Software Engineering Standards Prepared by: James W. Moore The MITRE Corporation [email protected] SIGAda - JWM0012 © 2000, The MITRE Corporation.

Software Engineering Standards

Prepared by:James W. MooreThe MITRE [email protected]

SIGAda - JWM0012© 2000, The MITRE Corporation.

Permission is granted to reproduce without modification.

1

For presentation toDC/Baltimore SIGAda

This presentation represents the opinion of the author and does

not present positions of The MITRE Corporation or of the U.S. Department of Defense.

Software engineering: Definition

(1) The application of a systematic, disciplined, quantifiable approach to the development, operation and maintenance of software, that is, the application of engineering to software.(2) The study of approaches as in (1).

-- IEEE Std 610.12



2

© IEEE, used by permission

Software Engineering Standards

• Concern the responsible practice of software engineering

• Usually deal with process rather than product ...• … although they sometimes deal with generic

product characteristics or supporting resources• Deal with subjects like Configuration Management,

Quality Assurance, Verification, and Validation



3

Importance of Software Engineering Standards

• They consolidate existing technology into a firm basis for introducing newer technology

• They increase professional discipline• They protect the business• They protect the buyer• They improve the product



4

Software Engineering Standards: Roles

• Specify techniques to develop software faster, cheaper, better, IEEE 982.1 (Measures for Reliable SW)

• Provide consensus validity for “best practices” that cannot be scientifically validated, IEEE 1008 (Unit Testing)

• Provide a systematic treatment of “ilities”, IEEE 730 (SW Quality Assurance)

• Provide uniformity in cases where agreement is more important than small improvements, IEEE P1320.1 (IDEF0)

• Provide a framework for communication between buyer and seller, IEEE/EIA 12207 (SW Life Cycle Processes)

• Give precise names to concepts that are fuzzy, complex, detailed and multidimensional, IEEE 1028 (SW Reviews)



5

More exciting

More effective

Organizational Goals for Software Engineering Standards

• Improve and evaluate software competence• Framework for two-party agreements• Evaluation of software products• Assurance of high integrity levels for software

products



6

Software Engineering Standards • Approximately 315 software engineering standards, guides,

handbooks, and technical reports are maintained by approximately 46 professional, sector, national, and international standards organizations. -- [Magee97]

• In 1981, IEEE had one software engineering standard. By year end 1997, the collection had grown to 44. -- [SESC97a]

• The 1994 edition of IEEE Standards Collection: Software Engineering is 1300 pages long. The 1999 edition is in four volumes totaling 2400 pages.



7

Users need help in integrating all of this advice.

Developers of international standards related to SWE



8

ISO IEC

JTC1TC176

SC22

Software Engineering Language, OS

SC7

TC56 SC65A

ITU

Quality Information Technology Dependability Functional Safety

The focal point in international standards is ISO/IEC JTC1/SC7.

Other committees, though, deal with related work.

Members of these committees are “national bodies,” i.e. countries, represented by “national delegations.”

......

SWE standards developers:US



9

About 550 organizations in the U. S. make standards.

About half of them are accredited by ANSI, allowing them to participate in international standardization activity.

The focal point (in the U. S.) is the Software Engineering Standards Committee of the IEEE Computer Society

ANSI

NCITS AIAA ANS ASTM EIA

SESC

IEEE

INCOSEPMI ......

International Software Engineering Standards



10

SC7 Plenary Attendance

0

20

40

60

80

100

120

140

160

180

200

1987FR

1988NL

1989GER

1990USA

1991SW

1992UK

1993JP

1994CND

1995AUS

1996TCR

1997USA

1998SA

1999BR



11

Based on a chart by François Coallier, SC7 Chair

Scope of ISO/IEC JTC1/SC7



12

RequirementsData Process

Technologies Resources (Human & Material)

Tools

Products

Product Data

Engineering Data

“Standardization of processes, supporting tools and supporting technologies for the engineering of software products and systems.”



13

ManagementSciences

ComputerSciences andEngineering Dependability

Engineering(IEC TC 56)

Safety(IEC TC65)

Industrial Engineering

QualityManagement(ISO TC 176)

APPLICATIONDOMAINS

(many TCs)

SOFTWARE and SYSTEM SOFTWARE and SYSTEM ENGINEERINGENGINEERING


ISO/IEC JTC1/SC7Working Groups

• WG2: System software documentation

• WG4: Tools and environment• WG6: Evaluation & metrics• WG7: Life cycle management• WG9: System & SW integrity• WG10: Process assessment• WG12: Functional size

measurement• WG13: Software

measurement process

• WG17: ODP enterprise lang• WG18: Quality mgmt• sWG18: Revise ISO 9000-3• WG19: ODP and modelling

languages• SWG1: Planning• SWG2: Vocabulary• SWG3: Process architecture• SWG4: Web and

communication support• Study Group: SE practices



14

SC7 Production (est.)(No new NWI assumed - exclude dependability)



15

0

10

20

30

40

50

60

70

1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003

Stds Pub.

Stds Maint


Current Standards of SC7 (1 of 2)

• Several “legacy” standards• ISO/IEC 6592:2000, Guidelines for the documentation of computer-based

application systems• ISO/IEC 9126:1991, Product quality characteristics• ISO 9127:1988, User documentation and cover information for consumer

software packages• ISO/IEC TR 9294:1990, Management of software documentation• ISO/IEC 11411:1995, Representation of state transition diagrams• ISO/IEC 12119:1994, Software packages: Quality requirements and testing• ISO/IEC TR 12182:1998, Categorization of software• ISO/IEC 12207:1995, Software life cycle processes• ISO/IEC 14102:1995, Evaluation and selection of CASE tools• ISO/IEC 14143-1:1998, Functional size measurement



16

Current Standards of SC7 (2 of 2)

• ISO/IEC TR 14471:1999 Information technology -- Software engineering -- Guidelines for the adoption of CASE tools

• ISO/IEC 14568:1997, Diagram exchange language for tree charts• ISO/IEC 14598:2000, Software product evaluation (6 parts)• ISO/IEC 14756:1999, Measurement and rating of performance• ISO/IEC TR 14759:1999, Mockup and prototype• ISO/IEC 14764:1999, Software maintenance• ISO/IEC 15026:1998, System and software integrity levels• ISO/IEC TR 15271:1998, Guide for ISO/IEC 12207• ISO/IEC TR 15504:1998, Software process assessment (9 parts)• ISO/IEC TR 15846:1998, SWLC processes - Configuration management• ISO/IEC 15910:1999, Software user documentation process• ISO/IEC TR 16326:1999, Software project management



17

ISO/IEC 9126:1991, Software Product Quality Characteristics

• 9126 defines the characteristics and subcharacteristics intended to cover all aspects of software quality resulting from the ISO definition of quality.



18

S u itab ilityA ccu racyIn te rop erab ilityC om p lian ceS ecu rity

F u n c tion a lity

M atu rityF au lt to le ran ceR ecoverab ility

R e liab ility

U n d ers tan d ab ilityL earn ab ilityO p erab ility

U sab ility

T im e b eh avio rR esou rce b eh avio r

E ffic ien cy

A n a lyzab ilityC h an g eab ilityS tab ilityTes tab ility

M a in ta in ab ility

A d ap tab ilityIn s ta llab ilityC on fo rm an ceR ep laceab ility

P ortab ility

IS O /IE C 9 1 2 6

New Generation of 9126 Family



19

14598-1: General Overview

14598-3: Developers Process

14598-6: Documentation of Evaluation Module9126-4

Quality in UseChar. and Metrics

9126-2External Char. and Metrics

9126-3Internal Char. and Metrics

9126-1Quality Model

14598-4: Acquirers Process

14598-5: Evaluators Process

14598-2: Planning and Management

ISO/IEC 9126 Quality Model and Metrics

ISO/IEC14598 Product Evaluation

ISO/IEC 12119Quality Requirement

and Testing

Adapted from a chart by Motoei Azuma

ISO/IEC 12207:1995,Software Life Cycle Processes

• To establish a common framework for the life cycle of software– Broad scope: Acquire, supply, develop, operate,

and maintain software– Recognizes that software is part of a system and

that a project is part of an enterprise• To establish a basis for world trade in software• Amendment underway to describe processes at

level of purpose and outcome



20

Example Use of 12207 Processes



21

SupplyDocumentation

Quality Assurance

Verification

Validation

Configuration Mgmt

Joint Review

Audit

Problem Resolution

Management

Infrastructure

Training

ImprovementAcquisition

An EnterpriseAn Enterprise

ISO/IEC TR 15504,Software Process Assessment

• A nine-part Technical Report -- not a standard• Currently under revision to become a five-part

Standard– Will have a “process dimension” provided by an

externally supplied process reference model*– Will have a 6 point “capability dimension”: Incomplete

through Optimizing• What conforms to 15504? -- the assessment.



22

*Processes other than software may be supplied.

ISO/IEC 15026, System and Software Integrity Levels



23

System Design

Risk Analysis

Software Design

Risks, threats, frequency, initiators,

consequences

System integrity level

Risk containment,

fault tolerance strategies

Risk dimensions

Software component integrity levels

Draft ISO/IEC 15939, Software Measurement Process

• Activities and tasks necessary to identify, define, select, apply and improve software measurement with a project or organization.

• Based on principles of Practical Software Measurement (PSM)

• Designed to fit with ISO/IEC 12207, ISO/IEC 15504, and ISO 9000 series.



24

How to Characterize Measurement as a Process?



25

Control

Process

Resources

ProductNeeds

Measurement Action

Goals

Constraints

Source: [SESC93]

Draft ISO/IEC 15939



26

Experience Base

Sustain Commitment

Plan the Measurement

Process

Perform the Measurement

Process

Evaluate Measurement

Tech & Mgmt

Processes

Core Measurement Process

Scope of Standard

Organization Project

Relationship to Quality Management Standards

• Current relationship will be made obsolete by the circa 2000 revision of the ISO 9000 series.

• SC7 will take responsibility for the replacement of ISO 9000-3 and other documents on SW QM



27

ISO 9001 ISO 9000-3 ISO/IEC 12207

Current Situation

Scope of SC7 wasChanged in 1997

• Standardization of processes, supporting tools and supporting technologies for the engineering of software products and systems



28

SC7

JTC1

Scope: Information technology

Scope: Engineering of software products and systems

Rationale

• Software generally is part of a system or product

• Tight integration of software and other component in many software based systems. Examples:– Telecommunication products (Hardware,

Silicon, Software)– Fly-by-wire aircraft



29


Working Groups withSystem Scope

• WG2: System software documentation

• WG4: Tools and environment• WG6: Evaluation & metrics• WG7: Life cycle management• WG9: System & SW integrity• WG10: Process assessment• WG12: Functional size

measurement• WG13: Software

measurement process

• WG17: ODP enterprise lang• WG18: Quality mgmt• sWG18: Revise ISO 9000-3• WG19: ODP and modelling

languages• SWG1: Planning• SWG2: Vocabulary• SWG3: Process architecture• SWG4: Web and

communication support• Study Group: SE practices



30

How Can You Participate?

• US delegates to meetings of ISO/IEC SC7 and its working groups are selected from representatives to the US Technical Advisory Group.

• US positions are developed by the TAG.• Any US-domiciled organization can join the

TAG -- $300 per year.• The TAG meets three times a year.



31

How the US is Represented in JTC1 and SC7



32

JTC1

SC7

Working Groups

JTC1 TAG: Members are US

domiciled organizations

SC7 TAG: Members are US

domiciled organizations

TAG Task Groups

TAG Administrator

TAG Administrator

ANSI

Help Wanted!

• SW product quality models and metrics• Systems processes and systems engineering practices• Extension of process assessment to systems and

business processes• System/software dependability issues• Non-traditional functional size measurement• Comprehensive SW quality management• Also... Documentation, CASE, electronic data

interchange



33

IEEE as an Integrator of Software Engineering

Standards



34

SW Engineering Standards Five Years from Now



35

SWE Body of Knowledge

Fundamental Principles

ISO/IEC

Integrating FrameworkIEEE

DoDISO/IEC

Umbrella Standard(s)IEEE

ISO/IEC

Integrated Collection of Practice StandardsIEEE

Motivate

Curricula

Licensing Exams

Professional Practice

Body of Standards

Quality Management

Project Management

System Engineering

Process Definition

Process Assessment

Process Measurement

Best Practices Competency Certification

Organizational Adoption

Buyer/Seller Agreement

Critical System Assurance

Guide to the Software Engineering Body of Knowledge

• Collaboration between IEEE CS and UQAM.• International participation from industry,

professional societies, standards bodies, academia, authors

• By the time the project is finished literally thousands of individuals will have touched it

• About to complete the middle of three phases• http://www.swebok.org



36

Guide to the Software Engineering Body of Knowledge

• Objectives– Better characterize the SWE discipline– Provide an index or taxonomy of the BOK– Promote a consistent view of SWE– Clarify relationship of SWE to other disciplines– Act as an indicator of maturity of the discipline– Provide a flagship document for SWE standards



37

Baseline List of Knowledge Areas• Requirements Analysis• Design• Construction• Testing• Evolution and Maintenance• Configuration Management• Quality Analysis• Engineering Infrastructure• Engineering Process• Engineering Management



38

• Computer Science (CC2001)

• Mathematics (CC2001)

• Project Management (PMBOK)

• Computer Engineering

• Cognitive Sciences and Human Factors

• Systems Engineering

• Management and Management Science

Related Disciplines

Knowledge Area Description



39

Classification of Topics

Matrix of Topics &

References References

Topic Descriptions

Classification by Vincenti’s Taxonomy

Classification by Bloom’s

Taxonomy

References to Related

DisciplinesNot in Stone-man Version




40



ISO/IEC


DoDISO/IEC


ISO/IEC


Motivate

Curricula

Licensing Exams


Body of Standards

Quality Management

Project Management

System Engineering

Process Definition

Process Assessment

Process Measurement





Examples of Fundamental Principles

• Currently, 16 candidates. Examples are …– Since change is inherent in software, plan for it and

manage it.Comme le logiciel est, de par sa nature même, sujet au changement, il faut planifier et gérer ce changement.

– Since tradeoffs are inherent to SWE, make them explicit and document them.Les compromis étant inhérents au GL, il faut les rendre explicites et les documenter.



41

Source: [Dupuis97]




42



ISO/IEC


DoDISO/IEC


ISO/IEC


Motivate

Curricula

Licensing Exams


Body of Standards

Quality Management

Project Management

System Engineering

Process Definition

Process Assessment

Process Measurement





Organizing the SESC standards:Objects of software engineering



43

interactswith AgentAgentCustomer Product

Process

Resource

performs

uses

produces

Project

transforms

applies to

aids

Source: [SESC95]

Organizing the SESC standards: Relationship to other disciplines



44

SoftwareEngineering

ComputerScience andTechnology

ApplicationDomains

ProjectManagement

SystemsEngineering

QualityManagement

Dependability Safety

Organization of SESC Collection



45

Overall Guide

Terminology

“Toolbox” ofTechnique Standards

CustomerResource Process Product

Principles or Policies

Element Standards

Application Guides

SystemDisciplines

ISO SWEngineering

QualityManagement

ProjectManagement

ContextualStandards

IEEE SESCStandards Program

Source: [SESC95]

Overall Guide to the IEEE Collection

• Rather than developing a consensus document, IEEE SESC “commissioned” the writing of an overall guide to the collection.

• Relevant standards are “sliced and diced” by:– Context– Object– Normative intent– Provider and subject

• Relationships are explained, even between standards from different providers.

• Published by IEEE Computer Society Press. 328 pages. 6" x 9" Softcover. ISBN 0-8186-8008-3. $40.00 List



46

Relating the Four-Volume Edition to the SESC Model



47

Customer Process Product Resource

Techniques Layer

Overall Guide

Terminology Layer

TechniquesTerminology

Software Engineering, 1999 Edition(4 Volume Set)ISBN 0-7381-1563-0List: $325.00IEEE Mbr: $260.00http://standards.ieee.org/catalog/softwareset.html




48



ISO/IEC


DoDISO/IEC


ISO/IEC


Motivate

Curricula

Licensing Exams


Body of Standards

Quality Management

Project Management

System Engineering

Process Definition

Process Assessment

Process Measurement





Relationship to other Standards

• IEEE SESC has adopted policy designating 12207 as a strategic, integrating standard for its collection.

• IEEE SESC will revise its other standards to improve their fit with 12207.

• Many of the other standards will detail the processes of 12207.



49

SWLC Process Standards:DoD Family Tree



50

DoD-Std2167A

DoD-Std2167

Mil-Std1679

Mil-Std498

EIA/IEEEJ-Std-016

NSA 1703

DoD-Std7935A

SWLC process standards:Commercial family tree



51

Future revisionISO/IEC 12207

ISO/IEC 12207

IEEE/EIAStd 12207

EIA/IEEEJ-Std-016

Mil-Std 498

IEEE Std 1074

Example Use of 12207 Processes



52

SupplyDocumentation

Quality Assurance

Verification

Validation

Configuration Mgmt

Joint Review

Audit

Problem Resolution

Management

Infrastructure

Training

ImprovementAcquisition

An EnterpriseAn EnterpriseAdding Reuse & Risk Mgmt

Improvements in IEEE/EIA 12207: Data Guidance

• Part 1 may be used as a Guide or a Standard• Defines 84 information items related to data

requirements of 12207.0– Defines seven generic kinds of data that account

for 44 of the information items– Defines 30 specific content guidelines that account

for the other 40 information items

• The information items are data requirements … not documents



53




54



ISO/IEC


DoDISO/IEC


ISO/IEC


Motivate

Curricula

Licensing Exams


Body of Standards

Quality Management

Project Management

System Engineering

Process Definition

Process Assessment

Process Measurement





Overall SESC Collection



55

Terminology

OverallGuide

Principles

ElementStandards

ApplicationGuides and

Supplements

“Toolbox” ofTechniques

See figurebelow for

Customerstandards

See figurebelow for

Processstandards

610.12IEEE glossary

Road MapBook

See figurebelow for

Productstandards

See figurebelow for

Resourcestandards

1044Classification of

anomalies

1044.1Guide to

1044Adapted from [Moore97a]



56

GeneralProcesses

Principles

ElementStandards


Supplements

PrimaryProcesses

SupportingProcesses

ProcessMeasurement

1074.1Guide to 1074

730.1Guide--SW QA

planning

12207.0Software life cycle (SWLC) processes

12207.1Guide--SW life

cycle data

12207.2Guide--SWLC

process

1220Systems eng.

process

1362Concept of

operations doc

1233Guide--System

rqmts spec

1058.1SW projectmgmt plans

1074DevelopingSWLC proc

J-Std-016Acq/sup

agreement

829SW test

documentation

830SW require-ments spec

1008SW unittesting

1219SW maint.

730SW QA plans

828SW CM plans

1012SW V & V

plans

1028SW reviewsand audits

1042Guide-SW CM

1059Guide--SWV & V plans

1298SW quality

mgmt system

1045SW product-ivity metrics

Charac-teristics

Principles

ElementStandards


Supplements

ProductMeasurement

ProductEvaluation

End ItemSpecification

982.2Guide to

982.1

730.1Guide--SW QA

planning

730SW QA plans

982.1Measures forreliable SW

1012SW V & V

plans

1059Guide--SWV & V plans

1063SW user

documentation

1228SW safety

plans

1233Guide--System

reqmts spec

1362Concept of

operations doc

1061Software quality

metrics methodology

Data Storage& Interchange

Principles

ElementStandards


Supplements

NotationReuse

LibrariesTools &

Environments

1016.1Guide to

1016

829SW test

documentation

830SW rqmts

specifications

1016SW designdescriptions

1175Tool inter-connection

P1471Architecturaldescription

P1320.xIDEF

1430Guide to1420.x

1420.xData model forreuse lib interop

1209Selection

of CASE tools

1348Adoption ofCASE tools

Pro

cess

Cus

tom

er

Pro

duct

Res

ourc

e

Process adoption: Is this the best way?



57

Project Manager

Total Quality Management

Process definition

Corporate initiatives

Contractual requirements

Government regulations

Safety standards

Project manage-ment needs

Systems engineering methods

Capability evaluation

Tooling needs “Best practices”

Traditional project-level compliance with standards



58

Enterprise

EnterpriseProcesses

Project

ProjectProcesses

ProjectProcesses

Software engineering standards

Project claims compliance to standards

Every project adopts a unique set of procedures, etc.

“Message”: SW engineering standards are irrelevant to organizational capability and maturity.

Procedures,practices,templates,etc.

Procedures, practices, templates, etc.

Enterprise level adoption



59

Enterprise

EnterpriseProcesses

EnterpriseProcesses

Project

ProjectProcesses

ProjectProcesses

Software engineering standards

Enterprise claims compliance

Project is able to use enterprise procedures, etc.

“Message”: SW engineering standards are the basis for implementing repeatable, improving processes.

Procedures,practices,templates,etc.

Procedures, practices, templates, etc.

Project complies with enterprise processes

Relationship of 12207 and CMM

• 12207 gives requirements for the implementation of processes -- definition

• CMM assesses the maturity of the implementation of the processes -- measurement



60

Control

Process

Resources

ProductNeeds

Evaluation / Assessment / Measurement

Action

Goals

Constraints

Source: [SESC93]

Focus of 12207

Focus of CMM

Relationship of 12207 to SEISW-CMM Key Process Areas

CMM Key ProcessAreas for Level 2

CMM Key ProcessAreas for Level 3

Addressed by12207compliance

Requirements mgmt SW project plan SW project tracking SW subcontract mgmt SW QA SQ CM

Training program SW product

engineering Intergroup

coordination Peer reviews

Addressed byenterprise-level adoptionof 12207

Organizationalprocess

Organizationalprocess definition

Integrated SW mgmt



61

How Can You Participate?

• IEEE standards working groups are open to anyone.• To vote on the standards, you must be a member of

IEEE or IEEE Computer Society and a member of the IEEE Standards Association.

• http://standards.ieee.org/• http://computer.org/• http://computer.org/standards/sesc/



62

Contact information• Address for postal mail:

The MITRE CorporationAttn: James W. Moore, M/S W5341820 Dolley Madison Blvd.McLean, VA 22102

• Address for express delivery services:The MITRE CorporationAttn: James W. Moore, (703) 883-739611493 Sunset Hills RoadReston, VA 20190

• Electronic mail: [email protected]• Phone numbers:

Office: +1.703.883.7396Fax: +1.703.883.5432



63

Software Engineering Standards Prepared by: James W. Moore The MITRE Corporation [email protected] SIGAda - JWM0012 © 2000, The MITRE Corporation.

Documents

mitre corporation

software competenceframework

maintenance of software

application of engineering

ieee p1320

ieee std

validationsigada jwm0012

productsigada jwm0012