SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 1 Software Engineering Standards Prepared by:

SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification.

1

Software Engineering StandardsSoftware Engineering Standards

Prepared by:James W. MooreThe MITRE [email protected]

For presentation toDC/Baltimore SIGAda

This presentation represents the opinion of the author and does

not present positions of The MITRE Corporation or of the U.S. Department of Defense.


2

Software engineering: Software engineering: DefinitionDefinition

(1) The application of a systematic, disciplined, quantifiable approach to the development, operation and maintenance of software, that is, the application of engineering to software.

(2) The study of approaches as in (1).

-- IEEE Std 610.12© IEEE, used by permission


3

Software Engineering StandardsSoftware Engineering Standards

Concern the responsible practice of software engineering

Usually deal with process rather than product ... … although they sometimes deal with generic

product characteristics or supporting resources Deal with subjects like Configuration Management,

Quality Assurance, Verification, and Validation


4

Importance of Software Importance of Software Engineering StandardsEngineering Standards

They consolidate existing technology into a firm basis for introducing newer technology

They increase professional discipline They protect the business They protect the buyer They improve the product


5

Software Engineering Software Engineering Standards: RolesStandards: Roles

Specify techniques to develop software faster, cheaper, better, IEEE 982.1 (Measures for Reliable SW)

Provide consensus validity for “best practices” that cannot be scientifically validated, IEEE 1008 (Unit Testing)

Provide a systematic treatment of “ilities”, IEEE 730 (SW Quality Assurance)

Provide uniformity in cases where agreement is more important than small improvements, IEEE P1320.1 (IDEF0)

Provide a framework for communication between buyer and seller, IEEE/EIA 12207 (SW Life Cycle Processes)

Give precise names to concepts that are fuzzy, complex, detailed and multidimensional, IEEE 1028 (SW Reviews)

More exciting

More effective


6

Organizational Goals for Organizational Goals for Software Engineering StandardsSoftware Engineering Standards

Improve and evaluate software competence Framework for two-party agreements Evaluation of software products Assurance of high integrity levels for

software products


7

Software Engineering Standards Software Engineering Standards

Approximately 315 software engineering standards, guides, handbooks, and technical reports are maintained by approximately 46 professional, sector, national, and international standards organizations. -- [Magee97]

In 1981, IEEE had one software engineering standard. By year end 1997, the collection had grown to 44. -- [SESC97a]

The 1994 edition of IEEE Standards Collection: Software Engineering is 1300 pages long. The 1999 edition is in four volumes totaling 2400 pages.

Users need help in integrating all of this advice.


8

Developers of international Developers of international standards related to SWEstandards related to SWE

ISO IEC

JTC1TC176

SC22

Software Engineering Language, OS

SC7

TC56 SC65A

ITU

Quality Information Technology Dependability Functional Safety

The focal point in international standards is ISO/IEC JTC1/SC7.

Other committees, though, deal with related work.

Members of these committees are “national bodies,” i.e. countries, represented by “national delegations.”

......


9

SWE standards developers:SWE standards developers:USUS

About 550 organizations in the U. S. make standards.

About half of them are accredited by ANSI, allowing them to participate in international standardization activity.

The focal point (in the U. S.) is the Software Engineering Standards Committee of the IEEE Computer Society

ANSI

NCITS AIAA ANS ASTM EIA

SESC

IEEE

INCOSEPMI ......


10

International Software International Software Engineering StandardsEngineering Standards


11

SC7 Plenary AttendanceSC7 Plenary Attendance

0

20

40

60

80

100

120

140

160

180

200

1987FR

1988NL

1989GER

1990USA

1991SW

1992UK

1993JP

1994CND

1995AUS

1996TCR

1997USA

1998SA

1999BR

Based on a chart by François Coallier, SC7 Chair


12

RequirementsData Process

Technologies Resources (Human & Material)

Tools

Products

Product Data

Engineering Data

“Standardization of processes, supporting tools and supporting technologies for the engineering of software products and systems.”

Scope of ISO/IEC JTC1/SC7Scope of ISO/IEC JTC1/SC7


13

ManagementSciences

ComputerSciences andEngineering Dependability

Engineering(IEC TC 56)

Safety(IEC TC65)

Industrial Engineering

QualityManagement(ISO TC 176)

APPLICATIONDOMAINS

(many TCs)

SOFTWARE and SYSTEM SOFTWARE and SYSTEM ENGINEERINGENGINEERING



14

ISO/IEC JTC1/SC7ISO/IEC JTC1/SC7Working GroupsWorking Groups

WG2: System software documentation

WG4: Tools and environment WG6: Evaluation & metrics WG7: Life cycle management WG9: System & SW integrity WG10: Process assessment WG12: Functional size

measurement WG13: Software measurement

process

WG17: ODP enterprise lang WG18: Quality mgmt sWG18: Revise ISO 9000-3 WG19: ODP and modelling

languages SWG1: Planning SWG2: Vocabulary SWG3: Process architecture SWG4: Web and

communication support Study Group: SE practices


15

SC7 Production (est.)SC7 Production (est.)(No new NWI assumed - exclude dependability)(No new NWI assumed - exclude dependability)

0

10

20

30

40

50

60

70

1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003

Stds Pub.

Stds Maint



16

Current Standards of SC7 Current Standards of SC7 (1 of 2)(1 of 2)

Several “legacy” standards ISO/IEC 6592:2000, Guidelines for the documentation of computer-based

application systems ISO/IEC 9126:1991, Product quality characteristics ISO 9127:1988, User documentation and cover information for consumer

software packages ISO/IEC TR 9294:1990, Management of software documentation ISO/IEC 11411:1995, Representation of state transition diagrams ISO/IEC 12119:1994, Software packages: Quality requirements and testing ISO/IEC TR 12182:1998, Categorization of software ISO/IEC 12207:1995, Software life cycle processes ISO/IEC 14102:1995, Evaluation and selection of CASE tools ISO/IEC 14143-1:1998, Functional size measurement


17

Current Standards of SC7 Current Standards of SC7 (2 of 2)(2 of 2)

ISO/IEC TR 14471:1999 Information technology -- Software engineering -- Guidelines for the adoption of CASE tools

ISO/IEC 14568:1997, Diagram exchange language for tree charts ISO/IEC 14598:2000, Software product evaluation (6 parts) ISO/IEC 14756:1999, Measurement and rating of performance ISO/IEC TR 14759:1999, Mockup and prototype ISO/IEC 14764:1999, Software maintenance ISO/IEC 15026:1998, System and software integrity levels ISO/IEC TR 15271:1998, Guide for ISO/IEC 12207 ISO/IEC TR 15504:1998, Software process assessment (9 parts) ISO/IEC TR 15846:1998, SWLC processes - Configuration management ISO/IEC 15910:1999, Software user documentation process ISO/IEC TR 16326:1999, Software project management


18

ISO/IEC 9126:1991, Software ISO/IEC 9126:1991, Software Product Quality Product Quality CharacteristicsCharacteristics

9126 defines the characteristics and subcharacteristics intended to cover all aspects of software quality resulting from the ISO definition of quality.

S u itab ilityA ccu racyIn te rop erab ilityC om p lian ceS ecu rity

F u n c tion a lity

M atu rityF au lt to le ran ceR ecoverab ility

R e liab ility

U n d ers tan d ab ilityL earn ab ilityO p erab ility

U sab ility

T im e b eh avio rR esou rce b eh avio r

E ffic ien cy

A n a lyzab ilityC h an g eab ilityS tab ilityTes tab ility

M a in ta in ab ility

A d ap tab ilityIn s ta llab ilityC on fo rm an ceR ep laceab ility

P ortab ility

IS O /IE C 9 1 2 6


19

New Generation of 9126 FamilyNew Generation of 9126 Family

14598-1: General Overview

14598-3: Developers Process

14598-6: Documentation of Evaluation Module9126-4

Quality in UseChar. and Metrics

9126-2External Char. and Metrics

9126-3Internal Char. and Metrics

9126-1Quality Model

14598-4: Acquirers Process

14598-5: Evaluators Process

14598-2: Planning and Management

ISO/IEC 9126 Quality Model and Metrics

ISO/IEC14598 Product Evaluation

ISO/IEC 12119Quality Requirement

and Testing

Adapted from a chart by Motoei Azuma


20

ISO/IEC 12207:1995,ISO/IEC 12207:1995,Software Life Cycle ProcessesSoftware Life Cycle Processes

To establish a common framework for the life cycle of software Broad scope: Acquire, supply, develop, operate,

and maintain software Recognizes that software is part of a system and

that a project is part of an enterprise To establish a basis for world trade in software Amendment underway to describe processes at

level of purpose and outcome


21

Example Use of 12207 ProcessesExample Use of 12207 Processes

SupplyDocumentation

Quality Assurance

Verification

Validation

Configuration Mgmt

Joint Review

Audit

Problem Resolution

Management

Infrastructure

Training

ImprovementAcquisition

An EnterpriseAn Enterprise


22

ISO/IEC TR 15504,ISO/IEC TR 15504,Software Process AssessmentSoftware Process Assessment

A nine-part Technical Report -- not a standard Currently under revision to become a five-part

Standard Will have a “process dimension” provided by an

externally supplied process reference model* Will have a 6 point “capability dimension”: Incomplete

through Optimizing

What conforms to 15504? -- the assessment.

*Processes other than software may be supplied.


23

ISO/IEC 15026, System and ISO/IEC 15026, System and Software Integrity LevelsSoftware Integrity Levels

System Design

Risk Analysis

Software Design

Risks, threats, frequency, initiators,

consequences

System integrity level

Risk containment,

fault tolerance strategies

Risk dimensions

Software component integrity levels


24

Draft ISO/IEC 15939, Software Draft ISO/IEC 15939, Software Measurement ProcessMeasurement Process

Activities and tasks necessary to identify, define, select, apply and improve software measurement with a project or organization.

Based on principles of Practical Software Measurement (PSM)

Designed to fit with ISO/IEC 12207, ISO/IEC 15504, and ISO 9000 series.


25

How to Characterize How to Characterize Measurement as a Process?Measurement as a Process?

Control

Process

Resources

ProductNeeds

Measurement Action

Goals

Constraints

Source: [SESC93]


26

Experience Base

Sustain Commitment

Plan the Measurement

Process

Perform the Measurement

Process

Evaluate Measurement

Tech & Mgmt

Processes

Core Measurement Process

Scope of Standard

Organization

Draft ISO/IEC 15939Draft ISO/IEC 15939Project


27

Relationship to Quality Relationship to Quality Management StandardsManagement Standards

Current relationship will be made obsolete by the circa 2000 revision of the ISO 9000 series.

SC7 will take responsibility for the replacement of ISO 9000-3 and other documents on SW QM

ISO 9001 ISO 9000-3 ISO/IEC 12207

Current Situation


28

Scope of SC7 wasScope of SC7 wasChanged in 1997Changed in 1997

Standardization of processes, supporting tools and supporting technologies for the engineering of software products and systems

SC7

JTC1

Scope: Information technology

Scope: Engineering of software products and systems


29

RationaleRationale

Software generally is part of a system or product

Tight integration of software and other component in many software based systems. Examples: Telecommunication products (Hardware,

Silicon, Software) Fly-by-wire aircraft



30

Working Groups withWorking Groups withSystem ScopeSystem Scope

WG2: System software documentation

WG4: Tools and environment WG6: Evaluation & metrics WG7: Life cycle management WG9: System & SW integrity WG10: Process assessment WG12: Functional size

measurement WG13: Software measurement

process

WG17: ODP enterprise lang WG18: Quality mgmt sWG18: Revise ISO 9000-3 WG19: ODP and modelling

languages SWG1: Planning SWG2: Vocabulary SWG3: Process architecture SWG4: Web and

communication support Study Group: SE practices


31

How Can You Participate?How Can You Participate?

US delegates to meetings of ISO/IEC SC7 and its working groups are selected from representatives to the US Technical Advisory Group.

US positions are developed by the TAG. Any US-domiciled organization can join the

TAG -- $300 per year. The TAG meets three times a year.


32

How the US is Represented How the US is Represented in JTC1 and SC7in JTC1 and SC7

JTC1

SC7

Working Groups

JTC1 TAG: Members are US

domiciled organizations

SC7 TAG: Members are US

domiciled organizations

TAG Task Groups

TAG Administrator

TAG Administrator

ANSI


33

Help Wanted!Help Wanted!

SW product quality models and metrics Systems processes and systems engineering practices Extension of process assessment to systems and business

processes System/software dependability issues Non-traditional functional size measurement Comprehensive SW quality management Also... Documentation, CASE, electronic data

interchange


34

IEEE as an Integrator of IEEE as an Integrator of Software Engineering Software Engineering

StandardsStandards


35

SW Engineering Standards Five SW Engineering Standards Five Years from NowYears from Now

SWE Body of Knowledge

Fundamental Principles

ISO/IEC

Integrating FrameworkIEEE

DoDISO/IEC

Umbrella Standard(s)IEEE

ISO/IEC

Integrated Collection of Practice StandardsIEEE

Motivate

Curricula

Licensing Exams

Professional Practice

Body of Standards

Quality Management

Project Management

System Engineering

Process Definition

Process Assessment

Process Measurement

Best Practices Competency Certification

Organizational Adoption

Buyer/Seller Agreement

Critical System Assurance


36

Guide to the Software Guide to the Software Engineering Body of KnowledgeEngineering Body of Knowledge

Collaboration between IEEE CS and UQAM. International participation from industry,

professional societies, standards bodies, academia, authors

By the time the project is finished literally thousands of individuals will have touched it

About to complete the middle of three phases http://www.swebok.org


37

Guide to the Software Guide to the Software Engineering Body of KnowledgeEngineering Body of Knowledge

Objectives Better characterize the SWE discipline Provide an index or taxonomy of the BOK Promote a consistent view of SWE Clarify relationship of SWE to other disciplines Act as an indicator of maturity of the discipline Provide a flagship document for SWE

standards


38

Baseline List of Knowledge Baseline List of Knowledge AreasAreas

Requirements Analysis Design Construction Testing Evolution and Maintenance Configuration Management Quality Analysis Engineering Infrastructure Engineering Process Engineering Management

• Computer Science (CC2001)

• Mathematics (CC2001)

• Project Management (PMBOK)

• Computer Engineering

• Cognitive Sciences and Human Factors

• Systems Engineering

• Management and Management Science

Related Disciplines


39

Knowledge Area DescriptionKnowledge Area Description

Classification of Topics

Matrix of Topics &

References References

Topic Descriptions

Classification by Vincenti’s Taxonomy

Classification by Bloom’s

Taxonomy

References to Related

DisciplinesNot in Stone-man Version


40




ISO/IEC


DoDISO/IEC


ISO/IEC


Motivate

Curricula

Licensing Exams


Body of Standards

Quality Management

Project Management

System Engineering

Process Definition

Process Assessment

Process Measurement






41

Examples of Fundamental Examples of Fundamental PrinciplesPrinciples

Currently, 16 candidates. Examples are … Since change is inherent in software, plan for it and

manage it.

Comme le logiciel est, de par sa nature même, sujet au changement, il faut planifier et gérer ce changement.

Since tradeoffs are inherent to SWE, make them explicit and document them.

Les compromis étant inhérents au GL, il faut les rendre explicites et les documenter.

Source: [Dupuis97]


42




ISO/IEC


DoDISO/IEC


ISO/IEC


Motivate

Curricula

Licensing Exams


Body of Standards

Quality Management

Project Management

System Engineering

Process Definition

Process Assessment

Process Measurement






43

Organizing the SESC standards:Organizing the SESC standards:Objects of software engineeringObjects of software engineering

interactswith AgentAgentCustomer Product

Process

Resource

performs

uses

produces

Project

transforms

applies to

aids

Source: [SESC95]


44

Organizing the SESC standards: Organizing the SESC standards: Relationship to other disciplinesRelationship to other disciplines

SoftwareEngineering

ComputerScience andTechnology

ApplicationDomains

ProjectManagement

SystemsEngineering

QualityManagement

Dependability Safety


45

Organization of SESC CollectionOrganization of SESC Collection

Overall Guide

Terminology

“Toolbox” ofTechnique Standards

CustomerResource Process Product

Principles or Policies

Element Standards

Application Guides

SystemDisciplines

ISO SWEngineering

QualityManagement

ProjectManagement

ContextualStandards

IEEE SESCStandards Program

Source: [SESC95]


46

Overall Guide to the IEEE Overall Guide to the IEEE CollectionCollection

Rather than developing a consensus document, IEEE SESC “commissioned” the writing of an overall guide to the collection.

Relevant standards are “sliced and diced” by: Context Object Normative intent Provider and subject

Relationships are explained, even between standards from different providers.

Published by IEEE Computer Society Press. 328 pages. 6" x 9" Softcover. ISBN 0-8186-8008-3. $40.00 List


47

Relating the Four-Volume Relating the Four-Volume Edition to the SESC ModelEdition to the SESC Model

Customer Process Product Resource

Techniques Layer

Overall Guide

Terminology Layer

TechniquesTerminology

Software Engineering, 1999 Edition(4 Volume Set)ISBN 0-7381-1563-0List: $325.00IEEE Mbr: $260.00http://standards.ieee.org/catalog/softwareset.html


48




ISO/IEC


DoDISO/IEC


ISO/IEC


Motivate

Curricula

Licensing Exams


Body of Standards

Quality Management

Project Management

System Engineering

Process Definition

Process Assessment

Process Measurement






49

Relationship to other StandardsRelationship to other Standards

IEEE SESC has adopted policy designating 12207 as a strategic, integrating standard for its collection.

IEEE SESC will revise its other standards to improve their fit with 12207.

Many of the other standards will detail the processes of 12207.


50

SWLC Process Standards:SWLC Process Standards:DoD Family TreeDoD Family Tree

DoD-Std2167A

DoD-Std2167

Mil-Std1679

Mil-Std498

EIA/IEEEJ-Std-016

NSA 1703

DoD-Std7935A


51

SWLC process standards:SWLC process standards:Commercial family treeCommercial family tree

Future revisionISO/IEC 12207

ISO/IEC 12207

IEEE/EIAStd 12207

EIA/IEEEJ-Std-016

Mil-Std 498

IEEE Std 1074


52

Example Use of 12207 ProcessesExample Use of 12207 Processes

SupplyDocumentation

Quality Assurance

Verification

Validation

Configuration Mgmt

Joint Review

Audit

Problem Resolution

Management

Infrastructure

Training

ImprovementAcquisition

An EnterpriseAn EnterpriseAdding Reuse & Risk Mgmt


53

Improvements in IEEE/EIA Improvements in IEEE/EIA 12207: Data Guidance 12207: Data Guidance

Part 1 may be used as a Guide or a Standard Defines 84 information items related to data

requirements of 12207.0 Defines seven generic kinds of data that

account for 44 of the information items Defines 30 specific content guidelines that

account for the other 40 information items The information items are data

requirements … not documents


54




ISO/IEC


DoDISO/IEC


ISO/IEC


Motivate

Curricula

Licensing Exams


Body of Standards

Quality Management

Project Management

System Engineering

Process Definition

Process Assessment

Process Measurement






55

Overall Overall SESC SESC CollectionCollection

Terminology

OverallGuide

Principles

ElementStandards

ApplicationGuides and

Supplements

“Toolbox” ofTechniques

See figurebelow for

Customerstandards

See figurebelow for

Processstandards

610.12IEEE glossary

Road MapBook

See figurebelow for

Productstandards

See figurebelow for

Resourcestandards

1044Classification of

anomalies

1044.1Guide to

1044Adapted from [Moore97a]


56

GeneralProcesses

Principles

ElementStandards


Supplements

PrimaryProcesses

SupportingProcesses

ProcessMeasurement

1074.1Guide to 1074

730.1Guide--SW QA

planning

12207.0Software life cycle (SWLC) processes

12207.1Guide--SW life

cycle data

12207.2Guide--SWLC

process

1220Systems eng.

process

1362Concept of

operations doc

1233Guide--System

rqmts spec

1058.1SW projectmgmt plans

1074DevelopingSWLC proc

J-Std-016Acq/sup

agreement

829SW test

documentation

830SW require-ments spec

1008SW unittesting

1219SW maint.

730SW QA plans

828SW CM plans

1012SW V & V

plans

1028SW reviewsand audits

1042Guide-SW CM

1059Guide--SWV & V plans

1298SW quality

mgmt system

1045SW product-ivity metrics

Charac-teristics

Principles

ElementStandards


Supplements

ProductMeasurement

ProductEvaluation

End ItemSpecification

982.2Guide to

982.1

730.1Guide--SW QA

planning

730SW QA plans

982.1Measures forreliable SW

1012SW V & V

plans

1059Guide--SWV & V plans

1063SW user

documentation

1228SW safety

plans

1233Guide--System

reqmts spec

1362Concept of

operations doc

1061Software quality

metrics methodology

Data Storage& Interchange

Principles

ElementStandards


Supplements

NotationReuse

LibrariesTools &

Environments

1016.1Guide to

1016

829SW test

documentation

830SW rqmts

specifications

1016SW designdescriptions

1175Tool inter-connection

P1471Architecturaldescription

P1320.xIDEF

1430Guide to1420.x

1420.xData model forreuse lib interop

1209Selection

of CASE tools

1348Adoption ofCASE tools

Pro

cess

Cus

tom

er

Pro

duct

Res

ourc

e


57

Process adoption: Process adoption: Is this the best way?Is this the best way?

Project Manager

Total Quality Management

Process definition

Corporate initiatives

Contractual requirements

Government regulations

Safety standards

Project manage-ment needs

Systems engineering methods

Capability evaluation

Tooling needs “Best practices”


58

Traditional project-level Traditional project-level compliance with standardscompliance with standards

Enterprise

EnterpriseProcesses

Project

ProjectProcesses

ProjectProcesses

Software engineering standards

Project claims compliance to standards

Every project adopts a unique set of procedures, etc.

“Message”: SW engineering standards are irrelevant to organizational capability and maturity.

Procedures,practices,templates,etc.

Procedures, practices, templates, etc.


59

Enterprise level adoptionEnterprise level adoption

Enterprise

EnterpriseProcesses

EnterpriseProcesses

Project

ProjectProcesses

ProjectProcesses

Software engineering standards Enterprise

claims compliance

Project is able to use enterprise procedures, etc.

“Message”: SW engineering standards are the basis for implementing repeatable, improving processes.

Procedures,practices,templates,etc.

Procedures, practices, templates, etc.

Project complies with enterprise processes


60

Relationship of Relationship of 12207 and CMM12207 and CMM

12207 gives requirements for the implementation of processes -- definition

CMM assesses the maturity of the implementation of the processes -- measurement

Control

Process

Resources

ProductNeeds

Evaluation / Assessment / Measurement

Action

Goals

Constraints

Source: [SESC93]

Focus of 12207

Focus of CMM


61

Relationship of 12207 to SEIRelationship of 12207 to SEISW-CMM Key Process AreasSW-CMM Key Process Areas

CMM Key ProcessAreas for Level 2

CMM Key ProcessAreas for Level 3

Addressed by12207compliance

Requirements mgmt SW project plan SW project tracking SW subcontract mgmt SW QA SQ CM

Training program SW product

engineering Intergroup

coordination Peer reviews

Addressed byenterprise-level adoptionof 12207

Organizationalprocess

Organizationalprocess definition

Integrated SW mgmt


62

How Can You Participate?How Can You Participate?

IEEE standards working groups are open to anyone.

To vote on the standards, you must be a member of IEEE or IEEE Computer Society and a member of the IEEE Standards Association.

http://standards.ieee.org/ http://computer.org/ http://computer.org/standards/sesc/


63

Contact informationContact information

Address for postal mail:The MITRE CorporationAttn: James W. Moore, M/S W5341820 Dolley Madison Blvd.McLean, VA 22102

Address for express delivery services:The MITRE CorporationAttn: James W. Moore, (703) 883-739611493 Sunset Hills RoadReston, VA 20190

Electronic mail: [email protected] Phone numbers:

Office: +1.703.883.7396

Fax: +1.703.883.5432

SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 1 Software Engineering Standards Prepared by:

Documents

software competenceframework

maintenance of software

mitre corporation

application of engineering

ieee p1320

ieee std

sw reviews

professional disciplinethey