11/18/19991 QUALITY SYSTEMS FOR SOFTWARE DEVELOPMENT The Software Engineering Institute Capability Maturity Model and ISO 9000.

11/18/1999 1

QUALITY SYSTEMS QUALITY SYSTEMS FOR FOR

SOFTWARE SOFTWARE DEVELOPMENTDEVELOPMENT

The Software Engineering InstituteThe Software Engineering InstituteCapability Maturity ModelCapability Maturity Model

andandISO 9000ISO 9000

11/18/1999 2

QUALITY SYSTEMS FOR SOFTWARE DEVELOPMENTQUALITY SYSTEMS FOR SOFTWARE DEVELOPMENT

Objectives:Objectives:

•Introduce two key Process Improvement Methodolgies used in industry

•Specifically the SEI CMM and the ISO 9000 Standard.

•Provide Some background, structure, comparisons, and case study

11/18/1999 3

QUALITY SYSTEMS FOR SOFTWARE DEVELOPMENTQUALITY SYSTEMS FOR SOFTWARE DEVELOPMENTQUALITY SYSTEMS FOR SOFTWARE DEVELOPMENT

COSTCOST

SCHEDULINGSCHEDULING

QUALITYQUALITY

PRIMARY SOFTWARE PRIMARY SOFTWARE PROJECT CHALLENGESPROJECT CHALLENGES

11/18/1999 4


A model that describes how software engineering practices in an organization evolve under certain conditions.

• The work performed is organized and viewed as a process

• The evolution of the process is managed systematically

CMMCMM

11/18/1999 5


ISO is “the organization” behind ISO 9000

It is NOT AN ACRONYM for the International Organization for Standardization (IOS)

It is NOT a Government organization

ISO has national standards institute organization voluntary members from more than 120 countries

ISO has developed more than 11,000 standards since it began operation in 1947.

ISO has NO Power to enforce the implementation of Stds

11/18/1999 6


• ISO Does NOT Certify nor determine conformance for the quality of an organizations systems

• Independent Certification Bodies in each country follow criteria of Accreditation Bodies to ensure good practice by all involved

• ISO Standards have been and continue to be developed based on Market Demands for quality

NO: ISO-certified, ISO-registered, ISO certification,ISO certificates nor ISO registration

CERTIFICATIONCERTIFICATION

11/18/1999 7

Quality Systems Model for quality assurance in design/development, production, installation and service

* Organizational Structure * Responsibilities

* Procedures * Processes

* Resources for implementing quality management


http://www.iso.ch/9000e/9000pub.htm

ISO 9001ISO 9001

11/18/1999 8

Certified to ISO StandardCertified to ISO Standard

The objective is to give the organization’s management and it’s customers confidence that the organization is in control of the way it does things.

ISO is not a product quality label. It is not a product standard, but rather a management process/system or in this case also a software development process standard.


11/18/1999 9


ISO operates no system for assessing the conformance of organizations

management systems with standards in the ISO 9000 family.

In addition, there exist many testing laboratories and certification bodies which offer independent (also known as "third party") conformity assessment services.

In partnership with the IEC (International Electrotechnical Commission), ISO has developed ISO/IEC Guideswhich contain the voluntary criteriarepresenting an internationalconsensus of what constitutes acceptablepractice.

Audits provide confirmation that products (including hardware, software and processed materials), services or systems measure up to ISO standards.

11/18/1999 10

ISO Structure ISO Structure

The ISO 9000 is a set of generic standards for quality management and assurance. It is a family of standards.

• The ISO 9001 contains 20 quality elements.

• ISO 9000-3 is a subset of guidelines and is divided into three groups:

* General company and management requirements * Projects and maintenance requirements * Supporting activities requirements


11/18/1999 11

• Define and document the way things are• Identify of best practices• Identify business processes• Simplify routine procedures• Conduct Internal audits• Provide Impetus and Incentive• Fosters Team spirit• Communication through Workshops and Meetings• Definition of common language• Customer perception


Key success factors Key success factors software process improvement software process improvement using ISO 9001 standards:using ISO 9001 standards:

11/18/1999 12


• Perceived high quality• Improved customer satisfaction• Competitive edge• Increased market share• Faster time to market

External benefits of ISO 9000 standardsExternal benefits of ISO 9000 standards

11/18/1999 13

Internal benefits of ISO 9000 standardsInternal benefits of ISO 9000 standards

• Better documentation• Greater quality awareness• Positive Cultural Change• Increased operational efficiency/productivity• Enhanced inter-company communications• Reduced rework • Everyone works better under stress !!!


11/18/1999 14

Auditors check that:

* Procedures are in place for all quality elements. * That the procedures followed do exist. * The document control system is valid. * Internal audits have been performed and corrective actions have been performed.


An Audit determines that you An Audit determines that you

““SAY WHAT YOU DO AND DO WHAT YOU SAY”SAY WHAT YOU DO AND DO WHAT YOU SAY”

11/18/1999 15


Agriculture Environment InformationProcessing

Ores andMetals

BasicChemicals

Graphics andPhotography

MechanicalEngineering

Packaging andDistribution ofGoods

Building Health andMedicine

Non-MetallicMaterials

SpecialTechnologies

Other Technical FieldsOther Technical Fields for Which for Which

ISO Develops StandardsISO Develops Standards

11/18/1999 16


Management ResponsibilityQuality SystemContract ReviewDesign ControlDocument and Data ControlPurchasingControl of Customer-supplied ProductProduct Identification and TractabilityProcess ControlInspection and Testing

QUALITY ELEMENTS OF ISO 9001QUALITY ELEMENTS OF ISO 9001

11/18/1999 17


QUALITY ELEMENTS OF ISO 9001, Cont.QUALITY ELEMENTS OF ISO 9001, Cont.

Control of Inspection, Measuring and Test EquipmentInspection and TestControl for Nonconforming ProductCorrective and Preventive ActionHandling, Storage, Packaging, Preservation and DeliveryControl of Quality RecordsInternal Quality AuditsTrainingServicingStatistical Techniques

11/18/1999 18


ISO CMM COMPARISONISO CMM COMPARISON

CMM ISO 9001

- Hierarchical Structure - Flat Structure- Popular in the USA - Popular in Europe and Asia- Mainly adopted by - Good for small and small organizations large organizations- Specifically for software - Common standards for all industries

ISO9001 to CMM mapping http://www.cs.njit.edu/~axp9532/

11/18/1999 19


SEI/CMMSEI/CMM

The Software Engineering Institute (SEI) is a federally funded research and development center sponsored by the U.S. Department of Defense and operated by Carnegie Mellon University.

® CERT and CMM are registered in the U.S. Patent and Trademark Office.

11/18/1999 20


Authorized Lead Assessors for CBA IPI are required to return a report to the SEI for each completed assessment. The data is entered into the Process Appraisal Information System.

All assessment data that is returned to the SEI is kept confidential. The information is available only to SEI employees who have a need to know associated with their R&D activities.

The SEI's Role in CMM-Based AppraisalsThe SEI's Role in CMM-Based Appraisals

11/18/1999 21


Assessment data are reported in industry aggregates with no identification of organizations and projects.

The Maturity Profile Report is published by the SEI twice a year and is available on the SEI web site as well as through SEI Customer Relations.

The SEI does not "certify" any assessment findings or maturitylevels.

The SEI's Role in CMM-Based Appraisals, ContThe SEI's Role in CMM-Based Appraisals, Cont..

11/18/1999 22


The SEI does not confirm or deny that any organization has performed an assessment. An assessment's findings and maturity level are owned by the assessment sponsor. The sponsor may publicize this information at their discretion.

The SEI trains and authorizes qualified persons to lead appraisals, i.e., CMM-Based Appraisal for Internal Process Improvement (CBA IPI), and to use SEI materials in conducting the appraisals.

The SEI does not validate or certify appraisal results.

The SEI's Role in CMM-Based Appraisals, ContThe SEI's Role in CMM-Based Appraisals, Cont

11/18/1999 23


The CMM provides a

well-defined evolutionary path

toward becoming a mature software organization.

Each level is a layer in a foundation for continuous process improvement.

11/18/1999 24

QUALITY SYSTEMS FOR SOFTWARE DEVELOPMENT

Over-all Goals

- Determine if the organization or project has effectively implemented the KPA (key process area). - Identify the scope, boundaries, and intent of each KPA.

Common Features

- Commitment to Perform - Ability to Perform - Measurement and Analysis - Verifying Implementation - Activities Performed

11/18/1999 25


Five Capability Maturity Model Levels

InitialInitial RepeatableRepeatable

DefinedDefined

ManagedManaged

OptimizedOptimized

11/18/1999 26


Five Capability Maturity Model LevelsFive Capability Maturity Model Levels

There are 18 Key Process Areas distributed over levelsTwo through five.

There are a total of 343 key practices grouped within the KeyProcesses according to the common features of one or more Key Indicators.

11/18/1999 27



The initial process is often called ad hoc. Organizations in this level are known as having chaotic processes. Outcomes are unpredictable and poorly controlled. Few processes are defined, and success depends primarily upon individual effort and heroics.

1.Initial

11/18/1999 28



At this level, software engineering management is used to establish basic project management processes to control cost, schedule, and quality of the product. The necessary process disciplines are established to repeat earlier successes on projects with similar applications.

2. Repeatable

11/18/1999 29



In this level there is a documented andimplemented standard software process for both management and engineering activities. All projects use an approved, tailored version of the organization’s standard software process for developing and maintaining software.

3. Defined

11/18/1999 30


In this level process and product are quantitatively measured and managed.

4. Managed


11/18/1999 31


This level emphasizes continuous, measurable process improvement, which is enabled by quantitative feedback from the process and from piloting innovative ideas and technologies

5. Optimizing


11/18/1999 32


Characteristics

-43.2% of organizations are at this level compared to 75%. -Over commitment is very common at this level. -During crisis, projects typically abandon planned procedures and revert to coding and testing. -Success depends on having an exceptional manager and effective software team.

Level 1: Initial

Percentages based on most recent assessment, since 1995, of 734 organizationscompared to percentages advertised in Late 1997

11/18/1999 33


Characteristics

-34.2% of organizations are at this level compared to 15%. -Software project standards are defined. -Planning and managing new projects is based on similar project experience.

Level 2: Repeatable

11/18/1999 34


Characteristics

- 17.3% of organizations are at this level compared to 8%. - A standard process for developing and maintaining software is documented and used across the organization. - Management has good insight into technical activities.

Level 3: Defined

11/18/1999 35


Characteristics

- 4.0% of organizations are at this level compared to 1.5%. - The organization sets quantitative quality goals for both software product and software processes. - All project’s productivity and quality are measured for important software process activities.

Level 4: Managed

11/18/1999 36


Characteristics

- 1.4% of organizations are at this level compared to 0.5%. - The organization has the means to identify weaknesses and strengthen the process proactively. - The goal is the prevention of defects.

Level 5: Optimized

11/18/1999 37

0

10

20

30

40

50

% of Orgs 43.2 34.2 17.3 4 1.4

InitialRepeat

ableDefine

dManag

edOptimi

zing


Data From SIE CMU 1999 Mid-Year ReportData From SIE CMU 1999 Mid-Year Report

11/18/1999 38


KEY PROCESS AREAS BY MATURITY LEVELKEY PROCESS AREAS BY MATURITY LEVEL

Initial (1)

Requirements Management [6]Software Project Planning Software Project Tracking and OversightSoftware subcontract managementSoftware quality assuranceSoftware Configuration management

Repeatable (2)

11/18/1999 39



Organization Process Focus [7]Organization process definitionTraining ProgramIntegrated software managementSoftware Product EngineeringIntergroup CoordinationPeer Reviews

Defined (3)

11/18/1999 40



Managed (4)

Defect Prevention [3]Technology Change Management Process Change Management

Optimizing (5)

Quantitative Process Management [2]Software Quality Management

11/18/1999 41


Level 2 - Repeatable - Goals - Requirements Management

1. System requirements allocated to software are controlled to establish a baseline for software engineering and management use.

2. Software plans, products, and activities are kept consistent with the system requirements allocated to software.

Level 1 - Poorly Planned - No Goals

Examples of KPA GoalsExamples of KPA Goals

11/18/1999 42


Level 3 - Defined - Goals - Organization Process Focus

1. Software process development and improvement activities are coordinated across the organization

2. The strengths and weaknesses of the software processes used are identified relative to a process standard

3. Organization-level process development and improvement activities are planned

11/18/1999 43


Level 4 - Managed - Goals - Quantitative Process Managment

1. The quantitative process management activities are planned

2. The process performance of the project’s defined software process is controlled quantitatively.

3. The process capability of the organization’s standard software process is known in quantitative terms.

11/18/1999 44


Level 5 - Optimizing - Goals - Defect Prevention

1. Defect Prevention activities are planned

2. Common causes of defects are sought out and identified

3. Common causes of defects are prioritized and systematically eliminated.

11/18/1999 45


Tree Diagram to define shortages

Created Work Breakdown Structure for

SEI Level 3 Project Activities for CAPPTask Id

Owner

Project Plan SectionTask DescriptionPlan Start DatePlan End DateActual Start Date

Actual End Date

11/18/1999 46


Software Engineering Institute Web Site http://www.sei.cmu.eduSoftware Productivity Research Web Site http://www.spr.comISO Web Site http://www.iso.ch

“A Comparative Study of CMM and ISO9000” Alok Pareek 1/24/98

A Guide to the CMM Kenneth M. Dymond 1995CMM for S/W V1.1 Paulk, Curtis, Chrissis & Weber Assessment of CAPP V4.0 Vs. SEI Level 3 Bauer, Hauge, Price

Acknowledgements:

Software Engineering Measurement and Analysis Team1999 Mid-Year Update on Process Maturity Profile of the Software Community

Lucent Technologies ISO 9001 Requirements Training Manual

11/18/1999 47

Acknowledgements Acknowledgements

Thanks to

Bonnie Jean Bauer

AVAYA/LUCENT Technologies

11/18/19991 QUALITY SYSTEMS FOR SOFTWARE DEVELOPMENT The Software Engineering Institute Capability Maturity Model and ISO 9000.

Documents

software development

iso certification

iso structure

iso certificates

countries iso

software development

quality systems model

government organization