Topic 14Summer 2003 1 ICS 52: Introduction to Software Engineering Lecture Notes for Summer Quarter, 2003 Michele Rousseau Topic 14.

Topic 14 Summer 2003 1

ICS 52: Introduction to Software Engineering

Lecture Notes for Summer Quarter, 2003

Michele RousseauTopic 14


Process Improvement

Capability Maturity Model

ISO 9000 PSP


Understanding existing processes Introducing process changes to

achieve organisational objectives usually focused on:

• quality improvement• cost reduction • schedule acceleration

Most work so far has focused on • defect reduction to improve Quality

Testing can’t do it all

Process improvement


Process analysis• Model and analyse (quantitatively if possible) existing

processes Improvement identification

• Identify quality, cost or schedule bottlenecks Process change introduction

• Modify the process to remove identified bottlenecks Process change training

• Train staff involved in new process proposals Change tuning

• Evolve and improve process improvements

Process improvement stages


The process improvement process

Processmodel

Process changeplan

Trainingplan

Feedback onimprovements

Revised processmodel

Analyseprocess

Identifyimprovements

Tuneprocess changes

Introduceprocess change

Trainengineers


Process quality product quality • These are closely related

A good process is usually required to produce a good product

For manufactured goods, process is the principal quality determinant

For design-based activity, other factors are also involved especially the capabilities of the designers

Process and product quality


Principal product quality factors

Productquality

Developmenttechnology

Cost, time andschedule

Processquality

Peoplequality


What is CMM?

Capability Maturity Model Developed by the software community in 1986

with leadership from the SEI. Has become a de facto standard for assessing

and improving processes related to software development

Has evolved into a process maturity framework Provides guidance for measuring software

process maturity Helps establish process improvement programs


What is the Software CMM?

“A common-sense application of process management and quality improvement concepts to the software development and maintenance”

A model for organizational improvement


Software Capability Maturity Model.

Maturity LevelsProcess

Capability Indicate

Key Process Areas

Goals

Contain

Achieve

Common Features

Implementation

Organized by

Address

Key Practices

Activities

Contain

Describe

CMU/SEI-93-TR-24 p. 29


What makes up the CMM?

The CMM is organized into five maturity levels: Initial Repeatable Defined Manageable Optimizing

Except for Level 1, each maturity level decomposes into several key process areas that indicate the areas an organization should focus on to improve its software process.


Each Maturity Level

1. Initial : » ad hoc process. Success depends on individual effort.

2. Repeatable : » Basic management processes: cost, schedule and

functionality 3. Defined :

» Activities are documented, standardized and integrated into an organization-wide software process.

4. Managed : » Detailed measures are collected: software and product

quality. 5. Optimizing :

» Continuous process improvement: quantitative feedback from the process and from testing new ideas and technologies.


Five Levels of Software Process Maturity

Initial

Optimizing

Managed

Defined

Repeatable

Unpredictable andpoorly controlled

Focus on process improvement

Process measured and controlled

Process characterized, fairly well understood

Can repeat previouslymastered tasks

Disciplined process

Continually improving process

Predictableprocess

Standard, consistent process

Level 1

Level 2

Level 3

Level 4

Level 5


Key Process Areas

• KPAs associated with each maturity level describe functions that must be present to satisfy good practice at a particular level. (Except Level 1)

• Each KPA is described by:

• Goals – Overall objectives

• Commitments – Requirements that must be met to achieve the goals.

• Abilities – Things that must be in place to enable the organization to meet the commitment.

• Activities – Specific tasks required to achieve the KPA function

• Methods for monitoring implementation

• Methods for verifying implementation• Each KPA is defined by a set of key practices that

contribute to satisfying its goals. (ie policies, procedures, and activities)


Level 2 KPAs - Purpose

KPA Purpose* Requirements Management

-Establish a common understanding between the customer and the project team of the requirements. -Artifacts are kept consistent with requirements

Software Project Planning

-Estimates are documented for planning and tracking, - Project activities are planned and documented

Software Project Tracking and Oversight

Provide adequate visibility into actual progress so significant deviations from the plan can be addressed.

* Paraphrased


Level 2 KPAs - Purpose (Cont.)

* Paraphrased

KPA Purpose* Software Subcontract Management

Select qualified subcontractors and manage them effectively.

Software Quality Assurance

-Provide management with visibility into the project’s process and products. - Plan and verify

Software Configuration Management

Establish and maintain the integrity of the project’s products.

Changes in commitments are agreed to by affected groups

All activities are planned and tracked



KPA Purpose* Organization Process Focus

-S/w process activities are coordinated with org. Establish responsibility for the definition and continuous improvement of the organization’s process.

Organization Process Definition

Develop and maintain the organization’s standard process (i.e., process assets).

Training Program Develop the skills and knowledge of individuals so they can perform efficiently and effectively.

* Paraphrased


Level 3 KPAs - Purpose (Cont.)

KPA Purpose* Integrated Software Management

Integrate the project team’s activities and the project management activities into a coherent, defined process tailored from the organization’s process.

Software Product Engineering

Consistently perform a well-defined software engineering process to produce consistent products effectively and efficiently.

Intergroup Coordination

Establish a means for the project team to actively participate with other engineering groups so the project is better able to satisfy the needs effectively and efficiently. Everybody agrees to requirements and their responsibilities

Peer Reviews

Remove defects from project’s products early and efficiently.

* Paraphrased



KPA Purpose* Quantitative Process Management

-Control the project’s process -performance quantitatively. -Measurement of the process activities -Time taken for activities to be completed - Resources required

Software Quality Management

Develop a quantitative understanding of the quality of the project’s products and achieve specific quality goals.

* Paraphrased



KPA Purpose*Defect Prevention Identify the cause of defects and prevent

them from recurring.

Technology ChangeManagement

Identify new technologies and transitionthem into the organization in an orderlymanner.

Process ChangeManagement

Continually improve the organization’sprocess with the intent of improvingquality, increasing productivity, anddecreasing cycle time.

* Paraphrased


Interesting CMM Facts

The number of companies using CMM to assess their software management practices more than doubles every five years (since 1987).

Software Quality Assurance is the biggest obstacle for organizations trying to move from level 1 to level 2.

Organization Process Definition is one of the biggest obstacles for organization trying to move from level 2 to level 3.


… more interesting facts

On average, it takes an organization:» 25 months to move from level 1 to 2» 22 months to move from level 2 to 3» 36.5 months to move from level 3 to 4

About a third of companies engaged in CMM are located overseas (primarily India), and are 3 times more likely to reach CMM level 4 or 5 than US organizations.

Only about 23% of organizations surveyed eventually move from level 2 to level 3 or higher.


What improved maturity provides

Source: Master Systems, Inc.

SEI Level

Calendar Time

(Months)

Effort (Person Months)

Defects Found

Defects Shipped

Median Cost

($,000)

Lowest Cost

($,000)

Highest Cost

($,000)

1 30 594 1,348 61 5,440 1,786 101,721

2 19 143 328 12 1,311 962 1,732

3 15 80 182 7 728 518 933

4 13 43 97 5 392 279 502

5 9 16 37 1 146 15 271

Based on data from 1300 applications, average 200,000 LOC


Characteristics of Immaturity

Software process improvised during the course of a project.

Even if process is specified, it is not rigorously followed or enforced.

Reactionary, focus on solving immediate crises. Hard deadlines often mean a compromise in

functionality and/or quality. No objective basis for judging product quality

or for solving process problems. Quality is difficult if not impossible to predict.


Characteristics of Maturity

Able to manage software development and maintenance organization/project wide.

There is a prescribed, mandated, and enforced process. Process is consistent with the way that work actually

gets done. Process is updated and improved as necessary. Roles and responsibilities within the process are clear. Quality is measured and monitored, and an objective

basis for judgment exists. The necessary infrastructure for supporting the process

exists. Workers see the value in the process.


Greater Maturity Can Bear Fruit

SE division of Hughes aircraft spent @$500K over a three year period for assessment and improvement programs. By the end of the three year period, assessed at CMM Level 3. Estimated savings of @$2M annually as a result (less overtime, less rework, greater productivity, etc.)

Equipment Division of Raytheon rise to CMM Level 3, at an estimated cost of @$580K resulted in 2-fold increase in productivity along with savings of @$15.8M in rework costs.

Motorola GED (CMM Level 4) documented significant• reduction in cycle time• reduction in defect rates• increase in productivity


ISO 9000 - Background

International set of standards for quality management (ISO 9000:2000, ISO 9001:2000, ISO 9004:2000, etc.)

ISO is name adopted by the International Organization for Standardization (not an acronym) – comes from isos “equal”

ISO 9000 is the most popular quality standard in the world

Over 13,000 standards issued since 1946 Made up of representative bodies from over

140 countries It applies to almost all types of organizations

regardless of their function or product.


From “The Dilbert Principle”


ISO 9000 – What is it?

ISO 9000 is primarily concerned with "quality management".

ISO 9001:2000 specifies requirements for a quality management system for any organization that needs to demonstrate its ability to consistently provide product that meets customer and applicable regulatory requirements and aims to enhance customer satisfaction, in all business sectors

It involves the development of a quality system that meets the quality requirements of the ISO standards.


Contents of Software Quality Planfrom ISO9000

Management responsibility Quality system Contract review Design control Quality control Purchasing Customer supplied info Configuration management Process control Inspection and testing Inspection and testing

equipment

Control of non-conforming product

Corrective action Handling, storage, packing and

delivery Quality records Internal quality audits Training Software maintenance Statistical techniques Control of the development

environment


ISO 9000 series of standards:

represents an international consensus on good management practices

guidelines on what constitutes an effective quality management system

serves as framework for continuous improvement


ISO 9000 and quality management

Project 1quality plan



Project qualitymanagement

Organizationquality manual

ISO 9000quality models

Organiza tionquality process

is used to develop instantiated as

instantiated as

documents

Supports


ISO 9000 - Certification

Quality standards and procedures should be documented in an organisational quality manual

External body may certify that an organisation’s quality manual conforms to ISO 9000 standards (namely ISO 9001)

Customers are, increasingly, demanding that suppliers are ISO 9000 certified


ISO vs CMM

CMM and the ISO 9000 series of standards share common concerns with quality and process management.

CMM emphasizes continuous improvement ISO deals with minimum criteria of quality

systems There is a clear correlation between the key

processes in the CMM and the quality management processes in ISO 9000

ISO 9000 has little explicit support for continuous improvement


ISO vs CMM (2)

The CMM is more detailed and prescriptive and includes a framework for improvement

An ISO 9001-compliant organization would not necessarily satisfy all of the CMM level 2 key process areas (it would satisfy most of the level 2 goals and many level 3 goals.

Organisations rated as level 2 in the CMM are likely to be ISO 9000 compliant


ISO9000 and CMM compared

CMM ISO 9001 Specific to software development Intended for most industries

Used in USA, less widely Recognised and accepted in most elsewhere countries

Provides detailed and specific Specifies concepts, principles and definition of what is required safeguards that should be in place for given levels

Assesses on 5 levels Establishes one acceptable level

CMM Level 2 - 3 ISO 9000

Relevant to Stabilises the customer - supplier s/w development process relationship

No time limit on certification Certification valid for three years

No ongoing audit Auditors may return for spot checks during the lifetime of the certificate

Topic 14 Summer 2003

Personal Software Process (PSP)

Developed by SEI in 1994A measurement and analyses framework to help you characterize your process

A defined procedure to help you to improve your performance

PSP principles•System quality depends on the quality of its worst components

•Component quality depends on individual developers


Overview of CMM and PSP

CMM sets out the principal practices for managing the processes in large-scale software development

PSP sets out the principal practices for defining, measuring and analysing an individual’s own processes


PSP

PSP applies a CMM-like assessment for individual work• Measurement & analysis framework to

help you characterize your process»Self-assessment and self-monitoring

• Prescribes a personal process for developing software» defined steps » Forms » Standards

• Assumes individual scale & complexity»Well-defined individual tasks of short

duration


PSP Overview

The PSP is introduced in 7 upward compatible steps (4 levels)

Write 1 or 2 small programs at each step•Assume that you know the programming language

Gather and analyze data on your work•Many standard forms & spreadsheet templates

Use these analyses to improve your work •Note patterns in your work


PSP Evolution

PSP0Current processTime recording

Defect recordingDefect type standard

PSP1Size estimating

Test report

PSP2Code reviews

Design reviews

PSP3Cyclic development

PSP2.1Design templates

PSP1.1Task planning

Schedule planning

PSP0.1Coding standard

Size measurementProcess improvement

proposal (PIP)

Baseline Personal Process

Cyclic Personal Process

Personal Quality Management

Personal Planning Process


PSP Evaluation

Humphrey has used in SE courses• Improvements in time-to-compile, quality and

productivity Patchy, but promising use in industry

• E.g. Nortel (Atlanta) Still immature Requires large overhead for data gathering

• Not clear that you should use permanently or continually


PSP/TSP/CMM

Topic 14Summer 2003 1 ICS 52: Introduction to Software Engineering Lecture Notes for Summer Quarter, 2003 Michele Rousseau Topic 14.

Documents

software process maturity

good process

process changes

designers process

process improvement

continuous process improvement

process maturity framework

process analysis model