Quality Management

©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 24 Slide 1

Quality Management

l Managing the quality of thesoftware process and products


Objectives

l To introduce the quality management process andkey quality management activities

l To explain the role of standards in qualitymanagement

l To explain the concept of a software metric,predictor metrics and control metrics

l To explain how measurement may be used inassessing software quality


Topics covered

l Quality assurance and standards

l Quality planning

l Quality control

l Software measurement and metrics


Software quality management

l Concerned with ensuring that the required level ofquality is achieved in a software product

l Involves defining appropriate quality standardsand procedures and ensuring that these arefollowed

l Should aim to develop a ‘quality culture’ wherequality is seen as everyone’s responsibility


What is quality?

l Quality, simplistically, means that a productshould meet its specification

l This is problematical for software systems• Tension between customer quality requirements (efficiency,

reliability, etc.) and developer quality requirements(maintainability, reusability, etc.)

• Some quality requirements are difficult to specify in anunambiguous way

• Software specifications are usually incomplete and ofteninconsistent


The quality compromise

l We cannot wait for specifications to improvebefore paying attention to quality management

l Must put procedures into place to improve qualityin spite of imperfect specification

l Quality management is therefore not justconcerned with reducing defects but also withother product qualities


Quality management activities

l Quality assurance• Establish organisational procedures and standards for quality

l Quality planning• Select applicable procedures and standards for a particular

project and modify these as required

l Quality control• Ensure that procedures and standards are followed by the

software development team

l Quality management should be separate fromproject management to ensure independence


Quality management and software development

Software developmentprocess

Quality managementprocess

D1 D2 D3 D4 D5

Standards andprocedures

Qualityplan

Quality review reports


ISO 9000

l International set ofstandards for qualitymanagement

l Applicable to a range of organisations frommanufacturing to service industries

l ISO 9001 applicable to organisations whichdesign, develop and maintain products

l ISO 9001 is a generic model of the qualityprocess Must be instantiated for each organisation


ISO 9001

Management responsibility Quality systemControl of non-conforming products Design controlHandling, storage, packaging anddelivery

Purchasing

Purchaser-supplied products Product identification and traceabilityProcess control Inspection and testingInspection and test equipment Inspection and test statusContract review Corrective actionDocument control Quality recordsInternal quality audits TrainingServicing Statistical techniques


ISO 9000 certification

l Quality standards and procedures should bedocumented in an organisational quality manual

l External body may certify that an organisation’squality manual conforms to ISO 9000 standards

l Customers are, increasingly, demanding thatsuppliers are ISO 9000 certified


ISO 9000 and quality management

Project 1quality plan



Project qualitymanagement

Organizationquality manual

ISO 9000quality models

Organizationquality process

is used to develop instantiated as

instantiated as

documents

Supports


l Standards are the key to effective qualitymanagement

l They may be international, national,organizational or project standards

l Product standards define characteristics that allcomponents should exhibit e.g. a commonprogramming style

l Process standards define how the softwareprocess should be enacted

Quality assurance and standards


l Encapsulation of best practice- avoidsrepetition of past mistakes

l Framework for quality assurance process - itinvolves checking standard compliance

l Provide continuity - new staff can understandthe organisation by understand the standardsapplied

Importance of standards


Product and process standards

Product standards Process standardsDesign review form Design review conductDocument naming standards Submission of documents to CMProcedure header format Version release processAda programming style standard Project plan approval processProject plan format Change control processChange request form Test recording process


Problems with standards

l Not seen as relevant and up-to-date by softwareengineers

l Involve too much bureaucratic form filling

l Unsupported by software tools so tedious manualwork is involved to maintain standards


l Involve practitioners in development. Engineersshould understand the rationale underlying astandard

l Review standards and their usage regularly.Standards can quickly become outdated and thisreduces their credibility amongst practitioners

l Detailed standards should have associated toolsupport. Excessive clerical work is the mostsignificant complaint against standards

Standards development


Documentation standards

l Particularly important - documents are thetangible manifestation of the software

l Documentation process standards• How documents should be developed, validated and maintained

l Document standards• Concerned with document contents, structure, and appearance

l Document interchange standards• How documents are stored and interchanged between different

documentation systems


Documentation process

Createinitial draft

Reviewdraft

Incorporatereview

comments

Re-draftdocument

Proofreadtext

Producefinal draft

Checkfinal draft

Layouttext

Reviewlayout

Produceprint masters

Printcopies

Stage 1:Creation

Stage 2:Polishing

Stage 3:Production

Approved document

Approved document


Document standards

l Document identification standards• How documents are uniquely identified

l Document structure standards• Standard structure for project documents

l Document presentation standards• Define fonts and styles, use of logos, etc.

l Document update standards• Define how changes from previous versions are reflected in a

document


Document interchange standards

l Documents are produced using different systemsand on different computers

l Interchange standards allow electronic documentsto be exchanged, mailed, etc.

l Need for archiving. The lifetime of wordprocessing systems may be much less than thelifetime of the software being documented

l XML is an emerging standard for documentinterchange which will be widely supported infuture


l The quality of a developed product is influencedby the quality of the production process

l Particularly important in software development assome product quality attributes are hard to assess

l However, there is a very complex and poorlyunderstood between software processes andproduct quality

Process and product quality


Process-based quality

l Straightforward link between process and productin manufactured goods

l More complex for software because:• The application of individual skills and experience is

particularly imporant in software development

• External factors such as the novelty of an application or the needfor an accelerated development schedule may impair productquality

l Care must be taken not to impose inappropriateprocess standards


Process-based quality

Define process Developproduct

Assess productquality

Standardizeprocess

Improveprocess

QualityOK

No Yes


l Define process standards such as how reviewsshould be conducted, configurationmanagement, etc.

l Monitor the development process to ensurethat standards are being followed

l Report on the process to project managementand software procurer

Practical process quality


Quality planning

l A quality plan sets out the desired productqualities and how these are assessed ande definethe most significant quality attributes

l It should define the quality assessment process

l It should set out which organisational standardsshould be applied and, if necessary, define newstandards


Quality plan structure

l Product introduction

l Product plans

l Process descriptions

l Quality goals

l Risks and risk management

l Quality plans should be short, succinct documents• If they are too long, no-one will read them


Software quality attributes

Safety Understandability PortabilitySecurity Testability UsabilityReliability Adaptability ReusabilityResilience Modularity EfficiencyRobustness Complexity Learnability


Quality control

l Checking the software development process toensure that procedures and standards are beingfollowed

l Two approaches to quality control• Quality reviews

• Automated software assessment and software measurement


Quality reviews

l The principal method of validating the quality ofa process or of a product

l Group examined part or all of a process or systemand its documentation to find potential problems

l There are different types of review with differentobjectives• Inspections for defect removal (product)

• Reviews for progress assessment(product and process)

• Quality reviews (product and standards)


Types of reviewReview type Principal purposeDesign or programinspections

To detect detailed errors in the design orcode and to check whether standards havebeen followed. The review should be drivenby a checklist of possible errors.

Progress reviews To provide information for managementabout the overall progress of the project.This is both a process and a product reviewand is concerned with costs, plans andschedules.

Quality reviews To carry out a technical analysis of productcomponents or documentation to find faultsor mismatches between the specificationand the design, code or documentation. Itmay also be concerned with broader qualityissues such as adherence to standards andother quality attributes.


l A group of people carefully examine part or allof a software system and its associateddocumentation.

l Code, designs, specifications, test plans,standards, etc. can all be reviewed.

l Software or documents may be 'signed off' at areview which signifies that progress to the nextdevelopment stage has been approved bymanagement.

Quality reviews


The review process

Selectreview team

Arrange placeand time

Distributedocuments

Hold review

Completereview forms


Review functions

l Quality function - they are part of the generalquality management process

l Project management function - they provideinformation for project managers

l Training and communication function - productknowledge is passed between development teammembers


Quality reviews

l Objective is the discovery of system defects andinconsistencies

l Any documents produced in the process may bereviewed

l Review teams should be relatively small andreviews should be fairly short

l Review should be recorded and recordsmaintained


l Comments made during the review should beclassified.• No action. No change to the software or documentation is

required.

• Refer for repair. Designer or programmer should correct anidentified fault.

• Reconsider overall design. The problem identified in thereview impacts other parts of the design. Some overalljudgement must be made about the most cost-effective wayof solving the problem.

l Requirements and specification errors mayhave to be referred to the client.

Review results


Software measurement and metrics

l Software measurement is concerned with derivinga numeric value for an attribute of a softwareproduct or process

l This allows for objective comparisons betweentechniques and processes

l Although some companies have introducedmeasurment programmes, the systematic use ofmeasurement is still uncommon

l There are few standards in this area


l Any type of measurement which relates to asoftware system, process or relateddocumentation• Lines of code in a program, the Fog index, number of person-

days required to develop a component

l Allow the software and the software process tobe quantified

l Measures of the software process or product

l May be used to predict product attributes or tocontrol the software process

Software metric


Predictor and control metrics

Managementdecisions

Controlmeasurements

Softwareprocess

Predictormeasurements

Softwareproduct


l A software property can be measured

l The relationship exists between what we canmeasure and what we want to know

l This relationship has been formalized andvalidated

l It may be difficult to relate what can be measuredto desirable quality attributes

Metrics assumptions


Internal and external attributes

Reliability

Number of procedureparameters

Cyclomatic complexity

Program size in linesof code

Number of errormessages

Length of user manual

Maintainability

Usability

Portability


The measurement process

l A software measurement process may be part of aquality control process

l Data collected during this process should bemaintained as an organisational resource

l Once a measurement database has beenestablished, comparisons across projects becomepossible


Product measurement process

Measurecomponent

characteristics

Identifyanomalous

measurements

Analyseanomalouscomponents

Selectcomponents to

be assessed

Choosemeasurements

to be made


Data collection

l A metrics programme should be based on a set ofproduct and process data

l Data should be collected immediately (not inretrospect) and, if possible, automatically

l Three types of automatic data collection• Static product analysis

• Dynamic product analysis

• Process data collation


Automated data collection

Instrumentedsoftware system

Faultdata

Usagedata


Data accuracy

l Don’t collect unnecessary data• The questions to be answered should be decided in advance and

the required data identified

l Tell people why the data is being collected • It should not be part of personnel evaluation

l Don’t rely on memory• Collect data when it is generated not after a project has finished


l A quality metric should be a predictor ofproduct quality

l Classes of product metric• Dynamic metrics which are collected by measurements made of

a program in execution

• Static metrics which are collected by measurements made of thesystem representations

• Dynamic metrics help assess efficiency and reliability; staticmetrics help assess complexity, understandability andmaintainability

Product metrics


Dynamic and static metrics

l Dynamic metrics are closely related to softwarequality attributes• It is relatively easy to measure the response time of a system

(performance attribute) or the number of failures (reliabilityattribute)

l Static metrics have an indirect relationship withquality attributes• You need to try and derive a relationship between these metrics

and properties such as complexity, understandability andmaintainability


Software product metricsSoftware metric Description

Fan in/Fan-out Fan-in is a measure of the number of functions that call some otherfunction (say X). Fan-out is the number of functions which are calledby function X. A high value for fan-in means that X is tightlycoupled to the rest of the design and changes to X will haveextensive knock-on effects. A high value for fan-out suggests that theoverall complexity of X may be high because of the complexity ofthe control logic needed to coordinate the called components.

Length of code This is a measure of the size of a program. Generally, the larger thesize of the code of a program’s components, the more complex anderror-prone that component is likely to be.

Cyclomaticcomplexity

This is a measure of the control complexity of a program. Thiscontrol complexity may be related to program understandability. Thecomputation of cyclomatic complexity is covered in Chapter 20.

Length ofidentifiers

This is a measure of the average length of distinct identifiers in aprogram. The longer the identifiers, the more likely they are to bemeaningful and hence the more understandable the program.

Depth ofconditional nesting

This is a measure of the depth of nesting of if-statements in aprogram. Deeply nested if statements are hard to understand and arepotentially error-prone.

Fog index This is a measure of the average length of words and sentences indocuments. The higher the value for the Fog index, the more difficultthe document may be to understand.


Object-oriented metricsObject-orientedmetric

Description

Depth ofinheritancetree

This represents the number of discrete levels in the inheritance tree wheresub-classes inherit attributes and operations (methods) from super-classes.The deeper the inheritance tree, the more complex the design as,potentially, many different object classes have to be understood tounderstand the object classes at the leaves of the tree.

Method fan-in/fan-out

This is directly related to fan-in and fan-out as described above and meansessentially the same thing. However, it may be appropriate to make adistinction between calls from other methods within the object and callsfrom external methods.

Weightedmethods perclass

This is the number of methods included in a class weighted by thecomplexity of each method. Therefore, a simple method may have acomplexity of 1 and a large and complex method a much higher value. Thelarger the value for this metric, the more complex the object class.Complex objects are more likely to be more difficult to understand. Theymay not be logically cohesive so cannot be reused effectively as super-classes in an inheritance tree.

Number ofoverridingoperations

These are the number of operations in a super-class which are over-riddenin a sub-class. A high value for this metric indicates that the super-classused may not be an appropriate parent for the sub-class.


Measurement analysis

l It is not always obvious what data means • Analysing collected data is very difficult

l Professional statisticians should be consulted ifavailable

l Data analysis must take local circumstances intoaccount


Measurement surprises

l Reducing the number of faults in a program leadsto an increased number of help desk calls• The program is now thought of as more reliable and so has a

wider more diverse market. The percentage of users who call thehelp desk may have decreased but the total may increase

• A more reliable system is used in a different way from a systemwhere users work around the faults. This leads to more helpdesk calls


Key points

l Software quality management is concerned withensuring that software meets its requiredstandards

l Quality assurance procedures should bedocumented in an organisational quality manual

l Software standards are an encapsulation of bestpractice

l Reviews are the most widely used approach forassessing software quality


Key points

l Software measurement gathers information aboutboth the software process and the softwareproduct

l Product quality metrics should be used to identifypotentially problematical components

l There are no standardised and universallyapplicable software metrics

Quality Management

Technology

quality managementlapplicable

quality managementlto

quality managementlmust

quality culture

quality managementlmanaging

quality compromiselwe

software engineering

quality management iso