Unit 2

Unit 2

July 27, 2011

1 DEVELOPMENTS IN MEASURING QUAL-ITY

Quality goals:

It is largely accepted that quality is the most importantobjective in software development. A software productof poor quality does not provide any benefit for its pro-ducer, but requires continuous correction, which entailsa wasteful use of resources.GRCM (goal-rule-checklist-metric)” model to provide thestructure for organizing quality information.

The GRCM model has three links with the SQM synthe-sis, the goals correspond to factors (e.g. usability) andthe rules to criteria (e.g. ease of use), and the metricssupport quality measurement in both models. The goalsare broken down into rules and further into checklists.The aim of the rules is to guide software engineers insoftware design, while checklists are generated from spe-cific rules and used as guidelines by inspectors, to helpof them check that these rules have been followed.

1

P.Selvapriyavadhana,Asst.Prof,ACT

Figure 1: The quality-intensive approach based on the GRCM model

We first prioritize the most important of the 11 alter-native goals (presented in the GRCM model) for the userinterface of the beer pump. We may choose to place ourfocus on usability aspects, e.g. striving for understand-able instructions, or we can emphasize reliability, e.g.ensuring that the beer flows into the glass, for example.In addition we choose the portability goal, because thiskind of system could have potential clients worldwide.

Reasons for the choice of quality goals

• A goal (or objective) is a statement of the desiredresult to be achieved within a specified period whichis aimed-at target.

• These goals then become basis for detailed planningof activities.

SQM 2


• Tactical goals are short range (up to 1 year), whereasstrategic goals are long range (say, 5 years).

• Examples of corporate quality goals.For a health prod-uct company, the quality goals over the next yearcould be: The average leakage rate for ”. productshall be reduced to ”

• Note that quality goal statements include quantifieddata.

• Typically Pareto analysis is used to develop the qual-ity goals .

Pareto analysis is a statistical technique in decisionmaking that is used for selection of a limited num-ber of tasks that produce significant overall effect.It uses the Pareto principle the idea that by doing20% of work, 80% of the advantage of doing the en-tire job can be generated. Or in terms of qualityimprovement, a large majority of problems (80% )are produced by a few key causes (20% ).

A quality measure is in effect a rule (or the re-sult of a rule) that assigns numeric values to aspecific quality indicator. The essential distinc-tion between quality indicators and quality mea-sures is that quality measures take on numericvalues, while quality indicators refer only to un-quantified attributes of care related to quality.

Measure - quantitative indication of extent, amount, di-mension, capacity, or size of some attribute of a productor process. E.g., Number of errors

SQM 3


Figure 2: Reasons for the choice of quality goals

2 Principles Of Measurement

The objectives of measurement should be established be-fore data collection begins.

1. Formulation - The derivation of software measuresand metrics appropriate for the representation of thesoftware that is being considered.

2. Collect information - The mechanism used to accu-mulate data required to derive the formulated met-rics.

3. Analysis information - The computation of metricsand the application of mathematical tools.

4. Interpretation -The evaluation of metrics results inan effort to gain insight into the quality of the rep-resentation.

SQM 4


Figure 3: Annual quality goals

SQM 5


Figure 4: Approaches to Measuring Quality

5. Feedback - Recommendations derived from the in-terpretation of product metrics transmitted to thesoftware team.

Types of Measures

• Direct Measures (internal attributes)

– Cost, effort, LOC, speed, memory

• Indirect (external attributes)

– Functionality, quality, complexity, efficiency, re-liability, maintainability

SQM 6


Why Measure Software?

• Determine the quality of the current product or pro-cess

• Predict qualities of a product/process

• Improve quality of a product/process

Measures of Software Quality

• Correctness degree to which a program operates ac-cording to specification

– Defects/KLOC

– Defect is a verified lack of conformance to re-quirements

– Failures/hours of operation

• Maintainability degree to which a program is opento change

– Mean time to change

– Change request to new version (Analyze, designetc)

– Cost to correct

• Integrity - degree to which a program is resistant tooutside attack

– Fault tolerance, security and threats

• Usability easiness to use

– Training time, skill level necessary to use, In-crease in productivity, subjective questionnaireor controlled experiment

SQM 7


3 Metric

Quantitative measure of degree to which a system, com-ponent or process possesses a given attribute. A handleor guess about a given attribute.

E.g., Number of errors found per person hours expended.

Motivation for Metrics

• Estimate the cost and schedule of future projects.

• Evaluate the productivity impacts of new tools andtechniques.

• Establish productivity trends over time.

• Improve software quality.

• Forecast future staffing needs.

• Anticipate and reduce future maintenance needs.

Example Metrics

Defect ratesError rates

SQM 8


Product Quality Metrics

The definition of software quality consists of two levels:intrinsic product quality and customer satisfaction. Themetrics we discuss here cover both levels:

• Mean time to failure

• Defect density

• Customer problems

• Customer satisfaction.

Intrinsic product quality is usually measured by the num-ber of bugs (functional defects) in the software or byhow long the software can run before encountering acrash. In operational definitions, the two metrics are de-fect density (rate) and mean time to failure (MTTF).The MTTF metric is most often used with safetycriti-cal systems such as the airline traffic control systems,avionics, and weapons. For instance, the U.S. govern-ment mandates that its air traffic control system cannotbe unavailable for more than three seconds per year. Incivilian airliners, the probability of certain catastrophicfailures must be no worse than 10-9 per hour (Little-wood and Strigini, 1992). The defect density metric, incontrast, is used in many commercial software systems.

Customer Satisfaction Metrics

Customer satisfaction is often measured by customer sur-vey data via the five-point scale:

• Very satisfied

• Satisfied

• Neutral

• Dissatisfied

SQM 9


• Very dissatisfied.

Satisfaction with the overall quality of the product andits specific dimensions is usually obtained through vari-ous methods of customer surveys. For example, the spe-cific parameters of customer satisfaction in software mon-itored by IBM include the CFUPRIMDSO categories(capability, functionality, usability, performance, relia-bility, installability, maintainability, documentation/information,service, and overall); for Hewlett-Packard they are FURPS(functionality, usability, reliability, performance, and ser-vice).Based on the five-point-scale data, several metrics withslight variations can be constructed and used, dependingon the purpose of analysis. For example:

1. Percent of completely satisfied customers

2. Percent of satisfied customers (satisfied and com-pletely satisfied)

3. Percent of dissatisfied customers (dissatisfied and com-pletely dissatisfied)

4. Percent of nonsatisfied (neutral, dissatisfied, and com-pletely dissatisfied)

SQM 10


Figure 5: Scopes of Three Quality Metrics

SQM 11


Usually the second metric, percent satisfaction, is used.In practices that focus on reducing the percentage of non-satisfaction, much like reducing product defects, metric(4) is used.

4 Quality Function Deployment

Definition

Quality Function Deployment, QFD, is a quality tech-nique which evaluates the ideas of key stakeholders toproduce a product which better addresses the customersneeds.Integrating customer requirements into product designQuality: Meeting the specificationsFunction: Function that forms qualityDeployment: Step-by-step deployment of that function

A structured method for translating the Voice of the Customer into

design requirements

A method to keep the organization focused on what is important to

the customer

A standard approach to present, document, track and create consen-

sus on customer needs

A technique to balance the voice of the executives.

QFD is the guiding design process for rapid, low costdevelopment of products that delight the customers. Thisversatile tool can be tailored to fit the needs of a verydiverse collection of projects.A common barrier to using QFD is the perceived com-plexity and subsequently large time commitment requiredfor implementation. However, project leadership andgood facilitation can clarify the mechanics of QFD and

SQM 12


significantly shorten the time required to complete thematrices of QFD.

QFD should not be:

A method to justify your own agendaA method to build cool looking charts to impress thebossA way to get out of the office and hang around with yourbuddiesAn exercise in futility, confusion, and aggravationA way to produce reports that are shelved

Why use QFD?

• Poor understanding of customer needs.

• Failure to strategically prioritize efforts.

• Willingness to take on unmanageable risks.

• Tendency toward unbuildable designs.

• Overreliance on formal specifications.

• Testing scenarios that fail to find key defects.

Benefits of QFD

1. Fewer, less expensive changes.

2. Less time, less cost of development.

3. Fewer start-up problems.

4. Lower warranty costs.

5. Delighted customers.

SQM 13


Figure 6: QFD

SQM 14


Characteristics of QFD

The 4 Main Phases to QFD

1.Product Planning including the House of Quality (Re-quirements Engineering Life Cycle)2.Product Design (Design Life Cycle)3.Process Planning -Design (Implementation Life Cycle)4.Process Control - Planning Production(Testing Life Cy-cle)

Phases of QFD

Phase 1-Led by the marketing department, Phase 1,or product planning, is also called The House of Qual-ity. Many organizations only get through this phase ofa QFD process. Phase 1 documents customer require-ments, warranty data, competitive opportunities, prod-uct measurements, competing product measures, and thetechnical ability of the organization to meet each cus-tomer requirement. Getting good data from the cus-tomer in Phase 1 is critical to the success of the entireQFD process.Phase 2-Phase 2 is led by the engineering department.Product design requires creativity and innovative teamideas. Product concepts are created during this phaseand part specifications are documented. Parts that aredetermined to be most important to meeting customerneeds are then deployed into process planning, or Phase3.Phase 3-Process planning comes next and is led by man-ufacturing engineering. During process planning, manu-facturing processes are flowcharted and process parame-ters (or target values) are documented.Phase 4-And finally, in the production planning, perfor-

SQM 15


mance indicators are created to monitor the productionprocess, maintenance schedules, and skills training foroperators. Also, in this phase decisions are made as towhich process poses the most risk and controls are putin place to prevent failures. The quality assurance de-partment in concert with manufacturing leads Phase 4.

QFD is a systematic means of ensuring that customerrequirements are accurately translated into relevant tech-nical descriptors throughout each stage of product devel-opment. Meeting or exceeding customer demands meansmore than just maintaining or improving product per-formance. It means building products that delight cus-tomers and fulfill their unarticulated desires.

Advantages of QFD

1. Customer / User involvement

2. Focus on customer needs

3. Team builder

4. Improve product or service quality

5. Shorter development cycles

6. Lower costs and greater productivity

SQM 16


Figure 7: QFD Planning Process

SQM 17


5 THE GOAL QUESTION METRIC

The Goal Question Metric (GQM) approach is basedupon the assumption that for an organization to mea-sure in a purposeful way it must first specify the goalsfor itself and its projects, then it must trace those goalsto the data that are intended to define those goals opera-tionally, and finally provide a framework for interpretingthe data with respect to the stated goals. Thus it is im-portant to make clear, at least in general terms, whatinformational needs the organization has, so that theseneeds for information can be quantified whenever possi-ble, and the quantified information can be analyzed a towhether or not the goals are achieved.

The result of the application of the Goal Question Met-ric approach application is the specification of a mea-surement system targeting a particular set of issues anda set of rules for the interpretation of the measurementdata. The resulting measurement model has three levels:

1. Conceptual level (GOAL): A goal is defined for anobject, for a variety of reasons, with respect to var-ious models of quality, from various points of view,relative to a particular environment. Objects of mea-surement are

• Products: Artifacts, deliverables and documentsthat are produced during the system life cycle;E.g., specifications, designs, programs, test suites.

• Processes: Software related activities normallyassociated with time; E.g., specifying, designing,testing, interviewing.

• Resources: Items used by processes in order toproduce their outputs; E.g., personnel, hardware,software, office space.

SQM 18


2. Operational level (QUESTION): A set of questions isused to characterize the way the assessment/achievementof a specific goal is going to be performed based onsome characterizing model. Questions try to charac-terize the object of measurement (product, process,resource) with respect to a selected quality issue andto determine its quality from the selected viewpoint.

3. Quantitative level (METRIC): A set of data is asso-ciated with every question in order to answer it in aquantitative way. The data can be

• Objective: If they depend only on the object thatis being measured and not on the viewpoint fromwhich they are taken; E.g., number of versions ofa document, staff hours spent on a task, size ofa program.

• Subjective: If they depend on both the objectthat is being measured and the viewpoint fromwhich they are taken; E.g., readability of a text,level of user satisfaction.

SQM 19


Figure 8: GQM Hierarchical structure

SQM 20


A GQM model is a hierarchical structure starting with a goal (spec-

ifying purpose of measurement, object to be measured, issue to be

measured, and viewpoint from which the measure is taken). The

goal is refined into several questions, such as the one in the example,

that usually break down the issue into its major components. Each

question is then refined into metrics, some of them objective such as

the one in the example, some of them subjective. The same metric

can be used in order to answer different questions under the same

goal. Several GQM models can also have questions and metrics in

common, making sure that, when the measure is actually taken, the

different viewpoints are taken into account correctly (i.e., the metric

might have different values when taken from different viewpoints).

THE GOAL QUESTION METRIC PROCESS

A GQM model is developed by identifying a set of qual-ity and/or productivity goals, at corporate, division orproject level; e.g., customer satisfaction, on-time deliv-ery, improved performance. From those goals and basedupon models of the object of measurement, we derivequestions that define those goals as completely as pos-sible. For example, if it is to characterize a softwaresystem (e.g., an electronic mail package, a word proces-sor) with respect to a certain set of quality issues (e.g.,portability across architectures), then a quality model ofthe product must be chosen that deals with those issues(e.g., list of functional features that can be implementedin different architectures).The next step consists in spec-ifying the measures that need to be collected in orderto answer those questions, and to track the conformanceof products and processes to the goals. After the mea-sures have been specified, we need to develop the datacollection mechanisms, including validation and analysismechanisms.The process of setting goals is critical by GQM approachand it is supported by specific methodological steps.

SQM 21


Figure 9: GQM Process

SQM 22


6 Quality Characteristics Tree

The quality characteristics are used as the targets for val-idation (external quality) and verification (internal qual-ity) at the various stages of development.They are refined into sub-characteristics, until the at-tributes or measurable properties are obtained.In this context, metric or measure is a defined as a mea-surement method and measurement means to use a met-ric or measure to assign a value.

In order to monitor and control software quality duringthe development process, the external quality require-ments are translated or transferred into the requirementsof intermediate products, obtained from development ac-tivities. The translation and selection of the attributesis a non-trivial activity, depending on the stakeholderpersonal experience, unless the organization provides aninfrastructure to collect and to analyze previous experi-ence on completed projects.

SQM 23


SQM 24


SQM 25


Quality factors

Each quality factors and its corresponding sub-factorsare defined as follows:

• Functionality: A set of attributes that relate tothe existence of a set of functions and their specifiedproperties.The functions are those that satisfy stated or impliedneeds.

– Suitability: Attribute of software that relatesto the presence and appropriateness of a set offunctions for specified tasks.

– Accuracy: Attributes of software that bare onthe provision of right or agreed results or effects.

– Security: Attributes of software that relate toits ability to prevent unauthorized access, whetheraccidental or deliberate, to programs and data.

– Interoperability: Attributes of software thatrelate to its ability to interact with specified sys-tems.

– Compliance: Attributes of software that makethe software adhere to application related stan-dards or conventions or regulations in laws andsimilar prescriptions.

• Reliability: A set of attributes that relate to thecapability of software to maintain its level of perfor-mance under stated conditions for a stated period oftime.

– Maturity: Attributes of software that relate tothe frequency of failure by faults in the software.

– Fault tolerance: Attributes of software that re-late to its ability to maintain a specified level ofperformance in cases of software faults or of in-fringement of its specified interface.

SQM 26


– Recoverability: Attributes of software that re-late to the capability to re-establish its level ofperformance and recover the data directly affectedin case of a failure and on the time and effortneeded for it.


• Usability: A set of attributes that relate to the ef-fort needed for use, and on the individual assessmentof such use,by a stated or implied set of users.

– Understandability: Attributes of software thatrelate to the users’ effort for recognizing the log-ical concept and its applicability.

– Learnability: Attributes of software that relateto the users’ effort for learning its application (forexample,operation control, input, output).

– Operability: Attributes of software that relateto the users’ effort for operation and operationcontrol.


• Efficiency: A set of attributes that relate to therelationship between the level of performance of thesoftware and the amount of resources used, understated conditions.

– Time behavior: Attributes of software that re-late to response and processing times and on through-put rates in performing its function.

SQM 27


– Resource behavior: Attributes of software thatrelate to the amount of resources used and theduration of such use in performing its function.


• Maintainability: A set of attributes that relate tothe effort needed to make specified modifications.

– Analyzability: Attributes of software that re-late to the effort needed for diagnosis of deficien-cies or causes of failures, or for identification ofparts to be modified.

– Changeability: Attributes of software that re-late to the effort needed for modification, faultremoval or for environmental change.

– Stability: Attributes of software that relate tothe risk of unexpected effect of modifications.

– Testability: Attributes of software that relateto the effort needed for validating the modifiedsoftware.

• Portability: A set of attributes that relate to theability of software to be transferred from one envi-ronment to another.

– Adaptability: Attributes of software that re-late to on the opportunity for its adaptation todifferent specified environments without apply-ing other actions or means than those providedfor this purpose for the software considered.

– Installability: Attributes of software that relateto the effort needed to install the software in aspecified environment.

SQM 28


– Conformance: Attributes of software that makethe software adhere to standards or conventionsrelating to portability.

– Replaceability: Attributes of software that re-late to the opportunity and effort of using it inthe place of specified other software in the envi-ronment of that software.

SQM 29


Figure 10: Characteristics and sub Characteristics of quality

SQM 30


Figure 11: Quality Characteristics and Guidelines for their Use

SQM 31


7 The FURPS Model

The FURPS model originally presented by Robert Grady.FURPS stands for:

• Functionality which may include feature sets, ca-pabilities and security

• Usability - which may include human factors, aes-thetics, consistency in the user interface, online andcontextsensitive help, wizards and agents, user doc-umentation, and training materials

• Reliability - which may include frequency and sever-ity of failure, recoverability, predictability, accuracy,and mean time between failure (MTBF)

• Performance - imposes conditions on functional re-quirements such as speed, efficiency, availability, ac-curacy, throughput, response time, recovery time,and resource usage

• Supportability - which may include testability, ex-tensibility, adaptability, maintainability, compatibil-ity, configurability, serviceability, installability, lo-calizability (internationalization)

The FURPS-categories are of two different types: Func-tional (F) and Non-functional (URPS). These categoriescan be used as both product requirements as well as inthe assessment of product quality.

Functional (what behaviors it does) and non-functional(how it does them)

• Functional requirements describe system behaviors

1. Priority: rank order the features wanted in im-portance

SQM 32


2. Criticality: how essential is each requirement tothe overall system?

3. Risks: when might a requirement not be satis-fied? What can be done to reduce this risk?

• Non-functional requirements describe other desiredattributes of overall system

1. Product cost (how do measure cost?)

2. Performance (efficiency, response time? startuptime?)

3. Portability (target platforms?), binary or byte-code compatibility?

4. Availability (how much down time is acceptable?)

5. Security (can it prevent intrusion?)

6. Safety (can it avoid damage to people or environ-ment?)

7. Maintainability (in OO context: extensibility, reusabil-ity)

SQM 33


Figure 12: FURPS classification

SQM 34


8 FURPS+ - Functionality

• All functional requirements

• Usually represent main product features.

– E.g. Order Processing requirements.

• Can also be architecturally significant.

– Auditing, Licensing, Localization, Mail, Onlinehelp, Printing, Reporting,Security, System man-agement, Workflow.

FURPS+

• Usability

– User interface issues such as accessibility, aes-thetics and consistency.

• Reliability

– Availability, accuracy, recoverability.

• Performance

– Throughput, response time, recovery time, start-up time.

• Supportability

– Testability, adaptability, maintainability, compat-ibility, configurability,installability, scalability andlocalizability.

SQM 35


FURPS+

• Design requirement

– Constrains the design.

– E.g. a relational database is required.

• Implementation requirement

– Constrains the coding or construction.

– E.g. required standards, platform or implemen-tation language.

• Interface requirement

– A requirement to interact with an external item.

• Physical requirement

– A physical constraint imposed on the hardwareused to house the system; for example, shape,size and weight.

9 GILB’ S Approach

Glib proposes four qualities attributes: workability, avail-ability, adaptability and usability, accompanied by theresource attributes of time, money, people and tools.

Workability

• Is defined as the raw ability of the system to do work,i.e. transact sign processing.

• Just as the quality and resource attributes are subdi-vided, sc each attribute may be further subdivided.

• Workability may be considered in terms of processcapacity, storage capacity and responsiveness, amongstother things.

SQM 36


Glib defines these terms in the following manner:

• Process capacity is the ability to process transac-tions within a given unit of time. storage capacityis the ability of the system to store things such asinformation.

• Responsiveness is a measure of the response to a sin-gle event.

SQM 37


Figure 13: Glib’s Attributes

SQM 38


Availability

• Availability is concerned with the proportion of elapsedtime that a system is able to be used. The sub-attributes highlighted here are reliability, maintain-ability and integrity.

Reliability

• Reliability is the degree to which the system doeswhat it is supposed to do.

• Because the purpose of each system is different, andthe purpose of different parts of the system is differ-ent, the way in which reliability is assessed may alsovary.

• Glib suggests that reliability may be assessed in termsof fidelity, veracity and viability for both logic ware(code) and data ware (data files), based upon ananalysis by Dickson (1972).

Maintainability

Maintainability refers to the process of fault handling.Some of the principal sub-attributes are:

1. Problem recognition time is the time required to rec-ognize that a fault exists.

2. Administrative delay is the time between recognitionof a problem and activity designed to rectify it.

3. Tool collection is the time required to gather all rel-evant information, e.g. program analysis and docu-mentation.

4. Problem analysis is the time needed to trace thesource of the problem.

SQM 39


5. Correction hypothesis time is the time required tocome up with a possible solution.

6. Inspection time is the time taken to evaluate saidsolution.

7. Active correction is the time to implement a hypoth-esized correction.

8. Testing is the time taken to adequately test cases tovalidate the change.

9. Test evaluation is the time needed to evaluate thetest results.

10. Recovery is the time required to recover and restorethe system.

SQM 40


Figure 14: Reliability criteria

SQM 41


Integrity

• The integrity of a system is a measure of its abilityto remain intact whilst under threat.

• It may be regarded as the ability of in-built securityfunctions to cope with threats.

• Threats may come from human action (deliberateor otherwise) or machine action, either hardware orsoftware driven.

• Integrity affects availability. A system with poor in-tegrity is likely to be unavailable for much of thetime.

Adaptability

Adaptability may be considered in terms of improvabil-ity, extendibility and portability.

• Improvability is the time taken to make minor changesto the system where the term ’ system’ is taken toinclude items such as documentation.

• Extendibility is the ease of adding new functionalityto a system.

• Portability is the ease of moving a system from oneenvironment to another.

Usability

• Usability may be considered as the ease of use andeffectiveness of use of a system.

• This may be considered in terms of handling ability,entry requirements, learning requirements and like-ability:

SQM 42


– Entry requirements are the basic human abilitiessuch as intelligence level, language proficiency orculture that are required to operate the system.

– Learning requirements are the resources, partic-ularly time, needed to attain a measurable levelof performance with the system.

– Handling ability is a measure of productivity af-ter error time is deducted.

– Likeability is a measure of how well people likethe system.

The resource attributes

The resource attributes highlighted by Glib include time,people, money and tools:

• Time resource is of two types: calendar timeto delivery and the time taken by the system oncedeveloped to carry out a task.

• People resources may be measured in terms ofman-years. However, this is a relatively crude ’ broad-brush’ approach since people resources are often gov-erned by scarce skills. In such cases, the availabilityof a particular person becomes a critical attribute.If you require a C programmer, all the COBOL pro-grammers in the world will not help you.

• Money resources concern both development andmaintenance costs. Since budgets are always a con-straint and many authors quote figures as high as80% for the proportion of software cost spent onmaintenance, this area is a favorite target for qualityimprovement programs.

• Tool resources encompass all physical resourcesfrom air- conditioning capacity to debuggers, a much

SQM 43


Figure 15: Constraints upon quality improvement

wider range of ’ tools’ than is conventionally consid-ered. Those tools which impose critical constraintsare those which should be carefully considered.

SQM 44


10 Quality Prompts - The COQUAMO project

Some of the most influential work in the UK in recentyears has been carried out by Kitchen ham and co-workers,resulting in the COQUAMO toolset.Their view of quality is based upon the five views of qual-ity setout by Garvin (1984) and described in detail in thefirst chapter of this book.Many of the ideas on measurement are based upon thework of Gilb. Garvin describes quality in terms of fiveviews: transcendent, product-based, user-based,manufacturing-based and value- based.In order to accommodate these differing views Kitchenham (1989b) introduces the concept of a ’ quality profile’, making a distinction between subjective and objectivemeasures of quality.

The quality profile is the view of the overall quality ofthe system, and is split into the following components:

• Transcendent Properties. These are qualitativefactors which are hard to measure, and about whichpeople have different views and definitions, for ex-ample, usability.

• Quality Factors. These are characteristics of thesystem which are made up of measurable factorscalled quality metrics and quality attributes. Thequality factors themselves are either subjective orobjective characteristics, for example, reliability andflexibility.

• Merit Indices. These subjectively define functionsof the system.They are measured by quality ratings,which are subjective value ratings.

SQM 45


Figure 16: Quality profile,after Kitchenham

SQM 46


The work has led to a constructive quality model knownas COQUAMO (Constructive Quality Model), namedafter the earlier COCOMO model (Constructive CostModel) of software economics, due to Boehm.This model forms the basis of tools developed to assistsoftware developers in their objective of supplying a highquality system.

The aim of this model is threefold:

• To predict final product quality

• To monitor progress towards a quality product

• To feed back the results to improve predictions forthe next project.

The model uses a similar approach to the earlier CO-COMO model (Boehm, 1981) and is delivered as threetools,-one to predict quality at the start (COQUAMO-1) -oneto monitor quality during development (COQUAMO-2)and-one to measure the quality of the final product (COQUAMO-3).

• The predictive and measuring tools are said to reflectthe user’ s view of quality;

• The monitoring tool is said to reflect a manufactur-ing or developer’ s view of quality.

SQM 47


Figure 17: The COQUAMO toolset

SQM 48


COQUAMO makes use of five types of quality drivers:

• Product attributes such as quality requirements, suc-cess criticality and difficulty of developing the prod-uct in question.

• Process attributes such as process maturity, tool useand method maturity.

• Personnel attributes such as staff experience and mo-tivation.

• Project attributes such as the, quality norms andleadership style

• Organizational attributes such as quality manage-ment and physical environment.

COQUAMO-1:Prediction

COQUAMO-1 can only consider those quality factorswhich are common to most applications and application-independent in nature. Thus it considers reliability, main-tainability, extensibility, usability and reusability as de-fined below.

• Reliability is defined as the expected time to nextfailure at the release date.

• Maintainabilit y is defined as the average elapsedtime to identify the cause of a fault once reported.

• Extensibility is defined as the average productivityachieved for code changes.

• Usability is defined as the expected time to next non-fault problem report.

• Reusability is defined as the effort used in creatingmodules (code and design) intended for reuse (a po-tential cost saving).

SQM 49


COQUAMO-2: Monitoring

COQUAMO-2 is based upon a set of guidelines to carryout a number of tasks to assist in the monitoring of qual-ity during a project. The guidelines set out to:

• identify appropriate metrics for each stage of the de-velopment process

• indicate methods for setting targets for project-levelmetrics

• suggest methods of analysis to identify unusual com-ponents

• indicate possible causes of unusual components anddeviations in performance, and indicate possible cor-rective action.

The automation of COQUAMO-2 is a complex task andat the time of writing is still a matter of research. Themanual guidelines have been tested and have proved worth-while in trials.

COQUAMO-3: Testing

COQUAMO-3 is intended to provide data about thequality of the end product.This is done both to validate the predictions made byCOQUAMO-1, and also to provide data for the use ofCOQUAMO-1 in future projects.

The current deliverables from this project show a numberof remaining limitations:

• They require a record of past performance, uponwhich predictions are based. This may not be avail-able or may be inapplicable if working practices arechanged to increase productivity, e.g. if CASE toolsare introduced.

SQM 50


• They are still dependent upon subjective assessment,although as the tools are used, these assessments aremodified in the light of experience.

• They exclude a number of common quality criteria,notably performance and portability. Some commoncriteria, e.g. usability, that are included are definedin an idiosyncratic way.

• The tools’ effectiveness cannot be empirically veri-fied.

SQM 51

Unit 2

Documents

quality measures

quality information

quality indicators

choice of quality goals

annual quality goals

quality goal statements

specic quality indicator

qualityintensive approach