Evolution of Software/System Quality Second Asia-Pacific Conference on QUALITY SOFTWARE Dr. Raymond A. Paul December 11, 2001 NOV 2000.

Evolution of

Software/System Quality

Second Asia-Pacific

Conference on

QUALITY SOFTWARE

Dr. Raymond A. Paul

December 11, 2001NOV 2000

2

DefinitionDefinition

QUALITY is the characteristic that distinguishes the grade of excellence or superiority of a process, product, or service.

• In general usage, QUALITY means different things.Meaning of QUALITY varies considerably across specific disciplines and applications.

•What is Quality?

3

Historical Aspects Historical Aspects of QUALITY Evolutionof QUALITY Evolution

• Edward Deming– Postulated Statistical QUALITY Control Principles– Famous 14 Points of QUALITY Management– A Subset of these Principles successfully adapted by

Japanese Manufactures

• William Crosby– Emphasized Humanistic Behavioral Aspects of QUALITY

Improvement– Becoming More Important Now

4

JuranJuran

• QUALITY TrilogyA. QUALITY Planning

• Set of QUALITY Goals• Set Plans for Operations Based on these Goals

B. QUALITY Control• Responsible for Meeting QUALITY Goals• Prevent Adverse Changes• Set and Observe

� Performance Measures� Compare with Industry Standards� Benchmarking� Compare with Deming, CMM

C. QUALITY Improvement• Moving from Current Level to the Next Higher Level• Organize Teams, Train Operators to identify and Correct QUALITY Problems

TOTAL STATISTICAL QUALITY MANAGEMENT

5

Six Sigma InitiativesSix Sigma Initiatives

• GE and Motorola• Recognized Widely by Manufacturing Industry

– Adopted in Various Ways by Software Industry

• Major Results– Total QUALITY Management (TQM)– ISO Certification– Malcolm Baldridge Nation of QUALITY Award (Dept. of Commerce)

Six Sigma Means no more that 3.4 defects per million opportunities (six standard deviations away from the average of an assumed normal distribution)

• IBM (late Harlan Mills) Clean Room “follows” this Metric

6

Perspectives on QUALITY (Garvin 1984)Perspectives on QUALITY (Garvin 1984)

• Transcendented View– Something we recognize but cannot define

EXAMPLE: Beauty, Aristotle's Concept of Form

• User View– Fitness of Purpose

• Manufacturing (Design) View– Conformance to Specification

• Product View– Tied to inherent product functions and characteristics

• Value-Based View– Depends on the price the customer is willing to pay

VIEWS

•What is Quality?

7

QUALITY ModelsQUALITY Models

• McCall’s QUALITY Model (1977)– Shows relationships between external QUALITY Factors

such as correctness, reliability, usability, testability, etc. and Product QUALITY Criteria such as traceability, completeness, error tolerance, etc.

• PROBLEM: Difficult to measure these QUALITY Factors– No Standards, No Methods, No Tools

(A LOOSE INTERPRETATION)

•What is Quality?

8

Boehm's QUALITY Model (1986)Boehm's QUALITY Model (1986)

• Asserts that QUALITY software satisfies the needs of the users, designers, testers, and maintainers

• Relates software’s general utility to maintainability, reliability, testability, etc., to device independence, completeness, accessibility, etc.

• Very difficult to apply in practice, but easy to understand and learn

• Parts of it can be supported by tools; therefore, useful in a restricted sense

•What is Quality?

9

ISO 9126 QUALITY Model (1991)ISO 9126 QUALITY Model (1991)

• Consolidates many views of software QUALITY• ISO hierarchy is strict and non-overlapping unlike

McCall’s and Boehm's– EXAMPLE: In the model, maintainability depends ONLY on

analyzability, changeability, and stability. Whereas, in Boehm’s model, maintainability and understandability depend on structuredness, legibility, etc., in an overlapping way.

• Like Boehm’s model, it does not tell us how to measure characteristics such as security, accuracy, and interoperability.

•What is Quality?

10

Software QUALITY Characteristics DesiredSoftware QUALITY Characteristics Desired

• Functional Correctness� Functions and their specified properties

• Reliability� Attributes maintained by software under stated conditions

• Usability� Effort needed for users

• Efficiency� Software performance with a given set of resources

• Maintainability� Effort needed to make specified modifications

• Portability� Effort needed to transfer software from one environment to another

specified environment

(MOST MODELS INCLUDE THESE)

•What is Quality?

11

Comments on the Software Comments on the Software QUALITY ModelsQUALITY Models

• Some important characteristics such as evolvability, safety, and security are left out

• Dromey’s Model (1996) tries to address and serve some of these issues but not satisfactory

•What is Quality?

12

MATURITY Models (MM)MATURITY Models (MM)

• Specification Process Product

• Complete MM emphasizes– Process MATURITY

• What is MATURITY?– Methods of continuously improving a procedure, activity, or

process by planned measurements and feedbacks.– In essence, it emphasizes incremental but positive

improvement of an activity via repeated use and measurement.

(Design andImplementation)

• Measuring Quality

13

Analysis of Current MATURITY ModelsAnalysis of Current MATURITY Models

– One Dimensional– Does Not Include Items such as

• Maintenance• Application• Customer Satisfaction• Correctness with Respect to Specifications• Ease of Use, Ease of Change (maintenance)

– No Common Set of Assumptions– No Common Set of Measurements (no standardization)– No Financial Business Considerations– No Interoperability and Scalability Considerations

The emphasis in all these models INCREMENTAL!Incremental advances based on current experience.

NO LEAP FROGGING!!!


14

Paul’s CallPaul’s Call

MATURITY means growing older and wiser

• Collecting and retrieving experiences

• Measure, observe, gather feedback from the product and process,

update lessons learned from the product and process

• Maturity techniques continuously strive to improve processes,

procedures, and activities by repeated use, planned measurements,

and feedback

In essence, maturity emphasizes incremental but positive improvement

of an activity via repeated use, measurement, analysis of results of the

product, and feedback refinement


15

QuestionsQuestions

What to measure? When to measure? Where to measure?How to measure? How to analyze and interpret results?

Remember the problems of measurement...– Assumptions– Assessment– Analysis

…have to be standardized

One Method GQM Paradigm ( )Goal, Question, Metric (BASILI, et al.)

Not Very Effective– Have to customize the product design goal from designers that

could be different from users

goal-primary-samegoal-primary-different


16

Hierarchy of Maturity ModelsHierarchy of Maturity Models

• We Propose a Hierarchy of MATURITY Models for Improvement of IT Processes and Products

IT Maturity Model (ITMM)

Computing MM Communications MM

Hardware MM Software MM


17

Assistive Technology to Assistive Technology to Improve QUALITY of Life Improve QUALITY of Life

• These technologies assist users with physical disabilities. Quality of Life Improvement

• 47 million Americans are in some form disabled

• 75% of Americans with major disabilities are unemployed (President Clinton) Major Disability – Vision Impairment “World is so Visual”

• Key: Augmenting Human Productivity

• Designers do not have people with disabilities in mind when they develop their products

• Philosophy – Design products that work with more people

• Methods of Design– Universal Designs

– Specialized Designs


18

QUALITY ParametersQUALITY Parameters

• Usage Based – Utilitarian– Satisfaction of User’s Needs

• Functionality

• Ease of Use and Handling

• Troubleshooting– Restart and Recovery– Maintenance and Analyzability

• Technology-Based Aspects– Design Complexity – Maintainability– Evolutionary Flexibility

• Reconfigurability• Agility• Growth Enhancement


19

QUALITY Parameters QUALITY Parameters (cont.)(cont.)

• Economic Considerations– Cost to the Seller, and Price to the Buyer– Competitiveness with Similar Products– Ease of Manufacture, Reuse, and Maintenance

• Aesthetic Considerations– Form and Function– Appearance and Content– Appropriateness for the Purpose– Color, Packaging, Appearance– Attractiveness to Users– Attention-Grabbing Features– Entertaining, Interesting, Attention Keeping, e.g. TV/Websites

• Mass Appeal• Class/Culture Appeal• Individual Appeal


20

QUALITY Parameters QUALITY Parameters (cont.)(cont.)

• Personality/Individualization– To support and help physical preferences and disabilities– EXAMPLES

• Right vs. Left Handedness• Poor Vision• Hearing Impairment• Web Browsing – Personalization

� My YAHOO� My EXCITE

• Personalization for:� News Junkies� Financial Analysts� Customized Information


21

Another QUALITY Life Cycle ModelAnother QUALITY Life Cycle Model

Builtto Last

DemandGoodies

Make it Work

New Functions/Enhancements

Activities Processes Products Services Enterprises

Another QUALITY Life Cycle ModelAnother QUALITY Life Cycle ModelAnother QUALITY Life Cycle ModelAnother QUALITY Life Cycle ModelAnother QUALITY Life-Cycle ModelAnother QUALITY Life-Cycle Model

Life-Cycle Objectives of a Designer

Works the First Time

WorksMany Times

Use BecomesHabit Forming

Establishes User’sDependency and

Brand Loyalty (SONY)

• Achieving Quality

22

Designing Methodologies for Embedding Designing Methodologies for Embedding QUALITY-Based QUALITY-Based (QUALITY SUPPORTING)(QUALITY SUPPORTING) Functions Functions

• Major Characteristics of These Designs– Reliability, Fault Tolerance via:

• Redundancy• Highly Reliable Proven Components/Parts, Interconnections,

Networks• Design Methods

– Flexibility - Reconfigurability– Dynamic Introduction of:

• Standby Optional Functions• Universal Designs

• Component-Based, Easy-Assembly Design:– Building Block Based – Lego like– Assembly – Disassembly Principle


23

Designing Methodologies Designing Methodologies (cont.)(cont.)

• The Methods Presented Give Easy-Assembly (Disassembable) Subsystems– Good for Maintenance (Repair, Modification)– Subsystems and their integral components are loosely

coupled. This helps system understandability, analyzability, and reuse

– Helps to add on or enhance functions

• Complex Systems are Synthesized by:– Layered Architectures – each layer assembled from

components– Ring-Based Architectures– Good for Security


24

Designing Methodologies Designing Methodologies (cont.)(cont.)

Components (Building Blocks)

Assemblers

Subsystems

System (made of loosely coupled subsystems)

To Improve Performance Characteristics (in real-time) the System can be Tightly Coupled

A more compact high performance design can begeneralized by coalescing the layers (rings), i.e., tightcoupling of some layers – SYSTEM B

For Better System Performance, Real-Time Applications use Tight-Coupling Principles (SYSTEM B). SYSTEM A is used as a Prototype.

In coalescing several small components into a bigger element, we reduce the inter-component overhead. More importantly, we eliminate or reduce the conflicts among assumptions made during the design of the components.

SYSTEM A

SYSTEM B


25

Distributed System and Distributed System and Reconfiguration DesignReconfiguration Design

• QUALITY implied here includes:– Fault Tolerance– Control (Concurrency and Event Control)– Reconfigurability (Part of Control)– Heterogeneity

EXAMPLES:– CORBA Components and Distributed Control Technologies– Object-Oriented Design Components– NASA Space Shuttle Avionics

• Hardware and Software� Structured (Preferred) Reconfiguration Philosophy� Very Simple Software Reconfigurations


26

Universal DesignsUniversal Designs

• Primarily for Mass-Produced, Batch-Processed Hardware Systems• Holds True for Software Systems as well

PRINCIPLEThe System Includes Many Functions – Primary, Auxiliary, and Supporting Functions• User/Supplier triggers as activities those supporting and auxiliary functions needed for

that specific application or circumstance along with the primary functionsEXAMPLE

Pentium II MMX– Has 57 supporting instructions (functions) to improve multimedia

application processingUnderlying Philosophy

Design products that work for most applications (or people)

INSIGHT: Easier to mask out unwanted functions from a universal design (cheaper) than to build specialized functions from scratch (more expensive, time consuming, and labor intensive)


27

QUALITY Manager ConceptQUALITY Manager Concept

• QUALITY embraces many aspects, for example:– Dependability, Fault Tolerance– Maintainability– Information Hiding

• QUALITY is a dynamic property at anytime. Different aspects of quality become more important depending on the circumstance and needs of the USER at that moment.

• QUALITY manager is a watch dog function that “detects and supplies” the quality aspect needed by the system at that moment.

• A method of implementing the quality manager function– Remember that computer power is like cash. It can be used to buy:

• Fault Tolerance• Multi-Precision Computation• Support or Take Over Failed Computing Functions

Use a high-performance computing function whose “excess” computing power can be used for quality manager functions that can invoke and support real-time quality needs.


28

Projected Improvement in QUALITYProjected Improvement in QUALITY

• Service Industry Growing Very Rapidly (80% of Workers in US)• Service Industry’s Growth can be Traced to Increased Automation and Decreasing Labor-Intensive (manual)

Functions• Increased Automation means better QUALITY of Products and Services

EFFORT

Time

Industry GrowthAUTOMATION

•Evolution of Quality to a User Perspective

29

QUALITY PyramidQUALITY PyramidIn

creasi

ng

Com

ple

xit

y ENTERPRISE

SERVICES

PRODUCTS

PROCESSESS

ACTIVITIES

Qu

alit

y In

pu

t


30

Customer PrioritiesCustomer Priorities

Works the FirstTime –

All the Time

Work Horse Make it Last

Legacy Dependency - Afraid to Change

A B C D E

Make it Work

Legacy

Failure – No Option Part of Lifestyle

Reliability isTaken for Granted

Maintenanceand Cost are

No Issue

Afraid of Anything New –

Not Similar to the One Used

A B C D E

Usage History and QUALITY is Assessed

Customer is Reluctant to ChangeCustomer is Prepared

to Change

Usage Life Cycle

DemandC D E

Make it Last Forever

User’sDependency

• Changes in Quality•Evolution of Quality to a User Perspective

31

Evolution of QUALITY Evolution of QUALITY Consciousness and MeasuresConsciousness and Measures

A

B

C

D

E

(HARDWARE, SOFTWARE, INFORMATION TECHNOLGY, AND SERVICE)

Productivity&

Performance

Reliability

SQC

BehavioralConsiderations

Total QualityManagementand Control

INDUSTRIAL REVOLUTION: Assembly Line and Mass Production, Productivity Consciousness - Time studies and ELI Whitney’s Standards and InterchangeableParts, Ford’s Assembly Line (Productivity Emphasis) Early Operations Research

Machines can be produced rapidly, and they are complex -need for improved reliability. EXAMPLE: Steam Engines, Automobiles, Airplanes

Statistical Quality Control- Edward Deming,Shewhert

Man is not a machine. Quality can be improved by focusing on human behavior and industrial psychology. Rise of industrial unions, human factors, and ergonomic emphasis improve quality and productivity by creating better working environments. ( Crosby)

TQM & C – J. Juran – Trilogy of Quality Management, Most Popular and Important


32

Evolution of QUALITY Evolution of QUALITY Consciousness and Measures Consciousness and Measures (cont.)(cont.)

F

G

H

I

J

K

GE and Motorola - Moved from mostly mechanical parts to electronic parts and assemblies - Jack Welch of GE

Structured Programming, High-level languages, object-oriented design, and implementation languages

Structured Programming and Statistical Testing– Harlan Mills, Mike Fagan Inspection Procedures

Watts Humphrey, Bill Curtis

Initiative of DARPA for Communications Networks

SIX SIGMAQuality

Initiatives

SoftwareQuality

Initiatives and Methodologies

IBM Clean Room

Methods

CMM of SEI

QoS

IT MM To Be Developed


33

QUALITY Concerns Change withQUALITY Concerns Change with

• User Needs• Technology Evolution• Product and Service Sophistication

EXAMPLE:

HARDWARE HARDWARE&

SOFTWARE

HARDWARE&

SOFTWARE& COMMUNICATION

EMBEDDED&

DISTRIBUTEDSYSTEMS


34

QUALITY Concerns with ChangeQUALITY Concerns with Change

LOW

HIGH

HIGHComplexity of Change

Degradation of QUALITY

Due to Change

LOW


35

Service TripletsService Triplets

PROCESS PRODUCT SERVICE

• Current Software Quality FADs Emphasize Process

Aspects Only

• CMM Emphasizes Continuous Improvement in

Software Development Processes (Well-Defined

Activities)

• Specification Process Software Product

• CMM Emphasizes Process Maturity


36

Change of Customer QUALITY Priorities with Change of Customer QUALITY Priorities with Product (Software) Life CycleProduct (Software) Life Cycle

Customer Expectation Increasing Dependency on the Product

The Ultimate Milestone of QUALITY

Make it Work(Works the First

Time and Allthe Time)

PRODUCT USAGE LIFE CYCLE

User’s Dependencyon Product

User’s Expectation

(Product WorkHorse Ethic)

Builtto Last

User Afraidto Change

LegacyMentality


37

QUALITY – Customer PrioritiesQUALITY – Customer Priorities

Usage Life Cycle – How QUALITY is assessed by the customer

(EVOLUTION OF QUALITY CONSCIOUSNESS)

= Reliability

= Reliability and Maintainability

= Failure No Option

= Failure No Option

= Refining, Enhancing User Preferred Features

Make it Work-Works the First

Time –EVERYTIME

User’s Dependency

Work HorseEthic

Builtto Last

User Afraidto Change,

LegacyMentality

A B C D E

A

B

C

D

E


38

Quality of DatabasesQuality of Databases

• Databases are used widely and extensively in all kinds of enterprises and their use will only increase.

• But what is the view of database quality?• From a database manager point of view, database quality is terms of

database size, transaction speed, data reliability, scalability and data integrity.

• However, from a software designer point of view, quality may mean the maintainability, code reliability, and software architecture attributes.

• However, from the enterprise point of view, quality may mean the contribution of a database to the enterprise mission.


39

Quality from a Higher Level PerspectiveQuality from a Higher Level Perspective

• We need to think in terms of a high level perspective from the user point of view rather than a design or process points of view– Design view such as redundancy for fault tolerance, buffers

for scalability.– Process views such as following certain techniques, work

flow, or using specific kinds of tools such as configuration control.

• Furthermore, we should not think in terms of system requirements only, we need to think about the overall mission of the enterprise, operation, enterprise capabilities, and enterprise evolution point of view.


40

Enterprise EvolutionEnterprise Evolution

• Technology is changing rapidly at the speed of the Internet.• Enterprise mission also changes, but at a much slower speed.• Enterprise capability also needs to change with knowledge and

technology changes .– This is the trade-off between budget growths at X% per year and

Moore’s law [IT technical capability grows at 67% per year].

• An orderly transition of enterprise capabilities, while taking advantage of technology change, is badly needed.

• This also assumes that an enterprise has a sound strategy to know what is needed and how to incorporate new technologies and new knowledge growth plans into the existing infrastructure and superstucture.


41

SummarySummary

• QUALITY - particularly software quality - has to be more adaptive

• Products and processes grow with their application and human

environments. Quality criteria have to change with the situation,

circumstance, and the environment. These are dynamic characteristics.

Therefore, quality has to be dynamic and adaptive.

• Quality aspects change with application and product life cycles. Software is

the right place to put quality adoption and evolution.

• The new IT environment will be multidisciplinary, fast changing with rapid

diversification and globalization.

• THE QUALITY JOURNEY CONTINUES, BE PREPARED

•Summary

LEADERSHIPLEADERSHIP

•The position or office of a leader•Capacity or ability to lead•A continuum of good decisions•Provide direction•To influence

43

HOPE HOPE

• Michelangelo lived a few days shy of 89 years, still sculpting, painting, writing, and designing in an age when 90 was about 60 years beyond normal life expectancy was speaking to this idea of having high hopes and aims in this quotation (next slide) of his.

• I recall being in front of the statue of David in Florence and being transfixed. The size, the majesty, the spirit that seemed to jump right out of the marble was Michelangelo saying to all of us, “Aim high.”

• High aim indeed. Look at the at the Sistine Chapel where Michelangelo painted the ceiling by lying on his back working every day for 4 years. Lesser artists considered the task impossible, yet Michelangelo took it on and many more and aimed high

44

HOPEHOPE

The greater danger for most of us is not that our aim is too highand we miss it,

but that it is too lowand we reach it.

Michelangelo(1475-1564)

45

HOPE cont.HOPE cont.

• No one knows enough to be a pessimist!!!• Michelangelo’s advice is just as applicable today as it

was almost 500 years ago. Never listen to those who try to influence you with their pessimism. Follow these guidelines:– Refuse to listen to or internalize the proclamations of those

who point to your limitations. You must always remember:• when you argue for your limitations, the only thing you get are

limitations

• imagine Michelangelo telling you that you can create the quality you desire, and the great danger is not in having too much hope, but in reaching what you have perceived as hopeless

Evolution of Software/System Quality Second Asia-Pacific Conference on QUALITY SOFTWARE Dr. Raymond A. Paul December 11, 2001 NOV 2000.

Documents

quality software

quality garvin

definition quality

boehms quality model

product quality criteria

points of quality management

meeting quality goals

quality control responsible