Can Computer Make Mistakes? Quality in Software Development Alex Hauber – IBM Haifa Labs June 2004.

Can Computer Make Mistakes?

Quality in Software Development

Alex Hauber – IBM Haifa LabsJune 2004

Agenda

Quality Software Quality Problems ISO 9000 Quality in HRL : 3 paradigmsLife after ISO

Can Computers Make Mistakes?

•Computers are pervasive and getting more so by the day

•Can they make mistakes?•Can we afford it?

Agenda

QualityQuality Principles in IBM ISO 9000Quality in HRL : organization, processes, procedures

what is Quality?

what is Quality?

Customer Satisfaction =

Requirements + Expectations

A customer is the most important visitor on our premise. He is not dependent on us; we are dependent on him. He is not an interruptionour work; he is the purpose of it. He is not anoutsider on our business; he is a part of it. We arenot doing him a favor be serving him; he is doingus a favor by giving us an opportunity to do so.

Mahatma Gandhi

"... People expect more from IBM, Maintaining our customers' confidence inus is a matter of execution -- in everycustomer engagement, by every IBMer,every single day ..." Lou V. Gerstner

From the CEO ...

Any Quality initiative at IBM must be consistent with our first three guiding principles:

The Marketplace is the driving force behind everything we do.

At our core, we are a technology company with an overriding commitment to Quality.

Our primary measures of success are customer satisfaction and shareholder value.

From the CEO ...

In support of those principles, we have instituted a single Corporate Policy on IBM Quality. It states:

IBM has an overriding worldwide commitment to the quality of the products, solutions and services we provide to our customers. Quality is recognized as a fundamental component of the value customers receive from IBM.

IBM is committed to the goals of achieving total customer satisfaction, delivering superior products, solutions, services and exceeding customer requirements. Recognizing that the marketplace is the driving force behind everything we do, IBM implements effective business processes that support value creation for our customers and our stakeholders.

IBM leaders are responsible for establishing objectives and using measurements to drive continual improvement in Quality and in customer satisfaction. All IBMers are expected to contribute to continual improvement as an integral part of our Quality management system.

From the CEO ...

Beyond that renewed policy, our longstanding existing Corporate Instruction #105 requires that:

Each new IBM product is to be better in reliability, availability and serviceability than the comparable product it replaces, including competitive offerings.

Agenda


SW Product

Customer view

Functional Specs

Programmer view

From Requirements to Product

Software Quality Problems

•Once the software is developed, how do we know it is correct?

•Validation vs Verification•How do we assure RAS ?


•Software is almost always part of a system, a complex system

SQA Problems

•Dynamic Testing–black box–white box

•Static Testing•none is complete

• “In most cases, the projects that achieve the lowest defect rates, also achieve the shortest schedules”

• “IBM was the first company to discover that…” Jones 1991


•Can (and frequently does) begin with the coding !

Agenda`


ISO 9000 Principles

Say What You DoDo what You SayProve It

ISO 9000 Principles

Say What You Do - ProceduresDo what You Say - ImplementProve It - Audits

•- complexity

The need for Procedure

The Process Management Premise

• The quality of a (software) system is largely governed by the quality of the process used to develop and maintain it

• This premise implies focus both on process and on product

IBM Geographies

NAEMEA

LA

AP

S&D ESGSWG TG

PSG GS

Tier 1 Corporate Processes

Tier 2 GroupCustomized Processes

Tier 3 Locally Customized Processes / Procedures

IBM G Q M S Framework

Divisions

Geographies

Business Units

Tier 4 LocalBusiness Unit Procedures

GS - Global Services

SWGTG

- Software GroupTechnology Group

PSGESG

- Personal Systems GroupEnterprise Systems Group

S&D - Sales and Distribution Group

Hursley

ISO 9001:2000

Quality Management System Documentation

Document ControlControl of Quality RecordsInternal AuditControl of Nonconforming ProductCorrective ActionPreventive Action

Quality Management SystemScope/ExclusionsProcess Interaction

Product RealizationProcesses & subprocesses

Meet product requirementsMeasurable & consistent withQuality Policy

Commitment to:Comply with RequirementsContinual Improvement

Framework for Establishing & Reviewing Quality Objectives

Procedures

Processes

Manual

Objectives

Policy

Qua

lity

Rec

ords

Spe

cial

Typ

e of

Doc

umen

t

Server Group

Personal Systems Group

Software Group

Global Services

Sales and Distribution

Market Planning

Integrated Product Development

Integrated Ser- vice OfferingDevelopment

Integrated Supply Chain

Procurement

Human Resources

Legal

Finance

Chief Information Officer

Technology Group

Customer Relationship Management

Fulfillment(Hardware)

Fulfillment(Software)

Storage Systems Group

Marketing

Corporate Business Processesand Supporting Organizations

ISO 9000 Principles

Internal Audits - ongoingExternal Audits - once a year

Agenda


Ordered Development:

1st paradigm: Planning

Plan, Plan, Then Re-PlanIdentify Risks EarlyDevelop in Parallel

Planning Philosophy and Strategy

Plans are uselessPlanning is priceless

Planning Philosophy and Strategy


2nd paradigm: Design before Implementation

Development ProcessRegular Project

DOU/Contract

Maintenance

Process

ReviewProcess Test

ProcessFunc Spec

High Level Design

Low Level Design

CODING

FVT

SVT`PDP

QualityRecords

Development Process

Research ProjectsReviewProcess

DOU/Contract

Feasibility

DesignQualityRecords

PDP

Prototype

Go/No GoDecision

The spiral Model

• Start small and expand the scope of the project in increments, only after reducing the risk to an acceptable level


3rd paradigm: Reviews !!!

Reviews

•Design Reviews•Code Reviews

Reviews - is not ...

A Review is NOT ....

Reviews - is rather ...

A Review is rather ....

Review guidelines

define attendees and review leaderprepare and distribute all material beforehand

DO NOT present review material at reviewno need to write minuteswrite a review summary: subject, date, attendees, final outcome

document action items: action, responsible,target date

follow up until all a.i. are closed

Discussion rules

Listen to your peersYou can not argue with factsIt is not the one who shouts the loudest who is right

Code review

Peer reviewImperative to have Coding StandardsCycling – fringe benefit of code familiarity throughout the group

Life after ISO

•Reuse•OO – Java•OO – UML•Open Source

Effective Deployment of 6 Best Practices- by Rational

• 1.Develop software iteratively

• 2. Manage requirements

• 3. Use component-based architectures

• 4. Visually model software

• 5. Verify software quality

• 6. Control changes to software

Life after ISO

•CMMI – Capability Maturity Model Integration

--- The End ---

? You are not allowed to comment to outsiders regarding IBM's business performance or prospects anywhere in the world at anytime. The only exception to this rule is when such comments are carefully coordinated in advance with the appropriate legal, communications and corporate Investor Relations functions.

? You are not allowed to make any comment on rumors. When asked about any rumor -- including any rumor about business results, acquisitions or divestitures or management changes -- we say “IBM has a policy of not commenting on rumors”. This means we do not deny, affirm, wink or nod. The reason for this policy is simple -- once a company begins to deny baseless rumors, it may be obligated to affirm rumors that have some basis in fact. If a company denies a rumor that has some truth to it, the company is exposed to litigation for misleading the market

Comments on IBM business performance

Comments on IBM business performance

When executives do not follow these rules, companies lose millions in litigation, reputations are ruined and jobs are lost. There are no excuses for failure, and you need to make sure that all your people understand these rules.

Karl TaborskyDirector of Customer Satisfaction & QualityEMEA - A - 1020 Vienna

ISO 9000 Principles

Internal Audits - ongoingExternal Audits - once a year

--- Backup Foils ---

Plan, Plan, Then Re-PlanIdentify Risks EarlyDevelop in Parallel

Design Philosophy and Strategy

Letter of the CEO In S&T we have set ourselves a demanding standard for quality. Our objective is to provide a competitive system, products and services which fullymeet our internal and external customers' requirements first time, on time and every time. The quest for quality in S&T is part of overall IBM commitment to excellence. The policies and principles contained in this manual are based on the Corporate QualityValues and ISO 9000 standards, and are designed to help S&T to adopt a systematic approach to quality. Our customers, now and in the future, will judge S&T on the overall quality of theproducts and services we deliver. Quality means conformance to requirements. If we get it right, if we satisfy the customers' needs, they will continue to work with us. Animportant part of our business strategy is developing unique high quality software and hardware products. A continuous progress in all areas of our business, acquisition of new knowledge assets, increase in productivity and quality will continue to keep S&T superior to the competition. I am pleased to approve the contents of this Quality Manual and the quality procedures listed therein for immediate implementation in S&T. Dr. J. Raviv, CEO Science and Technology, IBM Israel October 1996

a

Senior Vice President &Group ExecutiveSoftware Group

S.M. (Steve) Mills

Solutions & StrategyApplication & Integration MiddlewareDatabase ManagementTivoli SystemsLotus Development

Senior Vice President & Group Executive

Sales & Distribution

J.M. (Mike) Lawrie

Global Small BusinessGlobal Midmarket BusinessGlobal Industry SectorsGlobal Business PartnersGlobal Net Genibm.comAmericasAsia PacificEurope/Middle East/Africa

Senior Vice President & Group ExecutiveGlobal Services

D.T. (Doug) Elix

Integrated Technology ServicesBusiness Innovation ServicesStrategic Outsourcing

Senior Vice President &Group ExecutiveGlobal Financing

J.C. (Joe) Lane

Senior Vice PresidentCommunications

J.C. (Jon) Iwata

Senior Vice President &Chief Financial Officer

J.R. (John) Joyce

Senior Vice PresidentGeneral Counsel

L.R. (Larry) Ricciardi

Senior Vice PresidentHuman Resources

J.R. (Randy) MacDonald

Senior Vice PresidentMarketing

A.F. (Abby) Kohnstamm

ResearchProcurement

Senior Vice PresidentStrategy

J.B. (Bruce) Harreld

Vice Chairmanof the Board

J.M. (John) Thompson

Senior Vice PresidentCorporate Technology

& Manufacturing

N.M. (Nick) Donofrio

General ManagerPersonal & Printing

Systems Group

R.W. (Bob) Moffat, Jr.

Personal ComputersPrinting Systems Div.Retail Store Solutions


Technology Group

J.E. (John) Kelly III

MicroelectronicssStorage Technology DivisionDisplaysAPTO


Server Group

W.M. (Bill) Zeitler

Enterprise ServersWeb ServersMid-Market ServersAppliance ServerxSeries


Storage Systems Group

L.S. (Linda) Sanford

Storage Networking SolutionsDisk ProductsTape ProductsStorage Management Software

CORPORATE ORGANIZATION

January 2002

IGF AmericasIGF Asia PacificIGF EMEAIGF Acquisitions & StrategicAlliancesGlobal Asset Recovery ServicesIBM Credit Union

Chief Executive Officer& Chairman

L.V. (Louis) Gerstner, Jr.

Board of Directors

President & Chief Operating Officer

S.J. (Sam) Palmisano

Business Transformation/CIOBusiness Transformation/CIO

HRL Quality System Structure

Project 2Project 1 Quality Rep

Group Manager

Department Manager

Lab Director

Quality Manager

Quality Coordinator

Quality Steering Committee

Rank 1996 1997 1998 1999 2000 1H01

1 HITACHI DELL DELL CISCO CISCO CISCO

2 DELL HP HITACHI DELL DELL DELL

3 HP COMPAQ HP IBM IBM IBM

4 COMPAQ HITACHI IBM GWAY HP GWAY

5 SUN SUN SUN HP GWAY HP

6 NOVELL IBM COMPAQ SUN SUN NEC

7 IBM DEC DEC HITACHI COMPAQ COMPAQ

8 TOSHIBA NOVELL NOVELL NOVELL FUJITSU SUN

9 MSOFT. UNISYS FUJITSU COMPAQ SAP AG FUJITSU

10 DEC NEC NEC NEC ORACLE SAP AG

Cisco 79.0

Dell76.5

IBM 75.8

Worldwide Overall Satisfaction:IBM Relative To Competition

Actual 1H01 = +0.3 Target YE01 = +1.5

Customer Satisfaction - Direct 1H02

Agenda


Agenda


Balanced View of Quality

Use internal and external performance measures to obtain

a balanced view of quality Internal product quality metrics

IWTSMeasure satisfaction with the people and

process from customer's point of

view

External Measure

People and Process

External Measure

Product Quality

PIQTSMeasure

satisfaction with the

product from customer's

point of view

Internal Measures

ISO 9000 Principles

origins in European Common Marketcustomer's point of viewbased on process control vs product controlrequired in ECCwhy HRL chose to adopt it

Can Computer Make Mistakes? Quality in Software Development Alex Hauber – IBM Haifa Labs June 2004.

Documents

ibm quality

iso slide

product slide

gerstner slide

procedures slide

quality initiative

complex system slide

quality management system