OPRE 6364 1 Software Quality
OPR
E 63
641
Soft
ware
Qua
lity
OPR
E 63
642
Soft
war
e D
isas
ters
►G
reen
Par
ty C
onve
ntio
n fa
ils
(By
roun
ding
erro
r and
erro
neou
s us
e of
Exc
el th
e w
rong
num
ber
of d
eleg
ates
is c
ompu
ted,
200
2)
►M
ars
Clim
ate
Orb
iters
, Los
s (M
ixtu
re o
f pou
nds
and
kilo
gram
s, 1
999)
►Ar
iane
5, E
xplo
sion
(d
ata
conv
ersi
on o
f a to
o la
rge
num
ber,
1996
)►
Den
ver A
irpor
t(C
ompu
teriz
ed B
agga
ge H
andl
ing
fails
, 199
5)
►Pe
ntiu
m P
roce
ssor
, Div
isio
n Al
gorit
hm
(inco
mpl
ete
entri
es in
a lo
ok-u
p-ta
ble,
199
4)►
Patri
ot-S
cud
(roun
ding
erro
r, 19
91)
►N
ASA
Mar
iner
1 ,
Venu
s pr
obe
(per
iod
inst
ead
of c
omm
a in
FO
RTR
AN D
O-L
oop,
196
2)
OPR
E 63
643
Erro
r Ca
tego
ries
10 D
ocum
enta
tion
pro
blem
20 S
ynta
x30
Bui
ld40
Ass
ignm
ent
50 I
nter
face
60 E
rror
Han
dlin
g70
Dat
a80
Fun
ctio
n90
Sys
tem
100
Envi
ronm
ent
OPR
E 63
644
Def
ect c
lass
ifica
tion
•D
efec
ts a
re c
lass
ified
on
the
basi
s of
def
ect t
ype,
cla
ssan
d se
verit
y
•Type
is th
e ki
nd o
f def
ect c
ondi
tion.
–D
A (d
ata)
a de
fect
in in
tern
al d
ata
use
or s
peci
ficat
ion
–D
C (d
ocum
ent)
inad
equa
te, i
rrele
vant
or i
ncor
rect
des
crip
tion
–FN
(fun
ctio
nalit
y)an
inco
rrect
spe
cific
atio
n–
HF
(hum
an fa
ctor
s)a
defe
ct in
ope
ratio
nal p
roce
dure
or h
uman
inte
rface
–IF
(int
erfa
ce)
a de
fect
in th
e co
mm
unic
atio
n be
twee
n co
mpo
nent
s–
LO (l
ogic
)a
defe
ct in
pro
cedu
ral,
algo
rithm
ic, o
r con
trol l
ogic
–M
N (m
aint
aina
bilit
y) th
e co
mpo
nent
can
not b
e m
aint
aine
d ea
sily
–PF
(per
form
ance
)op
erat
iona
l effi
cien
cy m
ay g
et s
acrif
iced
–SN
(syn
tax)
a de
fect
in la
ngua
ge u
sage
–ST
(sta
ndar
ds)
a de
partu
re fr
om re
pres
enta
tiona
l sta
ndar
ds–
OT
(oth
er)
a de
fect
con
ditio
n th
at h
as n
ot b
een
spec
ified
OPR
E 63
645
Softw
are
Qua
lity
Assu
ranc
e•
Softw
are
Qua
lity
invo
lves
–En
surin
g co
nfor
man
ce w
ith u
ser r
equi
rem
ents
–Id
entif
ying
def
ects
in e
arly
pha
ses
of d
evel
opm
ent
–M
onito
ring
the
prod
ucts
thro
ugh
its p
hase
s of
de
velo
pmen
t•
Qua
lity
assu
ranc
e du
ring
the
deve
lopm
ent
–Ac
hiev
ed th
roug
h St
ruct
ured
Wal
kthr
ough
s(a
lso
know
n as
Sof
twar
e In
spec
tion)
•Q
ualit
y As
sura
nce
afte
r dev
elop
men
t–
Achi
eved
thro
ugh
pre-
plan
ned
Test
ing
of th
e en
d pr
oduc
t
OPR
E 63
646
Tota
l Q
ual
ity
Man
agem
ent
princi
ple
s an
d t
ools
apply
!Cont
inuous
Process I
mprovemen TQ
M
Cust
omer
Sat
isfa
ctio
n:10
0% C
ompl
ianc
e wi
th E
xpec
tati
ons
Statisti
cal Pr
ocess
Control
t
OPR
E 63
647
“So
Wha
t?”
•Be
caus
e a
SW is
bui
lt us
ing
proc
esse
s•
Proc
esse
s ar
e no
t alw
ays
perfe
ct•
We
can
impr
ove
whi
le a
void
ing
the
Dilb
ert
synd
rom
e•
Rem
embe
r, w
e ca
n’t t
est q
ualit
y in
to o
ur
softw
are,
we
have
to d
esig
n it
in!
OPR
E 63
648
Dem
ing’s
PD
CA C
ycle
Plan
Do
Che
ckA
ct
Wha
t’s th
e ne
xt p
robl
em?
Wha
t can
we
do?
Try
impr
ovem
ent
Smal
l tes
t
Gat
her d
ata
And
ana
lyze
Ado
pt n
ew p
roce
ss
OPR
E 63
649
Dilb
ert
Cyc
le
Ado
pt n
ewU
np
roven
pr
oces
sA
ctM
ake
wil
d g
uess
At w
hat i
s wro
ng
OPR
E 63
6410
Moder
n Q
ual
ity
Man
agem
ent
•M
oder
n qu
ality
man
agem
ent
–re
quire
s cu
stom
er s
atis
fact
ion
–pr
efer
s pr
even
tion
to in
spec
tion
–re
cogn
izes
man
agem
ent r
espo
nsib
ility
for
qual
ity•
Not
ewor
thy
qual
ity e
xper
ts in
clud
e D
emin
g,
Jura
n, C
rosb
y, Is
hika
wa,
Tag
uchi
, and
Fe
igen
baum
OPR
E 63
6411
Rec
all Q
ual
ity
Guru
s…
Dem
ing
was
fam
ous
for h
is w
ork
in re
build
ing
Japa
n an
d fo
r his
14
poin
tsJu
ran
wro
te th
e Q
ualit
y C
ontro
l Han
dboo
k an
d th
e 10
ste
ps to
qua
lity
impr
ovem
ent
Cro
sby
wro
te “Q
ualit
y is
Fre
e” a
nd s
ugge
sted
that
or
gani
zatio
ns s
houl
d st
rive
for z
ero
defe
cts
Ishi
kaw
a de
velo
ped
the
conc
ept o
f qua
lity
circ
les
and
the
use
of fi
shbo
ne d
iagr
ams
Tagu
chi d
evel
oped
met
hods
for o
ptim
izin
g th
e pr
oces
s of
eng
inee
ring
expe
rimen
tatio
nFe
igen
baum
dev
elop
ed th
e co
ncep
t of t
otal
qua
lity
cont
rol
OPR
E 63
6412
Lead
ersh
ip c
an’t b
e es
caped
!
•“It
is m
ost i
mpo
rtant
that
top
man
agem
ent b
e qu
ality
-m
inde
d. In
the
abse
nce
of s
ince
re m
anife
stat
ion
of
inte
rest
at t
he to
p, li
ttle
will
happ
en b
elow
.” (J
uran
, 194
5)•
A la
rge
perc
enta
ge o
f qua
lity
prob
lem
s ar
e as
soci
ated
w
ith m
anag
emen
t, no
t tec
hnic
al is
sues
OPR
E 63
6413
More
Soft
war
e D
isas
ters
…►
Rad
io T
eles
cope
VLA
, cal
ibra
tion
(roun
ding
erro
r, 19
90-1
995)
►El
ectio
ns in
Sch
esw
igH
olst
ein/
Ger
man
y (R
ound
ing
lifts
the
Gre
en P
arty
from
4.9
7% to
5.0
%, 1
992)
►
Slei
pner
Offs
hore
Pla
tform
(Sin
king
cau
sed
by th
e w
rong
use
of F
E-co
de N
ASTR
AN, 1
991)
►AT
&T lo
ng d
ista
nce
serv
ice
fails
for n
ine
hour
s(W
rong
BR
EAK
stat
emen
t in
C-C
ode,
199
0)►
Phob
os1,
Rus
sian
Mar
s Pr
obe
lost
(Wro
ng c
omm
and
lead
s to
rota
tion,
198
8)►
Airb
us d
owni
ng d
urin
g Ira
n-co
nflic
t(P
atte
rn re
cogn
ition
sof
twar
e, 1
988)
►Fa
lse
alar
m in
Sov
iet e
arly
-war
ning
mon
itorin
g sy
stem
(Pat
tern
reco
gniti
on, 1
983)
OPR
E 63
6414
The
Cost
of Q
ual
ity
Cos
t of q
ualit
y is
–
The
cost
of c
onfo
rman
ce o
r del
iver
ing
prod
ucts
that
mee
t req
uire
men
ts a
nd fi
tnes
s fo
r use
–Th
e co
st o
f non
conf
orm
ance
or t
akin
g re
spon
sibi
lity
for f
ailu
res
or n
ot m
eetin
g qu
ality
exp
ecta
tions
OPR
E 63
6415
Cost
s Pe
r H
our
of
Dow
ntim
e Cau
sed
by
Soft
war
e D
efec
ts
Bus
ines
sC
ost p
er H
our
Dow
ntim
e
Aut
omat
ed te
ller m
achi
nes (
med
ium
-siz
ed b
ank)
$14,
500
Pack
age
ship
ping
serv
ice
$28,
250
Tele
phon
e tic
ket s
ales
$69,
000
Cat
alog
sale
s cen
ter
$90,
000
Airl
ine
rese
rvat
ion
cent
er (s
mal
l airl
ine)
$89,
500
In a
dditio
n,
poor
qual
ity
may
cost
you
fore
goin
g r
epea
t busi
nes
s
OPR
E 63
6416
Five
Cost
Cat
egories
Rel
ated
to Q
ual
ity
•Pr
even
tion
cost
: the
cos
t of p
lann
ing
and
exec
utin
g a
proj
ect s
o it
is e
rror-f
ree
or w
ithin
an
acce
ptab
le e
rror
rang
e•
App
rais
al c
ost:
the
cost
of e
valu
atin
g pr
oces
ses
and
thei
r ou
tput
s to
ens
ure
qual
ity•
Inte
rnal
failu
re c
ost:
cost
incu
rred
to c
orre
ct a
n id
entif
ied
defe
ct b
efor
e th
e cu
stom
er re
ceiv
es th
e pr
oduc
t•
Exte
rnal
failu
re c
ost:
cost
that
rela
tes
to a
ll er
rors
not
de
tect
ed a
nd c
orre
cted
bef
ore
deliv
ery
to th
e cu
stom
er•
Mea
sure
men
t and
test
equ
ipm
ent c
osts
: cap
ital c
ost o
f eq
uipm
ent u
sed
to p
erfo
rm p
reve
ntio
n an
d ap
prai
sal
activ
ities
OPR
E 63
6417
Tw
o a
spec
ts o
f Soft
war
e Q
ual
ity
Proc
ess
Qua
lity
–En
surin
g co
nfor
man
ce w
ith u
ser r
equi
rem
ents
–Id
entif
ying
def
ects
–M
onito
ring
the
prod
uct t
hrou
gh it
s ph
ases
of
deve
lopm
ent
Prod
uct Q
ualit
y–
Iden
tifyi
ng u
ser s
peci
fied
qual
ity n
eeds
–Pr
iorit
izin
g qu
ality
nee
ds–
Res
olvi
ng q
ualit
y co
nflic
ts, i
f any
–Bu
ildin
g th
em in
to th
e de
velo
pmen
t pro
cess
–Al
loca
ting
effo
rt an
d tim
e fo
r the
m
OPR
E 63
6418
Mal
colm
Bal
drige
Aw
ard a
nd
ISO
9000
•Th
e M
alco
lm B
aldr
ige
Qua
lity
Awar
d w
as s
tarte
d in
19
87 to
reco
gniz
e co
mpa
nies
with
wor
ld-c
lass
qu
ality
•IS
O 9
000
prov
ides
min
imum
requ
irem
ents
for a
n or
gani
zatio
n to
mee
t the
ir qu
ality
cer
tific
atio
n st
anda
rds
•H
owev
er, n
one
assu
re th
e qu
ality
of t
he fi
nal
prod
uct!
OPR
E 63
6419
Qual
ity
Plan
nin
g
•It
is im
porta
nt to
(a)
“des
ign
in” q
ualit
y an
d (b
) co
mm
unic
ate
with
in th
e or
gani
zatio
n th
e im
porta
nt
fact
ors
that
dire
ctly
con
tribu
te to
mee
ting
the
cust
omer
’s re
quire
men
ts•
Tool
s su
ch a
s D
esig
ned
Expe
rimen
tatio
n ca
n id
entif
y fa
ctor
s ha
ve th
e m
ost i
nflu
ence
on
the
over
all
outc
ome
of a
pro
cess
•M
any
“sco
pe” a
spec
ts o
f IT
proj
ects
als
o af
fect
cu
stom
er s
atis
fact
ion
or q
ualit
y, s
uch
as fu
nctio
nalit
y,
feat
ures
, sys
tem
out
puts
, per
form
ance
, rel
iabi
lity,
and
m
aint
aina
bilit
y
OPR
E 63
6420
Qual
ity
Ass
ura
nce
•Q
ualit
y as
sura
nce
incl
udes
all
the
activ
ities
rela
ted
to
satis
fyin
g th
e re
leva
nt q
ualit
y st
anda
rds
for a
pro
ject
•An
othe
r goa
l of q
ualit
y as
sura
nce
is c
ontin
uous
qu
ality
impr
ovem
ent
•Be
nchm
arki
ng c
an b
e us
ed to
gen
erat
e id
eas
for
qual
ity im
prov
emen
ts•
Qua
lity
audi
ts h
elp
iden
tify
less
ons
lear
ned
that
can
im
prov
e pe
rform
ance
on
curre
nt o
r fut
ure
proj
ects
OPR
E 63
6421
Qual
ity
Contr
ol
•Th
e m
ain
outp
uts
of q
ualit
y co
ntro
l are
–ac
cept
ance
dec
isio
ns–
rew
ork
–pr
oces
s ad
just
men
ts•
Som
e to
ols
and
tech
niqu
es in
clud
e–
pare
to a
naly
sis
–st
atis
tical
sam
plin
g–
qual
ity c
ontro
l cha
rts–
test
ing
OPR
E 63
6422
Freq
uen
tly
sought
afte
r qual
ity
fact
ors
Use
r's N
eed
Use
r's C
once
rnQ
ualit
y Fa
ctor
FUN
CTI
ON
ALH
ow s
ecur
e is
it ?
INTE
GR
ITY
How
ofte
n w
ill it
fail
?R
ELIA
BILI
TYC
an it
sur
vive
dur
ing
failu
re ?
SUR
VIVA
BILI
TYH
ow e
asy
is it
to u
se ?
USA
BILI
TYPE
RFO
RM
ANC
EH
ow m
uch
reso
urce
s ar
e ne
eded
?EF
FIC
IEN
CY
Doe
s it
com
ply
with
requ
irem
ents
?C
OR
REC
TNES
SD
oes
it pr
even
t haz
ards
?SA
FETY
Doe
s it
inte
rface
eas
ily ?
INTE
RO
PER
ABIL
ITY
CH
ANG
EH
ow e
asy
is it
to re
pair
?M
AIN
TAIN
ABIL
ITY
How
eas
y is
it to
exp
and
?EX
PAN
DAB
ILIT
YH
ow e
asy
is it
to c
hang
e ?
FLEX
IBIL
ITY
How
eas
y is
it to
tran
spor
t ?PO
RTA
BILI
TYIs
it re
usab
le in
oth
er s
yste
ms
?R
EUSA
BILI
TYM
ANAG
EMEN
TIs
per
form
ance
ver
ifica
tion
easy
?VE
RIF
IABI
LITY
Is th
e so
ftwar
e ea
sily
man
aged
?M
ANAG
EABI
LITY
OPR
E 63
6423
Req
uirem
ents
Anal
ysis
Ana
lyze
nee
dfo
r qua
lity
Con
vert
qual
ityne
eds t
ore
quire
men
ts
Doc
umen
t SW
Qua
lity
requ
irem
ents
Use
r opi
nion
s
Crit
eria
for g
ood
requ
irem
ents
Req
spec
Der
ived
req’
s
Qua
lity
fact
ors
Nee
ds d
ata
base
Qua
lity
conf
licts
Cos
t of q
ualit
y
Fact
or a
nd c
riter
ia d
efin
ition
s
Leve
l of q
ualit
y m
atrix
Qua
lity
need
s dat
a ba
se
Engi
neer
ing
crite
ria
Trac
eabi
lity
mat
rix
Softw
are
qrs
Qua
lity
spec
ifica
tion
guid
elin
e
OPR
E 63
6424
The
Qual
ity
Nee
ds
dat
abas
e
•N
EED
Text
ual d
escr
iptio
n of
the
need
•SO
UR
CE
Prec
ise
desc
riptio
n of
the
sour
ce•
IMPO
RTA
NC
EPe
rcei
ved
impo
rtanc
e of
the
need
•R
EQU
IREM
ENT
Keyw
ord
for e
ach
need
•SU
BJEC
TKe
ywor
d fo
r the
sub
ject
of t
he n
eed
OPR
E 63
6425
Exa
mple
of
a U
ser
Nee
ds
Dat
abas
e
Nee
dSo
urce
Im
port
ance
R
equi
rem
ent
S
ubje
ct
Prec
isio
n an
d ac
cura
cy o
f the
P. S
mith
8
prec
ise
arith
met
icin
puts
shou
ld b
e pr
eser
ved
in10
/9/9
9al
l cal
cula
tions
.
Emph
asis
shou
ld b
e on
faul
tR
P, p
g 3
5
faul
tde
sign
tole
ranc
e.pa
ra2
tole
rant
Cod
e sh
ould
be
rege
nera
ble
SDD
,
7
com
plet
esu
ppor
tus
ing
deliv
ered
softw
are.
Pg 1
0,pa
ra 4
softw
are
Softw
are
to b
e us
ed b
yR
P, p
g 2
8
user
desi
gnN
on-te
chni
cal p
eopl
epa
ra4
frie
ndlin
ess
OPR
E 63
6426
Prio
ritizi
ng Q
ual
ity
Nee
ds
•O
ne p
ossi
ble
met
ric c
ould
be:
–E
(Exc
elle
nt)
G
(Goo
d)
A
(Ave
rage
)
NI (
Not
an
issu
e)•
Clie
nt’s
cho
ice
is b
ased
on
–Po
ssib
le q
ualit
y co
nflic
ts–
Cos
t•
Qua
lity
does
not
com
e fre
e an
d re
quire
s ad
ditio
nal e
ffort
and
cost
Rep
ercu
ssio
n on
revi
ew a
ctiv
ity, f
or e
xam
ple,
Qua
lity
Nee
d
Rev
iew
s ne
eded
Not
an
issu
e
int
erna
l rev
iew
cyc
le, i
nter
nal w
alkt
hrou
ghs
Aver
age
inte
rnal
revi
ew c
ycle
, for
mal
des
ign
revi
ew (s
ubse
t)G
ood
inte
rnal
revi
ew c
ycle
, ins
pect
ions
, for
mal
des
ign
revi
ews
(full
set)
Exce
llent
inte
rnal
revi
ew c
ycle
, ins
pect
ions
, for
mal
des
ign
revi
ew, I
V &
V
OPR
E 63
6427
-Ve
rifia
bilit
y (1
5)+
+-
Usa
bilit
y (1
4)+
--
+-
--
-Su
rviv
abilit
y (1
3)+
++
+-
Safe
ty (1
2)-
-+
++
--
Reu
sabi
lity
(11)
+-
Rel
iabi
lity
(10)
-Po
rtabi
lity
(9)
++
+M
anag
eabi
lity
(8)
++
++
-M
aint
aina
bilit
y (7
)-
-In
tero
pera
bilit
y (6
)-
Inte
grity
(5)
--
+-
-Fl
exib
ility
(4)
--
+-
-Ex
pand
abilit
y (3
)-
--
Effic
ienc
y (2
)+
++
++
Cor
rect
ness
(1)
1514
1312
1110
98
76
54
32
1Q
ualit
y N
eeds
in
tera
ct
OPR
E 63
6428
Res
olvi
ng Q
ualit
y C
onfli
cts
Alo
ng w
ith t
he
clie
nt,
go b
ack
to t
he
leve
l-of
-im
port
ance
and a
dju
st it
by
usi
ng t
he
follo
win
g p
roce
dure
:1.
For
the
each
soft
war
e co
mponen
t st
arting f
rom
the
firs
t co
lum
n,
2.
For
each
conflic
ting r
elat
ionsh
ip (
-) b
etw
een f
acto
r A a
nd f
acto
r B,
3.
Adju
st t
he
two lev
els
of
qual
ity
usi
ng t
he
follo
win
g t
able
(this
tab
le is
for
guid
ance
only
).If
spec
ifie
d f
or
Fact
or
AThen
at
most
for
Fact
or
BE
NI
GA
AG
NI
E4.
Rev
isit p
revi
ousl
y ad
just
ed r
elationsh
ip t
o m
ake
sure
that
they
hav
e not
bee
n c
han
ged
OPR
E 63
6429
Build
ing q
ual
ity
in d
evel
opm
ent
pro
cess
The
resu
lting
qua
lity
fact
ors,
alo
ng w
ith t
heir
leve
ls,
ente
r th
e So
ftwar
e R
equi
rem
ent S
peci
ficat
ion
docu
men
t as
attr
ibut
es.
For
exam
ple,
if
we
are
follo
win
g IE
EE S
RS
Doc
umen
tatio
n St
anda
rd
(Pro
toty
pe
outli
ne
1)
as
the
SRS
docu
men
tatio
n pr
actic
e, th
e qu
ality
req
uire
men
ts w
ill ge
t pla
ced
in s
ectio
n 3.
5 as
giv
en b
elow
:1.
. . .
2.. .
.3.
Spec
ific
requ
irem
ents
3.1
Func
tiona
l req
uire
men
ts3.
2Ex
tern
al in
terfa
ce re
quire
men
ts3.
3Pe
rform
ance
requ
irem
ents
3.4
Des
ign
cons
train
ts3.
5A
ttrib
utes
3.6
Oth
er re
quire
men
ts
OPR
E 63
6430
Docu
men
tation
•R
EQU
IREM
ENT
Text
of a
ctua
l req
uire
men
t•
SOU
RC
ESo
urce
of r
equi
rem
ent
•C
RIT
ERIO
NC
riter
ion
the
requ
irem
ent b
elon
gs to
•D
ESIG
N‘x
’ if r
equi
rem
ent i
s ve
rifia
ble
in th
ede
sign
pha
se•
CO
DE
‘x’ i
f req
uire
men
t is
verif
iabl
e in
the
codi
ng p
hase
•TE
ST‘x
’ if r
equi
rem
ent i
s ve
rifia
ble
in th
ete
stin
g ph
ase
OPR
E 63
6431
Adju
stin
g ef
fort
and
sche
dule
CO
CO
MO
stu
dy h
as b
een
exte
nded
and
can
now
be
used
to
giv
e an
idea
of
the
extra
cos
t ne
eded
for
the
qua
lity
conc
erns
. The
var
iatio
n in
the
CO
CO
MO
cos
t an
d tim
e sc
hedu
le c
an b
e as
giv
en in
the
tabl
e be
low
.
Qua
lity
leve
lR
elat
ive
Cos
tR
elat
ive
Sche
dule
Not
an
issu
e0.
69 to
1.0
00.
89 to
1.0
0Av
erag
e0.
90 to
1.1
60.
97 to
1.0
5G
ood
1.10
to 1
.30
1.03
to 1
.09
Exce
llent
1.48
to 2
.06
1.13
to 1
.26
OPR
E 63
6432
Qua
lity
by S
truct
ured
Wal
kthr
ough
•Is
use
d to
ver
ify in
telle
ctua
l pro
duct
s by
man
ually
ex
amin
ing
the
deve
lope
d pr
oduc
t, a
piec
e at
a ti
me,
by
sm
all g
roup
s of
pee
rs•
Expl
oits
the
syne
rgy
crea
ted
by a
sm
all g
roup
of
peop
le w
orki
ng to
geth
er•
Perfo
rms
exam
inat
ion
of w
ork
prod
ucts
at d
efin
ed
chec
kpoi
nts
•U
ses
a de
fined
six
ste
p pr
oced
ure
•Ke
eps
reco
rd o
f erro
rs d
etec
ted
for q
ualit
y co
ntro
l an
d pr
oces
s m
anag
emen
t•
Is n
ot a
foru
m fo
r des
ign
deci
sion
s, n
or it
is a
br
ains
torm
ing
sess
ion
OPR
E 63
6433
Stru
ctur
ed W
alkt
hrou
gh In
spec
tion
Req
uire
s m
etic
ulou
s pl
anni
ng a
bout
•W
hat c
an b
e in
spec
ted
•W
hen
it ca
n be
insp
ecte
d•
Who
can
insp
ect i
t•
Wha
t pre
para
tion
is n
eede
d•
How
the
insp
ectio
n is
to b
e co
nduc
ted
•W
hat d
ata
need
s to
be
colle
cted
•W
hat f
ollo
w u
p ac
tion
is re
quire
d
OPR
E 63
6434
The
Insp
ectio
n te
am
•Pr
oced
ural
role
s–
Auth
or–
Mod
erat
or–
Rea
der
–R
ecor
der
–In
spec
tor
•Fu
nctio
nal v
iew
poin
ts–
Lead
er–
Req
uest
er–
Peer
–R
ecei
ver
–Va
lidat
or–
stan
dard
izer
OPR
E 63
6435
Gui
delin
es fo
r tea
m
•Th
e m
inim
um s
ize
shou
ld b
e th
ree
: a
mod
erat
or/re
cord
er, a
read
er a
nd a
n au
thor
. Al
l tea
m
mem
bers
are
insp
ecto
rs.
•Te
am s
houl
d no
t hav
e m
ore
than
sev
en p
erso
ns.
•Th
e in
spec
tion
is a
pee
r pro
cess
. No
man
ager
s sh
ould
be
invo
lved
. The
se a
re te
chni
cal a
nd n
ot
man
agem
ent m
eetin
gs.
•O
ther
mem
bers
cho
sen
shou
ld b
e fa
milia
r with
the
insp
ectio
n pr
oces
s as
wel
l as
the
wor
k ar
ea.
OPR
E 63
6436
Step
1 :
Plan
ning
Purp
ose
: Org
aniz
atio
nR
oles
: Mod
erat
or, A
utho
rTa
sks
: App
rove
ent
ry c
riter
iaEs
tabl
ish
sche
dule
sD
esig
nate
par
ticip
ants
Det
erm
ine
over
view
Dis
tribu
te m
ater
ial
Prep
are
insp
ectio
n m
eetin
g no
tice
Inpu
ts: C
ompl
eted
wor
k pr
oduc
t, In
spec
tion
rate
sO
utpu
ts: M
eetin
g sc
hedu
le, I
nspe
ctio
n m
ater
ial
OPR
E 63
6437
Insp
ectio
n R
ates
Dev
elop
men
t Sta
geO
VIEW
Rat
ePR
EP R
ate
INSP
Rat
e
Req
uire
men
ts50
025
025
0Pr
elim
inar
y de
sign
500
200
200
Det
aile
d de
sign
500
150
150
Sour
ce c
ode
300
150
150
Test
pla
ns50
020
020
0Te
st c
ases
300
150
150
Rat
es a
re in
line
s pe
r hou
r. A
page
is ro
ughl
y eq
uiva
lent
to 5
0 lin
es.
The
abov
e ra
tes
wer
e fo
und
effe
ctiv
e in
sys
tem
con
trol a
pplic
atio
ns.
OPR
E 63
6438
Step
2 :
Ove
rvie
w
Purp
ose
: Tut
oria
lR
oles
: Mod
erat
or, A
utho
r, In
spec
tors
Task
: Pre
sent
atio
nIn
put
: Wor
k pr
oduc
t, vi
sual
aid
sO
verv
iew
is o
ptio
nal.
Rea
sons
for s
ched
ulin
g it
may
be:
–Th
e w
ork
prod
uct i
s cr
itica
l to
the
proj
ect
–Th
e w
ork
prod
uct i
s la
rge
or c
ompl
ex–
The
wor
k pr
oduc
t rep
rese
nts
use
of a
tech
nolo
gy th
at
is n
ew o
r inf
requ
ently
use
d–
The
wor
k pr
oduc
t is
the
resu
lt of
a ‘o
ne-p
erso
n’
proj
ect
OPR
E 63
6439
Step
3 :
Prep
arat
ion
Purp
ose
: Und
erst
andi
ng; i
dent
ifyin
g po
tent
ial d
efec
tsR
oles
: Ins
pect
ors
Task
s: S
tudy
mat
eria
l to
be in
spec
ted
Inpu
t: W
ork
Prod
uct
Insp
ectio
n ch
eckl
ist
Rec
omm
ende
d pr
epar
atio
n tim
eO
utpu
t: N
oted
def
ects
Not
ed p
repa
ratio
n tim
e
OPR
E 63
6440
Step
4 :
Insp
ectio
n M
eetin
g
Purp
ose
: Ver
ify p
rodu
ctR
oles
: Mod
erat
or, a
utho
r, re
ader
, rec
orde
r, in
spec
tors
Task
s: I
ntro
duce
insp
ectio
n m
eetin
gEs
tabl
ish
prep
arat
ion
Rea
ding
and
reco
rdin
g de
fect
sR
evie
w li
sted
def
ects
Det
erm
ine
prod
uct d
ispo
sitio
nIn
put
: Wor
k pr
oduc
tIn
spec
tor’s
pre
para
tion
time
Insp
ecto
r’s p
repa
ratio
n no
tatio
nsO
utpu
t: L
ist o
f ide
ntifi
ed d
efec
tsM
eetin
g re
port
and
defe
ct s
umm
ary
OPR
E 63
6441
Def
ect c
lass
ifica
tion
•D
efec
ts a
re c
lass
ified
on
the
basi
s of
def
ect t
ype,
cla
ssan
d se
verit
y
•Type
is th
e ki
nd o
f def
ect c
ondi
tion.
–D
A (d
ata)
a de
fect
in in
tern
al d
ata
use
or s
peci
ficat
ion
–D
C (d
ocum
ent)
inad
equa
te, i
rrele
vant
or i
ncor
rect
des
crip
tion
–FN
(fun
ctio
nalit
y)an
inco
rrect
spe
cific
atio
n–
HF
(hum
an fa
ctor
s)a
defe
ct in
ope
ratio
nal p
roce
dure
or h
uman
inte
rface
–IF
(int
erfa
ce)
a de
fect
in th
e co
mm
unic
atio
n be
twee
n co
mpo
nent
s–
LO (l
ogic
)a
defe
ct in
pro
cedu
ral,
algo
rithm
ic, o
r con
trol l
ogic
–M
N (m
aint
aina
bilit
y) th
e co
mpo
nent
can
not b
e m
aint
aine
d ea
sily
–PF
(per
form
ance
)op
erat
iona
l effi
cien
cy m
ay g
et s
acrif
iced
–SN
(syn
tax)
a de
fect
in la
ngua
ge u
sage
–ST
(sta
ndar
ds)
a de
partu
re fr
om re
pres
enta
tiona
l sta
ndar
ds–
OT
(oth
er)
a de
fect
con
ditio
n th
at h
as n
ot b
een
spec
ified
OPR
E 63
6442
Def
ect c
lass
ifica
tion
•Class
is th
e w
ay th
e de
fect
isex
pres
sed
orad
dres
sed.
–
M (m
issi
ng)
the
pro
duct
is m
issi
ng m
ater
ial t
hat n
eeds
to b
e ad
ded
–W
(wro
ng)
th
e pr
oduc
t con
tain
s in
corre
ct o
r unc
lear
mat
eria
l–
E (e
xtra
)th
e pr
oduc
t con
tain
s ex
tra m
ater
ial
•Severity
is th
eim
pact
that
the
defe
ct is
exp
ecte
d to
hav
e on
the
wor
k pr
oduc
t or t
he d
evel
opm
ent p
roce
ss.
–J
(maj
or)a
def
ect t
hat i
s ex
pect
ed to
cau
se p
rodu
ct fa
ilure
, dep
artu
refro
m s
peci
ficat
ions
, or p
reve
nt fu
rther
cor
rect
dev
elop
men
t.–
N (m
inor
)a d
efec
t tha
t red
uces
the
effe
ctiv
enes
s, o
r con
fuse
s a
prod
uct's
repr
esen
tatio
n or
form
at, b
ut is
not
exp
ecte
d to
impa
ct th
e op
erat
ion
or fu
rther
dev
elop
men
t of t
he p
rodu
ct.
OPR
E 63
6443
Insp
ectio
n D
efec
t Lis
t
Mee
ting
Dat
e :
Proj
ect :
Rel
ease
:Ac
tivity
:D
ocum
ent :
Com
pone
nt :
Mod
erat
or :
Mee
ting
Type
:In
spec
tion
(I)R
eins
pect
ion
(R)
Mai
nten
ance
(M)
Insp
ectio
n Ty
pe :
Hig
h-le
vel d
esig
n(H
D)
Det
aile
d de
sign
(DD
)C
ode
(CD
)
Loca
tion
Def
ect D
escr
iptio
nTy
peC
lass
Seve
rity
------
-----
------
------
----
----
-----
------
------
-----
------
------
------
----
----
-----
------
------
-----
------
------
----
----
-----
------
---
OPR
E 63
6444
Insp
ection D
efec
t Sum
mar
yM
eetin
g D
ate
:Pr
ojec
t :R
elea
se :
Activ
ity :
Doc
umen
t :C
ompo
nent
:M
oder
ator
:M
eetin
g Ty
pe :
Insp
ectio
n (I)
Rei
nspe
ctio
n(R
)M
aint
enan
ce (M
)In
spec
tion
Type
:H
DD
DC
DD
ispo
sitio
n :
Acce
pt (A
)C
ondi
tiona
l (C
)R
eins
pect
(R)
Min
or D
efec
tsM
inor
Def
ects
Def
ect
M
W
E T
otal
M
W
E T
otal
DA
DC
FN
… OT
Tota
l
OPR
E 63
6445
Product
Dis
posi
tion
Acce
pt(A
) :
Acce
pt t
he w
ork
prod
uct
as c
ompl
ete,
with
out
any
furth
er
verif
icat
ion
of r
ewor
k. T
his
does
not
req
uire
the
wor
k pr
oduc
t to
be
defe
ct fr
ee. B
ut it
doe
s re
quire
that
ther
e be
no
defe
cts
that
cau
se th
e pr
oduc
t to
devi
ate
from
its
spec
ifica
tions
, and
that
ther
e ar
e on
ly v
ery
few
triv
ial d
efec
ts.
Con
ditio
nal(
C) :
Con
ditio
nally
acc
ept t
he w
ork
prod
uct,
with
ver
ifica
tion
of
the
rew
ork
by th
e m
oder
ator
. In
this
cas
e th
ere
are
som
e m
ajor
def
ects
, bu
t the
y ar
e fe
w, r
elat
ive
to th
e w
ork
prod
uct,
and
thei
r re
wor
kis
not
ex
pect
ed to
cre
ate
any
subs
tant
ial c
hang
es in
the
'des
ign
prem
ise'
of
the
wor
k pr
oduc
t.
Rei
nspe
ct(R
) :
Rei
nspe
ctth
e au
thor
's r
ewor
k. T
his
disp
ositi
on r
equi
res
that
the
rew
ork
be e
xam
ined
by
the
mod
erat
or, t
he a
utho
r and
at l
east
on
e ot
her
mem
ber
of t
he i
nspe
ctio
n te
am.
For
this
cas
e, t
here
are
ei
ther
a s
ubst
antia
l num
ber
of m
ajor
def
ects
, re
wor
k th
at w
ill ch
ange
th
e de
sign
pre
mis
e of
the
wor
k pr
oduc
t.
OPR
E 63
6446
Step
5 :
Rew
ork
Purp
ose
: Mee
t exi
t crit
eria
Rol
es: A
utho
rTa
sks
: Res
olve
all
iden
tifie
d de
fect
sIn
put
: Ins
pect
ion
defe
ct li
stW
ork
prod
uct d
ispo
sitio
nSc
hedu
le fo
r mod
erat
or re
view
Out
put
: Wor
ked
upon
wor
k pr
oduc
tD
ocum
enta
tion
of d
efec
t res
olut
ion
OPR
E 63
6447
Step
6 :
Follo
w-u
p
Purp
ose
: Cer
tify
insp
ectio
nR
oles
: Mod
erat
or, A
utho
r, (In
spec
tors
)Ta
sks
: Ver
ify a
ll re
wor
k; R
epor
t res
ults
Inpu
t: R
evis
ed w
ork
prod
uct
Doc
umen
tatio
n of
def
ect r
esol
utio
nO
utpu
t: C
ompl
eted
insp
ectio
n re
port
OPR
E 63
6448
NASA S
hutt
le s
atel
lite
link
snag
s co
mpute
r pro
ble
ms
The
NAS
A sa
tellit
e lin
k th
at n
orm
ally
tran
smits
vid
eo o
f the
sta
tion
to E
arth
ha
s be
en in
terru
pted
by
the
com
pute
r pro
blem
s. T
he s
pace
age
ncy
cons
ider
ed th
e re
stor
atio
n of
the
vide
o lin
k a
low
prio
rity
in th
e re
cove
ry o
f th
e co
mpu
ters
. Th
ose
diffi
culti
es s
urfa
ced
Tues
day
whe
n th
e st
atio
n's
thre
e cr
itica
l co
mm
and
and
cont
rol c
ompu
ters
beg
an to
fail
inex
plic
ably
. The
dev
ices
co
ntro
l the
gui
danc
e an
d st
eerin
g of
the
stat
ion,
pro
duct
ion
of e
lect
rical
po
wer
, com
mun
icat
ions
with
Mis
sion
Con
trol a
s w
ell a
s th
e op
erat
ion
of th
e ne
w ro
bot a
rm.
By S
unda
y, e
xper
ts b
elie
ved
the
prob
lem
s co
uld
have
bee
n ca
used
by
near
-si
mul
tane
ous
failu
res
in th
e ha
rd d
rives
, or f
ile s
erve
rs, i
n tw
o of
the
mac
hine
s. T
he s
erve
rs s
tore
the
softw
are
appl
icat
ions
for m
any
of th
e st
atio
n's
criti
cal f
unct
ions
. H
owev
er, a
sof
twar
e pr
oble
m h
ad n
ot b
een
rule
d ou
t. ht
tp://
ww
w.c
hron
.com
/con
tent
/inte
ract
ive/
spac
e/m
issi
ons/
sts-
100/
stor
ies/
2001
0430
.htm
l
OPR
E 63
6449
Som
e in
spec
tion
expe
rienc
esJP
L Ex
perie
nce
Aver
ages
per
insp
ectio
nM
ajor
def
ects
foun
d4
Min
or d
efec
ts fo
und
12Pa
ges
of m
ater
ial
38N
umbe
r of p
artic
ipan
ts5
Tota
l sta
ff ho
urs
28Ap
prox
imat
e sa
ving
s$2
5,00
0
Bell
Nor
ther
n R
esea
rch
Expe
rienc
e2.
5 m
illion
s LO
C o
ver 8
sof
twar
e re
leas
esD
etec
ted
37 d
efec
ts p
er 1
000
lines
of c
ode
Foun
d on
e de
fect
for e
ach
staf
f-hou
r inv
este
dD
isco
vere
d de
fect
s 2
to 4
tim
es fa
ster
than
test
ing
Foun
d 80
% o
f all
defe
cts.
OPR
E 63
6450
Softw
are
Test
ing
•Th
e pu
rpos
e of
test
ing
is to
dem
olis
h th
e so
ftwar
e th
at h
as ju
stbe
en c
ompl
eted
•Te
stin
g ca
nnot
dem
onst
rate
abs
ence
of d
efec
ts•
Test
ing
unco
vers
erro
rs o
nly
if yo
u ar
e w
illing
to d
etec
t the
m•
If a
new
ly d
evel
oped
sof
twar
e do
es n
ot h
ave
erro
rs th
en th
e so
ftwar
e w
as to
o tri
vial
to b
e de
velo
ped
•So
ftwar
e er
rors
follo
w P
aret
o Pr
inci
ple
–ab
out 8
0% o
f the
er
rors
occ
ur in
abo
ut 2
0% o
f the
cod
e•
Exha
ustiv
e te
stin
g is
impo
ssib
le•
A ty
pica
l sof
twar
e de
velo
pmen
t pro
ject
ear
mar
ks a
bout
25%
of
the
tota
l effo
rt fo
r tes
ting
OPR
E 63
6451
Test
ing
phas
es•
Unit T
est
–Tes
ts t
he code
of ea
ch u
nit d
evel
oped
by
a pro
gra
mm
er.
Usu
ally
done
by
the
pro
gra
mm
er
him
(her
)sel
f.•
Inte
gra
tion T
est
–Tes
ts t
he design
of
the
syst
em b
y te
stin
g t
he
module
le
vel in
terf
aces
. U
sual
ly d
one
by
the
per
son in
char
ge
of th
e co
rres
pondin
g s
ubsy
stem
•Val
idation T
est
–Tes
ts t
he requirements
of th
e sy
stem
–U
sual
ly d
one
by
the
ITG
, In
dep
enden
t Tes
t G
roup
•Acc
epta
nce
Tes
t–
Tes
ts t
he
entire
system
acco
rdin
g t
o p
re-s
pec
ifie
d
criter
ia.
Usu
ally
done
by
the
use
r
OPR
E 63
6452
Test
cas
e de
sign
•Bl
ack
box
test
ing
•Te
sts
the
func
tiona
l req
uire
men
ts o
f the
uni
t. Te
st c
ases
ar
e de
sign
ed k
eepi
ng in
min
d w
hat t
his
porti
on o
f the
so
ftwar
e w
as s
uppo
sed
to d
o–
Inco
rrect
or m
issi
ng fu
nctio
ns–
Inte
rface
erro
rs–
Erro
rs in
ext
erna
l dat
a ac
cess
–Pe
rform
ance
erro
r
•G
lass
box
test
ing
–Te
sts
the
cont
rol s
truct
ure
of th
e un
it•
Exec
ute
each
inde
pend
ent p
aths
•Ex
erci
se a
ll lo
gica
l dec
isio
ns•
Exec
ute
all l
oops
at t
heir
boun
darie
s•
Exer
cise
inte
rnal
dat
a st
ruct
ures
OPR
E 63
6453
Inte
grat
ion
test
ing
•A
syst
emat
ic te
chni
que
for c
onst
ruct
ing
the
prog
ram
stru
ctur
e w
hile
at t
he s
ame
time
cond
uctin
g te
sts
to u
ncov
er e
rrors
as
soci
ated
with
inte
rfaci
ng•
Top-
dow
n in
tegr
atio
n–
Mod
ules
are
inte
grat
ed b
y m
ovin
g do
wnw
ard
thro
ugh
the
cont
rol h
iera
rchy
–U
ses
stub
s to
repr
esen
t low
er le
vel m
odul
es•
Botto
m-u
p in
tegr
atio
n–
Low
leve
l mod
ules
are
inte
grat
ed fi
rst i
nto
clus
ters
and
the
clus
ters
are
inte
grat
ed b
y m
ovin
g up
the
cont
rol s
truct
ure
–U
ses
driv
ers
to re
pres
ent u
pper
leve
l mod
ules
•R
egre
ssio
n te
stin
g
OPR
E 63
6454
Stu
bs
and D
rive
rs
Stub
ASt
ub B
Stub
CSt
ub D
Dis
pla
y a
trac
e m
essa
ge
Dis
pla
y pas
sed
par
amet
er
Ret
urn
a v
alue
from
a t
able
or
exte
rnal
file
Do a
table
sea
rch f
or
the
input
and r
eturn
an
outp
ut
Driv
er A
Driv
er B
Driv
er C
Driv
er D
Invo
ke
subord
inate
Sen
d a
par
amet
erD
ispla
y a
par
amet
erA c
om
bin
atio
n
of
Drive
rs B
an
d C
S T U
B S D
R I V
E R S
OPR
E 63
6455
Crit
eria
for t
est c
ompl
etio
n•
How
do
we
know
we
have
test
ed e
noug
h?•
Use
som
e st
atis
tical
mod
el to
pre
dict
the
num
ber o
f de
fect
s af
ter t
he s
oftw
are
has
been
test
ed fo
r tun
its o
f tim
e•
Whe
n th
e nu
mbe
r of d
efec
ts fo
und
in e
ach
of la
st n
cons
ecut
ive
hour
s of
test
ing
fall
belo
w a
pre
-spe
cifie
d lim
it•
Whe
n a
pre-
spec
ified
per
cent
age
of p
lant
ed e
rrors
get
di
scov
ered
OPR
E 63
6456
Valid
atio
n Te
stin
g•
Answ
ers
the
ques
tion
‘Are
we
deve
lopi
ng th
e rig
ht p
rodu
ct?’
rath
er th
an‘A
re w
e de
velo
ping
the
prod
uct r
ight
?’•
Alph
a te
stin
g–
Con
duct
ed a
t dev
elop
er’s
site
by
a cu
stom
er. C
ondu
cted
in
a co
ntro
lled
envi
ronm
ent i
n a
natu
ral s
ettin
g w
ith th
e de
velo
per “
look
ing
over
the
shou
lder
” of t
he u
ser a
nd
reco
rdin
g er
rors
•Be
ta te
stin
g–
Con
duct
ed a
t one
or m
ore
cust
omer
’s s
ites
by th
e en
d-us
er
of th
e so
ftwar
e. It
is a
‘liv
e’ te
st o
f the
sof
twar
e in
an
envi
ronm
ent n
ot c
ontro
lled
by th
e de
velo
per
OPR
E 63
6457
Acce
ptan
ce T
estin
g•
It is
a c
ompl
ete
test
of t
he e
ntire
sys
tem
by
the
end-
user
ac
cord
ing
to p
re-d
eter
min
ed c
riter
ia
–“T
he s
oftw
are
will
run
cont
inuo
usly
for 4
8 ho
urs”
–“A
vera
ge q
uery
tim
e sh
ould
not
exc
eed
1.5
seco
nd w
orki
ng
on a
dat
abas
e of
siz
e no
t exc
eedi
ng 1
000
reco
rds”
–“P
rodu
ctio
n pl
anni
ng w
ill be
don
e us
ing
real
dat
a of
last
four
m
onth
s”•
Syst
em te
stin
g–
Rec
over
y te
stin
g–
Secu
rity
test
ing
–St
ress
test
ing
–Pe
rform
ance
test
ing
OPR
E 63
6458
When
to T
est
Man
y IT
pro
fess
iona
ls th
ink
of te
stin
g as
a s
tage
that
co
mes
nea
r the
end
of I
T pr
oduc
t dev
elop
men
tTe
stin
g sh
ould
be
done
dur
ing
alm
ost e
very
pha
seof
th
e IT
pro
duct
dev
elop
men
t life
cyc
le
OPR
E 63
6459
Tes
ting T
asks
in t
he
Soft
war
e D
evel
opm
ent
Life
Cyc
le
OPR
E 63
6460
Typ
es o
f Tes
ts
•A
unit
test
is d
one
to te
st e
ach
indi
vidu
al c
ompo
nent
(o
ften
a pr
ogra
m) t
o en
sure
it is
as
defe
ct fr
ee a
s po
ssib
le•
Inte
grat
ion
test
ing
occu
rs b
etw
een
unit
and
syst
em
test
ing
to te
st fu
nctio
nally
gro
uped
com
pone
nts
•Sy
stem
test
ing
test
s th
e en
tire
syst
em a
s on
e en
tity
•U
ser a
ccep
tanc
e te
stin
g is
an
inde
pend
ent t
est
perfo
rmed
by
the
end
user
prio
r to
acce
ptin
g th
e de
liver
ed s
yste
m
OPR
E 63
6461
Sta
tist
ical
Tools
for
test
dat
a an
alys
is
•Q
uant
itativ
e Te
chni
ques
that
sup
port
PDC
A cy
cle
•C
ontro
l (R
un) C
harts
•Pa
reto
Cha
rts•
Ishi
kaw
a D
iagr
ams
•H
isto
gram
s, S
catte
r Cha
rts, F
low
Cha
rts,
Che
cksh
eets
OPR
E 63
6462
Pare
to A
nal
ysis
•Pa
reto
ana
lysi
s in
volv
es id
entif
ying
the
vita
l few
co
ntrib
utor
s th
at a
ccou
nt fo
r the
mos
t qua
lity
prob
lem
s in
a s
yste
m•
Also
cal
led
the
80-2
0 ru
le, m
eani
ng th
at 8
0% o
f pr
oble
ms
are
ofte
n du
e to
20%
of t
he c
ause
s•
Pare
to d
iagr
ams
are
hist
ogra
ms
that
hel
p id
entif
y an
d pr
iorit
ize
prob
lem
are
as
OPR
E 63
6463
Pare
to A
nal
ysis
•Th
e Pa
reto
Prin
cipl
e (8
0/20
rule
)•
Put o
ur e
ffort
on m
ost s
igni
fican
t pro
blem
•C
olle
ct d
ata
usin
g ch
ecks
heet
s
OPR
E 63
6464
Pare
to E
xam
ple
Time Worked (Hours)
42.2
icon
gen
40.8
18.5
157
hud_
ipc
hudd
sply
N=1
23.5
Percent of Total
34%
67%
hud_
proc
hud_
boot
82%
96%
100%
OPR
E 63
6465
Sam
ple
Par
eto D
iagra
m
OPR
E 63
6466
Exer
cise
Par
t I
•Ta
ke y
our d
efec
t dat
a•
Gro
up b
y de
fect
cat
egor
y (n
ext s
lide)
•D
raw
a P
aret
o C
hart
of y
our d
efec
ts
OPR
E 63
6467
Sort
dat
a by
Err
or C
ateg
orie
s10
Doc
umen
tati
on p
robl
em20
Syn
tax
30 B
uild
40 A
ssig
nmen
t50
Int
erfa
ce60
Err
or H
andl
ing
70 D
ata
80 F
unct
ion
90 S
yste
m10
0 En
viro
nmen
t
OPR
E 63
6468
Root
Cau
se A
nal
ysis
by
Ishik
awa
Dia
gra
ms
•Al
so c
alle
d C
ause
-Effe
ct o
r fis
hbon
e D
iagr
am•
Hel
ps R
oot C
ause
Ana
lysi
s•
Don
’t tre
at s
ympt
oms!
Dig
out
the
root
cau
se--t
he
fact
oraf
fect
ing
qual
ity•
Use
s di
agra
m p
lus
brai
nsto
rmin
g•
4 M
’s: M
anpo
wer
, Mac
hine
s, M
etho
ds,M
ater
ials
•4
P’s:
Pol
icie
s, P
roce
dure
s, P
eopl
e, P
lant
OPR
E 63
6469
The
Dia
gra
m loca
tes
Cau
ses
OPR
E 63
6470
Exa
mple
of Is
hik
awa
in S
W
Icongen
use
d 3
4%
of al
l ef
fort
People
Proce
ssEquip
men
t
Poor
IDE
Inex
per
ience
d m
anag
er
Har
d t
o m
odify
Bad
des
ign
Inex
per
ience
d d
evel
oper
s
Mea
sure
men
t
Poor
chan
ge
trac
king
OPR
E 63
6471
OPR
E 63
6472
Exe
rcis
e Pa
rt I
I
•N
ow s
elec
t you
r mos
t crit
ical
def
ect c
ateg
ory
from
the
Pare
to A
naly
sis
•Fo
rmul
ate
a pr
oble
m s
ympt
om•
Dra
w Is
hika
wa
Dia
gram
•Se
lect
app
ropr
iate
cat
egor
ies
(M,E
,P)
•Br
ains
torm
cau
ses
(try
to g
o at
leas
t 2 d
eep
on a
few
)
OPR
E 63
6473
Your
Ishi
kaw
a
OPR
E 63
6474
Exe
rcis
e Pa
rt I
II
•N
ow a
naly
ze y
our I
shik
awa
diag
ram
to p
ick
the
mos
t lik
ely
caus
e•
Form
ulat
e a
shor
t pla
n (P
DC
A) to
add
ress
cor
rect
ing
the
caus
e•
Wha
t met
rics
(che
cksh
eet)
wou
ld y
ou s
elec
t to
dete
rmin
e w
heth
er th
is s
olut
ion
wor
ked?
OPR
E 63
6475
Contr
ol Char
ts
•To
impr
ove
a pr
oces
s, it
mus
t be
“in c
ontro
l”•
In c
ontro
l = re
peat
able
•H
ow d
o w
e kn
ow?
•C
olle
ct d
ata,
dra
w c
ontro
l cha
rt•
Plot
any
def
ects
that
you
can
cou
nt o
r mea
sure
(nex
t sl
ide)
•C
hart
show
s ch
ange
s ov
er ti
me
•Ea
ch o
ut-o
f-con
trol p
oint
requ
ires
inve
stig
atio
n
http
://w
ww
.sei
.cm
u.ed
u/pu
b/do
cum
ents
/92.
repo
rts/p
df/tr
22.9
2.pd
f
OPR
E 63
6476
Con
trol C
hart
(mor
e lik
e p
Cha
rt)
Rang
e of
Va
lues
Ave
rage
Upp
er C
ontr
ol L
imit
(UCL
)
Lowe
r Co
ntro
l Lim
it (L
CL)
Spec
ific
atio
n Li
mit
(Opt
iona
l)
OPR
E 63
6477
Con
trol C
hart
Valu
es
Spec
Lim
its
are
prov
ided
(nor
mal
ly a
s go
als
or s
tand
ards
)Ce
nter
line
= s
um(v
alue
s)/n
wher
e n
is s
ampl
e si
ze
UCL
= A
vg+
3*sq
rt(A
vg)
LCL
= A
vg-
3 *s
qrt(
Avg
) (if
impo
ssib
le v
alue
the
n ze
ro)
Proc
ess
in c
ontr
ol w
hen
all p
oint
s be
twee
n U
CLan
d LC
L. M
ay s
till
be p
oor
and
outs
ide
spec
, ju
st m
eans
tha
t it
is p
redi
ctab
le.
OPR
E 63
6478
Contr
ol Char
t D
ata
MO
DU
LE B
YD
EFEC
TS/K
LOC
15Te
am 6
b
10Te
am 6
40Te
am 4
25Te
am 3
OPR
E 63
6479
Sam
ple
Contr
ol Char
t
X
Defects per KLOC
050
X
X
X
Avg
=(15
+10+
40+2
5)/4
=25
UCL
=25+
3(SQ
RT(2
5))=
LCL=
25-3
(SQ
RT(2
5))=
UCL
LCL
SPEC
OPR
E 63
6480
Sam
ple
Qual
ity
Contr
ol Char
t
OPR
E 63
6481
A p C
har
t of D
efec
ts
http
://w
ww
.sei
.cm
u.ed
u/pu
b/do
cum
ents
/97.
repo
rts/p
df/9
7hb0
03.p
df
OPR
E 63
6482
http
://w
ww
.sei
.cm
u.ed
u/pu
b/do
cum
ents
/97.
repo
rts/p
df/9
7hb0
03.p
df
OPR
E 63
6483
Sta
tist
ical
Sam
plin
g a
nd S
tandar
d
Dev
iation
•St
atis
tical
sam
plin
g in
volv
es c
hoos
ing
part
of a
po
pula
tion
of in
tere
st fo
r ins
pect
ion
•Th
e si
ze o
f a s
ampl
e de
pend
s on
how
repr
esen
tativ
e yo
u w
ant t
he s
ampl
e to
be
•Sa
mpl
e si
ze fo
rmul
a:Sa
mpl
e si
ze =
.25
X (c
erta
inty
Fac
tor/a
ccep
tabl
e er
ror)2
OPR
E 63
6484
Com
monly
Use
d C
erta
inty
Fac
tors
Desir
ed C
erta
inty
Cert
aint
y Fa
ctor
95%
1.96
0
90%
1.64
5
80%
1.28
1
95%
cer
tain
ty: S
ampl
e si
ze =
0.2
5 X
(1.9
60/.0
5)2
= 38
490
% c
erta
inty
: Sam
ple
size
= 0
.25
X (1
.645
/.10)
2=
6880
% c
erta
inty
: Sam
ple
size
= 0
.25
X (1
.281
/.20)
2=
10
OPR
E 63
6485
Sta
ndar
d D
evia
tion
•St
anda
rd d
evia
tion
mea
sure
s ho
w m
uch
varia
tion
exis
ts in
a d
istri
butio
n of
dat
a•
A sm
all s
tand
ard
devi
atio
n m
eans
that
dat
a cl
uste
r cl
osel
y ar
ound
the
mid
dle
of a
dis
tribu
tion
and
ther
e is
littl
e va
riabi
lity
amon
g th
e da
ta•
A no
rmal
dis
tribu
tion
is a
bel
l-sha
ped
curv
e th
at is
sy
mm
etric
al a
bout
the
mea
n or
ave
rage
val
ue o
f a
popu
latio
n
OPR
E 63
6486
Norm
al D
istr
ibution a
nd S
tandar
d
Dev
iation
OPR
E 63
6487
Sig
ma
and N
o.
of
Def
ective
Units
Spec
ifica
tion
Ran
ge
(in +
/- Si
gmas
) Pe
rcen
t of
Popu
latio
n W
ithin
Ran
ge
Def
ectiv
e U
nits
Pe
r B
illio
n
1 68
.27
317,
300,
000
2 95
.45
45,4
00,0
00
3 99
.73
2,70
0,00
0
4 99
.993
7 63
,000
5 99
.999
943
57
6 99
.999
9998
2
No
te:
“S
ix s
igm
a”
oft
en
refe
rs t
o +
/-3
sig
ma,
mean
ing
2.7
mil
lio
n d
efe
cts
per
bil
lio
n u
nit
s p
rod
uce
d,
or
2.7
defe
cts
per
mil
lio
n.
OPR
E 63
6488
Qual
ity
Contr
ol Char
ts,
Six
Sig
ma,
and
the
Sev
en R
un R
ule
•A
cont
rol c
hart
is a
gra
phic
dis
play
of d
ata
that
illu
stra
tes
the
resu
lts o
f a p
roce
ss o
ver t
ime.
It h
elps
pre
vent
de
fect
s an
d al
low
s yo
u to
det
erm
ine
whe
ther
a p
roce
ss
is in
con
trol o
r out
of c
ontro
l•
Ope
ratin
g at
a h
ighe
r sig
ma
valu
e, li
ke 6
sig
ma,
mea
ns
the
prod
uct t
oler
ance
or c
ontro
l lim
its h
ave
less
va
riabi
lity
•Th
e se
ven
run
rule
sta
tes
that
if s
even
dat
a po
ints
in a
ro
w a
re a
ll be
low
the
mea
n, a
bove
,the
mea
n, o
r in
crea
sing
or d
ecre
asin
g, th
en th
e pr
oces
s ne
eds
to b
e ex
amin
ed fo
r non
-rand
om p
robl
ems
OPR
E 63
6489
Red
ucin
g D
efec
ts w
ith S
ix S
igm
a
OPR
E 63
6490
Sum
mar
y of Tools
Use
•Pr
oces
ses
need
to b
e co
ntin
ually
impr
oved
. To
ols
mak
e yo
ur e
ffort
effe
ctiv
e an
d ef
ficie
nt•
Don
’t fo
llow
the
Dilb
ert M
odel
•Th
e pe
ople
who
do
the
wor
k us
ually
kno
w b
est h
ow
to im
prov
e it
•C
olle
ct d
ata
to s
uppo
rt de
cisi
ons
•U
se q
uant
itativ
e m
etho
ds w
hen
poss
ible
to s
uppo
rt de
cisi
ons
•Pl
an th
e fli
ght,
Fly
the
plan
OPR
E 63
6491
Qual
ity
Man
agem
ent
of
IT P
roje
cts
•M
any
peop
le jo
ke a
bout
the
poor
qua
lity
of IT
pr
oduc
ts
•Pe
ople
see
m to
acc
ept s
yste
ms
bein
g do
wn
occa
sion
ally
or n
eedi
ng to
rebo
ot th
eir P
Cs
•Th
ere
are
man
y ex
ampl
es in
the
new
s ab
out q
ualit
y pr
oble
ms
rela
ted
to IT
(Wha
t Wen
t Wro
ng??
)•
Qua
lity
is c
ritic
al in
eve
ry IT
pro
ject
man
agem
ent
also
!
OPR
E 63
6492
What
Is
Pro
ject
Qual
ity
Man
agem
ent?
•Th
e In
tern
atio
nal O
rgan
izat
ion
for S
tand
ardi
zatio
n (IS
O)
defin
es q
ualit
y as
the
tota
lity
of c
hara
cter
istic
s of
an
entit
y th
at b
ear o
n its
abi
lity
to s
atis
fy s
tate
dor
impl
ied
need
s•
Oth
er e
xper
ts d
efin
e qu
ality
bas
ed o
n–
conf
orm
ance
to re
quire
men
ts: m
eetin
g w
ritte
n sp
ecifi
catio
ns–
fitne
ss fo
r use
: ens
urin
g a
prod
uct c
an b
e us
ed a
s it
was
inte
nded
OPR
E 63
6493
Proje
ct Q
ual
ity
Man
agem
ent
Proce
sses
•Q
ualit
y pl
anni
ng: i
dent
ifyin
g w
hich
qua
lity
stan
dard
s ar
e re
leva
nt to
the
proj
ect a
nd h
ow to
sat
isfy
them
•Q
ualit
y as
sura
nce:
eva
luat
ing
over
all p
roje
ct
perfo
rman
ce to
ens
ure
the
proj
ect w
ill sa
tisfy
the
rele
vant
qua
lity
stan
dard
s•
Qua
lity
cont
rol:
mon
itorin
g sp
ecifi
c pr
ojec
t res
ults
to
ensu
re th
at th
ey c
ompl
y w
ith th
e re
leva
nt q
ualit
y st
anda
rds
whi
le id
entif
ying
way
s to
impr
ove
over
all
qual
ity
OPR
E 63
6494
Gan
tt C
hart
for B
uild
ing
Test
ing
into
a
Syst
ems
Dev
elop
men
t Pro
ject
Pla
n
OPR
E 63
6495
Impro
ving I
nfo
rmat
ion T
echnolo
gy
Proje
ct Q
ual
ity
•Se
vera
l sug
gest
ions
for i
mpr
ovin
g qu
ality
for I
T pr
ojec
ts
incl
ude
–Le
ader
ship
that
pro
mot
es q
ualit
y–
Und
erst
andi
ng th
e co
st o
f qua
lity
–Fo
cusi
ng o
n or
gani
zatio
nal i
nflu
ence
s an
d w
orkp
lace
fa
ctor
s th
at a
ffect
qua
lity
–Fo
llow
ing
mat
urity
mod
els
to im
prov
e qu
ality
OPR
E 63
6496
Org
aniz
atio
nal
Influen
ces,
Work
pla
ce
Fact
ors
, an
d Q
ual
ity
•St
udy
by D
eMar
co a
nd L
iste
r sho
wed
that
org
aniz
atio
nal
issu
es h
ad a
muc
h gr
eate
r inf
luen
ce o
n pr
ogra
mm
er
prod
uctiv
ity th
an th
e te
chni
cal e
nviro
nmen
t or
prog
ram
min
g la
ngua
ges
•Pr
ogra
mm
er p
rodu
ctiv
ity v
arie
d by
a fa
ctor
of o
ne to
ten
acro
ss o
rgan
izat
ions
, but
onl
y by
21%
with
in th
e sa
me
orga
niza
tion
•St
udy
foun
d no
cor
rela
tion
betw
een
prod
uctiv
ity a
nd
prog
ram
min
g la
ngua
ge, y
ears
of e
xper
ienc
e, o
r sal
ary
•A
dedi
cate
d w
orks
pace
and
a qu
iet w
ork
envi
ronm
ent
wer
e ke
y fa
ctor
s to
impr
ovin
g pr
ogra
mm
er p
rodu
ctiv
ity
OPR
E 63
6497
Good a
nd B
ad S
W p
roce
sses
:
“Mat
urity
Model
s”
•M
atur
ity m
odel
s ar
e fra
mew
orks
for h
elpi
ng o
rgan
izat
ion
impr
ove
thei
r pro
cess
es a
nd s
yste
ms
–So
ftwar
e Q
ualit
y Fu
nctio
n D
eplo
ymen
t Mod
el fo
cuse
s on
de
finin
g us
er re
quire
men
ts a
nd p
lann
ing
softw
are
proj
ects
–Th
e So
ftwar
e En
gine
erin
g In
stitu
te’s
Cap
abilit
y M
atur
ity M
odel
(C
MM
) pro
vide
s a
gene
ric p
ath
to p
roce
ss im
prov
emen
t for
so
ftwar
e de
velo
pmen
t–
Seve
ral g
roup
s ar
e w
orki
ng o
n pr
ojec
t man
agem
ent m
atur
ity
mod
els
http
://w
ww
.sts
c.hi
ll.af
.mil/
cros
stal
k/19
99/m
ay/p
utm
an.a
sp
OPR
E 63
6498
SW
Pro
ject
Man
agem
ent
Mat
urity
Model
s1.
Ad-
Hoc
: The
pro
ject
man
agem
ent p
roce
ss is
des
crib
ed a
s di
sorg
aniz
ed,
and
occa
sion
ally
eve
n ch
aotic
. The
org
aniz
atio
n ha
s no
t def
ined
sys
tem
s an
d pr
oces
ses,
and
pro
ject
suc
cess
dep
ends
on
indi
vidu
al e
ffort.
Ther
e ar
e ch
roni
c co
st a
nd s
ched
ule
prob
lem
s.2.
Abb
revi
ated
: The
re a
re s
ome
proj
ect m
anag
emen
t pro
cess
es a
nd
syst
ems
in p
lace
to tr
ack
cost
, sch
edul
e, a
nd s
cope
. Pro
ject
suc
cess
is
larg
ely
unpr
edic
tabl
e an
d co
st a
nd s
ched
ule
prob
lem
s ar
e co
mm
on.
3. O
rgan
ized
: The
re a
re s
tand
ardi
zed,
doc
umen
ted
proj
ect m
anag
emen
t pr
oces
ses
and
syst
ems
that
are
inte
grat
ed in
to th
e re
st o
f the
or
gani
zatio
n. P
roje
ct s
ucce
ss is
mor
e pr
edic
tabl
e, a
nd c
ost a
nd s
ched
ule
perfo
rman
ce is
impr
oved
.4.
Man
aged
: Man
agem
ent c
olle
cts
and
uses
det
aile
d m
easu
res
of th
e ef
fect
iven
ess
of p
roje
ct m
anag
emen
t. Pr
ojec
t suc
cess
is m
ore
unifo
rm,
and
cost
and
sch
edul
e pe
rform
ance
con
form
s to
pla
n.5.
Ada
ptiv
e: F
eedb
ack
from
the
proj
ect m
anag
emen
t pro
cess
and
from
pi
lotin
g in
nova
tive
idea
s an
d te
chno
logi
es e
nabl
es c
ontin
uous
im
prov
emen
t. Pr
ojec
t suc
cess
is th
e no
rm, a
nd c
ost a
nd s
ched
ule
perfo
rman
ce is
con
tinuo
usly
impr
ovin
g.