1 Lucent Technologies Bell Labs Innovations Some Successful Approaches to Software Reliability Modeling in Industry Daniel R. Jeske and Xuemei Zhang Bell Laboratories, Lucent Technologies Holmdel, NJ 1. Introduction and Context 2. Software Reliability Growth Models 3. Architecture-Based Software Reliability Models 4. Case Studies 5. Summary ASA Conference on Quality and Productivity May 21, 2003
33
Embed
Some Successful Approaches to Software Reliability Modeling in … · 2018-12-21 · Lucent Technologies Bell Labs Innovations Some Successful Approaches to Software Reliability Modeling
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1
Lucent TechnologiesBell Labs Innovations
Some Successful Approaches to Software Reliability Modeling in Industry
Daniel R. Jeske and Xuemei Zhang Bell Laboratories, Lucent Technologies
Holmdel, NJ
1. Introduction and Context2. Software Reliability Growth Models3. Architecture-Based Software Reliability Models4. Case Studies5. Summary
ASA Conference on Quality and ProductivityMay 21, 2003
2
Lucent TechnologiesBell Labs Innovations
When are Software Reliability Models Typically Applied?
Software Reliability Growth Models......reliability is quantified and influences the release decision
Unit Testing
SoftwareDevelopment
IntegrationTesting
SystemTesting
GeneralAvailability
RequirementsFormulation
ArchitectureDesign
Architecture-Based Reliability Models.....decisions are being made as to how to design reliability into the system
FirstOffice
Application
Software Reliability Growth Models......reliability predictions are verified......parameters useful for modeling reliability of next release are estimated
3
Lucent TechnologiesBell Labs Innovations
Software Reliability Growth Models (SRGMs)
4
Lucent TechnologiesBell Labs Innovations
SRGMs: Questions to Answer
•What would the user-perceived failure rate of the software be ifif the software was to be released now?
•How much more testing is needed to achieve the reliability targets?
•How many faults exist in the code at the end of system test?
5
Lucent TechnologiesBell Labs Innovations
Goel-Okumoto Model
Cumulative Test Time (days)
CumulativeFailures
T
( ) (1 )d tM t c e−= −( ) ( )
dT
T M T
cde
λ−
′==
Failure Rate Estimate
x
x
xx
A Well-Known SRGM
= initial number of faults = average per-fault failure rate
= cumulative test time, aggregated across all installations
= current value of
cdt
T t
6
Lucent TechnologiesBell Labs Innovations
The expected number of faults detected in a time interval is proportional to the number of faults remaining in the software at time t.
Thus,
or equivalently,
implying,
Goel and Okumoto further assume that the number of faults observed in (0 , t) is Poisson{ M(t) }.
),( ttt ∆+
( ) ( ) [ ( )]M t t M t d c M t t+ ∆ − = − ∆
( ) (1 )dtM t c e −= −
( ) ( )M t dM t cd′ + =
Motivation for Goel-Okumoto Model
7
Lucent TechnologiesBell Labs Innovations
( )1 d t
dd teβ −=
+
Cumulative Test Time (days)
CumulativeFailures
T
x
x x
x
1( ) (1 )(1 )
dtdtM t c e
eβ−
−= × −+
S-shaped Model
8
Lucent TechnologiesBell Labs Innovations
Challenges With Applications
Traditional SRGMs are applied to system test data with the hope of obtaining an estimate of software failure rate that will be observed in the field. The following issues have to be taken into consideration:
• The usage profile in the field is typically very different than the testing profile
• Fault removal in field environments is not instantaneous
• Quality of software reliability data
9
Lucent TechnologiesBell Labs Innovations
G-O Model
( ) [1 ]
( )
dt
dt
M t c e
t cdeλ
−
−
= −=
where: c initial number of faults at time of field deploymentd average per fault failure rate in a field environmentp probability that a detected fault is successfully removed
( ) [1 ]
( )
dpt
dpt
cM t ep
t cdeλ
−
−
= −
=
G-O Model with Imperfect Debugging
Non-Instantaneous Fault Removal TimesIn Field Environments
10
Lucent TechnologiesBell Labs Innovations
1 1 n dp
µ= +
•Perfect debugging assumption is an acceptable assumption (based on thorough regression tests)
•We relate non-instantaneous fault removal to imperfect debugging
Under the imperfect debugging model:
E(# of occurrences of each fault)= 1/p
Under the situation of non-instantaneous fault removals, if µ denotes the mean time to remove a fault and there are n systemsin the field
E(# of occurrences of each fault)= 1+ nµd
Non-Instantaneous Fault Removal TimesIn Field Environments
11
Lucent TechnologiesBell Labs Innovations
Non-Instantaneous Fault Removal TimesIn Field Environments
1
1
( ) (1 )[1 ]
( )
d tn d
d tn d
M t c n d e
t cde
µ
µ
µ
λ
− ×+
− ×+
= + −
=
where: c initial number of faults at time of field deployment d average per fault failure rate in the field environmentµ average time to remove a fault (µ = 0 gives back the G-O model)n number of systems in the fieldt cumulative exposure time, aggregated across all installations
12
Lucent TechnologiesBell Labs Innovations
Field Failure Rate Prediction if Testing Profile Matches the Field Usage Profile
The number of initial faults at time of field deployment time is the same as the number of residual faults after testing has completed
The average per fault failure rate in the field environment is identical to the average per fault failure rate in the test environment
For the field failure rate model, we can use the G-O model replacing with and adjusting for non-instantaneous removal times
1( ) ( )d t
dT n dt ce de µλ− ×
− +=
dTce−c
13
Lucent TechnologiesBell Labs Innovations
Field Failure Rate Prediction When Testing Profile Does Not Match the Field Usage Profile
1) Testing and field usage profiles drive the average per fault failure rate. Usually thefailure rate of faults is smaller in field environments than in test environments
2) Define , where is the average per fault failure rate in the field environment
3) K is the “per fault failure rate” calibration factor
4) Estimate K from previous releases of the software, or from related projects
*K /d d=
/1 /( ) ( ) ( / )
d Kt
dT n d Kadj t ce d K e µλ
− ×− +=
*d
14
Lucent TechnologiesBell Labs Innovations
BCF - Base Station Controller FrameBTS - Base Transceiver Station MSC - Mobile Switching CenterPSTN - Public Switched Telephone Network
BCF
MSC
PSTN
BTSs
BCF Software•setup calls•tear down calls•negotiate cell handoffs•dynamically control transmit
power levels•alarm handling•overload control•equipment provisioning
Case Study: GSM Wireless System
15
Lucent TechnologiesBell Labs InnovationsObjective
Estimate the field failure rate of R3 BCF software which is currently
in system test
1) R3 Test Data in a non-operational profile environment
2) Field failure data for R1 and R2
Goals
Data Available
16
Lucent TechnologiesBell Labs Innovations
Field Failure Data for R1
System Days FailuresMonth Days Cumulative Month Cumulative