Top Banner
The Many Faces of Software UnReliability Taz Daughtrey ©2011 ASQ & T. DAughtrey Presented live on Jan 12 th , 2012 http://reliabilitycalendar.org/ The_Reliability_Calendar/ Webinars_-_English/Webinars_- _English.html
45
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: ASQ RD Many Faces of Software un-Reliability

The Many Faces of Software

UnReliabilityTaz Daughtrey©2011 ASQ & T. DAughtrey

Presented live on Jan 12th, 2012

http://reliabilitycalendar.org/The_Reliability_Calendar/Webinars_-_English/Webinars_-_English.html

Page 2: ASQ RD Many Faces of Software un-Reliability

ASQ Reliability Division English Webinar SeriesOne of the monthly webinars

on topics of interest to reliability engineers.

To view recorded webinar (available to ASQ Reliability Division members only) visit asq.org/reliability

To sign up for the free and available to anyone live webinars visit reliabilitycalendar.org and select English Webinars to find links to register for upcoming events

http://reliabilitycalendar.org/The_Reliability_Calendar/Webinars_-_English/Webinars_-_English.html

Page 3: ASQ RD Many Faces of Software un-Reliability

The Many Faces ofSoftware Unreliability

Taz Daughtrey

Data and Analysis Center for Software

James Madison University

© 2011 Quanterion Solutions, Inc.

Page 4: ASQ RD Many Faces of Software un-Reliability

The Many Faces ofSoftware Unreliability

ASQ Reliability Division WebinarJanuary 12, 2012

Defining software reliability

Measuring software reliability

Achieving software reliability

The future

Page 5: ASQ RD Many Faces of Software un-Reliability

Errors => Defects => Failures

Page 6: ASQ RD Many Faces of Software un-Reliability

The Many Faces of Software Unreliability

Defining software reliability

Page 7: ASQ RD Many Faces of Software un-Reliability

Reliability: does what is expected

Page 8: ASQ RD Many Faces of Software un-Reliability

Unreliability: doesn’t do what is expected

Page 9: ASQ RD Many Faces of Software un-Reliability

unavailable

compromised

unsafeincorrect

Page 10: ASQ RD Many Faces of Software un-Reliability
Page 11: ASQ RD Many Faces of Software un-Reliability
Page 12: ASQ RD Many Faces of Software un-Reliability
Page 13: ASQ RD Many Faces of Software un-Reliability
Page 14: ASQ RD Many Faces of Software un-Reliability
Page 15: ASQ RD Many Faces of Software un-Reliability
Page 16: ASQ RD Many Faces of Software un-Reliability
Page 17: ASQ RD Many Faces of Software un-Reliability
Page 18: ASQ RD Many Faces of Software un-Reliability

The Many Faces of Software Unreliability

Measuring software reliability

Page 19: ASQ RD Many Faces of Software un-Reliability

Reliability: measured in……success/failure probability

…Mean Time To Failure

Page 20: ASQ RD Many Faces of Software un-Reliability

Mean Time To Repair

mission failure

$ lossrisk exposure

measured in …

Page 21: ASQ RD Many Faces of Software un-Reliability

Risk Exposure =

Probability of occurrence

X

Consequence of occurrence

Page 22: ASQ RD Many Faces of Software un-Reliability

Risk Avoidance

X

Consequence of occurrence

Risk Exposure =Risk Exposure =

ProbabilityProbability of of occurrenceoccurrence

Page 23: ASQ RD Many Faces of Software un-Reliability

Risk Mitigation

Risk Exposure =

Probability of occurrence

X

ConsequenceConsequence of occurrence of occurrence

Page 24: ASQ RD Many Faces of Software un-Reliability

R O I = return

investment

R O R I = risk exposure reduction

reliability investment

Page 25: ASQ RD Many Faces of Software un-Reliability

Risk Avoidance costs $3 x 103

4 x 10 4 x 10 --

4 4

$4 x 103

risk exposure reduction of $104 -$4 x 103 = $6 x 103

Page 26: ASQ RD Many Faces of Software un-Reliability

Risk Avoidance costs $3 x 103

4 x 10 4 x 10 --

4 4

$4 x 103

risk exposure reduction of $104 -$4 x 103 = $6 x 103

Return On Risk Exposure Reduction =$6 x 103 / $3 x 103 = 2.0

Page 27: ASQ RD Many Faces of Software un-Reliability

Risk Mitigation costs $5 x 103

10 10 -3 -3 ProbabilityProbability of occurrence of occurrence

$3 x 106

$3 x 103

risk exposure reduction of $104 -$3 x 103 = $7 x 103

Page 28: ASQ RD Many Faces of Software un-Reliability

Risk Mitigation costs $5 x 103

10 10 -3 -3 ProbabilityProbability of occurrence of occurrence

$3 x 106

$3 x 103

risk exposure reduction of $104 -$3 x 103 = $7 x 103

Return On Risk Exposure Reduction =$7 x 103 / $5 x 103 = 1.4

Page 29: ASQ RD Many Faces of Software un-Reliability

The Many Faces of Software Unreliability

Achieving software reliability

Page 30: ASQ RD Many Faces of Software un-Reliability
Page 31: ASQ RD Many Faces of Software un-Reliability

Software Reliability Engineering

E Establish quantitative reliability targets

Construct usage profiles of operational system

Test statistically to predict system reliability

Page 32: ASQ RD Many Faces of Software un-Reliability

Software Reliability Estimation+

Reliability Growth Modeling

Page 33: ASQ RD Many Faces of Software un-Reliability

Lifecycle Software Reliability Measurements

Page 34: ASQ RD Many Faces of Software un-Reliability

Statistical Modeling and Estimation of Reliability Functions for Software

Page 35: ASQ RD Many Faces of Software un-Reliability

Set measureable targets

Design, implement

Conduct appraisals

Release? Rework?

Page 36: ASQ RD Many Faces of Software un-Reliability

reviews tests

stakeholder agreement operational profiles

verifiable requirements fault-tolerant design

Page 37: ASQ RD Many Faces of Software un-Reliability

FAULT ENCOUNTER

FAULTSOURCE

intentional

accidental

intentionalaccidental

Page 38: ASQ RD Many Faces of Software un-Reliability

FAULT ENCOUNTER

FAULTSOURCE

intentional

accidental

intentionalaccidental

malicious

tradeoffs

Page 39: ASQ RD Many Faces of Software un-Reliability

Attack + Countermeasure Tree

Page 40: ASQ RD Many Faces of Software un-Reliability

The Many Faces of Software Unreliability

The future of software reliability

Page 41: ASQ RD Many Faces of Software un-Reliability

DACS Software Reliability Initiative

= “Roadmap to Dependability”

Page 42: ASQ RD Many Faces of Software un-Reliability

DACS projected products:

Handbook series Software Reliability and Security Testing Software Reliability and Security Specification

Software Reliability and Security Assessment and Improvement

Training offerings Software Reliability and Security Fundamentals

+ industry-specific modules

Research support

Page 43: ASQ RD Many Faces of Software un-Reliability
Page 44: ASQ RD Many Faces of Software un-Reliability

ongoing mentoring

on-the-job application

management-sponsored projectinitial class session

follow-up sessions

Page 45: ASQ RD Many Faces of Software un-Reliability

The Many Faces ofSoftware Unreliability

Taz Daughtrey

[email protected]

www.thedacs.com