We make technology work • 1 Risk Analysis Avoiding Future Impacts
Mar 27, 2015
We make technology work • 1
Risk AnalysisAvoiding Future Impacts
Futron Corporation • 1120 NASA Road 1, Suite 310 • Houston, Texas 77058Phone 281-333-0190 • Fax 281-333-0192 • www.futron.com
We make technology work
Risk Analysis Techniques Risk Analysis Techniques Schedule, Cost, and Other AspectsSchedule, Cost, and Other Aspects
INCOSE
Heartland Chapter
October 24, 2001
We make technology work • 3
What Is Risk and Risk What Is Risk and Risk Management? Management?
• Risk:» Definition: Risk is an event that threatens the accomplishment of one or
more future goals or objectives
» A Risk is described by the likelihood that an event will occur and the severity of the consequence of that event should it occur.
» Risk is about a possible future event and is often confused with issues and problems. If the likelihood is certain or near certain, then it is not a risk.
• Risk Management:» Definition: Risk management is the formal process of identifying
risks, assessing their magnitude, making decisions about how to handle, and then tracking the progress of the handling approach.
We make technology work • 4
The Basic Risk Management ProcessThe Basic Risk Management Process
Planning Identification Analysis HandlingCommuni-cation & Tracking
• Build the process• Allocate Responsibilities• Determine “What is to be Risk-Managed”
• Comparative Analysis• Assign Attributes• Assign Ownership
• Evaluate the Impact• “A” Risk• “All” Risks
• Prepare Decision-packages
• A Decision-Making Process
• Iterative with Analysis to select mitigation
• Keep everyone aware of decisions made
• Track progress • Evaluate effectiveness
We make technology work • 5
How to State a Risk – Formula for How to State a Risk – Formula for SuccessSuccess
• A risk statement is written as follows:
» “If A occurs because of B, then C will be the result.”
• “A” is symptomatic, “B” is causal, “C” is the outcome
• Common errors:
» Thinking that problems and issues are risks
» Using the Risk Management Database as a tool to bring attention to
problems and issues
We make technology work • 6
Exercise 1aExercise 1a
• Identify whether the following contains a risk or a problem:
• You have an 16 year old son who has a job at a restaurant in Baybrook Mall. He wants to drive your car to work.
• … a shady part of town. He works the night shift.
• Plus: He’s moved into an apartment with another young man who has already had his driver’s license revoked
We make technology work • 7
Exercise 1aExercise 1a
• Identify whether the following is a risk or a problem:
• You have an 16 year old son who has a job at a restaurant in Baybrook Mall. He wants to drive your car to work.
• … a shady part of town. He works the night shift.
• Plus: He’s moved into an apartment with another young man who has already had his driver’s license revoked
We make technology work • 8
Risk Handling Exercise:Risk Handling Exercise:Possible Handling ActionsPossible Handling Actions
• Accept» Take no action
» Make contingency plans
• Avoid» Change requirement
» Don’t do …
• Transfer» Give to someone else better equipped to deal with risk
» Insurance underwriters
• Mitigate» Reduce Likelihood
» Reduce Consequence
We make technology work • 9
Exercise 3aExercise 3a
• We’re back to your son…what would you do with the risk of him having an accident for which he is at fault? He is currently uninsured.
• How would you “Avoid” this risk?
• … “Transfer” : He has a good driving record, lives at home
• What if your son is in a hi-risk pool and collision insurance is $4500 per year?
• Can you think of some “Mitigation” actions?
We make technology work• 10
OutlineOutline
• What Is Risk Analysis?
• What Products Do We Need To Produce?
• How Are They Produced?
We make technology work• 11
What is it?: DefinitionWhat is it?: Definition
• Risk Assessment (about “a” risk)» The process of estimating the probability and impact for each risk
» Assigning other relevant attributes to a risk
• Risk Analysis (about “all” risks)» Combining and studying the effects of multiple risk assessments
» A "snapshot" in time of the project’s risk environment
» Often done with a Monte Carlo simulation
» Uses models to understand characteristics of the project
» Results derived include • Probability of meeting the project objective such as a completion date
• Contingency needed to achieve an acceptable level of likely project results
• Finding the risks which put the objective in greatest jeopardy
Source: PMI RiskSIG Lexicon: www.risksig.com/resource/lexicon.htm
We make technology work• 12
Assessment –Assessment –Assigning A Value To “A” Risk Assigning A Value To “A” Risk
• From previous Module, we have learned how to construct the Risk Matrix using qualitative techniques
Level-1 Very HighLevel-2 High
Level-n Very Unlikely
Likelihood
Cost Schedule Technical Mission
Level-1 Very High GT 50 daysVery
DifficultLoss of Vehicle
Level-2 High 25-50 days Moderate
Loss of primary science objective
Level-n Low 5 daysEasy as
pie
Loss of minor
function
ASN 5X5 Risk MatrixASN 5X5 Risk Matrix
(5)
(4)
(3)
(2)
(1)
(10)
(8)
(6)
(4)
(2)
(15)
(12)
(9)
(6)
(3)
(20)
(16)
(12)
(8)
(4)
(25)
(20)
(15)
(10)
(5)
High
Med.
Low
Consequence
Lik
elih
oo
d
1 2 3 4 5
e
d
c
b
a
M
L
VL
H
M
L
VH
H
M
VH= Very HighH = HighM = MediumL = LowVL = Very Low
3 X 3 Space Station & Genesis
4 X 4’s are common,e.g. JPL/MER
Does not have to be symmetrical
5 X 5 not a requirement
We make technology work• 13
Risk n
After “Assessment” what do you have?After “Assessment” what do you have?
• Ordered sets of individual risks• But, no clear understanding of the integrated risk
environment
Risk 1Item
Description
Cost Schedule Technical Mission
Level-1 Very High GT 50 daysVery
DifficultLoss of Vehicle
Level-2 High 25-50 days Moderate
Loss of primary science objective
Level-n Low 5 daysEasy as
pie
Loss of minor
function
Level-1 Very HighLevel-2 High
Level-n Very Unlikely
Likelihood
RED
Yellow
Green
Affinity Grouping
Consolidation
Classification
Evaluation
Development Risks
1 6 8 10 10 10 8 93 2
14
1616 14 14 13 15 15
9 9
12
52 2 4 5
3 3
1 1 1 1 58
20 212
0
5
10
15
20
25
30
35
40
PD
R
09
/24
/19
98
10
/20
/19
98
11/2
5/1
99
8
01
/14
/19
99
03
/09
/19
99
05
/21
/19
99
06
/03
/19
99
08
/17
/19
99
01
/31
/20
00
Time
Nu
mb
er o
f R
isks
# Low # Medium
# High # Retired
Top – 10 (or so)---------Risk n
Risk 1Item
Description
We make technology work• 14
Now what should I do with this stuff?Now what should I do with this stuff?
• Decision-making with “piles” of risks? How?
• These decisions need to be made based on the total integrated risk impact to the project
– That integrated impact is often counter-intuitive
The easy way out is to reduce all RED risks and most YELLOW risks, but can you afford it?
If, so, then stop here!
The easy way out is to reduce all RED risks and most YELLOW risks, but can you afford it?
If, so, then stop here!
We make technology work• 15
OutlineOutline
• What Is Risk Analysis?
• What Products Do We Need To Produce?
• How Are They Produced?
We make technology work• 16
Typical Decision Support Data ProductsTypical Decision Support Data Products
• Risk-based reserve allocation• Double Pareto Boxes• Analytical Hierarchy Process• Bubble charts• Stochastic critical path analyses• Risk-based schedule margin allocation• Cumulative distribution functions and their utility in decision-
making• Probability density functions and their utility in decision-making• Cost utility functions and their utility in decision-making• Stochastic trending charts• Importance Values
• Hyperlink indicates that a sample follows
We make technology work• 17
Double Pareto BoxDouble Pareto Box
• Focus management attention on drivers» Reduced list by order of magnitude: ~100 => ~10 (ISS Experience)
• Make effective decisions:» Which risks to accept» Which risks to manage
The Double - Pareto box is a very useful decision support tool for making mitigation decisions
The Double - Pareto box is a very useful decision support tool for making mitigation decisions
At-risk Days Duration
Risk #1 Risk #2 Risk #3 Risk #n
WBS Task #1 9 0 44 12
WBS Task #2 78 65 0 8
WBS Task #3 0 56 87 7
WBS Task #4 25 0 24 6
WBS Task #m 88 0 56 0
At-risk Days Duration
Risk #12 Risk #8 Risk #51 Risk #n
WBS Task #21 113 81 63 12
WBS Task #8 96 72 55 8
WBS Task #15 81 66 41 7
WBS Task #93 43 23 13 6
WBS Task #m 2 2 1 0
Whining
We make technology work• 18
Stochastic Critical PathStochastic Critical Path
• Critical Paths» Deterministic» Stochastic
• Risk-based• Sensitivity -- Problem-based
• Provide Decision Support and Watch Items for Project Manager
» Where can I relax my vigil» Where/when should I intensely focus
ATLO Launch
Other Bus Com
ponents
Star T
rack
erFS
W 1
& 2
FSW
3 &
4
SRC
Canister
FSW
5
PPIC, C&DH, PCAs
We make technology work• 19
AA
Utility of Cumulative Distribution Utility of Cumulative Distribution Functions, CDFsFunctions, CDFs
• To provide the decision maker with information to make risk-handling investments
• Gives you a much richer decision-making environment
• Decisions will be more mature
100%
80%
60%
40%
20%
0%Cost
What cost would you propose?
A
100%
80%
60%
40%
20%
0%
B
Cost
Which Option would you choose?
Margin
We make technology work• 20
Launch Readiness Probability
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Nov-
98
Dec-
98
Jan-9
9
Feb-9
9
Mar-
99
Apr-
99
May-
99
Jun-9
9
Jul-
99
Aug-9
9
Sep-9
9
Oct
-99
Nov-
99
Dec-
99
Jan-0
0
Feb-0
0
Mar-
00
Apr-
00
May-
00
Date
Pro
bab
ilit
y
8% Chance You will be ready!!!
Probability Density FunctionsProbability Density Functions
You were told to be ready to launch to the ISS on: 2/99
However, in your heart, you know that the ISS will not be ready for you!!
What do you do?
How do you plan?
Probability GBF is Ready When i Station is Ready
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
No
v-9
8
Jan
-99
Ma
r-9
9
Ma
y-9
9
Jul-
99
Se
p-9
9
No
v-9
9
Jan
-00
Ma
r-0
0
Ma
y-0
0
Date
Pro
babilit
y D
ensi
ty
GBF
ISSA
76% chance current plan will deliver facility when space station is ready
We make technology work• 21
Section RecapSection Recap
• There exist tools and techniques that produce good quality information that relate risk and probable future outcomes that threaten cost, schedule and mission success
• These tools and techniques greatly enhances the decision-maker’s quality of decisions
So, How is this done??Reminder: Tools Handout
We make technology work• 22
OutlineOutline
• What Is Risk Analysis?
• What Products Do We Need To Produce?
• How Are They Produced?
We make technology work• 23
How Is It Done?How Is It Done?
• Step 1. Decide “What is to be risk-managed”
• Step 2. Build a model
• Step 3. Select risks for simulation
• Step 4. Evaluate risks and input
• Step 5. Analyze, perform the trade studies
• Step 6. Create the decision-support reports for your project manager
What?
1Model
2Select Risks
3Evaluate
4Analyze
5Create
Reports
6
We make technology work• 24
There are only three classes of models needed
• Integrated Cost, Schedule, Technical» (Class participative Simulation)
What? Model Select Risks Evaluate AnalyzeCreate
Reports
1 2 3 4 5 6
WBS
Task 100
Task 101
Task 102
Task 103
Task 104
Task 105
Task 106
Event Trees Using
Failure Rates From
Analogies• Mission Models » Performance» Mission success» (Overview and set-up only)
1 2 3
5 6
4Defects
Defects
Re-work
Re-work
• Process Models» Operations and support» (Example not presented)
We make technology work• 25
CST Model – Project Network*
CST ModelCST Model
WBS
Task 100
Task 101
Task 102
Task 103
Task 104
Task 105
Task 106
Days, Facilities, Parts, People*
*This is KEY! This is the part of the technique that integrates the cost and schedule risk impacts
What? Model Select Risks Evaluate AnalyzeCreate
Reports
1 2 3 4 5 6
Probability Density
Function
Research ProjectFind AnalogiesExecute QuestionnaireAnalyze Databases---------------What can go wrong?How likely is it?What is the cause?What would be the consequence?
Monte Carlo Sim. Tool
1.0
.8
.6
.4
.2
0Days, Facilities, Parts, People
Cumulative Distribution Function
We make technology work• 26
Data Sources-1Data Sources-1• Data Required
» CST: Days at-risk, dollars at-risk» Mission: Probability of failure, failure rate
• Data Type» Probability distributions» 3-point distributions are fine: Pessimistic, Most-likely, and optimistic
• Expert opinion
• Analogies and databases of prior experiences
What? Model Select Risks Evaluate AnalyzeCreate
Reports
1 2 3 4 5 6
Probability Density
Function
Probability Density
FunctionRisk Correlation
Factors
Project Risk Information
Example: Q-Model
We make technology work• 27
Data Sources-2Data Sources-2
• Projects are at-risk of significant overrun if they proceed before they are ready
INVESTMENT IN DEFINITION
PHASES VS GROWTH
-
0
50
100
150
200
0 5 10 15 20
Definition Ratio, % Total Project
Cost Invested In Definition Phases
Proj
ect C
ost G
row
th, %
(Source: NASA Comptroller’s Office)
What? Model Select Risks Evaluate AnalyzeCreate
Reports
1 2 3 4 5 6
We make technology work• 28
Data Sources-3Data Sources-3
• System Management Process Maturity
• Deviations from norm introduce risk
What? Model Select Risks Evaluate AnalyzeCreate
Reports
1 2 3 4 5 6
OptimizingOptimizing
InitialInitialPerformedPerformed
ManagedManagedDefinedDefined
MeasuredMeasured
Reference: EIA/IS 731, System Engineering Capability Maturity Model
Level5 4 3210
0 0.5 1.0 1.5 2 2.5 3 3.5 4.0 4.5 5.0
2.0
1.5
1.0
.5Overall PM Maturity
CICost Index, CI, as a Function of Capability Maturity
Source: Ibbs and Kwak, The Benefits of Project Management, PMI
Best Fit
-10%
+10%
Futron Corporation • 1120 NASA Road 1, Suite 310 • Houston, Texas 77058Phone 281-333-0190 • Fax 281-333-0192 • www.futron.com
We make technology work
““Hands-on” Class ExerciseHands-on” Class Exercise
“Managing Cost Schedule and Technical Risk in a Project”
We make technology work• 30
Format of Class ExerciseFormat of Class Exercise
• A simplified space project will be presented
• An @Risk/MS Project simulation of the project will be presented
• The model will be run to assess the impact of the risks
• The students will be presented with the risks as identified by the simulated project, question? Is the risk list complete?
• The students must select a mitigation plan from several options presented
• The case will be rerun with the selected mitigation plan
• A random number generator will be used to obtain final outcome
• The Genesis Model will be shown using a more complex simulation tool in order to demonstrate other capabilities
We make technology work• 31
Risk ListRisk List
Risk ID Title WBS Area Impacted Likeli- hood
Conse- quence
1 CF is a new development and may not meet specified power density, causing a redesign to reduce mass in other areas.
Power & Structure Hi Hi
2 CF Design cooling requirements may be greater than expected causing redesign of the thermal system
Structure Med Hi
3 RFI from CF system may exceed TeleCom requirements causing a switch to an alternate system
TeleCom Lo Med
We make technology work• 32
Risk Manager’s ReportRisk Manager’s Report
Distribution for Project/Cost
0
0.2
0.4
0.6
0.8
1
1.2
175
179
183
187
191
195
199
203
207
211
Values in Millions
Pro
b o
f V
alu
e <
= X
-axi
s V
alu
e
Distribution for Complete/Finish
0
0.2
0.4
0.6
0.8
1
1.2
2/2/
06
3/9/
06
4/13
/06
5/18
/06
6/22
/06
7/27
/06
8/31
/06
10/5
/06
11/9
/06
12/1
4/06
Date
Pro
b o
f V
alu
e <
= X
-axi
s V
alu
e
Launch Window
Cost Cap
We make technology work• 33
Handling Options – Pick Your Poison!Handling Options – Pick Your Poison!
• Accept – Build contingency plan
• Avoid – Switch to PV
• Transfer – Sorry, not allowed by your organization!
• Mitigate Consequence
» Reduce mass
» Add thermal margin
» Buy better Telecom eqpt
• Mitigate Likelihood – Insert advanced tech development project up
front
• Other??? Skip to Closing
We make technology work• 34
AcceptAccept
• It’s a trap!!!
• All you have done is add cost and more risk
• The likelihood and consequences of the risks are unchanged
• You have done nothing to help your case
• Unfortunately this is a very common approach and very ineffective
We make technology work• 35
Avoid ResultsAvoid Results
Distribution for Project/Cost
0
0.2
0.4
0.6
0.8
1
1.2
165
170
175
180
185
190
195
200
205
210
Values in Millions
Pro
b o
f V
alu
e <
= X
-axi
s V
alu
e
Distribution for Complete/Finish
0
0.2
0.4
0.6
0.8
1
1.2
5/28
/05
7/17
/05
9/5/
05
10/2
5/05
12/1
4/05
2/2/
06
3/24
/06
5/13
/06
7/2/
06
8/21
/06
Date
Pro
b o
f V
alu
e <
= X
-axi
s V
alu
e
Launch WindowCost Cap
We make technology work• 36
The Winner: Avoid & Outsource SWThe Winner: Avoid & Outsource SW
Distribution for Project/Cost
0
0.2
0.4
0.6
0.8
1
1.2
160
165
169
174
178
183
187
192
196
201
Values in Millions
Pro
b o
f V
alu
e <
= X
-axi
s V
alu
e
Distribution for Complete/Finish
0
0.2
0.4
0.6
0.8
1
1.2
5/28
/05
6/27
/05
7/27
/05
8/26
/05
9/25
/05
10/2
5/05
11/2
4/05
12/2
4/05
1/23
/06
2/22
/06
Date
Pro
b o
f V
alu
e <
= X
-axi
s V
alu
e
Launch Window
Cost Cap
We make technology work• 37
Mitigate ConsequenceMitigate Consequence
Distribution for Project/Cost
0
0.2
0.4
0.6
0.8
1
1.2
155
163
170
178
185
193
200
Values in Millions
Pro
b o
f V
alu
e <
= X
-axi
s V
alu
e
Distribution for Complete/Finish
0
0.2
0.4
0.6
0.8
1
1.2
5/28
/05
8/11
/05
10/2
5/05
1/8/
06
3/24
/06
6/7/
06
8/21
/06
Date
Pro
b o
f V
alu
e <
= X
-axi
s V
alu
e
Launch Window
Cost Cap
We make technology work• 38
Mitigate LikelihoodMitigate Likelihood
Distribution for Project/Cost
0
0.2
0.4
0.6
0.8
1
1.2
180
184
188
191
195
199
203
Values in Millions
Pro
b o
f V
alu
e <
= X
-axi
s V
alu
e
Distribution for Complete/Finish
0
0.2
0.4
0.6
0.8
1
1.2
6/12
/06
7/15
/06
8/17
/06
9/19
/06
10/2
2/06
11/2
4/06
12/2
7/06
Date
Pro
b o
f V
alu
e <
= X
-axi
s V
alu
e
Launch Window
Cost Cap
We make technology work• 39
What About Good Luck??What About Good Luck??
• 2- .028
• 3 - .083
• 4 - .166
• 5 - .277
• 6 - .416
• 7 - .583
• 8 - .722
• 9 - .833
• 10 - .916
• 11 - .971
• 12 – 1.00
We make technology work• 40
A few carry-away thoughts …A few carry-away thoughts …
"History is a vast early warning system." Norman Cousins
“Those who cannot remember the past are condemned to repeat it.” George Santayana
"Insanity: doing the same thing over and over again and expecting different results.” Albert Einstein
Futron Corporation • 1120 NASA Road 1, Suite 310 • Houston, Texas 77058Phone 281-333-0190 • Fax 281-333-0192 • www.futron.com
We make technology work
ClosingClosing
We make technology work• 42
QUALITY ASSURANCE AND THE SINKING OF THE LARGEST OFFSHOREOIL PLATFORM
March 2001
We make technology work• 43
For those of you who may be involved in project cost control (at whatever level),
We make technology work• 44
please read this quote from a Petrobras executive,
We make technology work• 45
extolling the benefits of cutting quality assurance and inspection costs,
We make technology work• 46
on the project that sunk into the Atlantic Ocean off the coast of Brazil in March 2001.
We make technology work• 47
"Petrobras has established new global benchmarks for the generation of exceptional shareholder wealth
We make technology work• 48
through an aggressive and innovative programme of cost cutting on its P36 production facility.
We make technology work• 49
Conventional constraints have been successfully challenged
We make technology work• 50
and replaced with new paradigms appropriate to the globalised corporate market place.
We make technology work• 51
Through an integrated network of facilitated workshops,
We make technology work• 52
the project successfully rejected the established constricting and negative influences of prescriptive engineering,
We make technology work• 53
onerous quality requirements, and outdated concepts of inspection and client control.
We make technology work• 54 Elimination of these unnecessary straitjackets has empowered the project's suppliers and contractors to propose highly economical solutions,
We make technology work• 55
with the win-win bonus of enhanced profitability margins for themselves.
We make technology work• 56
The P36 platform shows the shape of things to come
We make technology work• 57
in unregulated global market economy of the 21st Century.“
We make technology work• 58
And now you have seen the final result of this proud achievement by Petrobras.
We make technology work• 59
QUIZ:
1. How many lives were lost to this cost saving effort and how did this impact the environment, needlessly?
2. Did the person giving this speech or anyone in upper management connected with this decision lose their job/bonus?
3. How much did Petrobras really save?
4. Does your company have a cost saving effort? If so, you’d better know how to swim.