Risk Analysis & Management 19 th March 2013 Original slides by Max Brooker Additional information by Karen Bradbury Max Brooker BSc(hons), MBA, AMIMEchE,

Risk Analysis & Management

19th March 2013Original slides by Max Brooker

Additional information by Karen BradburyMax Brooker

BSc(hons), MBA, AMIMEchE, MAPMPartner – OTB Engineering LLP

O: 020 7099 2608 M: 07799 765 [email protected]

www.otbeng.com

Agenda

• Introduction

• Qualitative & Quantitative - Risk Analysis

• Project Examples

• Application Case Study – PPP Risk Allocation

ABI BTS Joint Code of PracticeRisk Management of Tunnel Works

in the UK(source: www.britishtunnelling.org.uk)

ICE - Risk Analysis and management in Projects (RAMP)

(www.icevirtuallibrary.com/content/book/100868)

ICE Risk Analysis and Management in ProjectsRAMP

• Developed by a joint working party of professional actuaries, civil engineers and economists in 1994.

• Recommended by HM Treasury in its ‘Green Book’. • The RAMP process is based on four key activities:

– Process launch– Risk review – Risk management– Process close-down

• Available from the Warwick library via ICE virtual library

Uncertainty and Risk

1.Uncertainty: The lack of certainty, A state of having limited knowledge where it is impossible to exactly describe the existing state, a future outcome, or more than one possible outcome.

2.Measurement of Uncertainty: A set of possible states or outcomes where probabilities are assigned to each possible state or outcome – this also includes the application of a probability density function to continuous variables.

3.Risk: A state of uncertainty where some possible outcomes have an undesired effect or significant loss.

4.Measurement of Risk: A set of measured uncertainties where some possible outcomes are losses, and the magnitudes of those losses – this also includes loss functions over continuous variables

Douglas Hubbard (2010). How to Measure Anything: Finding the Value of Intangibles in Business, 2nd ed. John Wiley & Sons

Risk

• Risk differs from uncertainty because risk is a condition where a probability distribution can be applied to the occurrence of a risk event.

• Based on the work of Frank Knight, University of Chicago economist in 1921.

DefinitionsRisk Analysis

source (Wikipedia)

RiskThe ISO 31000 (2009) /ISO Guide 73:2002 definition of risk is the 'effect of uncertainty on objectives'. In this definition, uncertainties include events (which may or not happen) and uncertainties caused by ambiguity or a lack of information. It also includes both negative and positive impacts on objectives. Many definitions of risk exist in common usage, however this definition was developed by an international committee representing over 30 countries and is based on the input of several thousand subject matter experts.

Risk and Uncertainty• Known Knowns

– risk sources is identified and a probability assigned to the probability of risk, e.g. weather patterns, panel 13.1 North Sea.

• Known unknowns– Not possible to assign a probability, UK National

Audit Office found that only 11% of risk sources were external. In many projects the data is available but not analysed. Panel 13.3 Misreading Nasa data and 13.4 risk on public projects.

– Tackle this through partnership and collaborative working to share all information during project.

Risk and Uncertainty• Unknown knowns

– Someone knows the information but is keeping it private. Panel 13.8 Kariba Dam North Station

– Tackle this through partnership and collaborative working to share all information during project.

• Unknown Unknowns– Remarkably rare– Panel 13.2 on the Millenium Bridge

Risk Sources and Risk Events

Risk source Risk source response Risk event Risk event

response

What might happen

What you did to reduce or

eliminate the source

What happened

What you did about it

Terminal 5 H&S800 managers 2-day

course1200 supervisors

1-day50,000 workers inducted

Occupational H&S service

Terminal 5Predictions2 fatalities

600 serious injuries

Terminal 5Occupational H&S service

treated

Terminal 5Accidents2 fatalities

200 serious injuries

Project Risk Management ProcessBased on ISO 31000(The Institute of Risk Management – www.theirm.org)

Risk – cost overspend

Likelihood * Impact = Risk Exposure

Say 50% chance of task overspend of £100k = £50k Risk Exposure

Mitigation actions options inclue:-• Project action to reduce

likelihood or impact• Contractors all risk insurance• Subcontract package and risk• Don’t accept risk –>

compensation event

Project Example Qualitative Project Risk Analysis

Risk Severity Metrics

1 Set up Risk Register2 Identify and define Risks3 Assess Likelihood of Occurrence and Impact if it occurs4 Assign Risk Severity levels R/A/G5 Plan, Implement and Manage Risk Mitigation Actions

Impact

Low < £300k Medium £300k - £1,000k High > £1,000k

Likelihood

High >50%Mitigate Mitigate Not Acceptable

Medium 20% - 50%Mitigate Mitigate Mitigate

Low <20%Acceptable Mitigate Mitigate

Modelling

Most likely

90% level

0

10

20

30

40

50

60

70

80

90

100

10 20 30 40 50

Cost (£m)

Pro

babi

lity

(%)

Scenario 1

Scenario 2

No one is comfortable predicting 0-10% or 90%+See panel 11.1 for PERT and beta distribution

DefinitionsMonte Carlo Simulation & Risk Analysis

source (Wikipedia)

Monte Carlo SimulationMonte Carlo methods (or Monte Carlo experiments) are a class of computational algorithms that rely on repeated random sampling to compute their results. Monte Carlo methods are often used in computer simulations of physical and mathematical systems. These methods are most suited to calculation by a computer and tend to be used when it is infeasible to compute an exact result with a deterministic algorithm. This method is also used to complement theoretical derivations.

Oracle Crystal Ball

• http://www.oracle.com/us/products/applications/crystalball/crystal-ball-suite/overview/index.html

• Demo• http://

download.oracle.com/technology/products/bi/crystalball/demos/crystalball-prod-111.html

The Basic Process Quantitative Project Risk Analysis

(source: @RISK www.Palisade.com)

Step 1 Set up a risk model functions:- Likelihood * Cost Impact = Risk Exposure

Risk 1 Binomial 30% *

Risk 2 Binomial 50% *

Risk 3 Binomial 50% *

Exposure

Exposure

Exposure

Sum Cost Exposures =

Step 2 Run Monte Carlo Simulation and interpret results

What can you model?

Can model• Costs• Time delaysCan’t model competency

Project ExampleQRA Risk Modelling

Quick look at – Thames Tunnel Risk Register

Risk Analysis & ManagementCase Study

Private Public Partnership (PPP)Procurement

Risk Allocation ProcessPPP Agreement and Main Contracts

Procurement Authority

Project Co Ltd(SPV)

Lending Banks

PPP Project Agreement

Facility Agreement

ConstructorsDesign Operator Maintainer

Shareholders Shareholders Agreement

Main Contracts

Direct Agreement

PPP

• Client will employ a risk specialist because the contractor has an incentive not to record the risk and raise it as a compensation event.

• If the lenders are no happy with risk management they will charge a higher premium

• Cross rail use different tunnelling contractors to spread the risk

PPP Risk Allocation Principles

• Risks are allocated to the party best able to manage the risk.• Risk are transferred or allocated through the mechanism of the PPP

Agreement and Main Contracts.• All risks need to be owned and managed by parties to the project

contracts, except external and Force Majeure risk events.• Risk mitigation costs are included in each risk owners cost base.• The risk ownership and management responsibility needs to be clear,

unambiguous, agreed and funded.• Risk allocation and acceptance is proportional to cost.• Risk allocation and acceptance agreement is achieved through negotiation

(e.g. Competitive Dialogue & Risk Workshops)• Government guarantees, insurance and contract terms (e.g. restricting

competition) are legitimate risk mitigation mechanisms.

Abbreviations of Risk Allocation Matrix

• Cap – capped liability• Def – definition of risk, probability of

unexploded munitions, probability of bad weather days

• Ins - insurance

Typical PPP Risk Allocation MatrixRisk Description Authority Shared SPV

Requirements – Output Specification •

Demand/Usage • cap

Outline Planning Consent •

Outline Design •

Land Purchase/Provision •

Private Finance •

Detailed Design – design liability transferred •

Construction •

Availability – of the Service •

Operational Performance – set of KPIs •

Legislative Changes – specify categories •

Environmental Compliance • cap

Ex Risks - Delays & Cost IncreasesRisk Description Force

MajeureAuthority Shared SPV

Delayed start – Financial Close/Planning •

Interfaces at contract boundaries

Ground Condition – Geotechnical Baseline • ins

Archaeological Finds •

Unexploded Munitions • def

Severe Weather • def

Construction – Noise/Dust/Odours •

Changes in Requirements/Tech. Specifications •

Economic instability • cap

Material & Labour Cost Escalation • cap

Default of Contractor •

Insurance Arrangements • •

Procurement Authority’sPPP Shadow Project Risk Register

• Logical structure; award, design, site, construction, operations, maintenance, hand-back, transversal.

• Comprehensive and realistic mitigation strategies.• Analytical quantitative rigour.• What-if scenarios developed and analysed.• Clarity of risk ownership.• Cross referenced to PPP Agreement Terms and Project Plans.• Risk exposure estimated through project lifecycle.

Risk Identification Risk Analysis Risk Mitigation StrategyRef.ID No. Risk Description Cause Risk Impact/Effect Likelihood Consequence Risk Rating Basis of Evaluation Current Control Measures & Effectiveness Mitigation Action Owner

Risk Mitigation

Strategy

Mitigation

                       1

Contractor has inadequate experience, resources and equipment to achieve the required performance standards and response times.

1. Contractor has no local experience in road maintenance nor any experience in undertaking performance based maintenance contracts. 2. Contractor has not demonstrated that it has resources or equipment which will enable it to achieve the response times.

1. Expected performance and service outcomes not achieved (likely). 2. Contractor disputes over penalties applied for non performance (likely). 3. Significant effort and concessions by DOT to assist the Contractor (likely) 4. DoT required to reduce requirements to avoid major contractual issues (possible) 5. Contractor stops work and defaults, requiring termination and re-tender (worst case) > all leading to negative outcome for DOT and negative perception of the performance based approach.

C 2 I

It is likely that the Contractor will sub-contract the works to a more experienced maintenance contractor - however, no details of any nominated maintenance sub-contractor were included in submission

1. Financial penalties in contract for poor performance, and provisions for termination of contractor after three months poor performance. 2. Consultant to assist Contractor during early stage of contract to reach required standards and performance

DOT PM  

1. Request full details of the proposed maintenance sub-

contractors and the proposed contractual terms of

engagement / agreement with the sub-contractor/s. 2. Ensure

adequate reference checks have been undertaken in relation to

most recent three projects undertaken by each company (requesting references from

Bidder if not already provided).

4

No clear plan for the initial condition assessment

Lack of adequate planning and resources

1. Delays to the completion of the first year construction works claims 2. Contractor claims > all leading to negative outcome for DOT and negative perception of the performance based approach.

C 3 II

1. Contract penalties in place for late completion. 2. Requirement for submission of detailed plan during the mobilsation period

DOT PM  

Request that Bidder submits detailed method statement /

resources plan for the construction works.

5Financial insolvency of the contractor

Inadequate cash flow to meet the mobilisation and upfront costs

1. Delays (possible) 2. Termination and re-tender (worst case) E 3 III

 1. Bidder has signifcant site establishment allowance in Bid which will assist it in

meeting upfront costs DOT PM  

Request that Bidder Joint Venture companies submit company financial statements for past

three years

Contract Bidding NegotiationsCompetitive Dialogue Stages

Authority / Bidder Competitive Dialogue Stages

CD start

Project Co

Risk

Proposal

Version: a

CD end

Project Co

Risk

Proposal

Version: z

Structured Risk Workshops

• Initial Clarification / Understanding / Planning• First Focus on Design&Construction Stage Top 10 Risks• Second Focus on Other Design&Construction Risks• Third Focus on Operation&Maintenance Stage Risks• Fourth Focus on Financial & Legal Risks• Lastly Consolidate

Quantitative Analysis of RisksTo Support Cost Analysis and Debt Justification

Nº Nom de la tâche1 T0

2 System

3 System and Load anaylsis

4 Make or buy

5 Interfaces

6 Plan for Hardware Aspects for Certif

7 Reliability and maintenability

8 Sub-system performance

9 TRU 230VAC/28VDC

10 Definition phase

11 Prototype manufacturing

12 Delivery TRU Prototypes

13 Equipment performances

14 Environement validation

15 TRU Regulated

16 Definition phase

17 Trade off study

18 Trade off results

19 Delivery prototypes R TRU

20 Equipment perfomrances

21 Environment validation

22 Battery

23 Trade off study

24 Trade off results

25 Delivery prototypes batteries

26 Equipment performances

27 Envrionment validation

28 Auto Trasfo 230VAC/ 115VAC 75 KVA

29 Definition phase

30 Brass Board manufacturing

T0

T0+6mth

T0+6 mth

T0+18mth

T0+7mth

T0+18mth

06 07 08 09 10 11 12 01 02 03 04 05 06 07 08 09 10 11 12 01 02 03 04 05 062003 2004

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0.86

1.72

2.59

3.45

4.31

5.18

6.04

6.91

7.77

8.64

9.50

10.3

611

.2312

.0912

.9613

.8214

.6915

.5516

.4117

.2818

.14

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0.86

1.72

2.59

3.45

4.31

5.18

6.04

6.91

7.77

8.64

9.50

10.3

611

.2312

.0912

.9613

.8214

.6915

.5516

.4117

.2818

.14

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0.86

1.72

2.59

3.45

4.31

5.18

6.04

6.91

7.77

8.64

9.50

10.3

611

.2312

.0912

.9613

.8214

.6915

.5516

.4117

.2818

.14

model risk on planning, budget

Project Risk Register

Monte C

arlo simulation

probability curves

tornado / sensitivitydiagrams

Project Team

criticality diagram

Tornado and Critical analysis

• http://en.wikipedia.org/wiki/Tornado_diagram

• http://uk.ask.com/wiki/Failure_mode,_effects,_and_criticality_analysis

Procurement Authority’sPPP Competitive Adjudication - Risk

Most likely

90% level

0

10

20

30

40

50

60

70

80

90

100

10 20 30 40 50

Cost (£m)

Pro

babi

lity

(%)

Scenario 1

Scenario 2

C: Compare Scenario Risk Exposures

B: Compare Bidder Proposals& Cost Risk Exposure

Informed Evaluation & Decisions through Competitive Dialogue stage outputs

A: Compare Bidder Risk Management Competency

Examples of PPP Bidder Risk Mitigation Strategies

• Material cost escalation – set limits to indexed increases.• Delayed start – define cost escalation factors.• Environmental – Clarify Authority’s EIA parameters• Design liability – lock to Authority’s Output Specification

parameters.• Site Ground Conditions – Authority’s Geotechnical Baseline

Report, ground surveys.• Construction – use competent experts, team approach.• Project Assumptions – clarify boundaries of liability, clarity

external dependencies e.g. Planning Approvals.• Demand – set minimum thresholds in Payment Mechanism.

Summary & Conclusion

• Risk Management ISO31000, BTS Code of Practice, ICE RAMP

• Project Risk Management & Risk Registers• Qualitative & Quantitative Analysis• Risk Modelling by Monte Carlo Simulation• Application Case Study - PPP Procurement • Example of Bidder PPP Risk Mitigations

Thank you for your attention

Any Questions