Copyright 2010 Muschara Error Management Consulting, LLC Human Performance On the Horizon: HROs, Resilience Engineering, and Black Swans Tony Muschara,

Copyright Copyright 2010 Muschara Error Management Consulting, LLC 2010 Muschara Error Management Consulting, LLC

Human Performance On the Horizon:Human Performance On the Horizon: HROs, Resilience Engineering, and Black Swans HROs, Resilience Engineering, and Black Swans

Tony Muschara, CPTTony Muschara, CPTJanuary 25, 2010January 25, 2010

The Certified Performance Technologist (CPT) designation is awarded by the The Certified Performance Technologist (CPT) designation is awarded by the International Society for Performance Improvement (ISPI) to experienced International Society for Performance Improvement (ISPI) to experienced practitioners in the field of organizational performance improvement whose work practitioners in the field of organizational performance improvement whose work meets both the performance-based Standards of Performance Technology and meets both the performance-based Standards of Performance Technology and application requirements. For more information, visit www.certifiedpt.orgapplication requirements. For more information, visit www.certifiedpt.org

16th Annual HPRCT ConferenceJune 21-25 · Sheraton Baltimore Inner Harbor

co-sponsored by Constellation Energy and INPO


BP Deepwater HorizonBP Deepwater Horizon

2


3


4


Key PointsKey Points

1.1. Black Swans are rare, unpredictable, very severe Black Swans are rare, unpredictable, very severe events.events.

2.2. High Reliability Organizations (HRO) enhance safety High Reliability Organizations (HRO) enhance safety through optimizing reliability.through optimizing reliability.

3.3. Resilience Engineering (RE) improves safety by Resilience Engineering (RE) improves safety by improving the occurrence of success.improving the occurrence of success.

4.4. Safety is what an organization Safety is what an organization doesdoes..

5.5. Reducing human failures is not the most efficient Reducing human failures is not the most efficient and effective way to improve human performance.and effective way to improve human performance.

6.6. Recognize system health as a primary means to Recognize system health as a primary means to improving the safety state of an organization.improving the safety state of an organization.

5


Black Swan*Black Swan*

RareRareoutside the boundsoutside the boundsof known likelihoodsof known likelihoods

UnpredictableUnpredictableunable to anticipateunable to anticipate

SevereSevereunprecedented impactunprecedented impact

EventEventgood or badgood or bad

6* Taleb, N. (2007), * Taleb, N. (2007), The Black SwanThe Black Swan..


Challenger (7) / Columbia (7)Challenger (7) / Columbia (7) Deepwater Horizon (11)Deepwater Horizon (11) Texas City (15)Texas City (15) Chernobyl (>56)Chernobyl (>56) United 232 (108)United 232 (108) Kansas City Hyatt (114)Kansas City Hyatt (114) Piper Alpha (167)Piper Alpha (167) Herald of Free Enterprise (186)Herald of Free Enterprise (186) Tenerife (583)Tenerife (583) Bhopal (>2200)Bhopal (>2200)

Notable Black Swan Disasters*Notable Black Swan Disasters*

* Numbers in parentheses denote number of fatalities on each occasion.* Numbers in parentheses denote number of fatalities on each occasion. 7


High Reliability Organization (HRO)High Reliability Organization (HRO)

HROs are “Organizations [that] operate HROs are “Organizations [that] operate under trying conditions yet perform relatively under trying conditions yet perform relatively event-free over a long period of timeevent-free over a long period of time, , making consistently good decisions that making consistently good decisions that result in high quality and reliable result in high quality and reliable operations.”operations.”

-- Karlene Roberts-- Karlene Roberts

U.C. BerkeleyU.C. Berkeley

Roberts, K. (2003), “HRO has Prominent History,” Anesthesia Patient Safety Foundation Newsletter, Roberts, K. (2003), “HRO has Prominent History,” Anesthesia Patient Safety Foundation Newsletter, Vol. 18, No. 1, pp.1-16.Vol. 18, No. 1, pp.1-16. 8


Resilience Engineering (RE)*Resilience Engineering (RE)*

ResilienceResilience – the ability to – the ability to adjustadjust its its

functioning to sustain operations functioning to sustain operations

during expected conditions and in during expected conditions and in

the face of escalating demands, the face of escalating demands,

disturbances, and unforeseen disturbances, and unforeseen

circumstancescircumstances

Resilience EngineeringResilience Engineering – the – the

tools that promote resilience: tools that promote resilience:

anticipateanticipate, , monitormonitor, , respondrespond, , learnlearn

Erik HollnagelErik HollnagelEditor: Editor: Resilience Engineering Resilience Engineering Perspectives Volume 1:Perspectives Volume 1:Remaining Sensitive to theRemaining Sensitive to thePossibility of FailurePossibility of Failure

* Nemeth, C. et al. (2009), * Nemeth, C. et al. (2009), Resilience Engineering Perspectives, Volume 2: Preparation and RestorationResilience Engineering Perspectives, Volume 2: Preparation and Restoration, p.117., p.117. 9


1.1. Preoccupation with FailurePreoccupation with Failure

2.2. Reluctance to SimplifyReluctance to Simplify

3.3. Sensitivity to OperationsSensitivity to Operations

4.4. Commitment to ResilienceCommitment to Resilience

5.5. Deference to ExpertiseDeference to Expertise

Five HRO Principles*Five HRO Principles*

10* Weick K. and Sutcliffe, K. (2007), * Weick K. and Sutcliffe, K. (2007), Managing the UnexpectedManaging the Unexpected, 2, 2ndnd ed. p. ed. p.

Karl WeickKarl WeickCo-author: Co-author: ManagingManaging

the Unexpectedthe Unexpected


AAnticipatenticipate – knowing – knowing what to expectwhat to expect;;long-term threats and opportunitieslong-term threats and opportunities

MMonitor onitor – knowing – knowing what to look forwhat to look for;;near-term developments and threats near-term developments and threats (critical steps)(critical steps)

RRespondespond – knowing – knowing what to dowhat to do;;capable of addressing expected and capable of addressing expected and unexpected conditionsunexpected conditions

LLearnearn – knowing – knowing what has happenedwhat has happened (experience) and (experience) and what to changewhat to change (improvement)(improvement)

RE Cornerstones*RE Cornerstones*

* Source: Hollnagel, et al., * Source: Hollnagel, et al., Resilience EngineeringResilience Engineering, (2006), p.350, and , (2006), p.350, and Resilience Resilience Engineering Perspectives, Vol. 2 Engineering Perspectives, Vol. 2, (2009), pp.117-133., (2009), pp.117-133. 11


1.1. Limitations of Limitations of Six Sigma Six Sigma and and LeanLean manufacturing processes to manufacturing processes to adequately address inherent human variability—adequately address inherent human variability—randomrandom and often and often unpredictableunpredictable

2.2. Difficulty in identifying and controlling the effects of Difficulty in identifying and controlling the effects of social forcessocial forces on on information, its interpretation, knowledge, and actioninformation, its interpretation, knowledge, and action

3.3. The inability to specify all threats, combinations, and responses, and The inability to specify all threats, combinations, and responses, and the occurrence of the occurrence of human failureshuman failures in all phases of the plant’s life cycle in all phases of the plant’s life cycle

4.4. Multiple, concurrent human activities; Multiple, concurrent human activities; dynamicdynamic, ambiguous , ambiguous environment; confounding number of system interdependenciesenvironment; confounding number of system interdependencies

5.5. Incomplete or inaccurate knowledge about the Incomplete or inaccurate knowledge about the state of the systemstate of the system and the uncertain nature of resources, information, and time and the uncertain nature of resources, information, and time

6.6. Impossible to chase and eliminate all Impossible to chase and eliminate all latent weaknesses latent weaknesses in a system.in a system.

7.7. EntropyEntropy – things wear out – things wear out

SerendipitySerendipity

12


Intangibles – Sources of Intangibles – Sources of UncertaintyUncertainty

TrustTrust CompassionCompassionCaringCaring DedicationDedicationFortitudeFortitude CuriosityCuriosityHonestyHonesty WisdomWisdomInsightInsight CourageCourageConsensusConsensus EmpathyEmpathyPridePride InterpretationInterpretationThoughtsThoughts MotivesMotivesPerceptionsPerceptions FeelingsFeelings

CollaborationCollaboration PassionPassionUnderstandingUnderstanding CreativityCreativityCraftsmanshipCraftsmanship ValuesValuesForgivenessForgiveness UnityUnityPerseverancePerseverance InitiativeInitiativeFollow-throughFollow-through IntegrityIntegrityMeaningMeaning EffortEffortAssumptionsAssumptions BeliefsBeliefsInterpretationsInterpretations FearsFears

13


AccumulationAccumulationInconspicuous and seemingly harmless buildupof unusual conditions, hazards, threats, and system weaknesses without warning

Drift and Accumulation*Drift and Accumulation*

14

Saf

ety

Saf

ety

Dan

ger

Dan

ger

Hi

Lo Time

DriftDrift

ExpectationsExpectations

CurrentCurrentPracticePractice

Real Real Margin for ErrorMargin for Error

* Adapted from Dekker, S. (2007), * Adapted from Dekker, S. (2007), The Field Guide to Understanding Human ErrorThe Field Guide to Understanding Human Error..

Hidden hazards, threats, unusualHidden hazards, threats, unusualconditions, & system weaknessesconditions, & system weaknesses

Expectations: Desired approach to work (as imagined)

Practices: Work as actually performed (allowed by mgmt!)

ErrorError

ViolationViolation


Be Aggressive!Be Aggressive!

The causes ofThe causes oftomorrow’s events tomorrow’s events exist exist todaytoday!!

15

Latent SystemLatent SystemWeaknessesWeaknessesAccumulateAccumulate!


Safety Space – Where am I?*Safety Space – Where am I?*

16

Boundary: Economic FailureBoundary: Economic Failure

Boundary: Unacceptable WorkloadBoundary: Unacceptable Workload

AdministrativeAdministrativeSafety LimitSafety Limit

Boundary:Boundary: System Failure System Failure

Operating Point

* Rasmussen, J. (1997), “Risk Management in a Dynamic Society: A Modeling Problem,“* Rasmussen, J. (1997), “Risk Management in a Dynamic Society: A Modeling Problem,“ Safety ScienceSafety Science, 27(2/3), 183-213., 27(2/3), 183-213.


Commercial nuclear powerCommercial nuclear power Air traffic controlAir traffic control Wildland fire fightingWildland fire fighting Aircraft carrier operationsAircraft carrier operations HealthcareHealthcare Armored military brigadeArmored military brigade Space shuttleSpace shuttle Offshore platformsOffshore platforms Rail and train operationsRail and train operations ManufacturingManufacturing Submarine operationsSubmarine operations

Studies on HROs and REStudies on HROs and RE

17


Principal Features of an HRO*Principal Features of an HRO* High-Hazard and High-Hazard and

Complex ActivitiesComplex Activities High Technical High Technical

CompetenceCompetence Hazard-Driven Hazard-Driven

AdaptationsAdaptations Positive Design-Based Positive Design-Based

RedundancyRedundancy High PerformanceHigh Performance

and Close Oversightand Close Oversight Constant SearchConstant Search

for Improvementfor Improvement

High Pressures and High Pressures and Expectations for ReliabilityExpectations for Reliability

No SubstitutesNo Substitutesfor Reliabilityfor Reliability

Limitations on Trial-and-Limitations on Trial-and-Error LearningError Learning

Culture of ReliabilityCulture of Reliability Flexible AuthorityFlexible Authority

during Emergenciesduring Emergencies Mindfulness Mindfulness

18* Roe, E. and Schulman, P. (2008), * Roe, E. and Schulman, P. (2008), High Reliability Management: Operating on the Edge,High Reliability Management: Operating on the Edge, p.228-233. p.228-233.


Principal Features of REPrincipal Features of RE Adaptability, Flexibility, and Adaptability, Flexibility, and

CapacityCapacity Dynamic Nature of System Dynamic Nature of System

and Multipleand MultipleInterdependencies among Interdependencies among ComponentsComponents

Dependency on Dependency on Functioning of theFunctioning of theWhole SystemWhole System

Preparation and Preparation and SustainabilitySustainability

Learning from SuccessesLearning from Successes

Sustaining Normal Sustaining Normal Functioning, not just Functioning, not just Preventing FailuresPreventing Failures

Resourcefulness and Resourcefulness and RedundancyRedundancy

Safety more than Reliability Safety more than Reliability of Individual Componentsof Individual Components

Safety – What an Safety – What an Organization DoesOrganization Does

Minimizing the gap between Minimizing the gap between work-as-imagined andwork-as-imagined andwork-as-donework-as-done

19


RE emphasizes adapting – to maintain RE emphasizes adapting – to maintain successsuccess..

HRO emphasizes reliability – avoiding HRO emphasizes reliability – avoiding failurefailure..

Fundamentally, the only difference isFundamentally, the only difference is PERSPECTIVEPERSPECTIVE!!

There are advantages from both.There are advantages from both.

Are there any REAL Differences between HRO and RE?Are there any REAL Differences between HRO and RE?

20


Thanksgiving TurkeyThanksgiving Turkey

21

200200 400400 800800600600 10001000

Days

Ben

efit

s

SurpriseSurprise!!


AccumulationAccumulationInconspicuous and seemingly harmless buildupof unusual conditions, hazards, threats, and system weaknesses without warning

Drift and Accumulation*Drift and Accumulation*

22

Saf

ety

Saf

ety

Dan

ger

Dan

ger

Hi

Lo Time

DriftDrift

ExpectationsExpectations

““Normal”Normal”PracticePractice

Real Real Margin for ErrorMargin for Error

* Adapted from Dekker, S. (2007), * Adapted from Dekker, S. (2007), The Field Guide to Understanding Human ErrorThe Field Guide to Understanding Human Error..

Hidden hazards, threats, unusualHidden hazards, threats, unusualconditions, & system weaknessesconditions, & system weaknesses

Expectations: Desired approach to work (as imagined)

Normal Practices: Work as actually performed (allowed by mgmt!)


Chronic Sense of UneasinessChronic Sense of Uneasiness

An attitude of An attitude of mindfulnessmindfulness

regarding one’s regarding one’s capacity to err and capacity to err and

the presence of the presence of hidden threats; hidden threats; preoccupationpreoccupation

with failurewith failure

““When you stop being When you stop being scaredscared, you start making mistakes.”, you start making mistakes.”-- unknown-- unknown

23

--how you perceive, think, feel, and behave toward hazards--


1.1. People routinely make a choice between being People routinely make a choice between being efficientefficient (productive / less effort) and being (productive / less effort) and being thoroughthorough (safe / reliable), since it is rarely (safe / reliable), since it is rarely possible to be both at the same time.possible to be both at the same time.

2.2. If If demandsdemands for for productivityproductivity are arehigh, thoroughness is reducedhigh, thoroughness is reduceduntil productivity goals are met.until productivity goals are met.

3.3. If If demandsdemands for for safetysafety are high, are high,efficiency is reduced until theefficiency is reduced until thesafety goals are met.safety goals are met.

Efficiency/Thoroughness Trade-Off*Efficiency/Thoroughness Trade-Off*

* Hollnagel. E. (2009), * Hollnagel. E. (2009), The ETTO Principle, Efficiency-Thoroughness Trade-OffThe ETTO Principle, Efficiency-Thoroughness Trade-Off.. 24


1.1. Performance Performance conditions are always imperfectconditions are always imperfect: resources and time are : resources and time are finite, training and procedures are incomplete, and planning is finite, training and procedures are incomplete, and planning is approximate.approximate.

2.2. The reliability of individual The reliability of individual componentscomponents or subsystems cannot always or subsystems cannot always effectively predict how they effectively predict how they combinecombine to create novel pathways to to create novel pathways to failure.failure.

3.3. Safety is what you doSafety is what you do, not what you have (continuous adaptation to , not what you have (continuous adaptation to changes and disruptions—core business).changes and disruptions—core business).

4.4. People create safetyPeople create safety within inherently imperfect systems. within inherently imperfect systems.

5.5. It is difficult to do safety and production at the same time (trade-offs It is difficult to do safety and production at the same time (trade-offs between between time to thinktime to think vs. vs. time to dotime to do).).

6.6. Identifying and Identifying and eliminating latent system weaknesses eliminating latent system weaknesses continuously continuously offers the best route to improving system health.offers the best route to improving system health.

7.7. Organizations Organizations cultivate a capacity to adaptcultivate a capacity to adapt – preserve capacity, – preserve capacity, flexibility, reserves, and options and reduce volatile work situationsflexibility, reserves, and options and reduce volatile work situations. .

Key Conclusions about HRO and REKey Conclusions about HRO and RE

25


SuggestionsSuggestions

26

1.1. Make boundaries explicitMake boundaries explicit, and build a “preoccupation with failure.”, and build a “preoccupation with failure.”

2.2. Give people Give people coping skillscoping skills (adaptive capacities) at boundaries. (adaptive capacities) at boundaries.

3.3. When introducing change, 1) avoid introduction of local factors that When introducing change, 1) avoid introduction of local factors that intensify intensify volatilityvolatility for the front-line worker, (control) and 2) preserve for the front-line worker, (control) and 2) preserve local factors that support local factors that support flexibilityflexibility in off normal conditions in off normal conditions (adaptability).(adaptability).

4.4. Pay attention to: 1) avoiding Pay attention to: 1) avoiding human failures at critical stepshuman failures at critical steps, and 2) , and 2) improving system health improving system health (arresting the accumulation of latent system (arresting the accumulation of latent system weaknesses)weaknesses)

5.5. Expand Hu management perspective from a myopic operational Expand Hu management perspective from a myopic operational focus to an focus to an organizational focusorganizational focus..

6.6. Monitor how you monitorMonitor how you monitor system health and safety. system health and safety.

7.7. Learn continuouslyLearn continuously – learn from successes, not just failures. – learn from successes, not just failures.

8.8. AnticipateAnticipate – – MonitorMonitor – – RespondRespond – – LearnLearn


1.1. ReportingReporting

2.2. Observation and CoachingObservation and Coaching

3.3. Self-Assessment and TrendingSelf-Assessment and Trending

4.4. Operating Experience and BenchmarkingOperating Experience and Benchmarking

5.5. Causal AnalysesCausal Analyses

Multi-Tiered Approach – Finding LSWsMulti-Tiered Approach – Finding LSWs

““Eliminating latent [system weaknesses] is the mostEliminating latent [system weaknesses] is the most effective way to manage human error.” effective way to manage human error.”--Jop Groeneweg Author: Controlling the Controllable

27


Improving System Health (RE)Improving System Health (RE)

Internal MethodsInternal Methods

Field ObservationsField Observations Post-job ReviewsPost-job Reviews ReportingReporting Self-AssessmentsSelf-Assessments Metrics and TrendingMetrics and Trending Common Factor AnalysisCommon Factor Analysis Surveys & QuestionnairesSurveys & Questionnaires Cause AnalysisCause Analysis Effectiveness ReviewsEffectiveness Reviews

External MethodsExternal Methods

Operating ExperienceOperating Experience BenchmarkingBenchmarking Independent OversightIndependent Oversight

28


Organization

Local Factors(drivers)

HumanPerformance

““Human Performance is itsHuman Performance is its goal and its test.” goal and its test.”--Peter Drucker--Peter Drucker “ “Management: Tasks,Management: Tasks, Responsibilities, Practices” Responsibilities, Practices”

29

““We cannot change the humanWe cannot change the human condition, but we can change condition, but we can change the conditions under which the conditions under which people work.” people work.”--James Reason--James Reason “ “Managing the Risks ofManaging the Risks of Organizational Accidents” Organizational Accidents”


Effort vs. Importance*Effort vs. Importance*

Latent System WeaknessesLatent System Weaknesses

Human FailureHuman Failure

Equipment Failure

CurrentlyCurrently ActualActual

* Adapted from Kletz, T. (2001), * Adapted from Kletz, T. (2001), An Engineer’s View of Human Error An Engineer’s View of Human Error (3(3rdrd ed.); p.127.. ed.); p.127..

Effort ExpendedEffort Expended ImportanceImportance

30


Be skeptical about confirmatory data.Be skeptical about confirmatory data. Worry less about small failures, and moreWorry less about small failures, and more

about large, potentially terminal ones.about large, potentially terminal ones. Worry more about matters that lie outside beyond our Worry more about matters that lie outside beyond our

consciousness.consciousness. Worry about matters you can do something about.Worry about matters you can do something about. Be very conservative when exposed to the threat of a Be very conservative when exposed to the threat of a

negative Black Swan.negative Black Swan.

Avoiding the Black SwanAvoiding the Black Swan

31

““It is more difficult to be a loser in a game you set up yourself. It is more difficult to be a loser in a game you set up yourself. …this means that you are exposed to the improbable only if you …this means that you are exposed to the improbable only if you let it control you. You always control what you do.”let it control you. You always control what you do.”--Nassim Taleb--Nassim Taleb Author: Author: The Black SwanThe Black Swan


4724 Outlook Way4724 Outlook WayMarietta, Georgia 30066Marietta, Georgia [email protected]@muschara.comhttp://www.muschara.comhttp://www.muschara.com

Questions and CommentsQuestions and Comments

32

Copyright 2010 Muschara Error Management Consulting, LLC Human Performance On the Horizon: HROs, Resilience Engineering, and Black Swans Tony Muschara,

Documents

llc resilience engineering

unexpected slide

llc human performance

inpo slide

llc challenger

llc black swan

black swans tony muschara

llc bp deepwater horizon