Thread and error management

Threat and Error ManagementThreat and Error Management

James KlinectThe University of Texas Human Factors Research Project

IFALPA Caribbean / South America Regional Conference

November 19th, 1999

Presentation OutlinePresentation Outline

• Define error management

• External threats: An overlooked concept in error management

• Threat and error management model

• Line Operations Safety Audit (LOSA): Data source for threat

and error

• General LOSA results

• Behavioral markers

• Summary – The value of threat and error management

What is Error Management?What is Error Management?

• Error – Deviation from organizational expectations or crew intentions

• Error management - process of correcting an error before it becomes consequential to safety

ErrorError

OutcomeError

Response

Error Management

An Overlooked ConceptAn Overlooked Concept

• Adverse Weather• Terrain• Aircraft malfunctions• Poor airport conditions• Abnormal operations• Operational pressures

• ATC event or error• Maintenance event or error• Ground event or error• Dispatch event or error• Cabin event or error

• External Threats – safety threatening events that originate outside of the cockpit

• Threats increase the operational complexity for a flight

Threat and Error Management Threat and Error Management ModelModel

Error Response

Error Outcome

Error

External Threats

External Threat Management

Threat Induced Incident or

AccidentInconsequential

Data Sources for Threat and ErrorData Sources for Threat and Error

• Accident Reports• Problems

1. 1.5 accidents per million departures (not enough data)2. Retrospective data – only reflects failed system or crew

performance

• Incident Reports• Problems

1. Reports are subjective, first person accounts2. Retrospective data

• Need objective data that highlight both effective and ineffective threat and error management

Line Operations Safety Audit Line Operations Safety Audit (LOSA)(LOSA)

• Non-jeopardy observations of flight crew performance during normal operations

• Union supported• One to three months of observation• Team of airline and U.T. project observers (10-15)• Data do not identify pilots and are confidential

• Measures:• External threats and their management• Flight crew errors and their management• CRM behavioral markers• Pilot suggestions through interviews• Latent system failures

LOSA DemographicsLOSA Demographics

• Data taken from three airlines

• 184 flight crews

• 314 flight segments

External ThreatsExternal Threats

LOSA External Threat ResultsLOSA External Threat Results

• 72% of the flights had at least one external threat

• Two external threats per flight

• Most external threats on a flight = 11

• Most Frequent Threats for an airline that primarily flew into Latin America

• Adverse weather – 34% of the flights

• ATC events or errors – 34%

• Aircraft malfunctions – 15%

External Threats by Phase of FlightExternal Threats by Phase of Flight

Phase of FlightThreats by

PhaseMost Frequent

Threats by Phase

Pre-Departure / Taxi 22%

A/C malfunctionsGround events / errorOperational pressures

Takeoff / Climb 28%

Cruise 10%

Descent / Approach / Land

39%

TerrainAdverse weatherATC events and errors

Taxi / Park 1%Summary - External threats in the pre-departure phase are much different than those in the most threatening phase (descent / approach / land)

Flight Crew ErrorFlight Crew Error

Threat and Error Management Threat and Error Management ModelModel

Error Response

Error Outcome

Error

External Threats

External Threat Management

Threat Induced Incident or

AccidentInconsequential

LOSA Error ResultsLOSA Error Results

• 64% of the flights had an error

• Two errors per flight

• Most errors on a flight = 14

• Most common flight crew errors• Automation errors – 31% of all errors

• Wrong MCP and FMC modes and settings• Failure to cross-verify

• Checklist errors – 21% of all errors• Checklist performed from memory• Nonstandard usage• Missed items

Error by Phase of FlightError by Phase of Flight

Phase of FlightThreats by

PhaseError by Phase

Pre-Departure / Taxi 22% 23%

Takeoff / Climb 28% 24%

Cruise 10% 12%

Descent / Approach / Land

39% 39%

Taxi / Park 1% 2%

Summary - The most threatening phase (descent / approach / land) also contains the most error

Expanding the Model with Error Expanding the Model with Error TypesTypes

ErrorIntentional

NoncomplianceProcedural

CommunicationProficiencyDecision

Error Outcome

External Threats

External Threat Management

Threat Induced Incident or Accident

Inconsequential

Error Response

Error TypesError Types

1. Intentional Noncompliance – violationsex) Performing a checklist from memory

2. Procedural – Followed procedures but wrong executionex) Wrong altitude setting dialed into the MCP

3. Communication – Missing information or misinterpretationex) Miscommunication with ATC

4. Proficiency – Error due to a lack of knowledgeex) Lack of knowledge with automation

5. Decision – Discretionary crew decision that unnecessarily increased risk

ex) Unnecessary navigation through adverse weather

Error FrequenciesError Frequencies

6

5

6

29

54

0 20 40 60 80 100

Decision

Proficiency

Communication

Procedural

Noncompliance

Percent Frequency

Summary – Willful violations (intentional noncompliance errors) are the most frequently observed errors

Expanding the Model with Error Expanding the Model with Error ResponsesResponses

Error

External Threat Management

Threat Induced Incident or

Accident

Error Outcome

External Threats

Inconsequential

Error Response

Trap

Exacerbate

Fail to Respond

Error ResponsesError Responses

• Trap – 36% of all error responses • Error is successfully detected and managed

• Exacerbate – 11%• Error is detected but mismanaged to produce a consequential

outcome

• Fail to Respond – 53% • Error is not detected therefore not managed

Expanding the Model with OutcomesExpanding the Model with Outcomes

Recovery Additional Error

Undesired Aircraft State

Error

External Threat Management

Error Response

Threat Induced Incident or

Accident

External Threats

Inconsequential

Error OutcomesError Outcomes

• Inconsequential – 85% of all errors

• Consequential Outcomes – 15% of all errors• Additional Error – 3%

• Undesired Aircraft States – 12%

Undesired Aircraft StatesUndesired Aircraft States

• Lateral deviation• Vertical deviation• Speed too high or low• Unstable approach• Near miss• Fuel level below minimums

• Long landing• Hard landing• Landing off centerline• Wrong taxiway or ramp• Wrong runway• Wrong airport or country

• Undesired Aircraft States – Flightcrew induced deviations from normal flight that threaten safety

Error Frequencies and Error Frequencies and ConsequencesConsequences

43

69

13

6

5

6

29

54

23

2

0 20 40 60 80 100

Decision

Proficiency

Communication

Procedural

Noncompliance

Percent Consequential Percent Frequency

Summary – Proficiency and decision errors are the most difficult to manage

Intentional Noncompliance as a Intentional Noncompliance as a NormNorm

• One LOSA observer noted the following during a IOE ride• “The check airmen ran the entire taxi checklist from memory.

Bad example to set.”

• Airlines cannot allow violations to normalize

• Why?• It cultivates complacency and a disregard of rules

• Crews that commit at least one intentional noncompliance error are two times more likely to:

• Commit non-intentional errors• Have non-intentional errors with consequential outcomes

The Final Expansion: State ResponsesThe Final Expansion: State Responses

Undesired Aircraft State

ResponseMitigate

ExacerbateFail to Respond

Error Induced Incident or Accident

Error

External Threat Management

Error Response

Threat Induced Incident or

Accident

Recovery Additional Error

Undesired Aircraft State

External Threats

Inconsequential

Undesired Aircraft State Undesired Aircraft State ManagementManagement

• When an aircraft enters an undesired aircraft state, flightcrews manage the state not the error

• Undesired aircraft state responses• Mitigate – 79% of all error induced aircraft state responses• Exacerbate – 2%• Fail to respond – 12%

Threat and Error Threat and Error Management Behavioral Management Behavioral

MarkersMarkers

Behavioral MarkersBehavioral Markers

• After conducting observations on 3800 flights across seven airlines......

• Fifteen behavioral markers have been determined to be the most relevant in threat and error management

• The fifteen markers can be divided into four groups 1. Team Climate2. Planning3. Execution4. Review / Modify

Team Climate MarkersTeam Climate Markers

• Leadership • Captain showed leadership and coordinated flight deck activities

• Communication Environment • Environment for open communication was established and

maintained

• Flight Attendant Briefing• If performed, did the flight attendant brief address safety

concerns and crew coordination during emergencies

Planning Behavioral MarkersPlanning Behavioral Markers

• Briefing • Required briefing was interactive and operationally thorough

• Contingency Planning • Crew members developed strategies to manage threats before

they encountered them

• Workload Assignment • Roles and responsibilities were defined for normal and non-

normal situations

• Plans Stated • Operational plans and decisions were communicated and

acknowledged by other crew members

Execution Behavioral MarkersExecution Behavioral Markers

• Monitor / Cross-Check • Crew members actively monitored and cross-checked systems and

other crew members above and beyond procedures

• Workload Management • Operational tasks were prioritized and properly managed to handle

primary flight duties

• Vigilance • Crew members remained alert of the environment, safety threats and

the progression of the flight

• Automation Management • Automation was managed to balance situational and/or workload

requirements

Review / Modify MarkersReview / Modify Markers

• Evaluation of Plans • Existing plans were regularly reviewed and discussed

• Inquiry • Crew members asked questions to investigate and/or clarify

current plans

• Assertiveness • Crew members stated critical information and/or solutions with

the appropriate persistence

• Adaptability • Existing plans were altered under contingency situations in a

timely manner

One Last Look At The ModelOne Last Look At The Model

Undesired Aircraft State

Response

Error Induced Incident or Accident

Error

External Threat Management

Error Response

Threat Induced Incident or

Accident

Recovery Additional Error

Undesired Aircraft State

External Threats

Inconsequential

The Value of Threat and Error The Value of Threat and Error ManagementManagement

• Reinforces the goals of CRM training1. Manage external threats to safety2. Avoid flight crew error3. Manage flight crew error

• Encourages the examination of system defenses that help flight crews manage threat and error

• Aircraft design, technology, and manufacturing

• Regulations and procedures

• Training (Proficiency and CRM)

• Line support - ATC, Maintenance, Ground Handling, Dispatch and Cabin

Our WebsiteOur Website

www.psy.utexas.edu/psy/helmreich/nasaut.htm