July 23, 2013 Aviation Safety Reporting System (ASRS): What Information Tells Us Linda Connell, NASA ASRS Director Human Systems Integration Division NASA Ames Research Center Aviation Human Factors and SMS Seminar 2013 Frontiers of Flight Museum, Dallas TX
61
Embed
Aviation Safety Reporting System (ASRS): What … HF_SMS Seminar...July 23, 2013 Aviation Safety Reporting System (ASRS): What Information Tells Us Linda Connell, NASA ASRS Director
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
July 23, 2013
Aviation Safety Reporting System (ASRS):
What Information Tells Us
Linda Connell, NASA ASRS Director
Human Systems Integration Division
NASA Ames Research Center
Aviation Human Factors and SMS
Seminar 2013
Frontiers of Flight Museum, Dallas TX
Moffett Field - Hangar One
1932
Moffett Field - Hangar One
Aviation Safety Reporting System
ASRS Contributions to a System Safety Process
ASRS disseminates vital information to
the aviation community
This function is specifically designed to
help the aviation community recognize
accident precursors and take preventive
action
ASRS promotes the ability of safety
professionals to conduct more
prospective safety analysis
By evaluating the experience of other
reporters in the system, safety
professionals can learn from other reporters in similar situations without having to have to experience the undesired outcome for themselves.
Aviation Safety Reporting System Aviation Safety Reporting System
ASAP Review Committees could include an ASRS DataBase OnLine (DBOL) search of other events from a larger sample of similar events with ASRS Analysts providing preliminary analysis coding on numerous variables (contributing factors, human factors, etc).
As SMS procedures are implemented, an ASRS search could assist in efforts to discover information from additional data sources.
Topics for safety training can be enhanced with use of actual narratives describing events.
Newsletter and publication content can be expanded using illustrations of ASRS events.
Opportunities to Use ASRS Safety Data
Aviation Safety Reporting System Aviation Safety Reporting System 6
Since ASRS is a national repository of industry safety data, an ASAP Event
Review Committee (ERC) can rely on ASRS data to provide a systemic view
into the larger picture of emerging issues and hazards
The ERC team can use ASRS data to review the experiences of other
operators to help identify issues they have not yet encountered, or recognize
weaknesses in their own safety programs that may not have yet materialized
This may provide them the ability to detect potential hazards and consider
safety solutions which are discovered in reports submitted by similar operators
ASRS data can provide valuable insight into investigating factors that
contributed to a past event or provide insight into researching the potential of a
future issue
An open minded, forward looking approach to exploring potential hazards is
complementary to the carrier’s SMS program
What can ASAP ERCs gain from ASRS?
Aviation Safety Reporting System Aviation Safety Reporting System en Safety Reporting System
Database Searches
Direct access to an automated data search tool will provide de-identified reports from the ASRS Database. This is available online through the ASRS (http://asrs.arc.nasa.gov/search/database.html).
Information in the ASRS Database is publicly available. The ASRS can provide assistance upon request through Search Requests. ASRS will search its database, download relevant reports, and send to the person/organization requesting information.
For your convenience, selected relevant reports on several pre-selected safety topics are available at ASRS Database Report Sets. (http://asrs.arc.nasa.gov/search/reportsets.html).
The ASRS Database is also available through the FAA Aviation Safety Information Analysis and Sharing (ASIAS) website at http://www.asias.faa.gov/. This is updated monthly from ASRS.
Voluntary data are qualitative data
Data Typology
Qualitative
Describes the “why” and “how”
of events
Diverse responses through
narratives and storytelling
Captures themes, observations,
environmental context that
generates informed assertions
Holistic and contextual
Focus on in-depth
understanding of human
behavior and the reasons that
govern such behavior
Quantitative
Describes the “what” or “how
often” of events
Narrow categorizations for
unbiased measurements
Hypotheses driven focus on a
specific question and collects
sampling of numerical data
Reductionist and objective
Serves statistical, applied
inferential mathematics, or
computational techniques
Qualitative vs Quantitative Data
ASRS reports are subjective recollections of events experienced or witnessed by professionals operating in their specific domain
As a data source, these reports are characterized as qualitative and voluntary, which provide insights into the “why” of events from people operating within the system
Qualitative and voluntary data compliments quantitative sources that describing the “what” and “how often” an issue occurs in the system
Qualitative and voluntary systems provide a means to capture candid disclosures of the human factor contributions to safety events and can offer validation as to whether a mitigation has eliminated anomalies in the system
Statistical Nature of Voluntary Subjective Data
This data is not a random sample from the population of all events
Qualitative analyses might be appropriate, if properly constructed
Hypothesis generation possible for future research
Traditional quantitative statistics not appropriate (significance results,
trending, probability, frequency, correlation, etc.
Misuse of qualitative data in statistical analysis can
produce subtle, but serious errors in description and
interpretation –
Subtle in the sense that even experienced professionals can make
such errors, and
Serious in the sense that they can lead to decision errors.
S
Standards For Using Qualitative Data
Voluntary Data and Statistical Bias
Bias in Voluntary Sampling • In voluntary sampling, bias refers to the tendency of a sample statistic to
systematically over- or under- estimate a population parameter.
Non-response bias • Non-response bias results when respondents differ in meaningful ways from non-
respondents. Non-response is often problem with voluntary input, where the
response rate can be very low.
Voluntary Response Bias
• Voluntary response bias occurs when sample members are self-selected volunteers,
as in voluntary sampling. An example would be call-in radio shows that solicit
audience participation on controversial topics (affirmative action, gun control, etc.).
The resulting sample tends to over-represent individuals who have strong opinions.
. . . To provide information to the FAA and the aviation community to assist them in reaching the goal of identifying and eliminating unsafe conditions to prevent accidents.
Aviation Safety Reporting System
ASRS Gov’t/Industry Stakeholders
FAA provides reimbursable
funding to NASA for ASRS
Support & Management
NASA provides funding for
Director to provide overall
management
• Assures independence from
perceptions of potential conflicts of
interest
• Reinforces role of trust in success
The Aviation Community provides
support through advocacy for
reporting, feedback, and
communications
Aviation Safety Reporting System
System Recognition for Effectiveness
US News and World Report 2006 praises system FAA credited for a positive, proactive approach to safety
Proof that government agencies don’t have to act
like big-footed oafs, the FAA (and NASA) has a
system that allows pilots and air traffic controllers
to report problems anonymously, It’s a “Let’s
learn, let’s fix it” sort of culture. The goal is to
find systemic problems and solve them rather
than assign blame. Most errors and mistakes are
caught early, and accidents are rare.
Constellation of Aviation Safety Efforts
ASAP
FOQA
ASIAS
VDRP
LOSA
ASRS
CAST
HFAC
ASRS Overview and Status
Aviation Safety Reporting System
Aviation Safety Reporting System 19
ASRS Purpose and Mission
Identify deficiencies and
discrepancies in the National
Airspace System
Provide data for planning and
improvements to the future
National Airspace System
Precursors
Aviation Safety Reporting System
ASRS
FAA & NTSB
Normal Operations
Incidents
Injury
Accidents
Fatal Accidents
System-Wide Event Occurrences
ASRS is complementary to other systems of reporting
and focuses on precursors to the most severe events
Addressee agrees with AB/FYI but sees no problem 6%
Addressee in factual agreement but is unable to
resolve 3%
Addressee disputes factual accuracy of AB/FYI 19%
Information in AB/FYI insufficient for action 10%
For information only, no response expected 4%
Action not within addressee's jurisdiction 3%
64%
Aviation Safety Reporting System
1999 – 2012
ASRS Alert Messages and Feedback
Aviation Safety Reporting System
ASRS Alert Messages and Feedback
“The most troubling aspect of LED lighting concerns
is that they are often being discovered through
voluntary disclosure programs such as the Aviation
Safety Reporting System (ASRS). This strongly
suggests that not all of the unintended consequences
of LED lighting implementation have been discovered.
This conclusion alone should encourage the FAA to
take a more measured approach to LED lighting
implementation.”
Aviation Safety Reporting System
A Detailed Example of a Recent Alert
This Alert highlighted a critical flight safety anomaly
This anomaly had gone unnoticed during design,
certification, manufacture, oversight, and operations
in service
No SDR was issued since no event had yet occurred
An AD note was issued for a sister fleets but not for
this, newer fleet type that had the same design
configuration
ASRS was able to detect this anomaly though a single
report with no operational incident (weak signal)
Aviation Safety Reporting System
Did We Get Our Wires Crossed?
The configuration of the fire extinguishing agent
plumbing system can pose a potential challenge to
maintenance personnel
Several design features appear to have been used to
mitigate a possible system misconfiguration
However, this report shows it is still possible to
install the components in the wrong position
The reporter indicated that “in the worst case, if both
bottles were affected, neither engine would have fire
protection, with extreme risk for loss of life.”
Aviation Safety Reporting System
Embraer E170 Fire Bottle Installation Report 1050457
Normal Configuration
This graphic is for illustrative purposes only and not to be used for any other purpose.
Squib
1
A1 B1
Normal Configuration
Engine 1 Fire
Discharge Shot A
Normal Configuration
Engine 1 Fire
Extinguishing System
Abnormal Configuration
Engine 1 & 2 Fire
Extinguishing System
1
A1 B1
B2 B1
This graphic is for illustrative purposes only and not to be used for any other purpose
Embraer E170 Fire Bottle Installation Report 1050457
Aviation Safety Reporting System
Currently, there are no applicable Airworthiness
Directive or Service Bulletin for E170 series aircraft
AD 2004-23-19 is effective for E135 and E145 series
aircraft to correct the design anomaly
• General visual inspection to detect incorrect wiring
• Modify the electrical harnesses by adjusting the length
• Install identification sleeves on the harness and
connectors and matching color-coded stickers on the fire
bottles
Embraer E170 Fire Bottle Installation Report 1050457
• Structured Callback Studies
(funded by FAA)
• Weather Data Link
• Wake Vortex Encounter
• General Aviation Loss of Control • Accident/Incident Comparison
• RNAV – STARS and SIDS (TBD)
Aviation Safety Reporting System
Special Studies and Projects
Aviation Safety Reporting System
Meteorological and Aeronautical Information Services Data Link Services and Applications Study - ASRS is gathering reports of incidents that occurred while pilots were utilizing weather or AIS information in the cockpit (textual and/or graphical) obtained via data link (including ACARS) or other sources on the ground or in the air.
Wake Vortex Encounter Study - ASRS began this study in 2007 and will continue through 2013. Wake Vortex Encounter Study includes all airspace within the United States, en route and terminal. In quarterly reports, the ASRS documents event dynamics and contributing factors underlying unique wake vortex encounter incidents.
ASRS Conducts Studies on Special Topics
Aviation Safety Reporting System
ASRS Web Site
Launch Oct. 2007
• Over 9 million
sessions in 2012
File ASRS Report
• Electronic
• Print and Mail
Database Online
ASRS Publications
Program
Information
Immunity Policies
http://asrs.arc.nasa.gov
Aviation Safety Reporting System
ASRS Database Online (DBOL)
DBOL launched August 23,
2006
• Over 118,000 total online queries completed to date
• Over 19,600 queries completed in 2012
Fixed field and text search capability
Data formats (export)
• MS Word, Excel, CSV HTML
Experts version (DBOL II) being proposed
http://asrs.arc.nasa.gov
July 23, 2013
ASRS Model Applied to
Aviation & Other Industries
Aviation Safety Reporting System
International Confidential
Aviation Safety Systems (ICASS)
South Korea
KAIRS (2000) United States
ASRS (1976)
United Kingdom
CHIRP (1982)
Canada
CASRP (1985)
SECURITAS (1995)
Australia
CAIRS (1988)
REPCON (2006)
Russia
VASRP (1992)
Brazil
RCSV (1997)
Japan
ASI-NET (1999)
Taiwan
TACARE (2000)
Singapore
SINCAIR (2004)
France
REC (1999) China
SCASS (2004)
New Zealand
ICARUS
South Africa
SASCO
Germany
EUCARE
Spain
SNS (2007)
Aviation Safety Reporting System Aviation Safety Reporting System
ASRS - Model for Confidential Reporting
There is a growing interest in utilizing the
ASRS reporting model for application to other
disciplines such as medicine, fire fighting,
railroad, maritime, security, and others.
Aviation Safety Reporting System
ASRS Model Applied to Other Industries
• Confidential Close Call Reporting System (C3RS) • Railroad Safety Reporting System was modeled after ASRS
• Under development at NASA ASRS through collaboration with
Federal Rail Administration and Volpe National Transportation
System Center
• Fire Fighters Near Miss Reporting System • Launched August, 2005 was modeled after ASRS
• Development Task Force included FAA and NASA ASRS
• Bureau of Safety and Environmental Enforcement
(BSEE) • New Interagency Agreement for offshore oil and gas industry safety
reporting system
Aviation Safety Reporting System
NASA ASRS Model Applied
to Railroad Confidential
Close Call Reporting System
(C3RS)
Aviation Safety Reporting System
BSEE Safety Reporting System
Aviation Safety Reporting System
Unique Aspects of ASRS Confidential
Reporting Model
System-Wide Perspective - capability to identify hazards identified by aviation personnel and match reports from all segments of aviation community
• ASRS was catalyst for recent FAA focus on Teterboro Departures
System-Wide Alerting - both national and international capability to provide ASRS Alert Messages to industry and government
Data Processing through Aviation Expert Analysts
• ASRS Office staff include Aviation Expert Analysts with a combined total of 380 years of experience in aviation (air carrier pilots, corporate pilots, general aviation pilots, air traffic control, and maintenance)
• Experts read and review 100% of reports and reliably code information to databases
Comprehensive and Time Tested Coding Taxonomy
• Fixed Field Codes combined with Narrative Text yields qualitative data for further secondary analysis techniques (Perilog, special studies, focused analytic techniques, etc)
Aviation Safety Reporting System
Strong Immunity and Legal Provisions
• Federal Law specifically addressing ASRS (14 CFR 91.25)
• FAA Advisory Circular 00-46E
• ASRS Addressed by Congress in 1980’s
Information Sharing - both nationally and internationally with industry and government
• Database Search Requests, Database Publically Available, Topical Studies, Structured Telephone Callback Studies, Collaborations with Industry and Gov’t (FAA, NTSB, NASA, TSA, etc.)
• Largest source of airline ASAP data collected in central location
National and International Reputation
• ASRS Recognized Model for Proactive Contribution to Safety Process
• ASRS Model Being Utilized by Other Domains for Safety Improvements