Page 1
University of North DakotaUND Scholarly Commons
Theses and Dissertations Theses, Dissertations, and Senior Projects
12-1-2016
A Study Evaluating if Targeted Training for StartleEffect can Improve Pilot Reactions in HandlingUnexpected Situations in a Flight SimulatorMichael Gillen
Follow this and additional works at: https://commons.und.edu/theses
This Dissertation is brought to you for free and open access by the Theses, Dissertations, and Senior Projects at UND Scholarly Commons. It has beenaccepted for inclusion in Theses and Dissertations by an authorized administrator of UND Scholarly Commons. For more information, please [email protected] .
Recommended CitationGillen, Michael, "A Study Evaluating if Targeted Training for Startle Effect can Improve Pilot Reactions in Handling UnexpectedSituations in a Flight Simulator" (2016). Theses and Dissertations. 345.https://commons.und.edu/theses/345
Page 2
ASTUDYEVALUATINGIFTARGETEDTRAININGFORSTARTLEEFFECTCANIMPROVEPILOTREACTIONSINHANDLINGUNEXPECTEDSITUATIONSINAFLIGHTSIMULATOR
By
MichaelWilliamGillenBachelorofScience,UniversityofNorthDakota,1992MasterofScience,UniversityofNorthDakota,2008
ADissertation
SubmittedtotheGraduateFacultyofthe
UniversityofNorthDakotainpartialfulfillmentoftherequirements
forthedegreeofDoctorofPhilosophy
GrandForks,NorthDakotaDecember
2016
Page 3
ii
Copyright2016MichaelWilliamGillen
Page 5
iv
PERMISSION
Title AStudyEvaluatingifTargetedTrainingforStartleEffectCanImprovePilotReactionsinHandlingUnexpectedSituationsinaFlightSimulator.
Department Aviation
Degree DoctorofPhilosophy
InpresentingthisdissertationinpartialfulfillmentoftherequirementsforagraduatedegreefromtheUniversityofNorthDakota,IagreethatthelibraryofthisUniversityshallmakeitfreelyavailableforinspection.Ifurtheragreethatpermissionforextensivecopyingforscholarlypurposesmaybegrantedbytheprofessorwhosupervisedmydissertationworkor,inhisabsence,bytheChairpersonofthedepartmentortheDeanoftheGraduateSchool.Itisunderstoodthatanycopyingorpublicationorotheruseofthisdissertationorpartthereofforfinancialgainshallnotbeallowedwithoutmywrittenpermission.ItisalsounderstoodthatduerecognitionshallbegiventomeandtotheUniversityofNorthDakotainanyscholarlyusewhichmaybemadeofanymaterialinmythesis.
MichaelW.Gillen
September20,2016
Page 6
v
TABLEOFCONTENTS
LISTOFFIGURES……………………………………………………………………………………………………………vi
LISTOFTABLES……………………………………………………………………………………………………………vii
ACKNOWLEDGMENTS………………………………………………………………………………………………….xi
ABSTRACT…………………………………………………………………………………………………………………..xii
CHAPTER
I INTRODUCTION…………………………………………………………………………………..…1
II LITERATUREREVIEW……………………………………………………………………………..8
III METHODS……………………………………………………………………………………………45
IV RESULTS………………………………………………………………………………………………60
V DISCUSSION……………………………………………………………………………………..….93
APPENDICES………………………………………………………………………………………………………….…106
REFERENCES……………………………………………………………………………………………………………..132
Page 7
vi
LISTOFFIGURES
Figure Page
1.AutomationBias(Parasuraman&Manzey,2010)..........................................18
2.BehaviorModel(Rasmussen,1983)................................................................27
3.CausesofAccidents(Boeing,2012).................................................................30
4.Flight3407–OneMinutefromImpact–SituationNormal(NTSB,2010)......33
5.Flight3407–30SecondsfromImpact–StickShakerActivation(NTSB,2010)..........................................................................................................33
6.Flight3407-27SecondsfromImpact-RolltotheLeft(NTSB,2010)..............34
7.Flight3407–21SecondsfromImpact-RolltotheRight(NTSB,2010)...........34
8.Flight3407–10SecondsformImpact-FinalRolltotheRightandPitchDown(NTSB,2010).....................................................................................35
9.Parametersfrom2:10:50to2:11:46(BAE,2012)...........................................40
10.EvolutionofAirspeedandPitotIcing(BAE,2012).........................................41
11.AF447FDRData(MM43,2011).....................................................................42
12.OutsideFlying................................................................................................62
13.CivilianversusMilitaryFlying........................................................................63
Page 8
vii
14.AerobaticTraining.........................................................................................64
15.StartlingEvents..............................................................................................65
16.PitchandPowerSettings...............................................................................67
17.HandFlyingBelow10,000Feet....................................................................68
18.ChairFlying....................................................................................................70
19.RawDataFlying.............................................................................................71
20.SkillsPractice.................................................................................................73
21.AutopilotUsage.............................................................................................75
Page 9
viii
LISTOFTABLES
Table Page
1.DescriptiveStatisticsBetaGroup....................................................................49
2.TestsofBetween-SubjectsEffectsBetaGroup...............................................50
3.DescriptiveStatisticsLowAltitudeScenarioBetaGroup................................50
4.TestsofBetween-SubjectsEffectsLowAltitudeScenarioLowAltitudeBetaGroup................................................................................................51
5.DescriptiveStatisticsHighAltitude-BetaGroup............................................51
6.TestsofBetween-SubjectsEffectsHighAltitude-BetaGroup.......................52
7.Cohen’sDCalculationBetaGroup...................................................................52
8.HighAltitudeAnalysis......................................................................................55
9.LowAltitudeAnalysis.......................................................................................55
10.EvaluatedFactorsandSeatPositions............................................................56
11.FlyingOutsideofProfessionalJob.................................................................61
12.DidYouFlyintheMilitary?............................................................................63
13.DoYouHaveanyFormalAerobaticTraining.................................................64
Page 10
ix
14.StartleEvents.................................................................................................65
15.IKnowtheProperPitchandPowerSettings.................................................66
16.HandFlyingBelow10,000Feet.....................................................................68
17.ChairFlyScenariostoHelpDetermineCoursesofAction.............................69
18.ComfortFlyingRawData...............................................................................71
19.OftenPracticeRawDataSkills.......................................................................72
20.AutopilotUsageAbove1000Feet.................................................................74
21.DescriptiveStatisticsHighAltitude................................................................76
22.TestsofBetween-SubjectsEffectsHighAltitude...........................................78
23.DescriptiveStatisticsRegressionHighAltitude.............................................79
24.DescriptivesLowAltitude..............................................................................80
25.TestsofBetween-SubjectsEffectsLowAltitude...........................................81
26.DescriptiveStatisticsLowAltitude................................................................82
27.DescriptiveStatisticsCombined....................................................................84
28.HighandLowAltitudeMeanComparison.....................................................84
29.MeansComparisonwithTraining..................................................................85
30.MeansComparisonNoTraining....................................................................86
Page 11
x
31.TestsofBetween-SubjectsEffectsCombined...............................................87
32.ANOVAOtherFactors....................................................................................88
33.T-testTrainingversusFAAStandard..............................................................89
34.HandFlyingPreferences..............................................................................100
Page 12
xi
ACKNOWLEDGMENTS
Therearesomanypeoplewhohelpedmealongthisjourneyanditwouldbe
impossibletonamethemall.Iwouldliketoacknowledgeafewspecialpeoplewhom
withouttheirsupportthisworkwouldhaveneverbeencompleted.Iwouldliketo
thankmyfamilyincludingmyparentsforalwayssupportingmeinthiseffort.Their
wordsofencouragementkeptmegoingwhenthecourseworkseemeddaunting.Ialso
wouldlikethankmycommitteewhoalwaysofferedsoundadviceandguidance
developingandwritingthisdissertation.Theirpatienceandwillingnesstooffer
constructivecriticismandsuggestionswereinvaluable.Finally,Iwouldliketo
acknowledgetwoindividualswhosawpotentialinayoungaviatorandgavemesound
adviceandafewbreaksalongtheway.WithheartfeltthanksIacknowledgeCaptain
JimCorley(TWA)andMr.ElmerSchaal.Thesetwoindividualsarenolongerwithus,
however,theiridealsandphilosophyhashelpedcarrymethroughmyacademic
endeavorsandmycareerasaprofessionalaviator.
Page 13
DEDICATION
Itwouldhavebeennearlyimpossibletotakeonaworkofthismagnitudewithoutthe
lovingsupportofmywifeSonyaGillen.Thisworkisdedicatedtoherforherunfailing
love,guidance,andunderstandingthroughoutthislongendeavor.
Page 14
xii
ABSTRACT
Recentairlineaccidentspointtoacrew’sfailuretomakecorrectandtimely
decisionsfollowingasuddenandunusualeventthatstartledthecrew.Thisstudy
soughttodetermineiftargetedtrainingcouldaugmentdecisionmakingduringastartle
event.Followingastartleeventcognitivefunctionisimpairedforashortdurationof
time(30-90seconds).Inaviation,criticaldecisionsareoftenrequiredtobemade
duringthisbrief,butcritical,timeframe.
Atotalof40volunteercrews(80individualpilots)weresolicitedfromaglobal
U.S.passengerairline.Crewswerebriefedthattheywouldflyaprofileinthesimulator
butwerenotmadeawareofwhattheprofilewouldentail.Thestudyparticipantswere
askedtocompleteasurveyontheirbackgroundandflyingpreferences.Everyother
crewreceivedtrainingonhowtohandleastartleevent.Thetrainingconsistedofa
briefingandsimulatorpractice.Crewmembers(subjects)wereeitherpresentedalow
altitudeorhighaltitudescenariotoflyinafull-flightsimulator.
Themaneuverscenarioswereanalyzedusingaseriesofone-wayANOVAs,t-
testsandregressionforthemaineffectoftrainingoncrewperformance.Thedata
Page 15
xiii
indicatedthatthetrainedcrewsflewthemaneuverprofilessignificantlybetterthanthe
untrainedcrewsandsignificantlybetterthantheFederalAviationAdministration(FAA)
AirlineTransportPilot(ATP)standards.Eachscenario’ssubfactorswereanalyzedusing
regressiontoexamineforspecificpredictorsofperformance.Theresultsindicatethat
inthecaseofthehighaltitudeprofile,problemdiagnosiswasasignificantfactor,inthe
lowaltitudeprofile,timemanagementwasalsoasignificantfactor.Thesepredicators
canbeusefulinfurthertargetingtraining.
Thestudy’sfindingssuggestthattargetedtrainingcanhelpcrewsmanagea
startleevent,leadingtoapotentialreductionofinflightlossofcontrolaccidents.The
trainingwasbroadandintendedtocoveranoverallaircrafthandlingapproachrather
thanbeingaircraftspecific.Inclusionofthistypeoftrainingbyairlineshasthepotential
tobetteraidcrewsinhandlingsuddenandunusualevents.
Page 16
1
CHAPTERI
INTRODUCTION
Emergenciesinaircraftofteninvolvehigh-stressdecision-making,whichmustbe
accomplishedcorrectlyinrealtime,oftenwithlimitedinformation.Crewsareoften
startledattheonsetofsuchevents.Evencorrectdecision-makingattheoutsetofan
emergencymaynotguaranteeasuccessfuloutcome.Unfortunately,incorrectinitial
decisionsatthestartofanemergencyoftenresultindelayedaircraftrecoveryandin
somecasesleadtoanundesiredaircraftstate(UAS).Decisionsinstressful
environmentsareoftenmadewithinformationfrompastexperiences,training,and
patternmatching(Rasmussen,1983).Althougheachemergencyissurroundedby
uniquecircumstances,trainingoverabroadarrayofscenariosandcircumstancesmay
giveflightcrewsenoughbackgroundinformationtomanagethesituationfora
successfuloutcome.
AstudycompletedbyWoodhead(1969)founddecrementsonadecision-making
followingastartleevent.Thackray(1969)alsofoundthatmajorperformance
decrementfollowingastartleeventprobablyoccurswithinthefirstfewseconds.Inthe
officialreportonAirFranceflight447,theBureaud'Enquêtesetd'Analyses(BAE,
2012)stated,“Thestartleeffectplayedamajorroleinthedestabilizationoftheflight
pathandinthetwopilotsunderstandingthesituation.”Startletrainingmaybeakey
Page 17
2
elementineffectiveemergencyflighttraining.Duringsuchtraining,crewsareexposed
todifferent,complex,andunusualsituationstheywouldnotnormallyencounterunder
normalflightconditions.Thepurposeofthistypeoftrainingistodevelopthepattern
behaviorofsystematicallydealingwithcomplexemergencies.
ProblemStatement
Recentairlineaccidentspointtoarapiddegradationfromcontrolledflight
followinganunusualeventwhentheflightcrewbecomesstartled.Therehasbeenvery
littletrainingamongairlinecrewsonhowtosuccessfullymanageasuddenandoften
stressfuleventthatrequiresquickandaccuratedecision-making(BAE,2012).Accident
datahasindicatedthatwhenanincorrectdecisionismade,thelikelihoodofa
successfuloutcomedecreases(Hilscher,Breiter,&Kochan,2012).Thisstudyseeksto
determineifspecificandtargetedtrainingcanhelpmitigatetheeffectsofflightcrews
beingstartledbyimplementingasetoftechniquesdesignedtohelpstabilizethe
cognitivethoughtprocessandbridgethetimeofcognitivedegradation.
PurposeoftheStudy
Theintentofthismixedmethodsstudyistotestthetheorythatenhanced
specifictrainingcanprovideaneffectivecountermeasuretofillthetemporarycognitive
degradationthatoccursduringastartleevent.Thestudywillusebothsurveyand
observedsimulatorperformancedatatotestthetheory.Ifthehypothesisiscorrect,
specifictrainingcouldbeaddedtoairlinequalificationprogramstobetterequipairline
crewsindealingwithsudden,unusualevents.
Page 18
3
ResearchDesign
Theresearchisdesignedasamixedmethodsstudyusingquantitativeanalysis
methods.Inthefirstphase,asurveywillbeconductedoftheparticipatingaircarrier
pilots.Thissurveywillbeusedtogaugethepilot'sownperceptionsoftheirflyingskills
duringastartleevent.Theanalysiswillexploreforcommonthreadsofpilotthinking
andreactions.Theresultsofthesurveywillbecomparedandcorrelatedtothedata
fromtheaircraftsimulatorscenariosets.
Thesecondphaseofthestudyinvolvesevaluatingprofessionalairlinepilots
flyingtwodifferentscenariosinanFAAapprovedLevel-Dfullflightsimulator(FFS).The
scenarioswillbeflownbyacrewconsistingofacaptainandfirstofficer,similartowhat
wouldhappeninactuallineoperations.Eachcrewwillbepresentedeitheralowor
highaltitudescenariodependingonthedayoftheweek.Randomlyselectedcrewswill
receivetrainingonhandlingtheaircraftduringastartleevent.Thepilotgroupthatdoes
notreceivethestartletrainingwillbeconsideredthecontrolgroup.Thisgroupwillbe
referredtoastheuntrainedgroupforthepurposesofthisstudy.Thetrainingconsists
ofbothabriefingandsimulatorpractice.Thetrainingbriefingisviaaninperson
discussionontheproperpitch,power,andbank,settingsthatshouldbeflowninan
unusualevent.Thebriefingalsodiscussestimerecognitionespeciallyatlowaltitudes
andfuelstates.Thetrainingcontinueswithsimulatorpracticeusingthetechniques
discussedinthebriefing.Thepracticesessionsarenotthesameastheevaluation
profiles.Eachcrewwillpracticebothalowaltitudeandhighaltitudescenario.
Page 19
4
Thefirsttestscenarioisalowaltitudeandlowfuelprofile.Timepressureandan
unexpectedmissedapproachcombinetoformthestartleeventandeventevaluation
beginsatthemissedapproach.Thesecondtestscenarioisahighaltitudeprofile.The
profileinducesalossofairdatafollowedbyanenginefirebellthatcausesthestartle
event.Evaluationbeginsatthelossofairdata.Intermsofprocedures,thelossofair
dataisoftenreferredtobyaircraftmanufacturersasMach/AirspeedUnreliable.Data
analysiswillconsistofregression,ANOVA,andpost-hocTukeytests.Theanalysislooks
fordifferencesbetweentrainedanduntrainedgroupswithregressionlookingfor
differenceswithingroupssuchaspreviousexperience.
ResearchQuestions
Theresearchquestionssurroundthecognitivegapthatisperceivedtoexist
duringastartleeventandtowhatextenttrainingcanmitigatethegap.
1. Cantargetedtrainingbesuccessfulinhelpingpilotsmaintainaircraft
controlduringanunusualandsuddenstartleevent?
a. Doesthespatialproximityoftheeventhaveanyeffectonthe
outcome(loworhighaltitude)?
b. Sinceaccidentdataindicatesthataccidentsoccurmorefrequently
withthecaptainflyingandonthefirstdayandfirstlegofatrip,does
thepilotflying,eitherthecaptainorthefirstofficer,haveanyeffect
onthesuccessfuloutcomeoftheevent?
Page 20
5
2. Isthereanindicationinthepilot’ssurveyanswersthatisapredictorof
beingabletosuccessfullyhandleastartleevent?
Assumptions
Thelistbelowisnotmeanttobeall-encompassingbuttoinformthereaderasto
themajoraspectsinvolvedinthestudy.Thisstudydevelopsfindingsbasedonthe
followingassumptions.
1. Eachparticipantisaqualified,AirlineTransportPilot(ATP)certifiedFARpart
121jettransportpilotemployedbyaU.S.aircarrier.
2. Eachparticipanthasspentatleastoneyearinthespecificseat(Captainor
FirstOfficer)andtypeofaircraft.Itisassumedthatafteroneyearof
experienceonaparticularaircraft,thatthepilotwillbenormalizedtoflying
thatparticularaircraft(theaircraftwillnotbe“new”tothem).
3. Eachpilotiscurrentandqualifiedintherespectiveaircraft.Currentand
qualifiedindicatesthatthepilotcanbescheduledtoflyaregularpassenger
tripatanytime.
4. Eachpilotisconsideredalinepilot.Forthepurposeofthisassumption,line
pilotmeansthateachpilotfliestheirrespectiveairplaneatregularintervals.
Linepilotsinclude;Captains,FirstOfficers,LineCheckAirman,and
Instructors/Evaluators
Page 21
6
5. Exceptforthegroupthatreceivestraining,thepilotshavenoprior
knowledgeorpracticeofthemaneuverthatistobeflownandisgivenno
opportunitytopracticeitbeforehand.
6. Eachpilotisassumedtoflytothebestoftheirabilityduringthemaneuver.
Limitations
Thestudysoughttomitigatepossiblelimitationsthatcouldskewtheresults.
Althougheachstudyhasasetofuniquelimitations,theresultsofthisstudyshouldbe
consideredinthecontextofthelimitationslistedbelow.
1. Themaneuversetsareflowninasimulatorthatisrealisticinnaturebut
involvessimulationlimitsspecificallythegenerallackofg-forces.
2. Thestudyonlylooksattwomaneuversets.
3. Aircraftemergenciesareoftendynamicandareuniquetoeachsituation.
4. Thestudyonlyobservedprofessionalpilotsflyingtransportcategory
aircraft.
5. Crewsvolunteeredforthisstudywhichmayindicateahigherawarenessof
safety.
6. Crewsweresampleddirectlyaftertheirrecurrenttrainingwhichmay
increasetheirproficiencyabovewhatmightbeexpectedinnormalline
operations.
ExpectedFindings
Thestudyhasfourmajorgroupsofsubjectsthatarebeingcomparedtoaset
standardasdefinedtheFAAandairlinepolicyforsuccessfuloutcomes.Thestudy
Page 22
7
expectstofindcrewsthatreceivetrainingwillbecomemoreawareandproficienton
howtohandleaninflightemergencywithregardstothefirstfewcriticaldecisions
and/oractions.Theexpectationsarethatthetrainedgroupforbothlowandhigh
altitudescenarioswillshowastatisticallysignificantincreaseinperformancewhen
comparedtothenon-trainedgroups.Inaddition,whencollapsedformaneuverssets,
thetrainedgroupsshouldshowasignificantincreaseinperformancethanthenon-
trainedgroup.Itemssuchaspastexperienceandcurrentoutsideflyingmightbe
reasonablyexpectedtoalsoinfluencesuccess.
Page 23
8
CHAPTERII
LITERATUREREVIEW
Introduction
Aliteraturereviewwasconductedofpertinentarticlesrelatedtothisstudy.
Althoughtherewerenodirectstudiesonthisparticularproblem,thereweremany
articlesrelatedtodecisionmaking,startleeffect,andtimecriticalactionsrequiredby
flightcrews.Thereviewbeginsbyrelatingadiscussionontrainingofunusualeventsand
whypracticeisimportantforimprovedperformance.Theliteraturereviewthen
discusseswhatstartleeffectisandhowiteffectscognition,decisionmakingandpilot
responses.Thereviewthentakesthetheoreticaldiscussionandrelatesittoactual
aircraftaccidentsusingtheofficialreportsasabackground.Thisliteraturereviewisnot
meanttobeallencompassing,buttogivethereaderabroadoverviewoftheissues
surroundingthisstudy.
TrainingandUnusualEvents
Accidentreportsdescribemanysituationswherepilotsrespondedtoabnormal
eventsinwaysthatwereunexpectedfromthewaythattheyweretrained(Casner,
Geven,&Willliams,2012).Unfortunately,trainingandtestingofprofessionalairline
pilotshavebecomesomewhatroutineandpredictable.Inshort,theflightcrewsknow
whattoexpectastheyseethesamemaneuversateachtrainingevent.
Page 24
9
InastudybyDr.StephenCasner,(when)pilotswereevaluatedperforming
routinetrainingeventsandunexpected(butsimilar)ones.Pilotresponsestothe
routineeventsshowedlittlevariability.Incontrast,pilotresponsesintheunexpected
maneuversshowedgreatvariabilityfrompilottopilot(Casner,Geven,&Willliams,
2012).Theresultsofthestudyshowedthatmostpilotsgenerallyexperiencethesame
sequenceofabnormalevents,presentedundersimilarcircumstances.Thisisdueto
boththeairlinetrainingenvironmentandtheregulatoryenvironmentassetforthbythe
FAA.Thistrainingcallsintoquestiontheextenttowhichpilotshavetheabilityto
respondtoabnormaleventsinactualoperations(Casner,Geven,&Willliams,2012).
Casner,Geven,andWilliams,suggestedthatsuchtrainingcanleadtoshallowand
memorizedunderstandingsofproblemsituationswhichinturndonotleadtoanability
totransferthistrainingtodifferentencountersinactualoperations(Casner,Geven,&
Willliams,2012).Theendresultofthestudywasthatpilotsstruggletorecognize
unexpectedsituationswiththeresultofconsiderablydelayedresponses(Casner,Geven,
&Willliams,2012).
Itisunlikelythattrainingalonecaneliminatetheelementofsurprisefrom
unexpectedeventsalthoughskillandexperienceareknowntoreducetheoccurrence
and/orseverity(Merk,2009).Furthermore,forunusualevents,pilotswouldbenefit
fromexposurewithouttheuseofautomationtoenablethemtobetterrecognizethe
situationitselfratherthanrespondtoanalert(Wiener,1985).Finally,themost
importantstepintrainingistotrainabnormaleventsoverawidearrayofcircumstances
andoperationparameters(Casner,Geven,&Willliams,2012).
Page 25
10
Safetymanagementtendstofocusonpreventionoferrorsandfailures.Inmost
failurecases,thereareopportunitiestorecoverfromthefailurethroughthetimelyand
effectiveapplicationofcountermeasures.Theaimofthecountermeasurestoprevent
thenegativeconsequencesofthefailure(Kranse&vanderSchaaf,2001).Inthecaseof
unforeseenfailures,humanoperatorsplayakeyroleintheapplicationofeffective
countermeasures.Researchersgenerallyagreethatthefailurecompensationprocess
hasthreephases:
• PhaseOne:Detectionofthefactthatsomethinghasgonewrong
• PhaseTwo:Explanationorlocalizationofthecauses
• PhaseThree:Correctionoftheproblemthroughplanningandexecution
ofcountermeasures
AstudybyKranse,andvanderSchaaf,(2001)askedthequestion:howdoesthe
failurecompensationprocesswork,andwhatfactorsinfluencetheprocess?Afailure
canbeacombinationoftechnical,organizational,orhumanfactors.Detectionofthe
situationisalwaysthefirstfailurecompensationphasetooccur(Kranse&vander
Schaaf,2001).Afterthedetectionphase,timeoftendictatesthenextstep.The
correctiveactioncanbeperformedimmediately(usuallythecaseinaviation)orona
longertermforsystemicissues.Thestudyexamined50reportedfailuresatachemical
plant.Thefailureswereallreportedviaavoluntarysafetyreportingsystemsimilarto
AviationSafetyActionProgram(ASAP)reportingsystems.Thestudynotedthatinmost
detectionandlocalizationphasesunplannedactions(nottrained)occurred.Inthe
detectionphase,46outof50cases(92%)involvedunplannedactions(Kranse&vander
Page 26
11
Schaaf,2001).Forfailuresrequiringimmediatelocalization,100%involvedsome
unplannedactions.Suchwasthesameforeventsrequiringimmediatecorrection,
where80%involvedunplannedactions(Kranse&vanderSchaaf,2001).Eveninevents
thatwerenotastimecritical,wherelongertermcorrectionsweresought,77%
experiencedunplannedactions(Kranse&vanderSchaaf,2001).Generally,thetype
andseverityofpotentialconsequenceswerethemostpracticalfactorindecidingwhich
recoverypathstotake.TheKranseandvanderSchaafstudyhasimportantimplications
foranytypeofaviationtraininginasmuchasuntrainedactionswillinvariablytake
placeinarecoveryfromanunusualevent.
PracticeandPerformance
Astudythatwaspublishedin(McKinney&Davis,2003)researchedtheeffectsof
deliberatepracticeoncrisissituations.Withinprofessionaldomains,deliberatepractice
hasbeenpositivelycorrelatedwithimprovedperformance(McKinney&Davis,2003).
Thestudyexaminedthequestion,“dothebenefitsofdeliberatepracticecreatesuperior
performanceifpartofthetaskisunpracticed?”(McKinney&Davis,2003).Researchers
revieweddecision-makingundercrisisconditionsusingatotaldecisioneffectiveness
model.Themodelevaluatedboththeinitialassessmentandtheactionstakenforboth
practicedandunpracticedmaneuvers.Additionalstudieshaveindicatedthatdeliberate
practiceresultsinautomatedpatternmatchingofproblemswithsolutions(Richman,
Gobet,Stazewski,&Simon,1996).Practicedskillsallowforamoreaccuratediagnoses
ofthesituation,andimprovesbothspeedofactionandtheaccuracyofrecall(KleinG.,
1993).Practicemayaidinthecognitiveprocessesthroughenhancementofhigher
Page 27
12
levelsofsearchingandevaluating.This,inturn,mightenhancetheabilityto
extrapolatebeyondthepresenteddataandmakeuseoflong-termmemoryitemswhich
isricherandmoreorganizedthanshort-termmemory(Ericcsson,1996).
Forthisstudy,whollypracticedmaneuversareonesthatpilotshavedeliberately
practicedeitherinflight,ormorelikely,inthesimulator.Eachofthesemaneuvershad
abestpracticesolution,whichwasreinforcedonaregularbasis.Incontrastapartially
practicedmaneuverwasdefinedasaspecificaircraftmalfunction,occurringwithina
widerflyingscenariothatwasnovelorunique,andamaneuverthatthepilotcouldnot
havepracticed(McKinney&Davis,2003).Thesemaneuverscouldincludeitemssuch
asmultiplesystemfailures,flightcontrolmalfunctions,andunusualfailuremodes.The
dataforthestudywascompiledfrom173U.S.AirForcefighteraircraftmechanical
malfunctionincidents.Ineachcase,themishapwasclassifiedaseitherwhollypracticed
orpartiallypracticed.ThreeindependentpanelsofexperiencedAirForcepilots
reviewedandratedthemishaps.Theactiontakenbythepilotswasratedaseffectiveor
ineffective(McKinney&Davis,2003).Furthermore,athirdgroupofevaluatorswere
askedfurtherdefinewherethefailureoccurredincasesofineffectiveresponses.The
groupsoughttodefineiftheineffectiveresponseswereinthedecision-makingprocess,
orintheselectionofactionprocess.Toevaluatetheresearchquestions,logistic
regressionanalysiswasusedasatoolforpredictinggroupmembershipincaseswhere
dependentvariablesaredichotomous(McKinney&Davis,2003).Intotal83ofthe
eventswerecharacterizedaswhollypracticed,ofwhich68endedwitheffective
decisionsand15withineffectivedecisions(McKinney&Davis,2003).Thestudy
Page 28
13
concludedthatdeliberatepracticehasapositiveeffectoncrisisdecision-making
performance(McKinney&Davis,2003).Increasedperformancewasnotedforeach
decisionphaseforwhollypracticedmaneuvers.However,thestudyalsofoundthatno
relationshipexistedwherethecrisis-flyingscenariowasunpracticed(McKinney,2003).
Thestudyalsonotedthatdeliberatepracticewithintheflyingdomainwasnotrelated
tooveralldecision-makingperformance(McKinney&Davis,2003).
StartleEffect
Startleeffectisquitedifferentthanstartletraining.Thestartleresponsehas
beenwellresearchedanddocumentedoverthepast60years.Astartleresponse
happenswhenthehumanbrainispresentedwithasituationthatcompletely
overwhelmstheavailablecognitiveresourcesneededtoeffectivelymitigatethe
situation.Ithasbeenwidelyestablishedthroughpsychologicalresearchthatourability
toregulateourownthoughtsandbehaviorsbecomesdiminishedduringanemotional
event(Hilscher,Breiter,&Kochan,2012).Thisdiminishedabilityiscompoundedbythe
reliabilityoftoday’smodernaircraft,whichhascreatedaconditionedexpectationof
normalcyamongstpilots(Martin,Murray,&Bates,2012).Researchhasshownthat
thereareconsiderablecognitiveeffectsoninformationprocessingfollowingastartle
event.Theresultsindicatethatstrongcognitiveanddexterousimpairmentcouldlast
forupto30secondsfollowingastrongstartle(Vlasek,1969;WoodheadM.M.,1959;
WoodheadM.,1969;Thackray&Touchstone,1970).Apilotdescribinganencounter
withsevereturbulencedescribedthesituationas“theconstantaudiblewarningscame
fromfar-off,detachedfromthestruggleinthecockpit”(Hilscher,Breiter,&Kochan,
Page 29
14
2012).TheBureaud'Enquêtesetd'Analyses(BAE)intheofficialreportonAirFrance
Flight447(AF447)accident,themadethefollowingstatements:
“Thestartleeffectplayedamajorroleinthedestabilizationoftheflight
pathandinthetwopilotsunderstandingthesituation.Initialand
recurrenttrainingasdeliveredtodaydoesnotpromoteandtestthe
capacitytoreacttotheunexpected.Indeed,theexercisesarerepetitive,
wellknowntocrewsanddonotenableskillsinresourcemanagementto
betestedoutsideofthiscontext.
Alloftheeffortinvestedinanticipationandpredeterminationof
proceduralresponsesdoesnotexcludethepossibilityofsituationswitha
‘fundamentalsurprise’forwhichthecurrentsystemdoesnotgenerate
theindispensablecapacitytoreact.”(BAE,2012)
Theresponseofthebrain,andtheconsequentbehaviorisanamalgamation
resultingfrompastexperienceandgeneralexpertise(Isaac,2012).Onceanunusual
situationhasbeendeterminedtoexist,pilotsattempttocomparethesituationwith
pastexperiencesthroughasequenceofpatternmatchinganddecision-making.The
outcomeoftenreliesontheseverityofunusualcircumstanceandemergencytraining.
Inaddition,otherfactorsincludepriorexperienceandtheabilitytoaccepttheactual
factsofthesituation.Discrepanciesbetweenperceptionandtheactualaircraftstate
leadstoabreakdownofapilot’smentalpicture,whichinturncanleadtoalossof
situationalawareness(Hilscher,Breiter,&Kochan,2012).Surprisingeventscanplace
Page 30
15
thepilotintoaveryhighstateofarousalthatcanrenderthemineffectiveincomplex
decision-makingtasks(Hilscher,Breiter,&Kochan,2012).Thefinalresponseisoftena
stronglydevelopedbehaviorwiththepurpose,inextremecases,ofsurvival.Thereare
examplesinwhichhighlytrainedcrewsdiscardedindicationsfrominstrumentsand
flighttrainingafterastartleevent(Isaac,2012).Thisleadstodisbelievingwhatis
actuallypresentedtothecrewfromtheaircraft’ssystems.Onceanunusualor
emergencysituationispresented,apilotwillgenerallybelimitedintheirresponse.The
responsetendstofallintopatternsapilothasseenbefore,andwillalsobesubjectedto
severaldecision-making,behavioralbiases.Anobjectivethatisnotaddressedin
traditionalflighttrainingisbehavioralmanagementthatpromotesprogressive
functionalityunderconditionsofuncertaintyandfear(Hilscher,Breiter,&Kochan,
2012).
AutomationBiasandComplacency
Automateddecisionaidssupportdecision-makingincomplexenvironments.As
such,automationisassumingincreasingcontrolofcognitiveflighttasks,suchas
calculatingfuel-efficientroutes,navigating,ordetectinganddiagnosingsystem
malfunctionsandabnormalities(Mosier,1998).Thesesystemsaredesignedtosupport
thehumancognitiveprocessingofinformationtocorrectlyassessagivensituationand
torespondappropriately(Parasuraman&Manzey,2010).Automation-induced
complacencyandbiasrepresentcloselylinkedtheoreticalconceptsthatshow
considerableoverlapwithrespecttounderlyingprocesses(Parasuraman&Manzey,
2010).Automationcomplacencycanoccurwhentheautomationcompetesforthe
Page 31
16
pilot’sattentioninamultipletaskloadenvironment(Parasuraman&Manzey,2010).
Althoughsomewhatdifferentbutinterconnected,automationbiasresultsinmaking
bothomissionandcommissionerrorswhentheautomateddecisionaidsarenot
accurate(Parasuraman&Manzey,2010).Thesetwoissuesaffectbothnoviceand
expertpilotsandcannotgenerallybemitigatedthroughtraining.
Researchstudieshaveindicatedthatautomationdoesnotalwaysenhance
humanactivity.Insomecases,automationcanchangebehaviorpatternsinwaysthat
areunintended,andcannotbeunanticipatedbyautomationdesigners(Parasuraman&
Manzey,2010).Automatedsystemsintoday’smodernaircraftarehighlyaccurateand
reliable.Theendresultisthatpilotscandevelopaprematurecognitivecommitment
regardingtheinformationdisplayedbytheautomationanddisregardotherconflicting
information(Parasuraman&Manzey,2010).Automationbiascanleadtoincorrect
decisionsthatarenotbasedonacompleteanalysisoftheavailableinformationandcan
compromiseperformanceespeciallyinthecaseofautomationfailure.
Automateddecisionaidsaremisusedfortwomainreasons.Thefirstreasonis
thatautomationgeneratedcuesareusuallysalient,andbydesign,drawtheuser’s
attention(Parasuraman&Manzey,2010).Thesecondmajorfactoristhatusershavea
cognitivebiastoassigngreaterrelevancetoautomatedcuesoverothersourcesofdata
(Parasuraman&Manzey,2010).
InastudybyLayton,Smith,andMcCoy(1994),acomparisonofelectronicflight
planningtoolswasexamined.Pilotswhoweregivenhighlyautomatedflightplansspent
Page 32
17
lesstimeandeffortevaluatingalternateplansthangroupsworkingwithmanually
developedplans.Thisresultwasconsistentwiththecognitive-miserhypothesisof
automationbias(Layton,Smith,&McCoy,1994).Thepilotstendedtoaccepttheplan
generatedbytheautomationevenwhenitproducedlessthanoptimalsolutions.
Anotherstudyonautomationbiassoughttoquantifytheeffectsofautomation
over-relianceinmoderncockpits.Thisstudypointedouttheneedforpilotstobeable
toflytheairplanewhentheautomationdoesnotfunctioncorrectly.Automationbias
referstoomissionandcommissionerrorsresultingfromtheuseofautomatedcuesasa
heuristicreplacementforvigilantinformationseekingandprocessing(Mosier,Skitka,
Heers,&Burdick,1997).Highlyautomatedcockpitstendtochangethewaypilots
performtasksandmakedecisions.Researchershavedocumentedproblemsintheuse
ofadvancedautomatedsystems,includingmodemisunderstanding,failuresto
understandautomatedbehavior,confusionorlackofawarenessconcerningwhat
automatedsystemsaredoing,anddifficultytracingthefunctioningorreasoningprocess
oftheautomatedagent(Billings,1996).Figure1belowdiagramsbothpositiveand
negativefeedbackloopsassociatedwithautomation.Eachlooporpathwaycanleadto
biasonthepartofthepilot.
Page 33
18
Figure1.AutomationBias(Parasuraman&Manzey,2010)
Pilotsaretrainedanddeveloptheirskillsassessmentthroughtheuseofboth
systemandenvironmentalcues(crosscheckingofinformation).Inmostsituations,
processingisfacilitatedbyinter-correlationsamongcues(Wickens&Flach,1998).In
thecross-checkingenvironment,relatedtooldertechnologyaircraft,pilotsoftenlooked
formanycuesindeterminingifaproblemexisted.Usingtheseskills,pilotsknowand
lookforpatternsorcombinationofcuesthataremostecologicallyvalid,reliable,or
relevantfordiagnosingparticularsituations.Theyareabletoincorporatecontextual
informationtoformulateaworkableactionplanbasedontheirassessmentofthese
cues(Kaempf&Klein,1994).
Whenautomatedaidsareintroduced,thepatternofcueutilizationisdisrupted.
Automatedaidspresentpowerfulandgenerallyhighlyaccuratecues.Thisleadstothe
overallattitudethattheautomatedcuesarenotjustanothercue,butthemost
Page 34
19
powerfulandimportantcue.Theseautomateddecisionaidssupportthegeneral
humantendency“totraveltheroad”ofleastcognitiveeffort.Peoplewillgenerally
utilizeheuristics(cognitiveshortcuts)toreduceeffortandinformationload.
Forrigidtasksthatdonotrequireflexibledecision-makingautomationcanoften
providethebestsolution(Cummings,2016).Intime-criticalenvironmentsthathave
externalandchangingconstraints,higherlevelsofautomationmaynotbeadvisabledue
totherisksandthecomplexityofthedecisionaidsnotbeingperfectlystable.
(Cummings,2016).Situationawareness,complacency,andskilldegradationarethe
measurablecostsofautomationbias.
BreakdowninCoordination
Errorscanneverbecompletelyeliminatednecessitatingtheneedfordetection,
diagnosis,andrecovery(UnitedAirlines,2016).Eventdrivingtasksanddomainshave
seenalackofresearchinerrordiagnosisandrecovery(Nikolic&Sarter,2007).Astudy
jointlyconductedbytheBoeingCompanyandtheUniversityofMichiganin2007sought
tosomeinsightintoerroranddisturbancemanagementstrategies.Thestudynoted
thatpilotsseldomfollowthecanonicalpathtohandledisturbanceevents(Nikolic&
Sarter,2007).Acanonicalpathcanbeconsideredthemostoptimumsolutionthatis
technicallycorrectindevelopingdiagnosisandrecoveryoptions.Detectionofsuch
eventswereoftenobservedtobedelayedduetopilots’knowledgegapsandtime
criticality,andinmanycases,genericandinefficientrecoverystrategieswereobserved
(Nikolic&Sarter,2007).Inaddition,pilotsoftenreliedonhighlevelsofautomationto
Page 35
20
managetheconsequencesoftheinducederrors.Thestudynotedthatpilotsoften
attempttodiagnoseautomation-relatedproblemsbeforetheyrespondedtotheactual
disturbancehandling(Nikolic&Sarter,2007),meaningthatpilotstendedtobecome
focusedontheautomationinsteadofflyingtheaircraft.All18pilotcrewsintheBoeing
studystruggledatsomepointwithhandlingevents(actualflying)duringasimulated
flightintheB747.Itwasnotedthatthepilotsinthestudyrarelyattemptedtodiagnose
thesourceofthedisturbance(Nikolic&Sarter,2007).Thestudyfindingsofindicate
thatpoordisturbancemanagementissomewhatrelatedtothedesignofthe
automationinterfaces(Nikolic&Sarter,2007).
CognitiveResources
Asskilllevelsdecline,apilotmustdevotemorecognitiveresourceswhen
situationssuchasemergencies,systemfailures,orotherissuesthatforceapilotinto
manualflying.Inaddition,apilotbeginstolosetheabilitytomentallyprojectwhere
theairplaneisinspacewithregardstoaltitude,airspeed,andconfiguration.Simply
stated,apilot’scognitiveresources(infacteveryhuman)isfinite.
Twobasicparametersaffectperformance:theamountofcognitiveresources
availabletothepilotandthecomplexityofthetaskorsituation.Taskperformance
dependsontherelationbetweenthetwoparameters,cognitiveresourcesavailableand
thecomplexityofthesituation.Aslongastheamountofresourcesconsumedbythe
taskislowerthan,orequalto,theavailableamountofmemory,taskperformancewill
beadequate(Ippel,1987).However,taskperformancewillgraduallydeclinerelativeto
Page 36
21
thedegreethattasksimposecognitiveloadsthatexceedtheavailableamountof
resources(Ippel,1987).Iftoolittleprocessingresourceisapplied(becauseof
limitationstotheavailabilityofprocessingresources),performancefailureistobe
expected.Asmoreandmoreresourcesareappliedtothetaskthelikelihoodof
successfulperformanceincreases(Norman&Bobrow,1975).
StartleEffectandCognitiveConsequences
Thestartleeffectiscommontoallmammals(Simons,1996).Itconsistsofan
involuntaryreactiontoanunusualstimulus.Thisreflexusuallyhappensquickly
followingthestimulus,generallyinaslittleas14milliseconds(Yeomans&Frankland,
1996).Researchhassuggestedalinkbetweencommonpatternsofthestartlereflex
andtheneuralpathwaysinvolved(Davis,1986;Eaton,1984;Landis&Hunt,1939;Lang,
Bradley,&Cuthbert,1990;LeDouxJ.E.,2000;LeDouxJ.,1996;Whalen&Phelps,2009).
Theseactionsinvolvevarioussensesandmusclesandtheamygdalainthelimbicregion
ofthebrain.Theinitialanalysishappensveryquickly(500milliseconds)andresultsin
anaversivereflexawayfromthestimulus.Thestartlereactionmaylastbetween.3to
1.5seconds,dependingontheseverity(Martin,Murray,&Bates,2012).Anissuearises
whenthethreatpersistsandthestartlereactionbecomesafull-blownstartleor
surprisereaction,otherwiseknownas“flightorfight.”Thisprocesscanleadto
confusionordelaysinprocessing.Whenpeoplearestartledandthethreatpersists,
suchasinalife-threateningaircraftemergency,thenthestartlereflexislikelyto
transitionintoafullstartlereaction,withitsensuingactivationofthesympathetic
nervoussystem(Martin,Murray,&Bates,2012).Asuddenstartlingeventcanhave
Page 37
22
negativeeffectsonperformance(Martin,Murray,&Bates,2012).Thisisespecially
detrimentalinthecaseofanemergencywerecorrectdecision-makingisimportantto
resolveanissue.AstudyconductedfortheFAA(1969),demonstratedthatastartle
eventnegativelyaffectedperformanceand,alsonotedthatrecoveryofperformance
followingastartleeventappearstobequiterapid(Thackray&Touchstone,1970).
Uponinitialpresentationofthestartlestimulus,maximumdisruptionoccurredduring
thefirstfivesecondsafterthestimulation,withsignificantbutconsiderablyless
disruptionafterthesecond5-secondintervallastingfrom30secondstooneminute
(Thackray&Touchstone,1970).
Vlasak(1969)inhisstudy,investigatedtheeffectsofstartleonacontinuous
task.Thistaskwasmeasuredforaccuracyandconsistency.Testparticipantswere
givenataskofcontinuousmentalsubtraction.Subtractionwasfoundtobesignificantly
impairedfor15seccondsfollowingstimulation(Vlasek,1969).Forthereactiontime
tasks,therewasinsufficientdatagiventodeterminetheprecisedurationof
impairment,althoughbothwereimpairedtemporarilyfollowingstartleevent(Vlasek,
1969).Inasimilarstudy,Woodhead(1969)founddecrementsondecision-making
followingsuddennoisestimulationthatlasted,from17to31seconds.Itwouldappear
fromtheresultsofWoodhead’sstudy,andfromotherswhohaveinvestigated
performancerecoverythatmajorperformancedecrementfollowingstartleprobably
occurswithinthefirstfewseconds(Thackray&Touchstone,1970).Alesserbut
significantdecrementmaylastforperiodsfrom10to30secondsafterstartle.This
underperformancehasbeenshowninsomeaccidentstobeaperiodoftimewhere
Page 38
23
makingcorrectdecisionswerecriticaltorecovery.Interviewswithstartledpilotsand
qualitativedatainflightsimulatorexperimentssuggestthatthenegativeeffectsof
startleeffectarerealandinsomecasescanbesignificant(Martin,Murray,&Bates,
2012).
Decision-MakingModel
Thereareanumberofdecision-makingmodelsthatattempttoexplainhow
pilotsmakedecisions.Aeronauticaldecision-makingiscomplexandthereisnotalways
aclearlinkbetweenthedecisionsmadeandeventoutcome(Plant&Stanton,2013).
Schematheoryexplainshowpeopleinteractandmakedecisionsusingstoredmental
representations,andformsanintegralpartoftheperceptualcyclemodel(PCM).
Aeronauticaldecision-makingisaformofnaturalisticdecision-making(NDM);(Klein,
Calderwood,&Macgregor,1989)inwhichdecisionmakershavedomainexpertiseand
makedecisionsincontexts,whichareusuallycharacterizedbylimitedtime,goal
conflictsanddynamicconditions(Plant&Stanton,2013).Ahighproportionofpilot
errorsarerelatedtodecisionalerrors(Diehl,1991;Orasanu&Martin,1998;Shappell&
Wiegmann,2009).
Naturalisticdecisionmaking(NDM)iscomplexdueinparttotheweakly
correlatedlinkbetweeneventoutcomeandthedecisionprocess.Outcomescannot
alwaysbeusedasareliablemeanstoquantifyareasonabledecision(Orasanu&Martin,
1998).Theperceptualcyclemodel(Neisser,1976)isbasedupontheideaofareciprocal
andcyclicalrelationshipbetweentheoperatorandtheenvironment(Plant&Stanton,
Page 39
24
2013).Neisserpresentedtheviewthathumanthoughtiscloselyconnectedwitha
person’sinteractionintheworld,bothinformingtheother(Neisser,1976).
Worldknowledge(schemata)leadstotheanticipationofcertaintypesof
informationorclues.Accuratecueperceptioniscriticaltodecisionmaking.Inmost
operationalenvironments,therearemultiplecuesavailable;however,whenpilots
becomestartled,thereisatendencytoreducethenumberofcuesthataresampled
(Wickens&Flach,1998).Selectivecuesamplingcanleadtoacycleofconfusionthat
furthercomplicatesthesituation(Hilscher,Breiter,&Kochan,2012).Schematacanbe
conceptualizedashavingmental‘slots’thatareusedtostructuretheinformationlinked
tothem.Schematarepresentlinkedneuronsandmemoriesofabstractconcepts.They
aregenerallyformedfromspecificinstancesandallowabstractknowledgetobe
derivedatthetimeofretrievalbysamplingfromdomain-specificinstances(Plant&
Stanton,2013).Schemataareinternalknowledgestructuresthatarebasedonsimilar
experiencesthatcapturethecommonfeaturesofthisexperience(Lieberman,2012).
Theuseofschemataindecision-makingisadvantageous;theyactasnaturalstandard
operatingprocedures(SOPs)todirectdecisionmakerstomakeappropriateresponses
toenvironmentalstimulibasedonpreviouslysuccessfulexperiences(Plant&Stanton,
2013).
Accordingtotheperceptualcyclemodel(PCM),whenanenvironmental
experienceisencountered,relevantexperiences(schemata)areretrievedtohelp
developanappropriateresponse(Plant&Stanton,2013).Thisleadstoseekingout
certaintypesofadditionalinformationinasawayofinterpretingthatinformationusing
Page 40
25
aformofbottom-upprocessing.Theenvironmentalexperiencecanresultinthe
modificationandupdatingofcognitiveschemataandthus,inturn,influencefurther
interactionwiththeenvironment(Plant&Stanton,2013).
SmithandHancock(1995)havearguedthattheusefulnessofthePCM
explanationliesintheinteractionbetweenoperatorandenvironment,ratherthan
consideringthetwoseparately.Decisionissuesarisewhentheselectedschema(stored
andcatalogedmemories)isinappropriateforthecurrentsituation.Ingeneral,pilots
werefoundtoutilizeanumberofdifferentschemasindetermininganinitialresponse
toasituation.Theuseofschemaaidsperceptionanddecision-making(Plant&Stanton,
2013).Whentheunusualhappens,pilotstendtopaycloserattentiontoinformation
relatedtospecificcuesrelatingtotheunusualsituationinsteadofseekingout
additionalinformationtokeepthe“big-picture”inmind(Hilscher,Breiter,&Kochan,
2012).Alternativescenariointerpretationsareusuallyonlyconsideredwhentheyare
consistentwithpreexistingexpectations(Muthard&Wickens,2002).
MorrisandLeung(2006)foundthatmentalworkloadwasnotsignificantly
increased,whentaskdemandincreasedifpilotscouldreverttopre-existingschemata.
Wheninappropriateschemataareselected,incorrectactionsanddecisionscanfollow.
Over-relianceonpre-existingbutinappropriateschematahavebeenshowntoleadto
fixationoncertaincuesinrelationtoothercues(Stanton,etal.,2010;Plant&Stanton,
2013).
Page 41
26
Dual-ProcessAccountofDecisionBehavior
Inunfamiliarsituations,whenprovenrulesarenotavailable,behaviormay
becomegoal-controlledusingknowledge-basedreasoning(Rasmussen,1983).Coping
withcomplexityislargelyduetotheavailabilityofalargerepertoireofdifferentmental
representationsoftheenvironmentfromwhichrulescanbegeneratedadhoc
(Rasmussen,1983).Purposefulbehaviorisbasedonapilot’sperceptionofaneventand
isexperientialknowledgeofsimilarsituations.
Humanbehaviorcanbecharacterizedbythreelevelsofconstraintsor
performancelevels.Thelevelsmakeuseofpatternmatchingandaredefinedasskill-
based,rule-basedandknowledge-basedperformance.Skillbasedbehavioris
characterizedbysensor-motorperformanceduringactivitiesfollowingastateof
intentionandgenerallytakeplacewithoutconsciousthought.Theyareusuallysmooth,
automatedandhighlyintegrated(Rasmussen,1983).Thismodeismostlyusedforquick
andaccuratemovements.Thebodyactsasamultivariablecontinuouscontrolsystem
synchronizingmovementswiththebehavioroftheenvironment(Rasmussen,1983).
Whenasked,pilotscannotgenerallydescribetheirthoughtprocessinvolvedinthistype
ofcognition.Theyrefertoitasanautomatic-likeresponse.Thistypeofcognitionis
soughtbytrainingdepartmentsinresponsetotimecriticalaircraftemergenciessuchas
anenginefailureatrotationwhereaquick,automated,andpreciseresponseisneeded.
Attherulebasedlevel,informationistypicallyperceivedassigns,whichserveto
activateormodifypredeterminedactionsormanipulations(Rasmussen,1983).The
Page 42
27
boundarybetweenskillbasedandrulebasedperformanceisnotalwaysdistinct,and
dependsontheleveloftrainingandattentionoftheindividual(Rasmussen,1983).
Thesesignsareusedtoselectormodifytherulescontrollingthesequencingofskilled
sub-routines,andcannotbeusedforfunctionalreasoningtogeneratenewrules.
Duringunfamiliarsituationsthathavenoknownrulesforcontrol,performance
movestothehighestcognitionlevelthatisknowledgebased(Rasmussen,1983).Inthis
situation,thegoalisformulatedbasedonananalysisoftheenvironment.Thismode
canbecharacterizedbyevaluationofdifferentsolutionsandcanalsoincludetrialand
error.Figure2describesthevariouslevelsofhowRasmussendescribeshisbehavior
model.
Figure2.BehaviorModel(Rasmussen,1983)
Manydecision-processmodelsmarkthefirststepofassessingthesituationby
observinginformationanddatascanning(Salmon,etal.,2008).Thesecondpartofthe
processinvolvesexaminingpossiblesolutionsdependingontheinterpretationofthis
Page 43
28
assessment.Decisionerrorsoccurwhenthereisalackofconsiderationofimportant
datadisplays.TheperceptualstepbuildsthefoundationallevelofEndsley’sconceptof
situationawareness(Endsley,2006).
CognitionandEmotion
Emotionsareevolvedsituationresponsesthathavemultipleaspects.They
involvesubjectivefeelings,cognition,informationprocessing,expressivebehavior,
motivation,andphysiologicalresponses(Diamond&Aspinwill,2003).Infact,cognition
andemotionareintertwinedconstructs(Hilscher,Breiter,&Kochan,2012).Cognitions
thatpilotshavestoredinmemorymaynotbesufficientforexceptionalevents(Hilscher,
Breiter,&Kochan,2012).Insufficientcognitionschangespilotperceptionandasa
result,placemoreemphasisonhowpilotsperceiveandinterpreteventsbasedontheir
motivationalandbehavioralsignificance(Compton,etal,2003).Pressuressuchas
emotionalpressurescanalterrationalreasoningbyshiftingdecision-makingcriteria
fromsafetyrulestosubjectiveones(Causse,Dehais,Peran,&Pastor,2013).Emotion
andstresscanbiasdecision-makingandcognitivefunctioningparticularlyduring
complextasksthatinvolvehighercognitiveabilities(Causse,Dehais,Peran,&Pastor,
2013).Addingtothisissueisaningrainedconfidenceontheaircraft’sreliability.This
senseofsafetycanleavepilotsunpreparedforsuddenemergencies(Hilscher,Breiter,&
Kochan,2012).
Page 44
29
InflightLossofControl
Intermsofaircraftaccidents,peopleoftenincorrectlyassociatetakeoffand
landingphasestobetheareawherethehighestriskoccurs.Intotalnumbersof
accidentsandincidents,asdefinedbytheNationalTransportationSafetyBoard(NTSB),
takeoffsandlandingsarethelargestaccidentcategory.However,accordingtoBoeing
(2012),inflightlossofcontrolisthesinglelargestcategoryoffatalitiesoverthepastten
yearsaccountingfor1413fatalitiesfrom18accidents(Boeing,2012).Inflightlossof
controlaccidentshavemorefatalitiesthatbothcontrolledflightintoterrain(CFIT)and
landingaccidents.Manyoftheseinflightlossofcontrolaccidentsweretheresultofan
unusualeventatthebeginningoftheaccidentsequence.Lossofcontrolinflightcan
developrapidlyandsuddenlyfollowinginappropriatedecisionsbytheflightcrew.
Figure11belowshowsthevariousaccidentfatalitiesrankedbycategorywithinflight
lossofcontrolhavingthemost.
Page 45
30
Figure3.CausesofAccidents(Boeing,2012)
Thenextsectionsofthisliteraturereviewdiscussestwowidelyknownairline
accidentsinwhichinflightlossofcontroloccurredfollowingastartleevent.
AircraftAccidentColgan3407
Asreferencedinthelastparagraph,inflightlossofcontrolrepresentsamajority
ofairlineaccidentfatalities.Manyoftheseincidentshavebeenprecededbyastartle
event.Whenairlinecrewsarepresentedwithasuddenonsetofunusualcircumstances,
theysometimesreactcontrarytowhataregenerallyacceptedcorrectprocedures
(NTSB,2010).
Page 46
31
OnFebruary12,2009,aColganAirBombardierDHC-8-400(Q400)operatingas
ContinentalConnection3407crashedwhileonapproachtotheBuffaloInternational
Airport.All45passengers,4crewmembers,and1persononthegroundperishedasa
resultofthecrash.Theaircraftimpactedaresidentialareaapproximatelyfivenautical
milesnortheastoftheairportwhileattemptinganinstrumentapproach.Atthetimeof
theaccident,nightvisualmeteorologicalconditions(VMC)prevailedatthetime(NTSB,
2010).
TheMETARfortheairportindicatedthatthewindswerefrom240
degreesat15knotsgustingto27knots.Thevisibilitywas3milesinlight
snowandmistwithafewcloudsat1,100feet,brokencloudsat2,100
feetandovercastcloudsat2,700feet.Thetemperaturewas-1degreeC
withadewpointalsoat-1.PIREPSbothbeforeandaftertheaccident
reportedlighttomoderateicingfrom3,000to14,000feet(NTSB,2010).
TheflighthaddepartedNewarkLibertyInternationalAirportat2118EST(NTSB,
2010)forthe50-minuteflighttoBuffalo.Theflighthadbeenroutineupuntilthattime
withtheexceptionofnon-standardcommunicationduringsterileportionsoftheflight
(below10,000feet).
Whilepreparingfortheapproachat4000feet,thefirstofficeraskedthecaptain
iftheaircraftwasaccumulatingicetowhichherespondedthattherewasiceonhisside
ofthewindshield.Thefirstofficerthenresponded,“lotsofice”andthecaptainagain
commented,“that’sthemostI’veseen–mosticeI’veseenontheleadingedgesina
Page 47
32
longtime”(NTSB,2010).Airtrafficcontrol(ATC)continuedtomonitorthedescentof
theairplaneto2300feet(MSL)andat2212EST,theflightwasclearedfortheILSto
runway23.Thecrewhadtheautopilotengagedduringthisportionoftheflightandthe
airspeedwas180knots.Approximatelythreemilesfromtheoutermarker,thecaptain
begantoslowtheairplanetowarditsfinalapproachspeedbyreducingenginepower
towardsflightidle.At2216:21EST,thefirstofficerloweredthelandinggearand
selectedflapsto15degreesasrequestedbythecaptain.Theairspeedatthistimewas
145knotsanddecreasing.At2216:27(sixsecondslater)thecockpitvoicerecorder
(CVR)recordedasoundsimilartothestickshakerandtheautopilotdisconnecthorn
thatsoundeduntiltheendoftherecording(NTSB,2010).Theflightdatarecorder(FDR)
thatatthetimerecordedanairspeedof131knots.Within.5secondsoftheautopilot
disengaging,theFDRshowedthatthecontrolcolumnmovedaft(commandingapitch
up).Thepowerleverswerealsoadvancedtoabout75%torque(ameasureofengine
power).TheFDRalsoshowedthatwhilethepowerleverswerebeingadvancedthe
airplanepitchedupandrolledtotheleftapproximately45degreesandthenquickly
rolledtotheright(NTSB,2010).Concurrentwiththeroll,thestickpusheralsoactivated
(itwouldactivatetwomoretimes)attemptingtopushthenoseoftheaircraftdown.At
2216:34thefirstofficerselectedtheflapstozero(uncommentedbythecaptain),
airspeedatthattimewas100knots(NTSB,2010).FDRshowedthattherollangle
reached105degreesrightwingdownbeforetheairplaneagainbeganrollingleft.The
airplanerolledapproximately35degreestotheleftandthenbeganarapidrolltothe
rightreaching100degreesrightwingdown.At2216:50,theFDRindicatedthatthe
Page 48
33
airplanehadpitched25degreesnosedown(NTSB,2010).Impactwiththegroundwas
at2216:54.Fromtheonsetofthestickshakerwheretheairplanewasstillflyableto
impactwiththegroundwas26seconds.
Figure4.Flight3407–OneMinutefromImpact–SituationNormal(NTSB,2010)
Figure5.Flight3407–30SecondsfromImpact–StickShakerActivation(NTSB,2010)
Page 49
34
Figure6.Flight3407-27SecondsfromImpact-RolltotheLeft(NTSB,2010)
Figure7.Flight3407–21SecondsfromImpact-RolltotheRight(NTSB,2010)
Page 50
35
Figure8.Flight3407–10SecondsformImpact-FinalRolltotheRightandPitchDown(NTSB,2010)
TheaccidentwasinvestigatedbytheNationalTransportationSafetyBoard
(NTSB).Anextensivereviewofbothpilot’squalificationswasconductedbytheBoard.
Thecaptainhadexperiencedseveralunsuccessfuleventsduringhisflyingcareer
requiringadditionaltraining,however,otherpilotswhohadflownwithboththecaptain
andfirstofficerdescribedtheirperformanceas“good”(NTSB,2010).
TheNTSBinitsfindingsstatedoneoftheprimarycausesoftheaccidentwasthe
captain’sincorrectactionsinresponsetothestallwarningduringtheapproach(NTSB,
2010).Italsostatedthattheicingontheairplanewouldhaveresultedinminimal
performancedegradation(NTSB,2010).Whenthestickshakeractivated,thecaptain
respondedbyapplyinga37-poundpullforcetothecontrolcolumn,whichresultedina
noseupelevatordeflection.Theangleofattack(AOA)increasedto13degreeswitha
pitchof18degrees.Asaresultofthepowersettingsandthecaptain’sactions,the
Page 51
36
airspeeddroppedto125knots.Afterthefirststickpusheractivation,thecaptainagain
appliedanoseupforceonthecontrolcolumn(NTSB,2010).Thecaptainappliedtwo
additionalpullforcesofincreasingmagnitudeinresponsetothetwootherstickpusher
activations.TheNTSBcharacterizedthecaptain’sactionsas“abruptandinappropriate”
(NTSB,2010).
AccordingtotheNTSB,thecaptain’sperformancesuggestthathewasstartled
bytheactivationofthestickshakerandrespondedbymakinginappropriatecontrol
inputs(NTSB,2010).TheNTSBfurtherstated:
“Thecaptain’sfailuretomakeastandardcalloutorevenadeclarative
statementassociatedwitharecoveryattemptandhisfailuretosilence
theautopilotdisconnecthorn(whichcontinuedfortheremainderofthe
fightandcouldhavebeensilencedbypushingabuttononthecontrol
wheel)furthersuggestthathewasnotrespondingtothesituationusing
awell-learnedhabitpattern.Thefirstofficerwasnotprovidingguidance
consistentwithanunderstandingofthesituation(NTSB,2010)”.
AscientistattheNASA-AmesResearchforAerospaceHumanFactorsstatedthat
peopleunderstressmightnotrespondappropriatelytoeventsintheirenvironment
(NTSB,2010).Thecaptain’sresponsetothestickshakershouldnothaverequired
cognitiveefforttomakethecorrectinputsorcallouts(NTSB,2010).Inapossible
explanationtothecaptain’sresponse,theNTSBcitedColgan’strainingonicinginwhich
avideoontailplanestallswasshown(NTSB,2010).Therecoverythatthecaptain
Page 52
37
attemptedwassimilartothatwhichshouldbetakenduringatailplanestall,however,
theaircraftitselfpresentednoevidenceofsuchanevent(NTSB,2010).Itismore
probablethatinreactiontoastartleevent,thecaptainchosetheincorrectcognitive
pathwayforresolutionandwasneverabletocorrectlydiagnosethetrueissuewiththe
airplane.
AircraftAccidentAirFranceFlight447
AirFranceflight447(AF447)alsorepresentsacasewherestartleofthecrew
mayhaveadversuslyaffectedinitialdecisionmakingresultinginalossoftheaircraft
(BAE,2012).AF447wasaregularlyscheduledflightfromRiodeJaneirotoParis.AF447,
anA330,departedRioonJune1st,2009carrying216passengers,threepilots,andnine
flightattendants.RoutingoftheflightwasoverthecentralAtlanticOcean.Theflight
proceedednormallyforthefirsttwohoursandwasflyinglevelat35,000feet.
Approximately2:10:05,theaircraftencounteredfreezingprecipitationwhich
obstructedthepitotprobes(BAE,2012).Thelossofthepitotprobesaffectedthe
autoflightsystemandthecockpitairspeedindications.Thetotaltimefromonsetofthe
pitotissuetoimpactingtheoceanwas4:23.Whentheautoflightsystemdisconnected,
thepilotflying(PF)beganapplyinganoseupcommandonthesidestick.Thecockpit
speedindicationsdroppedfrom275knotsto60knots(typicalofanicingevent).At
2:10:16thepilotnotflying(PNF)stated“we’velostthespeeds”then“alternatelaw
protections”(BAE,2012).ThePFmaderapidandhighamplituderollcontrolinputs
(fromstoptostop).Healsomadeanadditionalnose-upinputthatincreasedthe
Page 53
38
airplane’spitchattitudeupto11degrees(BAE,2012).Theairplanewasinaclimb
through37,700feetatthispoint.At2:10:36theairspeedontheleftsideofthecockpit
becamevalidastheicemeltedinthepitotprobe.Airspeedatthispointwas223knots
whichrepresentedalossof50knots.ThePFreducedthepitchoftheairplane
momentarilyat2:10:47however,hethenresumedapitch-upbeyond10degreesand
theairplaneagainbegantoclimb.Thispitchcausedtheairplanesstall-warningsystem
totriggerinacontinuousmanner(BAE,2012).
ThePFselectedmaximumthrustontheenginesandmadeadditionalpitchup
inputstowards13degrees.Approximately15secondslatertherightsideairspeed
indicatorbecamevalidandrecordedanairspeedof185knots(BAE,2012).ThePF
continuedtocommandapitchupandtheairplanereachedamaximumaltitudeof
38,800feetandanangleofattack(AOA)of16degrees.At2:11:42thecaptainre-
enteredthecockpitfromarestbreak.Atthattimeallthreeairspeedindicatorswere
displayingvalidairspeeddata.Alsoaroundthistime,theairplanewasdescending
through35,000feetwithanAOAof40degrees,whichresultedinaverticalspeedof-
10,000feetperminute(fpm).Theairplanewasalsoexperiencingrolloscillations
exceeding40degrees(BAE,2012).
Duetotheextremelylowairspeed,thestallwarningceased.ThePF
momentarilyreducedthepitchoftheairplanewhichagaintriggeredthestallwarningas
theplanegainedalittlespeed.Unfortunately,thePFresumedthecommandedpitchup
andtheAOAapproached35degrees.Thelastrecordeddatafortheairplanewasat
2:14:28.Dataindicatesaverticalspeedof-10,912fpmwithanairspeedof107knots.
Page 54
39
Figure9isagraphicalreproductionoftheflightparametersfromtheflightdata
recorder.Thetimelineforthisdiagramisthefirst50secondsoftheeventwherethe
aircraftgoesfromcontrolledflighttoadescentrateofover10,000feetperminute.
Thechartshowstheaggressivehandlingbythefirstofficerinbothpitchandroll.Italso
showswhentheairspeedinformationbecamevalid.
Page 55
40
Figure9.Parametersfrom2:10:50to2:11:46(BAE,2012)
Figure10showsthattheairspeedindicationswerenormaljustpriortothe
event.Italsoshowsthattheairspeedindications(forthefirstofficer)mayhavenot
Page 56
41
beenreliableforaperiodof40seconds.Italsoshowsthatoncetheairspeed
indicationswerevalid,aconstantdecreaseinspeedoccurreduntiltheendoftheevent.
Figure10.EvolutionofAirspeedandPitotIcing(BAE,2012)
Figure11summarizestheeventfromtheonsetofthefrozenpitottubesuntil
impactwiththeocean.Largevariationsinpitchandangleofattackcanbeseen
throughoutmostoftheevent.Theaircraftiscompletelystalledforthelastminuteof
theevent,finallyimpactingtheoceanisanosehighwithalmostnoforwardairspeed
andatarateofdescentof-15,000feetperminute.
Page 57
42
Figure11.AF447FDRData(MM43,2011)
Initsfindings,theBAEstudied13incidentsrelatedtoicingandunreliable
airspeed.AirFrancehadfoursuchcasesintheirhistorywiththeA330aircraft.TheBAE
determinedthatinlessthanoneminuteaftertheautopilotdisconnected,theairplane
exiteditsflightenvelopefollowinginappropriatepilotinputs(BAE,2012).Theairplane
wentintoasustainedstallassignaledbythestallwarningsystemandstrongairframe
buffet.Eventhoughthestallwarningsoundedfor54seconds,neitherpilotmadeany
referencetothestallwarningortheassociatedbuffet.Thecrewneverappliedthestall
recoverymaneuver.Theincidentstartledthecrewandtheyhaddifficultieshandling
theairplane(BAE,2012).Theexcessivepitchandverticalspeedaddedtotheerroneous
indicationandemergency,caution,andmonitoring(ECAM)messages,whichadded
Page 58
43
complexityinthediagnosisofthesituation.Thecrewlikelyneverunderstoodthatit
wasasimplelossofairspeeddata(BAE,2012).
Conclusion
Althoughnotintendedtobeallencompassing,thisliteraturereviewseeksto
providethereaderabroadbackgroundonwhichtobasethisstudy.Thereview
discussedstartleeffectandthecurrentunderstandingofitseffectupondecision
making.Emergencies,whereflightcrewsmadetheincorrectinitialdecision,become
progressivelymoredifficulttosuccessfullymanageaspilotsoftenselectivelyfilter
informationtoconfirmtheirinitialdecision.Decisionsevolvefrompastexperiencesand
knowledgewherebitsofsimilarexperiencesarepastedtogetherinthemodel
generationphase.
Decision-makingisacomplexprocessthatresearchersareonlybeginningto
understand.Itisknownthatwhenpilotsarestartledbyasuddenemergencythattheir
decision-makingandsubsequentperformancecanbeadversuslyaffected.Thestartle
effectmayleadtoabreakdownincrewcoordinationandputsadditionalcognitiveload
ontheindividualpilot.Withover-relianceonautomation,crewsmaynotbewell
equippedtohandleasuddeninflightemergencythatrequirestheuseofhandflying
skills.Thestartleeffectmay,insomecases,resultinincorrectmodel
generation/selectionwithincorrectdecisionsand/oractionsbeingappliedtoan
emergencysituation.Boeing(2012)suggeststhatinflightlossofcontrolisthesingle
highestcategoryforairlinefatalities.Startleeffect,cognitiveoverload,andcrew
Page 59
44
breakdowncanallbeseeninboththeAirFranceandColganaccidents.Inmanycases,
simulatortraininginconjunctionwithdeliberatepracticehasbeendemonstratedto
increasecrewperformance.Deliberatepracticehasbeenwidelyappliedasanindustry
solutiontootherinflightemergenciesandapplyingitstartleeffectmaybeaneffective
waytomitigatesomeofitsinherentrisks.
Page 60
45
CHAPTERII
METHODOLOGY
Introduction
Thisstudywasamixedmethodologystudyfocusingonwhetherstartletraining
couldhelpsuccessfullymitigatethecognitivegapthatexistsduringastartleevent.The
firstpartofthestudywasasurveygiventoeachoftheparticipatingpilots.Thesecond
partofthestudyevaluatedthecrewsastheyflewoneoftheselectedmaneuversets
andwasquantitativeinnature.Thesimulatorpartofthestudywasaquasi-
experimentaldesignwithcrewsthatdidnotreceivestartletrainingservingasthe
controlgroup.Eachparticipatingcrewwasevaluatedeitheraloworhighaltitude
scenariodependingonthedayoftheweek.Randomlyselectedcrewsreceivedtraining
onhandlingtheaircraftduringastartleevent.Thetrainingconsistedofbothabriefing
andsimulatorpractice.Practiceinthesimulatorwasequalforthecrewsinthetrained
grouplastingapproximatelyonehour.Thebriefingconsistedofpersonalinstruction
usingapowerpointpresentationdiscussingtheproperpitch,power,bank,andtime
recognition(seeAppendixB).Thebriefingendedwithamnemonicdevicethatpilots
wereexpectedtouseandverbalizebothinthepracticeeventsandtheevaluation
event.Simulatorpracticeconsistedofastartleeventnotrelatedtotheevaluation
Page 61
46
profiles.Itwasintendedtohaveequallytrainedanduntrainedcrewstocompare
performancebetweenthegroups.
Thefirsttestscenariowasalowaltitudeandlowfuelprofile.Thescenario
degradeswithasystemfailurethatcausesamissedapproachandaresequencefor
landing.Timepressure,lowfuel,andtheunexpectedmissedapproachcombinetoform
thestartleeventandeventevaluationbeginsatthemissedapproach.
ThecrewflewastandardarrivalprocedureintoNewarkLibertyInternational
Airport(KEWR).Theroutingforthearrivalhadbothlateralandverticalrestrictionsand
wouldbeconsideredaroutineprocedureforapproachingtheairport.Thespecific
arrivalchosenforthisstudywastheDYLINarrival(seeAppendixD).Theweatherat
KEWRcombinedwithtrafficsaturationhascausedholding(attheMETROintersection).
Thisholdingisunexpectedbythecrewresultinginsomewhatofalowfuelsituationthat
addsaninitialstresselementtothescenario.Thecrewisclearedoutofholdingforthe
instrumentapproachtorunway4R(seeAppendixD)withapproximatelyonehourof
fuelremaining.Theexactamountoffueldependsontheaircraftinthescenario(See
AppendixD).Thevectorsandtheinitialpartoftheinstrumentapproachtorunway4R
werenormal.Theweatherwasinstrumentflightrules(IFR)witha500-footovercast
ceilingandavisibilityofonemile.
Whenthecrewselectedthelandinggeardown,aroundapproximately2000feet
ontheapproach,oneofthelandinggearfailstoextend.Itwasexpectedthatthecrew
willexecuteamissedapproachatthispointinordertotryandrectifythelandinggear
Page 62
47
issue.Thelandinggearmalfunctionisthestartleeventandtimepressureduetolow
fuelservetoaddstresstothecrew.
Thesecondtestscenariowasahighaltitudeprofile.Thecrewwasbriefedthat
theyareonaflightthatterminatesinKEWR,withroutingviatheDYLINarrival.The
flightisat35,000feet(FL350),withadescentplannedviathearrival.Therehavealso
beenreportsoflighticingdescendingintoKEWR.Pilotsareininstrumentconditions
withlightturbulence.Thescenarioinvolveslossoftheaircraft'sairdatasystem,which
disablesmanyoftheauto-flightsystems.Withtheairdataloss,anenginefirewarning
wasintroduced.Theairdatalosswasthestartleeventandtheenginefirewarning
addedadistractorandstresstothecrew.Evaluationbeganatthelossofairdata.The
airdatalossrenderstheaircraft’sspeed,altitude,andverticaltrendunreliablewiththe
sideeffectoftheautoflightsystemautomaticallydisconnecting.Theairdata
interruptionwasofshortdurationandonlyafewsecondselapsedbeforeinstrument
indicationsreturntonormal.Theautoflightdisconnectionforcesthecrewintoahand
flyingsituationandtheenginefirewarningservesasbothadistractionandastartle
event.
Datacollectionconsistedofcrewperformanceasitrelatestoaircraftcontrolfor
eachscenario.Eachscenarioismadeupoffivesub-taskswhichwereevaluatedand
usedtodetermineanoverallscore.Thescoringmethodologywastakendirectlyfrom
participatingairlines’FAAapprovedadvancedqualificationprogram(AQP)evaluation
manual.Eachpilotgroupwascomparedusingaone-wayANOVAagainsttheFAA
proficiencystandardandthencomparedtotheothergroupforsignificance.T-tests
Page 63
48
wereperformedonthedifferentmaneuversetstodetermineiftheproximityofthe
maneuveraffectsthepilot'sabilitytosuccessfullyflytheaircraft.Ifthestudy
hypothesisiscorrect,theuntrainedcrewsshouldshowasignificantstatisticaldifference
ascomparedtothestandardpilotperformanceasdefinedbytheFAA.Inaddition,
regressionanalysiswasperformedonvariousaspectsofthecollecteddatatogainan
insightastowherethevariabilitylies.
Subjects
ThepopulationforthisstudywereprofessionalpilotsofanFAR121commercial
aircarrier.Furthermore,thestudyfocusedonpilotsofscheduledpassengerairlines.
Flightcrewsfromtheparticipatingairlinewereaskedduringtheirrecurrenttraining
cycleiftheywishedtoparticipateinthisstudy.Theywereselectedbasedontheir
willingnesstovolunteerforthisstudy.Selectionofcrewsgenerallyoccurredduring
theirfinaldayoftrainingwhentherewasoftenextrasimulatortimeavailable.Ifthe
crewvolunteered,theywererequestedtofilloutasurveyontheirexperienceand
perceptionsoftheirflyingabilities,especiallyduringunusualevents.Thecrewswere
informedthattherewasnopersonaldatakeptandnodatalinkinganindividualtotheir
performance.
BetaTesting
Asmallgrouptestwascompletedonapproximately15subjects.Thenatureof
thebetatestwastodetermineanestimatedeffectsizeoftheindependentvariable.
Theeffectsizewasusedtocompleteapoweranalysis.Inaddition,thebetatestgroup
Page 64
49
wasusedtoestablishthesimulatorparametersforitemssuchasweight,fuel,and
inducedproblems.Thebetatestresultsarereportedhereandnotinthefindings
sectionastheyarenotconsideredpartoftheactualtestdata.IBMSPSSStatistics
(SPSS)wasusedtoanalyzetheoveralltrainingeffectandthenforeachscenario(low
andhigh).Tablesoftheinitialresultsarelistedbelow.Table1liststhebasic
descriptives.Atotalof15crewsparticipatedinthebetatest.
Table1.DescriptiveStatisticsBetaGroup
DependentVariable:OverallscorecollapsedacrosslowandhighaltitudeCrewtrainingprovided
Highorlowaltitudescenario
Mean Std.Deviation N
Yes HighAltitude 4.0000 .00000 5LowAltitude 4.5000 .57735 4Total 4.2222 .44096 9
No HighAltitude 2.3333 .57735 3LowAltitude 2.6667 .57735 3Total 2.5000 .54772 6
Total HighAltitude 3.3750 .91613 8LowAltitude 3.7143 1.11270 7Total 3.5333 .99043 15
Thebetatestfoundthatthetrainingeffectwassignificantwithp=.00whenthelow
andhighaltitudegroups(trainedanduntrained)wherecompared.Theresultsarelisted
intable2.
Page 65
50
Table2.TestsofBetween-SubjectsEffectsBetaGroup
Table3belowcomparesthetrainedanduntrainedgroupsforonlythelowaltitude
scenario.ThedescriptivestaticsandANOVAresultsareinTables3and4.
Table3.DescriptiveStatisticsLowAltitudeScenarioBetaGroup
DependentVariable:LowAltitudeScenarioCrewtrainingprovided Mean Std.Deviation NYes 4.5000 .57735 4No 2.6667 .57735 3Total 3.7143 1.11270 7
DependentVariable:OverallscorecollapsedacrosslowandhighaltitudeSource TypeIIISum
ofSquaresdf Mean
SquareF Sig. PartialEta
SquaredNoncent.Parameter
ObservedPowerb
CorrectedModel
11.400a 3 3.800 17.914 .000 .830 53.743 1.000
Intercept 163.209 1 163.209 769.414 .000 .986 769.414 1.000Training 10.970 1 10.970 51.716 .000 .825 51.716 1.000HighLow .622 1 .622 2.932 .115 .210 2.932 .346TrainingHighLow
.025 1 .025 .117 .738 .011 .117 .061
Error 2.333 11 .212
Total 201.000 15
CorrectedTotal
13.733 14
a.RSquared=.830(AdjustedRSquared=.784)b.Computedusingalpha=.05
Page 66
51
Table4.TestsofBetween-SubjectsEffectsLowAltitudeScenarioLowAltitudeBetaGroup
DependentVariable:LowAltitudeScenarioSource TypeIIISum
ofSquaresdf Mean
SquareF Sig. PartialEta
SquaredNoncent.Parameter
ObservedPowerb
CorrectedModel
5.762a 1 5.762 17.286 .009 .776 17.286 .908
Intercept 88.048 1 88.048 264.143 .000 .981 264.143 1.000Training 5.762 1 5.762 17.286 .009 .776 17.286 .908Error 1.667 5 .333
Total 104.000 7
CorrectedTotal
7.429 6
a.RSquared=.776(AdjustedRSquared=.731)b.Computedusingalpha=.05
Thesamecomparisonwasdoneonthebetagroupforthehighaltitudescenario.The
resultsarelistedinTables5and6.Likethelowaltitudebetagroup,themaineffectof
trainingshowedsignificance.
Table5.DescriptiveStatisticsHighAltitude-BetaGroup
DependentVariable:HighAltitudeScenarioCrewtrainingprovided Mean Std.Deviation NYes 4.0000 .00000 5No 2.3333 .57735 3Total 3.3750 .91613 8
Page 67
52
Table6.TestsofBetween-SubjectsEffectsHighAltitude-BetaGroup
DependentVariable:HighAltitudeScenarioSource TypeIIISum
ofSquaresdf Mean
SquareF Sig. PartialEta
SquaredNoncent.Parameter
ObservedPowerb
CorrectedModel
5.208a 1 5.208 46.875 .000 .887 46.875 1.000
Intercept 75.208 1 75.208 676.875 .000 .991 676.875 1.000Training 5.208 1 5.208 46.875 .000 .887 46.875 1.000Error .667 6 .111
Total 97.000 8
CorrectedTotal
5.875 7
a.RSquared=.887(AdjustedRSquared=.868)b.Computedusingalpha=.05
Analysissuggestthatbetween73%and86%ofthevariabilitycanbeaccounted
forduetothetrainingeffect.G-PowerwasthenusedtocalculateaCohen’sDto
estimatethetrainingeffectsize.Thecalculatedeffectsizewas.33whichreflectsa
mediumeffectsize.Thegiveneffectsizewasthenusedtoestimatethenumberof
subjectsneedstoensureanadequatesamplesizewhichwiththecalculatedeffectsize
isbetween17and60crews.
Table7.Cohen’sDCalculationBetaGroup
Scenario LowAltitude HighAltitudeMean 3.167 3.375Std.Deviation 0.916 1.112SampleSize 7.0 8.0Result Cohen'sd=(3.714-3.375)⁄1.018725=0.332769.Cohen'sd=(M2-M1)⁄SDpooledSDpooled=√((SD12+SD22)⁄2)
Page 68
53
SampleGroups
Theplanwastoevaluate30-60crews.Thisallowedforenoughdatatobe
collectedeveniftheeffectsizeintheratingscaleissmalltomedium
(Cohen’sd=.33/r=.19).Thenumberofpilotswerechoseninordertogaina
statisticallysignificantsampleapproximatingtheskilllevelofthegeneralprofessional
pilotpopulation.Therewerefourmaingroupsforcomparison.Thegroupswere
definedasfollows:
1. LowAltitude–NoTraining(LANT)
2. LowAltitude–Training(LAT)
3. HighAltitude–NoTraining(HANT)
4. HighAltitude–Training(HAT)
Equipment
ThisstudyusedprofessionalairlinepilotsflyingtwodifferentscenariosinanFAA
approvedLevel-Dfullflightsimulator(FFS).SimulatorsthatwereutilizedincludeA320,
B737,B757,B767,B777,B787,andB747.Thescenarioswereflownbyacrewconsisting
ofacaptainandfirstofficer,similartowhatwouldhappeninactuallineoperations.
Thetrainingthatsomeofthecrewsreceivedwasbroad-basedandnotaircraftspecific.
DataCollectionMethods/Procedures
Datacollectionforthisstudyfocusedontwoparts:asurveyandobserved
simulatorperformancedata.Thesurveyconsistedofapproximately10multiple-choice
Page 69
54
questionsthatwereevaluatedusingaLickertscale.Questionsfocusonsuchitemsas
outsideflying,previousmilitaryand/oraerobaticflying,andgeneralhandflying
attitudes.
ThesimulatorportionofthestudyinvolvedflyingmaneuverprofilesinanFAA
approvedLevel-Dfullflightsimulator.Themaneuversetswereevaluatedbythe
principleinvestigator.Theinvestigatorisaformerinstructorpilotonmultipletransport
categoryjetaircraftwithover11yearsofprofessionalinstructionalexperience.The
maneuverevaluationcriteriaweredevelopedinaccordancewithairlinetraining
proceduresandprotocolsassetforthbytheFAA.Thespecificevaluationcriteriawere
adoptedfromanFAR121passengerAdvancedQualificationProgram(AQP)program
(withpermission),whichwasapprovedbytheFAA.Thecriteriamatchcloselywith
evaluationstandardssetforthbytheFAAintheAirlineTransportPilot(ATP)practical
teststandards(PTS).Thecrewswereevaluatedonthesuccessofthemaneuveras
describedinTables8-10.
Page 70
55
Table8.HighAltitudeAnalysis
Table9.LowAltitudeAnalysis
Score Criteria
5Thecrewremainedwellwithinstandardsandperformancewasexemplary.Thecrewrecognizedtheissueandhandledpromptlywhilerecognizingthedeterioratingfuelstateoftheaircraft.
4 Thesituationwaswellhandledwiththesafetyoftheflightnotinjepordy.Thecrewwasawareofthetimepressuresandthefuelstateandmitigatedboth.
3 TheflightlandedsafelywithnomajordeviationsforSOPswithatleast30minutesoffuel.
2 Landedtheaircraftinlessthandesirableconditionswithregardstoconfiguration,fuelandtimemanagement.
1Thepilotcommittedmajordeviationsfromstandardsthatwerenotpromptlycorrectedand/orwereunsafe,orwasunabletoperformthemaneuver/taskwithoutassistance.Thepilotcrashedorlostcontroloftheaircraft.
Score Criteria
5Crewperformancewasexcellentinbothaircrafthandlingandproblemdiagnosis.Thecrewhadminimalaltitudeandheadingchanges,recognizedtheissuepromptlyandappliedthecorrectmitigationstrategies.
4 Crewperformancewasgood.Problemwascorrectlydiagnosedwithpitchandrollnotexceeding5deg/50feet.
3 Crewperformancewasaverage.Somedifficultydiagnosingtheproblem.Pitchandrollnotexceeding10deg/100feet.
2Crewperformancewasbelowaverage.Problemsand/orconfusiondiagnosingtheproblemormisdiagnosedoftheproblem.Majordeviationsinpitchandrollmorethan20deg/200feet.
1 Crewperformancewasunacceptable.Thecrewcouldnotdiagnosetheproblemandmisdiagnosedtheproblem.Excessivedeviationsandhandlingoftheaircraft.
Page 71
56
Table10.EvaluatedFactorsandSeatPositions
HighAltitude LowAltitude
Factors ProblemDiagnosis MissedApproach
Pitch IrregularChecklists
Roll TimeManagement
AltitudeControl FuelManagement
OverallControl ApproachandLanding
OverallScore OverallScore
SeatPosition PilotFlying PilotFlying
PilotMonitoring PilotMonitoring
DataAnalysisandStatisticalModeling
Analysiswasplannedforwithingroupsandbetweengroupscomparisonswith
furtheranalysisofsignificantfactors.Anadditionalanalysiscomparedthetraining
groupsandnon-traininggroupsversustheFAAstandard.Overallfactorandseat
positionscoreswerealsocomparedtotheFAAstandard,whichforthisstudywasa
gradeofthreeasdescribedaboveinthedatacollectionsection.Thestatistical
modelingprogramSPSSwasprimarilyusedtoanalyzethedata.Inaddition,thesurvey
datawascomparedandanalyzedtoseeifanysignificantcorrelationscanbe
determined.ANOVA,linearregression,andpost-t-testsweretheprimarystatistical
modelsused.
Page 72
57
WithinGroupComparison
Thefirstdatasetforanalysiswerethewithingroupcomparisons.Analysiswas
conductedbetweenLowAltitudeTrainedGroup(LAT)andLowAltitudeNon-Trained
Group(LANT)andtheHighAltitudeTrainedGroup(HAT)andtheHighAltitudeNon-
TrainedGroup(HANT).Thefirstpartoftheanalysisusedtheoverallgradescore(see
Appendix1).Analysislookedforsignificantfindingswithineachgroupusingaone-way
ANOVAwiththealphalevelsetat.05.Comparisonbetweentheoverallgradeandthe
FAAstandardgrade(3)wasalsocompared.Asecondroundofanalysisoccurredforthe
contributingfactorsoftheoverallgrade.Linearregressionwasusedtodeterminewhat
partofthevarianceeachfactor(ifany)aresignificant.Againthealphalevelwas.05.
Finally,aspartofthewithingroupscomparison,theseatposition(captainorfirst
officer)wastestedforsignificanceasacontributortotheoverallgradeusingregression.
BetweenGroupComparison
Thebetween-groupcomparisonwassimilartothewithingroupcomparison
usingthesametestsandtools.ThebetweengroupswasbetweenLATandHAT
followedbyLANTandHANTgroups.Theanalysissoughttodeterminesignificant
findingsofthefinalgrade,usingaone-wayANOVAwithanalphalevelof.05.Aswith
thewithingroupcomparison,linearregressionwasusedtoanalyzeboththe
contributingfactorsandtheseatposition.ComparisontotheFAAstandardwas
conductedinthebetweengroupcomparison.
Page 73
58
ComparisonversusaKnownStandard
Thefinalsetofcomparisonswasconductedbycollapsingacrossgroups(trained
versusuntrained,andthencomparingagainstaknownsetstandard.Thesetstandard
wasdeterminedbythetolerancessetforthbytheFAAforanAirlineTransportPilot
(ATP)certificate.Thedatacollectionsectionabovedescribesthegradingstandardsas
setbytheFAAandairlinepolicies,whichsincetheyareapprovedarealsopartofthe
FAAstandard.Crewsreceiving,atleast,anoverallgradeof(3)wereconsideredtomeet
theFAAstandard.Anygradebelow(3)wasconsideredbelowstandard.
Thetraininggroups(LATandHAT)andnon-traininggroups(LANTandHANT)
werecollapsedandthencomparedtotheFAAstandardusingaone-wayANOVAwith
analphalevelof.05.Comparisonofthecollapsedgroupswasalsocomparedtothe
surveyresponsesusingthePearsonCorrelationtest.
ProtectionofHumanSubjects
Thecrewsinthisstudywereexposedtounusualbutnotextraordinaryaircraft
failuresinthesimulator.Thesefailuresareregularlypracticedduringinitialand
recurrenttraining.Thecrewsareaccustomedtohavingtheirperformanceevaluated.
Thereisalsoaminimalriskofperforminginfrontofacolleagueiftheperformanceis
substandardeventhoughtheevaluationwillbeasawholecrew.Thisismitigatedby
notidentifyingaparticularpilotwithanindividualperformance.Thevolunteersubjects
werealsoencouragednottoevaluateeachother’scapabilitybasedonthistesting
scenario.Finally,crewswillberequestedtorefrainfromtalkingaboutthetestingwith
Page 74
59
othercrewsastonotcompromisethetestdata.Therewillbenoidentifiablelink
betweentheperformanceofthecrewandanyindividualcrewmember(aconditionset
bytheparticipatingairlineingrantinguseoftheirsimulators).Individualperformance
willnotbereportedtoanyairlineorentityoutsidetheresearchproject.
Sinceminimalriskswillbeinvolvedinthestudy,thesubjectswillbeinformed
thatthestudywillincludevariousmaneuversthatwillbeflowninthesimulatorand
thatthecrew'sperformancewillbeanalyzed.Therewillbeacheckboxonthesurvey
formindicatingthattheparticipantshavevolunteeredforthestudy.Thespecific
languageisasfollows:“Bycheckingthisbox,IagreethatIhavevolunteeredforthis
studyandhavefeltnounduepressurefromtheairline,theUniversityofNorthDakota,
ortheprincipleinvestigatortoparticipate.Ihavealsobeeninformedthatnodatawill
bekeptlinkinganysimulatorperformancetoaspecificpilot.Datacollectedisforthis
researchprojectonlyandwillnotbereportedtoanyentityorairline.Finalaggregate
resultsmaybeviewedinthepublisheddissertationthatwillbeavailableatthe
UniversityofNorthDakotaChesterFritzLibrary.TheresearcherhasinformedmethatI
willflyaspartofacrewandmayencountersomeunusualsituationsinthesimulator.I
alsounderstandthatIhavetherighttorefuseparticipationorwithdrawfromthestudy
atanypointwithoutachangeinrelationshipwithmyairline,theUniversityofNorth
Dakotaortheresearchteam.”Allparticipantsacknowledgedandsignedthe
participationconsentform.
Page 75
60
CHAPTERIV
RESULTS
Thisstudyconsistedoftwomainparts,asurveyandaflightevaluation.There
wasdatarecordedinanFAAapprovedLevel-Dflightsimulator,flownbypilotsfora
majorUSbasedpassengerairline.Volunteercrewswereaskedtoflyoneoftwo
differentscenarioprofiles.Randomcrewsreceivedtrainingthatconsistedofabriefing
andsimulatorpractice.Thetrainingsoughttomitigatethenegativecognitiveeffects
followingastartleevent.Thedatamainlyfocusesontheeffectthetraininghadonthe
trainedpilotgroup.Analysisconsistedofbothwithinandbetweenmaingroupswith
regressionanalysisonthecontributingfactorsthatmadeupthemaneuversetscores.
Crewswerepresentedwitheitheralowaltitudeandlowfuelscenarioorahighattitude
scenariowithalossofairdata.Thesurveywasconductedinordertogainaperspective
intohowpilotsatmajorairlinesflytheiraircraft,andhowtheyperceivetheirownflying
skills.
Demographics
FortycrewswhoflewforaU.S.Globalpassengerairlineparticipatedinthe
study.Allofthesubjectswereactivelinepilotsandvolunteers.Thepilotsflewasa
crewconsistingofaCaptainandFirstOfficerandhadflownintheirrespectiveaircraft
Page 76
61
foratleastoneyear.Crewswerealsodividedbywhichprofiletheyflewandwhether
theyreceivedtrainingandpracticepriortoflyingtheprofilescenario.Eachscenario
(loworhigh)wasflownby20crews.Inaddition,crewswereseparatedbywhattypeof
aircraftthattheyflew.Therewere21wide-bodyaircraftcrews(B747,B787,B777,
B767)and19narrow-bodyaircraftcrews(B737,A320,B757).
SurveyResponses
Thesurveywasdividedintotwodistinctparts:apilot’sexperience,andtheir
perceptionoftheirownskills.Thepilotflying(PF)thescenariowasaskedtocomplete
thesurvey.Thefirstsurveyquestionaskedifthepilotflewoutsideoftheircurrentjob
inanotherprofessionalmannersuchasflightinstructing.Theresultsaredisplayedin
Table11andFigure12
Table11.FlyingOutsideofProfessionalJob
Frequency Percent ValidPercent
CumulativePercent
Valid Yes 6 15.0 15.0 15.0
No 34 85.0 85.0 100.0
Total 40 100.0 100.0
Page 77
62
Figure12.OutsideFlying
Surveyresponsesindicatedthatonly15%ofthepilotsflewoutsideofthe
currentjob.Outsideflyinggenerallyconsistsoflesssophisticatedaircraftthatrequire
moreroutineflyingskillpractice.Sinceonly15%indicatedthattheyflewoutsideof
theirairlinejob,theresultswerenotconsideredsignificant.
Thenextsurveyquestionaskedwhetherthepilotflewinthemilitary.Ofthe
surveyresponses,32.5%indicatedthattheyhaveflownintheUnitedStatesmilitary.
TheresultsaredisplayedinTable12andFigure13
Page 78
63
Table12..DidYouFlyintheMilitary?
Frequency Percent ValidPercent
CumulativePercent
Valid Yes 13 32.5 32.5 32.5
No 27 67.5 67.5 100.0
Total 40 100.0 100.0
Figure13.CivilianversusMilitaryFlying
Anecdotalpersonalinterviewswithmilitarypilotssuggestedthatthoseexposed
tomilitaryflyinghaveagreaterexposuretostartleeventsandinsomecaseshave
developedcopingmechanisms.
Thenextsurveyquestionaskedifthepilothadanytypeofformalaerobatic
training.Thistypeoftrainingmayindicatebetterrecognitionofunusualattitudesand
Page 79
64
leadtoamoreeffectiveresponsetoanunusualevent.Theresultsaredisplayedin
Table13andFigure14.
Table13.DoYouHaveanyFormalAerobaticTraining
Frequency Percent ValidPercent
CumulativePercent
Valid Yes 23 57.5 57.5 57.5
No 17 42.5 42.5 100.0
Total 40 100.0 100.0
Morethan57%ofthepilotsindicatedthattheyhadreceivedsometypeofformal
aerobatictraining.
Figure14.AerobaticTraining
Thefinalsurveyquestionrelatedtopilotexperience,askedwhetherthepilots
hadeverencounteredanunusualeventthattheywoulddescribeas“startling”.This
Page 80
65
questionwasaskedtogagehowmanypilotshaveexperiencedevents(whileflying)that
caughtthembysurprise.TheresultsaredisplayedinTable14andFigure15.
Table14.StartleEvents
Frequency Percent ValidPercent
CumulativePercent
Valid Yes 32 80.0 80.0 80.0
No 8 20.0 20.0 100.0
Total 40 100.0 100.0
Amajorityofthepilots(80%)indicatedthattheyhadbeenstartledwhileflying,leading
totheconclusionthatstartleissomewhatcommonamongprofessionalfightcrews.
Figure15.StartlingEvents
Thenextsectionofthesurveysoughttogainaperspectiveonhowpilots
generallyflewtheaircraftinnormallineoperations.Thissectionofthesurveyasked
Page 81
66
thepilotstoratethequestionsbasedonaslidingscaleofagreement(1-5)from
“stronglyagree”to“stronglydisagree”.Thepilotswereaskedtoselecttheiragreement
withthesurveyquestions.Therewasnooptiontoselectresponsesoutsideofthefive
standardones.
Thefirststatementaskedwhetherapilotknowstheproperpitchandpower
settingsforphasesofflightsuchascruiseandapproach.Thisisimportantbecauseif
variousflightinstrumentsarelost,safeflightcanbecontinuedwithjustapitchand
powersetting.Nopilotsdisagreedwiththisstatementwith50%stronglyagreeingthat
theyknewthecorrectpitchandpowersettings.TheresultsaredescribedinTable15
andFigure16.
Table15.IKnowtheProperPitchandPowerSettings
Frequency Percent ValidPercent
CumulativePercent
Valid StronglyAgree 20 50.0 50.0 50.0
SomewhatAgree
15 37.5 37.5 87.5
Neutral 5 12.5 12.5 100.0
Total 40 100.0 100.0
Page 82
67
Figure16.PitchandPowerSettings
Whencombinedwith“somewhatagree”,positiveresponsesprovidedbythe
pilotsrecordedat87%.Noneofthepilotsindicatedthattheydisagreedwiththe
statement.Responsesindicatethatamajorityofpilotsbelievethattheyknowthe
correctpitchandpowersettingsforthephaseofflight.
Thenextsurveyquestionaskedifthepilotsoftenhand-flewtheaircraftduring
departureandapproachbelow10,000feet.Thesephasesofflightoftencontainthe
mostcomplexaircraftmaneuvering.Changesinaltitude,speed,andcourseareroutine
inthesephases.Departuresinvolvedchangesinroutingwhileclimbingandconfiguring
theaircraftforhighspeedflight.Arrivalsinvolveasimilarsequenceonlyinreversus
order.TheresultsaredisplayedinTable16andFigure17.
Page 83
68
Table16.HandFlyingBelow10,000Feet
Frequency Percent ValidPercent
CumulativePercent
Valid StronglyAgree 27 67.5 67.5 67.5
SomewhatAgree
9 22.5 22.5 90.0
SomewhatDisagree
4 10.0 10.0 100.0
Total 40 100.0 100.0
Responsestothisquestionwereindicatethat67.5%stronglyagreedwiththis
statementand22.5%somewhatagreed.Only10%ofthepilotsdisagreedwiththe
statement.Therewasnoneutralorstronglydisagreestatements.Thisindicatesthat
mostpilotsarehandflyingtheaircraftbelow10,000feet.
Figure17.HandFlyingBelow10,000Feet
Page 84
69
Mentalrehearsalofdifferentflightscenarioshasbeenfoundtohelpfulin
shapingpilotresponsestounusualsituations.Atthemajorairlinestudiedforthis
research,pilotsarerequiredtoviewandparticipateinorganized“chairflying”during
therecurrenttrainingcycle.Thenextsurveyquestionaskedwhetherthepilots
extendedthispracticeoutsideoftheirrecurrenttrainingcycle.Theresultsaredisplayed
inTable17andFigure18.
Table17.ChairFlyScenariostoHelpDetermineCoursesofAction
Frequency Percent ValidPercent
CumulativePercent
Valid StronglyAgree 6 15.0 15.0 15.0
SomewhatAgree
17 42.5 42.5 57.5
Neutral 11 27.5 27.5 85.0
SomewhatDisagree
3 7.5 7.5 92.5
StronglyDisagree
3 7.5 7.5 100.0
Total 40 100.0 100.0
Thepilotsindicatedthattheysomewhatagreedtothisstatement42%ofthe
time.Thenextlargestgroupwasneutralrepresenting27.5%oftheresponders.
Page 85
70
Figure18.ChairFlying
Thestatementrecordedresponsesinallcategories.Responsesindicatedthata
57.5%ofthepilotsusethispractice.
Beingabletoflytheairplanewithoutadvancedautomationhasbeenshownasa
keyelementinrecoveringfromanunusualsituation.Whenanaircraftisupset(outside
ofthenormalflightenvelope),theautomationwilloftendisconnect(UnitedAirlines,
2016).Thissurveyquestionaskedifthepilotswerecomfortableflyingtheaircraft
withouttheuseoftheflightdirector,autothrottles,andmapmode.Theresultsare
displayedinTable18andFigure19.
Page 86
71
Table18.ComfortFlyingRawData
Frequency Percent ValidPercent
CumulativePercent
Valid StronglyAgree 17 42.5 42.5 42.5
SomewhatAgree
15 37.5 37.5 80.0
Neutral 3 7.5 7.5 87.5
SomewhatDisagree
5 12.5 12.5 100.0
Total 40 100.0 100.0
Therewerenopilotswhostronglydisagreedwiththisstatement.Pilotsselecting
stronglyagreeandsomewhatagreewere80%oftheresponses.
Figure19.RawDataFlying
Page 87
72
Theseresponsesindicatethatmostpilotsfeelthattheyarecomfortableflying
theaircraftwithrawdataonly.
Handflyingtheairplaneduringthedayingoodweatheriscommon,however
deliberatepracticeinconditionsotherthandayvisualflightrules(VFR)isimportantin
maintainingflyingskills.Thenextsurveyquestionsaskedpilotsiftheyhandflewin
variousconditions.TheresultsaredisplayedinTable19andFigure20.
Table19.OftenPracticeRawDataSkills
Frequency Percent ValidPercent
CumulativePercent
Valid StronglyAgree
22 55.0 55.0 55.0
SomewhatAgree
10 25.0 25.0 80.0
Neutral 7 17.5 17.5 97.5
SomewhatDisagree
1 2.5 2.5 100.0
Total 40 100.0 100.0
Ofthepilotssurveyed,80%eitherstronglyagreedorsomewhatagreedwiththe
statement.Alargernumberofpilotsindicatedthattheywereneutral17.5%withthis
statementwhencomparedtotheotherquestions.
Page 88
73
Figure20.SkillsPractice
Theresponsesindicatethatmostofthepilotspracticehandflyingundervarious
conditions.
Thefinalquestionaskedifthepilotsusedtheautopilotforamajorityofthe
flightabove1000feet.Thisquestionisincontrasttothe10,000-footaltitudehand
flyingquestionandsoughttodeterminewhatpercentageofpilotspredominatelyuse
theautopilotforflight.TheresultsaredisplayedinTable20andFigure21.
Page 89
74
Table20.AutopilotUsageAbove1000Feet
Frequency Percent ValidPercent
CumulativePercent
Valid 1 2.5 2.5 2.5
StronglyAgree
8 20.0 20.0 22.5
SomewhatAgree
5 12.5 12.5 35.0
Neutral 2 5.0 5.0 40.0
SomewhatDisagree
13 32.5 32.5 72.5
StronglyDisagree
11 27.5 27.5 100.0
Total 40 100.0 100.0
Thisstatementhadthemostvariedresponseswithaslightmajority(56%)ofthe
pilotseitherdisagreeingorstronglydisagreeingwiththestatement.Pilotsagreedwith
thestatement32.5%ofthetime.
Page 90
75
Figure21.AutopilotUsage
FlightEvaluation-QuantitativeAnalysis
AquantitativeanalysiswascompletedusingSPSSonthehighandlowaltitude
profilesandthesub-factorsthatcomprisedeachprofile.Theprofileswereanalyzed
bothindividuallyandthencollapsedtogetherwiththeindependentvariablebeing
trainingasdescribedintheMethodsSection.Eachofthethreegroups(high,low,and
combined)wereanalyzedusingaone-wayANOVA.Inadditiontodescriptivestatistics,
regressionanalysiswasconductedonthefactorsthatmadeupeachindividualscenario
score.Finally,thecombinedgroupwascomparedtotheFAAstandardsforAirline
TransportPilot(ATP)certification.Theresultsarediscussedbelow.
Page 91
76
HighAltitude
Crewsflewthehighaltitudescenario20times.Therewerenineuntrainedcrews
and11crewsreceivedthetrainingasdescribedintheMethodsSection.SPSSwasused
tomodelaone-wayANOVAtestingfortheeffectsoftraining(independentvariable)on
theoverallscenarioeventscore.ThedescriptivestatisticsaresummarizedinTable21.
Table21.DescriptiveStatisticsHighAltitude
HighAltitudeScenario
N Mean Std.Deviation
Std.Error
95%ConfidenceIntervalforMean
Min-imum
Max-imum
Between-Compon-entVarianceLower
BoundUpperBound
No 9 2.666 .70711 .23570 2.1231 3.2102 2.00 4.00
Yes 11 3.727 .64667 .19498 3.2928 4.1617 2.00 4.00
Total 20 3.250 .85070 .19022 2.8519 3.6481 2.00 4.00
Mod-el
FixedEffects
.67420 .15076 2.9333 3.5667
RandomEffects
.53252 -3.5163 10.0163 .51653
Theuntrainedcrewshadameanscoreof2.67andastandarddeviationof.70,
whichisslightlybelowthestandardforFAAcertification(ascoreof3),whilethetrained
crewshadameanscoreof3.72andastandarddeviationof.65whichisabovetheFAA
standard.Therewerenocrewsthatreceivedascoreofone(1)indicatinglossofcontrol
oftheaircraft.Therewerealsonoscoresoffive(5)indicatinganearperfectlyflown
scenarioset.
Page 92
77
ALevene’stestwasconductedonthehighaltitudescenario.Thetestindicates
thattherewasnosignificantdifferenceinthetrainedanduntrainedgroupvariances.
ThetestresultswereF(1,18)=.674,p=.422.SeeAppendixEforadditionaldetailed
statisticaltestresults.
Aone-wayANOVAwasconductedcomparingthetrainedanduntrainedgroups
(seeTable22).Therewasasignificanteffectoftrainingonthehighaltitudescenario
score,F(1,18)=12.25,p=.003.Thetestconfirmstheresearchhypothesisthat
targetedtrainingcanbesuccessfulinhelpingpilotsmaintainaircraftcontrolduringan
unusualandsuddenstartleevent.Inthecaseofthehighaltitudescenario,thenull
hypothesisisrejected.
Page 93
78
Table22.TestsofBetween-SubjectsEffectsHighAltitude
DependentVariable:HighAltitudeScenario
Source TypeIIISumofSquares
df MeanSquare
F Sig. PartialEtaSquared
Noncent.Parameter
ObservedPowerb
CorrectedModel
5.568a 1 5.568 12.250 .003 .405 12.250 .911
Intercept 202.368 1 202.368 445.210 .000 .961 445.210 1.000
CrewTrng 5.568 1 5.568 12.250 .003 .405 12.250 .911
Error 8.182 18 .455
Total 225.000 20
CorrectedTotal
13.750 19
a.RSquared=.405(AdjustedRSquared=.372)b.Computedusingalpha=.05
TheeffectsizewascalculatedfromtheANOVAresultsusingtheformula
R2=SSM/SSTwiththeresultbeingn=.64.ThisrepresentsalargeeffectsizeR2=h2
=5.568/13.750Etah=.64.Theresultindicatesthattheaveragepersoninthe
experimentalgroupwouldscorehigherthan73%ofacontrolgroupthatwasinitially
equivalent(Coe,R.,2002).
RegressionHighAltitude
Regressionanalysiswasconductedonthefactorsthatmadeuptheoverall
scenarioeventscore.Thiswasdonetoexploreanysignificantfactorsleadingtothe
overallscoreandtopinpointpossibleareasoffuturetraining.Thefactorsare
summarizedinthetablebelow.Eachfactorwasevaluated20times.
Page 94
79
Table23.DescriptiveStatisticsRegressionHighAltitude
Mean Std.Deviation N
HighAltitudeScenario 3.2500 .85070 20
CrewTrainingReceived .5500 .51042 20
Problemdiagnosis 3.6000 1.14248 20
Pitchcontrol 3.5000 1.10024 20
Rollcontrol 3.8500 .81273 20
Altitudecontrol 3.3500 .93330 20
Theregressionanalysiswasconductedintwoblockswithblockonebeingcrew
trainingandblocktwoconsistingofthedescribedfactorsintheprevioustable.The
ANOVAindicatesthatbothmodelssignificantlyimprovetheabilitytopredictthe
outcomevariablecomparedtonotfittingthemodel.Model1hadF(1,18)=12.25,p=
.003andModel2hadF(5,14)=10.02,p=.00.SeeAppendixE.
Theregressioncoefficientswerealsoanalyzedtodeterminewhichfactors(other
thancrewtraininginmodel1)showedsignificance.Significancewasnotedforthe
factorofproblemdiagnosisp=.00.Theotherfactorsdidnotshowsignificance(p>
.05).Collinearityanalysisindicatesthattherearenoexamplesofmulticollinearity(VIF>
10).SeeAppendixE
LowAltitudeScenario
Thelowaltitude,lowfuelscenariowasflownby20crews.Therewere10
trainedand10untrainedcrewsasdescribedinthemethodssection.SPSSwasusedto
Page 95
80
analyzetheresultsusingaone-wayANOVA.Themeanscoreoftheuntrainedcrews
was2.60withastandarddeviationof.70andthetrainedcrewswas3.70witha
standarddeviationof.82.Aswiththehighaltitudescenario,theuntrainedcrews
performedbelowtheATPstandardsandthetrainedgroupperformedabovethe
standard.Therewerenoscoresofone(1)whichwouldindicatealossofcontrolor
crashoftheaircraft.Therewerescoresoffive(5)indicatingperformancewellabove
theFAAcertificationstandard.ThedescriptivestatisticsaresummarizedintheTable24
below.
Table24.DescriptivesLowAltitude
LowAltitudeScenario
N Mean Std.Deviation
Std.Error
95%ConfidenceIntervalforMean
Mini-mum
Maxi-mum
Between-Compon-entVarianceLower
BoundUpperBound
No 10 2.600 .69921 .22111 2.0998 3.1002 2.00 4.00
Yes 10 3.700 .82327 .26034 3.1111 4.2889 3.00 5.00
Total 20 3.150 .93330 .20869 2.7132 3.5868 2.00 5.00
Mod-el
FixedEffects
.76376 .17078 2.7912 3.5088
RandomEffects
.55000 -3.8384 10.1384 .54667
ALevene’stestwasconductedonthelowaltitudescenario.Thetestindicates
thattherewasnosignificantdifferenceinthetrainedanduntrainedgroupvariances.
ThetestresultswereF(1,18)=.450,p=..511.SeeAppendixE.
Page 96
81
Aone-wayANOVAwasconductedonthelowaltitudescenariototestthemain
effectofcrewtraining.Therewasasignificanteffectoftrainingonthelowaltitude
scenarioscore,F(1,18)=10.37,p=.005.Thetestconfirmstheresearchhypothesisthat
targetedtrainingbesuccessfulinhelpingpilotsmaintainaircraftcontrolduringan
unusualandsuddenstartleevent.Inthecaseofthelowaltitudescenario,thenull
hypothesisisrejected.TheANOVAresultsaresummarizedinTable25.
Table25.TestsofBetween-SubjectsEffectsLowAltitude
DependentVariable:LowAltitudeScenario
Source TypeIIISumofSquares
df MeanSquare
F Sig. PartialEtaSquared
Noncent.Parameter
ObservedPowerb
CorrectedModel
6.050a 1 6.050 10.371 .005 .366 10.371 .861
Intercept 198.450 1 198.450 340.200 .000 .950 340.200 1.000
CrewTrng 6.050 1 6.050 10.371 .005 .366 10.371 .861
Error 10.500 18 .583
Total 215.000 20
CorrectedTotal
16.550 19
a.RSquared=.366(AdjustedRSquared=.330)
b.Computedusingalpha=.05
TheeffectsizewascalculatedfromtheANOVAresultsusingtheformula
R2=SSM/SSTwiththeresultbeingh=.60.ThisrepresentsalargeeffectsizeR2=h2
=6.05/16.55Etah=.60.Theresultindicatesthattheaveragepersoninthe
Page 97
82
experimentalgroupwould,asinthehighaltitudescenario,scorehigherthan73%ofa
controlgroupthatwasinitiallyequivalent(Coe,2002).
RegressionLowAltitude
Regressionanalysiswasconductedonthefactorsthatmadeuptheoverall
scenarioeventscore,similartothehighaltitudescenario.Thiswasdonetoexploreany
significantfactorsleadingtotheoverallscoreandtopinpointpossibleareasoffuture
training.Thefactorsaresummarizedinthetablebelow.Eachfactorwasevaluated20
times.SeeTable26.
Table26.DescriptiveStatisticsLowAltitude
Mean Std.Deviation N
LowAltitudeScenario 3.1500 .93330 20
CrewTrainingReceived .5000 .51299 20
Missedapproach 2.8000 1.10501 20
Checklistprocedures 3.1500 .98809 20
TimeManagement 3.1500 1.03999 20
FuelManagement 3.2500 .91047 20
Approachandlanding 3.3000 .92338 20
Theregressionanalysiswasconductedintwoblockswithblockonebeingcrew
trainingandblocktwoconsistingoftheabovedescribedfactors.TheANOVAindicates
thatbothmodelssignificantlyimprovetheabilitytopredicttheoutcomevariable
Page 98
83
comparedtonotfittingthemodel.Model1hadF(1,18)=10.37,p=.005andModel2
hadF(6,13)=28.13,p=.00.SeeAppendixE.
Theregressioncoefficientswerealsoanalyzedtodeterminewhichfactors(other
thancrewtraininginModel1)showedsignificance.Significancewasnotedforthe
factorofmissedapproachp=.01andtimemanagementp=.00.Theotherfactorsdid
notshowsignificance(p>.05).Collinearityanalysisindicatesthatthereareno
examplesofmulticollinearity(VIF>10).SeeAppendixE.
Eachpredictorhadvarianceloadingontoadifferentdimension.Thisalso
indicatesnoissueswithmulticollinearity.
LowandHighAltitudeCombined
Thefinalsetofanalyseswereconductedbycollapsingtheeffectofcrewtraining
acrossboththelowaltitudeandhighaltitudescenarios.Therewasnoregressiononthe
sub-factorsofthecombinedscoresduetothefactorsalreadybeinganalyzedinthe
individualscenarios.Analysiswasalsoconductedontheeffectsofpilotflying(PF),pilot
monitoring(PM)andthetypeofaircraftinvolved(narrowbodyorwidebody).Further
analysiswasconductedtocompareboththehighaltitudeandlowaltitudescenariovs
theFAAstandardforATPcertificationforboththetrainedanduntrainedgroups.
Atotalof40crews(80individuals)volunteeredforthestudy,ofwhich19didnot
receivestartletrainingand21receivedstartletraining.Themeanfortheuntrained
crewswas2.58withastandarddeviationof.6.Themeanwas3.71withastandard
deviationof.71forthetrainedgroup.Thedeterminationofthescenarioscorewas
Page 99
84
describedintheMethodsSection.Table27summarizesthegroupmeansandstandard
deviation.
Table27.DescriptiveStatisticsCombined
DependentVariable:LowandHighAltitudeCombined
CrewTrainingReceived Mean Std.Deviation N
No 2.5789 .60698 19
Yes 3.7143 .71714 21
Total 3.1750 .87376 40
Afurtherbreakdownwasanalyzedtodetermineifthereweresignificant
differencesbetweenthehighandlowscenarioswhencomparingtrainedanduntrained
crews.TheresultsaresummarizedinTable28.
Table28.HighandLowAltitudeMeanComparison
CrewTrainingReceived HighAltitudeScenario LowAltitudeScenario
No Mean 2.6667 2.6000N 9.0 10.0Std.Deviation .70711 .69921
Yes Mean 3.7273 3.7000N 11.0 10.0Std.Deviation .64667 .82327
Total Mean 3.2500 3.1500N 20.0 20.0Std.Deviation .85070 .93330
Page 100
85
At-testwasutilizedtotestforsignificancebetweenthemeanofthetrained
crewsforboththelowandhighaltitudescenarios.Thetestdidnotshowsignificancein
eithercase(p=.89and.91)SeeTable29.
Table29.MeansComparisonwithTraining
TestValue=3.7/3.73
t dfSig.(2-tailed)
MeanDifference
95%ConfidenceIntervaloftheDifference
Lower Upper
HightoLow
.140 10 .892 .02727 -.4072 .4617
LowtoHIgh
-.115 9 .911 -.03000 -.6189 .5589
Theuntrainedgroupswerealsoanalyzedforsignificancebetweenthelowand
highaltitudescenarios.Themeanfortheuntrainedalsodidnotshowanystatistical
significancewithp=.78and.76.SeeTable30.
Page 101
86
Table30.MeansComparisonNoTraining
TestValue=2.60/2.67
t dfSig.(2-tailed)
MeanDifference
95%ConfidenceIntervaloftheDifferenceLower Upper
HightoLow
.283 8 .784 .06667 -.4769 .6102
LowtoHigh
-.317 9 .759 -.07000 -.5702 .4302
Theresultsofthet-testsindicatethattherewasnosignificantdifference
betweenthetrainedanduntrainedgroupswhenthetrainedanduntrainedgroupsare
combinedacrossscenariosets.Theresearchquestionthatsoughttodetermineifthe
proximityofthescenariohadanyeffectontheoutcomewasanswered.Thedata
indicatesthatthenullhypothesisinthiscaseisretained.
Aswithboththehighandlowaltitudescenario,aLevenetestforequalvariances
wasconducted.ThetestdidnotyieldsignificantresultsF(1,38)=.046,p=.83,
thereforetheassumptionisthatthevariancesareequalacrossthegroups.See
AppendixE.
ThenextstepintheanalysiswasanANOVAcalculatedusingSPSS.TheANOVA
testedforcrewtrainingwhencollapsedacrossbothhighandlowaltitudescenarios.
ThetestrevealedthattheeffectofcrewtrainingwassignificantF(1,38)=28.89,p=.00.
Thismeansthatthetrainedcrewsshowedastatisticallysignificantincreasein
performanceduetotheeffectofcrewtraining.TheresultsaresummarizedinTable31.
Page 102
87
Table31.TestsofBetween-SubjectsEffectsCombined
DependentVariable:LowandHighAltitudeCombined
Source TypeIIISumofSquares
df MeanSquare
F Sig. PartialEtaSquared
Noncent.Parameter
ObservedPowerb
CorrectedModel
12.858a 1 12.858 28.881 .000 .432 28.881 .999
Intercept 395.058 1 395.058 887.387 .000 .959 887.387 1.000
CrewTrng 12.858 1 12.858 28.881 .000 .432 28.881 .999
Error 16.917 38 .445
Total 433.000 40
CorrectedTotal
29.775 39
a.RSquared=.432(AdjustedRSquared=.417)
b.Computedusingalpha=.05
TheeffectsizewascalculatedfromtheANOVAresultsusingtheformula
R2=SSM/SSTwiththeresultbeingh=.66.ThisrepresentsalargeeffectsizeR2=h2
=12.86/29.78Etah=.66.Thecombinedeffectsizewaslargerthanboththehighand
lowaltitudescenarioeffectsizes.Theresultindicatesthattheaveragepersoninthe
experimentalgroupwouldscorehigherthan73%ofthecontrolgroupthatwasinitially
equivalent(Coe,2002).
Thenexttestmeasuredforsignificantdifferencesinthepilotflying,pilot
monitoring(CaptainorFirstOfficer),andthetypeofaircraft.ANOVAtestingfor
differencesbetweenPF,PMandNB/WByieldednosignificantresults.Theresultsare
summarizedinTable32.
Page 103
88
Table32.ANOVAOtherFactors
SumofSquares df MeanSquare F Sig.
LowandHighAltitudeCombined
BetweenGroups
12.858 1 12.858 28.881 .000
WithinGroups 16.917 38 .445
Total 29.775 39
CaptainorFirstOfficer
BetweenGroups
.226 1 .226 .877 .355
WithinGroups 9.774 38 .257
Total 10.000 39
PilotMonitoring
BetweenGroups
1.684 1 1.684 2.410 .129
WithinGroups 25.158 36 .699
Total 26.842 37
NarroworWidebodyacft
BetweenGroups
.095 1 .095 .367 .548
WithinGroups 9.880 38 .260
Total 9.975 39
TheANOVAindicatedthatwerenodifferencesinscenarioscoreswitheitherthe
CaptainorFirstOfficerflyingF(1,38)=.88,p=.36.Italsoindicatedthattherewasno
significancetowhichcrewmemberwasthepilotmonitoringF(1,36)=2.4,p=.129.
Finally,thetestindicatednosignificancebetweenaircrafttypeF(1,38)=.37p=.55.
Thistestansweredtheresearchquestionseekingtoexploreifsignificantdifferences
couldbedeterminedwithinthesecategories.Intheabovecases,thehypothesisof
significantdifferencesisrejectedandthenullhypothesisisretained.
Page 104
89
PerformanceversusFAAStandard
Thenextsegmentoftheanalysiswasconductedusingaseriesoft-teststo
exploredifferencesofcrewperformancefromtheFAAstandardforATPcertification.
Thecrewsforthisstudyhadjustcompletedtheirrecurrenttrainingcyclewherethey
werecertifiedtoFAAstandards.
Thefirstt-testexploredtheFAAstandardofthree(3)totheoverallmean(high
andlowaltitudescenario)oftheuntrainedcrews.TheresultsaresummarizedinTable
33.
Table33.T-testTrainingversusFAAStandard
TestValue=3t df Sig.(2-tailed) Mean
Difference95%ConfidenceIntervaloftheDifferenceLower Upper
NoCrewTraining
-3.024 18 .007 -.42105 -.7136 -.1285
CrewTraining
4.564 20 .000 .71429 .3878 1.0407
Thet-testrevealed,thatinthecaseofnocrewtraining,thatperformancewas
significantlybelowtheFAAcertificationstandards.Themeanscoredifferencewas-.42
thatresultedinasignificanceofp=.01.Thisindicatesthatcrewperformanceduringa
startleeventissignificantlydifferentfromtheFAAstandard.Inthecaseofcrew
training,theresultswerealsosignificantbutinapositivedirectionresultinginamean
scoredifferenceof.71withap=.000.
Page 105
90
SurveyandPerformanceCorrelations
Thefinalsetofanalysiswasconductedtodetermineifsurveyresponses
correlatedwithcrewperformance.Analysiswasconductedseparatelyforboththe
trainedanduntrainedcrews.Thefirstsetoftestsfortheuntrainedcrewsuseda
Pearson’scorrelationtohighlightsignificantresult.Theresultsaresummarizedin
AppendixE.
Therewerenosignificantsurveyresponsesthatcorrelatedwiththeperformance
oftheuntrainedcrews.Therewere,however,significantcorrelationsbetweentwoof
thesurveystatements.Thestatements“Ioftenhand-flybelow10,000feet”and“I
oftenpracticemyrawdataskills”wassignificantwithp=.02.Thestatement“Iam
comfortableflyingwithrawdata”and“Ioftenpracticemyrawdataskills”’alsoshowed
asignificantcorrelationwithp=.03.Thesequestionsshouldbecorrelatedbecauseone
questiongenerallyproducestheother.
Significantcorrelationswerealsoexploredforthegroupthatreceivedcrew
training.Theresultswereidenticaltotheuntrainedgroupwiththesametwo
statementsshowingsignificantcorrelationsp=.00andp=.01.Theresultsarealso
summarizedinAppendixE.
Page 106
91
ResultsSummary
Thedatawascollectedandanalyzedintwodistinctparts:thepilotsurvey,and
thecrewsflyingascenariointhesimulator.Thesurveywasdesignedtogainthepilot’s
perspectiveontheirexperiencesandattitudestowardsflying.Theprofilescenariosthat
wereflowninthesimulatorsoughttoproduceastartleeventforthecrewsandthen
recordtheirperformance.
Thesurveydataindicatedthatover80%ofthepilotsreportedthattheyhad
incurredaneventwhileflyingthatsurprisedthem.Inaddition,agoodmixofboth
civilian(67%)andmilitary(33%)pilotsparticipatedinthestudy.Thesurveyquestions
werenotcorrelatedwithcrewperformanceineitherthelowaltitudeorhighaltitude
scenarioandthuswerenotapredictorofcrewperformance.
TheeventscenarioswereallflowninanFAAcertifiedLevel-Dfullflightsimulator
(FFS),withaccuratevisualandvestibularsensoryinput.Inbothscenariotypes,the
crewsthatreceivedtrainingthatconsistedofabriefingandsimulatorpracticeshoweda
significantimprovementinperformancethanthecrewsthatdidnotreceivetraining.
Thisconfirmedthehypothesisthattargetedtrainingonmitigationofstartleeffectcould
increasecrewperformance.Regressionanalysiswasalsoconductedonthefactorsthat
madeupeachcrewperformance.Theanalysissuggeststhatproblemrecognitioninthe
highaltitudescenarioandthemissedapproachinthelowaltitudescenariowere
significantpredictorsofperformanceontheoverallprofile.
Page 107
92
Crewperformance,collapsedacrossbothscenarios,wasalsomeasuredagainst
theFAAstandardsforATPcertification.Whencrewsreceivedtraining,thedatashowed
asignificantimprovementinperformancevstheFAAstandard.Dataalsoindicatedthat
untrainedcrewsperformedsignificantlyworsethantheFAAstandardwhenpresented
withastartleevent.Thisresultwasunexpected.Therewerenosignificantresults
whenexaminingthepilotflying,pilotmonitoring,oraircrafttypeasitrelatedtocrew
performance,rejectingthehypothesisthatthesefactorswouldbesignificant.There
werealsonosignificantresultsbetweenthelowaltitudeandhighaltitudescenarios
whenlookingatbothtrainedanduntrainedcrews.Thenullhypothesiswasretainedfor
thisresearchquestionrelatedtoeventproximity(loworhighaltitude).
Finally,thesurveyresponseswerecorrelatedwithcrewperformance.There
werenosignificantcorrelationsbetweenperformanceandsurveyresponses.There
were,however,twosignificantcorrelationsthatweresolelyrelatedtothesurvey
questions.
Page 108
93
CHAPTERV
DISCUSSION
Introduction
Startleeffecthasbeenwelldocumentedforthepast50years.Vlasak(1969)
foundsignificantimpairmentincognitivefunctionforthefirst15secondsfollowinga
startleevent.Otherstudieshaveshownsimilarresults.Unfortunately,itisoftenduring
thisperiodthatcriticalaircrafthandlingdecisionsmustbemade.AccordingtoBoeing
(2012),inflightlossofcontrolistheleadingcauseofairlinefatalities.Recentaccidents
suchasAirAsiaandFlyDubaiindicatetheinflightlossofcontrolcontinuestobea
significantsafetyissueforairlines.Thisstudysoughttodetermineiftargetedtraining
couldimprovesimulatorperformanceofcrewsduringastartleevent.Volunteerairline
pilotsflewtwodifferentstartlescenariosinafullflightsimulator.Thescenarioswere
designedtobesimilartoeventsthathavecausedmajorairlineaccidents.Thevolunteer
groupsweredividedintolowaltitudescenariogroupsandhighaltitudescenariogroups.
Thegroupswerefurtherdividedintotrainedanduntrainedgroups.Dataanalysiswas
conductedonthemaineffectoftrainingwithinandbetweeneachvolunteergroup.
Additionalanalysiswasalsoconductedonthesub-factorsthatmadeupeachscenario
suchaspilotflying,aircrafttype,andmaneuversub-parts.
Page 109
94
Findings
Thedatashowedthattargetedstartletrainingcouldimprovecrewperformance
whileflyingstartlescenariosinthesimulator.Significancewasfoundforthetrained
crewsinboththelowandhighaltitudescenarioswhencomparedtotheuntrained
crews.Theeffectofthetrainingwasshowntobehigh,predictingthattrainedcrews
wouldperform73%betterthanuntrainedcrews(Coe,2002).Trainedcrewsalso
showedasignificantincreaseinperformancewhencomparedtotheFAAstandardsfor
ATPcertification.Theresultsansweredtheresearchquestionaskingwhethertargeted
trainingcouldincreasecrewperformanceduringastartleeventinthesimulator.
Inthehighaltitudescenario,crewswereexposedtofailuressimilartowhat
occurredintheAirFrance447accidentasdescribedintheliteraturereview(BAE,
2012).Crewsthatreceivedtrainingthatconsistedofabriefingandsimulatorpractice,
onhowtohandlestartleevents.ThisgroupperformedsignificantlybetterthantheFAA
standardandsignificantlybetterthanthecrewsthatflewthesameprofilebutdidnot
receivetraining.
ThelowaltitudescenariowasmodeledaftertheColganaccidentinBuffalo,NY,
andpresentedthecrewswithalowaltitudestartleeventthat,inmostcases,pushed
thecrewsintoamissedapproachinalowfuelsituation.Theresultsweresimilarto
thoseofthehighaltitudeprofileinthattrainedcrewsperformedsignificantlybetter
thantheuntrainedcrews.
Page 110
95
Unexpectedresultswerefoundinthecrewsthatdidnotreceivethestartle
trainingwhencomparedtotheFAAstandard.Thedataanalysisshowedthatthecrews
performedstatisticallysignificantlybelowthelevelforATPcertificationforatleasta
portionofthetimeduringthemaneuverprofile.Itshouldbenotedthattherewereno
crewswholostcontroloftheaircraftduringtheprofileandthatallcrewseventuallyhad
asuccessfuloutcome.Thisdataexploredtheinitialreactionofthecrewssincethisis
thecriticaldecisionmakingtimeframe.Thecrewsforthisstudyhadjustcompleted
theirannualrecurrenttrainingcyclefortheirrespectiveaircraftunderwhatisknownas
theadvancedqualificationprocess(AQP).InanAQP,TrainingProgram,whenacrew(or
individualpilot)fallsbelowtheFAAstandardtheyreceivetrainingandthenarerequired
toperformtheskillagain.Thisconceptistermed“traintoproficiency.”Crewsinthis
researchstudywereonlypresentedthescenarioonetime.This“firstlook”onlytakesa
snapshotofacrews’performanceintimeandindicateswheretrainingcouldbe
effective.Itisnotmeanttobeextrapolatedtooverallcrewcompetency
Eachmaneuverscenariowasmadeupofseveralsub-factorsorcomponentsthat
comprisedtheoverallscore.Thesefactorswereanalyzedtodeterminetheir
significanceinmakingupthetotalscoreandtouncoverpossibledimensionswhere
trainingshouldbetargeted.Theresultsforthehighaltitudescenarioindicatedthatthe
mostsignificantfactorindeterminingscenariosuccesswas“problemidentification”.
Thiswasconsistentwithpreviousresearchfindingswhichshowedthatwhencrews
makeaninitialwrongdecision,thein-flightissuetendstorapidlydegrade.Thelow
altitudescenariowassomewhatlessclearinsignificantfactors.Timemanagementwas
Page 111
96
asignificantpredictorofcrewperformance.Themissedapproachwasalsoasignificant
predictorwhichwasunexpected.Ifcrewsperformedthemissedapproachsucessfully,
therestofthescenariogenerallywasgradedbetterthanifthemissedapproachwas
incorrectlyflown.Minimumamountoffuel(45minutes)atstartofthemissed
approachlikelyinfluencedthisresult.
Theresearchalsosoughttodetermineifthepilotflying(CaptainorFirstOfficer)
resultedinsignificantdifferencesinmaneuverperformance.Thestudy’sdata
demonstratedthatcrewperformancewasnotaffectedbywhichpilotwasflying.The
studylookedatthisdimensionsinceinamajorityofairlineaccidents/incidents,the
Captainisthepilotflyinganditisthefirstflightofthetrippairing(United,2016).
Simulatordataalsodidnotuncoveranysignificancebetweenthetypesof
aircraftinpredictingthesuccessofthescenario.Aircraftweregroupedintonarrow
bodyandwidebodycategories.Narrowbodyaircraftpilotshaveagreaterfrequencyof
takeoffsandlandingsthanthosepilotwhoflywide-bodyaircraft.Thisdifferencein
frequencymayaddtonarrowbodyaircraftpilotsflyingproficiencyandincreasethe
successofrespondingtoastartleevent.Inaddition,mostwide-bodyaircraftrely
heavilyonautomationduetothelongdurationoftheirflightspossiblemakingasudden
startleeventmorechallenging.However,pilotswhoflywide-bodyaircraftgenerally
haveamoreexperiencedbackgroundthanpilotswhoflynarrow-bodyaircraftdueto
theairlines’senioritysystem.Theresearchquestiononpracticeandexperiencesought
todetermineifpracticeand/orexperiencecouldinfluencetheoverallmaneuverscore.
Theresearchquestionrelatingtoaircrafttypedidnotshowitaffectedthemaneuver
Page 112
97
scenarioineitherapositiveornegativeway.Therewasnosignificantdifferencein
eitherpilotgroup.
Thepilotsinthisresearchstudywerealsorequestedtocompleteasurveythat
askedquestionsabouttheirbackgroundandflyingpreferences.Thesurveyresponses
werenotlinkedtoanycrewmember’sindividualperformance,butwereanalyzedin
aggregatewithregardtosimulatorperformance.Thepilotflyingthemaneuverscenario
wastheonlycrewmemberaskedtocompletethesurvey.Thiswasdonetokeepthe
surveyresponsesequaltothenumberofcrewsobserved(40)andtocomparethe
responseswiththepilotwhoflewthesimulatorprofile.Thesurveyresponsesindicated
thatthepilotsgenerallyhandflewtheairplanebelow10,000feetandthattheyknew
theproperpitchandpowerforvariousphasesofflight.Furthermore,asignificant
portionofthepilotssurveyedindicatedthattheyhadreceivedformalaerobatic
training.Theresponsesgivendonotnecessarilycorrelatewithsimulatorperformance
whentakenasawhole.Ifthetrainedcrewsweretheonlyonesexamined,thenthe
surveyresponsescorrelatewithpositiveperformance;however,whenuntrainedcrews
wereadded,pilotstendedtooverestimatetheirflyingperformance.
Significance
Thedatarecordedforthisstudyshowedthattargetedtrainingcanhelppilots
bridgethecognitivegapwhenstartled.Crewsperformedequallywellinboththehigh
altitudeandlowaltitudescenario,suggestingthatthetraininghadabroadarrayof
effectiveness.Bothscenariosrecordedasimilarmaineffect:powerofeta=.6
Page 113
98
suggestedamediumtolargeeffectsize.Thetrainingofferedconsistedofabriefingto
explaintheeffectsofbeingstartledalongwithashortandsimpleproceduretohelp
mitigatethestartleeffectandregain(orkeep)controloftheaircraft.Thestudywasnot
designedtoeliminatethestartleresponsewhichwouldbeverydifficulttoaccomplish,
butsoughttohelpcrewsmanagetheperiodofcognitiveimpairment.Insummarizing
thetraining,themotto“liveforthenext60seconds”wasoftenused.Thisisthetimein
whichthemostcognitiveimpairmentoccurs.Unfortunately,crewsoftenhavetomake
criticaldecisionsinthistimetokeepcontroloftheaircraft.Thedataindicatedthat
trainedcrewsweremoresuccessfulinmanagingtheaircraftfollowingastartleevent
thanthosecrewsthatdidnotreceivetraining.Crewsthatreceivedtrainingflew
significantlybetterthantheFAAstandardsforATPcertification,indicatingapositive
shiftineventhandlingevenversusastandardcrew.
Thetrainingsuggestedinthisstudyhasimplicationsfortheairlineindustryasa
whole.Aspreviouslystated,crewsthatwerenottrainedshowedastatistically
significantdegradationbelowFAAATPstandards.Followingthestartleevent,the
untrainedcrewslapsedoutofATPstandardsasdescribedintheMethodsSectionofthis
study.Allofthecrewswereeventuallyabletosuccessfullyrecoverfromthesimulated
situation,howeveritisthedecisionmakingattheonsetwhichcanprovecriticalto
eventoutcome.Duringthestudytherewerenocrewsputtheaircraftintoanundesired
aircraftstate(UAS).Thissuggeststhatcurrentairlinetrainingmaybeimprovedby
incorporatingstartletraining.Severalpublishedpapersalludetothisideainthatairline
traininghasbecomeroteandroutine;notchallengingcrewswithnewsituationsand
Page 114
99
scenariosthatexpandflyingknowledgeandexperience(Casner,Geven,&Willliams,
2012).Mostairlineshaveastandardtrainingprofilethatisdeterminedbythe
regulatoryrequirementsoftheFAA.Thistrainingisgenerallythesamefromyearto
yearresultinginrepetitionandexpectedoutcomes.Trainingoutsideofthisset
standardisoftenreferredtoas“proficiencytraining”(UnitedAirlines,2016),andusually
exposesthecrewonlyatpredeterminedcyclesandhasmoretodowithtechnical
failuresandnotcognitiveloading.
Thepilots,throughtheirsurveyresponses,indicatedthattheygenerallyhandfly
theairplanebelow10,000feet.Handflyingbelowthisaltitudeprovidesaboosttoskill
maintenanceduetothefactthatchangesinallphasesofflightoccurfrequently.
Takeoffs,approaches,andlandingsallrequirechangestoaircraftspeed,configuration,
andnavigation(lateralandvertical).Thesemaneuverschallengepilotingskillsandkeep
themsharp(Gillen,2014).Overallpilotingskillsmaybeakeyelementinaircraft
control.Apilotproficientinhandflyingwillrequirelesscognitiveresources(toflythe
airplane)andmaybeabletodevotemorecognitiveprocessestoproblemdetection.
PilotsintheUnitedStatesgenerallyhandflytheaircraftmorethaninotherpartsofthe
world.InapaperpresentedattheLufthansaHumanFactorsConference(Gillen,2014),
pilotsemployedbyairlinesfromvariousglobalcarriersweresurveyedabouttheirhand
flyingpractices.TheresultsaresummarizedinTable34.
Page 115
100
Table34.HandFlyingPreferences
Company
Description
CompanyPolicy ActualPractice
UnitedStates-
Global
Companypolicystatesthat
theautomationlevelisat
thediscretionofthe
Captain.
Amajorityofpilotshandflythe
aircraftextensivelybelow
10,000feet.
MajorEuropean Handflyingisencouragedto
maintainproficiency
Mostpilotsreporthandflying
below10,000feet.
MiddleEast Companypolicyprohibits
handflyingabove10,000
feet.
Pilotsreportthattheygenerally
engagetheautopilotat1,000
feetondepartureanddisengage
onapproachoncetheaircraftis
fullyconfiguredforlanding.
Asia Companypolicyencourages
handflyingtoincreasepilot
proficiency
CompanyregularlyusesFOQA
dataindisciplinarycasesagainst
pilots.Asaresultpilotsrarely
handflytheaircraft.
SoutheastAsia Companypolicystatesthat
automationisatthe
discretionoftheCaptain
Manualflyingvariedwidely
dependingontheflightcrews.
Lackofflyingskillsbecomemoreapparentwhensystemfailurescausepilotsto
reverttomanualflyingskillstomaneuvertheaircraft.Simplefailurescanleadtoa
cascadeoferrorsandpilotconfusionthatinturncanleadtoanundesiredaircraftstate
(Gillen,2014).Thesesystemfailurescanalsotaxapilot’scognitiveresourceswell
beyondtheirabilitytocopewiththesituation(Gillen,2014).Basedonhandflying
Page 116
101
preferences,thepilotsinthisstudyshouldbeconsideredthemostproficientandthus
theresultsshouldtrendhigherwhencomparedtopilotswhohandflyless.
CriticalEvaluation
StudiesinvolvingairlinecrewsintheUnitedStatesareoftendifficultto
complete.Airlinesandtheirrespectivepilotunionsarereluctanttohavearesearcher
recordlivedataonacrews’performance.Theonlywaytoobtainpermissionforsucha
studyistohavethedatade-identifiedsothatnoindividualperformancecanbelinked
backtoaspecificpilotorcrew.Assuch,thisresearchwasonlyabletoobserve
volunteercrewsonetimetodeterminetheeffectofthetraining.Trainingresultscould
havebeenmoreconclusiveifarevisitofthetrainedcrewshadtakenplace.
Unfortunately,thiscouldnotoccurasitwouldmaketheindividualpilotsidentified
whichwouldviolatethepermissionletterfromtheparticipatingairline(s).Data
suggeststhatunusedtrainingskillswilldecreaseineffectivenessovertimeandthat
deliberatepracticeisrequired.Itistheopinionoftheresearcherthatifthetraining
presentedinthisstudyisnotpracticed,thentheeffectiveness(ofthetraining)willmost
likelydecreaseovertime.
Anytypeofsimulatortrainingcouldbereasonablyexpectedtoanincreaseinthe
crews’overallscenarioscores.Studysubjectswereaskedtovolunteerimmediately
followingtheirannualrecurrenttrainingcyclewhichincluded8-12hoursofsimulator
training/checking.Allcrewsinvolvedinthestudyreceivedsimulatortrainingand
Page 117
102
practiceimmediatelyprecedingparticipationinthisstudy.Asaresult,scoresmaybe
skewedhigherthanifcrewsflewthemaneuverswithoutanypreviouspractice.
Resultsinthisstudycouldpossiblybebiasedinapositivedirectionduetothe
voluntarynatureoftheparticipants.Generally,pilotswhovolunteerforthesetypesof
studiesarecomfortableintheirflyingskillsandinterestedinaviationsafety.Pilotswho
havedifficultyintraininggenerallywouldchoosetonotparticipate.Therefore,the
overallresultsmightbeskewedtowardsthehigherendthantheaverageairlinepilot
population.
Implications
Thestartleeffectisnotanewconceptandtheeffectsofbeingstartledarewell
known.Whatisnotwellknownishowtomitigatethestartleeffectinairlinecrews
wherecriticaldecisionmakingmustoftentakeplaceconcurrentwiththetimeof
cognitiveimpairmentfollowingastartleevent.Trainingandpracticehavebeenshown
toincreaseapilot’sresponsetoaircraftcontrolduringaneventthatcatchesacrew
unexpectedly.Targetedtrainingshouldbeproceduralinnatureandseektobecome
skillbase(best)orrulebasedbehavior.Thismethodrequiresaconsistentand
systematicapproachtodealingwithunusualevents.
Tobeeffective,trainingthatisdescribedinthisstudyshouldbeimplementedin
bothinitialandrecurrentpilottraininginadditiontobeingreinforcedinactualline
flying.Thisstudyhasshownthatsignificantpositiveeffectsoftrainingcanberealizedin
aslittletimeas60minutes.
Page 118
103
Followonstudiesshouldlookattheeffectivenessofthistypeoftrainingover
longerintervals.Thetrainingpresentedinthisstudywasdesignedtobebroadinnature
andcovervariousstatesofcontingenciesasitrelatestoastartleevent.Suchtrainingis
intendedtobeapplicableinageneralwayandisnotintendedtobeaircraftorairline
specific.Itismoreofaphilosophyindealingwithunusualeventsattheinitialdecision
makingpointtohelpbiasasuccessfuloutcome.IntheAirFranceandColganaccidents,
thatweredescribedintheliteraturereview,aircraftcontrolwaslostinthefirst30
secondsfollowingastartleevent.Trainingshouldfocusonthistimeperiodtobemost
effective.Althoughsuddenandunusualeventscannotbepreventedinaviation,a
pilot’sresponsetothem(especiallyattheonset)canbepositivelyinfluencedtoaidina
successfuloutcome.Thereisnotasinglesolutioninairlinetrainingtoeliminatetherisk
ofastartleevent,onlymitigatingfactors,thatwhenpresentedinmultiplelayersserve
toaidcrewsinsuccessfullyhandlingtheevent.
Recommendations
Startletrainingshouldbeaddedtothetrainingprogramsatairlinestomake
crewsawareoftheeffectsonperformanceofbeingstartledandmitigationstrategies
thatcanhelppilotssuccessfullyflytheaircraftimmediatelyfollowingastartleevent.
Startletraining,inordertobeeffective,hastobereinforcedatspecifictrainingintervals
suchasduringeachpilot’sinitialandannualrecurrenttrainingcycle.Positiveresults
wereshowninthisstudywheretrainingconsistedofbothclassroomandsimulator
practicelastingapproximatelyonehour.Trainingshouldfocusonwhathappensfroma
cognitivestandpointandwhatstepspilotsshouldtaketostabilizetheaircraftsothat
Page 119
104
theycanthendeterminethecourseofactiontosafelyflytheaircraft.Tobeeffective,
thistrainingwillhavetobevariedtopreventhabituation.
RecommendationsforFurtherResearch
Researchintostartletrainingshouldcontinue.Thisresearchshouldattemptto
identifythebesttrainingintervalforstartletrainingtopreventdegradationofstartle
responseskills.Theremaybealinkbetweenhandflyingability,andthegeneralability
tohandleastartleresponse.IncasessuchastheAirFranceandColganaccidents,the
pilotswerefacedwithasuddeneventthatforcedthemintohandflyingtheaircraft.
Pilotswhoarecompetentinhandflyingrequirelesscognitiveresourcestodoso,and
maybeabletodevotemoreresourcestoproblemdefinition.Thiswouldbeagoodarea
forfutureresearchaswell.Onefurtherareaforresearchmaylookatthelinkbetween
oftenpracticedunusualsituationsinthesimulatorandtheirnegativetransferto
unrelatedeventsintheactualaircraft.Duringtraining,mostofthemaneuversare
performedatlowaltitudesandrequireanimmediatepitchupoftheaircraft’snose
(enginefailures,windshear,andmissedapproachesareallexamples).Whilethese
responsesareappropriateforlowaltitudes,theinitialresponsetopitchthenoseup
maynotbeappropriateathighaltitudessuchasseenintheAirFranceaccident.
Conclusion
Thisstudyshowedthattargetedtrainingcanimprovecrewperformanceinthe
simulatorduringastartleevent.Giventhatskillslearnedinthesimulatoraregenerally
welltransferredtoactualoperations,thestudyresultsshouldalsobehighly
Page 120
105
transferrabletotheactualaircraft.Datashowsthatincreasedstartletrainingcould
significantlyimproveapilot’sreactiontoastartleevent.
Startleeventscontinuetobeamajortriggerresultinginaircraftinflightlossof
control.Althoughnoteveryeventwillresultinalossofaircraftcontrol,trainingcan
helpbridgethecognitivegapthatexistsduringtheinitialsecondsofastartleevent.
Trainingsuchaswhatwaspresentedinthisstudyshouldbeaddedtoairlinetraining
programstoaidcrewstowardsasuccessfuloutcomeofastartleevent.Akeyelement
indealingwithastartleeventofteninvolvesmanualmanipulationoftheaircraft
controls.Pilotswhoareproficientinhandflyingwillhaveanadvantageindealingwith
astartleevent.Training,practice,andhandflyingeachholdanimportantelementin
successfullymitigatingastartleeventandpreventinganinflightlossofcontrol.Further
studiesshouldseektodeterminetheoptimumintegrationofthesethreeelements.It
willtakeapartnershipbetweentheairlines,pilots,andtheregulatorstoimplement
startletraining.Suchtrainingcanbeakeymitigationstrategyinreducingtheleading
causeorairlinefatalities.
Page 121
106
APPENDICES
AppendixA
Definitions
AC:AdvisoryCircular
ACO:AircraftCertificationOffice
AD:AirworthinessDirective
AEG:AircraftEvaluationGroup
ALPA:AirlinePilotsAssociation
AQP:AdvancedQualificationProgram
ARAC:AviationRulemakingAdvisoryCommittee
ASAP:AviationSafety/AccidentPrevention
ASRS:AviationSafetyReportingSystem
ATA:AirTransportAssociationofAmerica
ATC:AirTrafficControl
ATIS:AutomaticTerminalInformationService
ATP:AirlineTransportPilot
ATS:AirTrafficServices
Page 122
107
BIS:BasicInstrumentSkills:Theabilitytoflytheaircraftsolelybyreferencetotherawdatawithouttheuseofauto-throttles,flightdirector,ormapmode.
CFIT:ControlledFlightintoTerrain
CMO:CertificateManagementOffice
CRM:CrewResourceManagement
FAA:FederalAviationAdministration
FAR:FederalAviationRegulations
FCOM:FlightCrewOperatingManual
FCU:FlightControlUnit
FMS:FlightManagementSystem
FOEB:FlightOperationsEvaluationBoard
FSB:FlightStandardizationBoard
FSDO:FlightStandardsDistrictOffice
GPS:GlobalPositioningSystem
GPWS:GroundProximityWarningSystem
HF:HumanFactors
ICAO:InternationalCivilAviationOrganization
IFR:InstrumentFlightRules
Page 123
108
IOE:InitialOperationalExperience
ILS:InstrumentLandingSystem
JAA:JointAviationAuthorities
JAR:JointAviationRequirements
LNAV:LateralNavigation
LOFT:LineOrientedFlightTraining
LOS:LineOperationalSimulations
ModernAircraft/GlassAircraft:Aircraftthathaveadvancedautomationtoinclude:CATIIIcapability,auto-throttles,flightdirector,FMC,andCRTdisplaysinsteadofactualinstruments,theabilitytoLNAVandVNAV
NASA:NationalAeronauticsandSpaceAdministration
NOAA:NationalOceanicandAtmosphericAdministration
NOTAM:NoticetoAirmen
NTSB:NationalTransportationSafetyBoard
PDC:Pre-DepartureClearance
PF:PilotFlying
PFD:PrimaryFlightDisplay
PM:PilotMonitoring
PTS:PracticalTestStandardsDefinedbytheFAAPilotQualification.
Page 124
109
RNP:RequiredNavigationPerformance
TCAS:TrafficAlertandCollisionAvoidanceSystem
VNAV:VerticalNavigation
VOR:VeryHighFrequencyOmnidirectionalRadioRange
Page 125
110
AppendixB
BriefingMaterials
ThefollowingareslidesfromaPowerPointpresentation.Thecrewsthat
receivedtrainingwereshowntheseslidesaspartoftheclassroombriefing.
Page 130
115
AppendixC
GradeSheetsandSurvey
Note:Thegradesheetsandsurveyarepresentedastheywereusedbythe
investigatorduringthedatacollectionprocess.Theyhaveonlybeenformattedtofit
withinthepagemarginsofthispaper.
Page 135
120
AppendixD
SimulatorSetup
Thissectiondescribesthesimulatorsetupforeachtypeofsimulatorusedinthis
study.EachsimulatorwasanFAAapprovedLevel-Dfullflightsimulator.Thesetupin
eachsimulatorwasdifferent,andtheselectionsrequiredtoachievethefailureswere
alsodifferent.Crewmembersflewthesimulatorfromtheirrespectiveseat(Captainor
FirstOfficer)andwerebriefedtotreatthesimulatorastheywouldanactualflight.
Simulator LowAltitude HighAltitude Notes
A320 4500–Fuel
15NMdoglegtofinal
35,000–cruise
Failall2airdatasources(CaptandFO).
AircraftgoestoALTlawinthehighaltitudescenario
B737 4300–Fuel
15NMdoglegtofinal
35,000–cruise
Fail3airdatasources(Capt,FO,Stby)
B747 19,000Fuel
15NMdoglegtofinal
35,000–cruise
Fail2airdatasources
B757 6000–Fuel
15NMdoglegtofinal
35,000–cruise
Mach/ASunreliable
B777 12000–Fuel
15NMdoglegtofinal
35,000–cruise
Mach/ASunreliable
Enginefire
HighAlt–uselessonplan14A
Page 136
121
DYLINFourArrivalKEWR(Jeppesen,2016).
Page 137
122
ILS4RatKEWR(Jeppesen,2016)
Page 138
123
AppendixE
AdditionalStatisticalTests
LeveneTestofHomogeneityofVariancesHighAltitude
HighAltitudeScenario
LeveneStatistic df1 df2 Sig.
.674 1 18 .422
RegressionANOVAHighAltitudeFactors
Model SumofSquares df MeanSquare F Sig.
1 Regression 5.568 1 5.568 12.250 .003b
Residual 8.182 18 .455
Total 13.750 19
2 Regression 10.748 5 2.150 10.024 .000c
Residual 3.002 14 .214
Total 13.750 19
a. DependentVariable:HighAltitudeScenario
b. Predictors:(Constant),CrewTrainingReceived
c. Predictors:(Constant),CrewTrainingReceived,Problemdiagnosis,Altitudecontrol,Rollcontrol,Pitchcontrol
Page 139
124
Regressio
nCo
efficients H
ighAltitud
e.
Mod
el
Unstan
dardize
dCo
efficients
Stan
dardize
d
Coefficients
tSig.
95.0%Con
fiden
ceIntervalfo
rB
Correlations
Colline
arity
Statistics
BStd.Error
Beta
LowerBou
nd
Uppe
rBou
nd
Zero-order
Partial
Part
Tolerance
VIF
1(Con
stan
t)2.66
7.225
11
.866
.000
2.19
53.13
9
Crew
Training
Received
1.06
1.303
.636
3.50
0.003
.424
1.69
7.636
.636
.636
1.00
01.00
0
2(Con
stan
t).165
.796
.207
.839
-1.542
1.87
1
Crew
Training
Received
.099
.356
.059
.278
.785
-.6
65
.862
.636
.074
.035
.342
2.92
6
Prob
lem
diagno
sis
.440
.124
.591
3.56
1.003
.175
.706
.812
.689
.445
.565
1.76
9
Pitchcontrol
.009
.304
.012
.031
.976
-.6
42
.660
.759
.008
.004
.101
9.88
3
Rollcontrol
.205
.246
.196
.834
.418
-.3
22
.732
.590
.218
.104
.283
3.53
1
Altitud
e
control
.186
.230
.204
.810
.432
-.3
07
.680
.679
.212
.101
.245
4.08
3
Page 140
125
Colline
arity
DiagnosticsaHighAltitud
e
Mod
elDimen
sion
Eigenvalue
Co
ndition
Inde
xVa
rianceProp
ortio
ns
(Con
stan
t)Crew
Training
Received
Prob
lem
diagno
sis
Pitchcontrol
Rollcontrol
Altitud
econtrol
11
1.74
21.00
0.13
.13
2.258
2.59
6.87
.87
21
5.59
81.00
0.00
.00
.00
.00
.00
.00
2.304
4.29
0.01
.40
.00
.00
.00
.00
3.049
10
.693
.03
.01
.74
.00
.04
.00
4.029
14
.012
.08
.19
.08
.04
.05
.32
5.016
18
.762
.32
.39
.00
.16
.18
.13
6.004
35
.529
.56
.01
.17
.80
.74
.55
a.De
pend
entV
ariable:HighAltitud
eScen
ario
Page 141
126
Levene'sTestofEqualityofErrorVariancesaLowAltitude
DependentVariable:LowAltitudeScenario
F df1 df2 Sig.
.450 1 18 .511
Teststhenullhypothesisthattheerrorvarianceofthedependentvariableisequal
acrossgroups.
a. Design:Intercept+CrewTraining
ANOVAaLowAltitude
Model SumofSquares df MeanSquare F Sig.
1 Regression 6.050 1 6.050 10.371 .005b
Residual 10.500 18 .583
Total 16.550 19
2 Regression 15.367 6 2.561 28.133 .000c
Residual 1.183 13 .091
Total 16.550 19
a. DependentVariable:LowAltitudeScenario
b. Predictors:(Constant),CrewTrainingReceived
c. Predictors:(Constant),CrewTrainingReceived,Approachandlanding,Time
Management,Checklistprocedures,FuelManagement,Missedapproach
Page 142
127
RegressionCoefficients
aLowAltitude
Model
UnstandardizedCoefficients
Standardized
Coefficients
tSig.
95.0%ConfidenceIntervalforB
Correlations
CollinearityStatistics
B
Std.Error
Beta
LowerBound
UpperBound
Zero-order
Partial
Part
Tolerance
VIF
1
(Constant)
2.600
.242
10.765
.000
2.093
3.107
Crew
Training
Received
1.100
.342
.605
3.220
.005
.382
1.818
.605
.605
.605
1.000
1.000
2
(Constant)
.088
.371
.238
.816
-.714
.891
Crew
Training
Received
.084
.194
.046
.434
.672
-.335
.504
.605
.119
.032
.483
2.070
Missed
approach
.449
.140
.531
3.201
.007
.146
.752
.847
.664
.237
.200
5.008
Checklist
procedures
.137
.115
.145
1.186
.257
-.113
.386
.773
.312
.088
.368
2.718
Tim
eM
agmt
.427
.118
.476
3.610
.003
.172
.683
.735
.708
.268
.316
3.165
FuelMgmt
-.109
.167
-.106
-.652
.526
-.469
.252
.759
-.178
-.048
.208
4.810
Approach
andlanding
.103
.124
.101
.827
.423
-.165
.370
.678
.224
.061
.366
2.736
Page 143
128
CollinearityDiagnosticsaLowAltitude
Mode
l
Dim
ensionEigenvalueCondition
Index
VarianceProportions
(Constant)Crew
Training
Received
Missed
approach
Checklist
procedures
Tim
e
Management
Fuel
Management
Approach
andlanding
1
1
1.707
1.000
.15
.15
2
.293
2.414
.85
.85
2
1
6.441
1.000
.00
.00
.00
.00
.00
.00
.00
2
.367
4.189
.01
.55
.00
.00
.00
.00
.00
3
.084
8.783
.01
.01
.11
.00
.14
.01
.06
4
.044
12.141
.10
.02
.03
.36
.05
.07
.01
5
.040
12.622
.35
.23
.06
.18
.10
.01
.03
6
.016
20.048
.15
.08
.31
.00
.12
.10
.86
7
.008
28.124
.38
.11
.48
.46
.58
.81
.04
a.DependentVariable:LowAltitudeScenario
Page 144
129
Levene'sTestofEqualityofErrorVariancesaCombined
DependentVariable:LowandHighAltitudeCombined
F df1 df2 Sig.
.046 1 38 .831
Teststhenullhypothesisthattheerrorvarianceofthedependentvariableisequal
acrossgroups.
a. Design:Intercept+CrewTraining
Page 145
130
SurveyCorrelatio
nsNoTraining
Lowand
High
Altitud
eCo
mbine
dIkno
wth
eprop
er
pitchan
dpo
wer
settings.
Hand
flyingbelow
10
,000
feet
Chairflysc
enarios
tohelpde
term
ine
courseso
faction
Comfortflyingra
w
rata
Ofte
npracticeraw
datask
ills
Autopilotu
sage
above10
00fe
et
NoCrew
Training
Pearson
Correlation
1.290
.101
.091
.081
.194
.373
Sig.(2
-tailed)
.228
.681
.712
.742
.425
.128
N
19
19
19
19
19
19
18
Ikno
wth
eprop
erpitchan
dpo
werse
ttings.
Pearson
Correlation
.290
1
.141
.193
.247
.022
.000
Sig.(2
-tailed)
.228
.566
.428
.309
.929
1.00
0N
19
19
19
19
19
19
18
Hand
flying
below10,00
0feet
Pearson
Correlation
.101
.141
1
.218
.411
.534
* -.2
37
Sig.(2
-tailed)
.681
.566
.370
.080
.019
.343
N
19
19
19
19
19
19
18
Chairfly
scen
ariostohe
lp
determ
ine
courseso
faction
Pearson
Correlation
.091
.193
.218
1
.217
.223
.248
Sig.(2
-tailed)
.712
.428
.370
.373
.359
.321
N
19
19
19
19
19
19
18
Comfortflying
rawra
ta
Pearson
Correlation
.081
.247
.411
.217
1
.497
* .154
Sig.(2
-tailed)
.742
.309
.080
.373
.030
.543
N
19
19
19
19
19
19
18
Ofte
npractice
rawdataskills
Pearson
Correlation
.194
.022
.534
* .223
.497
* 1
.279
Sig.(2
-tailed)
.425
.929
.019
.359
.030
.261
N
19
19
19
19
19
19
18
Autopilotu
sage
above10
00fe
et
Pearson
Correlation
.373
.000
-.2
37
.248
.154
.279
1
Sig.(2
-tailed)
.128
1.00
0.343
.321
.543
.261
N
18
18
18
18
18
18
18
*.Correlatio
nissig
nifican
tatthe
0.05level(2-tailed).
Page 146
131
SurveyCorrelatio
nswith
Training
Lowand
High
Altitud
eCo
mbine
dIkno
wth
eprop
er
pitchan
dpo
wer
settings.
Hand
flyingbelow
10
,000
feet
Chairflysc
enarios
tohelpde
term
ine
courseso
faction
Comfortflyingra
w
rata
Ofte
npracticeraw
datask
ills
Autopilotu
sage
above10
00fe
et
Crew
Training
Pearson
Correlation
1-.1
67
-.042
-.3
06
-.077
.264
-.2
04
Sig.(2
-tailed)
.470
.857
.177
.739
.248
.375
N
21
21
21
21
21
21
21
Ikno
wth
eprop
erpitchan
dpo
werse
ttings.
Pearson
Correlation
-.167
1
.427
-.1
90
.283
.190
-.1
12
Sig.(2
-tailed)
.470
.053
.408
.214
.409
.629
N
21
21
21
21
21
21
21
Hand
flying
below10,00
0feet
Pearson
Correlation
-.042
.427
1
.270
.424
.694
**
.016
Sig.(2
-tailed)
.857
.053
.237
.055
.000
.946
N
21
21
21
21
21
21
21
Chairfly
scen
ariosto
helpdetermine
courseso
factio
n
Pearson
Correlation
-.306
-.1
90
.270
1
-.085
.092
.103
Sig.(2
-tailed)
.177
.408
.237
.713
.691
.656
N
21
21
21
21
21
21
21
Comfortflying
rawra
ta
Pearson
Correlation
-.077
.283
.424
-.0
85
1.537
* .117
Sig.(2
-tailed)
.739
.214
.055
.713
.012
.615
N
21
21
21
21
21
21
21
Ofte
npractice
rawdataskills
Pearson
Correlation
.264
.190
.694
**
.092
.537
* 1
.052
Sig.(2
-tailed)
.248
.409
.000
.691
.012
.824
N
21
21
21
21
21
21
21
Autopilotu
sage
above10
00fe
et
Pearson
Correlation
-.204
-.1
12
.016
.103
.117
.052
1
Sig.(2
-tailed)
.375
.629
.946
.656
.615
.824
N
21
21
21
21
21
21
21
**.C
orrelatio
nissig
nifican
tatthe
0.01level(2-tailed).
*.Correlatio
nissig
nifican
tatthe
0.05level(2-tailed).
Page 147
132
References
BAE.(2012).FinalReportAF447.Ministèredel’Écologie,duDéveloppementdurable,
desTransportsetduLogement.Paris:Bureaud’Enquêtesetd’Analyses.
Billings,C.(1996).Human-CenteredAviation:PrinciplesandGuidelines.NASA,Ames
ResearchCenter.MoffettField:NASATechnicalMemorandum11038.
Boeing.(2012).StatisticalSummaryofCommercialJetAirplaneAccidents.Boeing
Company.Seattle:Boeing.
Casner,S.M.,Geven,R.W.,&Willliams,K.T.(2012).TheEffectivenessofAirlinePilot
TrainingforAbnormalEvents.TheJournaloftheHumanFactorsand
Ergonomics.,55(3),477-485.
Causse,M.,Dehais,F.,Peran,P.,&Pastor,J.(2013,July).Theeffectsofemotiononpilot
decision-making:Aneuroergonomicapproachtoaviationsafety..ransportation
ResearchPartC:EmergingTechnologies,33,272-281.
Coe,R.(2002).It'stheEffectSize,Stupid.Whateffectsizeisandwhyitisimportant..
AnnualConferenceoftheBritishEducationalResearchAssociation,Universityof
Exeter.Exeter:UniversityofDurham.
Page 148
133
Compton,R.,Banich,M.,Mohanty,A.,Milham,M.,Herrington,J.,Miller,G.,...Heller,
W.(2003).Payingattentiontoemotion:AnFMRIinvestigationofcognitiveand
emotionaltasks.Cognitive,Affective,andBehavioralNeuroscience3,81-96.,3,
81-96.
Cummings,M.(2016).AutomationBiasinIntelligentTimeCriticalDecisionSupport
Systems.RetrievedJanuary12,2016,fromAmericanInstituteofAeronauticsand
Astronautics:
http://web.mit.edu/aeroastro/labs/halab/papers/CummingsAIAAbias.pdf
Davis,M.(1986).Pharmacologicalandanatomicalanalysisoffearconditioningusingthe
fearpotentiatedstartleparadigm.BehaviouralNeuroscience,100,814-824.
Davis,M.(1992).Theroleoftheamygdalainfearandanxiety.AnnualReviewof
Neuroscience,15,353-375.
Diamond,L.,&Aspinwill,L.G.(2003).Emotionalregulationacrossthelifespan:An
integrativeapproachemphasizingself-regulation,positiveaffect,anddyadic
process..MotivationandEmotion,27,125-156.
Diehl,A.(1991).TheEffectivenessofTrainingProgramsforPreventingAircrewError.
SixthInternationalSymposiumonAviationPsychology(pp.640-655).Columbus:
TheOhioStateUniversity.
Eaton,R.(1984).Neuralmechanismsofstartlebehavior.NewYork,NY:PlenumPress.
Page 149
134
Endsley,M.(2006).SituationAwareness.HandbookofHumanFactorsandErgonomics.
Hoboken,NJ:JohnWileyandSons.
Ericcsson,K.(1996).Theacquisitionofexpertperformance:Anintroductiontosomeof
theissues.Theroadtoexcellence:Theacquisitionofexpertperformanceinthe
artsandsciences,sports,andgames.Mahwah,NJ:Erlbaum.
Gillen,M.(2014).PilotingSkillsUseThemorLoseThem.LufthansaFirstAnnualHuman
FactorsConference.Frankfurt:Lufthansa.
Hilscher,M.,Breiter,E.,&Kochan,J.(2012).FromtheCouchtotheCockpit:
PsychologicalConsiderationsDuringHigh-performanceFlightTraining.Retrieved
January20,2015,fromhttp://apstraining.com/wp-
content/uploads/Psychological-Considerations-During-High-Performance-Flight-
Training-2005-Hilscher-Breiter-Kochan.pdf
Ippel,M.J.(1987).CognitiveTaskLoadandTestPerformance.RetrievedFebruary1,
2016,fromhttp://www.ijoa.org/imta96/paper52.html
Isaac,A.(2012).EmergenciesandUnusualSituations–WhoseWorldView.Retrieved
January30,2016,fromHindSight15:
https://www.eurocontrol.int/sites/default/files/publication/files/120611-
hs15.pdf
Jeppesen.(2016).Charts.Denver,CO.
Page 150
135
Kaempf,G.,&Klein,G.(1994).AeronauticlDecisionMaking:Thenextgeneration.
AviationPsycholoyinPractice,223-254.
Klein,G.(1993).Arecognition-primeddecisionmodelofrapiddecisionmaking.Decision
makinginaction:Modelsandmethods.Norwood,NJ:Ablex.
Klein,G.A.,Calderwood,R.,&Macgregor,D.(1989).CriticalDecisionMethodfor
ElicitingKnowledge.IEEETransactionsonSystems,ManandCybernetics,19(3),
462-472.
Kranse,L.,&vanderSchaaf,T.(2001).RecoveryFromFailuresinChemicalProcess
Industry.5(3),199-211.InternationalJournalofCognitiveErgonomics,5(3),199-
211.
Landis,C.,&Hunt,W.(1939).Thestartlepattern.Oxford:FarrarandRinehart.
Lang,P.,Bradley,M.,&Cuthbert,B.(1990).Emotion,attention,andthestartlereflex.
PsychologicalReview,97,377-395.
Layton,C.,Smith,P.J.,&McCoy,C.E.(1994).Designofcooperativeproblem-solving
systemforen-routeflightplanning:anempiricalevaluation.HumanFactors,36,
94-119.
LeDoux,J.(1996).TheEmotionalBrain.NY,NY:SimonandSchuster.
LeDoux,J.E.(2000).Emotioncircuitsinthebrain..AnnualReviewofNeuroscience,23,
155-184.
Page 151
136
Lieberman,D.A.(2012).HumanLearningandMemory.Cambridge,MA:University
Press.
Martin,W.,Murray,P.S.,&Bates,P.(2012).TheEffectsofStartleonPilotsDuring
CriticalEvents:ACaseStudyAnalysis.Brisbane,Australia:GriffithUniversity.
McKinney,E.H.,&Davis,K.J.(2003).Effectsofdeliberatepracticeondecision
performance.HumanFactors:TheJournaloftheHumanFactorsandErgonomics
Society,45(3),436-444.
McKinney,E.,&Davis,K.(2003).EffectsofDeliberatePracticeonCrisisDecision
Performance.HumanFactors,45,436-444.
Merk,R.(2009).Acomputationalmodelonsurpriseanditseffectsonagentbehaviorin
simulatedenvironments(TechnicalReportNLR-TP-2009-637).Amsterdam,
Netherlands:.NationalAerospaceLaboratory,TechnicalReportNLR-TP-2009-
637,Amsterdam.
MM43.(2011).RetrievedMarch30,2016,fromProfessionalPilotRumorNetwork:
http://www.pprune.org/tech-log/460625-af-447-thread-no-6-a-31.html(FDR
analysispic).AccessedMarch30,2016
Morris,C.H.,&Leung,Y.(2006).PilotMentalWorkload:HowWellDoPilotsReally
Perform?Ergonomics,49(15),1581-1596.
Page 152
137
Mosier,K.,Palmer,E.,&Degani,A.(1992).Electronicchecklists:Implicationsfor
decisionmaking.ProceedingsoftheHumanFactorsSociety36thAnnualMeeting
(pp.7-11).SantaMonica:HumanFactorsSociety.
Mosier,K.,Skitka,L.,Heers,S.,&Burdick,M.(1997).AutomationBias:DecisionMaking
andPerformanceinHigh-TechCockpits.InternationalJournalofAviation
Psychology,8(1),47-63.
Muthard,E.,&Wickens,C.(2002).FactorsthatMediateFlightPlanMonitoringand
ErrorsinPlanRevision:AnExaminationofPlanningunderAutomatedConditions.
UniversityofIllinois,InstituteofAviation,Savoy.Retrievedfrom
http://www.aviation.illinois.edu/avimain/papers/research/pub_pdfs/isap/mutwi
c.pdf
Neisser,U.(1976).CognitionandReality.SanFrancisco,CO:W.H.FreemanandCo..
Nikolic,M.,&Sarter,N.(2007).FlightDeckDisturbanceManagement:ASimulatorStudy
ofDiagnosisandRecoveryfromBreakdownsinPilot-AutomationCoordination.
TheJournaloftheHumanFactorsandErgonomicsSociety,49,553-563.
Norman,D.,&Bobrow,D.(1975).Ondata-limitedandresource-limitedprocesses.
CognitivePsychology,7,44-64.
Page 153
138
NTSB.(2010).AircraftAccidentReport,LossofControlonApproachColganAir,Inc.
OperatingasContinentalConnectionFlight3407BombardierDHC-8-400,
N200WQClarenceCenter,NewYorkFebruary12,2009.Washington,D.C.:
NationalTransportationSafetyBoard.
Orasanu,J.,&Martin,L.(1998).ErrorsinAviationDecisionMaking:AFactorin
AccidentsandIncidents.WorkshoponHumanError,SafetyandSystem
Development.Seattle.
Parasuraman,G.,&Manzey,D.(2010).ComplacencyandBiasinHumanUseof
Automation:AnAttentionalIntegration.HumanFactors,52(3),381-410.
Plant,K.,&Stanton,N.(2013).Whatisonyourmind?Usingtheperceptualcyclemodel
andcriticaldecisionmethodtounderstandthedecision-makingprocessinthe
cockpit.Ergonomics,56(8),1232-1250.
Rasmussen,J.(1983).Skills,Rules,andKnowledge;Signals,Signs,andSymbols,and
OtherDistinctionsinHumanPerformanceModels.IEEETransactionsonSystems,
Man,andCybernetics,13(3),257-266.
Richman,H.,Gobet,F.,Stazewski,J.,&Simon,H.(1996).Perceptualandmemory
processesinacquisitionofexpertperformance:TheEPAMmodel.Theacquisition
ofexpertperformanceintheartsandsciences,sports,andgames.Mahwah,NJ:
Erlbaum.
Page 154
139
Salmon,P.M.,Stanton,N.,Walker,G.,Baber,C.,Jenkins,D.,McMaster,R.,&Young,M.
(2008).Whatreallyisgoingon?Reviewofsituationawarenessmodelsfor
individualsandteams.TheoreticalIssuesinErgonomicsScience,9(4).
Sarter,N.,&Woods,D.(1993).CognitiveEngineeringinAerospaceApplication;Pilot
InteractionwithCockpitAutomation.NASA,NASAAmesResearchCenter,
MoffettField.
Shappell,S.,&Wiegmann,D.(2009).AMethodologyforAssessingSafetyPrograms
TargetingHumanErrorinAviation.InternationalJournalofAviationPsychology,
19(3),252-269.
Simons,R.C.(1996).Boo!:Culture,experience,andthestartlereflex.Oxford:Oxford
UniversityPress..
Smith,K.,&Hancock,P.(1995).SituationAwarenessisAdaptive,ExternallyDirected
Consciousness.HumanFactors,37(1),137-148.
Stanton,N.A.,Rafferty,L.,Salmon,P.,Revell,K.,McMaster,R.C.-D.,&Cooper-
Chapman,C.(2010).DistributedDecisionMakinginMultihelicopterTeams:Case
StudyofMissionPlanningandExecutionFromaNon-CombatantEvacuation
OperationTrainingScenario.JournalofCognitiveEngineeringandDecision
Making,4(4),328-353.
Thackray,R.,&Touchstone,R.(1970).Recoveryofmotorperformancefollowingstartle.
PerceptualMotorSkills,30,279-292.
Page 155
140
UnitedAirlines.(2016).FlightOperationsManual.Denver,CO:UnitedAirlines.
Vlasek,M.(1969).Effectofstartlestimulionperformance.AerospaceMedicine,40,124-
128.
Whalen,P.,&Phelps,A.(2009).Thehumanamygdala.NewYork:GuidefordPress.
Wickens,C.,&Flach,J.(1998).Informationprocessing.HumanFactorsinAviation..(E.
E.L.Wiener,&D.Nagel,Eds.)London,UK:London:AcademicPressLimited.
Wiener,E.(1985).Humanfactorsofcockpitautomation:Afieldstudyofflightcrew
transition.NASA,NASAAmesResearchCenter.MoffettField:NASAAmes.
Woodhead,M.(1969).Performingavisualtaskinthevicinityofreproducedsonicbangs.
JournalofSoundVibration,9,121-125.
Woodhead,M.M.(1959).Effectofbriefnoiseondecisionmaking.JournalofThe
AcousticSocietyofAmerica,31,1329-1331.
Yeomans,J.,&Frankland,P.(1996).Theacousticstartlereflex:Neuronsand
connections.BrainResearchReviews,21,301-314.