Individual Differences in the Effect of Vertical Separation on Conflict Detection in Air Traffic Control

This article was downloaded by: [UQ Library]On: 28 August 2013, At: 21:08Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH,UK

The International Journal ofAviation PsychologyPublication details, including instructions forauthors and subscription information:http://www.tandfonline.com/loi/hiap20

Individual Differences in theEffect of Vertical Separation onConflict Detection in Air TrafficControlStéphanie Stankovic a , Shayne Loft b , Esa Rantanenc & Nicolas Ponomarenko da School of Psychology, The University ofQueensland, and National ICT, Australiab School of Psychology, The University of WesternAustralia, Crawley, Australiac Department of Psychology, Rochester Institute ofTechnology, Rochester, New York, USAd French Civil Aviation Authority, FrancePublished online: 03 Oct 2011.

To cite this article: Stphanie Stankovic , Shayne Loft , Esa Rantanen & NicolasPonomarenko (2011) Individual Differences in the Effect of Vertical Separationon Conflict Detection in Air Traffic Control, The International Journal of AviationPsychology, 21:4, 325-342, DOI: 10.1080/10508414.2011.606744

To link to this article: http://dx.doi.org/10.1080/10508414.2011.606744

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all theinformation (the “Content”) contained in the publications on our platform.However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness,or suitability for any purpose of the Content. Any opinions and views

expressed in this publication are the opinions and views of the authors, andare not the views of or endorsed by Taylor & Francis. The accuracy of theContent should not be relied upon and should be independently verified withprimary sources of information. Taylor and Francis shall not be liable for anylosses, actions, claims, proceedings, demands, costs, expenses, damages,and other liabilities whatsoever or howsoever caused arising directly orindirectly in connection with, in relation to or arising out of the use of theContent.

This article may be used for research, teaching, and private study purposes.Any substantial or systematic reproduction, redistribution, reselling, loan,sub-licensing, systematic supply, or distribution in any form to anyone isexpressly forbidden. Terms & Conditions of access and use can be found athttp://www.tandfonline.com/page/terms-and-conditions

Dow

nloa

ded

by [

UQ

Lib

rary

] at

21:

08 2

8 A

ugus

t 201

3

THE INTERNATIONAL JOURNAL OF AVIATION PSYCHOLOGY, 21(4), 325–342Copyright © 2011 Taylor & Francis Group, LLCISSN: 1050-8414 print / 1532-7108 onlineDOI: 10.1080/10508414.2011.606744

Individual Differences in the Effect ofVertical Separation on Conflict Detection in

Air Traffic Control

Stéphanie Stankovic,1 Shayne Loft,2 Esa Rantanen,3

and Nicolas Ponomarenko4

1School of Psychology, The University of Queensland, andNational ICT, Australia

2School of Psychology, The University of Western Australia, Crawley, Australia3Department of Psychology, Rochester Institute of Technology,

Rochester, New York4French Civil Aviation Authority, France

This research examined individual differences in judgments of the risk of aircraftseparation violation. Fourteen controllers were asked to judge the risk of conflict foraircraft pairs varying in geometry and vertical separation. A cluster analysis revealedindividual differences in how judgments of conflict risk changed with increased ver-tical separation. There were no individual differences in conflict risk judgmentswhen vertical separation was 0 ft. However, as vertical separation increased to2,000 ft and 4,000 ft, some controllers made progressively lower judgments of con-flict risk than others. These findings have implications for the design of automationtools and for training.

In efforts to improve the capacity of air traffic management systems, human per-formance models can be used to anticipate the effects of increased workloadand new automation on controller performance and system safety (Corker, Gore,Fleming, & Lane, 2000; Ravinder, Remington, & Lee, 2005). Building compu-tational modules that predict how expert controllers determine if a conflict existsbetween two aircraft as well as the subsequent likelihood that they will interveneto assure aircraft separation are central to the development of such performance

Correspondence should be sent to Stéphanie Stankovic, Department of Psychology, UniversitéParis Ouest Nanterre La Defense, 200 Avenue de la Républic, 92001 Nanterre Cedex, France. E-mail:[email protected]

Dow

nloa

ded

by [

UQ

Lib

rary

] at

21:

08 2

8 A

ugus

t 201

3

326 STANKOVIC ET AL.

models. Conflict in this context is defined as aircraft coming closer to each otherthan prescribed separation minima (both horizontal and vertical) should they con-tinue on their respective trajectories. Conflict detection requires controllers tocompare the relative trajectories of aircraft to assess whether they will violate min-imum lateral and vertical separation standards simultaneously in the future. Whenconflicts are detected, controllers develop and implement solutions (interventions)to assure aircraft separation. As controllers make conflict detection judgments andconflict resolution decisions under conditions of uncertainty and time pressure,near misses and midair collisions remain a distinct possibility in all areas of avia-tion (e.g., the tragic midair collision between Bashkirian Airlines Flight 2937 andDHL Flight 611 over the Swiss–German border in 2002; Johnson, Kirwan, Licu,& Stastny, 2009).

Many empirical studies have examined factors that influence the accuracy andtimeliness of conflict detection (e.g., Bisseret, 1981; Boag, Neal, Loft, & Halford,2006; Galster, Duley, Masalonis, & Parasurman, 2001; Leplat & Bisseret, 1966;Metzger & Parasuraman, 2001; Rantanen & Nunes, 2005; Remington, Johnston,Ruthruff, Gold, & Romera, 2000). Recent efforts have been directed toward for-mally describing the cognitive processes that underlie conflict detection (Loft,Bolland, Humphreys, & Neal, 2009; Neal & Kwantes, 2009; Stankovic, Raufaste,& Averty, 2008). This recent literature has several limitations, however. First,there is considerable disagreement regarding the effect of vertical separationbetween aircraft on conflict judgment (e.g., Loft et al., 2009; Stankovic et al.,2008). Second, current theories and models rest on data from quite limited airtraffic scenarios, often with aircraft only presented to participants at level flight(e.g., Neal & Kwantes, 2009) or on trajectories converging at limited angles (e.g.,Neal & Kwantes, 2009; Stankovic et al., 2008). Third, studies testing theories andmodels of conflict detection have only reported the “average” responses from con-trollers without explicitly considering the role of individual differences betweencontrollers (e.g., Loft et al., 2009). In this study we directly address these threeissues by using a wide range of air traffic scenarios to examine the effect of ver-tical separation on conflict risk judgments in air traffic control (ATC) from anindividual differences perspective.

THEORETICAL APPROACHES TO CONFLICT DETECTION

Conflict detection requires controllers to search ATC displays for aircraft pairs inpotential conflict (Galster et al., 2001; Metzger & Parasuraman, 2001; Remingtonet al., 2000). The focus of current theoretical models of conflict detection, andof this study, concerns how controllers determine the risk of conflict for specificaircraft pairs that have been selectively attended. Consider two aircraft flying atthe same altitude on converging courses. Determining whether these aircraft will

Dow

nloa

ded

by [

UQ

Lib

rary

] at

21:

08 2

8 A

ugus

t 201

3

INDIVIDUAL DIFFERENCE IN CONFLICT RISK JUDGMENT 327

violate lateral separation requires the integration of speed and distance informa-tion to predict the distance or time between the aircraft at the point of intersectionof their trajectories (Law et al., 1993; Loft, Neal, & Humphreys, 2007; Neal &Kwantes, 2009). When aircraft are also changing altitude, the prediction of lossof separation in the future also requires the integration of the vertical speeds andaltitudes of aircraft, and the subsequent computation of whether the differencebetween aircraft altitudes at the time of the position overlap on the lateral plane isbelow a minimum separation (Xu & Rantanen, 2003).

One common approach to examining conflict detection has been to establishstatistical relationships between environmental cues (e.g., lateral separation, ver-tical separation, time to closest point of approach) and conflict risk judgment(Bisantz & Pritchett 2003; Stankovic et al., 2008). For example, Stankovic et al.(2008) used three variables to predict controller judgments of conflict risk: (a)the distance between the crossing point of the aircraft pair trajectories and theclosest aircraft to that point, (b) the distance between the two aircraft at theirclosest (lateral) point of approach, and (c) the lateral distance between the twoaircraft when their growing vertical separation reached 1,000 ft. These variablesaccounted for up to 50% of the variance in conflict risk judgments. One impor-tant finding reported by Stankovic et al. was that the vertical separation betweenaircraft significantly influenced judgment of conflict risk.

A second general approach has been to develop theoretical accounts thatdescribe the process by which environmental cues (e.g., lateral and vertical sep-aration) are used to make judgments of conflict risk (Loft et al., 2009; Neal &Kwantes, 2009). For example, the Neal and Kwantes (2009) random-walk modelspecified how individuals use the distance-to-velocity ratios of aircraft to estimatethe difference in the lateral arrival times of pairs of aircraft at common intersectionpoints. This model closely predicted the conflict detection accuracy and responsetimes of naive participants on a laboratory conflict detection task that involvedaircraft simulation in the lateral plane.

Loft et al. (2009) further proposed that controllers use “safety margins” toassure separation between aircraft. These safety margins reflect expectationsregarding likely variation in aircraft trajectory (which can occur due to envi-ronmental factors such as wind shift, aircraft load, airline operating rules, etc.),and also the degree to which controllers are biased to favor safety over accuracy(cf. Bisseret, 1981). Depending on the magnitude of safety margins, controllerpredictions of aircraft position at specific points in the future will be some dis-tance closer or further (or higher or lower in the vertical plane) than the positionspredicted by aircraft state values (and that would be predicted by the Neal &Kwantes [2009] model). To test the claim that controllers apply safety margins,Loft et al. presented controllers pairs of aircraft varying in their lateral separation,vertical separation, and angle of convergence, and asked controllers to rate con-flict risk on a 4-point scale. A computational model that emulated how controllers

Dow

nloa

ded

by [

UQ

Lib

rary

] at

21:

08 2

8 A

ugus

t 201

3

328 STANKOVIC ET AL.

approximate aircraft trajectory closely predicted the conflict risk judgments madeby controllers. This two-parameter computational model assumed that the safetymargins used by controllers reflect both variability along the projected trajectoriesof aircraft and changes in controller decision thresholds.

VERTICAL SEPARATION BETWEEN AIRCRAFT

A key finding reported by Loft et al. (2009) was that there was no variability inrisk judgment as a function of the vertical separation between aircraft. Instead,risk judgment only varied with changes in aircraft lateral separation. To accountfor these data, the Loft et al. model was simplified to assume that controllersalways deem aircraft pairs to be in vertical conflict when aircraft are descendingor climbing through the levels of other aircraft. Loft et al. argued that controllersprefer to intervene to assure aircraft separation when aircraft are climbing throughthe levels of other aircraft to manage their own workload (Loft, Sanderson, Neal,& Mooij, 2007; Sperandio, 1971), and thus that their computational model shouldindeed be able to predict risk judgments without the setting of a vertical separationsafety margin parameter. That controllers’ ratings of scenario complexity typi-cally increase when conflict judgments involve the vertical dimension supportsthis assertion (Boag et al., 2006; Lamoureux, 1999). This increase in complexitystems from the fact that changing difference in altitudes between two aircraft isnot directly visually perceptible, but has to be deduced from the numerical altitudereadouts in aircraft data blocks and combined with estimation of groundspeed andfuture lateral separation.

However, there must be some limit to this. If experts intervened with all aircraftin altitude transition they would be proactive but overloaded with control tasks(see Hollnagel & Woods, 2005; Rouse, Edwards, & Hammer, 1993). In addition,whereas risk judgments made by controllers in the Loft et al. (2009) studies didnot decrease with increased vertical separation (even for presented aircraft pairswith minimum vertical separation of 4,000 ft, which was four times the prescribedstandard of 1,000 ft), Stankovic et al. (2008) reported differences in conflict riskjudgment as a function of vertical separation. Thus, it is important to further inves-tigate the role of aircraft vertical separation on conflict risk judgment. In addition,a limitation of Stankovic et al. was that the effect of vertical separation was onlydemonstrated for pairs of aircraft converging at a very limited set of angles. Inthis study we attempted to replicate this effect within a wider range of air trafficscenarios.

ROLE OF INDIVIDUAL DIFFERENCES

The Loft et al. (2009) model sets the same model parameters for all con-trollers when applying safety margins to approximate aircraft trajectory. Thus, the

Dow

nloa

ded

by [

UQ

Lib

rary

] at

21:

08 2

8 A

ugus

t 201

3

INDIVIDUAL DIFFERENCE IN CONFLICT RISK JUDGMENT 329

model implicitly assumes that no individual differences exist between controllers.Furthermore, Loft et al. did not examine the extent to which these standardizedparameters of the model can predict the performance of individual controllers (seeHorrey, Wickens, & Consalus, 2006). In fact, it is common for researchers pub-lishing in the conflict detection literature to only report the “average” decisionsand performance of controllers and not to consider individual differences (e.g.,Bisseret, 1981; Boag et al., 2006; Galster et al., 2001; Metzger & Parasuraman,2001; Rantanen & Nunes, 2005; Remington et al., 2000). As argued later, thereare strong theoretical and empirical precedents for the prediction that individualdifferences do indeed exist in the manner in which controllers make conflict riskjudgments.

A significant amount of research has focused on identifying domain-specificsituations in which experts outperform trainees and novices and designing tasksto elicit this superior performance (for a review see Ericsson, Charness, Feltovich,& Hoffman, 2006). In addition to such expert–novice differences, it has beenempirically demonstrated that experts can search for solutions in multiple waysthat can produce different acceptable solutions from one expert to another (e.g.,Parasuraman & Riley, 1997; Sanderson, 1989; Torenvliet, Jamieson, & Vicente,2000). Such individual differences in strategy selection are often referred to ascognitive styles (Cegarra & Hoc, 2006; Riding & Rayner, 2000; Sternberg &Zhang, 2001), and reflect differences in the manner in which operators managerisk between their resources (mental cost) and task performance (Hoc, 2005).Parallels can be drawn to ATC where controllers make conflict risk judgmentsunder dynamic conditions with concurrent demands, time pressure, and tacticalconstraints. Controllers maintain acceptable workload levels by using strate-gies that minimize the control activity required to meet their objectives (Loft,Sanderson, et al., 2007; Rouse et al., 1993; Sperandio, 1971). Although weacknowledge that most academics and human factors practitioners would agreethat not all controllers necessarily use the same strategy to detect conflicts, forthe purpose of building theories and computational models that describe the pro-cess underlying conflict detection and that can predict controller performance, itis crucial to provide real data on individual differences among real controllers.

PURPOSE OF THE STUDY

The first goal of this study was to replicate and establish the generality of the effectof vertical separation on conflict risk judgment reported by Stankovic et al. (2008)across a wider range of air traffic scenarios. Our second goal was to examineindividual differences between controllers on conflict risk judgments involvingvertical separation. We used minimum separation standards that corresponded tothe separations used in the approach control center at Toulouse-Blagnac Airport(France); aircraft were in conflict if they would violate minimum lateral (3 nm)

Dow

nloa

ded

by [

UQ

Lib

rary

] at

21:

08 2

8 A

ugus

t 201

3

330 STANKOVIC ET AL.

and vertical (1,000 ft) separation simultaneously in the future. Air traffic scenarioswere developed in consultation with a subject matter expert at Toulouse-BlagnacAirport.

The key experimental manipulation was the vertical separation between aircraft(0 ft, 2,000 ft, or 4,000 ft) at the time when lateral separation between aircraftreached 3 nm. To establish generality, in addition to aircraft on converging tra-jectories (crossing headings), we presented aircraft that followed each other onthe same flight path (same headings), or traveled toward each other on the sameflight path (opposite headings); all of these geometries presented a potential for aconflict in ATC.

METHOD

Participants

Fourteen licensed air traffic controllers (12 men and 2 women) from the Toulouse-Blagnac Airport volunteered to take part in the experiment. Their ages rangedfrom 26 to 59 years (M = 45.50, SD = 10.05). Their average length of experienceas an air traffic controller ranged from 2 to 37 years (M = 21.93, SD = 10.44),and experience since sector certification ranged from 0 to 25 years (M = 8.64,SD = 8.83).

Experimental Stimuli

The scenarios were chosen from historical flight data from the Toulouse-BlagnacAirport. Four independent variables were factorially manipulated across 36 airtraffic scenarios: (a) lateral conflict geometry, (b) vertical separation betweenaircraft, (c) time to lateral separation threshold (3 nm), and (d) groundspeeddifference between the aircraft.

There were three lateral conflict geometries: aircraft heading directly towardeach other on the same flight path (opposite headings), aircraft converging at a 90◦angle on a common intersection point (crossing headings), and situations where afaster aircraft followed a slower aircraft (same headings). For each pair, one air-craft was descending through the level of a second aircraft in level flight. Verticalseparation between the two aircraft was manipulated at three levels (0 ft, 2,000 ft,or 4,000 ft) at the time when the aircraft reached a lateral separation of 3 nm. Thetime that the aircraft would take to reach 3 nm lateral separation was manipulatedat two levels: 3 or 6 min. Finally, the groundspeed difference between the twoaircraft was manipulated at two levels, 20 kt or 40 kt.

A fully factorial design of 3 (conflict geometries) × 3 (vertical separation) ×2 (time to lateral separation threshold) × 2 (groundspeed difference) resulted

Dow

nloa

ded

by [

UQ

Lib

rary

] at

21:

08 2

8 A

ugus

t 201

3

INDIVIDUAL DIFFERENCE IN CONFLICT RISK JUDGMENT 331

in a total of 36 experimental conditions. One trial (i.e., aircraft pair) per condi-tion was presented to the participants in random order. Twelve of these aircraftpairs were conflicts (0 ft vertical separation), and the remaining 24 were notconflicts (2,000 ft or 4,000 ft vertical separation). The dependent variable wasconflict risk judgment, provided on a 12-point scale ranging from 1 (no risk) to12 (extreme risk).

Procedure

The experiment lasted about 30 min. The focus of this study was on the processesthat controllers use to make conflict risk judgments once they have selectivelyattended to an aircraft pair. We used the methodology employed by Bisseret(1981) and Boag et al. (2006) and presented the aircraft pairs involved in eachexperimental trial statically on a sheet of paper. A 3 nm scale marker was pre-sented on this display, and the rate of descent was set at 1,000 ft per minute. Eachaircraft had a data block that displayed its speed in knots, its current flight level(altitude in hundreds of feet), and a sign “=” for the aircraft in level flight or adown arrow followed by a cleared level for the descending aircraft. Two-minutevelocity vectors were displayed for the aircraft. Controllers were instructed tojudge the risk of conflict for each pair of aircraft. Specifically, controllers weretold to judge the risk that a conflict would occur in their sector for each pair ofaircraft (i.e., not the risk of conflict after aircraft pairs had entered hypotheticaladjacent sectors).

RESULTS

Significance was set at an alpha level of .05. Effect sizes for F tests were estimatedusing partial eta-squared (small = .01, medium = .06, large = .14; Cohen, 1988).Effect sizes for t tests were estimated using Cohen’s d (small = .20, medium =.50; large = .80; Cohen, 1988). We analyzed each of the three horizontal conflictgeometries separately because each scenario involved a different angle of conver-gence. Loft et al. (2009) and Neal and Kwantes (2009) demonstrated that anglehas a differential effect on perceived minimum lateral distance and subsequentconflict risk judgment.

To examine the role of individual differences in conflict risk judgment weperformed a cluster analysis. We wanted a cluster solution that would (a) pro-vide maximum differentiation of cases on the conflict risk judgments entered,and (b) provide the greatest interpretability. To achieve this, the cluster anal-ysis was processed in two steps (see Steinley, 2006). First, we constructed ahierarchical cluster solution and examined the joining distances (for details see

Dow

nloa

ded

by [

UQ

Lib

rary

] at

21:

08 2

8 A

ugus

t 201

3

332 STANKOVIC ET AL.

Wishart, 1982) to determine the optimal number of clusters to extract. The hier-archical analysis revealed that two clusters should be extracted. Second, usingthis predetermined number of clusters, we assigned observations to clusters usingthe K-means method. This K-means analysis maximized interclass distances andminimized intraclass distances to formally identify the two clusters of controllers.

Individual Differences

Controllers in the two clusters differed in their overall ratings of conflict risk,F(1, 13) = 21.57, p < .01, ηp

2 = .64. Controllers in Cluster 1 made lower judg-ments of conflict risk (M = 5.76, SD = .47) than controllers in Cluster 2 (M =8.84, SD = 1.57). Controllers in Cluster 1 were older (M = 51.43 years, SD =7.18) than controllers in Cluster 2 (M = 39.57 years, SD = 9.25), t(12) = 2.68,p < .05, d = 1.43. Controllers in Cluster 1 had (numerically) greater total expe-rience (M = 28.86 years, SD = 8.15) than controllers in Cluster 2 (M = 17years, SD = 10.63), t(12) = 1.95, p = .07, d = 1.04. In addition, controllers inCluster 1 had greater sector-specific experience (M = 13.43 years, SD = 10.05)than controllers in Cluster 2 (M = 3.86 years, SD = 3.80), t(12) = 2.36, p <

.05, d = 1.35. These three variables (age, experience, and sector-specific experi-ence) were highly correlated (r ranging from .55–.85), and together accounted forapproximately 50% of the variability in cluster membership.

For each scenario type, we performed a 2 (cluster: Cluster 1 vs. Cluster 2) × 3(vertical distance: 0; 2,000; and 4,000 ft) × 2 (time to lateral separation threshold;3 vs. 6 min) × 2 (velocity difference: 20 vs. 40 kt) mixed analysis of variance,with cluster as the between-subject variable, and vertical separation, time to lat-eral separation threshold, and velocity difference as within-subjects variables. Foreach of the three scenario types, there were no main effects of velocity differ-ence, nor did it interact with other variables. For this reason, we collapsed acrossvelocity when presenting the data. Conflict risk judgments as a function of verticalseparation and time to lateral separation threshold, for each of the three aircraftgeometries and for each cluster, are presented in Figures 1 through 6.

Opposite Headings

Conflict risk judgments as a function of vertical separation and time to lateralseparation threshold for opposite heading scenarios are presented in Figures 1and 2. There was a main effect of vertical separation, F(2, 24) = 29.31, p < .01,ηp

2 = .71, as controllers made lower judgments of conflict risk with increasedvertical separation (0 ft: M = 9.25, SD = 1.76; 2,000 ft: M = 6.80, SD = 3.56;4,000 ft: M = 4.79, SD = 3.54). There was also a significant main effect of clus-ter, F(1, 12) = 22.1, p < .01, ηp

2 = .65, with controllers in Cluster 1 makinglower conflict risk judgments (M = 5.07, SD = 1.60) than controllers in Cluster 2

Dow

nloa

ded

by [

UQ

Lib

rary

] at

21:

08 2

8 A

ugus

t 201

3

INDIVIDUAL DIFFERENCE IN CONFLICT RISK JUDGMENT 333

FIGURE 1 Conflict risk judgments made by controllers in Cluster 1, as a function of verticalseparation and time to lateral separation threshold, for opposite heading scenarios.

FIGURE 2 Conflict risk judgments made by controllers in Cluster 2, as a function of verticalseparation and time to lateral separation threshold, for opposite heading scenarios.

Dow

nloa

ded

by [

UQ

Lib

rary

] at

21:

08 2

8 A

ugus

t 201

3

334 STANKOVIC ET AL.

(M = 8.82, SD = 1.39). These main effects were qualified by an interactionbetween vertical distance and cluster, F(2, 24) = 17.34, p < .01, ηp

2 = .59.No other main effects or interactions approached significance (smallest p = .19).As illustrated in Figures 1 and 2, there was no difference in conflict risk judg-ments between controllers when vertical separation was 0 ft. As vertical separationincreased to 2,000 ft and 4,000 ft, controllers in Cluster 1 made lower judgmentsof conflict risk than controllers in Cluster 2.

Crossing Headings

Conflict risk judgments as a function of vertical separation and time to lateral sep-aration threshold for crossing heading scenarios are presented in Figures 3 and 4.There was a main effect of vertical separation, F(2, 24) = 17.79, p < .01, ηp

2 =.60, as controllers made lower judgments of conflict risk with increased verticalseparation (0 ft: M = 8.87, SD = 2.34; 2,000 ft: M = 6.14, SD = 3.41; 4,000ft: M = 5.30, SD = 3.40). The main effect of cluster was significant, F(1, 12) =40.28, p < .01, ηp

2 = .77, with controllers in Cluster 1 (M = 4.68, SD = .89)making lower judgments of conflict risk than controllers in Cluster 2 (M = 8.87,SD = 1.5). There was also an interaction between vertical distance and cluster,F(2, 24) = 12.30, p < .01, ηp

2 = .51, and between vertical distance and timeto lateral separation threshold, F(2, 24) = 4.55, p < .05, ηp

2 = .27. These maineffects and interactions were qualified by a three-way interaction among clus-ter, vertical separation, and time to lateral separation threshold, F(2, 24) = 3.31,p = .05, ηp

2 = .22. No other main effects or interactions approached significance(smallest p = .21). As illustrated in Figures 3 and 4, there was no differencein conflict risk judgments between controllers when vertical separation was 0 ft.As vertical separation increased to 2,000 ft and 4,000 ft, controllers in Cluster 1made lower judgments of conflict risk than controllers in Cluster 2. This interac-tion between cluster membership and vertical separation was stronger when thetime to lateral separation threshold was shorter.

Same Heading (Overtaking)

Conflict risk judgments as a function of vertical separation and time to lateralseparation threshold for same heading scenarios are presented in Figures 5 and 6.There was a main effect of vertical separation, F(2, 24) = 7.45, p < .01, ηp

2 =.38, as controllers made lower judgments of conflict risk with increased verticalseparation (0 ft: M = 9.28, SD = 1.48; 2,000 ft: M = 7.95, SD = 1.90; 4,000 ft:M = 7.29, SD = 2.83). The effect of cluster was not significant, F(1, 12) = 2.07,p = .18, ηp

2 = .15. There was a main effect of time to lateral separation threshold,F(1, 12) = 24.11, p < .01, ηp

2 = .67, with higher conflict risk judgments whentime to lateral separation threshold was shorter (3 min: M = 9.32, SD = 1.64)

Dow

nloa

ded

by [

UQ

Lib

rary

] at

21:

08 2

8 A

ugus

t 201

3

INDIVIDUAL DIFFERENCE IN CONFLICT RISK JUDGMENT 335

FIGURE 3 Conflict risk judgments made by controllers in Cluster 1, as a function of verticalseparation and time to lateral separation threshold, for crossing heading scenarios.

FIGURE 4 Conflict risk judgments made by controllers in Cluster 2, as a function of verticalseparation and time to lateral separation threshold, for crossing heading scenarios.

Dow

nloa

ded

by [

UQ

Lib

rary

] at

21:

08 2

8 A

ugus

t 201

3

336 STANKOVIC ET AL.

FIGURE 5 Conflict risk judgments made by controllers in Cluster 1, as a function of verticalseparation and time to lateral separation threshold, for same-heading scenarios.

FIGURE 6 Conflict risk judgments made by controllers in Cluster 2, as a function of verticalseparation and time to lateral separation threshold, for same-heading scenarios.

Dow

nloa

ded

by [

UQ

Lib

rary

] at

21:

08 2

8 A

ugus

t 201

3

INDIVIDUAL DIFFERENCE IN CONFLICT RISK JUDGMENT 337

as opposed to longer (6 min: M = 7.02, SD = 2.30). The main effect of verticalseparation was qualified by an interaction between vertical separation and clusterthat approached significance, F(2, 24) = 3.10, p = .06, ηp

2 = .20. No other maineffects or interactions approached significance (smallest p = .10). As illustratedin Figures 5 and 6, there was no difference in conflict risk judgments betweencontrollers when vertical separation was 0 ft. As vertical separation increased to2,000 ft and 4,000 ft, controllers in Cluster 1 made lower judgments of conflictrisk than controllers in Cluster 2. It is clear that the interaction between verticalseparation and cluster was not as large for same-heading scenarios as it was forthe other two scenario types.

DISCUSSION

Our results reveal several key findings, summarized here. Controller judgmentsof conflict risk were lower as the vertical separation between aircraft increased,and this effect was relatively consistent across the different conflict geometries.Controller judgments of conflict risk were higher for aircraft on same headingswhen the time to lateral separation threshold was shorter rather than longer (3vs. 6 min). This bespeaks a sense of urgency and lags in conflict resolutioninterventions in overtaking conflict situations. Crucially, there were significantindividual differences between controllers. One group of controllers (Cluster 1)made lower conflict risk ratings than the other group of controllers (Cluster 2),and the effect of vertical separation on conflict risk judgment was greater forcontrollers in Cluster 1 than for the controllers in Cluster 2. For crossing head-ing scenarios, this interaction between cluster membership and vertical separationwas particularly strong when the time to lateral separation threshold was shorter.Controllers in Cluster 1 were generally more experienced (i.e., total experi-ence, sector-specific experience, older) than controllers in Cluster 2, indicatingthat experienced controllers were less conservative than their less experiencedcounterparts. The theoretical and practical implications of these findings arediscussed next.

Theoretical Implications

The finding that conflict detection risk judgment varied as a function of the futurevertical distance between aircraft is consistent with Stankovic et al. (2008). Weextended Stankovic et al. by demonstrating that vertical separation can also influ-ence conflict judgment for pairs of aircraft on opposite headings and on the sameheadings in addition to aircraft on crossing headings. The second crucial findingwas that we found individual differences in the manner in which vertical sep-aration influenced conflict risk judgments. We found two quite distinct conflict

Dow

nloa

ded

by [

UQ

Lib

rary

] at

21:

08 2

8 A

ugus

t 201

3

338 STANKOVIC ET AL.

detection judgment policies in our population of expert controllers. Controllersin Cluster 1 were characterized by lower risk judgment, which varied as afunction of the vertical separation between aircraft. In contrast, controllers inCluster 2 made comparably higher risk ratings and were less affected by verticalseparation.

To our knowledge, this is the first study to systematically investigate anddemonstrate individual differences among a population of expert controllers. Inthe past, researchers describing the processes underlying conflict detection havenot explicitly considered the presence of individual differences within populationsof expert controllers. For example, although Loft et al. (2009) acknowledged thatcontroller safety margins can change with experience (trainees vs. experts), theydid not consider individual differences between experts. The individual differ-ences reported here challenge the notion that there is a common core of expertisein conflict detection in air traffic control (Boag et al., 2006; Loft et al., 2009;Rantanen & Nunes, 2005), and are in line with individual differences reported inother domains of expertise (e.g., Cegarra & Hoc, 2006; Parasuraman & Riley,1997; Riding & Rayner, 2000; Sanderson, 1989; Sternberg & Zhang, 2001;Torenvliet et al., 2000).

Yet some common characteristics could be identified among the controllers ineach risk judgment cluster. It was noteworthy that controllers in Cluster 1 wereolder and generally more experienced (total and sector specific) than the con-trollers in Cluster 2. Thus, more experienced experts rated scenarios as less risky(i.e., they were less conservative) and were influenced more by changes in verticalseparation than their less experienced counterparts. Interestingly, Bisseret (1981)and Loft et al. (2009) both have reported that expert controllers were more con-servative (i.e., perceive greater conflict risk) than trainees. Thus, although expertsas a group might be more conservative than trainees (Bisseret, 1981; Loft et al.,2009), the current data suggest that this response bias then decreases as licensedcontrollers gain more experience. With increased experience controllers mightbecome more skilled and confident at calculating the future vertical separationbetween aircraft, thereby placing smaller safety margins around the projectedvertical trajectories of aircraft (Loft et al., 2009).

The fact that Loft et al. (2009) found no variation in conflict judgments as afunction of vertical separation is at odds with Stankovic et al. (2008) and the cur-rent data. However, it is important to note that Loft et al. instructed controllersto rate conflict risk by indicating whether they would intervene to assure air-craft separation. They did this because the purpose of the Loft et al. model wasto predict when controllers intervene to change aircraft trajectories, and hencechange future task demands. Such models might one day allow the workloadimposed by ATC daily flight plans to be predicted (see Loft, Sanderson, et al.,2007). Risk rating scales might be more sensitive to detecting differences in con-trollers’ perceptions of conflict status with changes in vertical separation, and

Dow

nloa

ded

by [

UQ

Lib

rary

] at

21:

08 2

8 A

ugus

t 201

3

INDIVIDUAL DIFFERENCE IN CONFLICT RISK JUDGMENT 339

future research needs to establish the conditions under which vertical separationinfluences controller intervention (behavior). The fact remains, however, that theLoft et al. theoretical model claims to be a process model of conflict detection,yet currently assumes no variation in this process with changes in vertical sep-aration, and does not consider the role of individual differences between expertcontrollers.

Practical Implications and Conclusions

In the United States (NextGen) and Europe the key capabilities of future new airtraffic management systems are being rapidly prototyped and developed to dealwith projected increases in air traffic. It is critical that the designs of such sys-tems are informed by an understanding of the processes underlying controllerdecision making. We make two important contributions in this regard. First, wedemonstrated that controllers (but some more than others) are sensitive to changesin vertical separation between aircraft when making judgments of conflict risk.Vertical separation assessment is a complex activity, which relies on expertise,and automated conflict detection tools should be designed to facilitate the percep-tion and calculation of vertical separation. For example, the automated conflictdetection tool (Mid Term Conflict Detection, MTCD) currently implemented inthe European airspace by EUROCONTROL provides such functionality. One ofthe MTCD display windows provides a graphical representation of the air trafficon the vertical plane and displays various data about current and future verti-cal positions of aircraft (e.g., cleared flight level, predicted descent profile, etc.;EUROCONTROL, 2007).

Second, at least with respect to the task of conflict detection, controllers shouldnot be considered as a homogeneous group but rather as a collection of decisionmakers who can be divided into several categories, which reflect various judg-ment policies (Ericsson et al., 2006). On this basis, there needs to be a degree offlexibility in automated conflict detection tools because controllers differ in themanner in which they detect conflicts. For example, Corker, Howard, and Mooij(2005) found that controllers had difficulty posting and sorting entries in the air-craft list provided by the User Request Evaluation Tool (URET) conflict detectionaid, and that controllers rarely checked URET alerts because of the large numberof false alarms generated. It would be useful if the decision thresholds set by suchautomation could be set according to the preferences of individual controllers,perhaps also taking into consideration expected workload (Loft, Sanderson, et al.,2007). In modeling human performance it is also essential to consider the range ofperformance, or in some cases, worst performance as design criteria; performanceimportant to design typically involves the tails of the response distributions ratherthan the average responses, emphasizing the importance of attention to individualdifferences (Wickens, 2001).

Dow

nloa

ded

by [

UQ

Lib

rary

] at

21:

08 2

8 A

ugus

t 201

3

340 STANKOVIC ET AL.

Finally, the data have implications for the design of training programs. Theexperienced experts were making more informed decisions regarding changes inthe vertical separation between aircraft. This result suggests that particular atten-tion in training programs could be devoted to learning how to assess verticaldistance.

In conclusion, we demonstrated individual differences among expert con-trollers in the effect of vertical separation on judgments of conflict risk acrossa range of air traffic scenarios. Two distinct judgment policies were identified,with the more experienced controllers adjusting their judgment of conflict risk asa function of aircraft vertical separation. These data indicate that it is essentialthat researchers and designers of future air traffic management systems explicitlyconsider the role of individual differences in conflict detection.

ACKNOWLEDGMENTS

We thank the air traffic controllers of Toulouse-Blagnac Airport (France) whohave participated in this study.

REFERENCES

Bisantz, A. M., & Pritchett, A. R. (2003). Measuring the fit between human judgments and automatedalerting algorithms: A study of collision detection. Human Factors, 45, 266–280.

Bisseret, A. (1981). Application of signal detection theory to decision making in supervisory control:The effect of the operator’s experience. Ergonomics, 24, 81–94.

Boag, C., Neal, A., Loft, S., & Halford, G. (2006). An analysis of relational complexity in an air trafficcontrol conflict detection task. Ergonomics, 14, 1508–1526.

Cegarra, J., & Hoc, J.-M. (2006). Cognitive styles as an explanation of experts’ individual dif-ferences: A case study in computer-assisted troubleshooting diagnosis. International Journal ofHuman–Computer Studies, 64, 123–136.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ:Lawrence Erlbaum Associates.

Corker, K. M., Gore, B. F., Fleming, K., & Lane, J. (2000, June 13–16). Free flight and the con-text of control: Experiments and modeling to determine the impact of distributed air-ground airtraffic management on safety and procedures. Paper presented at the 3rd USA/Europe Air TrafficManagement Seminar, Naples, Italy.

Corker, K. M., Howard, H., & Mooij, M. (2005). Adaption and adoption of technologies: Examinationof the User Request Evaluation Tool (URET) (Hail Rep. No. 2005-2442-01). San Jose, CA: San JoseState University.

Ericsson, K. A., Charness, N., Feltovich, P. J., & Hoffman, R. R. (2006). The Cambridge handbook ofexpertise and expert performance. Cambridge, UK: Cambridge University Press.

EUROCONTROL. (2007). FASTI baseline description. Brussels, Belgium: EUROCONTROLHeadquarters.

Galster, S. M., Duley, J. A., Masalonis, A. J., & Parasuraman, R. (2001). Air traffic controllerperformance and workload under mature free flight: Conflict detection and resolution of aircraftself-separation. International Journal of Aviation Psychology, 11, 71–93.

Dow

nloa

ded

by [

UQ

Lib

rary

] at

21:

08 2

8 A

ugus

t 201

3

INDIVIDUAL DIFFERENCE IN CONFLICT RISK JUDGMENT 341

Hoc, J. M. (2005). Cooperation between human cognition and technology in dynamic situations. InR. J. Sternberg & D. Preiss (Eds.), Intelligence and technology: Impact of tools on the nature anddevelopment of human abilities (pp. 135–157). Mahwah, NJ: Lawrence Erlbaum Associates.

Hollnagel, E., & Woods, D. D. (2005). Joint cognitive systems: Foundations of cognitive systemsengineering. Boca Raton, FL: CRC Press/Taylor & Francis.

Horrey, W. J., Wickens, C. D., & Consalus, K. P. (2006). Modeling drivers’ visual attention allocationwhile interacting with in-vehicle technologies. Journal of Experimental Psychology: Applied, 12,67–78.

Johnson, C. W., Kirwan, B., Licu, A., & Stastny, P. (2009). Recognition primed decision making andthe organisational response to accidents: Überlingen and the challenges of safety improvement inEuropean air traffic management. Safety Science, 47, 853–872.

Lamoureux, T. (1999). The influence of aircraft proximity data on the subjective mental workload ofcontrollers on the air traffic control task. Ergonomics, 42, 1482–1491.

Law, D. J., Pellegrino, J. W., Mitchell, S. R., Fischer, S. C., McDonalds, T. P., & Hunt, E. B. (1993).Perceptual and cognitive factors governing performance in comparative arrival-time judgments.Journal of Experimental Psychology: Human Perception and Performance, 19, 1183–1199.

Leplat, J., & Bisseret, A. (1966). Analysis of the processes involved in the treatment of informationby the air traffic controller. Controller, 1, 13–22.

Loft, S., Bolland, S., Humphreys, M. S., & Neal, A. (2009). A theory and model of conflict detectionin air traffic control: Incorporating environmental constraints. Journal of Experimental Psychology:Applied, 15, 106–124.

Loft, S., Neal, A., & Humphreys, M. S. (2007). The development of a general associative learn-ing account of skill acquisition in a relative arrival-time judgment task. Journal of ExperimentalPsychology: Human Perception and Performance, 33, 938–959.

Loft, S., Sanderson, P., Neal, A., & Mooij, M. (2007). Modeling and predicting mental workload in enroute air traffic control: Critical review and broader implications. Human Factors, 49, 376–399.

Metzger, U., & Parasuraman, R. (2001). The role of the air traffic controller in future air trafficmanagement: An empirical study of active control versus passive monitoring. Human Factors, 43,519–528.

Neal, A., & Kwantes, P. (2009). A random walk model of conflict detection performance in a simulatedair traffic control task. Human Factors, 51, 164–180.

Parasuraman, R., & Riley, V. (1997). Humans and automation: Use, misuse, disuse and abuse. HumanFactors, 39, 230–253.

Rantanen, E. M., & Nunes, A. (2005). Hierarchical conflict detection in air traffic control. InternationalJournal of Aviation Psychology, 15, 339–362.

Ravinder, U., Remington, R., & Lee, S. (2005, October 10–12). A reactive computational model ofthe en-route controller. In Proceedings of the 2005 IEEE International Conference on Systems, Manand Cybernetics (pp. 1628–1633). Waikoloa, HI.

Remington, R. W., Johnston, J. C., Ruthruff, E., Gold, M., & Romera, M. (2000). Visual search incomplex displays: Factors affecting conflict detection by air traffic controllers. Human Factors, 42,349–366.

Riding, R. J., & Rayner, S. G. (2000). International perspectives of individual differences: Vol. 1.Cognitive styles. Stamford, CT: Ablex.

Rouse, W. B., Edwards, S. L., & Hammer, J. M. (1993). Modeling the dynamics of mental workloadand human performance in complex-systems. IEEE Transactions on Systems Man and Cybernetics,23, 1662–1671.

Sanderson, P. M. (1989). The human planning and scheduling role in advanced manufacturingsystems—An emerging human-factors domain. Human Factors, 31, 635–666.

Sperandio, J.-C. (1971). Variation of operator’s strategies and regulating effects on workload.Ergonomics, 14, 571–577.

Dow

nloa

ded

by [

UQ

Lib

rary

] at

21:

08 2

8 A

ugus

t 201

3

342 STANKOVIC ET AL.

Stankovic, S., Raufaste, E., & Averty, P. (2008). Determinants of conflict detection: A model of riskjudgments in air traffic control. Human Factors, 50, 121–134.

Steinley, D. (2006). K-means clustering: A half-century synthesis. The British Journal of Psychology,59, 1–34.

Sternberg, R. J., & Zhang, L.-F. (2001). Perspectives on thinking, learning, and cognitive styles.Mahwah, NJ: Lawrence Erlbaum Associates.

Torenvliet, G. L., Jamieson, G. A., & Vicente, K. J. (2000). Making the most of ecological interfacedesign: The role of individual differences. Applied Ergonomics, 31, 395–408.

Wickens, C. D. (2001, March 5–8). Attention to safety and the psychology of surprise. Keynote addressat the 11th International Symposium on Aviation Psychology, Columbus, OH.

Wishart, D. (1982). Supplement, CLUSTAN user manual (3rd ed.). Edinburgh, UK: Program LibraryUnit, Edinburgh University.

Xu, X., & Rantanen, E. M. (2003, April 14–17). Conflict detection in air traffic control: A task anal-ysis, a literature review, and a need for further research. Paper presented at the 12th InternationalSymposium on Aviation Psychology, Dayton, OH.

Manuscript first received: June 2010

Dow

nloa

ded

by [

UQ

Lib

rary

] at

21:

08 2

8 A

ugus

t 201

3