EUROPEAN ORGANISATION FOR THE SAFETY OF AIR NAVIGATION EUROCONTROL EXPERIMENTAL CENTRE Gaze analysis and psychophysiological parameters: A tool for the design and the evaluation of Man-Machine Interfaces Feasibility study EEC Report N° 350 Project HRS-Z-05_PHYS Issued: July 2000 The information contained in this document is the property of the EUROCONTROL Agency and no part should be reproduced in any form without the Agency’s permission. The views expressed herein do not necessarily reflect the official views or policy of the Agency. EUROCONTROL
65
Embed
EUROCONTROL EXPERIMENTAL CENTRE EEC Report N° 350 · EUROCONTROL EXPERIMENTAL CENTRE Gaze analysis and psychophysiological parameters: A tool for the design and the evaluation of
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
EUROPEAN ORGANISATIONFOR THE SAFETY OF AIR NAVIGATION
EUROCONTROL EXPERIMENTAL CENTRE
Gaze analysis and psychophysiological parameters: A tool for the design and the evaluation of Man-Machine Interfaces
Feasibility study
EEC Report N° 350
Project HRS-Z-05_PHYS
Issued: July 2000
The information contained in this document is the property of the EUROCONTROL Agency and no part shouldbe reproduced in any form without the Agency’s permission.
The views expressed herein do not necessarily reflect the official views or policy of the Agency.
EUROCONTROL
REPORT DOCUMENTATION PAGE
Reference:EEC Report N° 350
Security Classification:Unclassified
Originator:
EEC - IND
(INDependent Reseach)
Originator (Corporate Author) Name/Location:EUROCONTROL Experimental CentreCentre des Bois des BordesB.P.15F - 91222 Brétigny-sur-Orge CEDEXFRANCETelephone : +33 (0)1 69 88 75 00
Sponsor: Sponsor (Contract Authority) Name/Location:EUROCONTROL AgencyRue de la Fusée, 96B -1130 BRUXELLESTelephone : +32 (0)2 729 90 11
TITLE: GAZE ANALYSIS AND PSYCHOPHYSIOLOGICAL PARAMETERS : A TOOL FOR THE DESIGNAND THE EVALUATION OF MAN-MACHINE INTERFACES
FEASIBILITY STUDY
Authors P. Cabon, B. Farbos
R. MollardLAA, Université R. Descartes Paris V
Date
07/2000
Pages
x + 55
Figures/Photos
26 / 7
Tables
1
Annex
1
References
19
EATCHIP TaskSpecification
-
Project
HRS-Z-05_PHYS
Task No. Sponsor
-
Period
1999
Distribution Statement:(a) Controlled by: Head of IND(b) Special Limitations None(c) Copy to NTIS: Yes
Descriptors (keywords):
ATC, human factors, HMI, gaze analysis, perceptual aspect, information processing, EEG, HRV, workload,stress, fatigue.
Abstract:This study demonstrates the value of gaze analysis with simultaneous physiological data recording for theassessment of the human-computer interface in air traffic control.The literature on visual scanning was reviewed. Two studies using different eye-tracking systems, one head-mounted (I VIEW) and one remote (ASL 504) were conducted at EEC. In each study, a TRACON IIsimulator was used to present controllers with a representative ATC task and a set of relevant electro-physiological measures (EEG, HRV etc..) was employed.This study shows that selected eye-view and physiological parameters may be used together to evaluatestrain, fatigue, operational strategies of controllers in highly interactive situations.Further studies should investigate the effect of changes in the nature and extent of automation of the ATCtask, such electronic stripping, flat displays etc.
This document has been collated by mechanical means. Should there be missing pages, please report to:
GAZE ANALYSIS AND PSYCHOPHYSIOLOGICAL PARAMETERS :
A TOOL FOR THE DESIGN AND THE EVALUATION OF MAN-MACHINE INTERFACES
FEASIBILITY STUDY
By
P. Cabon, B. Farbos, R. Mollard LAA Université René Descartes - Paris V
SUMMARY
This study demonstrates the value of gaze analysis with simultaneous physiological datarecording for the assessment of the human- computer interface in air traffic control.
The literature on visual scanning was reviewed. Two studies using different eye-trackingsystems, one head-mounted (I VIEW) and one remote (ASL 504) were conducted at EEC.In each study, a TRACON II simulator was used to present controllers with a representativeATC task and a set of relevant electro-physiological measures (EEG, HRV etc…) wasemployed.
This study shows that selected eye-view and physiological parameters may be used togetherto evaluate strain, fatigue, operational strategies of controllers in highly interactive situations.
Further studies should investigate the effect of changes in the nature and extent of automationof ATC task, such electronc stripping, flat displays etc.. .
vi
vii
FOREWORD
The need for more ‘objective’ measures of the effects of Air Traffic Control on controllers haslong been apparent. Various studies have been undertaken at EEC on potential methods ofmeasuring these effects (EEC Reports n° 64, 164, 183, 226, 228, 323, 334 and 339).
The LAA (Laboratoire d’Anthropologie Appliquée, Université Paris V René Descartes) underthe direction of Professor Alex Coblentz, has carried out studies of fatigue, sleep disorder andassociated strain among Airline Pilots, Car Drivers and Air Traffic Controllers among others.
We therefore commissioned the LAA to undertake a series of feasibility studies on the use ofCortisol analysis, Electroencephalography, in particular Cortical Evoked Potential and otherrelated methods, which were reported in EEC Report n° 323 and in EEC Note n° 16/98. Thesestudies used a TRACON II autonomous simulator. LAA subsequently undertook the transferof these methods to real-time simulation, as reported in EEC Reports n° 334 and 339.
This report covers further TRACON II studies of eye movement recording methods, operatedin association with electrophysiological studies.
On behalf of the Director EEC, the experimenters for the LAA and the EEC Project Officerswish to thank the controllers and the EEC staff involved for their active and dedicatedco-operation.
Françoise CalooHugh David
EEC Project Officers.
viii
ix
TABLE OF CONTENTS
ABBREVIATIONS............................................................................................................................................... X
1 - INTRODUCTION AND AIMS - .................................................................................................................... 3
2 - LITERATURE REVIEW - ............................................................................................................................. 5
2.1 - THE EYE TRACKING TECHNIQUE -................................................................................................................ 52.2 - DATA PROCESSING METHODS - .................................................................................................................... 6
2.2.1 - Fixation chronology methods - ........................................................................................................... 62.2.2 - Statistical analysis of gaze in a specific area - ................................................................................... 82.2.3 - Analysis of fixation groups - ............................................................................................................... 92.2.4 Weighted Search Area (WSA) ............................................................................................................. 10
2.3 - APPLICATIONS OF VISUAL SCANNING IN ERGONOMICS - ............................................................................ 10
3 - SYNTHESIS OF THE IVIEW STUDY - ..................................................................................................... 14
4 - ASL STUDY - ................................................................................................................................................. 16
4.3. - RESULTS - ................................................................................................................................................ 204.3.1 - Fixation frequency and duration - .................................................................................................... 204.3.2 - Physiological data -.......................................................................................................................... 214.3.3 - Controllers’ orders -......................................................................................................................... 224.3.4 - Perceived workload -........................................................................................................................ 23
5 - ERRORS AND VISUAL SCANNING ANALYZIS - ................................................................................. 24
5. 1 - THE IVIEW SYSTEM - .............................................................................................................................. 255.1.1 - Analyzis of errors - 5 minutes -......................................................................................................... 255.1.2 - Analyzis of errors - 2 minutes -......................................................................................................... 27
5.2 - THE ASL SYSTEM - ................................................................................................................................... 305.2.1 - Analyzis of errors - 5 minutes -......................................................................................................... 305.2.2 - Analyzis of errors - Weighted Search Area method -........................................................................ 32
6 - DISCUSSION AND CONCLUSION - ......................................................................................................... 41
FRENCH TRANLATION .................................................................................................................................. 49
Green pages: French translation of the foreword, the summary, the introduction, objectives, discussion and conclusion.Pages vertes : Traduction en langue française de l'avant-propos, du résumé, de l’introduction, des objectifs, de la discussion
et conclusion.
x
ABBREVIATIONS
ASL Applied Science Laboratories
Astrips Active Strips
ATC Air Traffic Controller
CRT Cathodic Ray Tube
CTTT Controllers Transition Trials Tools
EEC Eurocontrol Experimental Centre
EEG Electroencephalogram
EUROCONTROL European Organisation for the Safety of Air Navigation
HMI Human Machine Interface
HRV Heart Rate Variability
IVIEW System Video-based contact-free evaluation of gaze position
KeyB Key Board
LAA Laboratoire d'Anthropologie Appliquée
NAC Nippon Automatic Corporation
NASA National Aviation and Space Agency
NASA TLX Nasa Task Load Index
NOLDUS Noldus Information Technology Inc
OBSERVER video pro Tool for the study of human behavior
POPUP Window displayed in response to behavior mouse click
Pstrips Pending Strips
SMI Senso Motoric Instruments
TRACON Terminal Radar Approach Control ( Simulator )
WSA Weighted Search Area
1
A N T H R O P O L O G I E A P P L I Q U E E
45, rue des Saints-Pères 75270 PARIS Cedex 06Téléphone : 01 42 86 20 37- 01 42 86 20 41 -Télécopie : 01 42 61 53 80
GAZE ANALYSIS AND PSYCHOPHYSIOLOGICAL PARAMETERS :
A TOOL FOR THE DESIGN AND THE EVALUATION OF MAN-MACHINE
INTERFACES FEASIBILITY STUDY
* * *
DOC AA 407/00 June 2000
2
This study was conducted with the fruitful collaboration of Mrs Françoise Caloo and MrHugh David.The laboratory expressed their grateful acknowledgements to the Air Traffic Controllers whowere actively involved in these experiments :
Ms Andy BarffFranck DowlingPeter EriksenRoger GerreauPaul HaworthYann KermarquerBernard KerstenneFrançois VergneAntoine VidalJean Paul Zabka
3
1 - INTRODUCTION AND AIMS -
The introduction of new technologies in the field of Air Traffic Control leads to
deep changes in the Controllers’ working procedures and tools. Traditional radar
displays and paper strips are progressively being replaced by cathodic displays,
electronic strips, a mouse and a keyboard. These changes can affect the performance of
Controllers in different ways and it is crucial to ensure that these new tools are capable
of helping Controllers in managing traffic. In this context, the previous studies
conducted at the Eurocontrol Experimental Centre were intended to validate a
methodology to address some important issues for the development of these new tools.
This methodology was based on physiological and subjective data concerning workload,
stress and fatigue.
The present study aims to complete these evaluations with gaze analysis. In fact,
visual scanning is one of the first steps of information processing. However, a basic
distinction should be made between “looking” and “seeing”. Considering that one can
“see” without “looking” gaze analysis itself is not sufficient for the evaluation of
human-machine interfaces. This is the reason why it is necessary to develop a
comprehensible approach that should allow to take into account some perceptual
aspects as well as the information processing of Controllers. Therefore, gaze analysis
and physiological data can maximise the evaluation both in terms of the impact of a new
device on the Controller’s state as well as for the design of new interfaces. In particular,
the integration of new tools using windows makes the use of large displays
(2000X2000) necessary. Thus, the way information is presented to Controllers is
changing. Because of these changes, the Controllers’ visual strategies are changing
drastically. For example, depending on its position on the display some information
could be neglected. It is thus useful to get some objective evaluations of the impact of
these new technologies on Controllers.
The main aim of the feasibility study was to evaluate the relevance of a
methodology using both gaze analysis and physiological data to assess interfaces. With
this aim in view, simple simulations using the TRACON software were presented to 10
Controllers. TRACON has already been used in previous studies (Cabon et al, 1997) for
the investigation of several issues such as the use of psychophysiological measures for
the evaluation of workload, learning process, or stress… Brookings et al (1996) also
used it on a study on workload. There were two main advantages to using TRACON for
the purpose of this feasibility study : it is easy to learn for Controllers and various
scenarios can be setup. Obviously, the simulations played with TRACON cannot be
4
considered either as reality or as the complex real-time simulations played at
EUROCONTROL.
However, as the aim of this study is not to provide some results on TRACON itself
but on the physiological measures and gaze data, TRACON appears to be very
adapted to this context. Moreover, in previous studies Controllers felt the simulations to
be realistic regarding the normal control activities and they were highly motivated in
using the software.
After a literature review on the topic of visual scanning, this report presents a synthesis
of the two feasibility studies conducted at the Eurocontrol Experimental Centre (EEC).
In these two studies, two eye tracking devices were used. In the first study, a head-
mounted device (IVIEW) was used. Its main results are summarised. In the second
study, a remote device was tested (ASL 504). Thanks to technical improvements, the
ASL device has allowed a very precise analysis of errors. The methodology and the
main results are presented.
5
2 - LITERATURE REVIEW -
The aim of this review is to summarize:
- the eye tracking technique,
- the data processing methods,
- the main applications of this method in ergonomics, in particular in aeronautics.
2.1 - The eye tracking technique -
The eye tracking use the following components (figure 1):
- a video camera and an infrared light that tracks the eye movements (eye system),
- a video camera recording the scene (scene camera),
- a real-time data processing system calculating the gaze direction.
Figure 1
Description of the principle for the acquisition of gaze
(adapted from Rousseau et coll., 1994)
Scenecamera
Eye ofsubject
Processingsystem of
data
Scenepicture
Eye camera
Infrared
Eye picture
6
This device displays the location of gaze points on the scene observed by the subject.
The device can be either head-mounted (e.g. IVIEW) or remote (e.g. ASL 504).
Depending on the device used, a head movement’s compensation is achieved. This
compensation permits to process automatically the gaze co-ordinates.
2.2 - Data processing methods -
Main data obtained from the eye tracking are the number and the duration of
fixations on specified area of the visual field. Depending on objectives, 4 classes of data
processing methods can be distinguished:
- fixation chronology,
- statistical analysis of gaze in a specific area,
- analysis of fixation groups,
- weighted search area.
2.2.1 - Fixation chronology methods -
These methods are based on the analysis of fixation sequences. These sequences
can be presented in two ways:
- by indicating the fixation points routes (figure 2),
- by indicating the fixation points succession in a specified area of the visual field
(figure 3).
The first method provides a spatial information whereas the latter gives a
temporal information.
7
Figure 2
Illustration of gaze point during one session
(adapted from Menu et al., 1994)
Figure 3
Fixation time in one selected area during a reading task
(adapted from Menu et al., 1994)
Y c
oord
inat
es
Time of session (s)
X coordinates
Fixa
tion
time
(ms)
Time of session (s)
8
2.2.2 - Statistical analysis of gaze in a specific area -
These methods are based on the statistical analysis of the fixation distribution in a
specific area (figure 4). This method allows an hierarchical organization of the various
areas in terms of information search.
Figure 4
Description of fixation frequency for three sectors defined from driving simulation
(adapted from Laya, 1994)
0
20
40
60
80
100
Left In front Right
Freq
uenc
y fi
xatio
n (s
)
9
2.2.3 - Analysis of fixation groups -
Two kinds of methods can be distinguished:
- Modelling of transition probability from one area to another (Papin, 1981)
- Factor analysis (Rousseau et al, 1994 (figure 5) which allows the separation of
some sectors which are statistically different and the evaluation of the chronology
of fixations and the quantity of fixations.
The width of the arrows indicate the number of fixation sequence
Figure 5
Illustration of factor analysis realized from sequence of fixations on a car
(adapted from Rousseau et al, 1994)
Sect
or 2
Sector 3
Sector 4Sector 4
Sector 1
sect
or 5
0
20
40
60
80
100
1 2 3 4 5
Sector
Num
ber
of f
ixat
ion
sequ
ence
Sector
10
2.2.4 Weighted Search Area (WSA)
The WSA developped by Chia and Fang-Tsan (1997) is based on the integration
of fixation frequency and duration. It is based on the description by vectors of visual
fixations and on assignment of weight to each vector by its respective fixation time.
Hence, the aim of WSA is to describe concentrations of visual strategies and potentially
quantify the amount of visual load (figure 6). This method was applied in our study of
error analysis (section 5.2.2).
Figure 6
Illustration of Weighted Search Area method (WSA) - This method is based on the
description by vectors of frequency and time of fixations
(adapted from Chia and Tsang, 1997)
2.3 - Applications of visual scanning in ergonomics -
For the operator, visual scanning consists of directing his gaze toward
informative areas. The analysis of these different fixation areas permits the evaluation
of the strategy used by the operator in taking the information and of the efficiency of
this strategy regarding his activity.
This efficiency is influenced by various factors. Drury and Sinclair (1983)
showed that visual scanning is influenced by the task difficulty. In that study, task
difficulty was mainly related to the size, the contrast and the variety of the objects.
11
The temporal organization, i.e. the duration and the rhythm of an activity is also
an important factor which influences the visual scanning pattern and the performance
(Moraal, 1974). Togami (1984) showed that when the activity rhythm is high, the
fixation number and duration increase. They also observed that the increase of the
fixation duration is more efficient to improve performance than the increase of the
fixation number.
In addition, Saito (1972) showed that the higher is the temporal demand, the less
the visual scanning is large (i.e., less objects are inspected). Simon et al (1993) observed
that the visual scanning is more structured when the workload increases. These results
are confirmed by Tole et al (1982) on aircraft pilots. When the pilot has to cope with a
high workload, he has less time to look at various instruments and therefore adopt a less
random visual scanning strategy. Other studies conducted in simulators found the same
results (Krebs et al, 1977, Harris et al, 1986).
These results are consistent with the conclusions of Sperandio (1991). Operators
have to adapt their strategy to cope with the increase of task demand by using a more
“economic” visual strategy to optimize the information taking.
In the field of aeronautics, visual scanning behaviour was shown to be very useful
for the interface design. (Table 1) summarized some recent studies conducted in this
field.
12
AUTHORS OBJECTIVE SAMPLE DATA
COLLECTIONRESULTS
Bellendes et al.(1997)
Examination of differences inattention control betweenexperienced and novice pilots frommeasurement of visual scanning
- Twelve students pilots
- Twelve Flight instructors
- Flight simulator
- Number of fixations- Mean dwell time
- Novices tended to dwell for a longer time aneach instrument
- Information is extracted more efficiently byexperts from nearly all instruments anparticularly from the most information withinstrument
Itoh (1990) Examination of the differences inscanning behaviour under conditionsof high mental workload betweenpilots using and integrated displayand pilots using the earlier electro-mechanical display
- Five experimented pilots - Number of fixations- Fixation time
- The proportion of fixation duration increases inresponse to the increase of mental workload
- In terms of the ease of obtaining informationfrom instruments the integrated display usingthe CRT might be more preferable for pilotstask
Tole et al.(1982)
Examination of the relationshipbetween pilot visual scan ofinstruments and mental workload
- Six subjects varying in skill-level from non pilot to ahighly experienced NASAtest pilot
- Flight simulation
- Time course of eyefixations
- Dwell time of eachfixation
- Scanning behaviour may serve as an indicatorof both workload and skill.
-As a pilot’s skills develop, his visual scanningbehaviour is less as less affected by non visualincrements in workload.
- Pilots with a high experienced skill present adecrease of the percentage of long dwells.
David(1977)
Provide a detail description of threemethods as means for measuring theController’s eye movement and thenecessary : direct observation, videotranscription and eye tracker system
- Two teams consisted foreach by 6 Controllers
- Video- Number and durationof fixation
- Eye mark recording (NAC Mark V) is precisebut intrusive.
- Video-recording requires about four times thelength of observation: it may be biased byomission.
- Direct observation is very "cheap" but requiresfour times also the length of observation: insomme circumstances it may not be acceptableto the Controller.
Table 1
13
For Air Traffic Control, the understanding of controllers visual search behaviour
becomes more and more important with the prevalent use of computer screens and other
visual display devices. In fact, the evaluation of controllers’ visual scanning patterns could
help to improve the design of these new interfaces.
In 1975, Karsten et al. found already that controllers spend approximately 80% of
their time looking at the radar display, 13% at flight strips and 5% at input devices.
However, in a real time simulation, David (1977) showed that the number of fixations
directed at various devices were not necessarily indications of the relative efficiencies of
displays. This observation seems to be related to the system limitation (NAC). New
generation eyes tracking systems (e.g. ASL) now overcome these limitations
More recently, the research of Stein (1987), showed also from a real time ATC
simulation that the longest fixation duration’s were observed for aircraft compared to other
objects on the radarscope. For the authors, the inclusion of other objects and scene planes
could drastically reduce the average duration of the fixations on aircraft.
However, only a few studies have associated gaze data and others objectives or
subjectives data to evaluate the information processing related to perception.
Itoh (1992) observed, in pilots, with the measure of heart rate variability, no
difference in mental workload between pilots using integrated CRT displays (Boeing 767)
and pilots using earlier electromechanical displays (Boeing 747-300) (table 1). Regarding
the visual scanning, they noticed that when the workload increased, the fixation duration
for certain instruments increased. In addition, they observed a significant difference of
visual scanning between the two interfaces during abnormal situation (e.g. taking off and
landing with engine failures). In fact, the changes in scanning pattern for B767 were found
to be smaller than those of the B747 pilots. Based on these results, the authors concluded
that the application of CRT helps pilots to obtain sufficient information more easily than
electromechanical displays do, even under abnormal situations.
Only one study has used both psychophysiological and visual scanning
measurements in Air Traffic Controller (Brookings et al, 1996). Although, the Brookings’
study did not evaluated the fixation duration and number of fixations but the vertical blink
and horizontal saccade activity, their results indicated that « as the visual demands
increased, the controllers blinked less often so as not to miss important events on the radar
scope ». For the author, this interpretation is supported by other investigations reporting
decreased blink rates when operators were under conditions of increased visual load.
14
3 - SYNTHESIS OF THE IVIEW STUDY -
The first step of this study was intended to evaluate the interest of using eye tracking
technique and physiological data for the evaluation of various aspects in the context of
interface design (photos 1 and 2). In the context of this study, it was decided to test the
following hypothesis: the use of different methods for entering data, the keyboard or the
trackball, changes the visual scanning of Controllers as well as their attention and
workload. In fact, it is assumed that the use of the keyboard forced the Controller to look
frequently at the keyboard and, consequently, reduced his visual inspection on the radar
and his attention towards the information displayed. The experimental protocol allowed to
compare 4 conditions combining the use of a trackball and a keyboard and 2 levels of
traffic:
- keyboard and low level of traffic,
- keyboard and high level of traffic
- trackball and low level of traffic,
- trackball and high level of traffic.
Photo 1
15
Photo 2
Eight subjects were involved in this study. The data collected were: fixation time
and frequency on various items of the visual field, Heart Rate Variability for mental effort
evaluation, the electroencephalogram (EEG) for the assessment of attention, the NASA-
TLX for perceived workload and simulator data for the Controllers’action and
performance.
The comparison between the Keyboard conditions and the Trackball conditions
shows that:
- changes in fixations are more frequent in the keyboard conditions but fixations are
longer in the Trackball conditions,
- performance is better in the Trackball conditions,
- workload is not affected,
- attention is higher in the Trackball conditions.
The comparison of the two levels of traffic indicates that:
- gaze frequency and duration are not affected,
- workload is higher in the 30-aircraft condition,
- handover errors are increased with traffic.
Therefore this study showed that the use of both gaze data and physiological
measures are able to evaluate the impact of different interfaces.
16
4 - ASL STUDY -
4.1 - Aims -
The aim of this second study is twofold:
- to examine the advantages of the ASL 504 device compared to IVIEW,
- to study the interest of the data collected as regards error analysis.
4.2 - Method -
4.2.1 - Controllers and sample -
Four Controllers were involved in this study. Among these 4 controllers, 2 had
already participated in the previous IVIEW study. Each Controller was presented with 2
simulations on the TRACON. The two Controllers who were not involved in the previous
study were presented a traffic of 15 aircraft in 30 minutes (session 1) and then a traffic of
30 aircraft in 30 minutes (session 2). As they were already familiar with TRACON, the 2
other Controllers were presented only with two sessions with 30 aircraft. For this study,
only the trackball was used by the Controllers as the previous study had demonstrated that
the graphical interface (trackball) was better than the numerical interface (keyboard
condition).
4.2.2 – Eye-tracking device description -
The eye-tracking device used in this study is the ASL 504. Contrary to IVIEW, the
ASL 504 is not head-mounted but totally remote (photos 3 and 4). It is composed of an
eye camera located beside the TRACON display. An infrared light, emitted by the camera,
is reflected on the controllers’cornea. The only thing that the controller wears on his
forehead is a transmitter that gives his head’s position. This information is sent to an aerial
located behind the Controller’s head and processed by a computer that allows to the
camera to track the eyes thanks to a motor.
17
Photo 3
Photo 4
18
Therefore, this device takes into account all the controllers’ eyes movements (photo 5).
The method to determine the gaze position is the same as for IVIEW. Compared to IVIEW,
one of the main advantages of ASL is that it compensates the head movements and
therefore provides for each gaze its co-ordinates relative to the display. The TRACON
display was filmed using a video camera during all the sessions.
Photo 5
4.2.3 - Data collected -
The data collected were the same as those collected in the previous study:
- gaze: fixation frequency and duration,
- physiological: Heart Rate Variability (HRV) and the electroencephalogram (EEG),
- simulator data: Controllers’ actions and errors,
- subjective: workload (NASA-TLX).
19
Contrary to the IVEW study, the ASL 504 allowed to process the data automatically.
In the IVIEW study, the fixation points were obtained by reviewing the video recording at
a lower speed (5 times less than the normal speed) and by manually coding the gaze
position using the OBSERVER (photo 6). In the present study, the OBSERVER was used
to collect the Controllers’ actions (photo 7).
Photo 6
Photo 7
20
4.3. - Results -
The results were analyzed in two ways:
- a global analysis in which the two levels of traffic were compared,
- an analysis focused on the errors.
4.3.1 - Fixation frequency and duration -
Figures 7 and 8 show the fixation frequency and duration on the menu, the radar,
the strips and the communication window for the two levels of traffic. It can be observed
that for all the items, the fixations frequency and duration are more frequent in 15-aircraft
condition. However, the fixations duration on the radar are longer in the 30-aircraft
condition. Therefore, the Controllers changed their fixation more in the lowest traffic
condition and spent less time looking at the radar. This result can be explained by the fact
that in the lowest traffic condition, the Controllers used were those with the least
experience in TRACON while the 30-aircraft condition included the 2 controllers who had
been involved in the first study. This also related to the fact that the traffic increase the
controller tend to focus to only a few important elements of the display.
Figure 7 – Fixation frequency as a function of items and traffic levels.
0
2
4
6
8
10
12
14
16
Menu Radar Strips Com Off
Traffic level : 15 aircraft (n = 2 ATC)
Traffic level : 30 aircraft (n = 4 ATC)
Fix
atio
n (m
n)
21
Figure 8. Fixation duration as a function of items and traffic levels.
4.3.2 - Physiological data -
HRV and EEG results are presented in figure 9. The results show that the power in
the middle frequency is increased (i.e. the effort is increased) in the 30-aircraft condition.
EEG data indicate also an increase of attention in the 30 aircraft condition for the 4
Controllers.
Figure 9. Power in the theta frequency band of the EEG and in the middle frequency of the HeartRate Variability as a function of traffic.
0
10
20
30
40
50
60
Menu Radar Strips Com Off
Traffic level : 15 aircraft (n = 2 ATC)
Traffic level : 30 aircraft (n = 4 ATC)
150
160
170
180
190
200
210
220
230
T15 T300
10
20
30
40
50
60
70
80
90
100
T15 T30
Pow
er in
the
mid
dle
freq
uenc
y (m
n)
Pow
er in
the
The
ta b
and
(mn)
Fix
atio
n du
rati
on (
%/m
n)
22
4.3.3 - Controllers’ orders -
The number of Controllers’ orders divided by the number of aircraft (figure 10)
show that Controllers changed the aircraft heading more than their speed or altitude. This
result is consistent with the results obtained in the previous study. It can also be observed
that Controllers provide more orders per aircraft at the highest level of traffic in order to
cope with the task difficulty.
Figure 10. Frequency of controllers’ orders as a function of traffic
0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
1.00
Altitude Speed Heading
Traffic level : 15 aircraft (n = 2 ATC)
Traffic level : 30 aircraft (n = 4 ATC)
Con
trol
lers
ord
ers
(mn)
23
4.3.4 - Perceived workload -
Figure 11 shows the results of the workload assessment after the two sessions for
the Controllers who were familiar and for the Controllers who were not familiar with
TRACON. In the first sessions, it can be observed that perceived workload is higher for
Controllers who were not familiar although they were only presented with 15 aircraft. In
the second session, the less experienced Controllers perceived a higher workload but with
less differences than in the first session.
Figure 11Assessment workload as a function of traffic.
Mentaldemand
Temporaldemand
Performance Effort Frustrationlevel
Physicaldemand
Mentaldemand
Temporaldemand
Performance Effort Frustrationlevel
Physicaldemand
Arb
itra
ry U
nits
Arb
itra
ry U
nits
100
80
60
40
20
0
100
80
60
40
20
0
Not experienced / 30 aircraft Experienced / 30 aircraft
Not experienced / 15 aircraft Experienced / 30 aircraft
24
5 - ERRORS AND VISUAL SCANNING ANALYZIS -
Three kinds of situations were analysed with the IVIEW and ASL systems:
separation conflicts, missed approaches and handover errors. The number of these errors
was determined with the TRACON software, at the end of each session. The time of errors
occurrence was determined with video recordings. Finally, each error’ analysis was
realised by taking into account gaze movements, physiological data and the conditions of
each session.
For the IVIEW system, the analyses were carried out in two ways:
- a mean was calculated for each set of data 5 minutes before Controller’ errors,
- data were processed per second, 2 minutes before errors.
With the ASL system, two kinds of analyses were also achieved:
- gaze movements were automatically analysed for 16 areas of the radar screen, 5
minutes before errors (figures 12 and 13),
- the application of the Chia and Fang-Tsan’s method: the Weighted Search Area.
Figure 12
Description of the areas analyzed with the Iview system
Radar
Communication
Strips
25
Figure 13
Description of the areas analyzed with the ASL system
5. 1 - The IVIEW system -
5.1.1 - Analyzis of errors - 5 minutes -
Five minutes before an error, the visual scanning of each Controller was determined
from this fixation frequency. The fixation frequency was calculated for the following
items: Radar, Keyboard, Astrips, Pstrips, Popup and Others (figure 14).
No significant difference was observed per item and error. Moreover, no effect of
the trackball-keyboard and traffic level conditions was observed on missed approach,
handover and separation conflict errors.
Nevertheless, two results can be noticed:
- the number of fixation decreased on the Keyboard, before a missed approach,
- the fixation frequency increased for the Astrips before a separation conflict error.
Radar
Communication
Strips
Radar Radar Radar
Radar Radar Radar Radar
Radar Radar Radar Radar
RadarRadarRadarRadar
26
Figure 14
Mean fixation frequency 5 minutes before missed approach, handover and
separation conflict errors
- Iview system -
0
50
100
150
200
250
Radar Keyb Astrips Pstrips Popup Others
Fix
atio
n fr
eque
ncy
Missed approach Handover Separation conflict
27
5.1.2 - Analyzis of errors - 2 minutes -
One subject’s duration and frequency of fixation, 2 minutes before a separation
conflict is presented on figure 15. Data obtained from these two gaze parameters were
processed per second.
Figure n 15
Description of duration and frequency fixation and the HRV of one Controller 2 minutes before a
L3sinß3 + L4sinß4 where L0(X+) is the total projected length on the X axis for X>0, L0(X-)
is the total projected length on the X axis for X<0, L0(Y+) is the total projected length on
the Y axis for Y>0, L0(Y-) is the total projected length on the Y axis for Y<0.
5 – Rotate the horizontal and vertical axes, where 00 < K0 < 900 . Both axes would
be rotated K degrees by R times, where R = (900 / K) –1. Then, reproject the compound
vector r V i onto the new references axes. Though the reprojection, Lr(X+), Lr(X-), Lr(Y+),
Lr(Y-) can be derived where Lr(X+) is the total projected length on the X axis for X>0,
Lr(X-) is the total projected length on the X axis for X<0, Lr(Y+) is the total projected
length on the Y axis for Y>0, Lr(Y-) is the total projected length on the Y axis for Y<0, in
the r rotated coordinate system. Procedures would be repeated R times for r = 1 until r = R.
48
49
Traduction en langue française de l’Avant-propos, du Résumé, del’Introduction, des Objectifs, de la Discussion et Conclusion.
ANALYSE DU REGARD ET MESURES PSYCHOPHYSIOLOGIQUES:
UNE METHODE POUR LE DEVELOPPEMENT ET L’EVALUATION DESINTERFACES HOMME-MACHINE.
ETUDE DE FAISABILITE
par
P. Cabon, B. Farbos, R. MollardLAA Université René Descartes - Paris V
AVANT - PROPOS
Le besoin d'évaluer plus objectivement l'impact du contrôle aérien sur les contrôleurs est unepréoccupation ancienne. De nombreuses études ont été conduites au CEE pour déterminer lesméthodes d'investigations les plus adaptées pour en mesurer les effets (CEE Rapports n° 64, 164,183, 226, 228, 323, 334 et 339).
Le LAA (Laboratoire d'Anthropologie Appliquée de l'Université Paris V, René Descartes) amené, sous la direction du Pr Alex Coblentz, de nombreuses études sur la fatigue, les troubles dusommeil et autres troubles associés sur les pilotes de lignes, les conducteurs d'automobiles et lescontrôleurs aériens.
C'est pourquoi, nous avons demandé au LAA, de faire une série d'études de faisabilité surl'utilisation d'analyse du cortisol, d'électro-encéphalographie en particulier la technique du PEA(Potentiel Evoqué Auditif) ainsi que d'autres méthodes ad hoc et qui sont décrites dans unrapport CEE n° 323 et une note CEE n°16/98. Ces études ont été réalisées par le LAA sur unsimulateur autonome TRACON II et ultérieurement sur des simulations temps-réel et fait l'objetdes rapports CEE n° 334 et 339.
Ce rapport couvre des études complémentaires d'analyses des mouvements oculaires associées àdes mesures électro-physiologiques, également réalisées sur TRACON II.
Le Directeur du Centre Expérimental, les chercheurs du Laboratoire d'Anthropologie Appliquéeainsi que les chefs de projets tiennent à exprimer leurs remerciements aux contrôleurs et auxpersonnels du CEE pour leur étroite et dévouée collaboration.
Françoise CalooHugh David
EEC project officers
50
51
RESUME
Cette étude démontre la pertinence d'une méthode reposant à la fois sur l'analyse des regardscouplée à des données physiologiques pour évaluer des interfaces Homme-Machine dans ledomaine du contrôle de la circulation aérienne.
Une revue de la littérature sur le thème des mouvements du regard a précédé deux études qui ontété conduites au Centre Expérimental Eurocontrol. Deux systèmes ont été utilisés, dans lapremière étude un système ( I VIEW) posé sur la tête et dans la deuxième, un système déporté(ASL504). Un logiciel TRACON II permettant de présenter aux contrôleurs des tâches ATCreprésentatives et une sélection de mesures electro-physiologiques (EEG, HRV etc….) ontégalement, été utilisés pour ces deux études.
Cette étude conforte l'intérêt d'associer des paramètres physiologiques et d'analyse du regardpour évaluer la charge de travail, la fatigue, les stratégies opérationnelles des contrôleurs dans unenvironnement hautement interactif.
Des études complémentaires devraient être poursuivies pour analyser l'impact des changementsdans les tâches ATC induit par l'introduction et la généralisation de nouvelles technologiesinformatiques tels que: strips électroniques, écrans plats etc…
52
53
1 - INTRODUCTION ET OBJECTIFS -
L’introduction de nouvelles technologies dans le domaine du contrôle aérien
entraîne des changements importants dans les procédures de travail et dans les outils
utilisés par les contrôleurs. Les écrans radar traditionnels et les strips papiers sont
progressivement remplacés par des écrans cathodiques, des strips électroniques, une
souris,... Ces changements peuvent affecter la charge de travail et la performance des
contrôleurs. Il apparaît essentiel de s’assurer que ces nouvelles technologies soient
capables d’aider les contrôleurs à gérer un trafic en constante augmentation. Dans ce
contexte, les travaux menés par le Laboratoire d’Anthropologie Appliquée en collaboration
avec le Centre Expérimental d’Eurocontrol ont pour objectif de valider des méthodes
d’évaluations permettant d’étudier l’impact de ces nouvelles technologies sur les
contrôleurs. Ces méthodes reposent sur le recueil de données physiologiques et subjectives,
concernant la charge de travail, le stress, la fatigue et l’activité des contrôleurs.
La présente étude vise à compléter ces évaluations avec l’utilisation de l’analyse des
mouvements du regard. Elle a pour but de prendre en compte les aspects perceptifs et de
traitement de l’information lors de l’évaluation d’un nouvel outil ou pendant la
conception d’une nouvelle interface. L’intégration de nouveaux outils informatiques se
traduit en effet fréquemment par l’utilisation d’écrans de grande taille (2000X2000), ce qui
peut influer sur la présentation des informations par rapport aux postes de travail
traditionnels mais aussi modifier les stratégies visuelles des contrôleurs. En fonction de
leur localisation sur l’écran, certaines informations peuvent éventuellement être négligées
par les contrôleurs.
L’objectif principal de cette étude consiste à évaluer la pertinence d’une méthode,
reposant à la fois sur l’analyse des regards couplée à des données physiologiques, pour
évaluer des interfaces. La démarche retenue repose sur l’utilisation de simulations
simplifiées à partir du logiciel TRACON, déjà utilisé dans des études précédentes (Cabon
et coll., 1997) pour évaluer la charge de travail, les processus d’apprentissage et le stress
des contrôleurs. Deux campagnes d’essais ont été conduites au Centre Expérimental
d’Eurocontrol. Dans la première campagne, un système (IVIEW) posé sur la tête du
contrôleur a été utilisé. Dans la seconde, un système déporté a été testé (ASL504). Ce
dernier est équipé d’un dispositif de compensation des mouvements de la tête permettant
de réaliser des traitements plus précis sur les stratégies oculaires adoptées par les
contrôleurs. Ce rapport présente à la fois une revue de la littérature sur le thème des
mouvements du regard ainsi qu’une synthèse des principaux résultats obtenus au cours de
ces deux campagnes.
54
2 - DISCUSSION ET CONCLUSION -
La première campagne de mesure a été effectuée avec la participation de 8
contrôleurs, la seconde avec 4 contrôleurs. Deux contrôleurs ont été impliqués dans les
deux campagnes. En complément des paramètres de regard (nombre et fréquence de
fixations), des données physiologiques (variabilité cardiaque pour l’évaluation de l’effort
mental, l’EEG pour les processus attentionnels) et des données subjectives (NASA-TLX
pour la perception de l’effort) ont été recueillies.
Les résultats de l’étude « IVIEW » (première campagne de mesure) montrent
l’intérêt des données recueillies pour évaluer l’impact de l’utilisation du clavier comparé à
une boule roulante. Cette étude a également mis en évidence les limites d’IVIEW en raison
de l’absence de compensation des mouvements de la tête. De ce fait, le traitement des
données reste manuel, long et n’est pas suffisamment détaillé pour effectuer des analyses
précises. A l’opposé, le système ASL 504, évalué dans la seconde étude, présente deux
avantages principaux. Le premier est qu’il réduit les contraintes pour le contrôleur
puisqu’il n’est pas posé sur la tête. Le second a trait au dispositif de compensation des
mouvements de la tête qui permet d’effectuer un traitement automatique, beaucoup plus
rapide et précis qu’avec IVEW. Par ailleurs, ASL504 peut être utilisé pour évaluer des
postes de travail qui comportent plusieurs écrans. La seule difficulté technique rencontrée
concerne les contrôleurs qui portent des lunettes. Dans ce cas, les données ne peuvent pas
être exploitables sur de courtes périodes (quelques secondes) d’enregistrements.
Le système ASL s’est révélé particulièrement utile pour l’analyse des erreurs. En
effet, il permet d’évaluer précisément la localisation du regard sur l’écran dans les minutes
précédant l’erreur. Cette analyse a permis de distinguer deux types d’erreurs : celles liées à
une absence de perception visuelle de la zone où l’erreur a eu lieu et celles associées à une
absence de traitement de la situation. Dans le premier cas, les regards ne couvrent pas la
zone de l’erreur alors que dans le second, le contrôleur regardait cette zone. Néanmoins,
cette analyse d’erreur doit être confirmée sur un échantillon plus important. Un entretien
d’auto-confrontation avec le contrôleur à la fin de la session, permettrait également de
mieux interpréter ce type d’erreur, en particulier sur le plan de la stratégie adoptée par le
contrôleur.
Les données concernant les erreurs collectées lors des essais avec le système IVIEW
bien que moins détaillées que celles recueillies avec ASL, suggèrent une relation entre la
variabilité cardiaque et les mouvements du regard. Généralement, dans les minutes
précédant une erreur, une augmentation de la puissance dans la bande de fréquence
moyenne (c’est-à-dire une augmentation de l’effort mental) s’observe pour la variabilité
55
cardiaque alors que la durée des fixations augmente. Ces résultats confirment ceux de Itoh
et coll. (1990) qui ont observé que l’augmentation de la durée de fixation se trouve
associée à une augmentation de la charge de travail. Ce résultat peut être expliqué par le
fait que l’augmentation de la difficulté de la tâche entraîne une focalisation de l’attention
sur un nombre limité d’informations.
En conclusion, cette étude de faisabilité démontre l’intérêt d’utiliser à la fois des
données physiologiques et de regard dans le contexte de l’évaluation d’une interface
spécifique. Il est donc proposé de transposer cette méthode à des simulations plus
complexes comme outil d’évaluation de nouveaux postes de travail des contrôleurs. Dans
ce contexte, les données recueillies peuvent s’avérer utiles pour l’évaluation des concepts
CTTT (Controllers Transition Trials Tools) et Skytools. Cette transposition pourrait être
centrée sur l’impact de l’automatisation et de l’utilisation des strips électroniques sur la