Effects of physical workload on cognitive task performance and situation awareness

New XML Template (2006) [16.9.2006–7:11pm] [1–19]{TANDF_FPP}TTIE/TTIE_A_195831.3d (TTIE) [First Proof]

Theoretical Issues in Ergonomics ScienceVol. ??, No. ?, Month?? 2006, 1–19

Effects of physical workload on cognitive task5 performance and situation awareness

CARLENE M. PERRY, MOHAMED A. SHEIK-NAINAR, NOA SEGALL,

RUIQI MA and DAVID B. KABER*

Edwards P. Fitts Department of Industrial and Systems Engineering, North CarolinaState University, Raleigh, NC 27695-7906, USA

10 Sixteen participants performed a military operations simulation directingloading of helicopters to weight capacity within an allotted timeframe andsubject to a set of decision rules. The participants stood, walked or jogged on atreadmill while performing the simulated cognitive task. Task performance wasmeasured in terms of helicopter loading rate and accuracy. Situation awareness

15 (SA) was measured using a simulation freeze technique and SA queries. Subjectiveworkload was measured using the NASA-TLX. Results indicated a general trendof decreasing SA with increasing physical workload for perceptual knowledge,comprehension and overall SA. Results also revealed higher subjective workloadduring jogging than during the walking and standing conditions. However, the

20 physical workload manipulations did not appear to affect cognitive taskperformance. This study has practical implications for defining physical andcognitive workloads in specific dynamic, complex work environments to supportoperator SA and performance.

Keywords: Situation awareness; Physical workload; Cognitive task performance;25 Mental workload

1. Introduction

1.1. Physical workload and cognitive task performance

Many occupations that require physical exertion also place demands on humanmental or cognitive resources. Examples include operators of manufacturing

30 systems, soldiers in combat operations, emergency search and rescue teams andemergency room medical staff (Mozrall and Drury 1996). There has been increasinginterest in how, and to what extent, physical exertion might affect performance inthese areas and the performance of cognitive tasks in general (Mastroianni et al.2003). According to reviews by Tomporowski (2003) and Mozrall and Drury (1996),

35 while physical exertion appears to facilitate performance on some cognitive tasksunder certain circumstances, it also appears to inhibit performance on other tasks oron the same tasks under different conditions.

Research on the effects of physical workload on simultaneous cognitive taskperformance has largely involved basic perceptual-motor performance (e.g. letter

*Corresponding author. Email: [email protected]

Theoretical Issues in Ergonomics ScienceISSN 1463-922X print/ISSN 1464-536X online � 2006 Taylor & Francis

http://www.tandf.co.uk/journalsDOI: 10.1080/14639220600959237

New XML Template (2006) [16.9.2006–7:11pm] [1–19]{TANDF_FPP}TTIE/TTIE_A_195831.3d (TTIE) [First Proof]

40 detection and simple and choice reaction time tasks) or psychomotor tasks(e.g. tracking). In an early review of over 27 studies performed to assess the effectsof exercise on simple cognitive tasks and five particular studies that evaluated short-duration, moderate-intensity exercise, Tomporowski and Ellis (1986) concluded thatmost research supports the position that task performance requiring information

45 processing is facilitated by an increase in physical arousal when evaluating reactionand movement times. In an updated review article, Tomporowski (2003) reviewed43 studies, 22 of which specifically studied the effects of short-duration aerobicexercise on simultaneous task performance. He again concluded that there iscompelling evidence that exercise increases speed of information processing

50 for various tasks, such as simple detection, visual search and discriminativechoice-response. Evidence was also found suggesting that exercise may, under certainconditions, facilitate more complex cognitive tasks, including basic decision-makingand problem-solving. For example, a study on handball players’ decision-makingabilities in simulated game situations found performance to be better while running

55 on a treadmill than when walking (Tenenbaum et al. 1993).Few studies have examined the effects of physical workload on more complex

cognitive tasks, such as mathematical reasoning (Bills and Stauffacher 1937). Resultsof this research have been mixed (Mozrall and Drury 1996, Tomporowski 2003). Forshort-duration physical activity, some studies found decreased accuracy in

60 performing cognitive tasks, such as map interpretation (e.g. Hancock andMcNaughton 1986); some found no effect on accuracy in complex mathematicalproblem-solving (e.g. Sjoberg 1980); and others found an inverted ‘U’-shapedrelation of physical loading and mental arithmetic task performance (i.e. an optimalperformance level was reached at an intermediate level of physical workload;

65 e.g. Reilly and Smith 1986).Whereas the results of physical workload effects have been anything but

conclusive, narrative review articles (Tomporowski and Ellis 1986, Mozrall andDrury 1996) have proposed that there is evidence that sensory tasks (such as visualsearch) are enhanced by all levels of exertion and that there is an improvement in

70 a majority of central processing tasks (i.e. memory, information manipulationand reasoning) under local, short-duration exertions. These review articles alsoidentified large gaps in this knowledge base. They concluded that the effects ofphysical demands on auditory and vocal tasks, whether alone or as part of amore complex task, are largely unstudied. They also reported no studies

75 involving planning/scheduling, a central processing task, within the combinedcognitive/physical task data.

1.2. Physical workload and situation awareness

As technology has evolved, many complex, dynamic systems have been created thatplace substantial information processing requirements on humans and tax their

80 ability to operate effectively. Situation awareness (SA) has been posited as a criticalmental construct on which performance in such complex systems is dependent(Endsley 1995a). For example, in technologically advanced systems, such as flexiblemanufacturing systems, operators must rely on up-to-date knowledge of machinetool parameters, as well as recognition of any patterns among the parameters that

85 might reveal clues as to the functioning of an overall cell and future process state

2 C. M. Perry et al.

New XML Template (2006) [16.9.2006–7:11pm] [1–19]{TANDF_FPP}TTIE/TTIE_A_195831.3d (TTIE) [First Proof]

changes (Usher and Kaber 2000). Without this understanding and prediction, human

control may not be effective.Military command personnel, as well as front-line soldiers, must also frequently

rely on SA to make decisions in the battlefield (Kaber et al. 2005). Inaccurate or90 incomplete SA in these environments can lead to devastating loss of life

or unnecessary expenditure of vital resources. Recent increases in the sophistication

of military equipment have brought high technology capabilities to portable

computing systems, requiring soldiers in physically demanding roles (e.g. squad

combat operations) to simultaneously perform cognitively demanding information95 processing and transmission tasks using these systems.

Endsley (1995a) defined SA as, ‘the perception of elements in an environment

within a volume of time and space, the comprehension of their meaning, and the

projection of their status in the near future’. This definition leads to three ‘levels’

of SA that are frequently measured and studied: Level 1 – perception of elements100 in the environment; Level 2 – comprehension of the current situation; and Level

3 – projection of future status (Endsley 1995b). SA is recognized as a cognitive

construct separate from decision-making and action, and as a construct separate

from those that may influence it such as attention, working memory and stress

(Endsley 1995a). However, an accurate internal situation model or ‘good’ SA is105 thought to lead to the use of accurate mental models in task performance

and accurate recognition-primed decision-making (Endsley and Jones 1997).

There are many studies across different domains, in which high SA has been

shown to support good task performance. For example, Venturino et al. (1989)

found that performance was predicted by a combination of SA and decision-110 making (fire-point selection) in combat pilots. Ma and Kaber (2005)

recently found a positive association between SA and driving performance

(measured by variations in headway distance and following speed) in a simulated

car-following task.Endsley (1995a) postulated a number of task and system factors to influence an

115 individual’s ability to achieve SA, including system design, interface design, task

complexity, automation, workload and stress. She also theorized that physical

stressors (e.g. noise, lighting, temperature, vibration, drugs, etc.) and social

psychological stressors (e.g. fear or anxiety, uncertainty, self-esteem, etc.) may

impact SA in different ways. First, an operator may narrow his or her field of120 attention, causing a tendency to sample dominant or probable sources of

information, while ignoring peripheral, but perhaps highly critical, information

located outside the main focus of attention. A similar viewpoint is supported by

Easterbrook (1959), who contends that the number of cues utilized in any

situation tends to become smaller with an increase in emotional stress/response.125 Secondly, stress may affect SA through decrements in working memory capacity

and retrieval (Endsley 1995a). Unfortunately, there is very little empirical research to

support the specific effects of each of these types of stressors on SA, either singly or

in combination. In fact, a review of the literature revealed no studies in

which physical stressors had been manipulated to assess effects on SA. There is a130 need to describe the effects of physical workload on SA, in order to develop

an understanding of changes in human perceptual knowledge, comprehension

and projection in dual-task scenarios involving both cognitive and physical

workloads.

Effects of physical workload on cognitive task performance and situation awareness 3

New XML Template (2006) [16.9.2006–7:11pm] [1–19]{TANDF_FPP}TTIE/TTIE_A_195831.3d (TTIE) [First Proof]

1.3. Motivation and hypotheses for current study

135 The motivation for the current study is two-fold. First, our review of relatedliterature revealed a gap in the knowledge of the effects of physical workload oncognitive task performance, specifically planning, scheduling and other complexdecision-making tasks. This is an area worth exploring because there are a numberof occupations that combine occasional or sustained physical workload with

140 requirements for cognitive planning. Secondly, there is a lack of research on theeffects of physical workload on operator SA, which is critical to complex planningand decision-making tasks. Again, there are many jobs performed in complex,dynamic environments where operators are taxed physically and must maintain anaccurate internal situation model of the environment (e.g. fighter aircraft piloting).

145 Endsley (1995a) predicted that some types of physical workload might act todeteriorate SA. On this basis, we hypothesized that physical workload in a dual-taskscenario involving cognitive task performance, as investigated in those studiesreviewed by Mozrall and Drury (1996) or Tomporowski (2003), might lead todecrements in SA. That is, as physical workload increases in a dual-task scenario, we

150 speculated that operator SA would decrease.In line with this expectation, it was also hypothesized that high-level cognitive

task performance (planning), in terms of both accuracy and responsiveness, would benegatively affected by physical workload manipulations in a dual-task scenario.Although Tomporowski (2003) stated that short-duration aerobic exercise

155 may increase the speed of information processing in complex problem-solvingtasks, no empirical evidence supporting this speculation was provided. Related tothis, the studies reviewed by Mozrall and Drury (1996) found decreased performanceor no performance effect when evaluating more complex cognitive tasks performedwith physical activity. Following the theory of SA, we therefore expected that as

160 SA in a high-level cognitive task is degraded by physical stressors, planning anddecision-making performance would be degraded as well. Finally, we hypothesized apositive relationship between physical workload and subjective workload: as physicalworkload increased, workload as perceived by the participant was also expected toincrease.

165 It is important to note here that there are considerable differences of opinion asto how the construct of SA is defined. Endsley’s (1995a) definition and modelprimarily focus on internal mechanisms of information processing (e.g. perception,attention, memory); whereas Smith and Hancock (1995) contend SA is externallydirected consciousness, primarily driven by data within the operating environment.

170 Although possibly more cognitively plausible, Endsley’s definition lends itself lessto direct observations. Smith and Hancock’s interpretation may provide formore direct assessment of SA, but it is less representative of goal-directed processingin humans. The lack of a consistent and uniform definition of the constructcontributes to difficulty in defining objective measures and can make SA research

175 challenging.SA measures generally fall into two categories, including process-oriented

measures for situation assessment and outcome-oriented measures for evaluating theresult of the situation assessment; that is, states of situation awareness. Processmeasures are purported to allow for dynamic assessment of SA and to not influence

180 the achievement of SA by interrupting operators during performance (Pritchettand Hansmann 2000). However, criticism has been raised that events used for

4 C. M. Perry et al.

New XML Template (2006) [16.9.2006–7:11pm] [1–19]{TANDF_FPP}TTIE/TTIE_A_195831.3d (TTIE) [First Proof]

process-oriented measurement are artificial and therefore may influence performance(Endsley 2000). Advantages of outcome-oriented measures include objectivity andthe capability to support global assessment of SA, but they typically require

185 interruptions of operator performance (Jones and Endsley 2000).Realizing the limitations of the different interpretations of SA, in this study we

adopted Endsley’s definition because it allows for a fine-grained assessment of theconstruct at multiple levels of information processing, which may be associated withvarious types of cognitive behaviour (e.g., skill, rule and knowledge-based behaviour;

190 Rasmussen 1983). Furthermore, Endsley (1995b) developed the Situation AwarenessGlobal Assessment Technique (SAGAT), to operationally define SA in a particularcontext and allow for objective assessment of the construct. That is, SAGAT is anoutcome measure that evaluates SA by comparing actual states of an environmentwith operator verbalizations of their internal models. We describe this measure in

195 further detail in x 2.It is also important for us to note here that cognitive workload, like SA, is an

aspect of information processing that is not directly observable, but must be inferredbased on various measurement techniques. Cognitive workload has been defined ashow much mental effort an operator must expend on a task relative to available

200 resources (Wickens 1992). This definition, albeit simple, has generally been acceptedby human factors and ergonomics researchers. With respect to measuringcognitive workload, Wickens (2001) says that the construct may best be assessedand defined using converging operations, to include subjective, performance and,if possible, physiological measures. Historical research (e.g. Hart and Staveland

205 1988) has also demonstrated that subjective measures of cognitive workload,including the NASA-TLX (Task Load Index) may be reliable for assessing actualdemands on operators and predicting performance. We also say more about thismeasure in x 2.

For this study, we contend that both workload response and SA do not ‘cause’210 performance but define and support the potential for task performance by mediating

the effects of resource demands (Wickens 2001). Lastly, it is possible that the natureof the workload and SA measures chosen may lead to certain patterns of resultsapplicable to specific cognitive performance situations.

2. Method

215 An experiment was designed to evaluate the effects of different levels of physicalworkload on performance, SA and perception of cognitive workload in a complexplanning and decision-making task. We conceived a scenario of a soldier locomotingthrough a combat environment while using high-tech information displays andcommunications channels to manage a tactical helicopter operation in order to

220 simulate an occupation combining physical workload with cognitive planningrequirements. Participants stood, walked or lightly jogged on a treadmill for shortdurations while performing the cognitive task. At random points in time, thecognitive and physical tasks were paused and questions were posed to participants toassess their SA on the cognitive task. Subjective workload ratings were made at the

225 close of task performance.

Effects of physical workload on cognitive task performance and situation awareness 5

New XML Template (2006) [16.9.2006–7:11pm] [1–19]{TANDF_FPP}TTIE/TTIE_A_195831.3d (TTIE) [First Proof]

2.1. Cognitive task

A military helicopter-loading simulation (see figure 1) was developed as part of this

research effort and used as the cognitive task in the experiment. The goal of the task

was to load helicopters to a maximum weight capacity within an allotted timeframe,230 subject to a set of decision rules or constraints (see table 1). There were seven rules/

constraints that created dependencies among resources which could be loaded on

helicopters. Based on Eggemeier’s (1988) work, this task can be classified as a

planning activity, which involves multi-attribute decision analyses. It was developed

to address the need identified by Mozrall and Drury (1996) for evaluation of the235 effects of physical workload on planning/scheduling tasks. In addition, it was

expected that this type of cognitive task, if carried out in a real combat situation,

would typically be performed at the same time as some physical activity, such as

walking, running or lifting. For our scenario, we imagined an infantry leader running

through a ‘hot’ combat zone while looking at a PDA-based or helmet-mounted240 display of ground elements in the area and calling commands to team members for

loading evacuation helicopters.During simulation run time, one helicopter was ‘on the ground’ at all times.

Personnel and equipment were loaded onto the helicopter or removed from it by

pressing the respective control buttons. In the experiment, participants verbally245 called out the names and numbers of resources to be loaded on or removed from

a helicopter and an experimenter carried out their requests at an adjacent

Figure 1. The helicopter-loading simulation interface.

6 C. M. Perry et al.

New XML Template (2006) [16.9.2006–7:11pm] [1–19]{TANDF_FPP}TTIE/TTIE_A_195831.3d (TTIE) [First Proof]

computer workstation. Every 1.5 minutes into the simulation, a new event occurredand the quantities of personnel and equipment on the ground were updated. Theseevents were displayed in the event window on the right side of the simulation

250 interface and were intended to make the task dynamic and to drive cognitivecomplexity (see right side of figure 1). The helicopter took off when it was filled tomaximum capacity or when the pre-defined task time elapsed, and a new helicopterreplaced it immediately. The time allotted to fill a helicopter was a function of itscapacity; that is, the larger the helicopter, the longer the time to fill it. During

255 each trial, participants loaded three large helicopters and three small helicopters.Trials were approximately 10 minutes in duration.

2.2. Physical task

While performing the helicopter-loading task, participants were also required to usean exercise treadmill under three levels of physical workload: standing (the control

260 condition), walking and jogging. During the walking condition, participants walkedon the treadmill at a self-chosen, normal speed. Speeds ranged from 3.2–5.4 kph forall participants. During the jogging condition, participants jogged lightly at a paceexactly 50% faster than their self-chosen walking speed. This resulted in joggingspeeds of 4.8–8.2 kph across all participants. This physical activity was selected since

265 it was relatively simple to quantify differences between levels and because it had beenused extensively in other studies evaluating effects of physical workload on taskperformance (e.g. Mozrall and Drury 1996).

2.3. Experimental design

A single-factor, randomized complete block design with replication was used in this270 study. Physical workload, the independent variable, was manipulated within

subjects. Each participant completed two trials under each physical workloadcondition, for a total of six trials.

Dependent variables in the experiment included cognitive task performance,SA and subjective workload. Cognitive task performance in the helicopter-loading

Table 1. Set of decision rules for cognitive helicopter-loading task.

1. Soldiers must be loaded in the following priority: severely wounded, lightly wounded,platoon leaders, infantry and deceased.

2. At least two platoon leaders must be left on the ground at all times.3. If wounded personnel are sent on a large helicopter, a paramedic and medical supply

bag must be sent with them. Whenever a paramedic is sent (on a large or smallhelicopter), a medical supply bag must go with him.

4. One radio operator and one radio must be on board every helicopter.5. Radio operators are the only soldiers that don’t travel with a backpack. Everyone else

(including wounded and deceased soldiers) must be loaded with a backpack.6. Paramedics and radio operators carry light arms on board. Everyone else (including

wounded and deceased soldiers) carry heavy arms.7. At least three water containers must be loaded on each large helicopter; at least one

water container must be loaded on each small helicopter.

Effects of physical workload on cognitive task performance and situation awareness 7

New XML Template (2006) [16.9.2006–7:11pm] [1–19]{TANDF_FPP}TTIE/TTIE_A_195831.3d (TTIE) [First Proof]

275 simulation was measured in terms of accuracy and loading rate. Performance

accuracy was determined by the number of rules violated in loading the helicopters.

Loading rate was calculated as the number of pounds of personnel and equipment

loaded on the helicopter per second. Participants were asked to place equal emphasis

on achieving good accuracy and a high loading rate.280 Participant SA during the task was measured using the SAGAT (Endsley 1995b).

SAGAT is an objective SA assessment technique in which a simulation is temporarily

frozen to pose SA queries to operators on the state of a task/system in order to assess

their perception (Level 1 SA), comprehension (Level 2 SA) and projection (Level 3

SA) of task/system states. Two SAGAT freezes were introduced at random points in285 time during each trial, once during the loading of a small helicopter and once

during the loading of a large helicopter. During each freeze, the treadmill was

stopped (if the participants were walking or jogging), the simulation was suspended

and the display was blanked. A random sample of nine queries (three queries for

each level of SA) was presented on the display, one query at a time. The queries290 were selected from a pool of 18 queries developed based on a goal-directed

task analysis of the helicopter-loading simulation (see table 2). Participants

answered the SA queries verbally. No time limit was imposed for responding to

Table 2. Situation awareness queries and percentage of correct responses during experimentfor each query across all participants.

Level Query % Correct

1 How many deceased soldiers do you have on the ground? 431 How many water containers do you have on the ground? 211 How many platoon leaders and infantry soldiers do you have on the

helicopter?72

1 How many medical supplies bags do you have on the helicopter? 801 What is the remaining time until the helicopter takes off? 351 What is the helicopter’s current load? 462 Given the rules you’ve learned, how many backpacks should there be on

the helicopter?51

2 Given the rules you’ve learned, how many heavy arms should there beon the helicopter?

49

2 Given the rules you’ve learned, how many light arms should there be onthe helicopter?

77

2 Given the rules you’ve learned, how many bags of medical suppliesshould there be on the helicopter?

77

2 How many helicopters have you already filled? 502 What is the remaining capacity of the helicopter? 363 How many severely wounded soldiers do you think will be left on the

ground after this helicopter takes off?98

3 How many lightly wounded soldiers do you think will be left on theground after this helicopter takes off?

89

3 Do you think you will be able to board any infantry soldiers on thishelicopter based on the remaining capacity?

77

3 Do you think you will be able to load any platoon leaders on thishelicopter based on the remaining capacity?

57

3 Given the remaining capacity and the priority of the soldiers, what typeof soldier should be loaded next?

71

3 Given the remaining capacity and the rules you’ve learned, how manymore soldiers (any type) can you load on this helicopter?

55

8 C. M. Perry et al.

New XML Template (2006) [16.9.2006–7:11pm] [1–19]{TANDF_FPP}TTIE/TTIE_A_195831.3d (TTIE) [First Proof]

the queries; however, participants never required more than 2 minutes to completeall nine queries. After completion of a set of queries, the simulation display and

295 treadmill were reactivated and participants immediately resumed the helicopter-loading task from where they left off at the time of the freeze. Participant answers toSA queries were compared with the ‘ground truth’ recorded by the simulationcomputer system at each stop. The SA score for each stop included the percentcorrect responses to Level 1, 2, and 3 queries (also see table 2) and an overall

300 SA score.Some research (Adams et al. 1995) has contended that SAGAT freezes may

disrupt the flow of operator performance and the achievement of SA over time,such that performance after a freeze might be degraded. Related to this, empiricalwork (Endsley and Kaber 1999) has found that SAGAT freezes in a multi-task

305 scenario were no more disruptive to operator performance than control mode shiftsamong levels of task automation. Endsley (1995b) also found in earlier researchthat SAGAT freezing as frequently as every 2 minutes and lasting as long as6 minutes had no significant negative effects on subsequent simulation performance.With these findings in mind and the capability of SAGAT to provide for a

310 comprehensive assessment of SA the method was considered viable for investigatingthe effects of the physical workload manipulation on SA in the helicopter loadingtask.

The NASA-TLX (Hart and Staveland 1988) was used to capture participants’perception of workload during experimental trials along six dimensions: physical

315 demand, mental demand, temporal demand, effort, frustration and performance.This workload measure was selected after considering other subjective techniques,due to its demonstrated sensitivity (Hill et al. 1992) and reliability (NATO 2001).After initial training on the cognitive task while jogging, participants ranked theimportance of the six demand factors in relation to the task to establish weights for

320 each factor (NATO 2001). At the close of each trial, participants completed aNASA-TLX form, on which they rated the perceived level of demand for each of thesix factors. In order to obtain a composite workload score for each trial, individualfactor ratings were multiplied by the factor weights and summed together. Accordingto the NASA-TLX implementation procedure, one set of weights was generated

325 based on the functional characteristics of the task. Whether subjects stood, walked orran, their cognitive function remained the same; planning and making decisions onthe helicopter-loading task. Thus, the TLX ratings revealed the influence of loadmanipulations.

2.4. Apparatus and participants

330 The experimental set-up consisted of a Biodex RTM 400 rehabilitation treadmill,a 122 cm� 91 cm projection screen, an OnFocus portable data projector and a DellPrecision 430 workstation. The simulation display was projected on the screenlocated in front of the treadmill (see figure 2).

Sixteen participants, primarily from the North Carolina State University student335 population, volunteered for the study by responding to web-based or hard copy

recruitment flyers. Participants ranged in age from 19–37 years. Thirteen participantswere male and three were female. All participants had 20/20 or corrected to normalvision. Participants with a history of cardiac problems were excluded from the study

Effects of physical workload on cognitive task performance and situation awareness 9

New XML Template (2006) [16.9.2006–7:11pm] [1–19]{TANDF_FPP}TTIE/TTIE_A_195831.3d (TTIE) [First Proof]

because of the physical task load manipulation and the potential for elevated aerobic340 demands during jogging. The participant who achieved the highest performance

score on the simulation received a gift certificate.

2.5. Procedure

Each participant completed a 10-minute familiarization with the cognitive task

(military helicopter-loading simulation) and 20 minutes of comprehensive training345 on the cognitive task and treadmill use. During this time, the participant’s self-

chosen walking pace was recorded. At the close of the training period, a task

proficiency test was administered to ensure participant understanding of the

helicopter-loading decision rules. If participants did not achieve a score of at least

90% on the test or if they were not able to load at least two helicopters without350 error during the training trial, further training was provided. In the entire study, only

one participant required additional training on the task, after which he loaded two

additional helicopters without error. Subsequently, a 10-minute familiarization

with SA queries was provided. This was followed by a dual-task practice trial

during which participants jogged while performing the cognitive task. Finally,355 participants were familiarized with the NASA-TLX and they completed the demand

ranking form.After the training period, participants were subjected to six 10-minute

experimental trials. Ten-minute breaks were included after walking and jogging

trials. The experiment lasted approximately 3 hours for each participant.

Figure 2. Experiment set-up.

10 C. M. Perry et al.

New XML Template (2006) [16.9.2006–7:11pm] [1–19]{TANDF_FPP}TTIE/TTIE_A_195831.3d (TTIE) [First Proof]

360 2.6. Data analysis

Statistical analyses included analyses of variance (ANOVA) to investigate theinfluence of physical workload on operator SA, perceived workload and taskperformance. An alpha level of 0.05 was used for establishing statistical significance.Further investigation of significant predictors was conducted using Duncan’s

365 Multiple-Range tests, also with an alpha criterion of 0.05.

3. Results

3.1. Situation awareness

Table 3 presents the mean Level 1, Level 2, Level 3 and overall SA scores (withstandard deviations) for the various physical workload conditions. This information

370 is presented graphically in figure 3. The plot reveals that, on average, participantsexhibited better overall SA on the cognitive task during the standing andwalking conditions than while jogging. Table 2 shows the percentage of correctresponses to each SA query.

ANOVA results revealed a significant effect of physical workload375 (F(2, 172)¼ 3.59, p¼ 0.0296) on the percentage of correct responses to Level 1 SA

queries. In agreement with our hypothesis, Duncan’s test revealed significantlygreater participant perceptual knowledge on the virtual helicopter-loading environ-ment during the walking condition, as compared to the jogging condition. That is, anincrease in the level of physical workload from the moderate to extreme setting led to

380 degradation of SA. It is also important to note that SA, when standing, did notstatistically differ from walking or jogging.

ANOVA results also revealed a significant effect of physical workload(F(2, 172)¼ 6.16, p¼ 0.0026) on Level 2 SA. According to Duncan’s post-hoc test,the standing and walking conditions produced a significantly greater (p<0.05)

385 percentage of correct responses to Level 2 SA queries in comparison to the joggingcondition. These findings were also in line with our hypothesis.

Contrary to expectation, ANOVA results showed no significant main effect ofphysical workload on Level 3 SA, that is participant ability to project future statesof the helicopter load planning task.

390 ANOVA results revealed that the total SA score (summation of percentage ofcorrect responses across all queries) was significantly affected by physical workload

Table 3. Situation awareness scores (in percentages) and standard deviations across allparticipants.

Level 1 Level 2 Level 3 Overall

Mean SD Mean SD Mean SD Mean SD

Stand 48 30 64 28 72 29 61 18Walk 57 29 58 29 77 24 64 17Jog 44 26 46 30 74 27 55 17Total 50 29 56 30 75 27 60 18

Effects of physical workload on cognitive task performance and situation awareness 11

New XML Template (2006) [16.9.2006–7:11pm] [1–19]{TANDF_FPP}TTIE/TTIE_A_195831.3d (TTIE) [First Proof]

(F(2, 172)¼ 5.04, p¼ 0.0075). Post-hoc tests indicated that scores were significantlyhigher (p<0.05) during standing and walking vs jogging.

3.2. Cognitive task performance

395 ANOVA results did not reveal any significant effect of physical workload oncognitive task performance, including accuracy and loading rate. The mean accuracyand loading rates across the three levels of physical workload are presented intable 4 along with standard deviations for each condition. The differences amongthe conditions did not meet our criterion for significance.

400 3.3. Subjective workload

Figure 4 presents the mean percentage NASA-TLX (overall and individual factor)scores for various physical workload conditions. ANOVA results revealed significanteffects of physical workload on the overall TLX score (F(2, 48)¼ 7.13, p¼ 0.0019),as well as individual factors, including perceived mental demand (F(2, 48)¼ 4.56,

405 p¼ 0.0153), physical demand (F(2, 48)¼ 174.10, p<0.0001) and effort(F(2, 48)¼ 8.10, p¼ 0.0009) components.

0%

20%

40%

60%

80%

100%

Level 1 SA Level 2 SA Level 3 SA Total SAPer

cen

t co

rrec

t re

spo

nse

s to

qu

erie

s

Stand Walk Jog

AB

AB

A A

B

A A A

A A

B

Figure 3. Mean percentage of correct responses to SA queries by physical workloadcondition. Different letters above the bars in the chart represent significantly different meansaccording to Duncan’s tests.

Table 4. Accuracy and loading rates in the helicopter simulation across all participants.

Accuracy (rule violations) Loading Rate (lb s�1)

Mean SD Mean SD

Stand 2.03 5.16 50.6 19.98Walk 2.27 5.96 50.49 18.37Jog 2.6 5.8 50.6 20.12

12 C. M. Perry et al.

New XML Template (2006) [16.9.2006–7:11pm] [1–19]{TANDF_FPP}TTIE/TTIE_A_195831.3d (TTIE) [First Proof]

In agreement with our hypothesis, Duncan’s tests revealed significantly higher(p<0.05) workload ratings during jogging than during the walking and standingconditions for the overall workload as well as the mental demand and effort

410 components. Not surprisingly, each of the three physical load conditions wasperceived as being significantly different in terms of physical demand. The highest(p<0.05) ratings of physical demand occurred under the jogging condition,followed by walking and then standing. This trend validated the independentvariable manipulation and our expectations of differences among the physical

415 workload settings.

4. Discussion

The goal of this experiment was to assess the effects of physical workload oncognitive task performance and SA. Earlier research revealed conflicting findingswith respect to the relationship between physical load and basic cognitive task

420 performance and no empirical evidence had been reported on the impact of physicalload on complex decision-making, planning or scheduling tasks (Tenenbaum et al.1993, Mozrall and Drury 1996, McMorris and Graydon 2000, Tomporowski 2003).In this experiment, although the physical and cognitive tasks were presented as anintegrated dual-task scenario, different levels of physical workload did not appear to

425 influence cognitive task performance. Our expectation of changes in cognitiveperformance in association with physical load manipulations was based on thepotential for depletion of attentional resources and degradation of SA. Morespecifically, Hancock and Warm (1989) previously demonstrated that environmentalstressors, including noise and temperature, led to reductions in attentional resources

0%

20%

40%

60%

80%

Men

tal d

eman

d

Pysica

l dem

and

Tempo

ral d

eman

d

Perfo

rman

ce

Frustr

ation

Effort

Overa

ll

Per

cen

t N

AS

A-T

LX

rat

ing

s

Stand

Walk

Jog

A AB

A

B

C A AB

A AB

Figure 4. Mean subjective workload ratings by physical workload condition. Differentletters above the bars in the chart represent significantly different means according toDuncan’s test, for those TLX demand components significantly affected by physical workloadmanipulation.

Effects of physical workload on cognitive task performance and situation awareness 13

New XML Template (2006) [16.9.2006–7:11pm] [1–19]{TANDF_FPP}TTIE/TTIE_A_195831.3d (TTIE) [First Proof]

430 and cognitive performance degradations. Endsley (1995a) also predicted that

physical stressors like noise and temperature might degrade SA, potentially leading

to problems in complex task performance.Several explanations for the current observed pattern of results are plausible.

First, the range of intensities of the exercise (physical workload) investigated may not435 have been sufficient to cause differences in helicopter loading speed and adherence to

decision rules. Secondly, it is possible that the complexity of the cognitive task used

in the study may not have been sufficient to cause performance to be sensitive to the

physical workload manipulations. Participants were able to fill most helicopters to

full or near full capacity within the allotted time, making only a minimal number of440 violations. Some evidence of a ceiling effect in cognitive task performance was

observed, specifically in terms of performance speed; however, the general trend on

loading accuracy did decrease with increasing physical workload. Thirdly, it is

possible that performance under higher physical workload was maintained by the

recruitment of additional cognitive resources (Hockey 1997), but only at the expense445 of increased subjective effort, an explanation supported by the NASA-TLX results.

Yeh and Wickens (1988) also described instances where performance and

subjective workload measures dissociated when greater resources were invested to

improve performance on a resource-limited task. Finally, it is possible that individual

differences in helicopter-loading strategy may have masked differences450 attributable to the experimental manipulation. Some participants chose to

immediately load large batches of personnel and equipment onto the helicopter

and then they fine-tuned their resource selections to fill the remaining helicopter

capacity. In contrast, other participants chose to load smaller batches of personnel

and equipment at a time and their helicopter fill rate was more incremental in nature.455 As evidenced by a negative relationship between loading rate and rule violations,

it appeared that participants choosing the former strategy loaded helicopters faster

and with fewer mistakes, suggesting an optimal strategy for the task. Across levels of

physical workload, there was very little difference between loading rates (ranging

from 50.5 lb s�1 for walking to 50.6 lb s�1 for jogging), whereas the differences across460 participants (ranging from 41.9–59.7 lb s�1) were comparatively large.

Perhaps the most interesting results of this study are the effects of physical

workload on SA. The findings support our hypothesis that physically demanding

conditions result in lower SA on the states of the task environment. The statistical

results on Level 1, Level 2 and total SA all showed SA to degrade with increasing465 physical load and for the worst SA to occur under the jogging condition. This is in

line with Endsley’s (1995a) contention that certain types of physical stressors may

deteriorate SA. The interesting departure of our results from expectation was that,

while SA was worst during the jog (for all three measures), there appeared to be no

difference between the standing and walking conditions. In general, participants470 appeared to achieve substantially greater comprehension, for example, of the

helicopter-loading environment across the less physically demanding conditions.

Therefore, some level of physical load may be tolerable with respect to achieving and

maintaining SA in planning tasks and may even be beneficial considering the

observed trends of mean SA across the physical workload conditions. Improvements475 in participants’ SA while walking could possibly be due to overall arousal and higher

engagement in the task, as compared to no physical load whatsoever. The very high

physical workload condition (jogging) may have led to difficulties in allocating

14 C. M. Perry et al.

New XML Template (2006) [16.9.2006–7:11pm] [1–19]{TANDF_FPP}TTIE/TTIE_A_195831.3d (TTIE) [First Proof]

resources to maintain the task state under stress and overload, thereby degrading SAas compared to the moderate physical load (Hockey 1997).

480 There may be several reasons to explain the deviation of the Level 3 SA resultsfrom the common finding on Levels 1, 2 and total SA; that is Level 3 SA did notdecrease with increasing physical load. First, it could be the case that the differentlevels of SA (perception, comprehension and projection) are truly differentiallyaffected by physical workload. This speculation is supported by the three-

485 dimensional classification scheme of (1) processing resources, (2) mode of processingand (3) physical exertion proposed by Mozrall and Drury (1996) in order to controlfor factors that might contribute to conflicting findings in this line of literature.A second possibility is that the Level 3 SA queries developed for this simulationmay not have been at the same level of difficulty as the Level 1 and 2 SA queries.

490 The questions may not have been sufficiently challenging for participants in terms ofintegrating simulation state information to predict future events or actions and,therefore, did not reveal the same significant effects of physical workload conditionsas all other SA measures.

Considering the performance and SA results together, there were significant495 differences in SA across the levels of physical workload, but this did not appear to

translate to differences in cognitive task performance. Accuracy and loading ratesacross the three levels of physical workload were not significantly different, though,on average, decision rule violations appeared to increase with increasing physicalworkload and jogging generated the highest mean number of errors. This finding was

500 counter to the hypothesis that significant decrements in SA due to physical loadingmanipulations would lead to corresponding changes in cognitive task performance.Specifically, we expected that higher physical workload would lead to increaseddemands on the operator in the dual-task scenario, causing a decrease in SA andreduced performance.

505 In general, good SA can be viewed as a critical factor to performance in complexcognitive tasks; however, it does not guarantee good performance (Endsley 1995a).For instance, in an air-to-air combat mission, Endsley (1990) found that SA wassignificantly related to performance only for those participants who had the technicaland operational capabilities to take advantage of such knowledge. The same study

510 found that poor SA did not necessarily lead to poor performance, if participantsrealized their lack of SA and were able to modify their behaviour. Some studies (e.g.Endsley and Kiris 1995) have shown that low SA does, in fact, correspond withperformance decrements in complex cognitive tasks.

Some factors in the lack of correspondence of SA and cognitive performance515 results in this study may include: the set of variables influencing cognitive task

performance versus SA; the sensitivity of the performance measures to differences inthe internal situation models of operators; the extent to which participants needed torely on their internal situation model for performance; and the complexity orworkload posed by the cognitive task in terms of achieving and maintaining SA. It is

520 possible that the simulated military helicopter-loading task used in the experimentdid not demand a very precise internal situation model for accurate taskperformance. Participants may have been able to perform reasonably well, asmeasured by how quickly and correctly they could load helicopters, without havingprecisely correct SA, as measured by the pool of SA queries regarding the numbers of

525 resources, load capacity, task time remaining, etc. Having ‘roughly right’ SA in thiscase (as evidenced by average SA scores between 45–75% correct responses to

Effects of physical workload on cognitive task performance and situation awareness 15

New XML Template (2006) [16.9.2006–7:11pm] [1–19]{TANDF_FPP}TTIE/TTIE_A_195831.3d (TTIE) [First Proof]

queries) may have been adequate to complete the basic aspects of the cognitivetask. We suspected that these factors were more likely candidates in the lack ofcorrespondence of the SA and cognitive performance results than, for example,

530 the limitations of the SAGAT measure (operational definition of SA) identifiedin the literature, in terms of implementation or administration procedures.We think that future research to quantify the margin of acceptable error in one’sinternal situation model in such tasks could be very useful from a training designperspective.

535 5. Conclusions

In general, the results of this study may have practical implications for designingmulti-tasking scenarios involving mixed (physical and cognitive) workload in acomplex and dynamic work environment. Although it is difficult to make specificrecommendations regarding the design of work environments where both physical

540 and cognitive tasks must be performed, this experiment provides general insightregarding performance and SA in these situations. First, the study begins to fill a gapin knowledge of the effects of physical workload on the cognitive tasks of planningand complex decision-making. Taking into account the specific task type andsimulation used, there appeared to be no effect of physical workload on objective

545 task performance measures on planning and complex decision-making, but asignificant increase in subjective workload was observed. Secondly, the studyprovided evidence of the effects of physical workload on operator SA. In general,high physical load appears to degrade SA, including perception and comprehension,but projection was not consistent in response. Although SA has previously been

550 identified as being important to complex decision-making tasks, in our simulation wedid not see direct evidence of decreases in SA translating to cognitive taskperformance problems. We believe this has to do with the task design and the degreeof reliance of participants on an internal situation model for task performance.The observations made here represent an important basis for other studies in this

555 area. The results provide a platform for more specific hypothesis formulation onthe general trends of perception, comprehension and projection across low to highphysical workload conditions. Based on the trends in our data, it may be worthwhileto investigate whether there is some physical arousal threshold below which SAcan be achieved effectively and consistently and above which SA begins to degrade.

560 Future research is warranted to more fully explore the effects of physical workloadon operator SA and to determine the specific relationship across other task typesand domains.

Acknowledgements

We would like to thank Gary Mirka for his instruction of a research practicum565 course, which was integral to the development of this project. We would also like to

thank him for his insightful comments on an early version of this paper. MohamedSheik-Nainar and David Kaber’s work on this project was supported in part by an

16 C. M. Perry et al.

New XML Template (2006) [16.9.2006–7:11pm] [1–19]{TANDF_FPP}TTIE/TTIE_A_195831.3d (TTIE) [First Proof]

Army/OSD Small Business Innovative Research Contract (No.DASW01-04-C-004)through SA Technologies, Inc. The opinions expressed in this paper are those of the

570 authors and do not necessarily reflect the views of the Army.

References

ADAMS, M.J., TENNEY, Y.J. and PEW, R.W., 1995, Situation awareness and cognitive management ofcomplex systems. Human Factors, 37(1), pp. 85–104.

BILLS, A. and STAUFFACHER, J.C., 1937, The influence of voluntarily induced tension on rational575 problem solving. Journal of Psychology, 4, pp. 261–271.

EASTERBROOK, J.A., 1959, The effect of emotion on cue utilization and the organization of behavior.Psychological Review, 66(3), pp. 183–201.

EGGEMEIER, F.T., 1988, Properties of workload assessment techniques. In Human Mental Workload,P.A. Hancock and N. Meshkati (Eds), pp. 41–62 (Amsterdam: North-Holland Elsevier

580 Science).ENDSLEY, M.R., 1990, Predictive utility of an objective measure of situation awareness.

In Proceedings of the Human Factors Society 34th Annual Meeting, pp. 41–45 (SantaMonica, CA: Human Factors and Ergonomics Society).

ENDSLEY, M.R., 1995a, Toward a theory of situation awareness in dynamic systems. Human585 Factors, 37(1), pp. 32–64.

ENDSLEY, M.R., 1995b, Measurement of situation awareness in dynamic systems. Human Factors,37(1), pp. 65–84.

ENDSLEY, M.R., 2000, Direct measurement of situation awareness: Validity and use of SAGAT.In Situation awareness analysis and measurement, M.R. Endsley and D.J. Garland (Eds),

590 (Mahwah, NJ: LEA).ENDSLEY, M.R. and JONES, W.M., 1997, Situation awareness, information dominance, and

information warfare ( Tech Report 97-01) (Belmont, MA: Endsley Consulting).ENDSLEY, M.R. and KIRIS, E.O., 1995, The out-of-the-loop performance problem and level of

control in automation. Human Factors, 37(2), pp. 381–394.595 HANCOCK, P.A. and WARM, J.S., 1989, A dynamic model of stress and sustained attention. Human

Factors, 31(5), pp. 519–537.HANCOCK, S. and MCNAUGHTON, L., 1986, Effects of fatigue on ability to process visual information

by experienced orienteers. Perceptual & Motor Skills, 62, pp. 491–498.HART, S.G. and STAVELAND, L.E., 1988, Development of NASA-TLX Task Load Index: Results of

600 empirical, & theoretical research. In Human Mental Workload, P.A. Hancock andN. Meshkati (Eds), pp. 139–183 (Amsterdam: North-Holland Elsevier Science).

HILL, S.G., IAVECCHIA, H.P., BYERS, J.C., BITTNER, A.C., ZAKLAD, A.L. and CHRIST, R.E., 1992,Comparison of four subjective workload rating scales. Human Factors, 34(4), pp. 429–439.

HOCKEY, G.R. J., 1997, Compensatory control in the regulation of human performance under stress605 and high workload: A cognitive-energetical framework. Biological Psychology, 45, pp. 73–93.

JONES, D.G. and ENDSLEY, M.R., 2000, Can real-time probes provide a valid measure of situationawareness? In Human Performance, Situation Awareness and Automation: User-centeredDesign for the New Millennium, D.B. Kaber and M.R. Endsley (Eds), pp. 245–250 (Atlanta,GA: SA Technologies).

610 KABER, D.B., RILEY, J.M., LAMPTON, D. and ENDSLEY, M.R., 2005, Measuring situation awarenessin a virtual urban environment for dismounted infantry training. In Proceedings of the 1stInternational Conference on Virtual Reality (CD-ROM), Las Vegas, NV, 22–27 July.

MA, R. and KABER, D.B., 2005, Situation awareness and workload in driving while using adaptivecruise control and a cell phone. International Journal of Industrial Ergonomics, 35(10),

615 pp. 939–953.MASTROIANNI, G.R., CHUBA, D.M. and ZUPAN, M.O., 2003, Self-pacing and cognitive performance

while walking. Applied Ergonomics, 34, pp. 131–139.MCMORRIS, T. and GRAYDON, J., 2000, The effect of incremental exercise on cognitive performance.

International Journal of Sport Psychology, 31, pp. 66–81.

Effects of physical workload on cognitive task performance and situation awareness 17

New XML Template (2006) [16.9.2006–7:11pm] [1–19]{TANDF_FPP}TTIE/TTIE_A_195831.3d (TTIE) [First Proof]

620 MOZRALL, J.R. and DRURY, C.G., 1996, Effects of physical exertion on task performance in modernmanufacturing: A taxonomy, a review, and a model. Ergonomics, 39(10), pp. 1179–1213.

NORTH ATLANTIC TREATY ORGANIZATION, RESEARCH AND TECHNOLOGY ORGANIZATION, 2001,NATO guidelines on human engineering testing and evaluation (Tech. Report RTO-TR-021)(Neuilly-Sur-Seine Cedex, France: Author).

625 PRITCHETT, A.R. and HANSMAN, R.J., 2000, Use of testable responses for performance-basedmeasurement of SA. In Situation Awareness Analysis and Measurement, M.R. Endsley andD.J. Garland (Eds), pp. 189–209 (Mahwah, NJ: Lawrence Erlbaum).

RASMUSSEN, J., 1983, Skills, rules, knowledge: signals, signs, and symbols and other distinctions inhuman performance models. IEEE Transactions on Systems, Man, and Cybernetics, 13,

630 pp. 257–267.REILLY, T. and SMITH, D., 1986, Effect of work intensity on performance in a psychomotor task

during exercise. Ergonomics, 29, pp. 601–606.SJOBERG, H., 1980, Physical fitness and mental performance during and after work. Ergonomics, 23,

pp. 977–985.635 SMITH, K. and HANCOCK, P.A., 1995, Situation awareness is adaptive, externally directed

consciousness. Human Factors, 37(1), pp. 137–148.TENENBAUM, G., YUVAL, R., ELBAZ, G., GAR-ELI, M. and WEINBERG, R., 1993, The relationship

between cognitive characteristics and decision-making. Canadian Journal of AppliedPhysiology, 18, pp. 48–62.

640 TOMPOROWSKI, P.D., 2003, Effects of acute bouts of exercise on cognition. Acta Psychologica, 112,pp. 297–324.

TOMPOROWSKI, P.D. and ELLIS, N.R., 1986, Effects of exercise on cognitive processes: A review.Psychological Bulletin, 99(3), pp. 338–346.

USHER, J.M. and KABER, D.B., 2000, Establishing information requirements for supervisory645 controllers in a flexible manufacturing system using GTA. Human Factors and Ergonomics in

Manufacturing, 10(4), pp. 431–452.VENTURINO, M., HAMILTON, W.L. and DVORCHAK, S.R., 1989, Performance-based measures of

merit for tactical situation awareness. In Situation Awareness in Aerospace Operations(AGARD-CP-478) (Conference Proceedings #478), pp. 4/1–1/5 (Neuilly Sur Seine, France:

650 NATO-AGARD).WICKENS, C.D., 1992, Engineering Psychology and Human Performance (New York: Harper

Collins).WICKENS, C.D., 2001, Workload and situation awareness. In Stress, Workload and Fatigue,

P.A. Hancock and P.A. Desmond (Eds), pp. 443–450 (Mahwah, NJ: Lawrence Erlbaum655 Associates).

YEH, Y.Y. and WICKENS, C.D., 1988, Dissociation of performance and subjective measures ofworkload. Human Factors, 30(1), pp. 111–120.

About the authors

Carlene M. Perry is currently a doctoral candidate in the Department of Industrial660 and Systems Engineering at North Carolina State University. She received her

Masters degree in Industrial Engineering from the Pennsylvania State University and

her Bachelors degree from the US Air Force Academy. She is currently serving asa Major in the active duty Air Force.

Mohamed A. Sheik-Nainar is currently a doctoral candidate in Industrial665 Engineering at North Carolina State University. He received his Masters degree in

Computer Engineering from NC State and Bachelors in Mechanical Engineering

from University of Madras, India. His research interests span virtual reality andlocomotion, telepresence and teleoperation and human–computer interaction.

18 C. M. Perry et al.

New XML Template (2006) [16.9.2006–7:11pm] [1–19]{TANDF_FPP}TTIE/TTIE_A_195831.3d (TTIE) [First Proof]

Noa Segall received her PhD in Industrial Engineering from North Carolina State670 University. She received her Masters degree from Oregon State University in

Industrial Engineering and her Bachelors degree from the Technion – Israel Institute

of Technology in Mechanical Engineering. Her past research involved developing

and implementing PDA-based exams and evaluating their usability. Her current

research interests include human factors in automation design, human–computer675 interaction and human factors in medical systems.

Ruiqi Ma received his PhD in Industrial Engineering in 2005 from North Carolina

State University. He also received his Masters degree in 2002 from NC State in the

same discipline and his Bachelors degree from Science and Technology University

Beijing in Industrial Automation. His current research interests include trust in680 human-automation interaction, human-computer interaction and telepresence

in teleoperations.

David B. Kaber is an associate professor of industrial engineering at North Carolina

State University. He received his PhD in industrial engineering from Texas Tech

University in 1996. His current research interests include human–automation685 interaction, multi-modal interface design for complex systems and presence in

teleoperations and virtual environments.

Effects of physical workload on cognitive task performance and situation awareness 19