Stress and Cognitive Load Dan Conway Page: 1 Stress and Cognitive Load NICTA Summer Scholarship Report ‐ 2011/2012. Dan Conway Supervisors: Dr Yang Wang, Dr Fang Chen ABSTRACT Human physiological signals have been suggested as proxies with which to non‐invasively quantify Cognitive Load (CL) in real‐time. A major challenge for any approach to CL detection is the presence of stress, which may affect physiological measurements in ways that confound reliable detection of CL. In this experiment we attempted to dissociate CL from stress. GSR was used as a proxy for stress and pre‐frontal cortical activity as measured by EEG signals as a proxy for CL. Three levels of CL were induced in 11 participants using math tasks in both ‘no‐stress’ and ‘stress’ conditions. The experiment used a modified version of the MIST protocol which utilizes feelings of lack of control, task failure and self and social‐evaluation to induce stress. Using basic statistical measures for eight subjects, GSR levels were shown to be significantly different between CL levels in the ‘no‐stress’ condition, but not in the ‘stress’ condition. This has important implications for CL quantification in that other physiological signals may also exhibit similar patterns where a stress response over‐rides signal variation owing to CL. Further analysis of the body of data generated by this experiment, utilising machine learning techniques is suggested. INTRODUCTION Physiological signals have previously been proposed as a method of quantifying Cognitive Load (CL). Signals include heart‐rate, heart‐rate variability, pupil‐dilation, blood‐pressure, respiration rate and GSR (Galvanic Skin Response). Some notable successes in CL quantification have been achieved via signals such as speech (Chen, 2006), Heart Period (Veltman & Gaillard, 1998), Pupillary Response (Xu, 2011), Heart Rate Variability (Aasman, Mulder, & Mulder, 1987). However, any given physiological signal is only a proxy for CL and is likely to be effected by countless additional inputs from the human body (Longo, et al., 2010). Thus a major task of CL measurement via physiological means is demonstrating the diagnosticity and construct validity of any nominated proxy physiological signal. One of the major contributors to change in human physiological systems is stress (Martin, 2007). Stress has been shown to effect both the sympathetic and parasympathetic nervous systems and, in its more extreme states, results in large changes to physiological function that may well obscure the relationship between a physiological indicator and CL. Furthermore – stress may, in some circumstances, be a confound for CL in that changes in CL may correlate with changes in stress‐levels (Veltman & Gaillard, 1998). Construct validity must be established before we can safely assert that changes in physiological indicators are the result of CL and not stress (or indeed some other factor).
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Stress and Cognitive Load Dan Conway Page: 1
Stress and Cognitive Load
NICTA Summer Scholarship Report ‐ 2011/2012.
Dan Conway
Supervisors: Dr Yang Wang, Dr Fang Chen
ABSTRACT
Human physiological signals have been suggested as proxies with which to non‐invasively quantify
Cognitive Load (CL) in real‐time. A major challenge for any approach to CL detection is the presence
of stress, which may affect physiological measurements in ways that confound reliable detection of
CL. In this experiment we attempted to dissociate CL from stress. GSR was used as a proxy for stress
and pre‐frontal cortical activity as measured by EEG signals as a proxy for CL. Three levels of CL were
induced in 11 participants using math tasks in both ‘no‐stress’ and ‘stress’ conditions. The
experiment used a modified version of the MIST protocol which utilizes feelings of lack of control,
task failure and self and social‐evaluation to induce stress. Using basic statistical measures for eight
subjects, GSR levels were shown to be significantly different between CL levels in the ‘no‐stress’
condition, but not in the ‘stress’ condition. This has important implications for CL quantification in
that other physiological signals may also exhibit similar patterns where a stress response over‐rides
signal variation owing to CL. Further analysis of the body of data generated by this experiment,
utilising machine learning techniques is suggested.
INTRODUCTION
Physiological signals have previously been proposed as a method of quantifying Cognitive Load (CL).
Signals include heart‐rate, heart‐rate variability, pupil‐dilation, blood‐pressure, respiration rate and
GSR (Galvanic Skin Response). Some notable successes in CL quantification have been achieved via
signals such as speech (Chen, 2006), Heart Period (Veltman & Gaillard, 1998), Pupillary Response
22 + 31 + 44 = 774 ‐ 447 + 315 ‐ 71 = 245 + 687 ‐ 22 x 29 + 558 = Table 2: Examples of the math problems presented.
The formulation of these problems came about through a reasonably intensive period of pilot testing
where it was found that the number of terms, carrying and having to remember numbers whilst
performing other operations (such as required by the order of operations in level 3 problems) were
the most reliable method of increasing subjective difficulty ratings. Math problems examples are
presented in Table 2 (with a full list available in Appendix A).
Apparatus
All experimental stimuli were presented on a VDU using custom software (SACL V1.0) whilst
participants were sitting comfortably at a desk.
An Emotiv ‘Epoc’ headset was fitted to the participants’ heads according to the protocol outlined in
‘Emotiv Beta EPOC Hardware Setup Guide Revision 1.0’. Saline solution was applied to the sensors
and contact was reliable and stable for all participants for all sensors with the exception of P7 and P8
which were either intermittent or made no contact at all for four participants (seemingly owing to
the shape of the participants head). Participants’ eyeglasses seemed to make no difference to
contact quality. EEG signals were then recorded using the Emotiv ‘Testbench’ application,
monitored visually during the experiment by the experimenter and later converted to XLSX files for
data analysis. Markers were sent by the SACL application to the Testbench software via virtual serial
ports at the end of each information screen and at the beginning and end of each block.
Stress and Cognitive Load Dan Conway Page: 6
Event Marker value sent by SACL to Testbench software
Experiment Begin Button is pressed 30
End of any information/Inter‐block pause screen.
32
End of the ‘Nominate Target Score’ screen (when the experimenter is to enter the room and enabled the video screens of the ‘observers’ for the beginning of the stress condition.
31
Block begins/ends. Block markers are three digit numbers. The format is:
1st digit: 1 (indicates it’s a block marker)
2nd digit: o 1 = Task begin. o 0 = Task end.
3rd Digit: Block Number. Eg: 103 = Block marker, end of block, block 3.
Table 3: Markers codes sent by SACL to Testbench software for insertion into EEG output EDF file.
GSR signals were collected using a Thought Technology ‘ProComp Infiniti’ interface and its ‘SC
Flex/Pro’ skin conductance sensor. The sensors were attached to D2 and D4 of the non‐dominant
had for all participants. GSR signals were sampled at a rate of 256Hz via custom software developed
by Ronnie Taib for NICTA.
Participants were asked to remain perfectly still and only move their dominant hand for mouse
control during the experiment. Participants were not asked to suppress blinking as this may have
added to the ‘stress’ during measurement of their baseline state and during the ‘No‐stress’
condition.
Procedure
All participants undertook the ‘no‐stress’ condition first.
Participants were told that they would be completing math tasks but it was emphasized that their
performance/accuracy was not important and in all likelihood that their performance and accuracy
data would not even be examined. They were told that the tasks were designed to induce different
levels of CL and the aim of the experiment was solely to try and measure CL via the GSR and EEG
systems. The experimenter maintained an informal and casual tone with the participants before
leaving the room to allow the participant to commence the experiment.
After submitting some basic demographic information, a two minute ‘baseline’ period was carried
out where the participants were told, via an on‐screen prompt, that they should just relax and let
their mind wander. Then three two‐minute blocks of math tasks were presented with 4 multiple
choice answers available for response by clicking on‐screen buttons with the mouse. Tasks were not
time‐limited and feedback was not provided. The blocks were not terminated until a participant
finished the current question, therefore some blocks were longer owing to participants finishing a
question after the 2 prescribed two minutes.
Stress and Cognitive Load Dan Conway Page: 7
The three blocks in the ‘no‐stress’ condition were of level 1, 2 and 3 difficulty in sequential order.
Between each block the participant was given a two minute ‘pause’ to allow physiological signals
time to return to baseline.
After block three, the participants were asked, via on screen prompts, to nominate a ‘target score’
for further tasks based on their estimation of their performance so far. Once submitted, the stress
condition ensued. They were told that their performance would be now be monitored. They were
also informed of time limits for further trials. At this point the experimenter entered the lab and
switched on two large LCD televisions behind and above the participant’s computer monitor, thus
directly in the field of view of the participant, and a video projector aimed at a large screen directly
to the participants left. One LCD screen displayed a video feed of the participants face from a small
webcam mounted on the desk. The second LCD screen displayed a video feed of ‘observers’ staring
into the camera, and therefore apparently at the participant. The large screen to the participants
left displayed a ‘mirrored’ image of the screen that the participant was using, thus rendering their
performance highly visible. Participant were then told that they would now be able to see the
observers for the rest of the experiment, implying that they had been being watched all along from
behind the one‐way glass of the observation room. The ‘observers’ were, unbeknownst to the
participant, actually only a pre‐recorded video, but appeared to be Hugh‐Durrant‐White, the CEO of
Nicta, a fellow student, and the experimenter.
Figure 3: A participant during the stress condition. His own image is displayed on the left hand LCD television screen, the ‘observers’ on
the right hand LCD television screen, and the experiment screen projected on the large screen to his left.
The participant was then told to continue and the experimenter returned to the observation room.
The pre‐recorded video of the ‘observers’ was timed so that the experimenter appeared to take his
place amongst the ‘observers’ at this point.
Stress and Cognitive Load Dan Conway Page: 8
Now in the stress condition, three more blocks of level 1, 2 and 3 difficulty level math tasks were
then carried out, again with two minute pauses in between each block but with time limits now
imposed on each trial. These were calculated by taking the mean of the RT’s for the same CL level
block in the ‘no‐stress’ condition and multiplying this time by .9. This time limit was also then
dynamically updated. In cases where participants correctly answered three questions in a row
within the trial time limits, the time limit was further multiplied by .9. In cases where the participant
either ran out of time or responded incorrectly to three questions in a row, the time limit was
multiplied by 1.1. The participants nominated target score was displayed on the left hand side of the
screen during all trials, and their current percentage of correct answers was displayed on the right
hand side. Feedback (‘Correct’, ‘Wrong’ or ‘Out of time’) was provided for one second after each
trial.
Figure 4: The screen presentation of a typical (level 1) math task in the ‘stress’ condition.
Once all six blocks had been completed the experiment concluded and the participant was
immediately told that the video of the ‘observers’ had been pre‐recorded and ‘the boss’ had not, in
fact, been watching them. They were then debriefed verbally, encouraged to ask questions, as well
as given a text debriefing document to take away with them. Participants were also asked to fill out
a questionnaire where they could report on their experience of the experiment and they were given
the option of revoking their consent for NICTA to use their data (no participants chose to do so).
Finally participants were asked to complete a further three blocks of math tasks each of a different
difficulty level and equivalent to those in the main experiment and rate each question via a nine
point Likert scale to ascertain subjective ratings of task difficulty (F. G. Paas & Van Merrienboer,
1994).
RESULTS
EEG Data was gathered for 11 participants but was only tentatively analysed owing to project time
constraints, as such it will not be discussed here.
GSR data was collected for 11 participants, however only data from the first eight were analysed
owing to project time constraints. The analysis described below is seen as preliminary in that it does
Stress and Cognitive Load Dan Conway Page: 9
not involve any machine learning or feature recognition which is suggested as being the most fruitful
method of analysis. The present analysis may, however, be taken as indicative.
In the Post‐Experiment questionnaire participants were asked (amongst other things) ‘Did you feel
‘stressed out’ during the experiment? If so – how much?’. All participants answered in the
affirmative with responses ranging from ‘A little bit’, through to ‘Yes, very’.
An ANOVA of pooled subjective ratings for each level showed differences (n = 8, df = 2, F = 82.32, p <
.0001). The means of each group were increasing by CL level (Level 1: M = 1.52, SD = .19. Level 2: M
= 4.17, SD = .27, Level 3: M = 7.46, SD = .47) and the difference between each group was significant
(1 to 2: p < .0001, 2 to 3: p <.0001).
Epoch generation
The ‘middle minute’ of each block was used as the basis for analysis with the aim of minimising
fatigue and practice effects. Once the epochs were extracted, a mean value of the two minute
baseline period was derived. This mean was subtracted from the GSR value at each time point
within the participant’s epochs. A mean was then calculated for the entire epoch, resulting in a
single value for each epoch (or block) for each participant.
Statistical Analysis
For GSR measurements within the ‘no‐stress’ condition, a repeated measures Anova showed
significant differences between CL level within subjects (F = 6.402, df = 2, p = .029). Group means
3 = .9299 µS). However for the ‘stress’ condition, a repeated measures Anova showed no significant
differences between CL level within subjects (F = 2.816, df = 2, p = .287). In this condition the means
of each group descended with increasing CL (CL level 1 = .3 µS, CL level 2 = .26 µS, CL level 3 = .24
µS). All measurements quoted are in Micro‐Siemens (µS), a measurement of conductance.
Figure 5. Mean of ‘Middle Minute Epochs’ of GSR for participants 2‐9 measured during the
'No‐stress' condition by Cognitive Load.
Stress and Cognitive Load Dan Conway Page: 10
Paired‐Samples T‐Tests of GSR measurements between the ‘no‐stress’ and ‘stress’ conditions were
then carried out for each CL level.
Significant differences were found between conditions for CL level 1 (t = ‐3.786, df = 7, p = .007)
where the ‘no‐stress’ condition exhibited a lower mean GSR (.24 µS) than ‘stress’ (2.99 µS).
Significant differences were found between conditions for CL level 2 (t = ‐5.051, df = 7, p = .001)
where the ‘no‐stress’ condition exhibited a lower mean GSR (.66 µS) than ‘stress’ (2.60 µS).
Significant differences were found between conditions for CL level 1 (t = ‐3.903, df = 7, p = .006)
where the ‘no‐stress’ condition exhibited a lower mean GSR (.93 µS) than ‘stress’ (2.42 µS).
Figure 6. Mean of ‘Middle Minute Epochs’ of GSR for participants 2‐9 measured during the
'Stress' condition by Cognitive Load.
Figure 7. Mean of ‘Middle Minute Epochs’ of GSR for participants 2‐9 by condition for Cognitive Load Levels 1, 2 and 3.
Stress and Cognitive Load Dan Conway Page: 11
DISCUSSION
The methods used here to induce stress seemed to deliver reliable increases in GSR as well as
subjective ratings of stress. Thus although we cannot categorically say that the ‘no‐stress’ condition
had no stress, and importantly, we cannot say that stress levels did not vary across the different
levels of CL tasks in the ‘no‐stress’ condition (more on this later), we CAN say that the two conditions
were quantitatively different in the levels of stress that they induced.
Although the analysis presented is somewhat rudimentary and does not utilise any feature
detection, the outcomes may be considered indicative of an underlying truth: In the ‘no‐stress’
condition GSR was seen to reliably increase with CL, whereas in the stress condition, this relationship
is not present.
This has important considerations for attempting to use physiological signals as means of assessing
CL. It is conceivable that similar patterns may exist for other physiological signals meaning that
stress needs to be controlled in order to accurately measure CL. Further research into other
physiological signals along similar lines as the experiment presented here seems validated.
A challenge for CL investigations is the nature of the task used to induce CL. A language based task
was not implemented in consideration of the variability of language backgrounds in the likely
participants, ie: NICTA staff, and this consideration is likely to be important in future experiments. A
math based task was chosen largely for its previous implementation in both the Trier Social stress
test (Kudielka, 2008) and Dedovic’s MIST protocol (2005). However task performance did
demonstrate variation owing to, it is assumed, individual differences in math ability. Although an
analysis of subjective ratings did show sufficient differences between the tasks, future investigations
may be better served by using a task that is less dependent on learnt abilities. An ‘n‐back’ task may
more specifically target Working Memory and thus be a more reliable method of inducing given
levels of CL across individuals of different abilities and backgrounds.
Figure8: A typical participant’s GSR response over the duration of the experiment.
Stress and Cognitive Load Dan Conway Page: 12
Any task that is assumed to induce different levels of CL must be assessed as to its criterion validity.
One cannot assume that a given manipulation of task parameters will monotonically increase task
difficulty. The human cognition system, with its capacity for parallel processing or even super‐
processing, variation in stopping rules for different tasks and variance in task expertise may in many
cases exhibit variation in performance for a given task that is not reflected in the assumed increasing
difficulty of a task. Altering a given parameter of a task (increasing the amounts of targets from 1 to
2 for example) may well not make a task twice as difficult. Indeed owing to the brains’ enormous
plasticity this is in fact quite unlikely. Thus assessment of task difficulty at the outset of the
experimental design process is critical.
Self‐reported, subjective ratings of task difficulty have been shown to be a reliable method of
quantification of task difficulty (Damos, 1991), and a number of different approaches to gathering
this data have been proposed. In the initial stages of this project the SWAT (Nygren, 1991) , NASA‐
TLX (Rubio, Díaz, Martín, & Puente, 2004) and Paas (F. G. Paas & Van Merrienboer, 1994) rating
systems were all trialed and assessed for suitability for implementation. The SWAT process was
quickly found to be both slow and laborious, and also has been reported as less sensitive to
differences in low CL than NASA‐TLX (Luximon, 2001). The NASA‐TLX paradigm is highly regarded
(Damos, 1991) and demonstrates good diagnosticity and as such was operationalised in a pilot
project (indeed is also an option within the SACL experiment application) to assess its suitability.
Two of the six factors (Physical Demand and Temporal Demand), were not relevant to our purposes
so a modified version was implemented with an appropriate mathematically modified version of the
weightings matrix. This was found to have extremely poor face validity with participants expressing
a large degree of frustration and dissatisfaction with the method. Finally a simple 9 point Likert
scale, emulating Paas was implemented and this was found to be highly appropriate to the situation
being both less intrusive and possessing higher face validity. Both Paas and the modified NASA‐TLX
ratings systems are available as options within the SACL application.
The Cognitive Load model, as proposed by Sweller (1988) and expanded upon by Paas (2003) and
others, was originally developed within the Pedagogical paradigm, and as such, is a sometimes
uneasy fit for the real‐time cognition focus that typifies NICTA’s approach. Specifically, the CL
model, does not incorporate the concept of the ‘central executive’ in Working Memory as proposed
by Baddeley (1974), but rather uses a conceptual model of ‘schema’ retrieval and application
(Schnotz & Kürschner, 2007). This may or may not have serious implications for our research
purposes, but a more concerning aspect of the CL model is the complete absence of any reference to
perceptual processing. Perceptual processing has been shown to require cognitive effort in that the
initial acquisition of information by the sensory system, when stimuli are indistinct or ambiguous,
induces load on the cognitive system (Eysenck, 2010). Signal Detection Theory (SDT) has been
grappling with these problems since the Second World War and an established body of knowledge
exists to allow the quantification and assessment of the nature of signal acquisition issues. The
question then becomes whether the load induced by perceptual processing falls within or outside of
the CL model. In one fMRI study, (Barch, et al., 1997) achieved a double‐dissociation between
activity associated with Working Memory in the dorsolateral prefrontal cortex and load induced by
Perceptual Coding which showed increased activity in the anterior cingulate as well as the frontal
cortex. This suggests that these are two separate sources of ‘load’ on the human cognition system,
and the perceptual demands of a task are likely to induce patterns of cortical activation that are
distinct from Working Memory tasks. This has important ramifications for EEG based approaches to
Stress and Cognitive Load Dan Conway Page: 13
detecting CL in that task difficulty imposed by perceptual coding (such as degraded stimuli,
ambiguous signals, etc…) may contribute to mental load but in patterns that are different to that of
Working Memory load. With the aim of developing real‐time detection of CL via EEG, this factor
then needs to be either excluded from the CL model, or if it is to be included, must be quantified.
Further research into the relationship between stress and Cognitive Load is suggested. Whether it
will be possible to ever create a motivated, no‐stress task remains to be seen, but of particular
interest would be to determine whether stress is automatically induced when the task exceeds
operator capacity. According to Lazarus and Folkman’s model this is likely, since the appraisal
processes would result in an ‘I cannot cope’ evaluation. An experiment where carefully calibrated
tasks induce load just below and just above participants’ cognitive capacity would be informative.
It also remains to be seen whether one can ever create tasks where increasing task difficulty does
not induce some corresponding increase in stress. This experiment was based on the assumption
that one can exert mental effort and not necessarily become stressed, but this assumption is open to
challenge and more direct proxies of stress such as blood‐cortisol levels as controlled by the
hypothalamus‐pituitary‐adrenal axis would need to be assessed in order to clarify this relationship.
An interesting study with many parallels to the work presented here was carried out by Setz (2010).
Unfortunately the title of the paper is somewhat misleading since they do not experimentally
manipulate cognitive load and therefore only demonstrate the ability to differentiate between
‘stress’ and ‘no‐stress’ conditions where CL is consistent between conditions (confusingly they label
the ‘no‐stress’ condition as the ‘Cognitive Load Condition’). Nonetheless, the feature detection
processes they outline appear promising, and suggest themselves as avenues with which to analyse
the body of data generated by the SACL experiment.
An additional intended outcome from this experiment was assessing the suitability of the Emotiv
Epoc headset for EEG recording. As mentioned earlier the P7 and P8 sensors were sometimes
unable to make contact with a participants scalp owing to head size/shape and a lack of flexibility by
the unit. Furthermore there were some problems with the build quality of the device (sensors
would sometimes drop out of their housing when being fitted, the plastic flanges holding sensors in
place would sometimes break, etc…). Having said this, the device seemed to perform remarkably
well given its price. Ideally a within‐subjects comparison of signals derived both by the Epoc and
another, higher quality headset on an identical series of tasks would now be carried out allowing
more exact comparison of the derived EEG signals.
One potential flaw in the design of the current experiment lies in the potential for the negative
relationship between CL and GSR levels in the stress condition to be the result of an initial spike of
GSR when the stress condition begins, followed by a gradual decline as the physiological signals
return to some other ‘baseline’ state. Future experiments should establish the recovery time of GSR
signals and ascertain that this effect is not present in the tasks within the stress condition.
The focus of this project (a Summer ‘Taste of Research’ scholarship) was on the experimental design
and data‐collection aspects. Thus, the body of data generated by this experiment begs for more
rigorous and deeper analysis. It is recommended that current Machine Learning approaches be
applied to both the GSR and EEG signals acquired to ascertain whether existing feature detection
methods can distinguish between CL levels in the stress condition.
Stress and Cognitive Load Dan Conway Page: 14
REFERENCES
Aasman, J., Mulder, G., & Mulder, L. J. (1987). Operator effort and the measurement of heart‐rate variability. Human Factors, 29(2), 161‐170.
Baddeley, A. D., . Hitch, G. (1974). Working Memory. Oxford: Oxford University Press.
Barch, D. M., Braver, T. S., Nystrom, L. E., Forman, S. D., Noll, D. C., & Cohen, J. D. (1997). Dissociating working memory from task difficulty in human prefrontal cortex. Neuropsychologia, 35(10), 1373‐1380.
Chen, F. (2006). Designing human interface in speech technology. NY: Springer Science+Business Media, Inc.
Damos, D., L. (Ed.). (1991). Multiple Task Performance. London: Taylor and Francis.
Dedovic, K., Renwick, R,. Mahani, N. K., Engert, C., Lupien, S. J., Pruessner, J. (2005). The Montreal Imaging Stress Task: using functional imaging to investigate the efects of percieving and processing psychosocial stress in the human brain. Journal of Psychiatry & Neuroscience, 30(5), 319‐325.
Dickerson, S. S., & Kemeny, M. E. (2004). Acute Stressors and Cortisol Responses: A Theoretical Integration and Synthesis of Laboratory Research. Psychological Bulletin, 130(3), 355‐391.
Duval, S., Wicklund, R.A. (1972). A Theory of objective Self Awareness. New York: Academic Press.
Eysenck, M. W., Keane, M.T. (2010). Cognitive Psychology. New York: Taylor and Francis.
Hagmueller, M., Rank, E., Kubin, G. (2006). Evaluation of the Human Voice for Indications of Workload‐induced Stress in the Aviation Environment EEC Technical/Scientific Report (EEC Note 2006/18 ed., Vol. 2006‐023). Bretigny‐sur‐Orge: Eurocontrol Experimental Centre.
Harmony, T. Ì. a., Fernandez, T. Ì. a., Silva, J., Bosch, J., Valdes, P., Fernandez‐Bouzas, A., et al. (1999). Do specific EEG frequencies indicate different processes during mental calculation? Neuroscience Letters, 266(1), 25‐28.
Kudielka, B. M. (2008). The Treir Social Stress Test. SciTopics. Retrieved from
Lazarus, R., S. Folkman, S. (1984). Stress, Appraisal and Coping. New York: Springer Pub. Co.
Longo, L., Barrett, S., Yao, Y., Sun, R., Poggio, T., Liu, J., et al. (2010). Cognitive Effort for Multi‐agent Systems
Brain Informatics (Vol. 6334, pp. 55‐66): Springer Berlin / Heidelberg.
Luximon, A., Goonetilleke, R, S. (2001). Simplified subjective workload assessment technique, from http://www‐ieem.ust.hk/dfaculty/ravi/papers/workload.pdf
Markus, H. (1978). The effect of mere presence on social facilitation: An unobtrusive test. Journal of Experimental Social Psychology, 14(4), 389‐397.
Martin, G. N., Carlson, N.R., Biskit, W. (2007). Psychology. Edibburgh Gate: Pearson.
Nygren, T. E. (1991). PSYCHOMETRIC PROPERTIES OF SUBJECTIVE WORKLOAD MEASUREMENT TECHNIQUES ‐ IMPLICATIONS FOR THEIR USE IN THE ASSESSMENT OF PERCEIVED MENTAL WORKLOAD. Human Factors, 33(1), 17‐33.
Stress and Cognitive Load Dan Conway Page: 15
Paas, F., Tuovinen, J. E., Tabbers, H., & Van Gerven, P. W. (2003). Cognitive load measurement as a means to advance cognitive load theory. Educational Psychologist, 38(1), 63‐71. doi: http://dx.doi.org/10.1207/S15326985EP3801_8
Paas, F. G., & Van Merrienboer, J. J. (1994). Variability of worked examples and transfer of geometrical problem‐solving skills: A cognitive‐load approach. Journal of Educational Psychology, 86(1), 122‐133. doi: http://dx.doi.org/10.1037/0022‐0663.86.1.122
Robert, G., & Hockey, J. (1997). Compensatory control in the regulation of human performance under stress and high workload: A cognitive‐energetical framework. Biological Psychology, 45(1‐3), 73‐93.
Rubio, S., Díaz, E., Martín, J., & Puente, J. M. (2004). Evaluation of Subjective Mental Workload: A Comparison of SWAT, NASA‐TLX, and Workload Profile Methods. Applied Psychology, 53(1), 61‐86. doi: 10.1111/j.1464‐0597.2004.00161.x
Schnotz, W., & Kürschner, C. (2007). A Reconsideration of Cognitive Load Theory. Educational Psychology Review, 19(4), 469‐508. doi: 10.1007/s10648‐007‐9053‐4
Setz, C., Arnrich, B., Schumm, J., La Marca, R., Tröster, G., & Ehlert, U. (2010). Discriminating stress from cognitive load using a wearable eda device. IEEE Transactions on Information Technology in Biomedicine, 14(2), 410‐417.
Shi, Y., Ruiz, N., Taib, R., Choi, E. H. C., Chen, F. (2007). Galvanic Skin Response (GSR) as an Index of Cognitive Load. Paper presented at the CHI, 2007, San Jose, California.
Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), 257‐285.
Tesser, A., Berkowitz, L. (1988). Toward a Self‐Evaluation Maintenance Model of Social Behavior Advances in Experimental Social Psychology (Vol. Volume 21, pp. 181‐227): Academic Press.
Veltman, J. A., & Gaillard, A. W. K. (1998). Physiological workload reactions to increasing levels of task difficulty. Ergonomics, 41(5), 656‐669.
Xu, J., Wang, Y., Chen, F., Choi, H., Li, G. Chen, S., Hussain, S. (2011). Pupillary Response Based Cognitive Workload Index under Luminance and Emotional Changes. Paper presented at the CHI 2011, Vancouver, BC, Canada.