ACI/HFES, Baltimore, October 1- 3, 2007 Experim entalD esign and Testing ofa M ultim odalC ognitive O verload C lassifier L eonard J . Trejo Neil J . McDonald Robert Matthews Q uantum A pplied Science and Research Brendan Z. Allison The ScrippsResearch Institute Sponsored by US Army Research Office SBIR Phase II: “Wearable Physiological Sensor Suite For Early Detection Of Cognitive Overload” US Army Aberdeen Test Center: “Remote Neurological Monitoring Program”
20
Embed
ACI/HFES, Baltimore, October 1-3, 2007 Sponsored by US Army Research Office SBIR Phase II: Wearable Physiological Sensor Suite For Early Detection Of Cognitive.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
ACI/HFES, Baltimore, October 1-3, 2007
Experimental Design and Testing of a Multimodal Cognitive Overload Classifier
Leonard J. Trejo Neil J. McDonald Robert Matthews
Quantum Applied Science and Research
Brendan Z. Allison
The Scripps Research Institute
Sponsored by US Army Research Office SBIR Phase II: “Wearable Physiological Sensor Suite For Early Detection Of Cognitive Overload”US Army Aberdeen Test Center: “Remote Neurological Monitoring Program”
ACI/HFES, Baltimore, October 1-3, 2007
Stress, Overload, and Performance
ACI/HFES, Baltimore, October 1-3, 2007
Mental State Estimation
Work-load
MentalFatigue
Non-specificFactors
Engage-ment
GeneralCognitive
Status
Engage-ment
Non-specificFactors
MentalFatigue
Work-load
BiosignalSources
ACI/HFES, Baltimore, October 1-3, 2007
Definitions
• Engagement: selection of a task as the focus of attention and effort
• Workload: significant commitment of processing resources to an engaged task
• Overload: task demands outstrip available processing resources
• Mental Fatigue: desire to withdraw attention and effort from an engaged task associated with extended performance (~45 min)
Work-load
MentalFatigue
Non-specificFactors
Engage-ment
GeneralCognitive
Status
ACI/HFES, Baltimore, October 1-3, 2007
Overload Patterns in Multimodal Signals
0 50 100 150 200 250 300 350 400 450
50
100
RT - blue; HRstd - red
0 50 100 150 200 250 300 350 400 4500
5
10
15
Left temporalis EMG - blue; Right temporalis EMG - red
0 50 100 150 200 250 300 350 400 4500
5
10
15
Fz/theta - blue; Pz/alpha - red
0 50 100 150 200 250 300 350 400 4500
100
200
300
vEOG - blue; hEOG - red
ACI/HFES, Baltimore, October 1-3, 2007
Multimodal Classifier Testbed
ACI/HFES, Baltimore, October 1-3, 2007
Multimodal Classifier Testbed
ACI/HFES, Baltimore, October 1-3, 2007
Multimodal Classifier Testbed
ACI/HFES, Baltimore, October 1-3, 2007
Multimodal Classifier Testbed
ACI/HFES, Baltimore, October 1-3, 2007
Multimodal Classifier Testbed
C2
C1
ACI/HFES, Baltimore, October 1-3, 2007
Engagement/Workload Related EEG Sources
Passive viewing: theta alpha
Engaged 5: theta alpha
10.55 Hz
10.30 Hz5.79 Hz
5.79 Hz
Anterior Cingulate Inferior Parietal Precuneus
ACI/HFES, Baltimore, October 1-3, 2007
Fatigue-Related EEG Sources Black = Alert Red = Mentally Fatigued
Fz Pz
FrontalTheta
ParietalAlpha
ACI/HFES, Baltimore, October 1-3, 2007
Other Sources
Source Effect of Workload
Heart rate Increase
Heart rate variability (and HFQRS) Decrease
Vertical and horizontal EOG (eye movements) Increase
Blinks May decrease for intake
Pupil diameter Increase
Skin conductance, SCR, GSR Increase
EMG (frontalis, temporalis, trapezius) Increase
ACI/HFES, Baltimore, October 1-3, 2007
Experimental Controls
• Task learning• Time of day and time on task• Test day• Food consumption • Neurotoxic effects • Test environment • Inadequate measurement of physiological variance • Inadequate definition of ground truth workload levels:
− Expert analysis and scoring of replayed videos− Logging all user inputs− Measuring reaction times to probes
ACI/HFES, Baltimore, October 1-3, 2007
Validation of Workload Manipulation
NASA - TLX questionnaires
P300
ACI/HFES, Baltimore, October 1-3, 2007
Within-day Results
• Trials 1 and 5: disengaged.
• Trials 2-4, 6-8: engaged increasing numbers of enemies.
• Trials 5-8: engaged and reported status in response to command tones