Brain Computer Interfacing for Firefighter Fatigue Assessment

Brain Computer Interfacing for Firefighter Fatigue Assessment

Fred Beyette Jr. (PhD) Pooja Kadambi (MS)

Joseph Lovelace (MS)

College of Engineering and Applied Science School of Electronics and Computing Systems

Acknowledgements

•  Research Support –  National Institute of Biomedical Imaging and Bioengineering

(5U54EB007954) –  National Institute for Occupational Safety and Health Pilot Research

Project Training Program of the University of Cincinnati Education and Research Center Grant #T42/OH008432-07.

•  Dr. Amit Bhattacharya •  Capt. Kammer and the Fairfield Firefighter Department

Problem Statement

•  Firefighters are subjected to intense physical and/or mental pressure in harsh work environments.

•  Mental and/or physical fatigue can affect task performance and in some cases jeopardize the health of the worker

•  Real time monitoring of neurological status is challenging

Causes of Firefighter Fatalities [1]

NIOSH report 2010

GE CT Scanner

Background •  Psychological, imaging and kinesthetic studies

demonstrated measurable physical and mental changes associated with fatigue [2] [3]

•  fMRI studies and cognitive assessment tests evaluated attention, decision making with fatigue

•  EEG has been a medical tool for nearly a half a century. It is now possible to design a mobile system that can potentially be integrated into existing equipment [4]

•  Sensory evoked potentials are generated in response to certain stimuli. Variations in the response characteristics aid assessments of mental status [5] [6]

http://www.youcanstaysharp.com/index.php?id=469

http://www.natus.com/index.cfm?page=products_1&crid=480&contentid=397

Event Related Potentials

•  Sound, Visual, Tactile Stimuli

•  Seen 100-600ms after an “event”

•  N100, P200, P300

•  Voluntary and Involuntary

Animation created by Aaron Kurosu

[7] [8]

Methods: Application Testing •  Software BCI200TM [7]

•  5,12,19 channels used

•  Wet Silver/Silver Chloride and Tin

electrodes

•  Visual and Audio Testing

•  Different Experimental States –  Sleep Deprivation –  Caffeine –  Physical Exercise –  Distraction Stimuli –  Motion

Methods: EEG Platform Design Signal Flow •  Analog Front End •  Digital Signal Processor •  Bluetooth Transmission •  Target Device

Features •  Portable •  1-8 Channels •  Minimal wiring •  Efficient Power management •  Custom Electrodes

[9]

[10]

Methods: Data Analysis •  Offline Analysis [7] [8]

–  Peak Characteristics –  Noise –  BCI2000TM –  MatlabTM

•  Hardware Comparison [9] –  Timing –  Location –  Signal to Noise Ratio –  Power Consumption

Results: Modular EEG Testing

•  Bluetooth transmission possible

•  Fast processing

•  Portable size

•  Up to 8 channels

•  Other sensor integration

•  Power consumption

[10]

Results: Visual Speller results

•  Delay with sleep deprivation

•  Delay with limited exercise

•  Speed up with caffeine

•  Control between 170ms and

450ms

•  Consistent with previous studies

[11] Response Profile Heat map

Response Time (ms)

Ele

ctro

de C

hann

el N

umbe

r

Control Response

Post Exertion Response

Control  State  

Average  Peak  Latency  Across  Experimental  States  

Results: Audio Testing

•  Response comparable to visual testing

•  Music video distraction

•  Delay seen with distraction

•  Delay seen with sleep deprivation

•  Reported “easier” test

•  All subjects had a response at Cz central electrode

[12]

Results: Motion Testing

•  Response present and extractable

•  Lower response peaks

•  More noise in signal

•  Planned motion impact

•  Speller not possible

Results: Audio vs. Visual

Audio Application Visual Application

Runs take 30-35 seconds Runs take 2-5 minutes

Motion has minimal impact Motion severely affects response

Competing stimuli problems with high volume or similar frequency

Competing stimuli cause problems especially ANY visual stimulus

Subjects reported test to be easy Subjects reported test to be stressful

Hearing impaired or tone deafness would cause functional problems

Visual impairment and seizure medical history would be problematic

Real world implementation simple Real world implementation complex

Results: Firefighter Cognitive Testing

•  Stroop Effect test

•  Evaluation –  Pre Simulation –  Post 3 simulations –  Post work shift

•  Key measures

–  Reaction time –  Trial Difference

Prototyping

•  Customized printed circuit boards

•  Dual ring “prong” electrode design

•  Firefighter helmet integration

10

Future Directions

Promising results warrant further testing

Pre and Post activity firefighter testing

Customized application

Continuous wireless testing and monitoring

Image created by Aaron Kurosu

Key References [1] University of Illinois, " Firefighters Fatalities and Injuries The Role of Heat Stress and PPE," Firefighter Life Safety, Research Illinois Fire Service Institute, Vol. N/A, no. N/a, July 2008.[Online], Bluetooth Enabled, Wireless Electroencephalograph (Eat:http://www.fsi.illinois.edu/documents/research/FFLSRC_FinalReport.pdf. [Accessed 3/5/2012]

[2] K.Saroj, L.Lal, A.Craig, " A critical review of the psychophysiology of driver fatigue," Biological Psychology, vol. 55, no. 3, pp. 173-194, February 2001.

[3] T.Sobeih, K.Davis, et.al, " Postural balance changes in on-duty firefighters: effect of gear and long work shifts.," Journal of Occupational and Environment Medicine, vol. 48, no. 1, pp. 68-75, January 2006.

[4] T. Morris, J. Miller, " Electrooculographic and performance indices of fatigue during simulated flight," Clinical Neurophysiology, vol. 42, no. 3, pp. 343-360, February 1996.

[5] MA Morris, Y So, et.al, " The P300 event-related potential. The effects of sleep deprivation.," Journal of Occupational Medicine, vol. 34, no. 12, pp. 1143-1152, 1992.

[6] N.Kawamura, et al. "Effects of caffeine on event‐related potentials: Comparison of oddball with single‐tone paradigms." Psychiatry and clinical neurosciences vol.50 no.4,pp 217-221, August 1996.

[7] Gerwin.Schalk, Jurgen.Mellinger, A Practical Guide to Brain-Computer Interfacing with BCI2000, 2010 ed. , Springer, April 2010.

[8]A.Furdea, S.Halder, et al., " An auditory oddball (P300) spelling system for brain-computer interfaces," Psychophysiological Research, vol. 46, no. 3, pp. 617-625, May 2009.

[9] Lovelace, et.al, “Modular, Bluetooth Enabled, Wireless Electroencephalograph (EEG) Platform” Presented at EMBS 2013, Osaka Japan. [10]Kadambi et al, “Changes in Behavior of Evoked Potentials in the Brain as a Possible Indicator of Fatigue in People.” 35th Annual International Conference of the IEEE Engineering in Medicine, July 2013

[11] Kadambi et al. “Audio based Brain Computer Interfacing for Neurological Assessment of Fatigue” Accepted for presentation NES 2013, San Diego USA

[12] ]Lovelace et al, “Bluetooth Enabled Electroencephalograph (EEG) Platform.” 56th IEEE Midwest Symposium on Circuits and Systems, August 2013

Thank You

? Questions Comments

Clarifications ?