Computational Physiology LabDepartment of Computer Science
University of HoustonHouston, TX 77004
Eustressed or Distressed?Combining Physiology withObservation in User Studies
Avinash WesleyDr. Peggy Lindner (Co-Advisor)Dr. Ioannis Pavlidis (Advisor)
2/13
Stress Signs
• Peripheral Physiological Measurement of Stress– Adrenergic response
• Elevates heart rate, respiration rate, and blood pressure – Cholinergic response
• Activates sweat glands on fingers and the perinasal area
• Introduction• Methods• Results and Discussion• Acknowledgements
- Stress Mechanism- Motivation- Background
3/13
Physiology and Observation
• Perspiratory response are – sympathetic in nature – Non-specific to positive or negative arousal
• Introduction• Methods• Results and Discussion• Acknowledgements
Distress Eustress
- Stress Mechanism- Motivation- Background
4/13
Emotions vs. Performance
• Introduction• Methods• Results and Discussion• Acknowledgements
- Stress Mechanism- Motivation- Background
Arousal
Performance
LOW MEDIUM HIGH
HIGH
Sleep Disorganization
AnxietyAlertness
Optimal
• An important goal in user studies: Study the role of emotions on human performance
• Emotions can be quantified via physiological response
• Physiological responses can be disambiguated via observation
5/13
Perspiration Signal and Observation
• Physiology– Perspiration extraction method in the Thermal Imagery [1]
• Observational Annotation– Traditional done in the Visual Imagery
• Manual
• Introduction• Methods• Results and Discussion• Acknowledgements
[1] D. Shastri, A. Merla, P. Tsiamyrtzis, and I. Pavlidis. Imaging facial signs of neurophysiological responses. IEEE Transactions on Biomedical Engineering, 56(2):477–484, 2009.
Courtesy of Science channel
- Stress Mechanism- Motivation- Background
6/13
Region Tracking
• Seven anatomical regions tracked over time by a dynamic template update tracker [2]
• Introduction• Methods• Results and Discussion• Acknowledgements
[2] Y. Zhou, P. Tsiamyrtzis, and I. Pavlidis. Tissue tracking in thermo-physiological imagery through spatio-temporal smoothing. Proc. of the 12th Int. Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2009),5762:1092–1099, 2009.
- Facial Expression Recognition- Field Study
7/13
Pattern Classification
• Feature Vector
• Classifier– Classify five action units (AU1+2, 4, 9, 10, and 12)– Multilayer Perceptron– 10-fold Cross Validation
• Introduction• Methods• Results and Discussion• Acknowledgements
AU 1+2 Inner + Outer Eyebrow Raise
d(x,5): Euclidean distance between ROI-x and 5, (x 5)
- Facial Expression Recognition- Field Study
8/13
Surgical Training
• Surgeon Pool (n=17)– Novices – Experienced
• Tasks– Running string (Task-1)– Pattern cut (Task-2)– Intracorporeal suture (Task-3)
• Dataset: 977 Thermal Clips
• Introduction• Methods• Results and Discussion• Acknowledgements
- Facial Expression Recognition- Field Study
9/13
Validation Results
• Using Thermal Imagery– 244 Facial Expressions – Ground Truth via Visual annotation– Method Accuracy 81.55%
• Introduction• Methods• Results and Discussion• Acknowledgements
* Use of visual images instead of thermal images for display purpose only
* Confusion matrix
- Quantitative Analysis- Qualitative Analysis- Conclusions
10/13
Results From The Field Study
• Distress is inversely related to experience
– EN (Perinasal perspiratory signal on portions of negative emotions)
• Introduction• Methods• Results and Discussion• Acknowledgements
- Quantitative Analysis- Qualitative Analysis- Conclusions
Novice Experienced
11/13
Example Visualizations
Eustress Distress
• Introduction• Methods• Results and Discussion• Acknowledgements
- Quantitative Analysis- Qualitative Analysis- Conclusions
12/13
Conclusions
• The proposed method is – Comprehensive (quantitative and qualitative) – Economical (single imaging modality with no labor)
• Conducted a study design that is applicable to a broad class of Human Machine Interaction
• Future Work– Expand the facial expression set– Apply the method to more field studies
• Detection of pain onset-offset
• Introduction• Methods• Results and Discussion• Acknowledgements
- Quantitative Analysis- Qualitative Analysis- Conclusions
13/13
• Support provided by NSF award # IIS-0812526
• Introduction• Methods• Results and Discussion• Acknowledgements