Emotion Classification using LSTM Based on Driving Behavior Hanyu Gao, Riya Sharma Technical University of Munich Department of Informatics 12.07.2019
Emotion Classification using LSTM Based on Driving Behavior
Hanyu Gao, Riya Sharma
Technical University of Munich Department of Informatics
12.07.2019
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Outline
!2
1. Introduction
2. Workflow and Data illustration
3. Long-Short-Term-Memory
4. Experiments and Result
5. Robustness and Proposal
6. Conclusion
7. Reference
1. Introduction
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Introduction Autonomous Driving & Deep Learning
!4
There are in general four questions a car needs to be able to answer to achieve
the final goal of autonomy.
1) Where am I? →Localisation and Mapping
2) Where is everybody else? →Scene Understanding
3) How do I get from A to B? →Movement Planning
4) What’s the driver up to? →Driver State
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Introduction Autonomous Driving & Deep Learning
!5
• via semantic abstraction -where each task is executed in a separate network and afterwards combined with classical control & decision-making algorithms.
• end-to-end approach -where a single DNN takes all the car’s inputs and computes a final output in a single step.
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Classification Standard
!6
• Emotion classification
• Driving behavior classification
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019 !7
• Emotion classification - Basic (primary) emotions: Ekman’s Big 6[1]
Classification Standard
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019 !8
• Emotion classification - Basic (primary) emotions: Ekman’s Big 6[1] - Plutchik's wheel of emotions
Classification Standard
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019 !9
• Emotion classification - Basic (primary) emotions: Ekman’s Big 6[1] - Plutchik's wheel of emotions - PAD emotion representation model
Classification Standard
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019 !10
• Emotion classification - Basic (primary) emotions: Ekman’s Big 6[1] - Plutchik's wheel of emotions - PAD emotion representation model
• Driving States Classification - normal driving, aggressive driving or drowsy driving
Classification Standard
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019 !11
• Emotion classification - Basic (primary) emotions: Ekman’s Big 6[1] - Plutchik's wheel of emotions - PAD emotion representation model
• Driving States Classification - normal driving, aggressive driving or drowsy driving - driving style: dissociative, anxious, risky, angry, high-velocity, distress reduction, patient, and careful…… [2]
Classification Standard
2. Workflow
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Workflow
!13
• Traditional Machine Learning Workflow
Data Model Application
Hanyu Gao (TUM), Riya Sharma |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Workflow —— Data
!14
• Vehicle dynamics-based
• Driver dynamics-based
Hanyu Gao (TUM), Riya Sharma |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Workflow —— Data
!15
• Vehicle dynamics-based technique - Internal data collectors
e.g:Controller Area Network bus (CAN Bus)
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Workflow —— Data
!16
• Vehicle dynamics-based technique - Internal data collectors
e.g:Controller Area Network bus (CAN Bus)
Hanyu Gao (TUM), Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Workflow —— Data
!17
• Vehicle dynamics-based technique - Internal data collectors, e.g:Controller Area Network bus (CAN Bus) - External data collectors, e.g: Accelerometer, Gyroscope, Smartphone
AccelerometerGyroscope Smartphone
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Workflow —— Data
!18
• Vehicle dynamics-based data - vehicle orientation - speed - acceleration - braking events - throttle - altitude - engine and fuel consumption - ……
Ocslab Driving dataset [4]
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Workflow —— Data
!19
• Driver dynamics-based technique - Video based
Car camera
Hanyu Gao, Riya Sharma(TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Workflow —— Data
!20
• Driver dynamics-based technique - Video based - Bio-signal based[3]
EMG: muscle activity ECG: manifestation of contractile activity of the heart Respiration: breathing depth EDA: skin conductance activity
Hanyu Gao, Riya Sharma(TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Workflow —— Data
!21
• Driver dynamics-based Data - electrocardiogram(EMG) - Electrocardiography(ECG) - Respiration - electrodermal activity(EDA) - eye gaze - EEG activities - head and body pose - ……
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Workflow —— Data
!22
• Problem - In-cab lighting changes - Camera position - Statistical data extraction(mean, median, mode ……) - Acquisition difficulty - ……
Hanyu Gao, Riya Sharma(TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Workflow —— Data
!23
• Problem - In-cab lighting changes - Camera position - Statistical data extraction(mean, median, mode ……) - Acquisition difficulty - ……
Feature Selection !
Hanyu Gao, Riya Sharma(TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Workflow —— Model
!24
• Machine Learning Model
Hanyu Gao, Riya Sharma(TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Workflow —— Model
!25
• Machine Learning Model - Support Vector Machine (SVM) - Bayesian Logistic Regression(BLR) - Hidden Markov Model (HMM)[5]
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Workflow —— Model
!26
• Machine Learning Model - Support Vector Machine (SVM) - Bayesian Logistic Regression(BLR) - Hidden Markov Model (HMM)[5]
Hanyu Gao, Riya Sharma(TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Workflow —— Model
!27
• Machine Learning Model - Support Vector Machine (SVM) - Bayesian Logistic Regression(BLR) - Hidden Markov Model (HMM)[5] - K-means - Symbolic Aggregate Approximation (SAX) - Gaussian Mixture Model (GMM)
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Workflow —— Model
!28
• Deep Learning Model VS Machine Learning Model
- Advantages : No more Feature selection, a holistic data-driven approach
- Disadvantages: More Computation, huge amount data
Hanyu Gao, Riya Sharma(TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Workflow —— Model
!29
• Deep Learning Model - Convolutional Neural Network (CNN)
Hanyu Gao, Riya Sharma(TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Workflow —— Model
!30
• Deep Learning Model - Convolutional Neural Network (CNN) - Recurrent Neural Network (RNN)
Hanyu Gao, Riya Sharma(TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Workflow —— Model
!31
• Deep Learning Model - Convolutional Neural Network (CNN) - Recurrent Neural Network (RNN) - Long Short-Term Memory (LSTM)
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Workflow —— Model
!32
• Deep Learning Model - Convolutional Neural Network (CNN) - Recurrent Neural Network (RNN) - Long Short-Term Memory (LSTM) - Gated Recurrent Unit (GRU)
3. Long Short-Term Memory
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Long Short-Term Memory
!34
• RNN
𝑠𝑡 = 𝑓(𝑈𝑥𝑡 + 𝑊𝑠𝑡−1)𝑦 = 𝑔(𝑉𝑠𝑡)
Hanyu Gao, Riya Sharma(TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Long Short-Term Memory
!35
• Vanishing Gradient Problem - Back-propagation through time - With multiple matrix multiplications, gradient values shrink exponentially - Gradient contributions from “far away” steps become zero
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Long Short-Term Memory
!36
• Long Short-Term Memory
memory block
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Long Short-Term Memory
!37
• Long Short-Term Memory - forget gate layer
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Long Short-Term Memory
!38
• Long Short-Term Memory - forget gate layer - input gate layer
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Long Short-Term Memory
!39
• Long Short-Term Memory - forget gate layer - input gate layer - current layer
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Long Short-Term Memory
!40
• Long Short-Term Memory - forget gate layer - input gate layer - current layer - output layer
Hanyu Gao, Riya Sharma(TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Long Short-Term Memory
!41
• Whole architecture of one LSTM cell
Hanyu Gao, Riya Sharma(TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Long Short-Term Memory
!42
• Limitations - Increase the number greatly of weights compared with RNN - Still unbalanced weight in time series (better than RNN)
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Long Short-Term Memory
!43
• Advantages[6] - LSTM can handle noise ,distributed representations and continuous values
Hanyu Gao, Riya Sharma(TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Long Short-Term Memory
!44
• Advantages[6] - LSTM can handle noise ,distributed representations and continuous values - Parameter fine tuning not really necessary, lstm works well over a broad
range of parameters
Hanyu Gao, Riya Sharma(TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Long Short-Term Memory
!45
• Advantages[6] - LSTM can handle noise ,distributed representations and continuous values - Parameter fine tuning not really necessary, lstm works well over a broad
range of parameters - Update complexity is O(1)
Hanyu Gao, Riya Sharma(TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Long Short-Term Memory
!46
• Advantages[6] - LSTM can handle noise ,distributed representations and continuous values - Parameter fine tuning not really necessary, lstm works well over a broad
range of parameters - Update complexity is O(1) - Able to deal with long time sequence
4. Experiment and Results
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Experiment and Results:Vehicle Based
!48
Classical Approaches
1) k-nearest neighbours classification algorithm:
- accelerometer sensor data
- classes(normal driving and aggressive driving)
- 177 features were extracted and fed to k-nearest classification model
Hanyu Gao, Riya Sharma(TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Experiment and Results:Vehicle Based
!49
2) k-mean clustering algorithm(unsupervised learning):
- accelerometer sensor data + vehicle dynamics data(braking and turning)
- classes(normal driving and aggressive driving)
- other statistical features were also
included(mean, max, variance) in feature vector
Hanyu Gao, Riya Sharma(TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Experiment and Results:Vehicle Based
!50
3) Support Vector Machine
- accelerometer sensor data + vehicle dynamics data(braking and turning)
- classes(normal driving and aggressive driving)
- other statistical features were
included(mean, max, variance) in feature vector
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Experiment and Results:Vehicle Based
!51
• Results
- For k-nearest neighbours , maximum(100%) classification precision can be reached by selecting certain features.
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Experiment and Results:Vehicle Based
!52
• Problems (Classical Approach)
- relies on hand crafting a set of features.
- requires a domain expert knowledge to determine feature selection.
- the separation happens between the feature extraction stage and the
learning algorithm stage which become a challenging task for deciding which
learning algorithm could be the best fit for the extracted features set.
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Experiment and Results:Vehicle Based
!53
• Problems (Classical Approach)
- relies on hand crafting a set of features.
- requires a domain expert knowledge to determine feature selection.
- the separation happens between the feature extraction stage and the
learning algorithm stage which become a challenging task for deciding which
learning algorithm could be the best fit for the extracted features set.
Solution:End to End Approach(RNN and LSTM)!
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Experiment and Results:Vehicle Based
!54
End to End Approaches
Dataset Used:UAH-DriveSet
- rich timestamped data with more than 500 minutes of driving sessions.
- two types of roads(motorway and secondary).
- 6 different drivers and vehicles.
- 3 types of driving behaviours (normal, aggressive and drowsy).
Hanyu Gao, Riya Sharma(TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Experiment and Results:Vehicle Based
!55
The 9 feature vectors of sensor data at each time step
Inertial measurement sensors: GPS sensors: Camera sensors:
1. Acceleration along x-axis & Vehicle Speed. Distance of vehicle
y-axis & z-axis -ahead.
2. Roll angle Number of-
3. Pitch angle detected vehicles
4. Yaw angle
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Experiment and Results:Vehicle Based
!56
1) RNNs
- time series classification (Many to one architecture)
- internal state h can capture the temporal dynamics
- input- a time-series window S of feature vectors
- outputs a classification scores vector Os .
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Experiment and Results:Vehicle Based
!57
• RNNs
Problems?
- memorising long sequence.
- also known as the “vanishing gradient” problem.
Hanyu Gao, Riya Sharma(TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Experiment and Results:Vehicle Based
!58
• RNNs
Problems?
- memorising long sequence.
- also known as the “vanishing gradient” problem.
Solution:LSTM!
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Experiment and Results:Vehicle Based
!59
2) Long Short Term Memory(2 layer)
- two LSTM memory cell layers.
- each layer have 100 hidden neurons.
- first layer input is a time-series window (64 feature vectors).
- second layer to output hidden feature vector.
- Finally, the last layer is a softmax layer.
Hanyu Gao, Riya Sharma(TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Experiment and Results:Vehicle Based
!60
• Results
• The LSTM clearly outperformed simple RNNs. • This may be because of the LSTM's greater ability to make use of long time
context.
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Experiment and Results: Driver based
!61
• Video Data Driven[10] • Dataset: EmotiW[11] —— AFEW 6.0 • 1.4K trimmed video clips from movies
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019 !62
• Model - LSTM - C3D – A Direct Spatio-Temporal Model - SVM
Experiment and Results: Driver based
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019 !63
• Result • Baseline —— 40.47%
Experiment and Results: Driver based
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019 !64
• Demo
Experiment and Results: Driver based
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019 !65
• Experiment II : Distraction Driving detection • 30 participants had driven at least 10.000 kilometres in 12 months • Data
Experiment and Results: Driver based
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019 !66
• Experiment II : Distraction Driving detection [8] • 30 participants had driven at least 10.000 kilometres in 12 months • Data(After correlation-based feature subset selection):
- speed (SP) - steering wheel angle (SA) - throttle position (TP) - heading angle (HA, angle between the longitudinal axis of the vehicle and
the tangent on the center line of the street) - lateral deviation (LD, deviation of the center of the car from the middle of
the traffic lane) - head rotation (HR, rotation around the vertical axis of the car)
Experiment and Results: Driver based
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019 !67
• Experiment II : Distraction Driving detection [8] • Model:
- Data collection - Statistical Processing - LSTM - Softmax
Experiment and Results: Driver based
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019 !68
• Experiment II : Distraction Driving detection [8] • Result
Experiment and Results: Driver based
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019 !69
• Experiment II : Distraction Driving detection [8] • Problems:
- Specific training condition - Bidirectional Long Short-Term Memory (BLSTM) - examine hybrid fusion of the low-level data streams
Experiment and Results: Driver based
5. Robustness and Proposal
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Robustness and Proposal
!71
• Fusion-RNN: Sensory Fusion RNN with LSTM units
- LSTM to solve vanishing gradient
- by concatenating the streams - Performs poorly as does not
capture the rich context for modelling
Solution ? Sensory Fusion Layer
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Robustness and Proposal
!72
- sensory fusion layer combines the high-level representations of sensor data. - passes two sensory streams {(x1,…..,xT),(z1,…..,zT)} through separate RNNs.
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Robustness and Proposal
!73
• Visual Feature Extraction • baseline video feature extractor
- Local Binary Patterns(LBP)
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Robustness and Proposal
!74
• Visual Feature Extraction • baseline video feature extractor
- Local Binary Patterns(LBP) • Proposed video feature extractor[12]
- Optical flow
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Robustness and Proposal
!75
• Visual Feature Extraction • baseline video feature extractor
- Local Binary Patterns(LBP) • Proposed video feature extractor[12]
- Optical flow
Baseline
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Robustness and Proposal
!76
• Multimodal • RECOLA dataset [13]:audio, video, electro-cardiogram (ECG) and electro-
dermal activity (EDA) modalities • Model
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Robustness and Proposal
!77
• Multimodal • RECOLA dataset [13]:audio, video, electro-cardiogram (ECG) and electro-
dermal activity (EDA) modalities • Model:
- Fusion layer
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Robustness and Proposal
!78
• Multimodal • RECOLA dataset [13]:audio, video, electro-cardiogram (ECG) and electro-
dermal activity (EDA) modalities • Model:
- Fusion layer
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Robustness and Proposal
!79
• Multimodal • RECOLA dataset [13]:audio, video, electro-cardiogram (ECG) and electro-
dermal activity (EDA) modalities • Result
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Robustness and Proposal
!80
• Attention-based Model • A neural attention mechanism equips a neural network with the ability to focus
on a subset of its inputs
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Robustness and Proposal
!81
• Attention-based Model • A neural attention mechanism equips a neural network with the ability to focus
on a subset of its inputs • Apply in NLP a lot
• Attention-based Model • A neural attention mechanism equips a neural network with the ability to focus
on a subset of its inputs • Apply in NLP a lot
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Robustness and Proposal
!82
• Attention-based Model • Example in Driving behavior and LSTM[14]
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Robustness and Proposal
!83
• Attention-based Model • Example in Driving behavior and LSTM
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Robustness and Proposal
!84
6.Conclusion
Hanyu Gao (TUM) , Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Conclusion
!86
• Conclusion: - LSTM outperforms in the Driving behavior temporal sequence analysis - Driving emotion recognition rely on a lot of factors and deep learning make
it possible to combine
• Future Work - Attention - Biosignal+video+vehicle data - Variants of LSTM, e.g Bilstm, conv-lstm……
Hanyu Gao, Riya Sharma (TUM) |Emotion Classification using LSTM Based on Driving Behavior , July 12, 2019
Reference
!87
[1] Ekman, Paul (1992). "An Argument for Basic Emotions". Cognition and Emotion. 6 (3/4): 169–200.
[2]Orit Taubman-Ben-Ari a,∗
, Mario Mikulincer b
, Omri Gillath “The multidimensional driving style inventory—scale construct and validation “ [3]R. Zheng, S. Yamabe, K. Nakano, and Y. Suda. 2015. Biosignal analysis to assess mental stress in automatic driving of trucks: Palmar perspiration and masseter electromyography. Sensors 15, 3 (2015), 5136–5150. [4] Kwak, B.I.; Woo, J.; Kim, H.K. Huy Kang. Know your master: Driver profiling-based anti-theft method. In Proceedings of the 14th Annual Conference on Privacy, Security and Trust, Auckland, New Zealand, 1214 December 2016. [5]Lee, C. C., Mower, E., Busso, C., et al. (2009). Emotion recognition using a hierarchical binary Decisi-on tree approach. INTERSPEECH, 53(9), 1162–1171. [6]Sepp Hochreiter and Ju ̈rgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735–1780. [7]C.Katsis,N.Katertsidis,G.Ganiatsas,andD.Fotiadis,“Towardemotion recognition in car-racing drivers: A biosignal processing approach,” IEEE Trans. Syst., Man, Cybern. A, Syst., Humans, vol. 38, no. 3, pp. 502–512, May 2008. [8]On-line Driver Distraction Detection using Long Short-Term Memory [9]Goldberger AL, Amaral LA, Glass L, Hausdorff JM, Ivanov PC, Mark RG, Mietus JE, Moody GB, Peng C-K, Stanley HE. Physiobank, physiotoolkit, and physionet components of a new research resource for complex physiologic signals. Circulation. 2000;101(23):215–20. [11]Dhall, A., Goecke, R., Joshi, J., Hoey, J. and Gedeon, T. 2016. EmotiW 2016: Video and Group -level Emotion Recognition Challenges, ACM ICMI 2016. [12]Martin W¨ollmer, Moritz Kaiser, Florian Eyben, Bj¨orn Schuller, Gerhard Rigoll,LSTM-Modeling of Continuous Emotions in an Audiovisual Affect Recognition Framework [13]Ringeval, F., Sonderegger, A., Sauer, J., & Lalanne, D.(2013, April). Introducing the RECOLA multimodal corpus of remote collaborative and affective interactions.In Automatic Face and Gesture Recognition (FG), 2013 10th IEEE International Conference and Workshops on (pp. 1-8). IEEE. [14]A Deep Learning Framework for Driving Behavior Identification on In-Vehicle CAN-BUS Sensor Data