Emotion based E-learning System using Physiological ...€¦ · Emotion based E-learning System using Physiological Signals Dr. Jerritta S, Dr. Arun S Introduction Emotions Mental
Post on 31-May-2020
14 Views
Preview:
Transcript
Emotion based E-learning System using
Physiological Signals
Dr. Jerritta S, Dr. Arun SSchool of Engineering, Vels University, Chennai
CHENNAI - INDIA
Emotion based E-learning System using Physiological Signals Dr. Jerritta S, Dr. Arun S
Outline
Introduction
Existing Research works on Emotion Recognition
Research Methodology
Results
Emotion based e-learning system
GUI
Future Work
MATLAB tools
Publications
Emotion based E-learning System using Physiological Signals Dr. Jerritta S, Dr. Arun S
Introduction
Emotions
Mental and physiological state associated with feelings,
thoughts and behavior
Learnt in diverse areas like Psychology, Cognitive Science,
Philosophy and Computer Science, However there hasn’t
been an universally accepted definition or categorization of
emotional states
Highly subjective and an integral part of any communication,
learning, perception and decision making
Emotion based E-learning System using Physiological Signals Dr. Jerritta S, Dr. Arun S
Introduction (Cont.)
Need of Emotion Recognition System
Intelligent machine interfaces for smooth interaction
between machines and human
Human Computer Intelligent Interaction (HCII) for natural
interaction between human and machines
Improves mutual empathy
Smart classrooms, Computer based Training, Medical
applications
Emotion based E-learning System using Physiological Signals Dr. Jerritta S, Dr. Arun S
Emotion Recognition Methods
Introduction (Cont.)
SELF REPORT methods
Questionnaires, Ratings and descriptions provided by the subject
Participant biased
Emotion based E-learning System using Physiological Signals Dr. Jerritta S, Dr. Arun S
Emotion Recognition Methods (Cont.)
Introduction (Cont.)
OBSERVOR (BEHAVIOUR) methods
Facial, Vocal and gesture cues
Dependant on external circumstances and prone to social
masking
Emotion based E-learning System using Physiological Signals Dr. Jerritta S, Dr. Arun S
Emotion Recognition Methods (Cont.)
Introduction (Cont.)
NEURO-PHYSIOLOGICAL signal processing methods
Physiological response of the Central Nervous System (CNS) and
Autonomous Nervous System (ANS)
Electroencephalogram (EEG), Electrocardiogram (ECG),
Electromyogram (EMG), Galvanic Skin Response (GSR), Skin
Temperature (ST), Skin Conductance (SC) etc.,
Complex, but provides the TRUE emotional state of the person.
Emotion based E-learning System using Physiological Signals Dr. Jerritta S, Dr. Arun S
Applications of Emotion recognition systems
Introduction (Cont.)
Robots
Dialogue systems
Computer based learning
Smart Classroom
Therapists for ASD
Medical Doctors
Emotion based E-learning System using Physiological Signals Dr. Jerritta S, Dr. Arun S
E-learning systems
Introduction (Cont.)
Feedback mechanisms in existing e-learning systems help to
study at the pace of the user.
They take into account only the ‘understanding on the
subject’ and not the state such as ‘fatigue’, ‘emotions’ etc.,
Emotion based E-learning System using Physiological Signals Dr. Jerritta S, Dr. Arun S
Emotion based E-learning systems
Introduction (Cont.)
Increases the receptiveness and productivity of the user
Suggests appropriate action to be taken depending on the
emotional state of the learner
Emotion based E-learning System using Physiological Signals Dr. Jerritta S, Dr. Arun S
Existing research works on EmotionRecognition
REFERENCES BIOSIGNALSNO OF
SUBJECTS
NO OF
EMOTIONS
EMOTIONAL
STIMULICLASSIFICATION RATE (%)
(Picard, Vyzas et al. 2001) EMG, BVP, SC, RR 1 8Personalized
Imagery 81(User Dependent)
(Lisetti and Nasoz 2004) GSR, HR, ST 29 6 Movies
72(User Dependent)
75(User Dependent)
84(User Dependent)
(Lan and Ji-hua 2006) ECG, ST, SC, RR 60 3 Movies 85.3 (User Dependent)
(Maaoui and Pruski 2008) BVP, EMG, SC,
ST, RR 25 6 Visual (IAPS) 88(User Dependent)
(Jonghwa and Ande 2008) EMG, ECG, SC,
RR
3 (22 trials)
MIT
database
4 Music 95(User Dependent)
70(User Independent)
(Kim and André 2009) EMG, ECG, SC,
RR
3 (22 trials)
MIT
database
4 Music 95% (user dependant)
70% (user independent)
(Kim and André 2009) EMG, ECG, SC,
RR
3 (22 trials)
MIT
database
4 Music 91% (user dependant)
Emotion based E-learning System using Physiological Signals Dr. Jerritta S, Dr. Arun S
Existing research works on EmotionRecognition (Cont.)
REFERENCES BIOSIGNALSNO OF
SUBJECTS
NO OF
EMOTIONS
EMOTIONAL
STIMULICLASSIFICATION RATE (%)
(Chuan-Yu, Jun-Ying et
al. 2010)
ECG, RR, GSR,
BVP 11 3 Movies 90.6% and 90.2% (user independent)
(Maaoui and Pruski 2010) BVP, EMG, ST,
SC, RR 10 6 Visual (IAPS)
90(User Dependent) and
45(User independent)
(Gouizi, Reguig et al.
2011)4 6 Visual (IAPS) 85% (user dependent)
(Valenza, Lanata et al.
2012) EDA, ECG, RR 35 2 Visual (IAPS) 90% (user independant)
(Vanny, Park et al. 2013) BVP, ST, SC 4 4 Visual (IAPS)100% for fear and joy, 60% for disgust
and neutral (User dependent)
(Chang, Chang et al.
2013) ECG, GSR, BVP 11 4 Movies 89.2% (user independant)
Emotion based E-learning System using Physiological Signals Dr. Jerritta S, Dr. Arun S
Existing research works on EmotionRecognition (Cont.)
Emotion classification rate varies from 45% to 100% for the different
research works.
They cannot be compared as they vary in,
Number of subjects
Type of elicitation
Type of physiological data
Number and placement of electrodes
Type of analysis (user dependency)
Though there is no standardization, some of the significant advances
are
IAPS, IADS database for emotion induction
Development of AuDB database (4 emotional states)
Statistical features for emotion recognition (Picard et al., 2001)
Emotion based E-learning System using Physiological Signals Dr. Jerritta S, Dr. Arun S
Existing research works on EmotionRecognition (Cont.)
Challenges
Emotion database of physiological signals
Emotional states must be elicited internally
The sensors should be less intrusive and at the same time
capture the emotional changes
Methodology
Complex, non-linear and non stationary nature of
physiological signals
Subject dependence of emotions
Reliability
Emotion based E-learning System using Physiological Signals Dr. Jerritta S, Dr. Arun S
Research Methodology
Step 1
•Development of Emotional data base (ECG, EMG)
Step 2
•Preprocessing
Step 3•Feature Extraction (Linear and non-linear methods)
Step 4
•Fusion of emotional features derived from ECG and EMG
Step5
•Emotion Classification
Step 6
•Development of GUI
Emotion based E-learning System using Physiological Signals Dr. Jerritta S, Dr. Arun S
Results
78.47
64.62
82.54
0
10
20
30
40
50
60
70
80
90
ECG EMG ECG and EMG
Max
imu
m C
lass
ific
atio
nA
ccu
racy
(%
)Performance of Emotion Recogntition System
Emotion based E-learning System using Physiological Signals Dr. Jerritta S, Dr. Arun S
Emotion based E-Learning system
User can recheck the emotion again before proceeding further
Suggests the activity based on the detected emotional state
Detects emotional state from ECG and EMG signals
EMOTIONAL STATE SUGGESTION
Neutral Start Lessons
Happiness Start Lessons
Sadness Listen to Music
Fear Listen to Music
Surprise Calm Down
Disgust Play a game
Emotion based E-learning System using Physiological Signals Dr. Jerritta S, Dr. Arun S
GUI
Emotion based E-learning System using Physiological Signals Dr. Jerritta S, Dr. Arun S
Future Work
On-line system
Develop efficient algorithms to capture the emotional states
More Automation
Integrate with mobile apps, robots and other personalized devices
Emotion based E-learning System using Physiological Signals Dr. Jerritta S, Dr. Arun S
MATLAB TOOLS
Signal processing
Filter Design for pre-processing
Feature Extraction algorithms – Wavelets, Fourier Transform, Hilbert Huang
Transform, Empirical Mode Decomposition, Hurst exponent
Machine Learning
Pattern Recognition – KNN, Regression Tree, Bayesian Classifier
Confusion Matrix
SIMULINK
Development of GUI
Emotion based E-learning System using Physiological Signals Dr. Jerritta S, Dr. Arun S
Publications
International Journals
– Jerritta S, M Murugappan, Sazali Yacob, Khairunizam Wan ‘ECG based Emotion Recognition System using Empirical Mode Decomposition and Discrete Fourier Transform,’ Journal: “Expert Systems’ by Wiley Publishers (IF : 0.733)
– Jerritta S, Murugappan M, Khairunizam Wan, Sazali Yaacob, "Emotion recognition from Facial EMG signals using Higher Order Statistics and Principal Component Analysis", Journal of Chinese Institute of Engineers (JCIE)(IF : 0.295) .
– Jerritta S, Murugappan M, Khairunizam Wan, Sazali Yaacob, "Classification of emotional states from electrocardiogram signals: a non-linear approach based on Hurst" Biomedical engineering online, May 2013. (IF:1.61).
– Jerritta S, Murugappan M, Khairunizam Wan, Sazali Yaacob, “Frequency study of Facial electromyography signals with respect to emotion recognition”, Biomedical engineering/ Biomedzinische technik, Degruter, (IF: 1.157) Dec 2013.
– Arun S, Sundaraj Kenneth, Murugappan M (2012) “Hypovigilance detection using energy of Electrocardiogram signals”, Journal of Scientific & Industrial Research, 71(12), 794-799. (ISI Impact Factor 0.505).
– Arun Sahayadhas, Kenneth Sundaraj and Murugappan Murugappan (2012) “Detecting Driver Drowsiness Based on Sensors: A Review”, Sensors, 12, 16937-16953. (ISI Impact Factor 1.953).
– Arun Sahayadhas, Kenneth Sundaraj and Murugappan Murugappan, (2013) "Drowsiness detection during different times of day using multiple features", Australasian Physical & Engineering Sciences in Medicine 36(2), 243-250 (ISI Impact Factor 0.885).
– Arun Sahayadhas, Kenneth Sundaraj M Murugappan, “Electromyogram signal based hypovigilancedetection”, Biomedical Research 2014; 25(3), ISI Impact Factor 0.177).
– Arun Sahayadhas, Kenneth Sundaraj M Murugappan, Rajkumar Palaniappan, “A Physiological Measures-Based Method for Detecting Inattention in Drivers Using Machine Learning Approach”, Biocybernetics and Biomedical Engineering, Accepted for publication, ISI Impact Factor 0.157
Emotion based E-learning System using Physiological Signals Dr. Jerritta S, Dr. Arun S
Publications
International Conferences
• Jerritta S, M Murugappan, R Nagarajan, Kahirunizam Wan, “Physiological Signals Based Human Emotion Recognition: A Review”, IEEE Colloquium on Signal Processing and Applications 2011, 3-5 March 2011, Penang, Malaysia.
• Jerritta S, M Murugappan, R Nagarajan, Khairunizam Wan, “Investigation on different Emotion Elicitation Methods for Human Computer Interactaion”, International Conference on Robotics Automation System 2011, 23-24 May 2011, Terungannu, Malaysia.
• Jerritta S, M Murugappan, R Nagarajan, Khairunizam Wan, “Digital Filtering based Pre-processing of Electrocardiogram (ECG) Signals for Human Emotion Recognition”, Malaysian Technical Universities International Conference in Engineering, 2011.
• M Murugappan, NQI Baharuddin, S Jerritta, “DWT and MFCC based human emotional speech classification using LDA”, International Conference on Biomedical Engineering (ICoBE), 2012.
• Jerritta S, M Murugappan, Khairunizam Wan, Sazali Bin Yaacob, “Emotion Recognition from ECG using Hilbert Huang Transform”, in IEEE STUDENT 2012, Oct 6-9, 2012.
• Jerritta S, M Murugappan, Khairunizam Wan, Sazali Bin Yaacob, “Emotion detection from QRS complex of ECG signals using Hurst Exponent for different age groups”, Submitted to IEEE 3rd Workshop on Affective Brain-Computer Interfaces, Geneva, Swizerland, Sept 2,2013.
• Arun S., Murugappan, M., & Sundaraj, K. (2011). “Hypovigilance warning system: A review on driver alerting techniques”. In Control and System Graduate Research Colloquium (ICSGRC), 2011 IEEE (pp. 65-69).
• Arun, S., Sundaraj, K., & Murugappan, M. (2012). “Driver inattention detection: A review” . In Sustainable Utilization and Development in Engineering and Technology (STUDENT), 2012 IEEE. (pp. 1-6)
Emotion based E-learning System using Physiological Signals Dr. Jerritta S, Dr. Arun S
top related