The ULg Multimodality Drowsiness Database (called DROZY) and Examples of Use IEEE Winter Conference on Applications of Computer Vision (WACV 2016); Lake Placid, NY; 7-10 March 2016 DROZY is a database containing various types of drowsiness-related data (signals, face images, etc.) and intended to help researchers to carry out experiments, and to develop and evaluate systems (i.e. algorithms), in the area of drowsiness monitoring. Description Quentin MASSOZ, Thomas LANGOHR, Clémentine FRANÇOIS, Jacques G. VERLY INTELSIG Laboratory, Dept. of Electrical Engineering and Computer Science, University of Liège, Liège, Belgium Contact: [email protected] • 14 young, healthy participants (3 M, 11 F) • PVT = Psychomotor Vigilance Test [1] (duration of 10 minutes) • Protocol approved by the Ethics Committee of our university. Data acquisition No stimulant No sleep 7:00 10:00 11:00 12:00 3:30 4:00 12:00 12:30 DAY 1 DAY 2 PVT1 PVT2 PVT3 Using 24 hour times: We thank: • the participants for enduring the acute sleep deprivation of 28-30 hours, • David Grogna and Philippe Latour (ULg researchers) for their help in supervising the data collection, • the Belgian FRIA F.R.S-FNRS for supporting Quentin MASSOZ with a fellowship. Acknowledgments [1] M. Basner and D. F. Dinges. Maximizing sensitivity of the psychomotor vigilance test (PVT) to sleep loss. Sleep, 34(5):581-591, 2011. [2] T. Åkerstedt and M. Gillberg. Subjective and objective sleepiness in the active individual. International Journal of Neuroscience, 52(1-2):29-37, 1990. [3] M. Gillberg, G. Kecklund, and T. Åkerstedt. Relations between performance and subjective ratings of sleepiness during a night awake. Sleep: Journal of Sleep Research & Sleep Medicine, 1994. References Database content Range images Standard drowsiness measures Kinect v2 data Electroencephalogram (EEG) = brain waves Electrooculogram (EOG) = eye movements Electromyogram (EMG) = muscle tension Electrocardiogram (ECG) = heart activity Annotations www.drozy.ulg.ac.be Karolinska Sleepiness Scale [2] KSS 1 = Extremely alert 9 = Very sleepy 2 3 4 5 6 7 8 Reaction times [1] RT 68 face landmarks (2D & 3D) Polysomnography signals [2,3] Near-infrared intensity images Time [ms] ×10 5 1.62 1.63 1.64 1.65 1.66 1.67 1.68 1.69 1.7 Normalized eye opening 0 0.5 1 Closing Opening Blink True reaction time [s] 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 Predicted reaction time [s] 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 5a) Regression: we use epsilon-SVR models (with an RBF kernel) to predict the post-stimulus 1-min mean reaction time (RT) from pre- stimulus ocular parameters. We obtain an RMSE of 105.84 ms and a Pearson’s correlation of 0.67 using leave-one-subject-out cross- validation. Two examples of use 5b) Classification: we use SVM classifiers (with an RBF kernel) to predict the post-stimulus lapses (i.e. RT>500 ms) from pre-stimulus ocular parameters. We obtain a specificity of 86%, a sensitivity of 78%, and an accuracy of 85% using leave-one-subject-out cross-validation. Time [ms] ×10 5 1.45 1.5 1.55 1.6 1.65 Our algorithm [mm] 0 5 10 Baseline Eyelids distance 4) Compute ocular parameters: • values of a 10-bin histogram • mean duration of blinks • number of microsleeps • etc. 1) Get the (global) eyelids distance from the 8 eyelids landmarks. 2) Compute the baseline, i.e. the maximum opening of the eye, with an adaptive exponential smoothing. 3) Normalize and segment the blinks. 2256 360 100 348 ≤.5s ≤.5s >.5s >.5s Predicted RT True RT Confusion matrix: