Page 1
Comunicazione non verbale:
computazione affettiva
Corso di Interazione uomo-macchina II
Prof. Giuseppe Boccignone
Dipartimento di Scienze dell’Informazione
Università di Milano
[email protected] ://homes.dsi.unimi.it/~boccignone/l
Ipotesi di lavoro: interazione fra organismi
metabolismo
emozioni
motivazioni
azione
percezione
riflessione
metabolismo
emozioni
motivazioni
azione
percezione
riflessione
verbalenon verbale
ambiente
Page 2
Riconoscere emozioni
metabolismo
emozioni
motivazioni
azione
percezione
riflessione
ambiente
PROCESSI
COSTITUTIVI
PROCESSI
INTERATTIVI
AUTONOMIA /
LIVELLI DI CONTROLLO
CONTROLLO
VISCERALE
REGOLAZIONE
OMEOSTATICA
CONTROLLO
EMOTIVO
CONTROLLO
RIFLESSIVO /
COGNITIVO
Goals
interni
Goals
esterni
confine del corpo
situato nell’ambiente
Riconoscere emozioni
Page 3
A. Vinciarelli, M. Pantic, H. Bourlard, Social Signal Processing: Survey of an Emerging Domain,Image and Vision Computing (2008)
Emotional expression
A. Vinciarelli, M. Pantic, H. Bourlard, Social Signal Processing: Survey of an Emerging Domain,Image and Vision Computing (2008)
Emotional expression
Page 4
Emotional expression
• Approcci possibili:
• Discrete (Ekman) vs. Dimensional (Russell)
• AU-based (Ekman) vs. holistic
Riconoscere emozioni da espressioni facciali
Paradigma dominante
Page 5
• Approcci possibili:
• Discrete (Ekman) vs. Dimensional (Russell)
• AU-based (Ekman) vs. holistic
Riconoscere emozioni da espressioni facciali
Paradigma dominante Approcci robusti
• Approcci possibili:
• Discrete (Ekman) vs. Dimensional (Russell)
• AU-based (Ekman) vs. holistic
Riconoscere emozioni da espressioni facciali
Meno consueti
Usati soprattutto nella simulazione virtuali
Analisi di emozioni in musica o nel parlato
Page 6
Riconoscere emozioni da espressioni facciali
//http://www.visual-recognition.nl/index.html
Riconoscere emozioni da espressioni facciali
Page 7
Expression vs identity
//A.J. Calder et al. : Vision Research 41 (2001)
Expression vs identity
//A.J. Calder et al. : Vision Research 41 (2001)
Page 8
Expression vs identity
//A.J. Calder et al. : Vision Research 41 (2001)
Expression vs identity
//A.J. Calder et al. : Vision Research 41 (2001)
Page 9
Expression vs identity
//A.J. Calder et al. : Vision Research 41 (2001)
Schema generale di un sistema per AU detection
Page 10
Riconoscere emozioni da espressioni facciali
//Bartlett et al.
Riconoscere emozioni da espressioni facciali
//Bartlett et al.
Page 11
Riconoscere emozioni da espressioni facciali
//Bartlett et al.
Unified Probabilistic Framework
for Spontaneous Facial Action Modeling
• Tong et al (2010)
Page 12
Unified Probabilistic Framework
for Spontaneous Facial Action Modeling
• Tong et al (2010)
Mind-Reading Machines:
Automated Inference of Complex Mental States
by Rana Ayman el Kaliouby
Page 13
Mind-Reading Machines:
Automated Inference of Complex Mental States
Mind-Reading Machines:
Automated Inference of Complex Mental States
Page 14
Mind-Reading Machines:
Automated Inference of Complex Mental States
Mind-Reading Machines:
Automated Inference of Complex Mental States
Page 15
Mind-Reading Machines:
Automated Inference of Complex Mental States
Mind-Reading Machines:
Automated Inference of Complex Mental States
Page 16
Mind-Reading Machines:
Automated Inference of Complex Mental States
Mind-Reading Machines:
Automated Inference of Complex Mental States
Page 17
Mind-Reading Machines:
Automated Inference of Complex Mental States
Mind-Reading Machines:
Automated Inference of Complex Mental States
Page 18
Mind-Reading Machines:
Automated Inference of Complex Mental States
Mind-Reading Machines:
Automated Inference of Complex Mental States
Page 19
Mind-Reading Machines:
Automated Inference of Complex Mental States
Mind-Reading Machines:
Automated Inference of Complex Mental States
Page 20
Mind-Reading Machines:
Automated Inference of Complex Mental States
Active and Dynamic Information Fusion
for Facial Expression Understanding
• Zhang & Ji
Page 21
Active and Dynamic Information Fusion
for Facial Expression Understanding
• Zhang & Ji
Active and Dynamic Information Fusion
for Facial Expression Understanding
• Zhang & Ji
Page 22
Active and Dynamic Information Fusion
for Facial Expression Understanding
• Zhang & Ji
Active and Dynamic Information Fusion
for Facial Expression Understanding
• Zhang & Ji
Page 23
Active and Dynamic Information Fusion
for Facial Expression Understanding
• Zhang & Ji
Active and Dynamic Information Fusion
for Facial Expression Understanding
• Zhang & Ji
Page 24
Active and Dynamic Information Fusion
for Facial Expression Understanding
• Zhang & Ji
Active and Dynamic Information Fusion
for Facial Expression Understanding
• Zhang & Ji
Page 25
Active and Dynamic Information Fusion
for Facial Expression Understanding
• Zhang & Ji
Active and Dynamic Information Fusion
for Facial Expression Understanding
• Zhang & Ji
Page 26
Dimensional approach via Gabor wavelets
• Lyons et al.
Dimensional approach via Gabor wavelets
• Lyons et al.
• Gabor and human similarity data was
analyzed usingnon-metric
multidimensional scaling (nMDS) using
theALSCAL algorithm [13].
• nMDS embeds points in a euclidean
space in such a way that the distances
between points preserves the rank order
of the dissimilarity values betweenthose
points.
• it was found that two dimensions
provide an adequate embedding of the
similarity data
Page 27
Dimensional approach via Gabor wavelets
• Lyons et al.
• Gabor and human similarity data was
analyzed usingnon-metric
multidimensional scaling (nMDS) using
theALSCAL algorithm [13].
• nMDS embeds points in a euclidean
space in such a way that the distances
between points preserves the rank order
of the dissimilarity values betweenthose
points.
• it was found that two dimensions
provide an adequate embedding of the
similarity data
Dimensional approach via Gabor wavelets
Page 28
Emotion by dynamical simulation
• Essa & Pentland (1997): We describe a computer vision system for observing
facial motion by using an optimal estimation optical flow method coupled with
geometric, physical and motion-based dynamic models describing the facial
structure. Our method produces a reliable parametric representation of the
face’s independent muscle action groups, as well as an accurate estimate of
facial motion.
• Essa & Pentland (1997):
Emotion by dynamical simulation
Page 29
• Essa & Pentland (1997):
Emotion by dynamical simulation
• Essa & Pentland (1997):
Emotion by dynamical simulation
Page 30
• Essa & Pentland (1997):
Emotion by dynamical simulation
• Essa & Pentland (1997):
Emotion by dynamical simulation
Page 31
Reconstruction of Facial Expressions in Embodied
Systems
• Karl Grammer & Elisabeth Oberzaucher
Reconstruction of Facial Expressions in Embodied
Systems
• Karl Grammer & Elisabeth Oberzaucher
Page 32
Reconstruction of Facial Expressions in Embodied
Systems
• Karl Grammer & Elisabeth Oberzaucher
Reconstruction of Facial Expressions in Embodied
Systems
• Karl Grammer & Elisabeth Oberzaucher
• The activation of single AUs in a
pleasure and arousal space. Note that
these
• distributions are different for different
Action Units.
Page 33
Reconstruction of Facial Expressions in Embodied
Systems
• Karl Grammer & Elisabeth Oberzaucher
Reconstruction of Facial Expressions in Embodied
Systems
• Karl Grammer & Elisabeth Oberzaucher
Page 34
Sociable robots
• Kismet
Sociable robots
• Kismet
Page 35
Sociable robots
• Kismet
Sociable robots
• Kismet
Page 36
Sociable robots
• Kismet
Sociable robots
• Kismet
Page 37
Sociable robots
• Kismet