8/26/11 1 Affective Computing Intro Class Stacy Marsella & Jon Gratch Institute for Creative Technologies University of Southern California 2 Background Computational Models of Human Behavior Cognitive and Emotional processes Social Interaction & Theory of Mind Nonverbal behavior (Facial Expressions, gesture, posture, gaze…) Applications Virtual Humans, Social Simulation, Interactive Drama Architectures Tools: PsychSim/Thespian, SmartBody, NVBG
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
8/26/11
1
Affective Computing Intro Class"
!!
Stacy Marsella & Jon Gratch!!!Institute for Creative Technologies!University of Southern California!
2!
Background"
Computational Models of Human Behavior"!Cognitive and Emotional processes"
!Virtual Humans, Social Simulation, Interactive Drama"Architectures!
!Tools: PsychSim/Thespian, SmartBody, NVBG!
8/26/11
2
Outline"
! Emotion and it’s impact on behavior"
! Why model emotion computationally?"– Applications!
! How we model it"– EMA ! !
! Can the model predict human behavior in a game"
Why study emotion?"
�Reason is, and ought to be, only the slave of the passions�
David Hume, 1711-1776
8/26/11
3
""
Marsella & Gratch, Journal of Cognitive Systems Research 2009
What is emotion?"
! Functions associated with individual autonomy"– Rapid, continually adjusting assessment of significant events!– Interruption of behaviors and changing of goals!– Action preparation: energizes body, changes physical orientation!
! Functions associated with social interaction"– Signaling: broadcast information about mental state!– Coordination: orient and coordinate group response!
! Immersion and contagion"
8/26/11
4
7!
Why Study Emotion? Emotion influences cognitive processes"
! Emotions change perception & decision-making"– Angry people blame others/outgroups
(Keltner et al 93; Mackie et al 00) !– Quicker to perceive threats (DeSteno et
! How:"– Model Player, User, Student !– Recognize emotional
behavior!! Facial Expression Recognition
(Movellen, Cohn)!!
! )!
8/26/11
7
Applications: Enhance human computer interaction II"
!
! Elicit emotional responses to engage user"
! Design-A-Plant’s Herman the Bug"
14!
Applications: Virtual Humans"
Intelligent agent that supports social interactions with human users in virtual reality!
!
Characters with a brain !– Reason about environment!– Understand and express emotion!– Communicate through speech & gesture!– Play the role of teachers, peers, adversaries!
8/26/11
8
15!
Support range of applications""Social-skills training"
Interview skills (TacQ/IEWTPT)!Meeting and negotiation skills (ELECT BiLAT)!Command and personal issues ( FITE, VOLT)!
"Clinical training"Medical or psychiatric assessment (Virtual Patient; People factory)!Medical triage (Sick Call)!
"Persuasive interfaces"Health seeking / social support (Sim Coach, TOPPS-VW)!Medical adherence / outpatient procedures (Bickmore)!Tutoring (Lester; Graesser; !Advertising / Marketing / Customer support (CodeBaby)!Informal science education (Twins)!
"Assessment and Experimentation"Explore factors that promote rapport / self-disclosure (Rapport Agent)!Personality assessment (Lester)!
"Entertainment"Gunslinger!!!!
Health Interventions • Social Skills Training (Marsella et al) • Autism (Tartaro & Cassell)
! How do we computationally model emotion?!– Both the causes and cognitive/behavioral
consequences of emotion?!
! How do we evaluate/validate these models?!– Can it predict human behavior?!– And model users?!
! How do we use the model in virtual humans?!
8/26/11
11
Theoretical Perspective:�Appraisal Theory
Magda Arnold
• Emphasizes cognitive antecedents of emotion – Emotion arises from an evolving subjective
interpretation of person’s relation to their environment – Well-suited to computational realization
• Emotion arises from inference over representations
(Arnold, Lazarus, Frijda, Scherer, Ortony et al.)
22!
(act on world) (act on self)
Emotion
Coping Strategy
Action Tendencies “Affect” Physiological
Response
Problem-Focused Emotion-Focused
Environment Goals/Beliefs/ Intentions
Theoretical Framework: Appraisal Theory " " "(Arnold, Lazarus, Frijda, Scherer, Ortony et al.)"
Desirability
Expectedness
Controllability
Causal Attribution
Appraisal
8/26/11
12
23!
(act on world) (act on self)
Emotion
Coping Strategy
Action Tendencies “Affect” Physiological
Response
Problem-Focused Emotion-Focused
Environment Goals/Beliefs/ Intentions
Theoretical Framework: Appraisal Theory " " " " (Arnold, Lazarus, Frijda, Scherer, Ortony et al.)"
Desirability
Expectedness
Controllability
Causal Attribution
Resignation
Distancing
Wishful Thinking
Take action
Seek support
24!
Appraisal Theory as Architectural Specification"
To derive a computational model need to model:"• Representation of Person-Environment relation!• Derivation of appraisal values.!• Map from appraisals to emotions.!• Behavioral, cognitive consequents of emotion!
Goal Utility: 50 Probability: 100% Belief: False Past Act
Cause: Other Intend: yes Prob: 100%
Facilitates Inhibits
8/26/11
14
EMA: Appraisal Derivation Model"
• Appraisal as evaluation of the causal interpretation"• Define appraisal variables in terms of features!• Fast, parallel pattern matching!
• Appraisal Variables"• Desirability: Does proposition facilitate/inhibit agent’s utility!• Likelihood: Likelihood of state.!• Expectedness: Could truth value of a state be predicted.!• Controllability: Can the outcome be altered by agent!• Changeability: Can outcome be altered by other causal agent!• Causal Attribution: what agent is blameworthy/praiseworthy!
• Scope"• Blueprint for a social agent!
EMA: Coping Model"
• Sequential, deliberate, mediated by focus of attention"• Problem-focused � Take Action, Make Plans"• Emotion-focused � Alters Attention, Beliefs & Desires:"
• Attention!• Avoidance � Take action that alters attention!
! Appraisals of utility and probability predict emotion intensity
Hi win motivation
Lo win motivation Winning Losing Winning Losing
48!
Coping"
! Emotion distorts desires and intentions
! H1 and H2 supported: losers distance and resign (sour grapes)
winners
losers
�Rational� behavior
Self-reported desire to win Self-reported willingness to play
losers
! But also �coping� with winning observed ! EMA implementation makes no prediction
winners
! Rationality assumption: desire should be independent of probability
8/26/11
25
49!
Individual differences: Impact depends on initial preferences"
! H1a and H2a also supported: strength of an individual�s desire predicts strength of coping response ! In Lose condition: Subject�s with initial high desire to win distanced
and resigned more
! In Win condition: Subject�s with initial low initial desire to win became more engaged (EMA makes no prediction)
50!
Discussion"
! Computational appraisal models (e.g., EMA) can improve predictive power of models"
! Overall, results suggest that coping moves subjects toward more positive emotional states"– Distancing and resignation of losing, high motivated subjects reduces
negative emotionality of threat to high utility goal!– Increased engagement of winning, low motivated subjects enhances
positive emotions from previously unattainable goal!
! Suggests appraisal theory as a tool for understanding and influencing user’s desires and intentions"
8/26/11
26
Behavioral Consequences: social influence"
! If an agent expressed its emotions, would you care?"
! Fact: People influenced by partner�s expressions"– Guilt/Anger fosters cooperation in Prisoner�s Dilemma (Frank)!– Anger promotes concessions in Integrative Bargaining (van Kleef)!– Smiling yields cooperation in Trust Games (Krunhumber)!
! Question: is there value in communicating emotion"– Give agents different goals!
– Appraise events in the game (according to EMA)!! Cooperative agent �feels� guilty if it exploits human!! Self-interested agent �feels� satisfaction!
– Display resulting emotions!! Are people influenced by displays of agent emotion?"
!"
Effects of Agent Emotions on People in experimental games"
8/26/11
27
! Two studies""Can emotion promote cooperation?"
– Cooperative vs. unemotional agent!
"Does impact depend on agent goals?"– Cooperative vs. Individualistic emotions!!Create patterns of displays consistent with!!cooperative or individualistic goal orientation!!!
! Play iterated prisoner�s dilemma for chance to win $100"! Participants believe they are playing against computer"! Agent behavior identical (tit-for-tat) except for face expressions"! Predict greater cooperation with cooperative agent""
Studies"
8/26/11
28
! Greater cooperation with emotional agent!
! More cooperation with cooperative emotions!
! Same emotion (e.g., smile) has different impact depending on context in which it is produced!
! Mechanism still unclear!– Is emotion just information?!– Is it inducing emotional
Summary: should emotions influence machine behavior? "! Social science theory can inform social agent design"
– In this talk we focused on emotion!– Use appraisal theory as architectural specification for agent design!
! Value for modeling and predicting human behavior"– EMA enhanced predictive power of classical decision models!!Predicted changes in desires and intentions in competitive game!
! Value for shaping human computer interaction"– Virtual human emotion alters human behavior in experimental games!– Not to mention considerable research into recognizing and reacting to
human emotional cues (Picard; Graesser; Movellan & Bartlett; Pantic; Cohn;…)!
8/26/11
29
Broader picture"
! AI born from interest in human behavior"– The cognitive revolution in psychology, linguistics, !!neuroscience, anthropology and philosophy!
! Emotion important in early discourse "– Simon, H. A. (1967). "Motivational and emotional controls of cognition."
Psychological Review 74: 29-39."
"but fell from favor "– as AI turned to formal models of individual problem solving!
Simon & Newell
! AAMAS a crucible for a social cognitive revolution"– Social cognitive theory clearly can benefit man-machine interaction!– Can it inform general theories of artificial intelligence?!