What are we modeling when we model emotion

What Are We Modeling WhenWe Model Emotion?

Eva HudlickaPsychometrix Associates

Blacksburg, VAhudlicka@ieee.org

AAAI Spring Symposium on“Emotion, Personality and Social Behavior”

March 27, 2008

What Are We Modeling WhenWe Model Emotion?

akaEmotion Modeling 101

HudlickaHudlicka

3

“Emotion Modeling”

• Lack of clarity re:– What is modeled and how

• Lack of design guidelines re:– Which theories to use?– Which data?– Which computations must be performed?– Which representational & reasoning formalisms

are best?– What type of architecture is best?

HudlickaHudlicka

4

Emotion Modeling Is ______

A: Emotion expression by machinesB: Emotion recognition by machinesC: Emotion generation in agents / robotsD: Emotion effects on agent / robot behaviorE: Affective user modelsF: All of the above.. And probably some others too

HudlickaHudlicka

5

“Emotion Models” Model ________

A: FeelingsB: MoodsC: EmotionsD: Affective StatesE: Personality Traits

HudlickaHudlicka

6

A Taxonomy of Affective Factors

Affective States Emotions Moods

Basic ComplexNegative Positive

Affective Factors

Traits States

“Big 5” “Giant 3” …

AngerJoyFear … ShameGuiltPride …

Attitudes, Preferences…

HudlickaHudlicka

7

Emotion Models Also Vary In…

• Specific roles of emotions modeled

• Specific aspects of emotions modeled

HudlickaHudlicka

8

Roles of Emotions

IntrapsychicIntrapsychicInterpersonalInterpersonal

WHAT?* Social coordination

* Rapid communicationof behavioral intent;

HOW?Express emotions via:

-Facial expression-Speech (content & properties)-Gesture, Posture-Specific actions

WHAT?* Motivation

* Homeostasis* Adaptive behavior

HOW?- Emotion generation (appraisal)- Emotion effects (processing biases)- Global interrupt system- Goal management- Prepare for coordinated actions

HudlickaHudlicka

9

How Do We Recognize anEmotion if We See One?

• Manifested across multiple, interactingmodalities:– Somatic / Physiological (neuroendocrine - e.g., heart rate, GSR)– Cognitive / Interpretive (“Nothing is good or bad but thinking

makes it so…”)– Behavioral / Motivational (action oriented, expressive, ‘visible’)– Experiential / Subjective (“that special feeling…”, consciousness)

• Much terminological confusion can be attributed to a lack ofconsideration of these multiple modalities of emotions– e.g., Is emotion a feeling or a thought? - It’s both

HudlickaHudlicka

10

Simple Fear “Signature”:Large, Approaching Object

Cognitive Subjective

Increasedheart-rate;

Attacked?Crushed?

Flee?Freeze?

Feeling offear

HudlickaHudlicka

11

Problem• Lack of consistency & terminological clarity• … makes it difficult to:

• Develop systematic guidelines for modeldevelopment

• Communicate effectively about alternativeapproaches

• Evaluate & compare different models & theories

HudlickaHudlicka

12

Objective:Deconstruct ‘emotion modeling’• View emotion models in terms of two

fundamental processes:– Emotion generation– Emotion effects

• Identify fundamental computational tasksnecessary to implement these– The emotion modeling “building blocks”

• …as a step toward more systematic design,comparison & evaluation of models

Emotion GenerationEmotion Generation Emotion Effects Emotion Effects

on Cognition & Behavioron Cognition & Behavior

Emotion Roles

implement

Social - Communication- Coordination

-….

Intrapsychic: - Goal management - Behavior preparation-……

Computational Tasks Computational Tasks

Emotion Model Emotion Model ““Building BlocksBuilding Blocks””

HudlickaHudlicka

HudlickaHudlicka

14

Emotion Modeling “Building Blocks”

• Should provide a basis for identifying:

– Theory requirements– Data requirements– Representational & reasoning requirements

HudlickaHudlicka

15

The Cognitive Caveat

• Primary focus on cognitive modality ofemotion

HudlickaHudlicka

16

HudlickaHudlicka

17

Core Processes of EmotionsEffects of Emotions

(on cognition & behavior)

Cognitive-Affective Architecture

Situations

Expectations

Goals

Cognitive Appraisal

Emotions

Generation of Emotions(via cognitive appraisal)

Stimuli

HudlickaHudlicka

18

Emotion Generation via Appraisal

StimuliStimuli

Appraisal ProcessAppraisal Process

EmotionsEmotionsAppraisal

Dimensions

RecalledPerceived

Existing emotions, moods, traits

Imagined

Goals (desires, values, standards) Beliefs, Expectations

HudlickaHudlicka

19

Emotion Generation via Appraisal

StimuliStimuli

Appraisal ProcessAppraisal Process

EmotionsEmotionsAppraisal

Dimensions

RecalledPerceived

Imagined

HudlickaHudlicka

20

Emotion Generation via Appraisal

StimuliStimuli

Appraisal ProcessAppraisal Process

EmotionsEmotionsAppraisal

Dimensions

RecalledPerceived

Existing emotions, moods, traits

Imagined

Goals (desires, values, standards) Beliefs, Expectations

HudlickaHudlicka

21

Emotion Generation via Appraisal

StimuliStimuli

Appraisal ProcessAppraisal Process

EmotionsEmotionsAppraisal

Dimensions

RecalledPerceived

Imagined

Goals (desires, values, standards) Beliefs, Expectations

Domain-Independent Appraisal Dimensions

NoveltyValenceGoal / Need relevanceGoal congruenceAgencyCoping potentialSocial and self norms and values

HudlickaHudlicka

22

STIMULISTIMULI

Novelty

Valence

Goal relevance

Outcomeprobability

Urgency

Goal congruence

Agency

Coping potential

Norms

high

high

v. high

low

other

lowlow

low

high

FEARFEAR

HudlickaHudlicka

23

STIMULISTIMULI

Novelty

Valence

Goal relevance

Outcomeprobability

Urgency

Goal congruence

Agency

Coping potential

Norms

high

high

v. high

low

other

highhigh

low

high

ANGERANGER

HudlickaHudlicka

24

Most Influential Appraisal Theories inMost Influential Appraisal Theories inComputational ModelsComputational Models

•• OrtonyOrtony, , Clore Clore and Collins (OCC) (1988 - and Collins (OCC) (1988 - ……))– Detailed taxonomy of triggers and emotions– Well-suited for computational implementation

•• Leventhal Leventhal and Scherer (1984 - and Scherer (1984 - ……))– Domain-independent appraisal dimensions– Appraisal as a dynamic, evolving process– Multiple levels of resolution

•• Arnold Arnold Lazarus Lazarus Smith and Kirby (1960 - ..) Smith and Kirby (1960 - ..)– Emphasis on coping– Appraisal as a dynamic process– Increasing emphasis on mechanisms

HudlickaHudlicka

25

What Do We Need To KnowTo Build a Model

• Stimulus - to - emotion mapping…– … for person… situation… affective state (?)– Implemented directly or via appraisal dimensions?

• Integrating internal & external stimuli– … remembered & imagined, seen & told…

• Distinct stages in the appraisal process– Functions implemented in each– Dependencies & interactions among them

• Factors influencing emotion intensity & how?– Emotion dynamics (ramp-up & decay)– … variations by person… situation… emotion

• Multiple emotions & conflict resolution

HudlickaHudlicka

26

Computational Tasks forAppraisal Models

Stimuli Emotions

Emotion attributes:- Complexity of emotion construct * type * intensity * cause … * direction * …

Types of stimuli:- Internal / External- Real / Imagined- Past / Present / Future- Domain specific / Abstract appraisal dimensions- Complexity of stimulus structure- Mental constructs required (e.g., goals, expectations)

- Stimuli-to-emotion mappings- Intensity calculation- Nature of mapping process: * Stages & functions * Degree of variability-Integrating multiple emotions-Emotion dynamics over time

HudlickaHudlicka

27

Cognitive-Affective Architecture

Stimuli

Situations

Expectations

Goals

Affect Appraiser

Emotions

Emotion Effects on Cognition

HudlickaHudlicka

28

Emotion Effects on Cognition

• Emotion and cognition function as closely-coupledinformation processing systems

• Emotions influence fundamental processes mediatinghigh-level cognition:– Attention speed and capacity– Working memory speed and capacity– Long-term memory recall and encoding

• Influences on processing & contents and structure– Transient biases influence processing– Long-term biases result in differences in long-term memory

content & structure

HudlickaHudlicka

29

Examples of Affective Biases• Anxiety

– Narrows attentional focus– Bias toward detection of threatening stimuli– Bias toward interpretation of ambiguous stimuli as threats– Promotes self-focus

• Anger– Increases risk tolerance– Bias toward impulsive action– Bias toward attribution of hostile intent in others

• Positive emotions– Increase estimates of degree of control– Overestimate of likelihood of positive events– Focus on “big picture”

HudlickaHudlicka

30

Theories

• Fewer theories exist than for appraisal

• Specific mechanisms of emotion effects not as welldeveloped

• Some available theories:– Spreading activation & priming (Bower, 1984; Derryberry, 1988)

• “Network theory of Affect”

– Distinct modes of processing associated with differentemotions (Oatley & Johnson-Laird, 1987)

– Emotions as patterns of parameters modulating processing (Fellous, Matthews, Ortony et al., Hudlicka, Ritter…)

HudlickaHudlicka

31

Modeling Threat BiasTRAITS / STATES

COGNITIVE ARCHITECTUREPARAMETERS

COGNITIVE ARCHITECTURE

Attention

ActionSelection

Situation Assessment

GoalManager

ExpectationGenerator

AffectAppraiser

Emotions

HigherAnxiety / Fear

Predisposes towards

ProcessingParameters

Module &Construct parms. -Cue selection - Interpretive biases

...Preferential processing of Threatening stimuli

Threat constructsRated more highly

ProcessThreatcues

ProcessThreateninginterpretations

Traits

Low Stability

MAMIDMAMID

HudlickaHudlicka

32

What Do We Need To KnowTo Build a Model

• Emotion - to - effects mapping– Which cognitive processes & structures affected… & how– How are contents & organization of LTM affected– How are cognitive appraisal processes affected

• Relationship between emotion & mood intensity andtype & magnitude of influence– Can distinct intensities have qualitatively distinct influences?

• What are the mediating variables of the effects?– Emotions OR individual appraisal dimensions

(Lerner & Tiedens, 2006)?

• Multiple emotions & conflict resolution– How & where are multiple emotions integrated?

HudlickaHudlicka

33

Computational Tasks forComputational Tasks forModeling Emotion EffectsModeling Emotion Effects

Emotion(s)Emotion(s)

--CognitionCognition Attention, perception, memory, learning, problem-solving, decision-making…)

-Behavior Verbal, non-verbal, action selection

…… & other affective& other affective factors:factors:- Affective States- Moods- Traits

Effect(s)Effect(s)

- Emotion-to-behavior mappings- Emotion-to-processes & structures mappings- Variability in effects (by intensity, by individual…)- Integration of multiple emotions - Similar vs. opposing - In cognition.. in behavior ..where?

HudlickaHudlicka

34

Ideally, theories would exactly specifyIdeally, theories would exactly specifyall computational tasksall computational tasks

.. and provide.. and provide necessary details fornecessary details fortheir implementationtheir implementation

•• In practice, that is rarely the caseIn practice, that is rarely the case

•• The act of constructing a computational modelThe act of constructing a computational modeltypically helpstypically helps define & refine thesedefine & refine these detailsdetails

HudlickaHudlicka

35

Related Work

• Lot of people have been thinking aboutemotion model design– Sloman– Ortony– Canamero– Velasquez– Scheutz– Lisetti– Reilly– …

HudlickaHudlicka

36

Summary• Analyze emotion models in terms of two categories

of processes:– Emotion generation– Emotion effects

• Identify associated computational tasks

• See if these “building blocks” provide basis for:– More systematic design of emotion models– Comparison & evaluation of existing models & modeling

alternatives– Comparison & evaluation of existing theories

• (Focus on cognitive modality of emotions)

HudlickaHudlicka

37

Parting Thought“Anyone can model emotions. That is easy.

But to model emotionsin the right contextto the right degreeat the right timefor the right reason, andin the right way

this is not easy.”

Paraphrasing “On anger”, Aristotle, Nichomachean Ethics

What Are We Modeling WhenWe Model Emotion?

Eva HudlickaPsychometrix Associates

Blacksburg, VAhudlicka@ieee.org

AAAI Spring Symposium on“Emotion, Personality and Social Behavior”

March 27, 2008