Emotions Many thinkers disagree on answers to these questions: Is surprise an emotion? (Some people say “always”, others say “only in certain cases”.)

EmotionsMany thinkers disagree on answers to these questions:

Is surprise an emotion?(Some people say “always”, others say “only in certain cases”.)

If you love your country, is that an emotion, or an attitude?What if your love for your country is far from your present thoughts?

Can you have an emotion without being aware of it?E.g. jealousy, infatuation

Does an emotion have to have some externally observable/measurable physiological manifestation?Can a fly feel pain, or have emotions?Is there a stage at which a human fetus becomes able to have emotions?

E.g. able to worry about how the birth will go?Could a disembodied mathematician have emotions?

E.g. feel disappointment at finding a flaw in a proof?

There is no consensus about what emotions are

Prejudices That Confuse the Investigation

Some people assume:

That minds must be “embodied” and emotions must involve bodiesIgnoring the possibility of passionate disembodied mathematicians.

That emotions are required for intelligence.That emotions evolved because they are useful

Maybe some did. It does not follow that all are useful. Many may be side-effects of other useful mechanisms.

That you can’t have an emotion without being conscious of it.What about the person who is obviously infatuated or jealous but unaware of the fact?

That it would be a “bad thing” if robots could have emotions.Could they possibly do worse things than humans do to other humans?

A Tip for Doing Conceptual Analysis

If someone puts forward a definition of mental state or process X, the question:

“Does this also apply to X in flies, rats, chimpanzees, newborn infants?”

often reveals that the definition was not based on sufficient thought.

Affect – Webster’s Collegiate Dictionary

Noun: The conscious subjective aspect of an emotion considered apart from bodily changes.

Verb: Applies to the acting of a stimulus strong enough to produce a noticeable response or reaction or modification

Synonyms: INFLUENCE, TOUCH, IMPRESS, SWAY

Emotions are a Subclass of “Affective” States

Affective states are of many kinds. They include not only what we ordinarily call emotions but also states involving desires, pleasures, pains, goals, values, ideals, attitudes, preferences, and moods.

The general notion of “affective state” is very hard to define but very roughly it involves using some kind of information that is compared (explicitly or implicitly) against what is happening, sensed either internally or externally.

–When there’s a discrepancy some action is taken, or tends to be taken to remove the discrepancy by acting on the sensed thing: affective states involve a disposition to change reality in some way to reduce a mismatch.– If the information is part of a percept or a belief, then detecting a discrepancy tends to produce a change in the stored “reference” information.

Emotions are a Subclass of “Affective” States

There is a more primitive type of control state which does not use any sort of description or representation that can be compared with reality, but merely generates action, or has a disposition to produce action (including resisting change). Many innate behaviors are like that. Are these “affective” states?

There’s no right definition of such a vague notion.

Emotions (from Latin emovere meaning moving, displacing) are, as its etymology suggests, manifest reactions to those affective conditions that, due to their intensity, move us to some kind of action.

Affective Computing Group – MIT

http://affect.media.mit.edu/“Affective Computing is computing that relates to, arises from, or deliberately influences emotions.

Emotion is fundamental to human experience, influencing cognition, perception, and everyday tasks such as learning, communication, and even rational decision-making. However, technologists have largely ignored emotion and created an often frustrating experience for people, in part because affect has been misunderstood and hard to measure. Our research develops new technologies and theories that advance basic understanding of affect and its role in human experience. We aim to restore a proper balance between emotion and cognition in the design of technologies for addressing human needs.”

Definition – Whatis?comhttp://whatis.techtarget.com/

Affective computing is human-computer interaction in which a device has the ability to detect and appropriately respond to its user's emotions and other stimuli. A computing device with this capacity could gather cues to user emotion from a variety of sources. Facial expressions, posture, gestures, speech, the force or rhythm of key strokes and the temperature changes of the hand on a mouse can all signify changes in the user's emotional state, and these can all be detected and interpreted by a computer. A built-in camera captures images of the user and algorithms are used to process the data to yield meaningful information. speech recognition and gesture recognition are among the other technologies being explored for affective computing applications.

A Control-basedConception of Emotion

What is there in common between

– a crawling woodlouse that rapidly curls up if suddenly tapped with a pencil,– a fly on the table that rapidly flies off when a swatter approaches,– a fox squealing and struggling to escape from the trap that has clamped its leg,– a child suddenly terrified by a large object rushing towards it,– a person who is startled by a moving shadow when walking in a dark

passageway,– a rejected lover unable to put the humiliation out of mind– a mathematician upset on realizing that a proof of a hard theorem is fallacious,– a grieving parent, suddenly remembering the lost child while in the middle of

some important task?

A Control-basedConception of Emotion

Proposed Answer:

In all cases there are at least two sub-systems at work in theorganism, and one of them, a specialized sub-system, somehow interrupts or suppresses or changes the behavior of others, producing some alteration in (relatively) global (internal or external) behavior of the system.

A Control-basedConception of Emotion

WE CAN MAKE IT MORE PRECISE BY:

– spelling out different kinds of information-processing control architectures inwhich such things (e.g. global interrupts or modulations of processing) canoccur– showing how different varieties of states with these general features can arise indifferent architectures.

Different sorts of emotions (and other affective states) arise out of different sorts of:– Interacting sub-systems–Ways one can interrupt or modulate another– Functional roles and side-effects

Affect and Architecture

As with emotions, which affective states are possible in an organism or machine will depend on the information-processing architecture of the whole system.

We’ll consider three architectural layers:reactivedeliberativemeta-management

and the different sorts of emotions that can be associated with them.

States and ProcessesThere are at least three different classes of mental phenomenacommonly referred to as “emotions”.

Primary emotionsEvolutionarily oldest – depend only on reactive mechanisms

Secondary emotionsDeliberative mechanisms generating these evolved later

Tertiary emotionsNewest and rarest: involve disruption of meta-management, e.g. loss of control of attention. These are usually not distinguished from secondary emotions

These rather vaguely defined categories must be re-defined in terms of the information-processing architectures (virtual machine architectures) that make them possible.

E.g., an animal without deliberative mechanisms cannot have secondary emotions.

Examples of primary emotions familiar in humansBeing startled by a loud noiseBeing frozen in terror as boulder crashes towards you,Being nauseated by a horrible smell

In primary emotions, sensor states and/or internal reactive states trigger a fastbut stupid reactive “alarm” mechanism that produces global changes in motorsand internal reactive states.

– Simple versions occur even in insects: when flee, fight, feed, freeze, or mate responses override other processes. (The five Fs!)– In humans these primary emotions often have sophisticated accompaniments that cannot occur in most other animals capable of having primary emotions.

E.g. when we are aware of having them we are using meta-management mechanisms that are not needed for primary emotions.

Often the primary emotion will immediately trigger some other kind, e.g. apprehension, a secondary or tertiary emotion.

Primary Emotions

Secondary Emotions

Examples — You are: Afraid the bridge you are crossing may give way Relieved that you got to the far side safely Afraid the bridge your child is crossing may give way Worried about what to say during your interview Undecided whether to cancel your holiday in ... Enjoying the prospect of success in your endeavor

Secondary emotions are triggered by events in a deliberative sub-system. Some ofthese are triggered by thinking about what might happen, what might havehappened, what did not happen, etc., unlike primary emotions which are triggered only by actual occurrences.

So secondary emotions require deliberative capabilities with ‘what if’, i.e.counterfactual, representational and reasoning capabilities. These are very subtleand complex requirements.

Counterfactual ReasoningCounterfactual: statement about the consequences of things which happen to be false.

In a case where proposition P is contrary to fact, counterfactual reasoning involves drawing inferences about what P would imply if it were true.

Because, by definition, a counterfactual is a conditional in which the antecedent is false, logically speaking all counterfactuals are true statements.

This has led some philosophers to argue that counterfactuals are not in fact cases of material implication.

This, in turn, has prompted the construction of formalisms with which to represent differences between this type of hypothetical reasoning and material implication.

Tertiary EmotionsExamples — You are:

Infatuated with someone you met recently Overwhelmed with grief Riddled with guilt about betraying a friend Full of excited anticipation of a loved one’s return Full of longing for your mother Basking in a warm glow of pride after winning an election Obsessed with jealousy about a colleague’s success

These involve disruption of high level self monitoring and control mechanisms. I.e., there is loss of control of thought processes. Thus they cannot occur in animals and machines that are incapable of having such control.

An architecture including meta-management capabilities is required for tertiary emotions.

Emotions and Architectures

In all of the categories (primary, secondary, tertiary emotions) thereis a subsystem that produces some relatively global changes in the rest of the system, or in much of it.

They differ in– What kind of subsystem does the disrupting– Where the information comes from that triggers the disrupting (e.g. does it come from a deliberative layer, or only sensors and internal states of a reactive layer?)– Which parts of the system are disrupted, e.g. is there externally visible behavior or only internal disruption? Which internal parts?– Also there are differences in kind of semantic content, time scale, what can and cannot suppress the disruption, whether learning is involved, etc.

Classes Not Mutually ExclusiveAll three kinds of emotional processes can coexist in complex situations.

As a result of this, the emotions labeled in ordinary language, e.g. “fear”, “anger”, “relief”, “distress”, cannot simply be classified as primary, secondary or tertiary. Often they are a mixture.

People involved in long and tiring adventure trips often describe multiple emotions at the end. E.g. they may be simultaneously:

Glad to have succeeded in their aims Regretful at not having done better Sad that the trip is over Relieved that some threat did not materialize (e.g. running out of fuel) Glad to see their families again Hoping to be selected for their national team Desperately longing for a good meal Worried about an injury incurred on the trip

Architectural Underpinnings

Different architectural underpinnings are required for different categories of emotions.

Primary emotions:Require sensors linked to fast reactive mechanisms that can sometimes trigger rapid global signal patterns sent to motors and other sub-systems.

Secondary emotions (central and peripheral):Require signals from deliberative mechanisms to fast reactive mechanisms that can, under certain conditions, trigger rapid global reactions.

Tertiary emotions (with and without peripheral effects):Presuppose self-monitoring self-controlling meta-management systems that can be disrupted or modulated by other sub-processes.

Further Refinements

Finer emotional distinctions can be made when we understand the underlying architecture better.E.g.

– “Purely central” vs “partly peripheral” secondary emotions.– Second-order emotions (being ashamed of feeling jealous).– Deliberately induced emotions (teacher who – reluctantly – allows himself to get angry to achieve control of a difficult class)– Emotions that involve constant activity (plotting, fretting, fuming, ranting).– Emotions that vary in intensity over time.– Long term, mostly dormant, emotions, e.g. jealousy, grief. (Often ignored.)

Important Human EmotionsSocially important human emotions involve rich concepts and knowledge and high level control mechanisms (architectures).

Example: longing for someone or something: Semantics:

To long for something you need to know of its existence, its remoteness, and the possibility of being together again.

Control:One who has deep longing for X does not merely occasionally think it would be wonderful to be with X. In deep longing, thoughts are often uncontrollably drawn to X. Moreover, such longing may impact on various kinds of high level decision making as well as the focus of attention.

Physiological processes (outside the brain) may or may not be involved.

Summary So Far1. We can reduce conceptual muddles regarding emotion by trying to use architecture-based concepts.2. Different architectures are relevant in different contexts (e.g. infants, adults, other animals). So we need to explore different families of concepts (e.g. for describing infants, chimps, cats, people with brain damage).3. Finding out which architectures are relevant is a hard research problem. One suggestion is that humans have three architectural layers that manifest themselves not only centrally but also in perception and action sub-systems. Most other animals have only a subset.4. At least three (and several more if we look closely) classes of affective states and processes can be distinguished, related to different architectural layers.5. Many other concepts (e.g. “learning”, “belief”, “motivation”, “intentional action”) can be refined on the basis of hypothesised architectures.

What Use are Emotions• In 1994 Antonio Damasio, a well known neuroscientist, published his book Descartes’ Error.

He argued that emotions are needed for intelligence, and accused Descartes and many others of not grasping that.

• In 1996 Daniel Goleman published Emotional Intelligence: Why It Can Matter More than IQ, quoting Damasio with approval.• Likewise Rosalind Picard a year later in her book Affective Computing.• Since then there has been a flood of publications and projects echoingDamasio’s claim, and many researchers in Artificial Intelligence have become convinced that emotions are essential for intelligence, so they are now producing many computer models containing a module called ‘Emotion’.• Before that, serious researchers had begun to argue that the study of emotions and affect had not had its rightful place in psychology, and cognitive science, but the claims were more moderate.

E.g. a journal called Cognition and Emotion was started in 1987.

What Use are Emotions

Damasio’s argument rested heavily on two examples:

• Phineas Gage: In 1848, an accidental explosion of a charge he had set blew his tamping iron through his head – destroying the left frontal part of his brain.

“He lived, but having previously been a capable and efficient foreman, one with a well-balanced mind, and who was looked on as a shrewd smart business man, he was now fitful, irreverent, and grossly profane, showing little deference for his fellows. He was also impatient and obstinate, yet capricious and vacillating, unable to settle on any of the plans he devised for future action. His friends said he was No longer Gage.”

http://www.deakin.edu.au/hbs/GAGEPAGE/Pgstory.htm

What Use are Emotions

• Elliot, Damasio’s patient (‘Elliot’ was not his real name.) Following a brain tumor and subsequent operation, Elliot suffered damage inthe same general brain area as Gage (left frontal lobe).

Like Gage, he experienced a great change in personality. Elliot had been a successful family man, and successful in business. After his operation he became impulsive and lacking in self-discipline. He could not decide between options where making the decision was important but both options were equally good. He perseverated on unimportant tasks while failing to recognize priorities. He had lost all his business acumen and ended up impoverished, even losing his wife and family. He could no longer hold a steady job. Yet he did well on standard IQ tests.

http://serendip.brynmawr.edu/bb/damasio/

What Follows?

Both patients appeared to retain high intelligence as measured by standard tests, but not as measured by their ability to behave sensibly.

Both had also lost certain kinds of emotional reactions.

Damasio’s Argument

In a nutshell, here is the argument Damasio produced which many people in many academic disciplines enthusiastically accepted as valid:

There are two factual premises from which a conclusion is drawn.P1 Damage to frontal lobes impairs emotional capabilitiesP2 Damage to frontal lobes impairs intelligenceC Emotions are required for intelligence

Is this a valid argument?

One can argue that the conclusion does not follow from the premises.

Whether the conclusion is true is a separate matter.

Compare This Argument

We ‘prove’ that cars need functioning horns in order to start, using two premises on which to base the conclusion:

P1 Damaging the battery stops the horn working in a carP2 Damage to the battery prevents the car startingC A functioning horn is required for the car to start

Does C follow from P1 and P2?

A Critical Observation

The following should occur to the thoughtful listener:

• two capabilities A and B could presuppose some common mechanism M, so that• damaging M would damage both A and B• without either of A or B being required for the other.

For instance, even if P1 and P2 are both true, you can damage the starter motor and leave the horn working, or damage the horn and leave the starter motor working!

Why Buy Damasio’s ArgumentA possible explanation for the surprising fact that so many intelligent

people so easily accept what appears to be an invalid argument is sociological: they are part of a culture in which people want the conclusion to be true.

There seems to be a wide-spread (though not universal) feeling, even among many scientists and philosophers, that intelligence, rationality, critical analysis, problem-solving powers, are over-valued, and that they have defects that can be overcome by emotional mechanisms.

This leads people to like Damasio’s conclusion. They want it to be true. And this somehow causes them to accept as valid an argument for that conclusion, even though they would notice the flaw in a structurally similar argument for a different conclusion (e.g. the car horn example).

On the Other Hand ...

In fact Damasio produced additional theoretical explanations of what is going on, so, in principle, even though the quoted argument is invalid, the conclusion might turn out to be true and explained by his theories.

Emotions and Interfaces

Expressive InterfacesUser FrustrationAnthropomorphism in InteractionSynthetic Characters (Agents)

Motivation

functionality and performance aspects are not always sufficient to make users comfortable expressive interfaces can convey additional information to the useraffective computing has to be used with care, it gets annoying very easily

Objectives

be aware of the effects that the use of expressive computing methods can have in user interactionidentify appropriate techniques that enhance the user’s comfort level with the systembalance the quality, quantity, and expressiveness of feedback from the system to the useridentify and avoid common mistakes that can lead to user frustration

How Interfaces Affect Users

Expressive interfaces how the ‘appearance’ of an interface can elicit positive responses

Negative aspects how computers frustrate users

Anthropomorphism and interface agentsThe pros and cons

Designing synthetic characters

Affective Aspects

HCI has generally been about designing efficient and effective systemsRecently, move towards considering how to design interactive systems to make people respond in certain ways

e.g. to be happy, to be trusting, to learn, to be motivated

Expressive Interfaces

Color, icons, sounds, graphical elements and animations are used to make the ‘look and feel’ of an interface appealing

Conveys an emotional state

In turn this can affect the usability of an interface

People are prepared to put up with certain aspects of an interface (e.g. slow download rate) if the end result is very appealing and aesthetic

Friendly Interfaces

Microsoft pioneered friendly interfaces for technophobes - ‘At home with Bob’ software

Microsoft Bob

Microsoft Bob was designed to be a user friendly interface for Microsoft Windows version 3.1. Bob included various office suite programs such as a calendar, a finance application, and a word processor. The user interface was designed to be helpful to novice computer users, but many saw its methods of assistance as too cute and involved. Each action, such as creating a new text document, featured the step-by-step tutorials no matter how many times the user had been through the process. Users were assisted by cartoon characters whose appearance was usually vaguely related to the task. These characteristics earned Bob the 7th place in PC World (March 26, 2006) Magazine's list of the 25 worst products of all time.

Microsoft Bob

Bob featured a diverse selection of "guides," representing an early form of the Office Assistant and Microsoft Agent technologies. Each guide possessed its own unique "personality" and had its own array of animations.

Bob’s Buddies

Rover, the yellow dog Blythe, a firefly from New York City Chaos, a fluffy cat from France Digger, an Irish earthworm Hopper, a blue stuffed bunny rabbitJava, a coffee-drinking dragon from Guatemala Ruby, a parrot Shelly, a turtle that wears a backpack and carries a walking stick Scuzz, a guitar-playing ratSpeaker, a black computer speakerWill, a cartoon of William ShakespeareThe Dot, a red ballBaudelaire, a gargoyle from a gothic cathedral

Friendly Interfaces

3D metaphors based on familiar places (e.g. living rooms)

Agents in the guise of pets (e.g. bunny, dog) were included to talk to the user

Make users feel more at ease and comfortable

User-created Expressiveness

Users have created emoticons - compensate for lack of expressiveness in text communication:

Happy :)Sad :<Sick :XMad >:Very angry >:-(

Also use of icons and shorthand in text and instant messaging has emotional connotations, e.g.

I 12 CU 2NITE

User Frustration

rational aspectthe system does not do what the user wants it to dotolerable if it is (more or less) expected

emotional aspectthe user is upset about this

mismatch with the mental model the user hasdisregard of the user’s perspective by the designer

may affect the system as a wholeeven if only a small part causes the frustration

may be difficult to recover from

Exercise: My Most Frustrating Computer Interaction

identify an interaction with a computer system that really frustrates youwhat are the causes for your frustration:

inability to do somethingunexpected outcome“the idiots who wrote the program”

try to contrast this with an example where the system does not work for you, but it is not frustrating for you

Causes for User frustration

Many causes:

When an application doesn’t work properly or crashes

When a system doesn’t do what the user wants it to do

When a user’s expectations are not met

When a system does not provide sufficient information to enable the user to know what to do

When error messages pop up that are vague, obtuse or condemning

When the appearance of an interface is garish, noisy, gimmicky or patronizing

Error Messages

“The application Word Wonder has unexpectedly quit due to a type 2 error.”

Why not instead:

“the application has expectedly quit due to poor coding in the operating system”

Shneiderman’s guidelines for error messages include:

avoid using terms like FATAL, INVALID, BADAudio warnings Avoid UPPERCASE and long code numbersMessages should be precise rather than vagueProvide context-sensitive help

Website Error Message

More Helpful Error Message

“The requested page /helpme is not available on the web server.

If you followed a link or bookmark to get to this page, please let us know, so that we can fix the problem. Please include the URL of the referring page as well as the URL of the missing page.

Otherwise check that you have typed the address of the web page correctly.

The Web site you seekCannot be located, butCountless more exist.”

Should Computers Say They’re Sorry?

Reeves and Naas (1996) argue that computers should be made to apologize

Should emulate human etiquette

Would users be as forgiving of computers saying sorry as people are of each other when saying sorry?

How sincere would they think the computer was being? For example, after a system crash:

“I’m really sorry I crashed. I’ll try not to do it again”

How else should computers communicate with users?

Anthropomorphism

Attributing human-like qualities to inanimate objects (e.g. cars, computers)

Well known phenomenon in advertising Dancing butter, drinks, breakfast cereals

Much exploited in human-computer interaction

Make user experience more enjoyable, more motivating, make people feel at ease, reduce anxiety

Which Do You Prefer?

1. As a welcome message

“Hello Chris! Nice to see you again. Welcome back. Now what were we doing last time? Oh yes, exercise 5. Let’s start again.”

“User 24, commence exercise 5.”

Which Do You Prefer?

2. Feedback when get something wrong

1. “Now Chris, that’s not right. You can do better than that. Try again.”

2. “Incorrect. Try again.”

Is there a difference as to what you prefer depending on type of message? Why?

Evidence to Support Anthropomorphism

Reeves and Naas (1996) found that computers that flatter and praise users in education software programs -> positive impact on them

“Your question makes an important and useful distinction. Great job!”

Students were more willing to continue with exercises with this kind of feedback

Criticism of Anthropomorphism

Deceptive, make people feel anxious, inferior or stupid

People tend not to like screen characters that wave their fingers at the user & say:

Now Chris, that’s not right. You can do better than that. Try again.”

Many prefer the more impersonal:“Incorrect. Try again.”

Studies have shown that personalized feedback is considered to be less honest and makes users feel less responsible for their actions (e.g. Quintanar, 1982)

Virtual Characters

Increasingly appearing on our screensWeb, characters in videogames, learning companions, wizards, newsreaders, popstars

Provides a persona that is welcoming, has personality and makes user feel involved with them

Virtual Characters in Video Games

interaction:if you know that an entity in the game is virtual, do you interact with it differently?

emotion:do virtual characters sometimes generate strong emotions?

Disadvantages

Lead people into false sense of belief, enticing them to confide personal secrets with chatterbots (e.g. Alice)Annoying and frustrating

E.g. Clippy

Not trustworthyvirtual e-commerce assistants?

Virtual Characters: Agents

Can be classified in terms of the degree of anthropomorphism they exhibit:• Synthetic characters• animated agents • emotional agents • embodied conversational agents

(i)Synthetic Characters -Silas the Dog

autonomous, with internal states and able to respond to external events

(Blumberg, 1996 - MIT)

(ii) Animated Agents

Play a collaborative role at the interfaceOften cartoon-likee.g. Herman the bug(Lester et al, 1997Intellimedia)

flies into plants& explains things on-the-fly & gives advice to students

(iii) Emotional Agents

Pre-defined personality and set of emotions that user can change

The Woggles, Bates, 1994

(iv) Embodied Conversational Agents

Rea, real-estate agent, showing useran apartment

Human-like body

Uses gesture, non-verbal communication (facialexpressions, winks) while talking

Sophisticated AI techniques used to enable this form of interaction Cassell, 2000, MIT

Conversation with Rea

Mike approaches screen and Rea turns to face him and says:Hello. How can I help you?Mike: I’m looking to buy a place near MIT.Rea nods, indicating she is following.Rea: I have a house to show you. (picture of a house appears on the screen)Rea: it is in Somerville.Mike: Tell me about it.Rea looks up and away while she plans what to say.Rea: It’s big.Rea makes an expansive gesture with her hands.Mike brings his hands up as if to speak, so Rea does not continue, waiting for him to speak.Mike: Tell me more about it.Rea: Sure thing. It has a nice garden...

Believable Agents

Believability refers to the extent to which users come to believe an agent’s intentions and personalityAppearance is very important

Are simple cartoon-like characters or more realistic characters, resembling the human form more believable?

Behaviour is very important How an agent moves, gestures and refers to objects on the screenExaggeration of facial expressions and gestures to show underlying emotions (cf animation industry)

Key Points

Affective aspects are concerned with how interactive systems make people respond in emotional ways

Well-designed interfaces can elicit good feelings in users

Expressive interfaces can provide reassuring feedback

Badly designed interfaces make people angry and frustrated

Anthropomorphism is increasingly used at the interface, in the guise of agents and virtual screen characters