Emotions in Argumentation: an Empirical Evaluation @ IJCAI 2015

Emotions in Argumentation

an Empirical Evaluation

Sahbi Benlamine, Maher Chaouachi, Claude Frasson

Serena Villata, Elena Cabrio, Fabien Gandon

Web Agora

web

read-write web

social web

argument web

debate web

emotion web

semantic web

debate traces

question:Connection between the arguments proposed by the participants of a debate and their emotional status?• correlation of polarity of

arguments and polarity of detected emotions?

• relation between kinds and amount of arguments, and the engagement of participants?

Web AgoraArgumentation ( )

argumentation support decision-

making and persuasion

e-democracy and online debates

abstract bipolar argumentation[Dung, 1995; Cayrol and Lagasquie-Schiex, 2013]

support and attack relations

e.g.A. Elena: This information is

important, we must publish it

B. Serena: It is a private information about a person who does not want to publish it

C. Elena: This person is the Prime Minister so the information is not private

D. Fabien: Yes, being a governmental officer makes the information about him public

attack

attack

support

argument network analysisattack

attack

support

A.

B.

C.

D.

Web AgoraArgumentationEmotions ( )

emotion computing beyond purely

rational behavior detect emotional

state to adapt reactions

e.g.(serious) games

emotion detection webcams for facial

expressions analysis [FACEREADER 6.0]

physiological sensors(EEG) for cognitivestates [Chaouachi et al., 2010]

real–time engagement engagement index

[Pope et al.,1995]

EEG frequency bands (4-8Hz) α (8-13Hz) β (13-22Hz)

real-time facial analysis classifying 500 key

points in facial muscles

neural network trained on 10 000

examples happy, sad, angry,

surprised, scared, disgusted.

valence, arousal neutral probability.

Web AgoraArgumentationEmotionsSeempad project

Seempad Joint Research Lab Emotions play an

important role in decision making [Quartz, 2009]

Assess connection between argumentation and emotions

Final goal = Detect on the Web…• a debate turning into a

flame war,• a content reaching an

agreement,• a good or bad emotion

spreading in a community…

+

1st Experiment & Public datasetfocus on associating participants’ arguments and the relations

among them mental engagement

detected by EEG facial emotions detected via

Face Emotion Recognition

protocol of the experiment topics from popular

discussions in iDebate & DebateGraph

12 debates - 4 participants and 2 moderators each

participants equipped with emotions detection tools

messages in plain English through IRC

participants are anonymous debate for 20 minutes debrief questionnaire

participants 6 sessions of 4 participants

(-1 session) 20 participants (7 women, 13

men) age range from 22 to 35

years not all of them were native

English speakers students in a North American

university signed an ethical agreement good computer skills

data collection during the experiment minimum, average and

maximum engagement of every participant in a debate

most dominant emotion (having maximum value)

pleased/unpleased valence active/inactive arousal

data annotation after the experiment synchronize arguments,

relations and emotional indexes

bipolar argumentation labelled with : sources, arguments, emotional states

two independent annotators with agreement of 91%

Cohen’s kappa 0.82 >> 0.6

publicly available dataset

dataset content xml structure of debate flow 598 arguments in 12

different debates 263 argument pairs

127 supports 136 attacks

gender, age and personality type

dominant emotion, valence and arousal

mental engagement levels

dataset extract (flow)<argument id="2" debate_id="4" participant="4" time-from="20:30" time-to="20:30">The religion is an independent factor, it should not be a dissociative factor separating people. </argument>

<argument id="3" debate_id="4" participant="1" time-from="20:32" time-to="20:32">The religion gives to his followers hope and help them to overcome some problem of the life so it's not all bad. </argument>

<argument id="4" debate_id="4" participant="4" time-from="20:32" time-to="20:32">Here in Canada it is appreciable to find the liberty of religion a practice in a peaceful way. </argument>

dataset extract (relations)<debate id="4" title="Religion" task="relation">

<pair id="1" relation="support">

<argument id="2" debate_id="4" participant="4" time-from="20:30" time-to="20:30">The religion is an independent factor, it should not be a dissociative factor separating people. </argument>

<argument id="3" debate_id="4" participant="1" time-from="20:32" time-to="20:32">The religion gives to his followers hope and help them to overcome some problems of the life so it's not all bad. </argument>

</pair>

<pair id="2" relation="attack">

<argument id="3" debate_id="4" participant="1" time-from="20:32" time-to="20:32">The religion gives to his followers hope and help them to overcome some problems of the life so it's not all bad. </argument>

<argument id="5" debate_id="4" participant="3" time-from="20:32" time-to="20:32">During all the existence of human being, religion makes a lot of issue. It make more hurts than curs. </argument>

</pair>

dataset extract (emotions)<argument id="30" debate_id="4" participant="4" time-from="20:43" time-to="20:43" emotion_p1="neutral" emotion_p2="neutral" emotion_p3="neutral" emotion_p4="neutral">

Indeed but there exist some advocates of the devil like Bernard Levi who is decomposing arabic countries. </argument>

<argument id="31" debate_id="4" participant="1" time-from="20:43" time-to="20:43" emotion_p1="angry" emotion_p2="neutral" emotion_p3="angry" emotion_p4="disgusted">

I don’t totally agree with you Participant2: science and religion don’t explain each other, they tend to explain the world but in two different ways.

</argument>

<argument id="32" debate_id="4" participant="3" time-from="20:44" time-to="20:44" emotion_p1="angry" emotion_p2="happy" emotion_p3="surprised" emotion_p4="angry">

Participant4: for recent wars ok but what about wars happened 3 or 4 centuries ago? </argument>

initial analysis H1 : some emotional and

behavioral trends can be extracted from a set of debates.

H2 : the number and the strength of arguments, attacks and supports exchanged between the debaters are correlated with particular emotions.

H3 : the number of expressed arguments is connected to the degree of mental engagement and social interactions.

initial analysis mean percentage of

appearance of each basic emotion (with 95% confidence interval)

that the most frequent : anger 8:15% to 15:6%

second most frequent : disgust 7:52% to 14:8%

negativity effect [Rozin and Royzman, 2001]

high level engagement (70:2% to 87:7% of time)correlated with anger (r=0.306)

emotional evolution participant 1 debate 1

before and after argument rejection(hypothesis 2)

correlation table for Session 3 “Distribute condoms at schools”“Encourage fewer people to go to the university”

number of attacks increased linearly with disgust (H2)

aggregated correlation table

supports increase linearly with engagement (r=0.31)more pronounced when no conflict (r=0.80)

high engagement for most participative participants (H3)

to wrap-upconnection between the arguments proposed by the participants of a debate and their emotional status? H1 : emotional trends can be

extracted from debates H2 : arguments, attacks,

supports correlated with emotions

H3 : the number of arguments is connected to the engagement

current work granularity: from sessions

to debates existence or not of a priori

positive/negative opinion actual changes of opinion impact of personality (big

5 test) dynamics of the debate

and emotion changes

perspectives deeper analysis, different

time scales and granularity new experiment with

improved protocol NLP analysis and

correlations

http://fabien.info

¿http://project.inria.fr/seempad/ 40/40/…40 :-)