NATURAL LANGUAGE AND DIALOGUE SYSTEMS LAB UC SANTA CRUZ Natural Language and Dialogue Systems Lab Does Personality Matter?: Expressive Generation for Dialog Systems IWSDS 2012 Prof. Marilyn Walker Baskin School of Engineering University of California, Santa Cruz
54
Embed
Does Personality Matter?: Expressive Generation for …...TTS Text-to-Speech Synthesis Spoken Language Understanding Dialogue Management ASR SLG Spoken Language Generation Data, Rules
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
NATURAL LANGUAGE AND DIALOGUE SYSTEMS
LAB
UC SANTA CRUZ
Natural Language and Dialogue Systems Lab
Does Personality Matter?:
Expressive Generation for
Dialog Systems IWSDS 2012
Prof. Marilyn Walker
Baskin School of Engineering
University of California, Santa Cruz
NATURAL LANGUAGE AND DIALOGUE SYSTEMS
LAB
UC SANTA CRUZ
Natural Language and Dialogue Systems Lab
Thank you for inviting me to this lovely
place!
NATURAL LANGUAGE AND DIALOGUE SYSTEMS
LAB
UC SANTA CRUZ
Expressive Language in Conversation
Expresses Speaker’s Personality & Identity
culture, style, origin, class
Dynamically Adapts to Conversational Partner
Convergent : Matching, e.g. two friends (extraverts)
talking
Divergent: Tailoring, e.g. parent to baby
Controlled by generation parameters
Content: Who is interested in what, who knows what
Linguistic: Lexical and Syntactic Choice
Pragmatic: Personality & Social Relationship
Acoustic: Speaking Rate, Amplitude, Prosody
NATURAL LANGUAGE AND DIALOGUE SYSTEMS
LAB
UC SANTA CRUZ 4
Personality in Language: People do it
From Mehl et al., 2006. Mairesse etal 2007.
Introvert Extravert
- I don't know man, it is
fine I was just saying I
don't know.
- I was just giving you a
hard time, so.
- I don't know.
- I will go check my e-mail.
- I said I will try to check
my e-mail, ok.
- Oh, this has been
happening to me a lot lately.
Like my phone will ring. It
won't say who it is. It just
says call. And I answer and
nobody will say anything.
So I don't know who it is.
- Okay. I don't really want
any but a little salad.
NATURAL LANGUAGE AND DIALOGUE SYSTEMS
LAB
UC SANTA CRUZ
Film Characters: Crafted Personalities
NATURAL LANGUAGE AND DIALOGUE SYSTEMS
LAB
UC SANTA CRUZ
Does personality matter for dialogue
systems?
Yes. Achieving communicative goals in dialogue often relies
on engaging user affect:
persuasion, motivation, increase in self-efficacy beliefs,
learning
People react socially to computational agents, thus social
norms such as liking people like yourself often hold (Nass &
Lee, 2001)
People make attributions beyond social level: task
competence
Personality matching in a robotic exercise coach increased the
time that stroke victims spent on their medically recommended
exercises (Tapus & Mataric 2008)
Tutoring oriented to the student’s ‘face needs’ improved
learning in training and tutoring (Porayska-Pomsta & Mellish
2004; Wang et al., 2004)
NATURAL LANGUAGE AND DIALOGUE SYSTEMS
LAB
UC SANTA CRUZ
Dialogue Systems Architecture
DM
SLU
TTS
Text-to-
Speech
Synthesis
Spoken Language
Understanding
Dialogue
Management
ASR
SLG Spoken
Language
Generation
Data,
Rules
Words
Meaning
Speech, Nonverbals Speech, Nonverbals
Goal
Words
Speech
Recognition
Personality?
NATURAL LANGUAGE AND DIALOGUE SYSTEMS
LAB
UC SANTA CRUZ
Outline of my talk
Expressive natural language
generation
PERSONAGE: personality models
for language generation (Mairesse &
Walker, UMUAI 2010; CL 2011, ACL2007; ACL
2008)
Generating nonverbal expression
of personality (Bee, Pollack, Andre’ &
Walker IVA 2010; Neff, Wang, Abbott & Walker
IVA 2010, Neff, Toothman, Grant, Walker IVA
2011)
Generation by learning models of film
characters from corpora (Lin & Walker
AIIDE 2011, Walker, Lin, Wardrip-Fruin, Buell,
Grant ICIDS 2011)
NATURAL LANGUAGE AND DIALOGUE SYSTEMS
LAB
UC SANTA CRUZ 9
Limitations of previous work
Writing character dialogue is an art: it is not described at a
level that supports computational models
User Experience design is largely based on intuition
Work on narrative (arts and humanities) does not suggest specific
linguistic or behavioral reflexes or parameters
There has been little systematic exploration of personality
or social parameters suggested by psycholinguistic
findings
Unclear which psycholinguistic findings have impact in
particular application domains
Almost no evaluation showing variation system produces
is perceived by the user as the system intended
NATURAL LANGUAGE AND DIALOGUE SYSTEMS
LAB
UC SANTA CRUZ
Need a general purpose generation
technology that can be easily ported from
one domain/task to another. Expressive
but also for mixed initiative dialogue (any
branching dialogue).
10
NATURAL LANGUAGE AND DIALOGUE SYSTEMS
LAB
UC SANTA CRUZ
Procedural Language Generation: A Key
Technology
Wide range of generation parameters
Different methods for creating models
that control the parameters
Dynamic Real-Time Adaptation
Trainable: Machine Learning
Techniques
Individual Personalization
11
NATURAL LANGUAGE AND DIALOGUE SYSTEMS
LAB
UC SANTA CRUZ
Dialogue Systems Architecture
DM
SLU
TTS
Text-to-
Speech
Synthesis
Spoken Language
Understanding
Dialogue
Management
ASR
SLG Spoken
Language
Generation
Data,
Rules
Words
Meaning
Speech, Nonverbals Speech, Nonverbals
Goal
Words
Speech
Recognition
Personality?
NATURAL LANGUAGE AND DIALOGUE SYSTEMS
LAB
UC SANTA CRUZ 13
Language Generation Module
What to say
Content
Planner
How to Say It
Sentence
Planner
Surface
Realizer
Prosody
Assigner
What is Heard
Speech
Synthesizer
NATURAL LANGUAGE AND DIALOGUE SYSTEMS
LAB
UC SANTA CRUZ 14
Variation controlled by the Language
Generator
What to say
Content
Planner
How to Say It
Sentence
Planner
Surface
Realizer
Prosody
Assigner
What is Heard
Speech
Synthesizer
Parametrized
Variation
• vary content and form easily depending on any
factor (context, personality, social relationship)
NATURAL LANGUAGE AND DIALOGUE SYSTEMS
LAB
UC SANTA CRUZ
Expressivity?: Which parameters and
models? Theories and Corpus Studies of Human Dialogue Behavior
Psychology: Big Five Theory of Personality
Sociolinguistics: Politeness Theory
Learn from Film Character Dialogue
NATURAL LANGUAGE AND DIALOGUE SYSTEMS
LAB
UC SANTA CRUZ
PERSONAGE Generator: BIG FIVE
Theory
Conscientiousness: Dutiful vs. impulsive
Emotional stability: Calm vs. anxious
Openness to experience: Imaginative vs.
conventional
Agreeableness: Kind vs. unfriendly
Extraversion: Sociable, assertive vs. quiet
NATURAL LANGUAGE AND DIALOGUE SYSTEMS
LAB
UC SANTA CRUZ
Linguistic Reflexes of Personality: 50 yrs of
studies
Extraversion (Furnham, 1990)
Talk more, faster, louder and more repetitively
Fewer pauses and hesitations
Lower type/token ratio
Less formal, more references to context (Heylighen & Dewaele, 2002)
More positive emotion words (Pennebaker & King, 1999)
E.g. happy, pretty, good
Neuroticism (Pennebaker & King, 1999)
1st person singular pronouns
Negative emotion words
Conscientiousness (Pennebaker & King, 1999)
Fewer negations and negative emotion words
Low but significant correlations
NATURAL LANGUAGE AND DIALOGUE SYSTEMS
LAB
UC SANTA CRUZ
18
NATURAL LANGUAGE AND DIALOGUE SYSTEMS
LAB
UC SANTA CRUZ
PERSONAGE Architecture: 67 Parameters
Realization
INPUT: Dialog Act,
Content Pool
OUTPUT
UTTERANCE
VERBOSITY
RESTATEMENTS
CONTENT POLARITY
…
SYNTACTIC
COMPLEXITY
SELF-REFERENCE
…
CONTRAST: e.g.
however, but
JUSTIFY: e.g.
so, since
PERIOD
…
EXCLAMATION
HEDGES: e.g. kind of,
rather, basically, you know
FILLED PAUSES: e.g. err…
SWEAR WORDS: e.g. damn
IN GROUP MARKERS: e.g. pal
STUTTERING: e.g. Ri-Ri-River
TAG QUESTIONS
…
FREQUENCY OF USE
WORD LENGTH
VERB STRENGTH
Content
Planner
Pragmatic
Marker
Insertion
Lexical
Choice Aggregation
Syntactic
Template
Selection
NATURAL LANGUAGE AND DIALOGUE SYSTEMS
LAB
UC SANTA CRUZ 20
Example of Pragmatic Transformation
Negation insertion “X has awful food” “X doesn’t have good food”
Wok Mania class: proper noun
number: sg
have class: verb
awful class:adjective
food class: noun
number: sg
article: none
Obj Subj
ATTR
WordNet
Database Look for antonym
“good”
- Negate verb
- Replace adjective
by antonym Wok Mania class: proper noun
number: sg
have class: verb
negated: true
good class:adjective
food class: noun
number: sg
article: none
Obj Subj
ATTR
NATURAL LANGUAGE AND DIALOGUE SYSTEMS
LAB
UC SANTA CRUZ 21
Recommendation: A Persuasive Task
Alt Realization Extra
5 Err... it seems to me that Le Marais isn’t as bad as the others. 1.83
4 Right, I mean, Le Marais is the only restaurant that is any good. 2.83
8 Ok, I mean, Le Marais is a quite french, kosher and steak house place, you know and
the atmosphere isn’t nasty, it has nice atmosphere. It has friendly service. It seems to
me that the service is nice. It isn’t as bad as the others, is it?
5.17
9 Well, it seems to me that I am sure you would like Le Marais. It has good food, the food
is sort of rather tasty, the ambience is nice, the atmosphere isn’t sort of nasty, it
features rather friendly servers and its price is around 44 dollars.
5.83
3 I am sure you would like Le Marais, you know. The atmosphere is acceptable, the
servers are nice and it’s a french, kosher and steak house place. Actually, the food is
good, even if its price is 44 dollars.
6.00
10 It seems to me that Le Marais isn’t as bad as the others. It’s a french, kosher and steak
house place. It has friendly servers, you know but it’s somewhat expensive, you know!
6.17
2 Basically, actually, I am sure you would like Le Marais. It features friendly service and
acceptable atmosphere and it’s a french, kosher and steak house place. Even if its
price is 44 dollars, it just has really good food, nice food.
6.17
NATURAL LANGUAGE AND DIALOGUE SYSTEMS
LAB
UC SANTA CRUZ
Training Models: Human Perceptions. 4
methods
Rule based: take the findings from the psych literature and encode them in a model
Overgenerate and Rank: generate many possibilities, collect human perceptual ratings, learn to select ones that match the human perception you are trying to achieve
Parameter Estimation: Assume parameters are independent and learn to set them as a function of personality desired
Film Corpora: Learn models from film dialogue (Mairesse&Walker07, Mairesse & Walker UMUAI 2010,