correction of oblig. 1 presentation of oblig. 2 · Assignment 2 • In this assignment, you will be asked to develop a small dialogue system 22 • The domain is a simple, simulated
Post on 26-Mar-2020
0 Views
Preview:
Transcript
Exercise session: correction of oblig. 1 & presentation of oblig. 2
Pierre Lison, Language Technology Group (LTG)
Department of Informatics
Fall 2012, October 1 2012
@ 2012, Pierre Lison - INF5820 course
Outline
• Correction of assignment 1
• Presentation of assignment 2
2
@ 2012, Pierre Lison - INF5820 course
Outline
•Correction of assignment 1
• Presentation of assignment 2
3
@ 2012, Pierre Lison - INF5820 course
Question 1: analysis of dialogue
• Turns mostly structured by
• Dialogue structure (questions followed by answers, etc.)
• Complete grammatical units
• + probably intonation, non-verbal cues etc. (although we don’t have direct access to them)
4
@ 2012, Pierre Lison - INF5820 course
Question 1: analysis of dialogue
• Speech acts:
• Assertives: «he sent it this afternoon I think», «we’ll be in Trondheim that day»
• Directives: «could you help me carry these groceries in the kitchen?»
• Commissives: «sure will do»
• Expressives: «poor you», «too bad»
5
@ 2012, Pierre Lison - INF5820 course
Question 1: analysis of dialogue
• Grounding and clarification:
• Lots of backchannels («mm» etc.)
• Clarification requests («which email?»)
• Explicit feedback («oh i see», «poor you»), on several levels (perception, understanding, attitudes)
• Implicit feedback («he sent his email this afternoon» - «didn’t check my email this afternoon»)
6
@ 2012, Pierre Lison - INF5820 course
Question 1: analysis of dialogue
• Conversational implicatures:
• «how was your day?» «well my boss put one more big pile ....»
• «I didn’t check my email this afternoon»
• «We’ll be in Trondheim that day»
• The indirect request «why don’t you tell him that ...» could also be seen as a conversational implicature
7
@ 2012, Pierre Lison - INF5820 course
Question 1: analysis of dialogue
• Deictic markers:
• Pronouns: I, you, we, my
• Adverbs: there
• Demonstratives: these, this
• Temporal phrases: this afternoon, the following weekend
• Tense markers: «he sent it»
• etc.
8
@ 2012, Pierre Lison - INF5820 course
Question 2: shared intentionality
• Shared intentionality
9
our cognition is directed towards things in the world (desires, goals, attention)
@ 2012, Pierre Lison - INF5820 course
Question 2: shared intentionality
• Shared intentionality
10
our cognition is directed towards things in the world (desires, goals, attention)
we can share this intentionality with others (joint attention, collaborative activities)
@ 2012, Pierre Lison - INF5820 course
Question 2: shared intentionality
• Shared intentionality presupposes advanced skills for reading the intentions (and beliefs) of others
• Humans can also have joint commitments towards goals and coordinate action plans to realise them
• Spoken dialogue is a prototypical example of collaborative activity:
• Need to read and interpret each other’s intentions
• Gradual refinement & expansion of common ground
• Joint attention towards particular topics in focus
• Role of imitation and alignment
11
@ 2012, Pierre Lison - INF5820 course
Question 3: phonetics
• Word boundaries difficult to detect on the waveform
• Stops: silence followed by a «burst»
• Fricatives: uneven, aperiodic waveform
12
@ 2012, Pierre Lison - INF5820 course
Question 3: phonetics
• Word boundaries difficult to detect on the waveform
• Stops: silence followed by a «burst»
• Fricatives: uneven, aperiodic waveform
12
Time (s)3.731 5.141
-0.2334
0.1682
0
untitled
@ 2012, Pierre Lison - INF5820 course
Question 3: phonetics
• Word boundaries difficult to detect on the waveform
• Stops: silence followed by a «burst»
• Fricatives: uneven, aperiodic waveform
12
@ 2012, Pierre Lison - INF5820 course
Question 3: phonetics
• Word boundaries difficult to detect on the waveform
• Stops: silence followed by a «burst»
• Fricatives: uneven, aperiodic waveform
12
Time (s)0.7956 2.387
-0.07474
0.07803
0
untitled
@ 2012, Pierre Lison - INF5820 course
Question 3: phonetics
• Pitch variation indicates the utterance’s intonation (e.g. declarative vs interrogative sentence)
13
• Vowels have distinct formant structures that can help us distinguish them
@ 2012, Pierre Lison - INF5820 course
Question 3: phonetics
• Intensity (in dB) can be calculated with the following formula:
14
for a sample of N time points t1,... tN
P0 is the human auditory threshold, = 2 x 10-5 Pa
Intensity = 10 log10Power
P0= 10 log10
1
NP0
N�
i=1
y(ti)2
Power =1
N
N�
i=1
y(ti)2
@ 2012, Pierre Lison - INF5820 course
Question 3: phonetics
15
[Marta’s solution]
@ 2012, Pierre Lison - INF5820 course
Question 4: probabilistic models
16
[Henning’s solution]
@ 2012, Pierre Lison - INF5820 course
Question 4: probabilistic models
17
[Henning’s solution]
@ 2012, Pierre Lison - INF5820 course
Question 4: probabilistic models
18
[Henning’s solution]
@ 2012, Pierre Lison - INF5820 course
Question 4: probabilistic models
19
P (B|JC,¬MC)
= αP (JC,¬MC|B)P (B)
= α
�
a={T,F}
P (JC,¬MC,A = a|B)
P (B)
= α
�
a={T,F}
P (JC|A = a)P (¬MC|A = a)P (A = a|B)
P (B)
= α
�
a={T,F}
P (JC|A = a)P (¬MC|A = a)
�
e={T,F}
P (A = a|B,E = e)P (E = e)
P (B)
Marginalisation on A
Bayes’ rule
Marginalisation on E
NB: there are several alternative ways to get to this result
@ 2012, Pierre Lison - INF5820 course
Question 4: probabilistic models
• Then we simply calculate the results for Burglary=true and Burglary=false and renormalise at the end:
20
P (¬B|JC,¬MC) = α× (0.9× 0.3× 0.29× 0.002
+0.9× 0.3× 0.001× 0.998 + 0.05× 0.99× 0.71× 0.002
+0.05× 0.99× 0.999× 0.998)× 0.999 = α× 0.049798
P (B|JC,¬MC) = α× (0.9× 0.3× 0.95× 0.002
+0.9× 0.3× 0.95× 0.998 + 0.05× 0.99× 0.05× 0.002
+0.05× 0.99× 0.05× 0.998)× 0.001 = α× 0.0002589
Renormalising with α = 19.977, we then have P(B|JC, ¬MC) = 0.00517
@ 2012, Pierre Lison - INF5820 course
Outline
• Correction of assignment 1
•Presentation of assignment 2
21
@ 2012, Pierre Lison - INF5820 course
Assignment 2
• In this assignment, you will be asked to develop a small dialogue system
22
• The domain is a simple, simulated human-robot interaction scenario:
• A robot is standing on a table where several objects are present
• The robot can move around on the table, perceive the objects in front of him, and pick them up
@ 2012, Pierre Lison - INF5820 course
Assignment 2
• Basic skeleton for the dialogue system is provided
• The speech recognition and synthesis will be done «in the cloud», using the API provided by AT&T (but you will have to provide a speech recognition grammar for the ASR)
• Libraries for recording/playing sounds and connecting to AT&T servers for the ASR and TTS
• As well as a GUI for the simulation and display of the dialogue history
• Your task is to implement a dialogue policy processing the input (N-best list) and returning an appropriate action:
23
public Action processInput(NBest u_u, DialogueState dstate, WorldState wstate) { ... }
@ 2012, Pierre Lison - INF5820 course
Assignment 2
• The dialogue policy must be able to handle the following:
• Greetings («hi») and closings («goodbye»)
• Commands to move the robot («go forward», «turn left», «turn right»)
• Talking about objects in the scene («do you see an object?», «what do you see?»)
• Picking up and releasing objects («pick up the object», «put down the object»)
• The system must also be able to produce clarification requests («sorry could you repeat?») when things are unclear
24
@ 2012, Pierre Lison - INF5820 course
Assignment 2
• You can implement your policy in Java or in Python
• For Python, you’ll have to use Jython (a python implementation running on the JVM, and which can directly use Java classes)
• The package is now ready to use
• You’re of course free to help me improve or debug it (the code is made available on a SVN server)
• Submission deadline: October 21st
25
top related