Jun 12, 2022 1 Multiparty Communication with a Tour Guide ECA Aleksandra Čereković HOTLab group Department of telecommunications Faculty of electrical engineering and computing Zagreb University http://hotlab.tel.fer.hr
Jan 12, 2016
Apr 21, 20231
Multiparty Communication with a Tour Guide ECA
Aleksandra ČerekovićHOTLab group
Department of telecommunicationsFaculty of electrical engineering and computing
Zagreb University
http://hotlab.tel.fer.hr
Outline
Apr 21, 20232
Project overview Background Information eNTERFACE ‘08 system
features scenario configuration
Project members
Project Overview
Apr 21, 20233
Project target System with an Embodied Conversational Agent
(ECA) ECA is a tour guide which describes the city of
Dubrovnik to two visitors and interacts with them We aim to explore issues of a multiparty
support in ECA system with focus on: Dialogue support
Participant roles Interaction Management Dialogue structure
Nonverbal behavior model of an ECA Getting attention, model of gazing, maintaining a
conversational flow
Background Information
Apr 21, 20234
During eNTERFACE ‘06 workshop we developed a Tour Guide system
System is based on common GECA framework GECA Platform, GECA Protocol, GECA Plugs Connects various system components Tranfers messages between components in the real
time Enables simple system integration
eNTERFACE ‘08 system scenario
Apr 21, 20235
In our system scenario ECA has a role of narrator ECA takes visitor through 5 different scenes in the
city Users can interrupt ECA
They can raise their hands to take a speaker’s role or just say something
User utterances are predefined: they can make comments, have requests or just express thier willingness to leave the session
Durring interaction ECA keeps initiative and gets attention
eNTERFACE ‘08 system design
Apr 21, 20236
Component functionalities (I)
Apr 21, 20237
Input Captures speech and users’ position, size, openness of
the eyes and the mouth, facial orientation (Okao’s Vision)
Deliberative phase components Input understanding
Interprets raw messages coming from the system input Detects speaker and his utterance (request, comment, leave the
system) Detects speech collision and mutual conversation between users
Decision Making Planner (Dialogue Management) Generates transitions between dialogue states in a multiparty
interaction (Information State, Nonverbal Event, Specifying addressee, Grounding judgment, Assuming the next speaker)
MIDIKI DM toolkit will be extended to manage multiparty interaction
Component functionalities (II)
Apr 21, 20238
Deliberative phase components Behavior Planner
Generates appropriate response by an ECA Dynamically changes ECA’s behaviors according the Information
State
Output Player
Generates ECA’s behaviors selected by Behavior Planner Has a support to stop utterance and start a new one
Project Members
Apr 21, 20239
Project Researches Takuya Furukawa* (Input, Input Understanding) Huang Hsuang- Hung* (DM, Behavior Planner), Shinya
Takeda** and Yuji Yamaoka** (DM) Aleksandra Cerekovic*** (Output)
Project supervisors Toyoaki Nishida* Yukiko Nakano** Igor S. Pandzic***
*University of Kyoto, Graduate School of Informatics, Nishida-Sumi Lab** Seikei University (Tokyo), Faculty of Science and Technology***University Of Zagreb, Faculty of Electrical Engineering and Computing,
HOTLab group