XISL: Content-Independent, Device-Independent MMI Description Language Tsuneo Nitta Graduate School of Tech., Toyohashi Univ. of Tech.

XISL: Content-Independent, Device-Independent MMI Description Language

Tsuneo Nitta

Graduate School of Tech., Toyohashi Univ. of Tech.

Issues on Interaction with Web

■ The coming of heterogeneous devices

and interaction manner

• access by : PC, Mobile phone, PDA, AutoPC, DTV, …

• interaction with: modality-rich terminals, mixed initiative, ...

Ex. On-line Shopping Devices - order goods at Kiosk ..... touch panel

- confirm the order with a mobile phone ..... voice- change the order at home ..... DTV remote terminal

XML Document Server(Contents ; View ; Interaction)

Our Purpose: Seamless Services over Internet

XML Document Server(Contents ; View ; Interaction)

How to do it?

<xhtml ........

<head>

VoiceXML

</head>

<body>

........

CSSCSS

<xhtml .......

Form

<salt: ......

CSS

XISL: Extensible Interaction-Sheet Language

CSS

<xhtml .......

Form

<xisl>

<dialog>

<exchange>

........

contents-

independent

<xhtml .......

Form

VoiceXML

........

remodels VoiceXML to meet MMI requirements

XISL: Device-Independent - 1/3

no modality-specific tag : input modality

<input type=“speech” event=“recog” target=“grammar.gxml”

match=“/grammar/goods” return=“goods_name”>

<param name=“mode”>data</param>

</input>

<input type=“dtmf” event=“push” target=“grammar.gxml”

match=“/grammar/goods_select”return=“goods_name”>

<param name=“mode”>digit</param>

</input>

...........


Front-end Module

<input ….. >type: speech

event:recognize recognizegrammar rule:

introductionreturn: word1

<input ….. >

type: pointing

:

ASR

Pointing

Keyboard

Hello!

Hellowor

d1: H

ellogr

amm

ar

type: speech

event: recognize

target: ***.xml

match: /page

return:

word1:Hello

Dialog Manager


no modality-specific tag : output modality

type event modewindow open, close, …speech play, stop, … TTS, rec_speech, …video play, stop, …audio play, stop, …agentmove, speak, …

MMI Description Language: XISL

- is content ( & view ) -independent.

- has no modality-specific tags that enable XISL device-independent.- has MMI control tags and attributes.

→ • realizes seamless services over internet and reuse of XML documents. • meets modality-rich services and avoids patent issues.

XISL enables combinatorial usage of modalities.

Sequential Parallel

adding a new modality

Alternative

<?xml version=“1.0” encoding=“Shift-JIS”?><!DOCTYPE xisl SYSTEM “xisl.dtd”><xisl version=“1.0”> <head>.........</head> <body> <dialogue id = “WAP_phone”> <exchange> <operation comb = “alt”>　 <input type=“dtmf” event=“push” target=“/d_gram.gxml” match=“/d_gram/goods_select” return=“number”> <param name=“mode”>digit</param> </input> <input type=“speech” event=“recognize” target=“/gram.gxml” match=“/gram/goods_select” return=“goods_name”> <param name=“mode”>data</param> </input> < /operation >

XISL: example -1/2

< action comb = “seq” > <output type=“speech” event=“play”/> <param name=“speaker”>nancy</param> <param name=“speech_text”> You selected No. One. < /output > < call dialogue_name=“2”/ > < /action > ：：：

XISL: example -2/2

Implementation -1/4

XISLXMLXSL

Inp

ut

Mod

alities

Ou

tpu

t M

odalitie

s

Document Server

Interpreter Interpreter

Dialogue

Manager

Dialogue

Manager

Input Integrator

Input Integrator

Response Controller

Response Controller

System Architecture

Dialogue Control

<output>

I/O profile

XML editing

transition

<action>

Op. Control

GLRparser

Input Integration

<input>

user operation

<operation>

Inte

rpre

ter

Start Dialog Dia

log M

anager

Action

Execution

Implementation -2/4 D

ocu

men

t Server

Do

cum

ent S

erver

Fro

nt - en

d

Fro

nt - en

d

Do

cum

ent B

uffer

Dialogue Control by Using GLR TBL

REDUCE-2５REDUCE-1４

SHIFT-5３SHIFT-4２

ACCEPT１SHIFT-2SHIFT-1SHIFT-3０EXCH1BUYGOODScompletetimeoutgoods_spbuy_btn

GOTO TBLACTION TBL INState

０ SHIFT-3

３ SHIFT-5

５ REDUCE-2

Implementation -3/4

①

②

③

Implementation -4/4

XISL Tree View　ＸＩＳＬの構造をツリー形式･･･

で表示Ｉｎｔｅｒａｃｔｉｏｎ　Ｗｉｎｄｏｗ

　生成したＸＩＳＬを表･･･示

Interaction Builder - binding MMI -1/4


<input type="touch" event="click" match="/OLS/goods_list/category/goods[@id='23']/object/picture[1]"/>


<output type="speech" event="agent-speech"><![CDATA[<param name="agent-name">Merlin</param><param name="speech-text">このコートは着心地抜群です</param>]]>


Future Work

- XHTML + XISL

- Implementation of 3GPP applications

- MMI Requirements: more complicated usage of MMI

- Rapid-prototyping: GUI-based developer support system (IPA project)

http://www.vox.tutkie.tut.ac.jp/XISL/XISL-E.html

XISL: Content-Independent, Device-Independent MMI Description Language Tsuneo Nitta Graduate School of Tech., Toyohashi Univ. of Tech.

Documents

internet slide

xisl implementation

end document buffer

state shift

interaction manner access

digitdata xisl

complicated usage of

mobile phone