towards a ubiquitous semantics of interaction

towards a ubiquitous semantics of interaction

Alan Dix

Lancaster University

formalism in HCI

adolescence

or …

rigormortis

formalism in HCI

• blocks to uptake?– expertise– utility– tools– cost– fear

• use only where value exceeds cost– safety critical, ?generic widgets?

ubiquity

• lots of simple devices

• but complex context:– lots of users– mobile– context dependent behaviour

• good for FM

ubiquity

simple devices complex context

good for formal methods!

models

discrete

• most common• good for GUI• familiar notations

continuous

• very few• VR, ubicomp• simpler semantics

what do they mean?

notationfever!

pattern

notations

• lots of them

• why?– cult of notation in computer science– differing paradigms and problems– tractability– familiarity

myriad notations

• problems– communication– education– tools– meaning and satisfaction

?? moving to ubiquity ??

routes

myriadnotations

interoperability UML-ishnesssingle notationone size fits all

translation sharedsemanitics

transformation

model 1 model 2(notation A) (notation B)

Galois connection

common semantics?

model 1 model 2

complete meta-semantics

model 1 model 2

shared sub-semantics

common semantics?

model 1 model 2

external phenomena

trace / scenario

issues

advantages

• obvious• physical/real• common• relates to real time

problems

• not obvious– e.g. CCS/CSP

• internal structure• binding to real• granularity/instants• real time

meta-model

a b c

x

Z

Y*

model

time

happenings

instances

a

b

c

xa

b

xa

x

Y

event stream

P a b P | a c P

time

model

a – b – a – b – a – c – a – b … instances

< < < < < < <happenings

turntaking & states

somestate

otherstate

actionresponse

s0 – a1 – r1 – s1 – a2 – r2 – s2 – …

one happening or two?

mapping

< < < < < <

model 1

model 2

< < < < <

P a b P | a c P

a b c

x

Z

Y*

<

<

NO?

interstices

status and events

S1 S2

A

A

S1 – A – S2 – A – S1 – A – S2 ….

hierarchies – task analysis

1.prepare

2.1introduction

2.2show slide

2.3conclusions

2.talk

3.1question

3.2answer

3.discussion

0.DSV-IS talk

1. prepare

time

2. talk

2.1 intro 2.2 sld 1 2.2 sld 1 2.3 conc

3. discussion

3.1 Q 3.2 A 3.1 Q 3.2 A

0. DSV-IS talk

layers – UI events

lexical level

dialogue level

application level

mouse activity on menu

underline is ‘word’

font size menu

size is 10subscript

clickradio btn click OK

finished!

change selection format

NoneWordSingleDoubleDotted

NoneWordSingleDoubleDotted

continuous

• status–event analysis

• CS hybrid system literature

• TACIT – hybrid Petri Nets& continuous interactors

• Wuther???? – systems theory

common features

• actions– at events, discrete changes in state

• interstitial behaviour– between events, continuous change

other continuous issues

• granularity issues – do it today

• two timing – infinitely fast times

• temporal gestalt – words, gestures

summary

• harder than it looks!

… but easier than pairwise matching

• consequences:less sloppy semantics

clear semantics shared toolsfoundation …