media x – dnp visit · 5 December 2006

stanford hci group / 2006

http://hci.stanford.edumedia x – dnp visit· 5 December 2006

Designing Sensor-based Interactions by Example

Scott Klemmer, Björn Hartmann, Leith Abdulla, Manas Mittal

2

[O’Sullivan]

3

Tools for Exploration

START

???

B

?

A??

4

d.tools

5

Design Video

6

Current Work: Exemplar

PROTOTYPE APPLICATION LOGIC

SPECIFY RELATIONSHIP BETWEEN SENSOR DATA AND APPLICATION LOGIC

PROVIDE SOFTWARE ABSTRACTIONFOR HARDWARE

SPECIFY RELATIONSHIP BETWEEN SENSOR DATA AND APPLICATION LOGIC

(with Leith Abdulla, Scott R. Klemmer, Manas Mittal)

Current Tools

8

Idea: Programming by Demonstration

9

Crux: Generalization

exemplar

11

First Video

12

Exemplar UI

13

Exemplar UI

14

Exemplar UI

15

Exemplar UI

16

Exemplar UI

17

Exemplar UI

18

Exemplar + d.tools

19

Recognition I: Thresholds

20

Recognition I: Thresholds

21

Editing & Generalizing

22

Editing & Generalizing

23

Recognition II: Patterns

24

Editing & Generalizing

25

Editing & Generalizing

26

Extensibility Multiple sensors,

multiple events

User-written filters

Continuous output

[more: http://hci.stanford.edu/exemplar/documents/exemplar.pdf]

Evaluation

Eval Video

29

Future Directions Improve pattern matching performance

with Hidden Markov Models (HMMs)

Generate and export high level specificationsfrom Exemplar (pseudo or compilable code)

Bricolage programming:Combine sensing with web services

stanford hci group / 2006

http://hci.stanford.edu

http://hci.stanford.edu/exemplar

Examples

33

Examples

Examples