Towards User-defined Cross-Device Interaction

Towards User-defined Cross-

Device InteractionAUDREY SANCTORUM & BEAT SIGNER

VRIJE UNIVERSITEIT BRUSSEL

0

1

2

3

4

5 or more

0% 5% 10% 15% 20% 25% 30%

How many connected devices do people use?

Japan Spain USA Switzerland BelgiumPercentage of users

Num

ber o

f dev

ices

The Connected Consumer Survey 2014/2015Google Inc.

65%

Conductor (2014)

HuddleLamp (2014)

Connichiwa (2015)

Panelrama (2014)

MultiSpace (2006)

ReticularSpaces (2012)

Existing Classifications

"The 4C Reference Model for Distributed User Interfaces" by Demeure et al.• computation, configuration, communication and coordination

"Distributed User Interfaces: State of the Art"by Niklas Elmqvist• input, output, platform, space and time

UI + data + UI elements

UI + data

Table / Camera Room Network connection

to the server Anywhere

HuddleLamp

MultiSpace

ReticularSpaces

Panelrama

Conductor

Connichiwa

Granularity of distribution

Location constraint

UI + data + UI elements

UI + data

Table / Camera Room Network connection

to the server Anywhere

HuddleLamp

Airlift

MultiSpace

ReticularSpaces

ARIS

GroupTogether

iLand

iRoom

Panelrama

Conductor

IMPROMPTU

THAW

Deep Shot

Weave

Connichiwa

XDStudio

WebSplitter

Melchior et al. (2009)

CAMELEON-RT

Granularity of distribution

Location constraint

Robertson et al. (1996)

Frosini et al. (2013)MultiMasher

Pick-and-Drop

End Users

How to allow end users to define customised cross-device interactions?

How much control will end users have in terms of the granularity of the UI components to be distributed?

Will end users be limited by a specific location, space or office setting?

Will end users be able to share their configuration of customised DUIs?

Can end users reuse parts of other configurations?

What will end users be able to modify?

…

<?xml version="1.0"encoding="UTF-8"?> <activeComponents><activeComponent> <identifier>Sound< /identifier> <logic>…</logic></activeComponent></activeComponents>

Active Component

Architecture

ActiveComponents

Resource/Selector Plug-ins

Developer RegistryCore

RSL

User Profile

End-User RegistryConfiguration

Pool

User Developer

Button 1

Scenarios

Swipe AC

Double Swipe AC

Triple Swipe AC

Direction AC

Swipe Area

TABLET

Sound AC

TV Runtime environment

Runtime environment

...Gesture AC

Play AC

Data Transfer AC

Data Transfer AC

Proposed Approach

[28] Signer and Norrie As We May Link: A General Metamodel for Hypermedia Systems.(2007)

Authoring Rather Than Programming

Swipe GestureDevice: Tablet

AreaDevice: Tablet

Data TransferSDevice:

TabletTDevice: TV

PlayDevice: TV

SoundDevice: TVVolume: 80%

_ X Components

Data Transfer +Play +Sound +Swipe +

Area +

Double Swipe +Triple Swipe +Direction

+... +

Conclusion•Classification of DUI systems•User-defined cross-device interactions• Linking UI components and application logic• RSL hypermedia metamodel• Arbitrary level of granularity • Sharing of user-defined interactions

Takeaway Message

Hypermedia based DUI development

Authoring rather than programming approach

ReferenceSanctorum A. and Signer B. :Towards User-defined Cross-Device Interaction. In the Workshop on Distributed User Interfaces. Lugano, Switzerland (2016).