Composite Device Computing Environment: A Framework for Situated Interaction Using Small Screen Devices Thai-Lai Pham, Georg Schneider, Stuart Goose and.

Composite Device Computing Composite Device Computing Environment: A Framework for Situated Environment: A Framework for Situated Interaction Using Small Screen DevicesInteraction Using Small Screen Devices

Thai-Lai Pham, Georg Schneider, Stuart Goose and Arturo PizanoThai-Lai Pham, Georg Schneider, Stuart Goose and Arturo Pizano

A Multimedia Systems Presentation byA Multimedia Systems Presentation byModupe OmuetiModupe Omueti

Spring 2005Spring 2005

IntroductionIntroduction MotivationMotivation Implementation Implementation Current statusCurrent status Challenges and future outlookChallenges and future outlook

SchemaSchema

IntroductionIntroduction

Mobile and wireless computingMobile and wireless computingInformation access and communication across Information access and communication across

a diverse range of environments through a diverse range of environments through traditional computing systems such as WWW traditional computing systems such as WWW and Internet.and Internet.

Ubiquitous computingUbiquitous computing““everywhere computing not a desktop-centric everywhere computing not a desktop-centric

system rather a human-centric system”.system rather a human-centric system”.ContextContext

Set of environmental states and settings that Set of environmental states and settings that determine an application’s behaviour.determine an application’s behaviour.

IntroductionIntroduction

Examples of context-aware systems include Examples of context-aware systems include the Cyberguide, Active Badge System.the Cyberguide, Active Badge System.

Situated ComputingSituated Computing Methodology imbuing applications and services Methodology imbuing applications and services

with more personal and appropriate behaviour.with more personal and appropriate behaviour. Small Screen DevicesSmall Screen Devices

These are contemporary communication These are contemporary communication devices that have become our “everyday devices that have become our “everyday companion” e.g. Personal Digital Assistants companion” e.g. Personal Digital Assistants (PDAs), Symbian Mobile Phones.(PDAs), Symbian Mobile Phones.

MotivationMotivation A dramatic growth in the small screen devices A dramatic growth in the small screen devices

market.market. Desire of users to browse rich multimedia Desire of users to browse rich multimedia

information and diverse services while mobile.information and diverse services while mobile. Size limitation of the small screen devices in Size limitation of the small screen devices in

rich content presentation.rich content presentation. Places limit on legibility of text and image display.Places limit on legibility of text and image display. Graphics and multi-media originally intended for Graphics and multi-media originally intended for

desktop PCs are normally not suitable for small desktop PCs are normally not suitable for small screen devices.screen devices.

MotivationMotivation

A lot of hard disk and RAM space is required for good A lot of hard disk and RAM space is required for good performance and storage of multimedia programs.performance and storage of multimedia programs.

Efforts focused on producing explicitly one service or Efforts focused on producing explicitly one service or shrink the content to match the capabilities of the shrink the content to match the capabilities of the small screen client.small screen client.

Other factors determining the design of the small Other factors determining the design of the small screen client devices can be improved on but the screen client devices can be improved on but the small physical screen size will remain for a long time.small physical screen size will remain for a long time.

The environment is becoming more and more The environment is becoming more and more equipped with several variety of computing equipped with several variety of computing resources.resources.

MotivationMotivation

So why not make interactive use of So why not make interactive use of the potential multitude of computing the potential multitude of computing resources available to us in our resources available to us in our immediate environment for diverse immediate environment for diverse application and servicesapplication and services

Composite Device Computing Composite Device Computing Environment (CDCE)Environment (CDCE)

This is the framework that exploits This is the framework that exploits and incorporates surrounding and incorporates surrounding available resources as another facet available resources as another facet of situated computing to overcome of situated computing to overcome the physical constraints of small the physical constraints of small screen devices.screen devices.

Composite Device Computing Composite Device Computing Environment (CDCE)Environment (CDCE)

Makes a pool from surrounding available Makes a pool from surrounding available computing resources.computing resources.

The resources form it elements.The resources form it elements. Provides mechanisms for seamlessly Provides mechanisms for seamlessly

exploiting and interacting with elements.exploiting and interacting with elements. Combines location and mobility with static Combines location and mobility with static

computing resources.computing resources.

Composite Device Computing Composite Device Computing Environment (CDCE)Environment (CDCE)

Based on small screen client Based on small screen client request, request, dynamically creates a unified dynamically creates a unified composite composite or or virtual device from virtual device from pool.pool. This This composite composite device is composed device is composed of of an an appropriate mix for the appropriate mix for the request.request. Augments small screen device.Augments small screen device. Creates a mobile collaborative Creates a mobile collaborative working working environment.environment.

ImplementationImplementation

Working ScenarioWorking ScenarioConceptConceptDesign goalsDesign goalsFramework componentsFramework components

Mobile Healthcare ScenarioMobile Healthcare Scenario

Each doctor is equipped with a PDA affording wireless access to the hospital patient information system.

Doctor wishes to query a patient’s medical history, including symptoms, diagnoses, prescriptions and X-rays.

PDA first detects the composite elements in the room, for example a TV and telephone, using the infrared interface.

PDA submits the doctor’s request, details about the detected devices to the CDCE gateway server.

CDCE server routes the medical symptoms, diagnoses, and prescription information to the doctor’s PDA

The X-ray image is redirected via RF to the TV for viewing.

Mobile Healthcare ScenarioMobile Healthcare Scenario

CDCE ConceptCDCE Concept

Small screen client does not need to perform all tasks.

Tasks not suitable for the PDA are outsourced to more appropriate device(s).

Multi-tier communication infrastructure is a key unit of CDCE.

Support of multi-modal interaction across the range of available CDCE supported computing resources

Design GoalsDesign Goals

Standard interfaces such as HTML/HTTP, WML/WAP, IR/Bluetooth.

Additional functionality supported through the browser extension mechanisms.

Adaptation for exploiting an ever changing number and diverse range of available elements in the surrounding environment.

Optimization of information presentation and selection of output devices.

CDCE Framework CDCE Framework ComponentsComponents

Small screen clientSmall screen client CDCE Smart GatewayCDCE Smart Gateway Network communication modelNetwork communication model The “environment”The “environment”

Fig. 1. Elements of CDCE FrameworkFig. 1. Elements of CDCE Framework

Small screen clientSmall screen client Detects resources available in close Detects resources available in close

proximity.proximity. Reserves the resources.Reserves the resources. Sends request to Smart Gateway Server.Sends request to Smart Gateway Server.

CDCE Framework CDCE Framework ComponentsComponents

Smart Gateway ServerSmart Gateway Server Intelligently manages the pool of services

available to the users. Creates and manages a composite device

based on the small screen client request. Maps requests to services, and

corresponding output to appropriate nodes in the virtual device.

performs any dynamic conversion needed.

CDCE Framework CDCE Framework ComponentsComponents

Network Communication ModelNetwork Communication Model Manages the convergence of wireless and Manages the convergence of wireless and

wire line networks.wire line networks. Manages corresponding communication Manages corresponding communication

protocols.protocols.

CDCE Framework CDCE Framework ComponentsComponents

““The environment”The environment” Represents the dynamic pool of resources

available at the user’s current location. Varies considerably from location to

location.

CDCE Framework CDCE Framework ComponentsComponents

Current StatusCurrent Status

CDCE prototype consists of a PDA running Windows CE and multiple Windows NT workstations equipped with infrared serial interface adapters.

PDA and Smart Gateway communication realized using HTTP over both Cellular Digital Packet Data (CDCP) and WLANs.

Current StatusCurrent Status

Smart Gateway and output client communication currently achieved using a LAN.

Distributed Component Object Model Distributed Component Object Model (DCOM) used by Smart Gateway for (DCOM) used by Smart Gateway for remote invocation of processes without remote invocation of processes without requirement for propietary client coderequirement for propietary client code

How the prototype works PDA client detects NT workstationsPDA client detects NT workstations Sends request to Smart GatewaySends request to Smart Gateway Gateway responds by initiating requested Gateway responds by initiating requested

processes on the NT workstationsprocesses on the NT workstations

Current StatusCurrent Status

Challenges and Future Challenges and Future OutlookOutlook

Intelligent information management Intelligent information management and distributionand distributionContents adaptation Contents adaptation

SplittingSplittingConversionConversionFilteringFiltering

Smart delivery of informationSmart delivery of information

Mobile user interfaceMobile user interfaceModes of interaction with the Modes of interaction with the

“environment”“environment”AbdicationAbdicationCooperativeCooperativeExclusiveExclusive

Network ArchitectureNetwork ArchitecturePDA-Environment-Gateway-Environment PDA-Environment-Gateway-Environment

CommunicationCommunicationSecuritySecurity

Challenges and Future Challenges and Future OutlookOutlook

The EndThe End