11-4-05 Kishore Ramachandran - Dfuse

8/2/2019 11-4-05 Kishore Ramachandran - Dfuse

1/56

DFuse and MediaBroker:System support for sensor-based

distributed computing

Kishore Ramachandran

(http://www.cc.gatech.edu/~rama)

College of Computing

Georgia Tech

Colleagues:Rajnish Kumar, Bikash Agarwalla,Junsuk Shin,

David Hilley, Dave Lillethun, Jin Nakazawa, Bin Liu,Xiang Song, Nova Ahmed, Seth Horrigan,

Matt Wolenetz, Arnab Paul, Sameer Adhikari,Ilya Bagrak, Martin Modahl, Phil Hutto
http://www.cc.gatech.edu/~ramahttp://www.cc.gatech.edu/~rama


2/56

Computing/Communication Continuum

Sensor

Network

High connectivity Low connectivity / Wireless

Cameras, sensor nodes High Performance Computing (HPC) resources

HPC

resources

Ambient Computing Infrastructure


3/56

What does this enable?

Application context distributed sensors with varying capabilities

control loop involving sensors, actuators

rapid response time at computational perception speeds

Sample applications Video based surveillance

Transportation

Emergency Response

Collaborative search and rescue Evacuation management

Aware Environments


4/56

Application Characteristics

Physically distributed heterogeneous devices Interfacing and integrating with the physical

environment

Diverse stream types (low to high BW)

Diverse computation, communication and powercapabilities (from embedded sensors to clusters)

Stream fusion/transformation, with loadable code

Resource scarcities

Dynamicjoin/leave of application components


5/56

Key Requirements

Middleware Programming infrastructure

Distributed data fusion

Stream data management

Network Level Protocol stack

Energy efficient routing

Ambient HPC resources Grid

Stampede

DFuseMediaBroker

SensorStack

Streamline


6/56

Stampede (TPDS 2003)

Distributed programming system coveringthe continuum Temporal stream data transport Multilingual (C, C++, Java) program

components sharing data abstractions Multiple platforms (x86-Linux, ARM-Linux,

x86-Windows, x86-Solaris, Alpha-Tru64)

Channel

thread

many to many connections

time sequenced data

correlation of streams

automatic GC

put(ts, item)

get(ts, item)

consume(ts)


7/56

Key Requirements







Stampede

DFuseMediaBroker

SensorStack

Streamline


8/56

DFuse (ACM SenSys2003)

Future Sensor Networks Todays handhelds, gateways Ubiquitous high-bw sensors

Challenges Overlaying the application onto the physical network

Programming abstraction for data fusion

Fusion Channel(a Virtual Sensor)

Producers

(sensors orother fusionchannels)

Consumers

(actuators orother fusionchannels)

.

..

.

..f()

SinkFilter

Collage

Cameras


9/56

Fusion Module

Placement Module

SinkFilter

Collage

Cameras

Fusion( Arg[ ]){

..

}

Resource Monitor,

Routing Layer Interface

Operating System

Hardware

Fusion Module

Placement Module

Resource Monitor,

Routing Layer Interface

Operating System

Hardware

Cost

Function

DFuse functions:1. Placement of fusion and relay points2. Plumbing as required

3. Dynamic migration of fusion points


10/56

Status of DFuse

Fusion and placement modules implementedon top of Stampede A prototype iPAQ farm (simulating future

sensor network) runs DFuse

Stampede and DFuse available for downloads MSSN, a simulator for sensor networks

middleware design guidance (BaseNets04,IJNM05, Wolenetzs thesis) Available for downloads


11/56

Key Requirements







Stampede

DFuseMediaBroker

SensorStack

Streamline


12/56

MediaBroker

An architecture for stream management

A clearing house for sensors and actuators in agiven space

Stream registry, discovery, plumbing, sharing,

Dynamic connection of sources (producers) and

sinks (consumers) Dynamic injection, and safe execution of

transformation codeFeature extraction, fusion

Dynamic sharing of transformations andstreams


13/56

Elements Type server: stores data types, relationships, and

transformation code Transformation engine: allow safe execution of injected

code on cluster nodes

Scheduler: manages workload, and allows prioritizingtransformation requests

Data brokers: manages connections between producers andconsumers

Type Server

DataBroker

DataBroker

Scheduler

TransformationEngine



ProducerProducer Consumer Consumer

Data ItemsTransformation RequestsTransformation Code


14/56

What it enables

Dynamic instantiations and sharing oftransformations


15/56

MediaBroker Status

MediaBroker V.1 A subset of the functionalities

Application example: Family intercom

IEEE PerCom 2004, PMC 2005

MediaBroker++ Currently under development

EventWeb built on top


16/56

Key Requirements







Stampede

DFuseMediaBroker

SensorStack

Streamline


17/56

SensorStack

Adaptability Vs. Stackability of protocollayers Adaptability deals with cross-layer data

A must for wireless sensor networks

Stackability deals with cross-layerfunctionalitiesA must for modular design

Principle behind SensorStack

Decouple data from functionalities


18/56

SensorStack withoutCross-layering Support

AODV routingFlood routing

MAC

Application

Time SyncService

Fusion layer

Fusion datatransmissionrequirement

Data periodicity,

Size, delay tolerance

NeighborhoodInformation,Topology

Link qualityinformation,Neighborhood statuschange, topology

Datatransmissionrequirement

Time synchronization accuracy,accuracy requirement

Fusion requirement


19/56

Information Exchange Service (IES)

Design Goals:

1. Efficient use of limited memory

2. Simplifying information sharing interface

3. Extensibility

4. Asynchronous delivery of the information

5. Complex event notification


20/56

IES Design

Data management module Stackability by using standard data interface

Publish/subscribe based shared memory system

Fully-associative cache for performance

Event management module Adaptability by notifying when to adapt

Complex event notification

Reactive memory access


21/56

DatePublisher

DataSubscriber

Event

Subscriber

Publisher

List

Subscriber

List

Shared

Memory

CacheDRE DAE

EMMRSE

Set rules

getput

DRE: Data request event RSE: Rule satisfied eventDAE: Data available event EMM: Event management module


22/56

SensorStack withCross-layering support

Application

Data Fusion Layer

Data Service Layer

Medium Access Layer

InformationExchangeService

HelperServiceLayer

Radio

Application Logic\

In-stack fusion

Next-hop selection,Logical naming, Packet

scatter/gather

Medium Access, ErrorControl, Radio Control

Attribute-Value publish/

subscribe

Locali-zation,

Synchro-nization

Service

Connection

(A) Stack Lay-out (B) Functionalities

Implemented in TinyOS and iPAQ LinuxInitial results (increasing application lifetime) very promising


23/56

Application Lifetime Improvementwith Cross-layer Information

DFuse performance withoutCross-layer information

DFuse performance withCross-layer information forrole-migration


24/56

Key Requirements







Stampede

DFuseMediaBroker

SensorStack

Streamline


25/56

C

V

CCameraP/T/Z

Recorder

CCamera Humidity

Pressure

Proximity

Cluster

Cluster

Cluster

Grid Infrastructure

Gateway

Gateway

Gateway

Gateway

MediaBroker

DFuse

Stampede

SensorStack


26/56

Streamline (MMNC 2006)

Scheduling problem

Input: Computation and communication requirements of various

stages of a coarse-grain dataflow graph

Application-specified constraints

Current resource (processing and bandwidth) availability Resource specific constraints

Output: Placement of the stages of the pipeline on available HPC

resources Performance criteria:

latency and throughput of the application


27/56

Streamline Scheduler

Expects to maximize throughput by assigning best

resource to most needy stage Additional policies concerning resources,

applications, and local schedulers can beincorporated in the cost of a particular assignment

ApplicationPolicies

S0

S1

S2 S3

Resource Filtering

Resource Selection

Stage Prioritization

{S2 S0 S1 S3}

Resource

Policies{S2{R0R2R3}}

{S2R0}

R0

R1

R2

R3


28/56

Streamline System Architecture

Results Outperforms condor

by an order ofmagnitude for bothcompute andcommunication bound

kernels,particularlyunder non-uniform loadconditions

Performs close toSimulated Annealing

but at considerablylow scheduling time (bya factor of 1000)

DataSources

ResourceInformation

Service

QoS

MonitoringService

ApplicationInformation

Service

GridBoundary

Streaming Application

HPC

Resource

GRAM

HPCResource

GRAM

HPC

Resource

GRAM

StreamingAppication

Application

Policies

SchedulingService

AuthenticationService


29/56

Streaming Grid

Streamline schedulerintegrated into Globustoolkit

Example of a mock trafficmonitoring app as aservice composition usingWeb Services

Blue boxes are theStreaming Grid services

D


30/56

Demo


31/56

Demo Output (live if stars arealigned!)

http://www.cc.gatech.edu/~bikash/sgrid/trafficapp/http://www.cc.gatech.edu/~bikash/sgrid/trafficapp/

trafficapp.html

What is the takeaway? Several technologies working together Service composition, Streamline scheduling, Web

Services, Globus toolkit to process the video stream andshow the output in the browser Streaming grid instantiates, connect, manages the

streaming app
http://www.cc.gatech.edu/~bikash/sgrid/trafficapp/http://www.cc.gatech.edu/~bikash/sgrid/trafficapp/trafficapp.htmlhttp://www.cc.gatech.edu/~bikash/sgrid/trafficapp/trafficapp.htmlhttp://www.cc.gatech.edu/~bikash/sgrid/trafficapp/trafficapp.htmlhttp://www.cc.gatech.edu/~bikash/sgrid/trafficapp/trafficapp.htmlhttp://www.cc.gatech.edu/~bikash/sgrid/trafficapp/


32/56

Computing/Communication Continuum

Sensor

Network

High connectivity Low connectivity / Wireless

Cameras, sensor nodes High Performance Computing (HPC) resources

HPCresources

Ambient Computing Infrastructure


33/56

Conclusions

MediaBroker stream transformation and typed transport

engine

DFuse data fusion architecture

Stampede distributed programming environment

SensorStack Information Exchange Service for cross-

layer support

Streamline Scheduling support for streaming apps on

grid

MediaBroker

DFuse

Stampede

SensorStack

Streamline


34/56

Ongoing Work

Programming tools Adaptive resource management

Marrying grid computing and ubiquitouscomputing

Wireless networking considerations sensorstack

energy efficient protocols

mobility considerations


35/56

Web Links

http://www.cc.gatech.edu/~rama http://www.cc.gatech.edu/~rama/stampede

http://www.cc.gatech.edu/~rama/up

http://www.cercs.gatech.edu/
http://www.cc.gatech.edu/~ramahttp://www.cc.gatech.edu/~rama/stampedehttp://www.cc.gatech.edu/~rama/uphttp://www.cercs.gatech.edu/http://www.cercs.gatech.edu/http://www.cc.gatech.edu/~rama/uphttp://www.cc.gatech.edu/~rama/stampedehttp://www.cc.gatech.edu/~rama


36/56

Applause!!!


37/56


38/56

Sensor Networks

Current Sensor Network Nodes

Limited capabilities (Mica-2) habitat monitoring, vineyard

monitoring

Recent trends iMotes

8x radio, 4x CPU, power++

Telos motes>3x radio, similar CPU & power

Future Sensor Network Nodes Todays handhelds, gateways Ubiquitous high-bw sensors Source: CACM #47-6 The platforms enabling wireless

sensor networks, Hill, Horton, Kling, Krishnamurthy, 2004.


39/56

HPC resources

Unix / Linux / XP cluster


40/56

Fusion Applications : need hierarchical fusion support

Overlaying the fusion graph

collage

filter

uncompress

F il I t


41/56

Family Intercom Clients connected via

MediaBroker

Type attributes includeaudio rate and bufferspecs

A client trackingsystem used for I-combo selection

Colocated clients canperform mixing forN-way conferencing


42/56


43/56

Fusion Module

Structure Management

Producers Consumers

.

..

.

..

Plumbing Issues


44/56

Fusion Module

Computation Management

Dynamic embedding of user-specified fusion function

Correlation and aggregation of input streams

Fusion channel migration

.

..

.

..f()f1()

f2()


45/56

Fusion Module

Computation Management

Dynamic embedding of user-specified fusion function

Correlation and aggregation of input streams

Fusion channel migration

Memory Management

.

..

.

..f()f1()

f2()


46/56

Fusion Module

Error and Status management

.

..

.

..f()f1()

f2()

Failure / Latency hiding


47/56

Placement Module

User inputs the task graph

Sink (Display)Filter

Collage

Sources

S1

S2

S3

And, a cost function.


48/56

Simple Solution ?

Why not push the fusion points towards itssources ?

Data sources may be lying all around

Fusion points may cause data expansion

Network is dynamic


49/56

DFuse: Placement Modules Algorithm

Three phases:1. Nave role assignment

Deploy task graph into the network

Start app at a designated root node and delegate taskgraph subtree roles to richest neighbors recursively

2.

Optimization Given anticipated application behavior (attributes in thetask graph), perform rapid local decisions to adjust whichnode performs which role.

Decisions guided by application-provided cost-function

3. Maintenance Monitor actual application behavior and perform less

frequent optimizations given application-provided cost-function

E l


50/56

Example Cost Function

MT1 (Minimize Transmission Cost-1)

Source

Source

f() Sink

2 Kbps

2 Kbps

2 Kbps

2 Kbps

1 Kbps

n1 n2

CMT1 ( n1, f ) = 6 kbps

CMT1 ( n2, f ) = 9 kbps

DF F


51/56

DFuse: Cost Functions MT1 (Minimize Transmission Cost 1)

Used in intuitive illustration slides minput data sources (fan-in) noutput data consumers (fan-out)

MPV (Minimize Power Variance) Attempts to keep the power of

network nodes at similar levels.

MTP (Minimize Ratio of TransmissionCost to Power) Attempts to keep the time for how

long nodes can run a fusion functionsimilar.

MT2 (Minimize Transmission Cost 2) Like MT1, but allows role transfer

when node power is below athreshold.

c(k,f)= cost of node kto perform role ft(x)= transmission rate of data source xhopCount(i,k)= network hops between nodes iand k

power(k)= remaining power at node k

P DF l


52/56

Prototype DFuse Implementation

Goal Investigate utility of Fusion Point Migration

Hypothesis Migration will increase application lifetime, for constant

QoS

Implementation Fusion Module: ARM Stampede port Simulated placement module Interface for coupling, transmission monitoring

Simple camera sources, fusion functions and sink

Collage

Filter


53/56

Experimental Setup 12 iPAQ 3870s in a familiar 0.6.1 Linux basedD-

Stampede 802.11b farm. Only directly adjacent iPAQsin the figure below are considered mutually reachable in1 hop.

Placement module run as a simulation of the distributedalgorithm coupled to the farm via an extended fusionmodule interface

Power usage is modeled to be linear to the number ofapplication-level bytes transmitted across a farm hop(simple model)

Prototype DFuse Results:


54/56

rototyp DFus su tsTransmission Cost over Time

Prototype DFuse Results:


55/56

yp FVariance, Role Transfers, Lifetime

MPV vs MT2

4x lessvariance

Migrations++

70% lifetime

MTP

good variance

good lifetime

h d k


56/56

Pervasive Computing with MediaBroker

Family Intercom and Sign Post Being developed at Georgia Techs Aware Home

Replacing a legacy hardware mixing system with a much morescalable system

Integration with existing RFID and Vision based tracking

system called Sign Post to enable rapid call dispatching andmobility-aware communication

11-4-05 Kishore Ramachandran - Dfuse

Documents