3/14/2005 CS851: Data Services for Ad vanced System Applications 1 DFuse: A Framework for Distributed Data Fusion Rajnish Kumar, Matthew Wolenetz, Bikash Agarwalla, JunSuk Shin, Phillip Hutto, Arnab Paul, and Umakishore Ramachandran Georgia Institute of Technology Presented by Lei Fang, Mar 14, 2005
57
Embed
3/14/2005 CS851: Data Services for Advanced System Applications 1 DFuse: A Framework for Distributed Data Fusion Rajnish Kumar, Matthew Wolenetz, Bikash.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
3/14/2005 CS851: Data Services for Advanced System Applications
1
DFuse: A Framework for
Distributed Data Fusion
Rajnish Kumar, Matthew Wolenetz, Bikash Agarwalla, JunSuk Shin, Phillip Hutto,
Arnab Paul, and Umakishore RamachandranGeorgia Institute of Technology
Presented by Lei Fang, Mar 14, 2005
3/14/2005 2CS851: Data Services for Advanced System Applications
3/14/2005 9CS851: Data Services for Advanced System Applications
DFuseDFuse
Fusion API: Help to develop Fusion API: Help to develop distributed sensor fusion distributed sensor fusion applicationsapplications
Fusion Placement: Support Fusion Placement: Support automatic management of fusion automatic management of fusion point placement and migrationpoint placement and migration
3/14/2005 10CS851: Data Services for Advanced System Applications
3/14/2005 11CS851: Data Services for Advanced System Applications
DFuse Architecture - Input
Target applicationsTarget applications HierarchicalHierarchical fusion functions on fusion functions on time time
sequencedsequenced data items data items Input Input
Task GraphTask Graph• Vertex: Vertex: source, sink, or immediate fusion points• Assume data sources are known at query time
Fusion Function• Transform a correlated set of inputs to a “fused” output item• May contract or expand the streaming data• Each fusion point performs one fusion function
3/14/2005 12CS851: Data Services for Advanced System Applications
DFuse Architecture - Components
Data Fusion Module Data Fusion Module Implement fusion API: Implement fusion API: allows any
synthesis operation to on stream of data to be specified as a fusion function
Fusion Placement Module Fusion Placement Module Determine the best placement of fusion
functions
3/14/2005 13CS851: Data Services for Advanced System Applications
DFuse Architecture
3/14/2005 14CS851: Data Services for Advanced System Applications
DFuse Architecture – Working principles
Fusion Module provides fusion API which Fusion Module provides fusion API which allows any synthesis operation to on stream of data to be specified as a fusion function. The user inputs the task graph and fusion functions
to DFuse at some designated node (root node)
The placement module at the root node maps the fusion functions onto the physical network Subject to some cost function Periodically re-evaluate the health of fusion point
nodes and their neighbors, and moves the fusion functions
3/14/2005 15CS851: Data Services for Advanced System Applications
3/14/2005 17CS851: Data Services for Advanced System Applications
DFuse – Fusion Module
Fusion API CapabilitiesFusion API Capabilities Structure ManagementStructure Management
• Fusion Channel – Abstracts inputs and Fusion Channel – Abstracts inputs and encapsulates a general fusion function encapsulates a general fusion function (filter, collage, …) (filter, collage, …)
• provide data buffering and synchronization provide data buffering and synchronization facilities facilities
• E.g. items are fused and accessed by E.g. items are fused and accessed by timestamptimestamp
3/14/2005 18CS851: Data Services for Advanced System Applications
DFuse – Fusion Module
Fusion APIFusion API Correlation controlCorrelation control
• Specify and collect correlation sets (related Specify and collect correlation sets (related input items supplied to the fusion function)input items supplied to the fusion function)
Computation managementComputation management• Handle the specification, application and Handle the specification, application and
migration of fusion functionsmigration of fusion functions Memory managementMemory management
• Caching, prefetching, and buffer Caching, prefetching, and buffer managementmanagement
3/14/2005 19CS851: Data Services for Advanced System Applications
DFuse – Fusion Module
Fusion APIFusion API Failure/latency handlingFailure/latency handling
• Allow partial fusionAllow partial fusion• Accept a variable number of input itemsAccept a variable number of input items
Status and feedback handlingStatus and feedback handling• Allow interaction between fusion functions Allow interaction between fusion functions
and data sourcesand data sources• Supply Supply status informationstatus information via “status” register via “status” register• Support Support a command seta command set via “command” via “command”
registerregister
3/14/2005 20CS851: Data Services for Advanced System Applications
3/14/2005 22CS851: Data Services for Advanced System Applications
Placement Module – Role Assignment
Distributed role assignment algorithmsDistributed role assignment algorithms Map a fusion function to a network nodeMap a fusion function to a network node Three roles: end point, relay, fusion pointThree roles: end point, relay, fusion point
3/14/2005 23CS851: Data Services for Advanced System Applications
Placement Module – Role Assignment
Role Assignment HeuristicRole Assignment Heuristic Phase 1: Naive role assignment
• Start at root node• Algorithm: Naive Tree Building
Upon receiving the application task graph:if (I can host the root fusion function) { accept the root fusion function; for each input {
delegate subtree to “richest” neighbors; }else delegate entire tree to the “richest” neighbor;
3/14/2005 24CS851: Data Services for Advanced System Applications
Placement Module – Role Assignment
Role Assignment HeuristicRole Assignment Heuristic Phase 2: Optimization (Role Transfer)
Improve overall health of the overlay network
• Health is valued by cost function (power, data transfer rate)• Each fusion node periodically broadcasts its health• If a neighboring node has better health, it informs the fusion node• The fusion node transfer fusion task to the neighbor with best
health
Phase 3: Maintenance (application is running)
3/14/2005 25CS851: Data Services for Advanced System Applications
Placement Module – Sample Cost Functions
The lower the cost function value, the better the node health!
Minimize transmission cost 1 (MT1):•
Minimize power variance (MPV):• CMPV( k ) = 1/power ( k )
Minimize Transmission cost vs. power (MTP):• CMTP ( k, f ) = CMT1 ( k, f ) * CMPV ( k )
Minimize transmission cost 2 (MT2):• CMT2( k, f ) = ( power ( k ) > threshold ? (CMT1 ( k, f ) :
Infinity )
How to transfer the role ?
3/14/2005 26CS851: Data Services for Advanced System Applications
Placement Module – Local Minima
Two simple metrics Simple hop-count Fusion data expansion or contraction information
Two types of optimization Linear: All inputs to a fusion node are coming from one
relay node Triangular: Multiple paths for inputs to reach a fusion
point
3/14/2005 27CS851: Data Services for Advanced System Applications
Placement Module –Optimization
Linear Optimization
1000
1000
1000 1000
1500
SourceRelay Fusion Point(data contraction)
Sink
1000
1000
1500 1500
SourceFusion Point
Relay Sink
1000
1000
1500 1500
SourceFusion Point(data expansion)
Relay Sink
1000
1000
1000 1000
1500
SourceRelay Fusion Point Sink
3/14/2005 28CS851: Data Services for Advanced System Applications
Placement Module –Optimization
Triangular Optimization
1000
1000
1000
1500
fp1
Relay
fp3(data contraction)
fp4
fp2
1000
1000
1000
1500
fp1
fp3
fp2
3/14/2005 29CS851: Data Services for Advanced System Applications
3/14/2005 51CS851: Data Services for Advanced System Applications
ConclusionsConclusions
Future sensor networks will support data fusion applications
DFuse provides fusion API to ease the development such applications
Distributed fusion placement increases application runtime by dynamic role assignment
3/14/2005 52CS851: Data Services for Advanced System Applications
CritiquesCritiques Inflexible:
• DFuse can’t be implemented in current sensor network• Assume data sources are known before• Stationary sources and sinks• Complicated, resource demand• Assume every two node in the network is reachable• Migrate a particular fusion channel is a local decision
and only immediate neighbor nodes are queried to see if their heath for hosting the fusion channel would be better.
Insufficient evaluation:• Only consider network communication cost when Only consider network communication cost when
evaluate power consumptionevaluate power consumption• Implemented on iPAQ farm, but most of the work via
event-based simulation on desktop
3/14/2005 53CS851: Data Services for Advanced System Applications
Related workRelated work--- Data Fusion --- Data Fusion
InterfaceInterface TAG (Tiny Aggregation): TAG (Tiny Aggregation): PProvides a simple SQL-like declarative language for expressing
aggregation queries over streaming sensor data. Intelligently distributes and executes aggregation queries in the sensor
network in a time and power-efficient manner. Example:
Difference between Tag query and SQL query Output of a Tag query is a stream of values Record : <group id, aggregate value>. Each group is time-stamped and the readings
used to compute an aggregate record all belong to the same time interval, or epoch.
SELECT AVG(volume),room FROM sensorsWHERE floor = 6GROUP BY roomHAVING AVG(volume) > thresholdEPOCH DURATION 30s
SELECT (agg(expr), attrs) FROM sensorsWHERE {selPreds}GROUP BY {attrs}HAVING {havingPreds}EPOCH DURATION i
3/14/2005 54CS851: Data Services for Advanced System Applications
Related workRelated work--- Data Fusion --- Data Fusion
InterfaceInterface TAG (Tiny Aggregation): TAG (Tiny Aggregation): Structure of Data aggregationStructure of Data aggregation
• Three pieces:– Merging function f– Initializer i– Evaluator e
• General form: <z> = f(<x>,<y>)– <x>, <y>, <z> are multi-valued partial state record– Example:
f- for Average Each partial state record consists of <SUM, COUNT>
f(<S1, C1>,<S2,C2>) = <S1+S2, C1+C2>i- How to initiate a state record for a single sensor value
Eg. i(x) = <x,1> :Average over a sensor value of xe- Compute the actual value of aggregation e(<S,C>)
Pros:Pros:• Proven to give more accurate results then centralized approach• In-network aggregation: Save powerIn-network aggregation: Save power• SimpleSimple