This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
IQ-ECho: Middleware Principles for Real-time Interaction
Large-scale CollaborativeApplications on Heterogeneous
Systems:Terastream Services and Teragrid
Data Transport
Visualization
Caching, Recovery, Logging, Security
Real-timeCollaborationand Inspection
WirelessUsers andDisplays
Data Source(e.g., spallationneutron source)
InstrumentedTestbed/FacilityLocal
Users
ORNLORNL
InstrumentedTestbeds/Facilities(e.g., for spallation neutron source)
High End Users and Displays
Real-time Collaboration: Molecular Dynamics
Requirements: Multiple
collaborators explore common data space
Personalized views, with ability to annotate and manipulate
Real-time sharing of data, even between different representations
Undamaged
Damaged
Undamaged
vs.
W / and L /
interatomic spacing
L
W
L
W
• Mechanical Engineering
• Physics
• Chemistry
•Aerospace Engineering
FCC
Twinning Plane
IQ-ECho: Middleware Principles for Network-aware Collaboration
Adaptive Peer-to-Peer Data Exchange:
IQ-ECho: High performance events:– Event-based peer-to-peer streaming data communications -binary
data exchanges (PBIO) for interactive apps (steering, real-time collaboration, …)
– Source-based filtering: IQ-services deployed to meet required application QoS, i.e., by disposition of application-specific code into remote sites and underlying platform
– Dynamic quality attributes: coordinated adaptation of platform (e.g., communication protocols) and of interactive applications
– Protecting critical connections from large-data traffic
Adaptive Communication
Adapt what?
– Congestion windows + data ratesIssues:– Transport cannot delegate all adaptation choices to
applications and still be fair to the network– Applications cannot delegate all adaptation to the
transport without limiting their choices or incurring difficulties (e.g., QoS translation)
Goal: – provide a mechanism to allow effective application
adaptations while remaining network-friendly
Coordinated Adaptation
Use `quality attributes’ to share information across middleware/protocol - IQ-Services
`Coordination methods’ Services/Protocol to address:– Conflicting adaptations– Combined effect of adaptation that may lead to overreaction– Limited application adaptation granularity– Others, ...
Problems important in networks where (delay * bandwidth) is large:– cost of adaptation– delay before correction of mistakes
Middleware/Protocol Interactions
IQ-Services in Middleware:– Application-relevant data manipulation:
Using an Instrumented Protocol: IQ-RUDP extends Reliable UDP– TCP-friendly congestion control (LDA algorithm)– Exposes network performance metrics– Supports application-registered callbacks– Application-controlled adaptive reliability
Middleware Architecture
Evaluation of Coordinated Adaptation
How effective is coordination in two-layer adaptations?
– Metric is “smoothness” of delay over time– Evaluate three cases where coordination is necessary– Hold application traffic pattern constant, vary network
bandwidth• iperf used to generate background traffic
– Hold network bandwidth constant, vary application traffic
• Emulate content delivery server using MBONE trace– Drive adaptations using callbacks on error ratio
Example: Conflicting Adaptations
No Coordination– Transport unaware of adaptation– All packets sent regardless of priority– More unmarked packets delivered– Larger delay for marked packets
Coordination– Transport can drop non-priority packets– Better delay/jitter for high priority packets– Average delay improves due to spacing
Conflicting Adaptations
IQ-RUDP (on right) achieves lower avg delay
(emulation results)
Example: Metadata-based Filtering
Without IQ-ECho Level Adaptation
0
10
20
30
40
50
60
70
80
40 90 140 190 240
Time(s)
Fra
me
Ra
te(f
/s)
Message Deliver Rate(WithoutAdaptation)
Perturbation Introduced
IQ-RUDP (on right) achieves substantially higher frame rate (measured results)
Qi He and Karsten Schwan, “IQ-RUDP: Coordinating Application Adaptation with Network Transport”, High Performance Distributed Computing (HPDC-11), July 2002.
Matt Wolf, Zhongtang Cai, Weiyun Huang, Karsten Schwan, ``SmartPointers: Personalized Scientific Data Portals in Your Hand'', Supercomputing 2002.
Fabian Bustamante, Patrick Widener, Karsten Schwan, ``Scalable Directory Services Using Proactivity'', Supercomputing 2002.
Patrick Widener, Greg Eisenhauer, Karsten Schwan, and Fabián E. Bustamante, "Open Metadata Formats: Efficient XML-Based Communication for High Performance Computing", Cluster Computing: The Journal of Networks, Software Tools, and Applications, 2003.
Greg Eisenhauer, Fabián Bustamante and Karsten Schwan, "Native Data Representation: An Efficient Wire Format for High-Performance Computing", IEEE Transactions on Parallel and Distributed Systems, 2003.