Network Computing Laboratory iOverlay: A Lightweight Middleware Infrastructure for Overlay Application Implementations Baochun Li, Jiang Guo, Mea Wang.

Network Computing Laboratory

iOverlay: A Lightweight iOverlay: A Lightweight Middleware Infrastructure for Middleware Infrastructure for

Overlay Application Overlay Application ImplementationsImplementations

Baochun Li, Jiang Guo, Mea WangDepartment of Electrical and Computer Engineering

University of TorontoMiddleware 2004

Presenter: Sunghwan Ihm

Korea Advanced Institute of Science and Technology

Network Computing Laboratory | 2

Table of ContentsTable of ContentsOne-Line CommentsMotivating ScenarioArchitectureHighlightsInternal DesignInterfacePerformance and CorrectnessCase StudiesCritiques



One-Line CommentOne-Line CommentThey propose iOverlay, a lightweight and high-performance middleware infrastructure that is specifically designed from scratch to support rapid development of distributed applications and protocols over realistic testbeds



Motivating ScenarioMotivating ScenarioWhen implementing overlay applications such as SPoN or PRISM for PlanetLab……

Tedious and mundane programming tasks includeBootstrappingMessage switchingFault detection and recoveryMeasurements of QoS metrics

Throughput emulationDebugging and monitoring facilitiesVirtualizing distributed nodes

“Reinventing the wheel”Hard to focus on distributed algorithms or protocols!

Any suitable middleware framework???



ArchitectureArchitecture(1) message switching engine

(2) algorithm

(3) application

The ultimate objective is for the application developer to build new algorithms based on the engine, and to select an application to be deployed on top of the algorithms.



Architecture - ObserverArchitecture - Observer



HighlightsHighlightsSimplified interface

Minimize the cost of entry to use iOverlayOnly needs to be aware of one function Engine::send()

Message drivenOnly needs to implement message handler

Executed in a single threadNo need to use thread-safe data structures

Virtualized nodesEach physical node may easily accommodate from one to up to dozens of iOverlay nodesEach iOverlay node has its own bandwidth specifications

Total bandwidth available to and fromUpload and download available bandwidthPer-link bandwidth limits

Maximized performance and portabilityImplemented with the C++ and native POSIX thread libraryObserver: implemented in Windows using the C#



Internal DesignInternal Design•Multi-threaded architecture

•Thread-per-receiver and thread-per-sender design•Separate engine thread•Blocking receive and send operations•Thread-safe circular queue for shared buffers•Engine thread constantly monitors for incoming messages using non-blocking select() function



Internal DesignInternal DesignTight coupling of the algorithm’s and the engine’s message processing components

Reside in the same thread

No paradoxThe algorithm is always reactive and never proactive

Engine::switch()Weighted round-robin

The hold mechanismn-to-m mapping



Internal DesignInternal DesignSalient features

Handling of failures: supports the automatic detection of failed nodes and links, and the automatic tear-down of relevant links after such failures.

(1) exceptions thrown and timeouts at the socket level(2) abnormal signals caught by the engine, such as the Broken Pipe(3) long consecutive periods of traffic inactivityDo not use active probes or “heartbeat updates” to avoid overhead

Measurement of QoS metrics: periodically reported to the algorithm and the observer

TCP throughput, round-trip latency, and number of bytes lostAvailable bandwidth and latency

Emulation of bandwidth availability(1) per-node total bandwidth(2) per-link bandwidth(3) per-node incoming and outgoing bandwidth (cf. DSL)



Internal DesignInternal DesignPerformance considerations

Persistent connectionsAll the messages between two nodes are carried with the same connection, regardless of the applications they belong to

Zero copying of messagesOnly the references of messages are passed from the incoming socket all the way to the outgoing socket

Minimized and stable memory footprint: 100 KB ~ 4 MBThe observer and its proxy

Bootstrap supportControl panelTrace, Debug

Basic elements of algorithmsA generic base class: iAlgorithmImplements a default message handlerImplements a disseminate function with a specific probability p

Gossiping behavior



InterfaceInterfaceObject orientation

New algorithms may be built based on existing algorithm implementations

Improved learning curve of the interface

The algorithm only needs to call one function of the engine: the send function

The algorithm is designed as a message handlerDestruction of messages

Stipulation: “All message destructions are the responsibility of the engine”



Performance and CorrectnessPerformance and CorrectnessEnvironment

C++ on Linux, C# on Windows, and around 19,000 lines of codeA single dual-CPU server with two Pentium III 1GHz processors, 1.5GB of memory, and Linux 2.4.25Compiled with gcc 3.3.3 with the most aggressive optimizations

EvaluatesThe raw message switching performanceCorrectness

Bandwidth emulationNode terminationsEffects of bottleneck with large buffers



Raw PerformanceRaw PerformanceChain Topology(# of node: 2~32)

48.4 MBps 46.8 MBps3.3% overhead

Higher than the typicalthroughput of wide-area connections

Total bandwidth = End-to-end throughput x number of links

…



Correctness – bandwidth Correctness – bandwidth emulationemulation

“back pressure” effectBuffer size: 5 messages



Correctness – node Correctness – node terminationsterminations



Correctness – with large buffersCorrectness – with large buffersBuffer size: 10000 messages



Case StudiesCase StudiesHow useful is iOverlay? – informal discussions

Content-based networking(1) the engine passes messages to the content-based decision-making algorithm(2) once decisions are made, it forwards the message to all selected downstreams

Load balancing, rationality and self-interestsiOverlay supports bandwidth and latency measurements

Fault tolerance, robustness and availabilityiOverlay supports the transparent detection of link and node failures

Case StudiesNetwork CodingConstruction of Data Dissemination TreesService Federation in Service Overlay Networks



Case Study – Network CodingCase Study – Network Coding

With one developer, within a few days!!!

Galois Field, GF(28)

Helpernodes



Case Study – Construction of Data Case Study – Construction of Data Dissemination TreeDissemination Tree

•node stress = the degree of a node in a data dissemination topology / the available “last-mile” bandwidth of the node

•ns-aware: select the minimum-stress node

random

unicast

ns-aware

Experiment-based insights!!!

C

S

A

D B

200

100100 200

better case




81 wide-area ndoes in PlanetLabPer-node bandwidth: 50~200 KBpsSource node bandwidth: 100 KBps

Using the deployment scripts in iOverlay




???

ns-aware performs best



sFlowSelects the most bandwidth efficient downstream service node using iOverlay’s feature that measures point-to-point throughput

sAware messageDisseminates a new service’s existence to all its known hosts

sFededrate messageThe requirement for the complex service

Case Study – Service Federation in Case Study – Service Federation in Service Overlay NetworksService Overlay Networks

Over a period of three weeks!!!

16 real-world nodes in PlanetLab




Left untouched

sFederate Message overhead is small

Fig. 15. (a) Control message overhead; (b) per-link and per-node bandwidth measurements on each of the overlay nodes. The overlay nodes are sorted by their per-node bandwidth availability.




sFederate Message grows at a slower rate

Less than 2 bytes per second

Overhead is moderate and acceptable

Fig. 16. The total control message overhead in a 30-node service overlay network, within a period of 22 minutes.

Fig. 17. The total control message overhead under different network sizes, over a period of 10 minutes, with 50 new service requirements specified and requested every minute.




Source service nodes

Source service nodes or nodes involved in the federation

sFlow performs best

Given the complexity of sFlow, its rapid implementation demonstrates the effectiveness of iOverlay in supporting realistic algorithm implementations!!!

Fig. 18. The per-node control message overhead within a period of 22 minutes, with 50 new service requirements specified and requested every minute.

Fig. 19. End-to-end bandwidth of federated complex services under different network sizes, comparing sFlow to the random and the fixed algorithm.



CritiquesCritiquesStrong points

Observer greatly reduces the cost of deployment and management of overlay nodesThey presented rich case studiesThey presented their strong points in detail

Weak pointsThe informal discussions and case studies seem to be not soundThe centralized observer might be a single point of failureMessage switching engine itself seems to be straightforward

Message switching utility?

They did not show the effect when not using iOverlay

Network Computing Laboratory iOverlay: A Lightweight Middleware Infrastructure for Overlay Application Implementations Baochun Li, Jiang Guo, Mea Wang.

Documents