23 September 2004 Evaluating Adaptive Middleware Load Balancing Strategies for Middleware Systems Department of Electrical Engineering & Computer Science.

23 September 2004

Evaluating Adaptive Middleware Load Evaluating Adaptive Middleware Load

Balancing Strategies for Middleware Balancing Strategies for Middleware

SystemsSystems

Department of Electrical Engineering & Computer Science

Vanderbilt University, Nashville

Jaiganesh Balasubramanian

Ossama Othman

Douglas C. Schmidt

Lawrence Dowdy

8th EDOC Conference, Monterey, California

Vanderbilt University

Balasubramanian et all Evaluating Load Balancing StrategiesR&D Motivation: Scaleable Distributed

Systems•Heterogeneous environments

•Large number of bursty clients

•Stringent QoS requirements, e.g.:

– 24x7 availability

– Low latency & high throughput•Examples

– Online trading systems– Mission-critical systems for critical infrastructure

•e.g., air transportation, power grid control

Local AreaNetwork

NYSE

NASDAQ

Our R&D goal is to assure the scalability of these types of distributed systems


Balasubramanian et all Evaluating Load Balancing StrategiesGeneral Approach: Scalability via Load

Balancing•Goal: Improve the efficiency &

scalability of distributed systems

•Approach: Partition & balance application tasks across many computers via load distribution strategies

•Load balancing be implemented at multiple layers, e.g.:

– Network layer

– OS layer

– Middleware layer


Balasubramanian et all Evaluating Load Balancing Strategies

Our Approach: Middleware Load Balancing•Capabilities of middleware

– Control end-to-end resources & QoS

– Leverage hardware/software advances

– Handle new environments & requirements

– Provide reusable COTS services

•Motivation for middleware load balancing

– Can take into account distributed system state, system runtime behavior, & request content

– Application-level control over load balancing policiesHardware

Domain-SpecificMiddleware Services

CommonMiddleware Services

DistributionMiddleware

Host InfrastructureMiddleware

Operating Systems & Protocols

Applications



TAO & Cygnus: Our Middleware Environment

•Based on The ACE ORB (TAO) – an open-source, high-performance, real-time CORBA Object Request Broker

– deuce.doc.wustl.edu/ Download.html

•Cygnus is a middleware load balancing service implemented atop TAO

•TAO also provides many other middleware services

LOAD LOAD BALANCINGBALANCING

RT-CORBA

EVENT SERVICE SECURITYTRANSACTIONS

DYNAMIC/STATICSCHEDULING

AV STREAMS



Cygnus Load Balancing Architecture •Multiple Clients can invoke

requests

– Potentially non-deterministic

– Duration called a session

•Members

– Multiple instances of the same object implementation

•Object Groups

– Collections of members among which loads will be distributed equitably

•Load Balancer

– Transparently distributes requests to members within an object group

LoadBalancer

Clients

Req

uest

s

Rep

lies

MembersObject Groups



•Load Monitor– Provides load feedback

•Load Analyzer– Determines location and

member load conditions•Member Locator

– Binds client to appropriate object group member

•Load Alert– Facilitates load control (load

shedding)•Load Manager

– Mediates all interactions between load balancer components

Client

Location/Node

LoadManager

MemberLocator

LoadAnalyzer

next_member

push_loads

POArequests

member

requests

*

* LoadMonitor

LoadAlert alert

Cygnus Load Balancing Components



Cygnus Load Balancing Strategies•Non-adaptive strategies

– RoundRobin & Random strategies

•Adaptive strategies

– LeastLoaded strategy rejects new sessions so that load is balanced

•RejectThreshold controls runtime rejection decision

– LoadMinimum strategy migrates client sessions from one machine to another to balance load

•MigrationThreshold controls runtime migration decision

Network

Clients

Req

uest

s

Rep

lies

IDC

Servers



Cygnus Adaptive Load Balancing Interactions

Client LoadManager MemberLocator LoadAnalyzer LoadMonitor LoadAlert Member

1: send_request

2: send_request

3: next_member

4: LOCATION_FORWARD

5: send_request

6: push_loads

7: is_overloaded

8: alert

9: LOCATION_FORWARD

Acceptance strategies

Migration strategies



Evaluating Load Balancing Strategies with LBPerf

We use LBPerf to empirically determine appropriate load balancing strategies for applications with various workload characteristics

We use LBPerf to empirically determine appropriate load balancing strategies for applications with various workload characteristics

•Open-source tool suite for evaluating middleware load balancing strategies

•Can tune different configurations of middleware load balancing

– e.g., systematically choose different load balancing strategies & runtime parameters associated with those strategies

•Evaluate strategies using different metrics

– e.g., throughput, latency, CPU utilization, overhead, etc.

Client code generated by LBPerf

Server code generated by

LBPerf

Load balancer code with option to change strategies

& run-time parameters

Client workloads can vary!!



Summary of Testbed & Experiments•Emulab is a NSF-sponsored testbed

for conducting large-scale network/system experiments

– www.emulab.net

•LBPerf experiments conducted with 16 clients & 32 servers

•RedHat Linux 7.1, Pentium III 850Mhz & 600Mhz machines

•LBPerf experiments run in Emulab

– Overhead measurements

– Throughput measurements

– CPU utilization

– Migration threshold values

– Reject threshold values



Measuring Overhead of Cygnus (1/2)

CPU-intensive requests

Single server

•Single-threaded clients generate CPU-intensive requests on one server

•Repeat experiment with & without load balancer

•Measure throughput, defined as # of requests processed per-second by server

Without load balancer

With load balancer



Measuring Overhead of Cygnus (2/2)•Single-threaded

clients generate CPU-intensive requests on one server

•Repeat experiment with & without load balancer


Overhead of using Cygnus is negligible


Balasubramanian et all Evaluating Load Balancing StrategiesMeasuring Load Balancing Effectiveness

(1/3)

Load balancer code with option to change strategies

& run-time parameters


•Single-threaded clients generate CPU-intensive requests on 4 servers

•Repeat experiment, choosing each of four strategies

•Measure – Throughput,

defined as # of requests processed per-second by server

– CPU utilization at each server



(2/3)•Single-threaded

clients generate CPU-intensive requests on 4 servers





Adaptive strategies

Non-adaptive strategies

Adaptive strategies behave better than non-adaptive strategies with non-uniform

CPU intensive workloads



(3/3)

Adaptive strategies ensure uniform utilization of servers

•Single-threaded clients generate CPU-intensive requests on 4 servers






Balasubramanian et all Evaluating Load Balancing StrategiesMeasuring Effect of Migration Threshold

(1/2)•Single threaded


•Use LoadMinimum strategy

•Repeat experiments for different MigrationThreshold values


Load balancer code with LoadMinimum strategy

Change MigrationThreshold

values




•Single threaded clients generate CPU-intensive requests on 4 servers

•Use LoadMinimum strategy

•Repeat experiments for different MigrationThreshold values


Migration’s not always effective

Measuring Effect of Migration Threshold (2/2)


Balasubramanian et all Evaluating Load Balancing StrategiesMeasuring Effect of Reject Threshold



•Use LeastLoaded strategy

•Repeat experiments for different RejectThreshold values


Load balancer code with LeastLoaded strategy

Change RejectThreshold

values



Balasubramanian et all Evaluating Load Balancing StrategiesMeasuring Effect of Reject Threshold



•Use LeastLoaded strategy

•Repeat experiments for different RejectThreshold values


Reject sessions to ensure QoS for admitted sessions



Concluding Remarks•Cygnus adds little overhead for load

balancing, while providing increased scalability & utilization

•Adaptive load balancing strategies can perform much better than non-adaptive strategies in presence of non-uniform loads

•Different run-time parameters associated with adaptive strategies must be tuned to perform well

•Future work: Devise self-adaptive load balancing architectures

– Take QoS specifications as input

– Tune run-time parameters dynamically to achieve those specifications

Client

Location/Node

LoadManager

MemberLocator

LoadAnalyzer

next_member

push_loads

POArequests

member

requests

*

* LoadMonitor

LoadAlert alert

23 September 2004 Evaluating Adaptive Middleware Load Balancing Strategies for Middleware Systems Department of Electrical Engineering & Computer Science.

Documents

load balancing goal

load feedback load analyzer

middleware environment

object group slide

california slide

types of distributed

tao tao

systems protocols applications