© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 1 Case Study: Load Balancing Thursday, 01 June 2006
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 1
Case Study:Load Balancing
Thursday, 01 June 2006
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 2
IntroductionOptimize the utilization of resources to reduce the user response time In telecom load balancing goal is: search the optimal routing given a performance index valueIn computer networks lb goal is: find the optimalallocation of processes given distributed resources (e.g. minimize response time, maximize throughput)
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 3
Load Balancing categoriesStatic
Computed during capacity planning process (batch mode)
DynamicModified at runtime using dedicated hardware or software to collect performance indices (on-line mode)
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 4
Network topology – Example A
open single-class modelexponential arrival rate distribution with μ=1FIFO queue strategyservice time distribution const(1), exp(1), hyperexp(1,2)
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 5
Model definition - JMODELSee example Load_Balancing_Single_Class.jmodel
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 6
Performance indicesResponse Time at each stationSystem Response Time
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 7
1 – Random routingStatic routing strategyAll stations have the same probability to be selected
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 8
2 – Round Robin routingStatic routing strategySelect stations according to a cyclic algorithm
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 9
3 – Shortest Queue Length routingDynamic routing strategySelect station with the smallest queue length
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 10
4 – Least Utilization routingDynamic routing strategySelect station with the smallest average utilization
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 11
Results – Example A
0
0,5
1
1,5
2
2,5
3
3,5
4
4,5
Syst
em R
espo
nse
Tim
e
const(1) exp(1) hyper(1,2)Service Time Distribution
Random Round Robin Least Utilization Shortest Queue Length
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 12
Network topology – Example B
open single-class modelexponential arrival rate distribution with μ=1FIFO queue strategyhyperexponential service time distribution with μ=1 and increasing coefficient of variation c
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 13
Results – Example B
0
10
20
30
40
50
60
1 3 5 7 9 11 13 15 17 19
Coefficient of variation c
Syst
em R
espo
nse
Tim
e
Random Round Robin Least Utilization Shortest Q Length
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 14
QuestionIf we are forced to use a random routing strategy, how can we improve system response time in the case of hyperexponential service times with high values of c?
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 15
Network topology – Example C
exponential arrival rate distribution with μ=4hyperexponential service time distribution:
Two servers with μ = (1 + δ) and c = 4Two servers with μ = (1 - δ) and c = 4One server with μ = 1 and c = 4
from Asymptotic analysis we expect saturation with Random and Round Robin algorithms:
λi = λ / 5 = 4 / 5λi < 1 / DMAX 4 / 5 < 1 / (1 + δ)δ < 0.25
μ = (1 + δ)
μ = (1 - δ)
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 16
Results – Example C
0
5
10
15
20
25
30
0% 5% 10% 15% 20% 25% 30%δ
Syst
em R
espo
nse
Tim
e
Random Round Robin Least Utilization Shortest Q Length
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 17
Problem definitionFind optimal load balancing in a replicated web server farm
n serversRequests from a Poisson process with mean rate λService times exponentially distributed
Mean response time of server i is:Ri = 1 / (μi- λi) i = 1,2,…,n
μi and λi are the service rate and the arrival rate of device i
Mean response time of the subsystem is:
Compute optimal λ*i that minimize R given:
0 ≤ λi < μi
( ) ∑= −
=n
i ii
inR
121
1,,,λμ
λλ
λλλ K
∑=
=n
ii
1λλ
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 18
Chen algorithm (1)
∑
∑ ∑
=
= =
=+
=−=
n
jj
k
j
k
jjkj
nL
nkkL
1
1 1
)1(
,,2,1)(
μ
μμμ K
Order servers by increasing service times:μ1 ≥ μ2 ≥ … ≥ μn
Compute quantities L(k), given by:
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 19
Chen algorithm (2)Compute the value of k such that:
L(k + 1) > λ ≥ L(k)
The optimal values are given by:
ni
i
i
pp
pp
iii
,,10
,,2,1
*1
1*
K
K
+==
=−
−=
∑
∑
=
=
k
kk
k
λ
μ
λμμμλ
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 20
Chen algorithm – Example (1)
Find optimal load subdivision for replicated web servers:
two new servers (S1,2 = 1/μ1,2 = 8ms) four old servers (S3,4,5,6 = 1/μ3,4,5,6 = 12ms)
System arrival rate is λ = 88 req/s
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 21
Chen algorithm – Example (2)We compute L(k):
∑
∑ ∑
=
= =
=+
=−=
n
jj
k
j
k
jjkj
nL
nkkL
1
1 1
)1(
,,2,1)(
μ
μμμ K
L(1) = L(2) = 0L(3) = L(4) = L(5) = L(6) = 45.87L(7) = 585.33
Since λ = 88 req/sL(7) > 88 > L(6)k = 6
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 22
Chen algorithm – Example (3)Now we apply:
λ*1=λ*
2= 30.9 req/sλ*
3=λ*4= λ*
5=λ*6= 6.55 req/s
System response time R = 11.34msU1,2 = 24.72%U3,4,5,6= 7.84%
ni
i
i
pp
pp
iii
,,10
,,2,1
*1
1*
K
K
+==
=−
−=
∑
∑
=
=
k
kk
k
λ
μ
λμμμλ
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 23
Questions
What happen if arrival rate is λ = 44 req/s ?
What happen if arrival rate is λ = 586 req/s ?
© Bertoli Marco A.A. 2005/2006 Dimensionamento degli impianti Informatici LoadBal - 24
ReferencesBERTOLI, M. jSIM: Un Tool per la Simulazione di Modelli a Reti di Code. Tesi presso il Politecnico di Milano, 2005.BUZEN, J.P., AND CHEN, P.P. Optimal load balancing in memory hierarchies. In Information Processing 74. North-Holland, New York, 1974, pp. 271-275. CHEN, P.P. Optimal file allocation in multi-level storage systems. In AFIPS Conference Proceedings (New York, June 1973), vol. 42. AFIPS Press, Reston, Va., 1973, pp. 277-282. SERAZZI G., "The dynamic behavior of computer systems", in Experimental Computer Performance Evaluation, pp. 127-163, Elsevier, 1981.