NETE4631:Capacity Planning (2)- Lecture 10 Suronapee Phoomvuthisarn, Ph.D. Email: [email protected] / [email protected].

NETE4631:Capacity Planning (2)- Lecture 10

Suronapee Phoomvuthisarn, Ph.D.Email: [email protected] / Q305

Lecture Outline Basic terminology Queueing Analysis M/M/1 M/M/m M/M/m/m Exercise

Capacity planning strategies Capacity planning problems:

A web server needs to complete an HTTP request within 0.5s when there are 500 HTTP requests per second, what CPU speed do you need?

Let us turn the capacity planning question into a performance analysis question

Performance analysis question: If the web server has a CPU with x MIPS, what is the

response time when there are 500 HTTP requests per second?

If you can solve the performance analysis question for any value of x, you can also solve the capacity planning question

Solutions: Build the system and perform measurement Simulation Mathematical modeling.

Modelling computer systems Single server queue considers only a

component within a computer system A request may require multiple resources E.g.

CPU, disk, network transmission We model a computer systems with multiple

resources by a Queueing Networks (QNs)

Representation of queue

A simple database server

Queueing analysis - Scale up or Scale out better?

Queues with Poisson arrivals

Open, single server queues and How to find:

Waiting time Response time Mean queue length etc.

The technique to find waiting time etc. is called Queueing Theory

Single Server Queue: Terminology

Response Time T = Waiting time W + Service time S

Single Server Queue: Terminology

Motivating example Given

Observation period = 1 minute CPU

Busy for 36s. 1790 transactions arrived 1800 transactions completed

Find Mean service time per completion Utilisation Arrival rate System throughput

Little’s Law Applicable to any “box” that contains some

queues or servers Mean number of jobs in the “box” = Mean response time x Throughput We will use Little’s Law in this lecture to

derive the mean response time We first compute the mean number of jobs in the

“box” and throughput

Single server system In order to determine the response time, you

need to know The inter-arrival time probability distribution The service time probability distribution

Possible distributions Deterministic

Constant inter-arrival time Constant service time

Exponential distribution We will focus on exponential distribution

Exponential inter-arrivals (λ) Exponential service time (μ)

Exponential inter-arrival with rate λ

We assume that successive arrivals are independent

Probability that inter-arrival time is between x and x + δx

= λ exp(- λx) δx

Probabilistic distribution function

Poisson distribution (1) The following are equivalent

The inter-arrival time is independent and exponentially distributed with parameter λ

The number of arrivals in an interval T is a Poisson distribution with parameter λ

Mean inter-arrival time = 1 / λ Mean number of arrivals in time interval T = λ T Mean arrival rate = λ

Poisson distribution (2) Assumption of Poisson arrival process

The number of customers in the system is very large.

Impact of a single customer on the performance of the system is very small, i.e. a single customer consumes a very small percentage of the system resources.

All customers are independent, i.e. their decision to use the system are independent of other users.

Explore …

Service time distribution Service time = the amount of processing time a job

requires from the server We assume that the service time distribution is

exponential with parameter μ The probability that the service time is between t and t +

δt is:

Here: μ = service rate = 1 / mean service time Another interpretation of exponential service time:

Consider a small time interval δ Probability [ a job will finish its service in next δ seconds ]

= μ δ

Sample queueing problems Consider a call centre

Calls are arriving according to Poisson distribution with rate λ The length of each call is exponentially distributed with

parameter μ Mean length of a call is 1/ μ

Queueing theory will be able to answer these questions: What is the mean waiting time for a call? What is the probability that a call is rejected?

Markov chain The state-transition model that we have used is

called a continuous-time Markov chain The transition from a state of the Markov chain

to another state is characterised by an exponential distribution E.g. The transition from State p to State q is

exponential with rate rpq, then consider a small time interval δ

Probability [ Transition from State p to State q in time δ ] = rpq δ

Method for solving Markov chain A Markov chain can be solved by

Identifying the states (may not be easy) Find the transition rate between the states Solve the steady state probabilities

You can then use the steady state probabilities as a stepping stone to find the quantity of interest (e.g. response time etc.)

Call centre with 1 operator and no holding slots Let us see how we can solve the queuing problem for a

very simple call centre with 1 operator and no holding slots

What happens to a call that arrives when the operator is busy?

The call is rejected What happens to a call that arrives when the operator is idle?

The call is admitted without delay. We are interesting to find the probability that an arriving

call is rejected.

Solution (1) There are two possibilities for the operator:

He is busy or He is idle

Let State 0 = Operator is idle (= zero call in the call

centre) State 1 = Operator is busy (= 1 call in the call

centre)

Solution (2) Transition from State 0 to State 1

Caused by an arrival, the rate is λ Transition from State 1 to State 0

Caused by a completed service, the rate is μ State diagram representation

Each circle is a state Label the arc between the states with transition

rate

Solution (3) Steady state means

We have for state 0:

Solution (4) We can do the same for State 1: Steady state means

We have for state 1:

Solution (5) We have one equation

We have 2 unknowns and we need one more equation.

Since we must be either one of the two states:

Solving these two equations, we get the same steady state solution as before

Solving a queueing problem Procedure:

Draw a diagram with the states Add arcs between states with transition rates Derive flow balance equation for each state, i.e. Rate of entering a state = Rate of leaving a state Solve the equation for steady state probability

It is harder to find how a queue evolves with time It is simpler to find how a queue behaves at steady

state

Kendall’s notation To represent different types of queues, queueing

theorists use the Kendall’s notation The call centre example on the previous page can

be represented as:

The call centre example on the last page is a M/M/m/(m+n) queue If n = ∞, we simply write M/M/m

M/M/1 queue

Consider a call centre analogy Calls are arriving according to Poisson distribution with rate λ The length of each call is exponentially distributed with parameter μ

Mean length of a call is 1/ μ

Queueing theory will be able to answer these questions: What is the mean waiting time for a call?

Solving M/M/1 queue (1) We will solve for the steady state response Define the states of the queue

State 0 = There is zero job in the system (= The server is idle)

State 1 = There is 1 job in the system (= 1 job at the server, no job queueing)

State 2 = There are 2 jobs in the system (= 1 job at the server, 1 job queueing)

State k = There are k jobs in the system (= 1 job at the server, k-1 job queueing)

The state transition diagram

Solving M/M/1 queue (2)

Solving M/M/1 queue (3)

Solving M/M/1 queue (4)

Solving M/M/1 queue (5)

Solving M/M/1 queue (6)

Solving M/M/1 queue (7)

Solving M/M/1 queue (8)

Solving M/M/1 queue (9)

Observing response time

Multi-server queues M/M/m

A call centre analogy of M/M/m queue Consider a call centre analogy

Calls are arriving according to Poisson distribution with rate λ

The length of each call is exponentially distributed with parameter μ Mean length of a call is 1/ μ

State transition for M/M/m

M/M/m This is a Markov chain, we have mean

response time T is

Multi-server queues M/M/m/m with no waiting room

A call centre analogy of M/M/m/m queue Consider a call centre analogy

Calls are arriving according to Poisson distribution with rate λ

The length of each call is exponentially distributed with parameter μ Mean length of a call is 1/ μ

State transition for M/M/m/m

What configuration has the best response time?

References Most of the slides have been modified from

COMP9334 Capacity Planning of Computer Systems and Networks Week 1-4: Introduction to Capacity Planning, Chou, C. T., 2008

Recommended reading from Chou Queues with Poisson arrival are discussed in

Bertsekas and Gallager, Data Networks, Sections 3.3 to 3.4.3

Note: Chou has derived the formulas here using continuous Markov chain but Bertsekas and Gallager used discrete Markov chain.

Example You have a computer system with a single

CPU. Both inter-arrival and service times are exponentially distributed. The job only requires services at the CPU. Each job only visits the CPU once. A finished job will leave the system. Mean arrival rate is 9 request/s. Mean service time at the CPU is 0.1s. What is the utilisation of the CPU? What is the mean response time? The utilisation is pretty high and you want to

change the system. -> You can think of 3 alternatives.

Alternative 1 Replace the existing CPU by one that is 2

times faster You may assume that the service time is

inversely proportional to CPU speed.

Alternative 2 Buy a system which is identical to the current

one Put the two system in parallel Add a switch in front of the system

Route 1st,3rd,5th,… requests to System 1 Route 2nd,4th,6th … requests to System 2

Assume the switch requires negligible time

Alternative 3 Similar to Alternative 2, we buy a system

which is identical to the current one and we also buy a switch

However, we only maintain a queue at the switch.

If both system are busy, the request waits at the switch; otherwise, the switch dispatches the request to any of the available systems

Assuming that it takes negligible time for the switch to find out whether a system is idle.

Question 1 Part (a): Calculate the resulting mean response

time for each for the three alternatives Part (b): Repeat part (a) for a number of

different mean arrival rates. Plot a graph of arrival rates against the mean response time.

Part (c): What observations can you make from these calculations?

Part (d): What is the best way to upgrade the system in terms of performance? However, the best way to upgrade in terms of performance may not be the best way to upgrade in terms of cost, why?