Introduction to Queuing Theory
Dec 28, 2015
Introduction to Queuing Theory
2
Queueing theory definitions (Bose) “the basic phenomenon of queueing arises
whenever a shared facility needs to be accessed for service by a large number of jobs or customers.”
(Wolff) “The primary tool for studying these problems [of congestions] is known as queuing theory.”
(Kleinrock) “We study the phenomena of standing, waiting, and serving, and we call this study Queuing Theory." "Any system in which arrivals place demands upon a finite capacity resource may be termed a queueing system.”
(Mathworld) “The study of the waiting times, lengths, and other properties of queues.”
3
Applications of Queuing Theory Telecommunications Traffic control Determining sequence of computer
operations Predicting computer performance Health services (e.g., control of hospital
bed assignments) Airport traffic, airline ticket sales Layout of manufacturing systems.
4
Example application of queuing theory In many retail stores and banks
multiple line/multiple checkout system a queuing system where customers wait for the next available cashier
We can prove using queuing theory that : throughput improves increases when queues are used instead of separate lines
5
Example application of queuing theory
6
Queuing theory for studying networks View network as collections of queues
FIFO data-structures Queuing theory provides probabilistic
analysis of these queues Examples:
Average length Average waiting time Probability queue is at a certain length Probability a packet will be lost
7
Model Queuing System
Server System Queuing System
Queue Server
Queuing System
Use Queuing models to Describe the behavior of queuing systems Evaluate system performance
8
Characteristics of queuing systems Arrival Process
The distribution that determines how the tasks arrives in the system.
Service Process The distribution that determines the task
processing time Number of Servers
Total number of servers available to process the tasks
9
Kendall Notation 1/2/3(/4/5/6)
Six parameters in shorthand• First three typically used, unless specified
1. Arrival Distribution2. Service Distribution3. Number of servers 4. Total Capacity (infinite if not specified) 5. Population Size (infinite) 6. Service Discipline (FCFS/FIFO)
10
Distributions
M: stands for "Markovian", implying exponential distribution for service times or inter-arrival times.
D: Deterministic (e.g. fixed constant) Ek: Erlang with parameter k
Hk: Hyperexponential with param. k G: General (anything)
11
Kendall Notation Examples
M/M/1: Poisson arrivals and exponential service, 1 server,
infinite capacity and population, FCFS (FIFO) the simplest ‘realistic’ queue
M/M/m Same, but M servers
G/G/3/20/1500/SPF General arrival and service distributions, 3 servers,
17 queue slots (=20-3), 1500 total jobs, Shortest Packet First
12
Analysis of M/M/1 queue
Given: • : Arrival rate of jobs (packets on input link) • : Service rate of the server (output link)
Solve: L: average number in queuing system Lq average number in the queue W: average waiting time in whole system Wq average waiting time in the queue
13
M/M/1 queue model
Wq
W
L
Lq
14
Little’s Law
Little’s Law: Mean number tasks in system = mean arrival rate x mean response time Observed before, Little was first to prove
Applies to any system in equilibrium, as long as nothing in black box is creating or destroying tasks
Arrivals Departures
System
15
Proving Little’s Law
J = Shaded area = 9
Same in all cases!
1 2 3 4 5 6 7 8
Packet #
Time
123
1 2 3 4 5 6 7 8
# in System
123
Time
1 2 3
Time inSystem
Packet #
123
Arrivals
Departures
16
Definitions
J: “Area” from previous slide N: Number of jobs (packets) T: Total time : Average arrival rate
N/T W: Average time job is in the system
= J/N L: Average number of jobs in the system
= J/T
17
1 2 3 4 5 6 7 8
# in System(L) 1
23
Proof: Method 1: Definition
Time (T) 1 2 3
Time inSystem(W)
Packet # (N)
123
=
WL TN )(
NWTLJ
WL )(
18
Proof: Method 2: Substitution
WL TN )(
WL )(
))(( NJ
TN
TJ
TJ
TJ Tautology
19
M/M/1 queue model
Wq
W
L
Lq
L=λWLq=λWq
20
Poisson Process
For a poisson process with average arrival rate , the probability of seeing n arrivals in time interval delta t
0...)2Pr(
)1Pr()(...]!2
)(1[)1Pr(
1)0Pr()(1...!2
)(1)0Pr(
)(!
)()Pr(
2
2
ttott
ttte
ttott
te
tnEn
ten
t
t
nt
21
Poisson process & exponential distribution Inter-arrival time t (time between
arrivals) in a Poisson process follows exponential distribution with parameter
1)(
)Pr(
tE
et t
22
M/M/1 queue model
1
Wq
W
L
Lq
L=λWLq=λWqW = Wq + (1/μ)
23
Solving queuing systems
4 unknowns: L, Lq W, Wq
Relationships: L=W Lq=Wq W = Wq + (1/)
If we know any 1, can find the others
0
n
nnPL
24
Analysis of M/M/1 queue
Goal: A closed form expression of the probability of the number of jobs in the queue (Pi) given only and
25
Equilibrium conditions
n+1nn-1
)(tPnDefine to be the probability of having n tasks in the system at time t
0)()(
lim,)(lim, when Stablize
)()()()()()(
)()()()(
)]1)()[(()]1)()[((])1)(1)[(()(
)]1)()[((])1)(1)[(()(
11
1000
11
100
t
tPttPPtP
tPtPtPt
tPttP
tPtPt
tPttP
tttPtttPtttttPttP
tttPtttttPttP
nn
tnn
t
nnnnn
nnnn
26
Equilibrium conditions
11
10
)(
nnn PPP
PP
n+1nn-1
27
Solving for P0 and Pn
Step 1
Step 2
0,0
2
201 , PPPPPPn
n
0
0
000
1,1,1
n
n
n
n
nn PPthenP
28
Solving for P0 and Pn
Step 3
Step 4
, then
n
n0
n 1
1 n0
1
1 1
P0 1
nn0
1 and Pn n 1
29
Solving for L
0
n
nnPL )1(0
n
nn )1(1
1
n
nn
(1 ) dd
11
0
)1(n
ndd
(1 ) 1(1 )2
)1(
30
Solving W, Wq and Lq
W L
1 1
Wq W 1
1
( )
Lq Wq ( )
2
( )
L
31
Online M/M/1 animation
http://www.dcs.ed.ac.uk/home/jeh/Simjava/queueing/mm1_q/mm1_q.html
32
Response Time vs. Arrivals
1W
Waiting vs. Utilization
0
0.05
0.1
0.15
0.2
0.25
0 0.2 0.4 0.6 0.8 1 1.2
W(s
ec)
33
Stable Region
Waiting vs. Utilization
0
0.005
0.01
0.015
0.02
0.025
0 0.2 0.4 0.6 0.8 1
W(s
ec)
linear region
34
Example
On a network gateway, measurements show that the packets arrive at a mean rate of 125 packets per second (pps) and the gateway takes about 2 millisecs to forward them. Assuming an M/M/1 model, what is the probability of buffer overflow if the gateway had only 13 buffers. How many buffers are needed to keep packet loss below one packet per million?
35
Example
Measurement of a network gateway: mean arrival rate (l): 125 Packets/s mean response time (m): 2 ms
Assuming exponential arrivals: What is the gateway’s utilization? What is the probability of n packets in the gateway? mean number of packets in the gateway? The number of buffers so P(overflow) is <10-6?
36
Example
Arrival rate λ = Service rate μ = Gateway utilization ρ = λ/μ = Prob. of n packets in gateway =
Mean number of packets in gateway =
37
Example Arrival rate λ = 125 pps Service rate μ = 1/0.002 = 500 pps Gateway utilization ρ = λ/μ = 0.25 Prob. of n packets in gateway =
Mean number of packets in gateway =
(1 )n 0.75(0.25)n
1
0.25
0.570.33
38
Example
Probability of buffer overflow:
To limit the probability of loss to less than 10-6:
39
Example Probability of buffer overflow:
= P(more than 13 packets in gateway)
To limit the probability of loss to less than 10-6:
40
Example Probability of buffer overflow:
= P(more than 13 packets in gateway) = ρ13 = 0.2513 = 1.49x10-8
= 15 packets per billion packets To limit the probability of loss to
less than 10-6:
41
Example Probability of buffer overflow:
= P(more than 13 packets in gateway) = ρ13 = 0.2513 = 1.49x10-8
= 15 packets per billion packets To limit the probability of loss to
less than 10-6:
n 10 6
42
Example To limit the probability of loss to
less than 10-6:
or
25.0log/10log 6n
n 10 6
43
Example To limit the probability of loss to
less than 10-6:
or
= 9.96 ≈10 buffers
25.0log/10log 6n
n 10 6
Example
One fast server vs. m slow servers? In terms of delay, what will happen?
44
Example
Customers arrive at a fast-food restaurant at a rate of 5/minute and wait to receive their order for an average of 5 minutes. Customers eat in the restaurant with probability 0.5 and carry out their order without eating with probability 0.5. A meal requires an average of 20 minutes. What is the average number of customers in the restaurant?
45
Example
Empty taxis pass by a street comer at a Poisson rate of 2 per minute and pick up a passenger if one is waiting there. Passengers arrive at the street corner at a Poisson rate of 1 per minute and wait for a taxi only if there are fewer than 4 persons waiting; otherwise, they leave and never return. Find the average waiting time of a passenger who joins the queue.
46
Example
The average time,T, a car spends in a certain traffic system is related to the average number of cars N in the system by a relation of the form T = a + bN2, where a > 0, b > 0 are give in scalars. What is the maximal car arrival rate that the
system can sustain?
47
Example
A person enters a bank and finds all of the four clerks busy serving customers. There are no other customers in the bank, so the person will start service as soon as one of the customers in service leaves. Customers have independent, identical, exponential distribution of service time. What is the probability that the person will be the
last to leave the bank assuming that no other customers arrive?
If the average service time is 1 minute, what is the average time the person will spend?
48