Queueing Theory and Stochastic Service Systemscfins.au.tsinghua.edu.cn/personalhg/xiali/teaching/queue_2018/lectures/... · Terminology in queueing theory • Basic element in queue

Queueing Theory and Stochastic Service Systems

Li Xia

Tsinghua University, 2018 Fall

Syllabus • Instructor

– Li Xia 夏俐, FIT 3-618, 62793029, [email protected]

• Text book – D. Gross, J.F. Shortle, J.M. Thompson, and C.M. Harris,

Fundamentals of Queueing Theory, 4th Edition, Hoboken: Wiley, 2008. (copy is provided)

• Reference books: – Mor Harchol-Balter, Performance Modeling and Design of

Computer Systems—Queueing Theory in Action, Cambridge Press, 2013.

– Leonard Kleinrock, Queueing Systems, vol. 1: Theory, John Wiley, 1975.

– Caltech course (Prof. Adam Wierman): http://courses.cms.caltech.edu/cs147/

2 Li Xia, Tsinghua Univ.

Syllabus • Grading

– Homework: 30% (4 assignments, 1 simulation task, in English) plagiary is prohibited

– Midterm: 30%

– Final Project: 30% (the 9th week)

– Course Interaction: 10%

• Lecture notes and assignments are available online (in English)

– http://cfins.au.tsinghua.edu.cn/personalhg/xiali/teaching/queue_2018/course_queues.htm


http://cfins.au.tsinghua.edu.cn/personalhg/xiali/teaching/queue_2018/course_queues.htm

http://cfins.au.tsinghua.edu.cn/personalhg/xiali/teaching/queue_2018/course_queues.htm

What’s your purpose to take this course

• What do you expect to learn from this course?

– Open discussion

• Let’s see some examples in practice


Telephone switch • A.K. Erlang studied the problem:

– No automatic switch, operator

– How many telephone operators were needed to handle a given volume of calls

– Erlang A / Erlang B formula

Li Xia, Tsinghua Univ. 5

Agner K. Erlang

(1878 –1929)

Danish engineer

Erlang (1909) "The Theory of Probabilities and Telephone Conversations" –

which proves that the Poisson distribution applies to random telephone traffic.

Erlang (1917) "Solution of some Problems in the Theory of Probabilities of Significance in

Automatic Telephone Exchanges"

https://en.wikipedia.org/wiki/Poisson_distribution

Beijing Subway

Throughput?

Safety?

More lines Increase buffer So what?


Railway ticket online booking in 2012 Chinese new year

• Crash of ticket booking system

– Large number of tickets for sale (4million)

– Huge visit requests 秒杀? (billion)

– System architecture is not optimal

• Bandwidth of network

• CPU/RAM of computer

• Business logic – Queue + Feedback, greatly reduce the repeating request

• Other factors…


Modeling and Analysis

• How to solve it?

– Performance analysis and optimization

– Queueing scheme, increase bandwidth…

Internet

Web server Application server

client Database server

IE browser data input interaction display…

passwd verify cookies/other application…

ticket data booking records… 8 Li Xia, Tsinghua Univ.

Applications in daily life

• Supermarket

– How to define the express line (# of items)?

– How to determine the number of checkouts?

– How long customers have to wait at checkouts?

– Behavior of waiting time during peak-hours

• Line at bank counters

– Multiple lines v.s. single line

– Specialist purpose v.s. generalist purpose

– Number of counters?


Applications in engineering

• Computer/circuit architecture design – 1 fast disk v.s. 2 slow disks?

– Invest on large buffer v.s. fast CPU?

– Scheduling policy to improve performance

• Communication network design – Buffer size design of switch/router

– Data packet scheduling policy in sensor or mobile network

• …


List of applications areas

• Production system (machine, different products)

• Computer system (CPU, disk, RAM design)

• Communication network (buffer design, link capacity)

• Transportation system (traffic lights control)

• Bank branches operation (counter/type design)

• Airlines scheduling (takeoff/landing arrangement)

• Data center (optimal control, energy saving)

• Call center (optimize the operators, hotlines,…)

• Post office (multi-class, specialization)

• Healthcare (hospital procedure design, booking, …)

• …


What’s queue? • A general queueing system

Customer arrival

waiting room

Service facility

Customer departure

r22

r11

r12

r23

r31 r21 r32

r13 r33 1

2

3

r20

r10

r30

γ1

γ2

γ3 queueing

network:

A single

server queue:


Terminology in queueing theory

• Basic element in queue

– Arrival pattern, service pattern, number of servers, service discipline, system capacity, customer type, server type, ..

• Performance metrics

– Average number of customers

– Queue length, average number of queueing customers

– Throughput

– Response time, sojourn time, system time

– Waiting time, queueing time

– Blocking probability, utilization ratio, …


Why we need queueing theory?

• Resource constraints

– Why queues appear? How to make them go away?

• Goal of queueing theory

– Predict the performance

– Design the architecture

– Optimize the parameter/policy

• Counter-intuitive

– Randomness is complicated

– Some examples


Example 1: CPU design

• A simple model of CPU

– Job arrival at rate λ=3/s, Poisson process

– Job mean size is 1/μ, exponential • i.e., service rate is μ=5/s

– FCFS(first come first serve), buffer is infinite

– assume λ < μ, [question]why?

CPU

buffer

Model of a cpu

λ μ

NOTE: modern CPU may have other features, multi-core/PS, etc. 15 Li Xia, Tsinghua Univ.

CPU design, cont.

• If the arrival rate λ doubles, how to upgrade?

– If want to maintain the same delay of jobs, [question] what you choose?

• A. double μ

• B. less than double μ

• C. more than double μ

– Why? Double μ will cut the delay in half

• prove with M/M/1 queueing theory

• Physical intuition, time speeds up with scale 2


Example 2: Lines in bank

• Customer arrival in Poisson with rate λ

• Counter service rate is μ, exponential

• FCFS, infinite waiting capacity

3

3 lines 1 line


L1=1.5, L2=0.237

W1=0.5min, W2=0.079min

T1=1min, T2=0.579min

Lines in bank, cont.

• Assume μ = 2/min, λ = 1/min

– queue length,

– waiting time,

– response time,

• [question] how is the following queue?

33

L3 = 0.5

W3 = 0.1667

T3 = 0.3333


Example 3: many slow v.s. one fast

• CPU selection:

– 1 core CPU with 3GHz freq.

– 3 core CPU with 1GHz freq.

• Which one has a better mean response time?

33

3


• Depend on the variability of jobs

– Job size variability is high, choose many slow CPU

– Job size variability is low, choose one fast CPU

• Exponential distr.: coefficient of variation(cv) = 1; – For the case of M/M/c and M/M/1, the latter is better

• Uniform distr. or deterministic: cv < 1;

• Hyper-exponential distr. or other distr. (PH, MAP): cv > 1. (self-similarity of Internet traffic)

– If workload is low, one fast is preferred

– If jobs are preemptible (priority, stop, resume)

• One fast is preferred

many slow v.s. one fast, cont.


many slow v.s. one fast, cont. • many slow v.s. a few fast, widely exist in practice

– Power allocation in data center with server farm

• Fast freq., more power consumption, green data center

– Bandwidth partition in communication systems

• Small chunks of bandwidth, TDMA/FDMA/CDMA …

– Road network in transportation

• Few wide roads v.s. many small roads (Beijing’s problem) – Traffic is bursty with high variability, prefer many slow

– etc. Consider economic factors…

– Service rate control in Jackson network

• (Xia and Shihada, IEEE-TAC 2013) 21 Li Xia, Tsinghua Univ.

Example 4: Closed queueing network

• Model the intensive traffic with N capacity of network

– Batch system, intensive queue with limited capacity, etc.

1

1

3

0.5

0.5

N=6 jobs

2

1

3


Closed queueing network, cont.

• If we double the speed of server 1

– How it effects the response time of job?

– How it effects the throughput? • [Answer] only change by a small amount

• Suppose N is very large, how is above question?

– Change 0, if N ∞

• What if N is very small

– If N=1, changed amount is large


Closed queueing network, cont.

• What if the queueing network is open?

– remarkable improvement of throughput and average response time

0.5

0.5

1

1

3

1

1

3


Example 5: Task assignment in a server farm

• Front-end dispatcher, web server farm, assign tasks among back-end servers

– used in engineering, Cisco/IBM network device

1

2

Arrivals Dispatcher

(Load Balancer)


• Task assignment policy

– Determine the task should go to which server

– Based on the system state, policy in MDP

• Different policy

– Random

– Shortest-Queue (SQ)

– Size-Interval-Task Assignment (SITA)

– Least-Work-Left (LWL)

– Central-Queue (CQ)

• Question: which one has best mean response time?

Task assignment, cont.


• Answer

– Depend on the property of job size • If job size is known, LWL is usually the best

• LWL = CQ ?

– If server discipline is processor-sharing (PS) • SQ is the near optimal

• Task assignment problem

– FCFS/PS, modeled as an MDP optimization problem

– Minimize variance of response time, rather than its mean

– Variance (fairness, risk) v.s. Mean (social welfare) • Armony&Ward (2010,2013) Oper. Res., “fair routing problem in

heterogeneous queues”

Task assignment, cont.

Chinese Proverb: 不患寡而患不均


Example 6: Scheduling

• How service disciplines affect response time?

– FCFS, first come first serve

– LCFS, last come first serve

– Random

– [answer] all the same


Scheduling, cont.

• What if PR-LCFS, preemptive-resumed LCFS?

– Depends on the randomness of job size

• High randomness, big improvement

• No randomness, twice worse


M/G/1:

Summary of examples

• Why counter-intuition?

– Randomness of queueing

– Interactions among customers and servers

• Toy examples, but many insights

– Models

– Analysis

– Design

– Optimization

– …


Queueing Theory and Stochastic Service Systemscfins.au.tsinghua.edu.cn/personalhg/xiali/teaching/queue_2018/lectures/... · Terminology in queueing theory • Basic element in queue

Documents