Queuing Theory and Stochastic Service Systems

Queuing Theory and Stochastic Service Systems

Li Xia Tsinghua University, 2014 Fall

Syllabus • Instructor

– Li Xia 夏俐, FIT 3-618, 62793029, [email protected] • Text book

– D. Gross, J.F. Shortle, J.M. Thompson, and C.M. Harris, Fundamentals of Queueing Theory, 4th Edition, Hoboken: Wiley, 2008. (copy is provided)

• Reference books: – Leonard Kleinrock, Queueing Systems, vol. 1: Theory, John Wiley, 1975. – Mor Harchol-Balter, Performance Modeling and Design of Computer

Systems—Queueing Theory in Action, Cambridge Press, 2013. – Caltech course (Prof. Adam Wierman):

http://courses.cms.caltech.edu/cs147/ – 林闯，计算机网络和计算机系统的性能评价，清华大学出版社，

2001.

2 Li Xia, Tsinghua Univ., 2014

Syllabus • Grading

– Homework: 25% (4 assignments, 1 simulation task, in English) plagiary is prohibited

– Midterm: 25% – Final Project: 40% (the 9th week) – Course Interaction: 10%

• Lecture notes and assignments are available online (in English) – http://cfins.au.tsinghua.edu.cn/personalhg/xiali

/teaching/course_queues.htm


http://cfins.au.tsinghua.edu.cn/personalhg/xiali/teaching/course_queues.htm�

http://cfins.au.tsinghua.edu.cn/personalhg/xiali/teaching/course_queues.htm�

What’s your purpose to take this course

• What do you expect to learn from this course? – Open discussion

• Let’s see some examples in practice


Beijing Subway

Throughput?

Safety?

More lines Increase buffer So what?


Railway ticket online booking in 2012 Chinese new year

• Crash of ticket booking system – Large number of tickets for sale (4million) – Huge visit requests 秒杀? (billion) – System architecture is not optimal

• Bandwidth of network • CPU/RAM of computer • Business logic • Or other factors…


Modeling and Analysis • How to solve it?

– Performance analysis and optimization – Queueing scheme, increase bandwidth…

Internet

Web server Application server

client Database server

IE browser data input interaction display…

passwd verify cookies/other application…

ticket data booking records…


Applications in daily life

• Supermarket – How to define the express line (# of items)? – How to determine the number of checkouts? – How long customers have to wait at checkouts? – Behavior of waiting time during peak-hours

• Line at bank counters – Multiple lines v.s. one line – Specialist purpose v.s. generalist purpose – Number of counters?


Applications in engineering

• Computer/circuit architecture design – 1 fast disk v.s. 2 slow disks? – Invest on large buffer v.s. fast CPU? – Scheduling policy to improve performance

• Communication network design – Buffer size design of switch/router – Data packet scheduling policy in sensor or mobile

network • …


List of applications areas

• Production system (machine, different products) • Computer system (cpu, disk, RAM design) • Communication network (buffer design, link capacity) • Transportation system (traffic lights control) • Bank branches operation (counter/type design) • Airlines scheduling (takeoff/landing arrangement) • Data center (optimal control, energy saving) • Call center (optimize the operators, hotlines,…) • Post office (multi-class, specialization) • …


What’s queue? • A general queuing system

Customer arrival

waiting room

Service facility

Customer departure

r22

r11

r12

r23

r31 r21 r32

r13 r33 1µ

2µ

3µ

r20

r10

r30 γ1

γ2

γ3 Queuing network:

A single server queue:


Terminology in queuing theory

• Basic element in queue – Arrival pattern, service pattern, number of servers, service

discipline, system capacity, customer type,..

• Performance metrics – Average number of customers – Queue length, average number of queuing customers – Throughput – Response time, sojourn time, system time – Waiting time, queuing time


Why we need queuing theory?

• Resource constraints – Why queues appear? How to make them go away?

• Goal of queuing theory – Predict the performance – Design the architecture – Optimize the parameter/policy

• Counter-intuitive – Randomness is complicated – Some examples


Example 1: CPU design

• A simple model of CPU – Job arrival at rate λ=3/s, Poisson process – Job mean size is 1/μ, exponential

• i.e., service rate is μ=5/s

– FCFS(first come first serve), buffer is infinite – assume λ < μ, [question]why?

CPU

buffer

Model of a cpu

λ μ

NOTE: modern CPU may have other features, multi-core/PS, etc. 14 Li Xia, Tsinghua Univ., 2014

CPU design, cont.

• If the arrival rate λ doubles, how to upgrade? – If want to maintain the same delay of jobs,

[question] what you choose? • A. double μ • B. less than double μ • C. more than double μ

– Why? Double μ will cut the delay in half • prove with M/M/1 queuing theory • Physical intuition, time speeds up with scale 2


Example 2: Lines in bank • Customer arrival in Poisson with rate λ • Counter service rate is μ, exponential • FCFS, infinite waiting capacity

3λµ

µ

µ

µ

µ

µ

λ

λ

λ

3 lines 1 line 16 Li Xia, Tsinghua Univ., 2014

L1=1.5, L2=0.237

W1=0.5min, W2=0.079min

T1=1min, T2=0.579min

Lines in bank, cont.

• Assume μ = 2/min, λ = 1/min – queue length, – waiting time, – response time,

• [question] how is the following queue?

3λ 3µL3 = 0.5 W3 = 0.1667 T3 = 0.3333


Example 3: many slow v.s. one fast

• CPU selection: – 1 core CPU with 3GHz freq. – 3 core CPU with 1GHz freq.

• Which one has a better mean response time?

3λ 3µ3λ

µ

µ

µ18 Li Xia, Tsinghua Univ., 2014

• Depend on the variability of jobs – Job size variability is high, choose many slow CPU – Job size variability is low, choose one fast CPU

• Exponential distr.: coefficient of variation(cv) = 1; – For the case of M/M/c and M/M/1, the latter is better

• Uniform distr. or deterministic: cv < 1; • Hyper-exponential distr. or other distr. (PH, MAP):

cv > 1. (self-similarity of Internet traffic)

– If workload is low, one fast is preferred – If jobs are preemptible (priority, stop, resume)

• One fast is preferred

many slow v.s. one fast, cont.


many slow v.s. one fast, cont. • many slow v.s. a few fast, widely exist in

engineering problems – Power allocation in data center with server farm

• Fast freq., more power consumption, green data center

– Bandwidth partition in communication systems • Small chunks of bandwidth, TDMA/FDMA/CDMA …

– Road network in transportation • Big trunk road v.s. many small roads

– etc. Consider economic factors… – Service rate control in Jackson network

• (Xia and Shihada, IEEE-TAC 2013) 20 Li Xia, Tsinghua Univ., 2014

Example 4: Closed queueing network

• Model the intensive traffic with N capacity of network – Batch system, intensive queue with limited

capacity, etc.

113

µ =0.5

0.5

N=6 jobs

113

µ =


Closed queueing network, cont.

• If we double the speed of server 1 – How it effects the response time of job? – How it effects the throughput?

• [Answer] only change by a small amount

• Suppose N is very large, how is above question? – Change 0, if N ∞

• What if N is very small – If N=1, changed amount is large


Closed queueing network, cont.

• What if the queueing network is open? – remarkable improvement of throughput and

average response time

0.5

0.5

λ 113

µ =

113

µ =


Example 5: Task assignment in a server farm

• Front-end dispatcher, web server farm, assign task among back-end servers – used in engineering, Cisco/IBM network device

λ 1µ

2µ

Arrivals Dispatcher (Load Balancer)


• Task assignment policy – Determine the task should go to which server – Based on the system state, policy in MDP

• Different policy – Random – Shortest-Queue (SQ) – Size-Interval-Task Assignment (SITA) – Least-Work-Left (LWL) – Central-Queue (CQ)

• Question: which one has best mean response time?

Task assignment, cont.


• Answer – Depend on the property of job size

• If job size is known, LWL is usually the best • LWL = CQ ?

– If server discipline is processor-sharing (PS) • SQ is the near optimal

• Task assignment problem – FCFS/PS, modeled as an MDP optimization problem – Minimize variance of response time, rather than the mean

response time – Variance (fairness, risk) v.s. Mean (social welfare)

Task assignment, cont.

Chinese Proverb: 不患寡而患不均 26 Li Xia, Tsinghua Univ., 2014

Example 6: Scheduling

• How service disciplines affect response time? – FCFS, first come first serve – LCFS, last come first serve – Random – [answer] all the same

λ µ


Scheduling, cont.

• What if PR-LCFS, preemptive-resumed LCFS? – Depends on the randomness of job size

• High randomness, big improvement • No randomness, twice worse

λ µ


Summary of examples

• Why counter-intuition? – Randomness of queuing – Interactions among customers and servers

• Toy example, but many insights – Models – Analysis – Design – Optimization – …


Queuing Theory and Stochastic Service Systems

Documents