3/24//2004 Fairness in Queues, H. Levy, CS, TAU 1
How Fair is your Queue
March, 2004
Hanoch Levy
School of Computer Science, Tel Aviv University
Jointly with
Benjamin Avi-Itzhak, RUTGERS University
David Raz, Tel-Aviv University
3/24//2004 Fairness in Queues, H. Levy, CS, TAU 2
Why QUEUES?
“Not Fair!!!”
To provide FAIRNESS in waiting/service
“This is more Fair…”
Sophisticated queues to address fairness issues
3/24//2004 Fairness in Queues, H. Levy, CS, TAU 3
Queueing Theory, queues and fairness
• Queueing theory: Decades of very deep and elaborate research– Queueing structures / policies, distributions– Focus on delay of individual: Moments/ distribution / optimal
operations, many more!
• Practical Applications: Efficient control / operation of: – Bank, computer system, web server, telecom
• Fairness in queues: – Many statements: “this is fair”, “that system is unfair”. – Very little analysis (job fairness):
• Larson (1988), Palm (1953), Mann (1969), Whitt (1984): Discuss justice related / overtaking
• Morris & Wang (85). • Avi-Itzhak & Levy (96)• Wierman & Harchol-Balter (Sigmetrics 2003):
We don’t know how to quantify queue fairness!
3/24//2004 Fairness in Queues, H. Levy, CS, TAU 4
How Important is fairness in queues?
• Recent studies, Rafaeli et. al. [2003] (experimental psychology):– Experiments on humans in multi-queue and single queue – Fairness in queue is very important to people– Perhaps even more than delay itself!
• WFQ: 10’s of papers – fairness among jobs whose duration is O(1)
microsecond • Economic value of queues of humans:
• O(1%) of GNP?
• FAIRNESS INHERENT/CRUCIAL part of queues
3/24//2004 Fairness in Queues, H. Levy, CS, TAU 5
“So – What is the problem? “
ShortLong
• “Take social-science/economics utility-fairness measures and apply to queues”
The difficulty (B): Size vs. seniority dilemma
• HOW??? What is the “PIE”? A “moving target”…
The difficulty (A): Whom to compare a customer against?
• The problem in a nutshell: Short vs. Long
3/24//2004 Fairness in Queues, H. Levy, CS, TAU 6
Motivation / Questions • Should Long be served ahead of Short?
• Is it fair?
• How fair (how much fair) is it to serve Short Ahead of Long?
• Quantify/ Measure Fairness in Waiting lines!
Short(S)Long (L)
3/24//2004 Fairness in Queues, H. Levy, CS, TAU 7
Motivation / Questions (2):Do You like your supermarket?
• How fair is a scheduling policy?
• E.g:• FIFO• LIFO
Or This?
This?
• Multiple Queue (Multi server)• Single queue (multi server)• Queues by job size?
3/24//2004 Fairness in Queues, H. Levy, CS, TAU 8
Motivation / Questions (3):
and your airport? • Multiple Queue (Multi
server)
• Single queue (multi server)
• Smith & Whitt (81), Larson (87), Rothkopf & Rech (87), Wolff (77, 87, 88)
3/24//2004 Fairness in Queues, H. Levy, CS, TAU 9
Applications (computer world)
• Internet revolution: Service shift to computer systems.
• Responsibility of control: shifts to computer programmer/operator
• Examples:• Call centers:
• Web services:
• How should I operate my web-server? FIFO? LIFO? Priorities?
Call center
Web server
3/24//2004 Fairness in Queues, H. Levy, CS, TAU 10
Related Work(1)
• Avi-Itzhak & Levy (96):• Axiomatic approach • Departure point+emphasis: Seniority (Order of
service) • If service times are identical variance of
waiting time measures fairness• Extend to service times • When is it good?
3/24//2004 Fairness in Queues, H. Levy, CS, TAU 11
Recent Related Work
• Wierman & Harchol-Balter (Sigmetrics 2003):• Propose a Fairness Criterion • Class-based approach: For job of size x compute
E[T(x)/x] – If this is bounded by 1/(1-rho) for all x FAIR.
• Results: Analyze the classification for a large variety of policies.– FIFO (FCFS) – is “Always UNFAIR”– LIFO (preemptive) – is FAIR.
• When is it good?
3/24//2004 Fairness in Queues, H. Levy, CS, TAU 12
Conflicting (disturbing) Views
View
Policy
Ordinary person
Queueing theory (WHB 03)
FIFO
LIFO (preemptive)
Unfair (most?)
Fair
Unfair
Fair
(Fairest?)
3/24//2004 Fairness in Queues, H. Levy, CS, TAU 13
Basic Requirements of a Queue Fairness measure
1. Aim for standard: Have a consistent view/intuition
2. Deal with individual, scenario, and system3. Account for both seniority and service requirement:
• Seniority: Service times are identical: – Fairness is a function of seniority
• FIFO most fair / LIFO most unfair • Senior ahead of junior is more fair
• Service requirement: Arrival times are identical – Fairness is a function of service requirement
• Short ahead of long is more fair
4. Yields to analysis
3/24//2004 Fairness in Queues, H. Levy, CS, TAU 14
RAQFM: A Resource Allocation Queueing Fairness Measure
• Aims at meeting these requirements
• In particular:– Long vs. Short– Seniority vs. service
times
SL
3/24//2004 Fairness in Queues, H. Levy, CS, TAU 15
Approach: individual discrimination
• To whom should a job be compared? (moving target!)• Compare to the “public”.• Focus on server resources (aim at equal division)• Weigh the warranted service with the granted service
• Equal Share philosophy (“axiom”): at t: )(/1)( tNtRi
:iC i
i
d
a
i dttN
R)(
1• Warranted service of
• Granted service of :iC i
i
d
a
iii sdttsG )(
iii RGD • Individual discrimination of
(net service)
:iC
3/24//2004 Fairness in Queues, H. Levy, CS, TAU 16
Basic Properties of RAQFM• Resource allocation is proper = zero-sum discrimination (work conserving, non idling)
0)()()( i
ii
ii
i tRtGtD
0 i
iD 0][ DE
• System measure of discrimination: aggregate statistics of
iD
– Eliminate: Expected discrimination
– Reasonable: distance from mean, Var(D), E[|D-E[D]|].
– Measure of Unfairness
• Individual discrimination: iD
3/24//2004 Fairness in Queues, H. Levy, CS, TAU 17
Property 1: Processor Sharing: Ultimate Fairness
• Single server queue• Processor sharing service policy (Kleinrock (64), (67),
Coffman, Muntz & Trotter (70))
0iD• Theorem 1: Under PS
Var[D] = 0 PS is the most fair policy!!
3/24//2004 Fairness in Queues, H. Levy, CS, TAU 18
Property 2: SENIORITY (identical service times)
• Single server queue• Service times are all identical• Arrival times are arbitrary • Theorem 2:
– Serving by order of seniority (FIFO) is most fair
– Serving in reverse order of seniority (LIFO) is most unfair
– Pushing a junior ahead of senior reduces fairness
L L
3/24//2004 Fairness in Queues, H. Levy, CS, TAU 19
Property 3: Service Time (identical arrival times)
• Single server queue
• Arrival times are all identical
• Service times are arbitrary
• Theorem 3: 2
1
)(1
][ i
N
i
SiN
iNDVar
– Serving shortest job first (SJF) is most fair
– Serving Longest job first (LJF) is most unfair
– Pushing a large job ahead of small job reduces fairness
S
L
3/24//2004 Fairness in Queues, H. Levy, CS, TAU 20
Property 4: More advancedThe Long vs. Short case
• Single server queue
• Long and short arrive at different times
S L
• Fairness of two orders depends on relative seniority and relative service times.
3/24//2004 Fairness in Queues, H. Levy, CS, TAU 21
Property 5: BoundsHow bad (good) can it be?
• Bound on individual discrimination
• Use for scale of reference / sanity check
• How good: ii sD (service time)
• How bad: 2/ii WD (waiting time)
• Bound on system discrimination:
• How bad: 2/][0,2|][|0 max2
max NsDEsDE
..# pbinservedN
3/24//2004 Fairness in Queues, H. Levy, CS, TAU 22
Property 6: Locality of Comparison
• Measure can be evaluated by comparing all customers (across busy periods)
• Unique to RAQFM within a large function family.
• Important for fairness: One should compare only relevant customers (within busy periods).
3/24//2004 Fairness in Queues, H. Levy, CS, TAU 23
Property 7: Discrimination Monotonic in Service time
• THM: For an arbitrary system, if service decision is independent of service time, then: – Discrimination monotonically increases with
service time (deterministic)• Larger customers get preferential service
– Discrimination monotonically increases with service time distribution
justification to the prioritization of short jobs!
3/24//2004 Fairness in Queues, H. Levy, CS, TAU 24
RAQFM is Analyzable
• RAQFM yields to analysis via standard queueing theory techniques
• Can compute
• E[D| x] (class discrimination)
• Var[D] (system unfairness)
• Conducted analysis for M/M/1 type: Variety of service orders (FIFO, LIFO, ROS, more…).
• Conducted analysis for Multi-queue / multi-class systems.
3/24//2004 Fairness in Queues, H. Levy, CS, TAU 25
Individual discrimination under FIFO: M/M/1( conditioned on # customers found ahead)
• Discrimination as a function of # customers found at queue
UtilizationTime to cry
Traffic jam at 4AM
Indifferent
Time to smile
Empty super-market Friday noon
3/24//2004 Fairness in Queues, H. Levy, CS, TAU 26
System Unfairness: Compare operation policies
• System Unfairness as a function of system load
Empty system: everyon
e is alone
LIFO: Severe seniority
discrimination
FIFO: no seniority
discrimination
PS:
Absolute
Fairness!
3/24//2004 Fairness in Queues, H. Levy, CS, TAU 27
How other measures relate:Bridging the gap
LIFO (preemptive)
FIFO
Queueing theory (WHB 2003)
Ordinary person
View
Policy
Unfair
Fair
Unfair
Fair
• Wierman & Harchol-Balter (2003) – FIFO (FCFS) – is UNFAIR
– LIFO (preemptive) – is FAIR
Seniority Service times
3/24//2004 Fairness in Queues, H. Levy, CS, TAU 28
RAQFM: account for all factors - bridge the gap
Seniority + service time differences play role (MOST CASES!)
RAQFM agrees with
ordinary person
Service time differences very radical (A few cases)
RAQFM agrees with
Wierman & Harchol Balter
dwarfsmany
giantsfewa
1.0/1,5.0
10/1,01.0
22
11
FIFO: 0.9
LIFO: 0.15
3/24//2004 Fairness in Queues, H. Levy, CS, TAU 29
Comparison of Methods• [AL96]: (SENIORITY)
• Easy to compute
• Order fairness: When the issue is ORDER
• [WHB03]: (SERVICE TIMES)
• Easy to compute
• When jobs do not see each other / do not care of each other.
• RAQFM: (SENIORITY & SERVICE TIMES)
• Somewhat harder to compute
• When issue is waiting time. Also for ORDER fairness.
3/24//2004 Fairness in Queues, H. Levy, CS, TAU 30
Summary• Fairness is fundamental for queueing systems
• No agreed upon measure exists
• RAQFM is a queueing fairness measure that:1. Has a consistent view 2. Deals with individuals, scenario, and system
3. Accounts for both seniority and service requirement4. Admits logically to special cases:
• Service times are identical: – Senior ahead of junior is more fair
• Arrival times are identical – Short ahead of long is more fair
5. Yields to analysis• We analyzed a large variety of queueing systems• Much more work is needed
3/24//2004 Fairness in Queues, H. Levy, CS, TAU 31
Closing Words
• Thank you
• If you have applications at which fairness is relevant – we will be glad to hear.
• Whenever you “enjoy” the queues of your supermarket / bank / airport / … give us a call…