-
1
Weighted Fair Queuing (WFQ)
Prof. T. Szymanski McMaster University
Prof. T Szymanski 4DN4 - WFQ, pg 2
Overview
! Internet Flows & Fairness ! Fluid-Flow Generalized
Processor Sharing (GPS) ! Packet-Flow Weighted Fair Queuing
(WFQ)
! Definition ! Computing Finishing Times ! Examples
! Jitter (Inter-Departure Time) Distribution ! Commercial
Implementations ! Other Scheduling Methods
-
Prof. T Szymanski 4DN4 - WFQ, pg 3
Internet Flow/Session
" Definition: A flow is a sequence of packets sent from one IP
source to one IP destination [text, p594]
" the source and destination addresses are defined by an IP
address and a port number
Example of the bandwidth requirement of a traffic flow. [1, Fig
7.52, p551]
Bandwidth
Prof. T Szymanski 4DN4 - WFQ, pg 4
Flow of Packets [text, p539]
" An IP router = a node where IP packets arrive, are segmented
into fixed-size cells, queued at the input queues, scheduled and
transmitted through the switch, reassembled and buffered at the
output queues, queued at the output fiber awaiting transmission,
and eventually transmitted onto the outgoing fiber .
" An IP router will contain Input Queues (IQs) and Output Queues
(OQs)
" A typical backbone router with 40 Gbps links will buffer
typically up to 5 million IP packets per port
" Packets from competing traffic flows compete for buffer space
and transmission bandwidth along the outgoing transmission line
" How does one define fairness?
-
Prof. T Szymanski 4DN4 - WFQ, pg 5
Flow of Packets Through Routers
Consider the end-to-end QoS of a packet traversing N switches
[1, Fig 7.40, p539]
Each packet traverses N queueing systems, and experiences delay
and jitter at each queueing system.
Prof. T Szymanski 4DN4 - WFQ, pg 6
Fairness
" Intuitively
" Competing flows should be guaranteed the bandwidth they have
reserved (assuming they can reserve bandwidth)
" Any excess bandwidth should be equally shared amongst the
flows.
" Protection against Starvation
" Fairness prevents Starvation or Deadlock, where flows are
deprived of access to buffers/bandwidth [text, p540]
" Fair scheduling
" Automatically isolates heavy users, to avoid resource
hogging
" Provides dynamic sharing of unused bandwidth allocations.
-
Prof. T Szymanski 4DN4 - WFQ, pg 7
Requirements of Fair Scheduling
" An ideal scheduler: " is easy to implement " is totally fair
in resource sharing " provides protection against starvation "
provides guaranteed performance bounds " offers different flows
different QoS; each flow given a
weight, ie a guaranteed fraction of resource usage
Interactive video
Packet voice
Email
r
Flows entering the same queue are given weights, w1, w2, and w3,
and share the bandwidth r accordingly.
Q
w1
w2
w3
Prof. T Szymanski 4DN4 - WFQ, pg 8
Problem with Round-Robin Service (pg 544, text)
" Suppose 4 flows arrive at an IP router. Why not service the
packets in round-robin order ?
" Suppose packets in flow 1 are large, packets in flows 2 and 3
are small, and packets in flow 4 are very small.
" Servicing queues in round-robin order would give flow 1 much
more service than the other flows, since it has much larger
packets. Similarly, flow 4 would receive very little bandwidth.
-
Prof. T Szymanski 4DN4 - WFQ, pg 9
Generalized Processor Sharing(GPS)
" GPS is an idealized Fluid Model, developed at MIT, which
provides an upper bound on Fairness [1, p542] " Each flow has its
own buffer. " Transmission bandwidth is divided among the flows
according to flow weights " Conceptually, packets in buffers
like a fluid that is drained
continuously. " How it works (assume all flow weights are equal
for now) :
" Assume initially that we service individual bits from each
flow, in a round-robin order
" Given n flows in the system and transmission capacity = C
bits/sec, then each flow is guaranteed transmission rate = (C/n)
bits/sec
" A flow could experience a higher rate when the other flows are
not using their allocation of bandwidth.
Prof. T Szymanski 4DN4 - WFQ, pg 10
Fluid (Bit-by-Bit) Round-Robin Service in GPS
let R(t) denote the current round of service at time t; R(t) is
also called Virtual Time. Virtual time increases quickly when few
buffers are active. Virtual time increases slowly when many buffers
are active.
-
Prof. T Szymanski 4DN4 - WFQ, pg 11
" One round consists of a round-robin service cycle in which all
n queues are offered service (1 bit of service each, for now).
" Therefore, the amount of real time elapsed per round = (number
of active queues) * (time serving one bit from each active
queue)
" nactive(t) queues = number of active (non-empty) queues at
time t
" In round of service, all nactive(t) queues (i.e. non-empty
queues) are serviced (1 bit for now).
" The Round Number R(t) = the number of rounds of service
completed by the GPS round-robin server at the current time t
Definition of a GPS Round (text, Fig. 7.46, pg. 544)
Prof. T Szymanski 4DN4 - WFQ, pg 12
Packet Finishing Time
" The Fluid-Based GPS system effectively computes the ideal
finishing time of a packet in a perfectly fair system
" The finishing time of a packet is easily expressed in terms of
rounds (Virtual-Time), rather than real time (i.e. a packet with
100 bits will finish service after 100 rounds have expired)
" In fluid-based GPS, packets are said to depart the system when
they reach their finishing times => Packets with smallest
finishing times are transmitted first.
" The GPS scheduler maintains several state variables: 1) The
current round number R(t) or the current Virtual Time 2) The
finishing time (in rounds) for every packet in the system,
3) The last finishing time (in rounds) of the last packet in
every queue.
-
Prof. T Szymanski 4DN4 - WFQ, pg 13
Finishing time: Empty Queue Case (text, pg 545)
" GPS Finishing times are expressed in rounds of service.
" Let F(f,k) denote the finishing time of packet k from flow f
in a queue " Suppose packet k from flow f arrives at an empty queue
at real time t
(f,k) and round (ie virtual time) R( t(f,k) ). " The Packet has
length P(f,k) bits. Therefore, it will complete its
transmission after P(f,k) rounds of additional service (since
each round services one bit)
" Assuming all flows request equal service: " The finishing
time, expressed in rounds, is clearly:
F(f,k) = R( t(f,k) ) + P(f,k) (1)
Prof. T Szymanski 4DN4 - WFQ, pg 14
Finishing time: NonEmpty Queue Case
" Suppose packet k from flow f arrives at a nonempty queue " It
must wait for all the packets ahead of it (and in particular
packet
k-1 which is directly ahead of it) to be serviced before it is
serviced. " The finishing time of the packet ahead of it (packet
k-1) in the queue
is F(f, k-1). " Assuming all flows request equal service:
" The current packet needs P(f,k) additional rounds of service,
therefore the finishing time of this packet is clearly:
F(f,k) = F(f,k-1) + P(f,k) (2)
-
Prof. T Szymanski 4DN4 - WFQ, pg 15
Finishing time: Empty and NonEmpty Queues
From the last slides we have:
" F(f,k) = R( t(f,k) )+ P(f,k) - Empty Queue Case (1) " F(f,k) =
F(f,k-1) + P(f,k) - NonEmpty Queue Case (2)
Combining equations (1) and (2) we get
F(f,k) = max{F(f,k-1), R( t(f,k) )} + P(f,k)
This is all great, but how does it help us? We can now compute
the ideal departure time of packets in a perfectly fair GPS
system, expressed in rounds, when each flow requests equal
service. However, can we really build IP routers which service
packets in round-robin order
at the bit-level (ie one bit of service per round) ? No, but we
do not need to. We only need to know the ideal finishing time of
each packet in a GPS system.
Also, what if flows request different levels of service?
Prof. T Szymanski 4DN4 - WFQ, pg 16
Packet-oriented GPS == WFQ (pg 544) Let a Weighted Fair Queueing
(WFQ) Server run in parallel with an ideal GPS server. The WFQ
server only services whole IP packets (it doesnt split them into
bits).
Given variable length IP packets, suppose the WFQ server will
service entire packets in the same order as the perfectly fair
service order of a fluid-flow GPS system, i.e., the next packet to
be transmitted over a transmission line by the WFQ server is the
packet with the smallest GPS Virtual Finishing time amongst all
packets waiting for that transmission line
Call this system Packet-oriented GPS, to distinguish it from the
Fluid-flow GPS. It is also called Weighted Fair Queueing. This
scheme is implemented in CISCO routers.
-
Prof. T Szymanski 4DN4 - WFQ, pg 17
Packet-by-Packet WFQ vs Fluid-Flow GPS
" Following slides will illustrate the differences between ideal
fluid flow GPS and Packet-by-Packet Weighted-Fair-Queuing.
" Assume buffers 1 and 2 each have one L-bit packet to transmit
at time t = 0, and that no subsequent packets arrive
" Assume the transmission capacity is C = L bits/second, then we
can transmit 1 packet/sec .
" The idealized fluid-flow GPS system transmits both packets
simultaneously at a rate = ! packet/sec and both packets complete
at time t = 2 seconds.
" The packet-by-packet WFQ system transmits (1) the packet from
buffer 1 at rate = 1 packet/sec, then (2) the packet from buffer 2
at rate = 1 packet/sec, so the packet completion times are 1 and 2
seconds, respectively.
Prof. T Szymanski 4DN4 - WFQ, pg 18
GPS vs Packet-by-Packet WFQ, (Fig 7.45, pg 543) Note: Y-axis =
fraction of packet in buffer1 and buffer 2. Both buffers contain 1
L -bit packet initially.
-
Prof. T Szymanski 4DN4 - WFQ, pg 19
Packets with Different Lengths (Fig 7.46, pg 546)
Prof. T Szymanski 4DN4 - WFQ, pg 20
Adding Flow Weights to GPS or WFQ (pg 546) " In a real IP
router, different flows will have different bandwidth requirements
" Let each flow j have its own buffer and a weight Wj which
determines its
relative share of bandwidth C (bits/sec) " In each service
round, the server services Wj bits from flow j " When all flows are
active, each flow j receives the fluid service at the rate C*
(Wj) /sum(Wj == guaranteed service rate for each flow " When
some flows are idle, the excess bandwidth is shared amongst active
flows
according to the weight of the active flows
" the next slide illustrates an example of Weighted Fair
Queueing - WFQ. Let flow 1 have weight W1=(1/4), and flow 2 have
weight W2 = (3/4)
" Both flows have packet size = 1. The time to complete each
packet is (L bits)/(Wj * C bits/sec), where Wj is expressed as a
fraction. At t=0, based upon the 2 active flows, the time to
complete flow #1 = L bits/(1/4 * C bits/sec) = 4 L/C seconds. Time
to complete flow #2 = L bits/(3/4 * C bits/sec) = (4/3) L/C
seconds. In this case (L/C) = 1. Once flow 2 is finished it
service, since there are no other active flows, flow 1 starts
receiving service at the rate L bits/sec. (When making such graphs,
one difficulty is keeping track of the discontinuities, when the
service rates change.)
-
Prof. T Szymanski 4DN4 - WFQ, pg 21
Weighted GPS and Packet Fair Queueing (Fig. 7.48, pg 546)
Prof. T Szymanski 4DN4 - WFQ, pg 22
GPS Finishing Time: Empty Queue
" Finishing times are expressed in rounds of service.
" Suppose packet k from flow f arrives at an empty queue at time
t(f,k) and round R( t(f,k) )=>
" the packet arrives when R( t(f,k) ) rounds of service have
already finished.
" The Packet has length P(f, k) bits. Therefore, it will
complete its transmission after P(f,k)/Wf rounds of additional
service (Wf expressed as an integer), since each round services Wf
bits
" The finishing time is clearly: F(f,k)= R( t(f,k) ) + P(f,k)/Wf
(1)
-
Prof. T Szymanski 4DN4 - WFQ, pg 23
GPS Finishing time: Non-Empty Queue
" If packet k from flow f arrives at a nonempty queue, it must
wait for all packets ahead of it (in particular packet k-1 from
flow f) to be serviced before it is serviced.
" The finishing time of the packet k-1 ahead if it is F(f, k-1).
" The current packet needs P(f,k)/Wf additional rounds of
service,
therefore, the finishing time of this packet is clearly:
F(f,k) = F(f,k-1) + P(f,k)/Wf (2)
Prof. T Szymanski 4DN4 - WFQ, pg 24
Comparing Both Finishing Times:
" Recall the finishing times using GPS fluid-flow fair queuing,
when all flows have the same weight:
F(f,k) = max{F(f,k-1), R( t(f,k) } + P(f,k)
" When each flow has its own weight, the equation for the
finishing times using GPQ fluid-flow fair queueing becomes:
F(f,k) = max{F(f,k-1), R( t(f,k) } + P(f,k)/Wf
" Each flows weight determines the flows share of the
bandwidth.
-
Prof. T Szymanski 4DN4 - WFQ, pg 25
Example " Consider 3 flows being serviced by a link with
capacity C = 1 Mbps (ie 1 usec
per bit). Assume each flow has its own queue with weights 5, 10,
and 20 (bits/round).
" Flow 1: pkt with 100 bits arrives at time t = 0 usec, weight =
5 " Flow 2: pkt with 300 bits arrives at time t = 50 usec, weight =
10 " Flow 3: pkt with 200 bits arrives at time t = 100 usec, weight
= 20 " " (a) Plot the # Queued bits versus the round, for each
queue in a fluid-flow
GPS queueing system. (A real router does not need plot this. We
ask this question to make sure you understand why the VFT equations
are valid.)
" (b) Plot the # Queued bits versus the real-time, for each
queue in a fluid-flow GPS queueing system. (A real router does not
need to plot this. We ask this question to make sure you understand
why the VFT equations are valid.)
" (c) Label the slope of each piecewise linear segment. Label
the (x,y) coordinates of each end point of a piece-wise linear
segment
Prof. T Szymanski 4DN4 - WFQ, pg 26
Example
" (d) What are the departure times of the last bit in each flow,
first expressed in Rounds (Virtual-Time), and then expressed in
Real-Time ?
" (e) Now consider a packet-by-packet WFQ approximation of the
fluid-flow GPS system. Only one whole (entire) packet is
transmitted at a time.
" Find the start time and end time for each packet in each flow,
expressed in real-time. (A real router needs to know these
numbers.)
-
Prof. T Szymanski 4DN4 - WFQ, pg 27
Solution Summary
Flow
Bits Weight Arr. Time * (usec)
Arr. Time (round)
Exit Time (round)
ExitTime* (usec)
1 100 5 0 us 0 30 320
2 300 10 50 us 10 40 600
3 200 20 100 us 14 24 440
4 Nill
Assume new flows arrive at the beginning of a round, for
simplicity.
Prof. T Szymanski 4DN4 - WFQ, pg 28
Bits vs Virtual Time (Round)
This slide helps explain the GPS equations.
The bits in a GPS queue drain at a smooth rate, when the x-axis
is the round of service
-
Prof. T Szymanski 4DN4 - WFQ, pg 29
Bits vs Real Time
This slide helps explain the GPS equations.
The bits in a GPS queue drain at a variable rate, when the
x-axis is real-time.
That is why we use the virtual time in the GPS equations It is
easier to compute the virtual finishing times using virtual time
(round), than real-time.
Prof. T Szymanski 4DN4 - WFQ, pg 30
Queued Bits vs Real-Time, WFQ
A real WFQ server in a router implements this behaviour:
The green packet arrives at time 100 usec, just as the red
packet finishes transmission. It has a smaller VFT than the blue
packet (see slide 28), so the WFQ server sends the green packet
before the blue packet, even though the blue packet arrived before
the green packet.
-
Prof. T Szymanski
What Does a Real Router Do?
4DN4 - WFQ, pg 31
" A real router must implement the 4 GPS equations, to compute
the virtual finishing time of each packet that arrives at the
router.
" A router must have 2 clocks, a real-time clock, and a
virtual-time clock (i.e., the current round of service).
" Every time a packet arrives at the router, the router must
compute the packets VFT in rounds (virtual time).
" A real router has a WFQ server, which transmits outgoing
packets in the order of increasing packet VFTs, so that packets
leave the real router in the same order as if a perfectly fair
fluid-flow GPS server was used.
Prof. T Szymanski 4DN4 - WFQ, pg 32
Jitter Comparison, TCP vs WFQ
Compare: TCP vs WFQ traffic sources, generating packets at a
average rate of 50 Mbp, for a 1 Gbps transmission line.
A TCP source could generate bursts of packets, at the full
transmission rate of 1 Gbps, every time a sequence of ACKs is
received (roughly every round-trip time): bursts could occur 1/20
of the time, idle periods could occupy 19/20 of the time, and if
the network is not too loaded, the bursts will travel without much
dispersion => potentially overloading queues
-
Prof. T Szymanski 4DN4 - WFQ, pg 33
Inter-Departure Time Distribution, TCP vs WFQ
" Assume packets are fixed-size; the TCP departures vs. Time
will be bursty:
" An unconstrained TCP source produces bursts of packets(red),
consuming the full 1Gbps bandwidth for the duration of the
burst.
" Bursts also be caused by the uneven delays in the reception of
ACKs.
" TCP can cause instantaneous congestion at intermediate nodes ,
causing potential dropping of packets when Random Early Detection
(RED) is used, introducing wasted bandwidth and resource usage.
Prof. T Szymanski 4DN4 - WFQ, pg 34
Departure Time Distribution , TCP vs WFQ
! The WFQ jitter is much less bursty ! With WFQ, each flow has a
fair share of the 1 Gbps bandwidth. At a
50 Mb/sec sustained rate, WFQ packet departures are evenly
spaced, with one packet in every 20 packet time-slots, followed by
19 empty time-slots.
! The traffic is much less bursty and will reduce/eliminate the
instantaneous congestion at intermediate routers that TCP bursts
may experience
-
Prof. T Szymanski 4DN4 - WFQ, pg 35
Commercial Implementations
! Consider a national network: ! WFQ can be used per individual
flow. ! Very expensive when used for individual flows, since we may
have over
1 billion flows for a national backbone network.
http://nsssc.superb.net/img/algx-usmap.gif
Prof. T Szymanski 4DN4 - WFQ, pg 36
Class-Based Weighted Fair Queuing [2]
! A commercial implementation of WFQ is Class Based Weighted
Fair Queuing (CBWFQ) , used by Cisco Systems
! CBWFQ extends the WFQ functionality by providing support for
user-defined traffic classes instead of individual flows.
! IP doesnt have any easy way to specify flows. Therefore,
classes are defined by a match criteria on packet headers which
include:
! Protocols (TCP, UDP, RTP, etc) ! IP addresses ! TCP and UDP
ports
! A queue is reserved for each traffic class. Incoming packets
are filtered and IP packets that meet the match criteria for a
class are placed in their corresponding class queue. Network
management software allows for IP routers to specify class-match
criteria (class filters)
-
Prof. T Szymanski 4DN4 - WFQ, pg 37
Class-Based Weighted Fair Queuing
! After defining a class according to its filter, the
characteristics of the class must be assigned.
! Class characteristics include: ! Bandwidth guaranteed amount
to be delivered to a class
determined by class weight.
! Maximum Packet limit maximum amount of packets allowed to
accumulate in a queue for that class.
! The weight for each packet is derived from class weight and
CBWFQ uses the class weights to ensure that each class is serviced
fairly.
! More information available at Cisco Systems website:
www.cisco.com/univercd/cc/td/doc/product/software/ios120/120newft/120t/120t5/cbwfq.htm
Prof. T Szymanski 4DN4 - WFQ, pg 38
Other scheduling methods
" There are many other proposed fair queue scheduling
algorithms, and many more variations of Weighted Fair Queueing,
including:
" Deficit Round Robin
" Self-Clocked Fair Queuing " Rate Controlled Service
Discipline
" Worst case fair queuing
" GPS and WFQ define the ideal fair bandwidth-sharing
performance. In practice, one cannot implement fluid-flow GPS, so
an algorithm which approximates the ideal scheme, such as
packet-based WFQ, can be used.
-
Prof. T Szymanski 4DN4 - WFQ, pg 39
Appendix
1. Communication Networks: Fundamental Concepts and Key
Architectures Second Edition. Alberto Leon-Garcia and Indra
Widjaja. McGraw-Hill.
2. Class-Based Weighted Fair Queuing. Cisco Systems Inc.
www.cisco.com/univercd/cc/td/doc/product/software/ios120/120newft/120t/120t5/cbwfq.htm
3. A. Parekh and R.G. Gallager, A Generalized Processor Sharing
Approach to Flow Control in Integrated Services Networks: The
Single-Node Case IEEE/ACM Trans. Networking, vol. 1, no. 3, pp.
344-357, June 1993. (Based upon A. Parekhs PhD Thesis at MIT).
Prof. T Szymanski 4DN4 - WFQ, pg 40
Example (Ignore in 2013)
-
Prof. T Szymanski 4DN4 - WFQ, pg 41
Example (Ignore in 2013)
Prof. T Szymanski 4DN4 - WFQ, pg 42
Example (Ignore in 2013)
-
Prof. T Szymanski 4DN4 - WFQ, pg 43
Example (Ignore in 2013)
Prof. T Szymanski 4DN4 - WFQ, pg 44
Example (Ignore in 2013)
-
Prof. T Szymanski
Some Fun With Matlab
! Lets use Matlab to implement WFQ as a real router would ! Most
Internet routers implement WFQ in real time to achieve some
type of fairness, processing millions of packets per second
! We generally have around a microsecond per packet to compute
the fair GPS virtual finishing times
! Lets (1) generate one flow = random stream of arriving packets
! Lets (2) plot the flow so we can visualize the packets and
their
sizes
! Lets (3) generate 4 flows ! Lets (4) plot the 4 flows on one
graph, so we can see the
competition for bandwidth
! Let (5) figure out how to implement WFQ in a router !
4DN4 - WFQ, pg 45
Prof. T Szymanski
Generate One Flow
4DN4 - WFQ, pg 46
-
Prof. T Szymanski
Plot the Flow on One Graph
4DN4 - WFQ, pg 47
Prof. T Szymanski
The Main program : Generate 1 Flow
4DN4 - WFQ, pg 48
-
Prof. T Szymanski
The Figure : One Flow (Packet Stream)
4DN4 - WFQ, pg 49
Prof. T Szymanski
Generate 4 Flows (Packet Streams)
4DN4- WFQ, pg 50
-
Prof. T Szymanski
Plot All 4 Flows on One Graph
4DN4 - WFQ, pg 51
Prof. T Szymanski
The Result: 4 Arriving Flows
4DN4 - WFQ, pg 52
Now lets use WFQ to achieve fairness in a router.
-
Prof. T Szymanski
Next: In Class Example for WFQ
4DN4 - WFQ, pg 53
! Ill ask the class for ideas on how to use Matlab to implement
WFQ as a real router would
! Lets experiment with the Matlab, and see what we can achieve !
Assignment #2 will cover WFQ