QoS 1 The Access Company QoS Presented by: Yaakov (J) Stein CTO.

QoS 1The Access Company

QoS

Presented by:Yaakov (J) SteinCTO

QoS 2

What am I going to talk about today ?

• The telecommunications service model– SLAs

• QoE and QoS• Soft QoS (DiffServ)

– packet marking (PCP, DSCP)– PHBs – BE, EF, AF– queuing mechanisms (strict priority, WFQ)– specifying datarate, bucketing algorithms (leaky, token)– traffic policing– traffic shaping

• Hard QoS (IntServ)– service levels – BE, CLS, GS– Network Engineering (planning) vs. Traffic Engineering (resource reservation)– RSVP – Routing protocols and RSVP-TE

QoS 3

thetelecommunications

service model

QoS 4

Why do we pay for services ?

Generally good (and frequently much better than toll quality)voice service is available free of charge (Skype, Fring, Nimbuzz, …)

So why does anyone pay for voice services ?

Similarly, one can get free • (WiFi) Internet access• email boxes• file storage and sharing• web hosting• software services

So why pay for any service ?

QoS 5

Paying for QoS

The simple answer is that one doesn’t pay for the serviceone pays for Quality of Service guarantees

In our voice model

But what does QoS meanand why are we willing to pay for it ?

To explain, we need to review some history …

QoS

price

BE

toll qualitywith mobility

QoS 6

Father of the telephone

Everyone knows that the father of the telephone wasAlexander Graham Bell (along with his assistant Mr. Watson)

But Bell did not invent the telephone network

Bell and Watson sold pairs of phones to customers

The father of the telephone network wasTheodore Vail

QoS 7

Father of the telephone network

Theodore Who?• Cousin of Alfred Vail (Morse’s coworker)• Ex-General Superintendent of US Railway Mail Service • First general manager of Bell Telephone• Father of the PSTN

Organized telephony as a service (like the postal service!) *

Why else is he important?• Established principle of reinvestment in R&D• Established Bell Telephones IPR division• Executed merger with Western Union to form AT&T• Solved major technological problems

• use of copper wire• use of twisted pairs

* Vailism is the philosophy that public services should be run as closed centralized monopolies for the public good

QoS 8

What’s the difference ?In the Bell-Watson model

the customer pays once, but is responsible for • installation

– wires– wiring

• operations– power– fault repair– performance (distortion and noise)

• infrastructure maintenancewhile the Bell company is responsible only

for providing functioning telephones

In the Vail model the customer pays a monthly feebut the provider assumes responsibility for everythingincluding fault repair and performance maintenance

The telephone company owns the telephone sets and even the wires in the walls !

+

QoS 9

Service Level Agreements

In order to justify recurring paymentsthe provider agrees to a minimum level of service in an SLA

An SLA is a legal commitment between a service provider (SP) and a customer, for example:

• Telco and subscriber• ISP and Internet user • VPN operator and enterprise• cloud application provider and cloud user

SLAs typically include (financial) penalties for breaches

If objectives or penalties are too low, SLA is uselessIf objectives or penalties are too high, cost will be prohibitiveBadly defined SLAs may damage operations by setting incorrect goals

QoS 10

SLAs and QoS parameters

SLAs should capture Quality of user Experience (QoE)but this is often hard to quantify

So SLAs usually actually detail measurable network parameters that influence QoE, such as :

Connectivity parameters• availability (e.g., the famous five nines)• time to repair (e.g., the famous 50 ms)

Noise (error) level parameters• SNR• BER• Packet Loss Ratio• defect densities

Information rate parameters• bandwidth, throughput, goodput

Information latency parameters• 1-way delay,• round trip delay

performance parameters

QoS 11

Connectivity vs. the rest

Basic connectivity (availability) always influences QoE

The other parameters may influence QoEdepending on service/ application (voice, video, browsing, …)

Some services only require basic connectivitySome also require minimum available throughputSome require delay less then some end-end (or RT) delaySome require packet loss ratio (PLR) less than some percentage

Note: these parameters are not necessarily independent

For example, TCP throughput drops with PLR 1000 B packets

50 ms RTT

QoS 12

Some rules of thumb

Mission Critical (and life critical) services require• high availabilityIf there are any MC services

then system traffic requires high availability tooMC services do not necessarily require strict throughput

but always indirectly require • a certain minimal average throughput • bounded delay

If the MC service uses TCP then it requires • low PLR

Real-time services require• sufficient throughputbut not necessarily low PLR (audio and video codecs have PLC)

Interactive services require • low RT delay

QoS 13

QoS monitoring

RECAP: SLA compliance is the SP’s justification for payment

To ensure SLA compliance, the SP must : • monitor the SLA parameters• take action if parameter is dropping below compliance levels

But how does the SP verify/ensure that the SLA is being met ?

Monitoring is carried out usingOperations, Administration, Maintenance (OAM)

The customer too may use OAM to check that the SP is compliant !Technical note:OAM is a user-plane function

but may influence control and management plane operationsfor example• OAM may trigger protection switching, but doesn’t switch• OAM may detect provisioned links, but doesn’t provision them

QoS 14

OAM – FM and PM

The difference between connectivity and performance parametersleads to two types of OAM :

1. Fault Monitoring required for maintenance of connectivity (availability)– detection and reporting of anomalies, defects, and failures– OAM runs continuously/periodically at required rate– used to trigger mechanisms in the

• control plane (e.g. protection switching) and • management plane (alarms)

2. Performance Monitoring required for maintenance of all other QoS parameters– measurement of performance criteria (delay, PDV, etc.)– OAM run :

• before enabling a service• on-demand or • per schedule

QoS 15

QoS assurance : availability

The difference between connectivity and performance parametersleads to 2 types of QoS assurance – availability and performance

Availability is usually specified in “nines”

In order to ensure high availability, one employs• FM OAM• Automatic Protection Switching (APS)

nines up % permitted down time typical service

3 nines 99.9% < 7 hour 18 min / month electric power service

4 nines 99.99% < 44 minutes / month

5 nines 99.999% < 4 min 23 sec / month PSTN

6 nines 99.9999% < 26 sec / month

QoS 16

QoS assurance : performance

There are two main approaches to ensuring performance QoS

IntServ (guaranteed QoS) – hard QoS– define traffic flows (CO approach)– guarantee QoS attributes for each flow– reserve resources at each router along the flow– signaling protocol (e.g., RSVP) needed

DiffServ (statistical QoS) – soft QoS– retain CL paradigm– no guaranteed QoS attributes– no resource reservation– mark packets (differentiated – e.g., gold, silver, bronze)

• marking can be by VLAN, P-bits, IP-ToS/DSCP, or general “flow”– offer special treatment (priority) relative to other packets

DiffServ is the preferred approach for Ethernet and IPIntServ is used in MPLS-TE

QoS 17

QoE

QoS 18

QoE and MOS

ITU-T defines QoE as the acceptability of a service

as perceived subjectively by the end-user

A well-known QoE measure for telephony-grade voice is Mean Opinion Score (MOS) (ITU-T P.800)

MOS is measured by having a number of listeners listen and score speech on a scale from 1 (bad) to 5 (excellent)and averaging over these scores (finding the mean)

• Toll quality voice has MOS = 4• Cellphone voice has MOS 3.5• Synthetic or military voice has MOS = 2 and below

QoS 19

QoE and QoS

Theory - QoE for a given application is a function of QoS parameters

QoE = f (service; QoS1, QoS2, … QoSn)

Researchers have found various functional forms for the dependence of QoE on a particular QoS parameter

see e.g., work of Markus Fiedler (BTH, Sweden)

form expression examples

Linear QoE QoSk perceived download time vs. PLR

Logarithmic QoE log(QoSk) perceived download time vs. datarate

Exponential QoE exp(QoSk) VoIP MOS vs. PLR

Power Law QoE QoSkp perceived streaming video quality vs. PDV

QoS 20

Absolute vs. Comparative QoE

QoE measures may beabsolute determined by observing the degraded message orcomparative determined by comparing the degraded message to the original

Comparative measures are often more accuratebut can not be used unintrusively on a live network scenarios

Absolute measures can be used single-ended (non-intrusively)

MOS variations Absolute Category Rating (ACR) : listeners hear only the degraded speechDegradation Category Rating (DCR) : listeners hear first the original and then the

degraded speech and score 1 = very annoying degradation … 5 inaudible degradationComparative Category Rating (CCR) : listeners hear the original and the degraded

speech in random order and score -3 2nd is much worse than 1st ... 3 2nd is much better than 1st

even simpler : AB test – simply report which sounds better

QoS 21

Subjective vs. Objective QoE

Direct human QoE scoring is expensive and time-consumingITU-T has defined objective measures that can be automatedThese entail algorithms that produce scores

that correlate well with human QoE

PSQM (ITU-T P.861) and PESQ (ITU-T P.862) are objective comparative MOS-like measures for telephone grade speech

They model the human auditory perception system (Bark scale, masking, etc.)

PEAQ (ITU-R BS-1387) similarly scores wideband audioThese were selected in competitions to have highest correlation with human MOS

ITU-T P.563 is a single-ended (absolute) objective MOS-like score It determines un-naturalness of telephone-grade speech sounds

and the amount of non-speech-like noise

QoS 22

PSQM processing

T1207950-96

x [n]

FFT

xwi[n]

y [n]

ywi[n]

FFT

Xi[k] Yi[k]

Pxi[k] Pyi[k]

Px'i[j] Py'i[j]

Si Py"i[j]

PFyi[j]

PHyi[j]

PFxi[j]

PHxi[j]

Lyi[j]Lxi[j]

Ly'i[j]

S1i

Ni[j]

Ni

Nwsil

Hanning window

frequency warping

filter with receivingcharacteristics of

handset

intensity warping

calculatelocal scaling

factor

Hoth noise

calculateloudness

scaling factor

Hanning window

frequency warping

filter with receivingcharacteristics of

handset

intensity warping

Cognitivesubtraction

asymmetry processing

silent interval weighting

QoS 23

E-model

The E-model defined in ITU-T G.107 is a planning tool

It predicts a “mouth-to-ear” transmission rating factor R • between 0 and 100• higher values signify better voice quality • should be uniquely convertible to a MOS level

R = f(QoS1, … QoSn) and is additive in individual QoSk degradations

R starts with the basic signal to noise ratioR is reduced to account for various impairments, including • simultaneous impairments (loudness, sidetone, clipping, quantization noise)• delay impairments (delay, echo delay and loudness) • equipment impairments (codec distortion, packet loss)

R is increased when there are additional advantages such as mobility (cellphone receives A=10)

R = R0 – Is – Id – Ie + A

QoS 24

R value meanings

R values meaning Equivalent MOS

90 - 100 Very satisfied 4.3-5.0

80- 90 Satisfied 4.0-4.3

70-80 Some users dissatisfied 3.6-4.0

60-70 Many users dissatisfied 3.1-3.6

50-60 Nearly all users dissatisfied

2.6-3.1

Below 50 Not recommended 1-2.6

QoS 25

VQMON

Years before P.563 ETSI specified VQmonTIPHON (Telecommunications and Internet Protocol Harmonization Over Networks) TS

101 329-5 Annex E

VQmon (developed by Telchemy) is a single-ended method for estimating the E-model factors for VoIP audiobased on QoS parameters (packet loss statistics, delay)

Depends on codec typeTakes human perception phenomena into account (e.g., recency effect)

VQmon was later extended to • audio (MOS-A)• video (MOS-V)• audio-video (MOS-AV)

QoS 26

Video quality

ITU-R produced BT.500 for subjective assessment of TV quality

Similar to MOS :• television sequences are shown to a group of viewers• subjective opinions are averaged

ITU-T has produced many Recommendations for video and multimedia quality :

• Subjective (P.9xx, J.140)• Objective (J.143, J.144, J.147, J.148, J.24x, J.34x)

Since 1997 the Video Quality Experts Group (VQEG)has been producing standards and tutorials

PEVQ (J.247) is a comparative pixel by pixel objective measurethat models the human visual tract

and returns a 5-point MOS score and further KPIs

QoS 27

QoE for other applications

G.1011 is a reference guide to existing standards for QoEand provides a taxonomy

G.1010 discusses many applications, including• conversational voice, voice messaging, streaming audio• videophone, one-way video• web-browsing, bulk data transfer, email, e-commerce, • interactive games• SMS, instant messagingand gives performance targets for delay, PDV, and PLR

G.1050 gives an IP network model for evaluating the performance of IP streams based on QoS parameters (delay, PDV, PLR).

J.163 treats real-time services over cable modems

X.140 defines QoS parameters for public data networks

QoS 28

Network planning tools

In addition to subjective/objective methods to quantify the QoEof a specific (live or simulated) service instance

Network planners need tools to predict service qualityin order to efficiently allocate resources

G.1030 provides network planners with end-to-end (E-model-like) tools for applications over IP networks

It includes an appendix devoted to web browsingthat presents empirical perception of users to response times and proposes a MOS measure

G.1070 proposes an algorithm for network planners to estimate videophone quality

QoS 29

Useful background documents

ISO 8402 defines quality as :the totality of features and characteristics of a product or service that bears its ability to satisfy stated or implied needs

E.800 Definitions of terms related to QoSG.1000 Communications quality of service: A framework and definitions

ETSI ETR 003 General aspects of QoS and Network Performance (NP)

* Note – terminology in these documents is outdated

QoS 30

TR-126

The Broadband Forum (BBF) has produced TR-126, which includes• a tutorial on QoE • guidelines for QoE vs. QoS for triple play applications

TR-126 also discusses :• QoE dimensions: service set-up, operation, and tear-down• QoE facets: user effort, application responsiveness• information fidelity, security, and dependability/availability;• localization of QoE contributions (access, ISP, application SPs)

Guidelines are given for :• video (conferencing, surveillance, streaming)• voice (wired, wireless, voice messaging, IVR)• best-effort Internet data (browsing, email, file transfer, VPN, P2P, ecommerce, …)

QoS 31

TMF

The TeleManagement Forum (TMF) discusses QoE SLA management TMF’s Wireless Services Measurement Handbook GB923 defines :• Key Quality Indicators (KQIs) (like QoE scores)• Key Performance Indicators (KPIs)

KQIs may be determined from KPIs (the mapping may be complex) KPIs are derived from QoS parameters

TMF has defined a set of KQIs including :• response time• service availability• speech/video quality• transaction rate• offered throughput

An SLA consists of a set of KQI and KPI thresholds (see SLA Management Handbook GB917 and its Application Notes)

QoS 32

Apdex

The Apdex Alliance is a consortium of companies functions as a IEEE-ISTO (Industry Standards and Technology Organization) program

Apdex develops open standardized methods to • report• benchmark and • track application performance.

The Apdex (Application Performance Index) • is a number between 0 and 1 • is meant to capture user satisfaction from an application • 0 means no user would be satisfied• 1 means that all users would be satisfied

QoS 33

Apdex (cont.)

To compute the Apdex N users are divided into 3 categories• satisfied (S users) e.g., web page completely loads within 2 seconds• tolerating (T users) e.g., web page completely loads within 8 seconds• frustrated (F users) e.g., web page takes > 8 seconds to load

The Apdex is given by Apdex = ( S + T/2 ) / N

Apdex hierarchically deconstructs application transactions into sessions processes tasks turns protocols packets

Sessions consist of the entire connect timeProcesses are interactions accomplishing a goalTasks are individual interactions

The user is mainly aware of the task response timesince must wait for the task to complete before proceeding!

QoS 34

Behavioral QoE

All of the above subjective and objective QoE measuresare service/application-specific.

But new services and applications are created every dayand different users use different features of a single application

So it is no longer feasible to study each application in depth

A new approach is behavioral QoE estimationthe user’s satisfaction is estimated based on actions / reactions

Example : there is a high measured correlation between a user being unsatisfied with a service levelhis aborting the application (or at least waiting until the service level improves)

Behavioral QoE can be used instead of traditional QoE measurementor to automatically find QoE(new app, QoS)

QoS 35

soft QoS

QoS 36

Queuing theory

Why isn’t the traffic distribution problem simple ?If we have available data rate A,

simply allow traffic from all sources that sums to AWrong !Even if the average rates sum to much below A

there may be peak rates that exceed AIn order to accommodate peaks, we insert a queue

Customers/packets/whatever wait in queue to be serviced

Queue behavior can be counter-intuitive Problem 1 :

two buses arrive at my bus stop every 10 minutesI take the first bus that arrivesWhy do I take bus A much more than bus B ? (hint : correlation)

Problem 2:buses leave their first stop every 10 minutesand pick up passengers at intermediate stopsWhy do the buses bunch ?

Squeuearrivals

QoS 37

Scheduling disciplines

When there is a single queue, service order is usually First In First Out (FIFO) AKA First Come First Servedbut may be Last In First Out (LIFO/stack), random order, etc.

When there are multiple queues packets belonging to a particular flow are consistently mapped to a single queuethere may be one or more flows in each queue

Service order may be :• Round Robin : each queue visited in order• Strict Priority : take from non-empty queue of highest priority• Fair Queuing : preserve average datarate from each queue• Weighted Fair Queuing : fair queuing with priority• hybrid : mixture of several disciplines

QoS 38

ErlangIn early 1900s telcos needed to calculate the number of

switches, lines, and operators they needed per call volume • too little and subscribers would be unhappy• too much wastes labor and moneySame problem for customers in stores, cars at traffic lights,

manufacturing processes, call centers, etc.

Agner Krarup Erlang developed queuing theory to solve this problem for the Copenhagen telephone exchange

The unit of traffic use is called the Erlang in his honor1 Erlang is 1 channel being used 100%, or 2 channels used 50%

when averaged over some time (generally an hour)

There is also a functional programming language(used by Ericsson for telephony applications) named after him

QoS 39

Kendall notationLet’s callA – the statistics of arrival times (customers / packets / whatever)

B – the statistics of service timesC – the number of servers (there may be only 1 server)K – the maximum queue length (if too many arrivals, need to drop)

Then we can describe a queuing system by A/B/C/Kand if K= then we call it A/B/C

Important statistics types:

Example:M/M/1 is a queue with a single server

and Poisson distributed arrivals and service times

D Deterministic distribution (often fixed intervals)

M Markov process (Poisson (exponential) distribution)

E(k) Erlang distribution with parameter k

QoS 40

Queuing theory results

Queuing theory derives important values, such as• waiting times• number of customers/packets waiting• number of customers/packets being processed

We will not develop queuing theory here

Some models are completely understood (M/M/1, M/M/K, M/D/1)

Some formulae are true in general

Little’s law L = λWL : long-term average number of customers in a queue λ : long-term average arrival rateW : average time a customer (waits) in the system

This law is true for any arrival distribution, service distribution, number of servers, service order, etc.

QoS 41

Recap: soft QoS

Soft QoS does not provide any hard service level guaranteesIt merely breaks fairness by giving priority to certain

users or applications or flows or individual packets

When there aren’t network resources to forward all packets packets are forwarded in order of priority from highest to lowest

Low(er) priority packets may be delayed or discarded(when K < )

In order to correctly prioritorize packetsthey need to be priority marked

Marking is best accomplished by the originatorbut may need to be performed by an intermediate elementbased on port, or header fields, or even DPI

QoS 42

Marking Ethernet

DA (6B)

SA (6B)

ET=8100 (2B) P(3b) CFI(1b) CVID(12b)

ET=88A8 (2B) P(3b) DEA(1b) SVID(12b)

ET (2B)

VLAN ID (VID) indicates priority

In addition, for VLAN tagged framesPriority Code Point (PCP) AKA user priority field, P-bits

•3 bits so takes values 0 … 7•monotonically increasing priority•can use priority tagging (VLAN=0) if no VLAN•P=0 means non-expedited traffic•802.1Q gives recommends mappings

QoS 43

Marking MPLS

Only top label is relevantLabel can indicate priority (L-LSP)or TC field (previously called EXP field, previously called COS field) (E-LSP)• 3 bits so takes values 0 … 7• no recognized TC value meanings

Top Label (20b) TC(3b) S(1b) TTL (8b)

Label (20b) TC(3b) S(1b) TTL (8b)

Bottom Label (20b) TC(3b) S(1b) TTL (8b)

QoS 44

Marking IPv4

The IPv4 header has a Type of Service (ToS) fieldRFC 2474 redefined ToS to consist of • 6 bit DSCP (see also RFC 4594)• 2 bit ECN (least significant bits)Guidelines for use of DSCP in many documents

Ver(4b) IHL(4b) ToS(1B) Len(2B)

Source IP Address (4B)

Destination IP Address (4B)

. . .

QoS 45

Marking IPv6

The IPv6 header has a Traffic Class (TC) fieldRFC 2460 states that it is to be used like the IPv4 ToS field

The Flow Label (intended to ensure flow of packets follow the same path) may be indirectly used

Ver(4b) TC(1B) Flow Label (20b)

Source IP Address (16B)

Destination IP Address (16B)

payload len(2B) next(1B) hop(1B)

QoS 46

IP DiffServ methods

DiffServ was developed in IETF after IntServ RFCs 2474, 2475 because• IntServ was considered too heavy-weight for most purposes• resource reservation is against IP-philosophy

if not enough BW, then more democratic for all to sufferif reserve BW and don’t use, then this is simply over-provisioning

DiffServ is evolutionary “coarse-grained” approach to IP QoS

DiffServ – divides traffic into service classes

and allocates resources on a per-class basis– uses 6 bits of ToS byte in IP header to mark packets

• field is renamed Differentiated Services Code Point • no setup or router state required

– DSCP defines per-hop behaviors (PHB)• tells router how to treat packet

– three standard PHBs (BE, AF, EF) but you are free to create more

QoS 47

DiffServ Per Hop Behaviors (PHBs)

Best Effort– standard IP service– QoS depends on momentary network load

Assured Forwarding– AF specifies class that determines queue– in addition, three drop-precedence levels (low, med, high)– AF packets from a single source should not be mis-ordered even if have different drop-precedence (i.e. single queue)

Expedited Forwarding– EF packet should experience no queuing delays– EF packets should have low loss– implemented by dedicated EF router queue

WARNING: DiffServ does not provide true assurances

QoS 48

What about MPLS ?

MPLS DiffServ - each FEC is defined by– destination address– service class

L-LSP (Labeled-inferred LSP)– behavior based on label alone– support different service classes by using different labels– LSP BW allocated from specific queue (class)– TC field may be used for drop precedence

E-LSP (EXP-inferred LSP)– behavior based on label + TC (ex-EXP) field– TC bits in MPLS shim header are similar to DSCPs, but

only three bits (like P-bits) while DiffServ ended up with 6 bits– but 8 service classes is usually more than enough commonly 4 classes are offered (bronze, silver, gold, platinum)– LSP BW allocated from link

QoS 49

Queuing in switches/routers

Switches and routers have queues FIFO bufferson each output port

If there were only one queue then traffic handling would be FCFS

To enable DiffServ prioritization multiple queues are used

Outgoing frames are inserted into queues according to priority marking

Queues are emptied according to scheduling discipline

(strict priority, WFQ, etc.)

switchfabric

input port

input port

input port

output port

output port

output port

output portqueue

queuequeuequeue

QoS 50

Traffic conditioning

One of the most important parts of an SLA is theCommitted Information Rate (bps)

This is the datarate (bandwidth) SP tries to forward

There may also be an Extra Information Rate (bps)

This is a datarate that the SP will forward if possible

A customer who did not send data for a whilewill expect to be able to send a higher rate afterwards

Enforcement of these rates is accomplished via traffic conditioningThree strategies :• policing (rate limiting, throttling)• shaping (in ATM – GCRA)• metering

WARNING: DiffServ does not provide true commitments

QoS 51

Traffic policing

Traffic exceeding the committed datarate is immediately discarded(or at least marked as discard eligible)

CIRtime

CIRtime

policed to CIR

QoS 52

Traffic shaping

Traffic exceeding the committed datarate is delayed until it can be forwarded (placed or remains in a buffer)

(discarded only if rate exceeds CIR for an extended time)

CIRtime

CIRtime

shaped to CIR

QoS 53

Traffic metering

Charge extra for traffic exceeding the committed datarate(leads customer to self-police)

CIRtime

CIRtime

extra charge over CIR

QoS 54

BW specification

What does an SLA commitment of X bps mean ?BW usage naturally varies for many servicesMust the customer send < X bps at all times

even if he transmitted much less than X up to now ?May the customer remain silent for 9 minutes

and then send 10X bps for 1 minute ?If the measurement interval is 10 minutes

then this is precisely X bps !

A BW cap is only meaningful when we specify the integration time

Or, we can specify the rate and the maximum burst size (in bytes)and enforce these using a bucketing algorithm

A bucketing algorithm allows bursts above X for a limited timeas long as the average remains at X or below

QoS 55

Leaky and token buckets

Leaky bucket model water is poured into bucket as neededwater leaks out at a constant rateif too much water poured in it overflows

Interpretationbps of traffic are added to bucketcommitted rate is continuously removedif packet fits into bucket it is sentunused data rate is lost

Token bucket model water is poured into bucket at a constant ratewater is removed as neededif too much water poured in it overflows

Interpretationtokens are added at committed rateto send traffic there have to be enough tokensunused data rate is lost

QoS 56

How does a token bucket work ? Part 1

Let’s look at a token bucket policer (other cases are equivalent)

The bucket is configured withheight - Committed Information Rate (CIR)filling rate - Committed Burst Size (CBS)

If packets are sent at precisely the committed rate– the bucket height stays constant– and all packets are forwarded

If packets arrive at less then the committed rate– the bucket height increases– all packets are forwarded– excess information rate overflows and is lost

If packets arrive at more then the committed rate– the bucket height decreases– when no tokens are left packets are discarded

CBS

CIR

continued

Note:Some people complicate formulas by specifying CIR in bps and CBS in BytesSuch people should be committed

QoS 57

How does a token bucket work ? Part 2

If no packets have been sent for some timeand then CBS worth of packets are sent– the bucket is initially full of CBS tokens– the tokens are all removed– all packets are forwarded

If more than CBS information rate in burst– the first CBS of packets are forwarded– the rest are discarded until new tokens arrive

Note: adding of tokens can be in discrete time - every T (e.g., 1 sec) the token are addedcontinuous time – tokens are continuously added (in practice, when new packet arrives, calculate number of tokens added since the last packet) for continuous time, the maximum burst size is larger than the configured CBS

CBS

CIR

QoS 58

Dual token buckets

Sometimes SPs sell (and customers purchase) Extra trafficThis is a rate above the committed rate

that the SP will forward if it can (but doesn’t commit to forward)

Extra traffic is priced much lower than committed rate traffic

To handle Extra traffic, we use two (token or leaky) buckets, C and Ethe C bucket is of height CBS and is filled at rate CIRthe E bucket is of height EBS and is filled at rate EIR

continued

CBS

CIR

EBS

EIR

C E

QoS 59

Dual token buckets (cont.)

Furthermore, we classify packets asgreen if passes C bucket test (green packets are forwarded)

yellow if fails C bucket test, but passes E bucket test(yellow packets may be forwarded, but SLA objectives don’t apply)

red if fails both bucket tests (red packets are always discarded)

More precisely :

if ingress traffic < number of tokens in C bucketframe is green and its length in tokens is debited from C bucket

elseif ingress frame length < number of tokens in E bucket

frame is yellow and its length of tokens is debited from E bucketelse frame is red

QoS 60

More token bucket variations

As if this isn’t complicate enough …MEF added two more twists – coupling and sharing

Unused rate is not lost – it is coupled or shared !

coupling

sharing

coupling

prioritysharing

QoS 61

hard QoS

QoS 62

CO vs. CL networks

To guarantee QoS (and thus QoE) we need to• find path through network that can provide the needed QoS• reserve resources along this path to guarantee the QoS• not accept flows for which there are insufficient resources (CAC)• optionally – optimize path placement (to maximize number of flows that can be accommodated)

ATM and (some) MPLS networks are Connection Oriented (CO), thus• we specify (or at least know) the path the packets will take• we can reserve resources at the network elements along the path

Standard IP networks are ConnectionLess (CL), thus• it is hard to ensure packets will go where we want them to • it is meaningless to reserve resources

as we don’t know which network elements a packet will traverse

Thus hard QoS is not easy to add to IPwhich is why hard QoS is not popular in pure IP networks …

QoS 63

Network and Traffic Engineering

Network Engineering (planning)putting the bandwidth where the traffic is– physical cable deployment (thick pipes)– over-provisioning and backup connection provisioning– does it violate provider objectives ?

Traffic Engineering (TE)putting the traffic where the bandwidth is– explicit traffic routing– route optimization– can it meet user objectives?

QoS 64

Traffic Engineering (TE)

TE is control of network traffic to achieve specific objectivesunfortunately users and providers have contradictory objectives

user objectives (QoS)– network availability– packet loss– end-to-end delay– round-trip delay– packet delay variation (PDV)– error rate

provider objectives (CAPEX, OPEX)– bandwidth utilization– resource utilization– speed of failure recovery– ease of management– monetary outlay

QoS 65

Simple example - fish diagram

In the above example, there is sufficient BW for all traffic(were these ATM switches, 1G over ACDG, ½ over

BCEFG

Without TE, can’t use all the physical bandwidth!

standard IP routing : minimum hop count is CDG, so all traffic flows there 1½ G over 1G link, so ½G is dropped !with administrative cost can force all traffic to go CEFG - which is worse !with ECMP half (750M) goes CDG and half (750M) goes CEFG – still drops !

A

B

C

D

E F

Gall links 1G except EF ½ G1G

½ G

QoS 66

Constraint-based routing

IP uses distributed routing protocols, not centralized management

Distributed protocols are very good at finding basic connectivityand minimizing an additive metric (e.g., hop count)but are not good at optimally utilizing network resourcesor obeying constraints

Common constraints include :• explicit include/exclude links/routers (local constraint)• conform to link BW constraints (local inequality constraint)• meet end-end delay / PLR objectives (global inequality constraint)

Routing that takes constraints into account is called constraint-based routing (CR)

After CR finds an acceptable pathwe may need to set it up (reserve resources) using another protocol

QoS 67

OSPF-TE and IS-IS-TE

Constraint-based routing needs link attribute information most importantly - available BW

Link-state protocols have mechanisms to flood link-up/link-downto every router in the domain

We can piggyback attributes as TLVs on these messages– OSPF add to Link-State Advertisement (LSA) (RFC 3630)– IS-IS add to Link-State Packets

and the routing protocol builds an extended “TE” RIB

When attribute information changes need to reflood informationTo decrease overhead:

– only flood when change passes threshold– inherent timing bounds

Note: CR routing can be NP-hardStandards don’t include (proprietary) efficient algorithms

QoS 68

IP IntServ

IntServ is an overall QoS architecture (not just RSVP)initially developed for VoIP QoS

IntServ is a radical departure from pure IPand requires IntServ-enabled routers

IntServ– enables providing end-to-end QoS guarantees– defines flows (introduces CO to IP’s CL architecture)

flows are classified into three service classes (BE,CLS,GS)– specifies admission control and policing– like all CO architectures, requires signaling protocol (RSVP)

IntServ-enabled routers– reserve needed resources along the flow’s path– must retain state

RFCs 2205-2216, 2379-2382

QoS 69

IntServ CoS levels

Best Effort– standard IP service– QoS depends on momentary network load

Controlled Load Service– service equivalent to unloaded network– low packet loss– most packets will experience delay close to minimum– no quantitative guarantees

Guaranteed Service– bounded worst case delay (no PDV guarantee)– low packet loss (zero if node buffers correctly provisioned)– quantitative guarantees

QoS 70

RSVP

The primary signaling protocol for IntServ is Resource reSerVation Protocol (RSVP)

RSVP protocol• runs between hosts and routers• runs over raw IP or UDP/IP• is unidirectional• does not find path (fed by routing protocols)• sessions identified by source and destination socket numbers• requests unidirectional QoS characteristics from network• causes routers along path to reserve link and node resources• network responds with success/failure• reservations are soft-state - time-out unless refreshed

two main message types: PATH and RESV

RSVP

QoS 71

RSVP Messages

PATH– message from sender to receiver(s)– carries classification info and TSpecs

RESV– response of receiver to PATH message– carries session ID and RSpec specifying QoS required– contains the actual request for resource reservation

receivers

sender

QoS 72

MPLS TE

MPLS-TE protocols enable Fast ReRouteto guarantee fault recovery (connectivity assurance)

Also, MPLS FECs can take QoS constraints into account

MPLS-TE LSPs can be setup according to constraints– include/exclude specific LSRs (for any reason)– only include in LSP LSRs with sufficient available BW– only include in LSP LSRs that guarantee sufficiently low delay

OSPF-TE or IS-IS-TE can be used to get needed network information

But how can the path be set-up ?

Vanilla LDP has no TE capabilitiesand its extension CR-LDP is now obsolete

The answer is a set of extensions to RSVP called RSVP-TE

Unlike RSVP, this protocol runs only between routers (LSRs)

QoS 73

RSVP-TE

RSVP-TE (RFC 3209 …) is a label distribution protocol

• For downstream-on-demand binding• Creates and distributes bindings between RSVP flows and labels• Uses labels instead of source and destination socket numbers• Extends RSVP by adding new objects (e.g. label) and procedures• Allows strict/loose explicitly routed LSPs• Has peer discovery, label requests, binding messages (like LDP)• Transparent transport of QoS and traffic parameters

in TSpecs and RSpecs • Although between routers - still soft state !

Note: RSVP-TE is frequently used to set up FRR alongside LDP

QoS 74

RSVP-TE LSP setup procedure

Example setupwith explicit routes

• A sends PATH message to B w/ explicit route BC and resource requirements

• B forwards PATH message to C after changing explicit route to C• C determines required resources, reserves,

locally binds label and sends RESV to B• B matches, reserves resources, remotely binds,

and sends RESV to A• A matches, remotely binds label and reserves resources

A CBingress egress

QoS 75

PCE

To optimally utilize network resources (e.g., link BW), we needto gather all the topology and network constraints into one centralized

Traffic Engineering Database (TED) enough computational power to solve the complex optimization problemto send path set-up commands to the routers to set up TE-LSP

RFC 4655 defines a Path Computation Element (PCE nicknamed Godbox) and Path Computation Clients (PCCs)

The PCE may be • a designated router or• the management system or • a dedicated computational platform

PCE is an evolutionary solutionto adding computational resources

PCE

QoS 76

PCEP (RFC 5440)

The protocol between PCE and PCCsand between multiple PCEs (if there are)

is called the Path Computation Element Protocol (PCEP)

PCEP runs over TCP with registered TCP port 4189Messages are objects (with common object header) with optional TLVs

PCEP Messages :• Open between PCE and PCC to open a new session • Keepalive optional heartbeat sent if no other PCEP messages• PCReq PCC → PCE to request a path computation • PCRep PCE → PCC with set of computed paths or negative reply• PCNtf event notification from either PCE or PCC • PCErr protocol error message• Close session close

QoS 77

Optimization

How does the PCE compute paths ?

The general path optimization problem is intractable (NP-hard)But there are many combinatorial optimization problems

with known efficient algorithms that return approximate solutions

For example, the (1-dimensional) bin packing problem :Given N values V1 V2 … VN between 0 and 1how can we place them in bins of maximum size 1using the minimum number of bins ?

This problem comes up in many applications :• multiprocessor scheduling• stock cutting• mapping short messages into time slots of ATM cells• mapping CBR flows onto WDM wavelengths• mapping flows onto orthogonal paths

Full PCE problem is even harder !

QoS 78

SDN

An even more radical centralized solution is the Software Defined Network (SDN)

Like PCE, SDN replaces distributed routing protocols with a centralized controllerthat communicates with SDN switches

An SDN switch is a forwarding device that can be programmedto match an arbitrary set of fields in the packetand edit / forward the packet accordingly

SDN switches need not obey standard network layers

The most popular controller-switch protocol is OpenFlow

Google uses SDN switches to optimize utilization in its inter-datacenter network