1 Congestion Control and Traffic Management in High Speed Networks Carey Williamson University of Calgary.

1

Congestion ControlCongestion Controland Traffic Management and Traffic Management

in High Speed Networksin High Speed Networks

Carey Williamson

University of Calgary

2

Introduction Introduction

The goal of congestion control is to regulate traffic flow in the network in order to avoid saturating or overloading intermediate nodes in the network

3

Congestion: EffectsCongestion: Effects

Congestion is undesirable because it can cause:

Increased delay, due to queueing within the network

Packet loss, due to buffer overflow Reduced throughput, due to

packet loss and retransmission Analogy: “rush hour” traffic

4

Congestion: CausesCongestion: Causes

The basic cause of congestion is that the input traffic demands exceed the capacity of the network

In typical packet switching networks, this can occur quite easily when:

- output links are slower than inputs - multiple traffic sources competing

for same output link at the same time

5

Buffering: A Solution?Buffering: A Solution? Buffering in switches can help alleviate

short term or transient congestion problems, but...

Under sustained overload, buffers will still fill up, and packets will be lost– only defers the congestion problem

More buffers means more queuing delay– beyond a certain point, more buffering

makes the congestion problem worse, because of increased delay and retransmission

6

MotivationMotivation

The congestion control problem is even more acute in high speed networks

Faster link speeds mean that congestion can happen faster than before

e.g., 64 kilobyte buffer @ 64 kbps: 8.2 seconds @ 10 Mbps: 52 milliseconds @ 1 Gbps: 0.52 milliseconds

7

Motivation (Cont’d)Motivation (Cont’d)

Buffer requirements increase with link speeds

e.g., to store 1 second worth of traffic

@ 64 kbps: 8 kilobytes @ 10 Mbps: 1.25 Mbytes @ 1 Gbps: 125 Mbytes

8

Motivation (Cont’d)Motivation (Cont’d)

Heterogeneity of link speeds - just because you add new high

speed links to a network doesn’t mean that the old low speed links go away

- interconnecting high speed and lower speed networks creates congestion problems at the point of interconnect

9

Motivation (Cont’d)Motivation (Cont’d)

Traffic is bursty - high peak-to-mean ratio, peak rates - e.g., data traffic: 10-to-1, 1-10 Mbps - e.g., video traffic: 20-to-1, 5-100

Mbps - can statistically multiplex several

channels, but if too many are active at the same time, congestion is inevitable

10

Motivation (Cont’d)Motivation (Cont’d)

Reaction time is bounded by the propagation delay

- in a high-speed wide-area network, the delay x bandwidth product is HUGE!!!

- d x b tells you how many bits fit in the “pipe” between you and the receiver

- by the time you realize that network is congested, you may have already sent another Mbit or more of data!!!

11

Reactive versus Reactive versus PreventivePreventive There are two fundamental

approaches to congestion control: reactive approaches and preventive approaches

Reactive: feedback-based– attempt to detect congestion, or the

onset of congestion, and take action to resolve the problem before things get worse

Preventive: reservation-based– prevent congestion from ever happening

in the first place, by reserving resources

12

Reactive versus Preventive Reactive versus Preventive (Cont’d)(Cont’d) Most of the Internet approaches are reactive schemes– TCP Slow Start– Random-Early-Detection (RED) Gateways– Source Quench

The large d x b product means that many of these approaches are not applicable to high speed networks

Most ATM congestion control strategies are preventive, reservation-based

13

Congestion Control in ATMCongestion Control in ATM

When people discuss congestion control in the context of high speed ATM networks, they usually distinguish between call-level controls and cell-level controls

14

Call-Level ControlCall-Level Control

An example of the call-level approach to congestion control is call admission control (to be discussed later this semester)

Tries to prevent congestion by not allowing new calls or connections into the network unless the network has sufficient capacity to support them

15

Call-Level Control (Cont’d)Call-Level Control (Cont’d)

At time of call setup (connection establishment) you request the resources that you need for the duration of the call (e.g., bandwidth, buffers)

If available, your call proceeds If not, your call is blocked E.g., telephone network, busy signal

16

Call-Level Control (Cont’d)Call-Level Control (Cont’d)

Tradeoff: aggressive vs conservative Want to accept enough calls to have

reasonably high network utilization, but don’t want to accept so many calls that you have a high probability of network congestion (which might compromise the QOS requirements that you are trying to meet)

17

Call-Level Control (Cont’d)Call-Level Control (Cont’d)

Problems: Can be unfair - denial of service, long access delay Hard to specify resource requirements

and QOS parameters precisely - may not know, or may cheat - congestion can still occur

18

Cell-Level ControlCell-Level Control Also called input rate control Control the input rate of traffic sources

to prevent, reduce, or control the level of congestion

Many possible mechanisms: Traffic shaping, traffic policing, UPC Leaky bucket (token bucket) Cell tagging (colouring), cell

discarding Cell scheduling disciplines

19

Congestion Control in ATMCongestion Control in ATM

There is actually a complete spectrum of traffic control functions, ranging from the very short-term (e.g., traffic shaping, cell discarding) to the very long-term (e.g., network provisioning)

See [Gilbert et al 1991]

20

ATM Traffic Control ATM Traffic Control SchemesSchemes

Time Scale

21

ATM Traffic Control ATM Traffic Control SchemesSchemes

Time Scale

ShortTerm usec

22

ATM Traffic Control ATM Traffic Control SchemesSchemes

Time Scale

ShortTerm usec

LongTerm Months, years

23

ATM Traffic Control ATM Traffic Control SchemesSchemes

Time Scale

Cell Time

Usage Parameter ControlPriority ControlTraffic ShapingCell Discarding

PropagationDelay Time

Explicit Congestion NotificationFast Reservation ProtocolNode to Node Flow Control

CallDuration

Admission ControlRouting, Load Balancing

LongTerm

Resource Provisioning

24

ATM Traffic Control ATM Traffic Control SchemesSchemes

Time Scale

Cell Time

Usage Parameter ControlPriority ControlTraffic ShapingCell Discarding

25

ATM Traffic Control ATM Traffic Control SchemesSchemes

Time Scale

PropagationDelay Time

Explicit Congestion NotificationFast Reservation ProtocolNode to Node Flow Control

26

ATM Traffic Control ATM Traffic Control SchemesSchemes

Time Scale

CallDuration

Admission ControlRouting, Load Balancing

27

ATM Traffic Control ATM Traffic Control SchemesSchemes

Time Scale

LongTerm

Resource Provisioning

28

ATM Traffic Control ATM Traffic Control SchemesSchemes

Time Scale

Cell Time

Usage Parameter ControlPriority ControlTraffic ShapingCell Discarding

PropagationDelay Time

Explicit Congestion NotificationFast Reservation ProtocolNode to Node Flow Control

CallDuration

Admission ControlRouting, Load Balancing

LongTerm

Resource Provisioning

29

ATM Traffic Control ATM Traffic Control SchemesSchemes Preventive controls: Resource provisioning Connection admission control Call routing and load balancing Usage parameter control Priority control Traffic shaping Fast reservation protocol

30

ATM Traffic Control ATM Traffic Control SchemesSchemes Reactive controls: Adaptive admission control Call routing and load balancing Adaptive usage parameter control Explicit congestion notification (forward or backward) Node to node flow control Selective cell discarding

31

Leaky BucketLeaky Bucket

One of the cell-level control mechanisms that has been proposed is the leaky bucket (a.k.a. token bucket)

Has been proposed as a traffic policing mechanism for Usage Parameter Control (UPC), to check conformance of a source to its traffic descriptor

Can also be used as a traffic shaper

32

Leaky Bucket (Cont’d)Leaky Bucket (Cont’d)

Think of a bucket (pail) with a small hole in the bottom

You fill the bucket with water Water drips out the bottom at a

nice constant rate: drip, drip, drip...

33

Leaky Bucket (Cont’d)Leaky Bucket (Cont’d)

34

Leaky Bucket (Cont’d)Leaky Bucket (Cont’d)

Bucket

35

Leaky Bucket (Cont’d)Leaky Bucket (Cont’d)

Bucket

Empty

36

Leaky Bucket (Cont’d)Leaky Bucket (Cont’d)

Bucket

Hole

37

Leaky Bucket (Cont’d)Leaky Bucket (Cont’d)

Bucket

Water

Hole

38

Leaky Bucket (Cont’d)Leaky Bucket (Cont’d)

39

Leaky Bucket (Cont’d)Leaky Bucket (Cont’d)

Drip

40

Leaky Bucket (Cont’d)Leaky Bucket (Cont’d)

41

Leaky Bucket (Cont’d)Leaky Bucket (Cont’d)

42

Leaky Bucket (Cont’d)Leaky Bucket (Cont’d)

43

Leaky Bucket (Cont’d)Leaky Bucket (Cont’d)

Constantrate stream ofdrips, all nicelyspaced, periodic

44

Leaky Bucket (Cont’d)Leaky Bucket (Cont’d)

Constantrate stream ofdrips, all nicelyspaced, periodic

Storage areafor dripswaiting to go

45

Leaky Bucket (Cont’d)Leaky Bucket (Cont’d) A leaky bucket flow control mechanism

is then a software realization of this very simple idea

Packets (cells) waiting for transmission arrive according to some (perhaps unknown) arrival distribution

Tokens arrive periodically (deterministically)

Cell must have a token to enter network

46

Leaky Bucket (Cont’d)Leaky Bucket (Cont’d)

Incoming Cells(generated bytraffic sourcewith rate X)

Incoming Tokensat rate r tokens/sec

+ ToNetwork

47

Leaky Bucket (Cont’d)Leaky Bucket (Cont’d)

Incoming Cells

Incoming Tokensat rate r tokens/sec

+ ToNetwork

12345

48

Leaky Bucket (Cont’d)Leaky Bucket (Cont’d)

Incoming Cells

Incoming Tokens

+ ToNetwork

12345

49

Leaky Bucket (Cont’d)Leaky Bucket (Cont’d)

Incoming Cells

Incoming Tokens

+ ToNetwork12345

50

Leaky Bucket (Cont’d)Leaky Bucket (Cont’d)

Incoming Cells

Incoming Tokens

+ ToNetwork12345

51

Leaky Bucket (Cont’d)Leaky Bucket (Cont’d)

Incoming Cells

Incoming Tokens

+ ToNetwork12345

52

Leaky Bucket (Cont’d)Leaky Bucket (Cont’d)

Incoming Cells

Incoming Tokens

+ ToNetworkX 123

45

53

Leaky Bucket (Cont’d)Leaky Bucket (Cont’d)

Incoming Cells

Incoming Tokens

+ ToNetworkXX 123

45

54

Leaky Bucket (Cont’d)Leaky Bucket (Cont’d)

Incoming Cells

Incoming Tokens

+ ToNetworkXX 123

55

Leaky Bucket (Cont’d)Leaky Bucket (Cont’d)

Cells must obtain tokens in order to proceed into the network

If no token available, then cell is discarded

Constrains the rate at which cells enter the network to be the rate negotiated at the time of call setup

Shapes traffic, reduces burstiness

56

BufferedBuffered Leaky Bucket Leaky Bucket

Arriving cells that find a token waiting can proceed directly into the network

Arriving cells that find no token ready must wait in queue for a token

Cells that arrive to a full queue are lost Tokens that arrive to a full token pool

are simply discarded

57

Buffered Leaky BucketBuffered Leaky Bucket

Incoming Cells

Incoming Tokensat rate r tokens/sec

+ ToNetwork

Queue ofat most Bwaiting cells

Pool of at most Mwaiting tokens

58

Buffered Leaky Bucket Buffered Leaky Bucket (Cont’d)(Cont’d) Incoming cell rate: X

Token rate: r If X > r, then cells wait in buffer until

tokens are available Output traffic is r cells/sec, nicely

paced If X < r, then tokens always ready Output traffic is X (< r) Use for traffic shaping or UPC

59

Buffered Leaky Bucket Buffered Leaky Bucket (Cont’d)(Cont’d) A station can “save up” at most M

tokens Limits the maximum burst size in

the network Can send at most M cells back to

back B can be set to balance the tradeoff

between cell loss and cell delay

60

Leaky Bucket UPCLeaky Bucket UPC The token rate r is set based on the

rate declared at the time of call setup Makes sure that each source obeys

rate that was used when the call admission decision was made (i.e., descriptor)

Can use “single leaky bucket” to police just the peak cell rate (PCR)

Can use “dual leaky bucket” to police both PCR and SCR

61

VariationsVariations

There are several different variations of the basic leaky bucket concept described in the literature, such as the virtual leaky bucket, spacer, others

Basic idea: rather than strictly enforcing rates, allow senders to occasionally exceed their prescribed rate, as long as they mark or tag their extra cells

62

Cell Marking SchemeCell Marking Scheme

Uses leaky bucket to regulate cell transmissions as before, but rather than having cells wait for tokens when there are no tokens ready, the station can transmit the cell and mark it as a violation cell (i.e., cell colouring)

Green (CLP = 0) for cells that obey rate Red (CLP = 1) for cells that don’t

63

Cell Colouring (Cont’d)Cell Colouring (Cont’d)

If the network detects congestion at any point, then it does not hesitate to throw away red cells (CLP = 1), but always gives preference to green cells

Green cells must get through Red cells get through only if there is

spare capacity in the network “No harm in trying!” principle

64

Selective Cell Discard Selective Cell Discard (SCD)(SCD) A cell-level control mechanism in ATM

switches called selective cell discard can be implemented quite easily using a CLP threshold on each queue/buffer

Below the threshold, can accept both green and red cells

Beyond the threshold, can only accept green cells

65

Selective Cell Discard Selective Cell Discard (Cont’d)(Cont’d)

Buffer in an ATM switch

66

Selective Cell Discard Selective Cell Discard (Cont’d)(Cont’d)

Buffer in an ATM switch

67

Selective Cell Discard Selective Cell Discard (Cont’d)(Cont’d)

Buffer in an ATM switch

Some cellswaiting to go

68

Selective Cell Discard Selective Cell Discard (Cont’d)(Cont’d) CLP

Threshold

Buffer in an ATM switch

69

Selective Cell Discard Selective Cell Discard (Cont’d)(Cont’d) CLP

Threshold

70

Selective Cell Discard Selective Cell Discard (Cont’d)(Cont’d) CLP

Threshold

71

Selective Cell Discard Selective Cell Discard (Cont’d)(Cont’d) CLP

Threshold

72

Selective Cell Discard Selective Cell Discard (Cont’d)(Cont’d) CLP

Threshold

73

Selective Cell Discard Selective Cell Discard (Cont’d)(Cont’d) CLP

Threshold

74

Selective Cell Discard Selective Cell Discard (Cont’d)(Cont’d) CLP

Threshold

75

Selective Cell Discard Selective Cell Discard (Cont’d)(Cont’d) CLP

Threshold

76

Selective Cell Discard Selective Cell Discard (Cont’d)(Cont’d) CLP

Threshold

77

Selective Cell Discard Selective Cell Discard (Cont’d)(Cont’d) CLP

Threshold

78

Selective Cell Discard Selective Cell Discard (Cont’d)(Cont’d) CLP

Threshold

79

Selective Cell Discard Selective Cell Discard (Cont’d)(Cont’d) CLP

Threshold

80

Selective Cell Discard Selective Cell Discard (Cont’d)(Cont’d) CLP

Threshold

81

Selective Cell Discard Selective Cell Discard (Cont’d)(Cont’d) CLP

Threshold

82

Selective Cell Discard Selective Cell Discard (Cont’d)(Cont’d) CLP

Threshold

83

Selective Cell Discard Selective Cell Discard (Cont’d)(Cont’d) CLP

Threshold

84

Selective Cell Discard Selective Cell Discard (Cont’d)(Cont’d) CLP

Threshold

85

Selective Cell Discard Selective Cell Discard (Cont’d)(Cont’d) CLP

Threshold

86

Selective Cell Discard Selective Cell Discard (Cont’d)(Cont’d) CLP

Threshold

87

Selective Cell Discard Selective Cell Discard (Cont’d)(Cont’d) CLP

Threshold

88

Selective Cell Discard Selective Cell Discard (Cont’d)(Cont’d) CLP

Threshold

89

Explicit Congestion Explicit Congestion NotificationNotification There are some proposals to use

reactive congestion control approaches for end-to-end flow control in ATM

One of the mechanisms proposed is called Explicit Forward Congestion Notification (EFCN) (or EFCI, for Explicit Forward Congestion Indication)

90

EFCI: Basic OperationEFCI: Basic Operation

Switches can detect the onset of congestion (e.g., buffers filling up)

Switches set a control bit in cell headers to indicate this congestion condition

Sources react by reducing the volume of traffic that they are sending through that switch

Suitable for VBR or ABR traffic

91

EFCI: Basic Operation EFCI: Basic Operation (Cont’d)(Cont’d)

Traffic Source

Traffic Sink

Switch Switch Switch

92

EFCI: Basic Operation EFCI: Basic Operation (Cont’d)(Cont’d)

Traffic Source

Traffic Sink

Switch Switch Switch

Buffer

93

EFCI: Basic Operation EFCI: Basic Operation (Cont’d)(Cont’d)

Traffic Source

Traffic Sink

Switch Switch Switch

OccupiedUnoccupied

94

EFCI: Basic Operation EFCI: Basic Operation (Cont’d)(Cont’d)

Traffic Source

Traffic Sink

Switch Switch Switch

EFCI Threshold

95

EFCI: Basic Operation EFCI: Basic Operation (Cont’d)(Cont’d)

Traffic Source

Traffic Sink

Switch Switch Switch

EFCI Threshold

DataCell

96

EFCI: Basic Operation EFCI: Basic Operation (Cont’d)(Cont’d)

Traffic Source

Traffic Sink

Switch Switch Switch

EFCI Threshold

97

EFCI: Basic Operation EFCI: Basic Operation (Cont’d)(Cont’d)

Traffic Source

Traffic Sink

Switch Switch Switch

EFCI Threshold

98

EFCI: Basic Operation EFCI: Basic Operation (Cont’d)(Cont’d)

Traffic Source

Traffic Sink

Switch Switch Switch

EFCI Threshold

99

EFCI: Basic Operation EFCI: Basic Operation (Cont’d)(Cont’d)

Traffic Source

Traffic Sink

Switch Switch Switch

EFCI Threshold

100

EFCI: Basic Operation EFCI: Basic Operation (Cont’d)(Cont’d)

Traffic Source

Traffic Sink

Switch Switch Switch

EFCI Threshold

101

EFCI: Basic Operation EFCI: Basic Operation (Cont’d)(Cont’d)

Traffic Source

Traffic Sink

Switch Switch Switch

EFCI Threshold

102

EFCI: Basic Operation EFCI: Basic Operation (Cont’d)(Cont’d)

Traffic Source

Traffic Sink

Switch Switch Switch

EFCI Threshold

103

EFCI: Basic Operation EFCI: Basic Operation (Cont’d)(Cont’d)

Traffic Source

Traffic Sink

Switch Switch Switch

EFCI Threshold

!!!

104

EFCI: Basic Operation EFCI: Basic Operation (Cont’d)(Cont’d)

Traffic Source

Traffic Sink

Switch Switch Switch

EFCI Threshold

AckCell

105

EFCI: Basic Operation EFCI: Basic Operation (Cont’d)(Cont’d)

Traffic Source

Traffic Sink

Switch Switch Switch

EFCI Threshold

106

EFCI: Basic Operation EFCI: Basic Operation (Cont’d)(Cont’d)

Traffic Source

Traffic Sink

Switch Switch Switch

EFCI Threshold

107

EFCI: Basic Operation EFCI: Basic Operation (Cont’d)(Cont’d)

Traffic Source

Traffic Sink

Switch Switch Switch

EFCI Threshold

108

EFCI: Basic Operation EFCI: Basic Operation (Cont’d)(Cont’d)

Traffic Source

Traffic Sink

Switch Switch Switch

EFCI Threshold

!!!

109

EFCI: Basic Operation EFCI: Basic Operation (Cont’d)(Cont’d)

Traffic Source

Traffic Sink

Switch Switch Switch

EFCI Threshold

110

EFCI: Basic Operation EFCI: Basic Operation (Cont’d)(Cont’d)

Traffic Source

Traffic Sink

Switch Switch Switch

EFCI Threshold

111

EFCI: Basic Operation EFCI: Basic Operation (Cont’d)(Cont’d)

Traffic Source

Traffic Sink

Switch Switch Switch

EFCI Threshold

112

EFCI: Basic Operation EFCI: Basic Operation (Cont’d)(Cont’d)

Traffic Source

Traffic Sink

Switch Switch Switch

EFCI Threshold

113

EFCI: Basic Operation EFCI: Basic Operation (Cont’d)(Cont’d)

Traffic Source

Traffic Sink

Switch Switch Switch

EFCI Threshold

114

EFCI IssuesEFCI Issues How to set EFCI threshold What should sources do when EFCI

signal is seen What should sources do when no

EFCI signal is seen Forward versus backward

notification Effect of feedback delay Delay x bandwidth product

115

SummarySummary

Congestion control in high speed ATM networks is a difficult problem

Lots of good ideas of how to do it, but no real standard (yet?)

Will likely require a combination of schemes at different time scales and for different classes of traffic

Lots more remains to be done