The Gaussian Nature of TCP Mark Shifrin Supervisor: Supervisor: Dr. Isaac Keslassy M.Sc Seminar Faculty of Electrical Engineering.

The Gaussian Nature of TCP

Mark ShifrinMark Shifrin

Supervisor: Supervisor: Dr. Isaac Keslassy

M.Sc SeminarM.Sc Seminar

Faculty of Electrical EngineeringFaculty of Electrical Engineering

2

Large Network PresentationCharacterizing traffic is essential to understanding the behavior of large Internet networks.

Network is large and complex – contains millions of flows. The interaction and mutual impact is difficult to understand.

3

Large Network Related Challenges –What do we pursue?

Ability to analyze problems: CongestionsHigh packet losses

Ability to make planning decisions:Bottleneck capacity decision Buffer sizeRTT (Round Trip Time) distribution

4

Obstacles and Difficulties Today: no existing efficient large network

model because of numerous problems. Problems:

Many TCP feedback behaviors (congestion avoidance, slow-start, …)

Complex interactions among flowsComplex queueing analysis Many topologiesMany protocols (TCP, UDP, …)Complex interactions between flows and

ACKs

5

Reducing the Network into Basic Components

Bottlenecks dictate the behavior of the flows. Assumption: The network can be subdivided into

basic dumbbell topologies with a single forward bottleneck.

6

Previous work – rule-of-thumb

Universally applied rule-of-thumb for TCP flows: A router needs a buffer size:

RTT is the two-way propagation delay C is capacity of bottleneck link

Context Mandated in backbone and edge routers. Appears in RFPs and IETF architectural guidelines. Usually referenced in [Villamizar and Song, ’94] Already known by inventors of TCP [Van Jacobson, ‘88] Has major consequences for router design

B RTT C

CRouterSource Destination

RTT

7

Previous Work – Stanford Model[Appenzeller et al., ’04 | McKeown and Wischik, ’05 | McKeown et

al., ‘06] Assumption 1: TCP Flows modeled as i.i.d.

total window W has Gaussian distribution Assumption 2: Queue is the only variable part

queue has Gaussian distribution: Q=W-CONST use smaller buffer than in rule of thumb

We also consider small buffers. But, with small buffers: Buffers lose Gaussian property Link variations cannot be neglected

8

Previous Work – Total Arrival is Poisson Distributed

Assumption: M/M/1/K model of buffer with TCP ]Avrachenkov et al., ‘01]

Using the model of [Padhye et al., ’98] for the M/M/1/K analysis.

We show that the Poisson model is far from reality.

9

What is new in our analysis?

Dividing the Dumbbell Topology into components (Lines). Finding separately the pdf of the traffic of each

component we can describe the statistics of the entire network.

We show the impact of one component on another. We find a new model of TCP congestion window

(cwnd) distribution, considering correlated losses. We use it to find the pdf of the traffic on the Lines.

We prove that distributions of the packets on the Lines are Gaussian.

We find Transmission rate, Queue (Q) distribution and the packet loss.

10

Line L2

Line L1

Line L3

Line L4

Line L5

Line L6

Contents of the Dumbbell Topology

Q

11

Line L2

Line L1

Line L3

Line L4

Line L5

Line L6

Contents of the Dumbbell Topology

Q

12

Outline Models of single objects

Finding the model of cwnd . Correlated losses Packet Loss Event derivation Model of li1

Transmission rate and packet loss derivation Arrival rate of single flow Total arrival rate Q distribution Packet loss

Gaussian models of Lines L1 model , Lindeberg Central Limit Theorem Models of other lines and of W.

Conclusions

13

Model Development Flow

cwnd

Q pdf

total arrival rate

li1 pdf

l1i arrival rates

packet loss

14

TCP Newreno Congestion Control (Reminder)

Slow Start w → w*2 Congestion Avoidance: w → w+1 Fast Retransmit and Fast Recovery: w → w/2

Treats multiple losses in the same window Timeout (TO): w →1

15

CWND Simple Markov Chain Solution

Simple Markov Chain. Assumption: Timeout (TO) probability is small

comparatively to the halving.

Model 1: Independent losses [Altman et al., 04]

16

Model 2: Packet loss event derivation

p: packet loss probability. Original transition was given by p*n. We cannot use p because the losses are correlated. probability of the Loss Event for window size n is equal to pe(n)n Assumption: In window of size n with losses , each packet in the

same window has equal probability to experience the packet loss event.

Intuition: pe is the effective packet loss, which is independent for every packet.

Fast Retransmit and Fast Recovery able to recover almost in the same way no matter how many packets are lost in the window.

w=n/2 w=n

pe(n)*n

17

Correlated Losses

Packet losses are strongly correlated: given one packet was lost in a TCP window, the probability is higher for others in the same window to be lost.

Assumption: For a given window size n, given at least one lost packet, the distribution of the number of lost packets is independent of the network condition! (No actual impact of p, RTT distribution, Buffer size)

Denote this distribution as Pr(l(n)=k), for k=1…n

18

Packet Loss Burst Size Distribution

19

Packet Loss Burst Size Distribution

21

Expected packet loss burst size Pr(l(n)=l) = Pr(l lost packets for w=n|loss event

happened) E[l(n)]- expected value of the size of the bursty

loss, given at least one packet was lost, for window size n.

At every loss event in window of size n, E[l(n)] packets fall in average.

Every loss event is effectively treated as a single loss (approximation)

pe – effective packet loss probability.

22

What is this pe actually? Theorem: pe(n)/p =1/ E[l(n)], i.e. independent of

p for a given n. Proof outline:

For arbitrary lost packet B denote AB as P(some packets lost in window containing B)

p=P(B|AB)*p(AB)+ P(B|ABC)*p(AB

C)

P(B|AB)=E[l(n)]/n

p(AB)=pe(n)*n

Conclusion: pe(n)=p/ E[l(n)] Intuition: E[l(n)] losses treated as a single loss!

0

23

pe/p - simulation results

500 flows traced by NS2 tools, event by event.

24

MC Transitions (Approach 1 & 2)

Approach 1:Pr(w(t+1)=w(t)/2 | w(t)=n)=n*pe(n)

Pr(TO)=const (known)Pr(w(t+1)=w(t)+1 | w(t)=n)=1-Pr(TO)-

Pr(w(t+1)=w(t)/2 | w=n). Approach 2:

Pr(w(t+1)=w(t)/2 | w(t)=n)=Ph1+Ph2

Then Ph1 is the probability of loss in the first half of the window and Ph2 is the probability of loss in the second half of the window.

Improvement: introduce Slow Start in the model

26

Results ( pdf,low p)

27

Results (cdf, low p)

28

Results (pdf, medium p)

29

Results (cdf, medium p)

30

Results (pdf, high p)

31

Results (cdf, high p)

32

Model of li1

cwnd

li1 pdf

packet loss

33

li1 distribution – uniform vs. bursty

li3li

1li

5

li2

li4 li

6

• Approach 1: Uniform packet distribution. l1(t)i = wi(t)* tpi

1/rtti.

• Approach 2: Bursty Packet Distribution

Bsource dest.

rtti=tp1i+tp2

i+tp3i+tp4

i+tp5i+tp6

i

34

Bursty Transmission and li1 model

Assumption: All packets in a flow move in a single burstTransmission and queueing times of the

entire burst negligible comparatively to propagation latencies

Conclusion 1: All packets belonging to an arbitrary flow i are present almost always on the same link (l1i,l2i,l3i,l4i,l5i,l6i).

Conclusion 2: The probability of burst of flow i being present on the certain link is equal to the ratio of its propagation latency to the total rtti.

35

li1 probability density function

We compare the measured results vs. the models 1 and 2

Reminder of the model 1: l1(t)i = wi(t)* tpi1/rtti.

36

Model results li1 , logarithmic scale

38

Outline Models of single objects

Finding the model of cwnd . Correlated losses Packet Loss Event derivation Model of li1

Transmission rate and packet loss derivation Arrival rate of single flow Total arrival rate Q distribution Packet loss

Gaussian Models of Lines L1 model , Lindeberg Central Limit Theorem Models of other lines and of W.

Conclusions

39

Model Development Flow

cwnd total arrival rate

li1 pdf

l1i arrival rates

40

Rate Transmission Derivation The objective is to find the pdf of the number

of packets sent on some link i, ri, in a time unit δt.

Assumptions: The rate on each one of the links in L1 is

statistically independent We assume that the transmissions are bursty. We assume that the rate is proportional to the

distribution of l1i and to the ratio δt/tp1i.

tp1i is the latency on the corresponding link.

41

Arrival rate of a single flow

li1

li2

sourceB

δt

tp1i

li1 *δt/tp1i

• δt is the same for all flows.• We find the arrival distribution for every flow in δt msec.

42

Rate PDF on the single line

The distribution is the same, but with different parameters.

For finding the total rate Lindeberg Central Limit Theorem is needed.

43

Total Rate Theorem: The proof is using the Lindeberg condition.

The idea: Central Limit Theorem holds if the share of each flow comparatively to the sum is negligible as the number of the flows grows.

Argument for the proof: cwnd is limited by maximum value so does the l1i and ri.

Another alternative: Using a Poisson arrival rate - too small variance (result of the bursty transmission not reflected)

45

Rate Model - Results

Probability

Total arrival rate – number of packets per δt

46

Model Development Flow

cwnd

Q pdf

total arrival rate

li1 pdf

l1i arrival rates

packet loss

47

Q pdf

Two possible ways to find the Q distribution:Running MC simulation using samples of RG/D/1/K analysis using R for arrivals [Tran-

Gia and Ahmadi, ‘88] Both ways yield almost the same results. Packet loss p is derived from the Q

distribution.

48

Q pdf - results

Probability

Q state – 0 to 585 packets

49

Fixed Point Solution: p=f(p)

cwnd

Q pdf

total arrival rate

li1 pdf

l1i arrival rates

packet loss

50

Packet Loss Rate Solution by gradient algorithm

1. Start with approximate p

2. Find the cwnd and all l1i

3. Find all rates of all flows4. Find the total rate and Q pdf5. Find new p6. Make a correction according to the error

weighted by step size and go to 2.7. Till the error small enough – fixed point

found

51

Packet loss - results

Model gives about 10%-25% of discrepancy. Case 1: Measured: p=2.7%, Model: p=3% Case 2: Measured: p=0.8%, Model: p=0.98% Case 3: Measured: p=1.4%, Model: p=1.82% Case 4: Measured: p=0.452%, Model: p=0.56%

52

Outline Models of single objects

Finding the model of cwnd . Correlated losses Packet Loss Event derivation Model of li1

Transmission rate and packet loss derivation Arrival rate of single flow Total arrival rate Q distribution Packet loss

Gaussian Models of Lines L1 model , Lindeberg Central Limit Theorem Models of other lines and of W.

Conclusions

53

L1 (reminder)

Line L1

Line L3

Line L5

Q

54

Lindeberg Central Limit Theorem

All li1 are statistically independent. The distribution is the same but with different

parameters – because of different RTT – simple Central Limit Theorem won’t fit.

Theorem: Conclusion : Using the model for cwnd and

then for li1 we are able to find the L1 distribution.

56

L1 Model Results

57

What else can we know from our model?

Line L2

Line L1

Line L3

Line L4

Line L5

Line L6Q

•Stanford Model: Gaussian Queue, Gaussian W, constant traffic on the links•Our model: non-Gaussian Queue, Gaussian W, Gaussian traffic on the links.

58

Packets on other lines

li3

li1

li5

li2

li4 li

6

Bclient servercwnd bcwnd

bcwnd

59

Model Development Flow

cwnd W

bcwnd

li2

li4

li5

li6

L2

L4

L5

L6

Total

60

cwnd on the Lines We know the pdf for the cwnd – the window at

the source. This window travels the lines as the single burst The burst which passes through the Q loses

some packets. How can we know the distribution of the window

of packets after it passes the Q? Suggestion: Using l(n) distribution for window of

size n. Reminder: Pr(l(n)=l) = Pr(l lost packets for w=n|loss

event happened)

61

Packet Loss Burst Size Distribution (Reminder)

62

Markov Chain of bcwnd

w=n-1 w=nw=2 w=3w=1

1-Ploss

Ploss*P(l(n)=n-3)

Ploss*P(l(n)=n-2)

Ploss*P(l(n)=n-1)

Ploss*P(l(n)=1)

63

Back cwnd algorithm (bcwnd) The fall probability for window of size n:

Ploss= pe(n)*n. (TO is small and omitted) For each n from 1 to 64 update:

Pr(bcwnd=n)=P(cwnd=n)-P(cwnd=n)* Ploss

For each I, i=0 to n-1 update: P(bcwnd=i)=P(cwnd=i)+ P(cwnd=n)* Ploss*P(l(n)=n-i)

The update done from low to the highest state.

64

bcwnd and cwnd comparison

65

Models for L2, L4,L5,L6 Typical case: tp1

i, tp2i, tp4

i, tp5i, tp6

i, all have a different distribution

Approach 1: Use model of L1 with cwnd and scale by ratio of means of tp. ratio of tp gives unreliable result for both the expected

value and variance Approach 2: Use bcwnd for the model for l2i, l4i,

l5i, l6i. The model is exactly the same as for l1i, but bcwnd

instead of cwnd used. Corresponding latencies for each line used. Use The Lindeberg Central Limit Theorem, exactly in

the same way as for L1!

66

L2 Model – result demonstration

67

Topology – Reminder of the Lines

Line L2

Line L1

Line L3

Line L4

Line L5

Line L6

Q

68

The Gaussian distributions

NS2 simulation of 500 flows with all different time propagations

L4

L2

L1

L5L6

W

69

Are these lines really Gaussian?

70

W – sum of all cwnd Two approaches to find W.

cwnd model – is known. All TCP connections are i.i.d – classical Central Limit Theorem is applicable.

Counting by summing all the components:W=L1+L2+L3+L4+L5+L6+Q.

L3 and Q – are non Gaussian components. cwndi of different flows have some small correlation.

L1,L2,L4,L5,L6 - all Gaussian. L1 – almost the perfect Gaussian.

71

What Size of Buffer do We Need?

Avg cwndpFormulaBuffer210.8C*RTT/sqrt(N)/17.5154

24.50.664C*RTT/sqrt(N)/3.577228.50.45C*RTT/sqrt(N)/0.73862

Case 1

Avg cwndp (%)FormulaBuffer83.26C*RTT/sqrt(N)/8154

8.82.98C*RTT/sqrt(N)/245011.52.01C*RTT/sqrt(N)/0.51802

Case 2

(%)

72

Outline Models of single objects

Finding the model of cwnd . Correlated losses Packet Loss Event derivation Model of li1

Transmission rate and packet loss derivation Arrival rate of single flow Total arrival rate Q distribution Packet loss

Gaussian Models of Lines L1 model , Lindeberg Central Limit Theorem Models of other lines and of W.

Conclusions

73

Conclusions.. We know the probabilistic models of

different components of the networkSingle flowsGeneral components

We can plan routers or networks now – by deciding the C and the B, given some topology and packet loss limit.

74

Thanks are going to:

Dr. I. Keslassy for guidance and inspiration

Lab Staff for relentless support:Hai Vortman, Yoram Yihyie, Yoram Or-Chen,

Alex Students for some simulations. Listeners in this class for the patience..