Utility, Fairness, TCP/IP Steven Low CS/EE netlab.CALTECH.edu Feb 2004.

Utility, Fairness, TCP/IP

Steven Low

CS/EEnetlab.CALTECH.edu

Feb 2004

Acknowledgments Caltech

Bunn, Choe, Doyle, Jin, Newman, Ravot, Singh, J. Wang, Wei

UCLA Paganini, Z. Wang

CERN Martin

SLAC Cottrell

Internet2 Almes, Shalunov

Cisco Aiken, Doraiswami, Yip

Level(3) Fernes

LANL Wu

Protocol Decomposition

Applications

TCP/AQM

IP

Transmission

WWW, Email, Napster, FTP, …

Ethernet, ATM, POS, WDM, …Topology, power control Maximize capacity

Shortest-path routing Minimize path costs

Duality model (Kelly, Low et al)

Maximize aggregate utility

HOT (Doyle et al)

Minimize user response time

Heavy-tailed file sizes

Outline

Network model FAST TCP

Equilibrium Stability Implementation Experiments

TCP/IP interaction Fairness-efficiency

Applications

TCP/AQM

IP

Transmission


Ethernet, ATM, POS, WDM, …

Congestion control

xi(t)

pl(t)

Example congestion measure pl(t) Loss (Reno) Queueing delay (Vegas)

TCP/AQM

Congestion control is a distributed asynchronous algorithm to share bandwidth

It has two components TCP: adapts sending rate (window) to congestion AQM: adjusts & feeds back congestion information

They form a distributed feedback control system Equilibrium & stability depends on both TCP and AQM And on delay, capacity, routing, #connections

pl(t)

xi(t)TCP: Reno Vegas

AQM: DropTail RED REM/PI AVQ

Network model

F1

FN

G1

GL

Rf(s)

Rb’(s)

TCP Network AQM

x y

q p

liRlif link uses source if 1

liR lib link uses source if 1

Network model

F1

FN

G1

GL

Rf(s)

Rb’(s)

TCP Network AQM

x y

q p

lieR lis

lif link uses source if

lieR lislib link uses source if E

Outline




Applications

TCP/AQM

IP

Transmission



Methodology

Protocol (Reno, Vegas, RED, REM/PI…)

Equilibrium Performance

Throughput, loss, delay

Fairness

Dynamics Local stability Global stability

))( ),(( )1(

))( ),(( )1(

txtpGtp

txtpFtx

Network model

F1

FN

G1

GL

R

RT

TCP Network AQM

x y

q p

))( ),(( )1(

))( ),(( )1(

tRxtpGtp

txtpRFtx T

Reno, Vegas

DT, RED, …

liRli link uses source if 1 IP routing

Duality model

))( ),(( )1(

))( ),(( )1(

txtpGtp

txtpFtx

Primal-dual algorithm:

)( )( max )( min

subject to )( max

00

0

:Dual

:Primal

ll

ll

sss

xp

sss

x

xcpxUpD

cRxxU

s

s

Duality Model of TCP

))( ),(( )1(

))( ),(( )1(

txtpGtp

txtpFtx

Primal-dual algorithm:

Reno, Vegas

DropTail, RED, REM

Source algorithm iterates on rates Link algorithm iterates on prices With different utility functions

Summary: duality model

cRx

xUs

ssxs

subject to

)( max0

Flow control problem (Kelly, Malloo, Tan 98)

TCP/AQM Maximize utility with different utility functions

Primal-dual algorithm

))( ),(( )1(

))( ),(( )1(

tRxtpGtp

txtpRFtx T

Reno,

VegasDropTail, RED, REM

Result (L 00): (x*,p*) primal-dual optimal iff 0 ifequality with ** lll pcy

Example utility functions

1 log

1 )1( :General

log : Vegas

32log

1 :2-Reno

3/2tan23

:1-Reno

11

1

i

i

ii

ii

ii

i

iii

x

x

x

Tx

Tx

T

TxT

/

FAST, STCP

(Mo, Walrand 00)

Methodology

Protocol (Reno, Vegas, RED, REM/PI…)

Equilibrium Performance

Throughput, loss, delay

Fairness

Dynamics Local stability Global stability

))( ),(( )1(

))( ),(( )1(

txtpGtp

txtpFtx

222

2

3

33

)1(4

)1 )(

2

-(Nc

N

c

Theorem (Low et al, Infocom’02) Reno/RED is locally stable if

Stability: Reno/RED

F1

FN

G1

GL

Rf(s)

Rb’(s)

TCP Network AQM

x y

q p

TCP: Small Small c Large N

RED: Small Large delay

Stability: scalable control

F1

FN

G1

GL

Rf(s)

Rb’(s)

TCP Network AQM

x y

q p

lll

l ctyc

tp )(1

)()(

)(tq

mii

iii

i

extx

Theorem (Paganini, Doyle, L, CDC’01) Provided R is full rank, feedback loop is locally stable for arbitrary delay, capacity, load and topology

Linear Stability: scalable control

Theorem (Paganini, Doyle, Low, CDC’01) Provided R is full rank, feedback loop is locally stable for arbitrary delay, capacity, load and topology

2

1

( )fy R s x ( )Tbq R s p

0i ii i

i

xx q

M

1l l

l

p yc

Globally stable in presence of delay?

Stability: Stabilized Vegas

)()(1)( tan)(

1 )()(1-

2tqtt

tTx iid

tqtxi ii

ii

F1

FN

G1

GL

Rf(s)

Rb’(s)

TCP Network AQM

x y

q p

lll

l ctyc

tp )(1

)(

Theorem (Choe & L, Infocom’03) Provided R is full rank, feedback loop is locally stable if

),( max aTx ii

Stability: Stabilized Vegas

)()(1)( tan)(

1 )()(1-

2tqtt

tTx iid

tqtxi ii

ii

F1

FN

G1

GL

Rf(s)

Rb’(s)

TCP Network AQM

x y

q p

lll

l ctyc

tp )(1

)(

Application Stabilized TCP with current routers Queueing delay as congestion measure has right scaling Incremental deployment with ECN

Outline




Applications

TCP/AQM

IP

Transmission



Reno TCP

Packet level Designed and implemented first

Flow level Understood afterwards

Flow level dynamics determines Equilibrium: performance, fairness Stability

Design flow level equilibrium & stability Implement flow level goals at packet level

Packet level

ACK: W W + 1/W

Loss: W W – 0.5W

Reno AIMD(1, 0.5)

ACK: W W + a(w)/W

Loss: W W – b(w)W

HSTCP AIMD(a(w), b(w))

ACK: W W + 0.01

Loss: W W – 0.125W

STCP MIMD(a, b)

RTT

baseRTT W W :RTT FAST

Flow level: Reno, HSTCP, STCP, FAST

Similar flow level equilibrium

= 1.225 (Reno), 0.120 (HSTCP), 0.075 (STCP)

pkts/sec

Flow level: Reno, HSTCP, STCP, FAST

Different gain and utility Ui

They determine equilibrium and stability

Different congestion measure pi Loss probability (Reno, HSTCP, STCP) Queueing delay (Vegas, FAST)

Common flow level dynamics

windowadjustment

controlgain

flow levelgoal=

Implementation strategy

Common flow level dynamics

windowadjustment

controlgain

flow levelgoal=

Small adjustment when close, large far away Need to estimate how far current state is wrt target Scalable

Window adjustment independent of pi Depends only on current window Difficult to scale

FAST TCP

Theorem (Jin, Wei, L ‘03) In absence of delay at a single link Mapping from w(t) to w(t+1) is contraction Global exponential convergence Full utilization after finite time Utility function: i log xi (proportional fairness)

Outline




Applications

TCP/AQM

IP

Transmission



Network

(Sylvain Ravot, caltech/CERN)

FAST TCPutil: 95%

Linux TCPutil: 19%

1Gbps path; 180 ms RTT; 1 flowJin, Wei, Ravot, etc (Caltech, Nov 02)

Dynamic sharing: 3 flowsFAST Linux

Dynamic sharing on Dummynet capacity = 800Mbps delay=120ms 3 flows iperf throughput Linux 2.4.x (HSTCP: UCL)

Dynamic sharing: 3 flowsFAST Linux

HSTCP STCP

Steady throughput

FAST Linux

throughput

loss

queue

STCPHSTCP

Dynamic sharing on Dummynet capacity = 800Mbps delay=120ms 14 flows iperf throughput Linux 2.4.x (HSTCP: UCL)

30min

FAST Linux

throughput

loss

queue

STCPHSTCP

30min

Room for mice !

HSTCP

Outline




Applications

TCP/AQM

IP

Transmission



Network model

F1

FN

G1

GL

R

RT

TCP Network AQM

x y

q p

))( ),(( )1(

))( ),(( )1(

tRxtpGtp

txtpRFtx T

Reno, Vegas

DT, RED, …

liRli link uses source if 1 IP routing

Motivation

ll

li l

lliR

iiixp

iii

xR

cppRxxU

cRxxU

ii

max)( max min

subject to )( maxmax

00

0

:Dual

:Primal

Motivation

Can TCP/IP maximize utility?

ll

li l

lliR

iiixp

iii

xR

cppRxxU

cRxxU

ii

max)( max min

subject to )( maxmax

00

0

:Dual

:Primal

Shortest path routing!

TCP-AQM/IP

Theorem (Wang, et al 03)

Primal problem is NP-hard

Ai

iAi

i cc

Proof Reduce integer partition to primal problem

Given: integers {c1, …, cn}Find: set A s.t.

TCP-AQM/IP

Theorem (Wang, et al 03)

Primal problem is NP-hard

Achievable utility of TCP/IP?

Stability? Duality gap?

Conclusion: Inevitable tradeoff between

achievable utility routing stability

Ring networkdestination

r

Single destination Instant convergence of

TCP/IP Shortest path routing

Link cost = pl(t) + dl

price static

TCP/AQM

IPr(0)

pl(0)

r(1)

pl(1)

… r(t), r(t+1) , …

routing


r

TCP/AQM

IPr(0)

pl(0)

r(1)

pl(1)

… r(t), r(t+1) , …

Stability: r ?

Utility: V ?r* : optimal routing

V* : max utility


r

link cost = pl(t) + dl

0

0||*

*

VV

rr

Theorem (Infocom 2003)

Solve primal problem asymptoticallyas

Stability: r ?

Utility: V ?


r

link cost = pl(t) + dl

Theorem (Infocom 2003)

large: globally unstable small: globally stable medium: depends on r(0)

Stability: r ?

Utility: V ?

General network

Conclusion: Inevitable tradeoff between

achievable utility routing stability

random graph20 nodes, 200 links Achievable

utility

Outline




Applications

TCP/AQM

IP

Transmission



TCP/AQM: duality model Flow control problem (Kelly, Malloo, Tan 98)

TCP/AQM Maximize utility with different utility functions (L 00): (x*,p*) primal-dual optimal iff

Primal-dual algorithm

Reno, Vegas, FAST

DT, RED, REM/PI, AVQ

Fairness

Identify allocation with An allocation is fairer if its is

larger

(Mo, Walrand 00)

Fairness

maximum throughput

proportional fairness

min delay fairness infinity maxmin

fairness

(Mo, Walrand 00)

Efficiency

Unique optimal rate x() An allocation is efficient if T() is

large

Conjecture

Conjecture T() is nonincreasing

i.e. a fair allocation is always inefficient

Example 1



1/(L+1)

L/(L+1)

1/2

1/2

maxmin proportional

Example 1



Example 2



Example 3

)(T



Intuition

“The fundamental conflict between achieving flow fairness and maximizing overall system throughput….. The basic issue is thus the trade-off between these two conflicting criteria.”

Luo,etc.(2003), ACM MONET

Results

Theorem: Necessary & sufficient condition for general network

Corollary 1: true if N(R)=1

1/(L+1)

L/(L+1)

1/2

1/2

maxmin proportional

Results


Corollary 1: true if N(R)=1

Results


Corollary 2: true if N(R)=2 2 long flows pass through same# links

Counter-example There exists a network such that

dT/d > 0 for almost all >0 Intuition

Large favors expensive flows Long flows may not be expensive

Maxmin may be more efficient than proportional fairness

Counter-example

Theorem: Given any 0>0, there exists network where

Compact example

0

netlab.caltech.edu/FAST

Utility, Fairness, TCP/IP Steven Low CS/EE netlab.CALTECH.edu Feb 2004.

Documents

stcppktssecflow level

arbitrary delay

presence of delay

stcpqueueing delay vegas

utility ui

current routersqueueing

congestion informationthey

topologylinear stability