Network Sharing Issues

Network Sharing Issues

Lecture 15Aditya Akella

• Is this the biggest problem in cloud resource allocation? Why? Why not?

• How does the problem differ wrt allocating other resources?

• FairCloud: Sharing the Network in Cloud Computing, Sigcomm 2012

• What are the assumptions? Drawbacks?

Motivation

Network?

Context

Networks are more difficult to share than other resources

X

Context

• Several proposals that share network differently, e.g.:– proportional to # source VMs (Seawall [NSDI11])– statically reserve bandwidth (Oktopus [Sigcomm12])– …

• Provide specific types of sharing policies

• Characterize solution space and relate policies to each other?

FairCloud Paper

1. Framework for understanding network sharing in cloud computing– Goals, tradeoffs, properties

2. Solutions for sharing the network – Existing policies in this framework– New policies representing different points in

the design space

Goals

1. Minimum Bandwidth Guarantees– Provides predictable performance– Example: file transfer finishes within time limit

A1 A2

Timemax = Size / Bmin

Bmin

Goals

1. Minimum Bandwidth Guarantees2. High Utilization– Do not leave useful resources unutilized– Requires both work-conservation and proper

incentives

A

B B B

Both tenants active Non work-conserving Work-conserving

Goals

1. Minimum Bandwidth Guarantees2. High Utilization3. Network Proportionality– As with other services, network should be shared

proportional to payment– Currently, tenants pay a flat rate per VM

network share should be proportional to #VMs (assuming identical VMs)

Goals

1. Minimum Bandwidth Guarantees2. High Utilization3. Network Proportionality– Example: A has 2 VMs, B has 3 VMs

A1

A2

BwA

B1

B3

B2

BwB

BwB

BwA =

23

When exact sharing is not possible use max-min

Goals

1. Minimum Bandwidth Guarantees2. High Utilization3. Network Proportionality

Not all goals are achievable simultaneously!

TradeoffsMin

GuaranteeHigh

UtilizationNetwork

Proportionality

Not all goals are achievable simultaneously!

TradeoffsMin

GuaranteeNetwork

Proportionality

Tradeoffs

BwBBwA

A BAccess Link LCapacity C

BwB = 11/13 CBwA = 2/13 C

10 VMs

Network Proportionality

BwA ≈ C/NT 0

#VMs in the network

BwB = 1/2 CBwA = 1/2 C

Minimum Guarantee

Min Guarantee


TradeoffsHigh

UtilizationNetwork

Proportionality

Tradeoffs

L

B1

B3

B2

B4

A1

A3

A2

A4

High Utilization


Tradeoffs

L

B1

B3

B2

B4

A1

A3

A2

A4

BwB = 1/2 C

BwA = 1/2 CNetwork Proportionality

High Utilization


Tradeoffs

L

B1

B3

B2

B4

A1

A3

A2

A4

P

High Utilization


Uncongested path

Tradeoffs

L

B1

B3

B2

B4

A1

A3

A2

A4

BwB BwA <


L L

Tenants can be disincentivized to use free resources

If A values A1A2 or A3A4 more than A1A3

BwA+BwA = BwBLLP

P

High Utilization


Uncongested path


Tradeoffs

L

B1

B3

B2

B4

A1

A3

A2

A4

PNetwork proportionality applied only for flows traversing congested links shared by multiple tenants

High Utilization

Uncongested path Congestion

Proportionality


Tradeoffs

L

B1

B3

B2

B4

A1

A3

A2

A4

Uncongested path

P

BwB BwA =

Congestion Proportionality

L L

High Utilization



Tradeoffs

Still conflicts with high utilization

High Utilization


Tradeoffs

B1

B3

A1

A3

B2

B4

A2

A4L2

C1 = C2 = C


High Utilization

L1


Tradeoffs

B1

B3

A1

A3

B2

B4

A2

A4

L1

L2

BwB BwA =

Congestion ProportionalityL1 L1

BwB BwA =L2 L2


High Utilization

C1 = C2 = C


Tradeoffs

B1

B3

A1

A3

B2

B4

A2

A4

L1

L2

Demand drops to ε


High Utilization

C1 = C2 = C


Tradeoffs

B1

B3

A1

A3

B2

B4

A2

A4

ε

C - ε

ε

C - ε

Tenants incentivized to not fully utilize

resources


High Utilization

C1 = C2 = C


L1

L2

Tradeoffs

B1

B3

A1

A3

B2

B4

A2

A4

ε

C - 2εUncongested

ε

C - ε

L1

L2

C1 = C2 = C


High Utilization


resources


L2

Tradeoffs

B1

B3

A1

A3

B2

B4

A2

A4

ε

C - 2ε

C/2

C/2

Uncongested

C1 = C2 = C


High Utilization



resources L1

Tradeoffs

Proportionality applied to each link independently


Link Proportionality


High Utilization

L2

B1

B3

A1

A3

B2

B4

A2

A4

L1

L2

Tradeoffs

B1

B3

A1

A3

B2

B4

A2

A4

Full incentives for high utilization




High Utilization

L1

Goals and TradeoffsMin

Guarantee


High Utilization



Guiding PropertiesMin

Guarantee


High Utilization



Break down goals into lower-level necessary properties

PropertiesMin

Guarantee


High Utilization



Work Conservatio

n

Work Conservation

• Bottleneck links are fully utilized• Static reservations do not have this property

PropertiesMin

Guarantee


High Utilization



Work Conservatio

n

UtilizationIncentives

Utilization Incentives

• Tenants are not incentivized to lie about demand to leave links underutilized

• Network and congestion proportionality do not have this property

• Allocating links independently provides this property

PropertiesMin

Guarantee


High Utilization



Work Conservatio

n


Comm-PatternIndependence

Communication-pattern Independence

• Allocation does not depend on communication pattern

• Per flow allocation does not have this property– (per flow = give equal shares to each flow)

Same Bw

PropertiesMin

Guarantee


High Utilization



Work Conservatio

n



Symmetry

Symmetry

• Swapping demand directions preserves allocation• Per source allocation lacks this property– (per source = give equal shares to each source)

Same Bw

Same Bw

Goals, Tradeoffs, PropertiesMin

Guarantee


High Utilization



Work Conservatio

n



Symmetry

Outline

1. Framework for understanding network sharing in cloud computing– Goals, tradeoffs, properties

2. Solutions for sharing the network – Existing policies in this framework– New policies representing different points in

the design space

Per Flow (e.g. today)Min

Guarantee


High Utilization



Work Conservatio

n



Symmetry

Per Source (e.g., Seawall [NSDI’11]) Min

Guarantee


High Utilization



Work Conservatio

n



Symmetry

Static Reservation (e.g., Oktopus [Sigcomm’11])Min

Guarantee


High Utilization



Work Conservatio

n



Symmetry

New Allocation Policies

3 new allocation policies that take different stands on tradeoffs

Proportional Sharing at Link-level (PS-L)Min

Guarantee


High Utilization



Work Conservatio

n



Symmetry

Proportional Sharing at Link-level (PS-L)

• Per tenant WFQ where weight = # tenant’s VMs on link

A

B

WQA= #VMs A on L

BwB

BwA =

#VMs A on L#VMs B on L

Can easily be extended to use heterogeneous VMs (by using VM weights)

Proportional Sharing at Network-level (PS-N)Min

Guarantee


High Utilization



Work Conservatio

n



Symmetry

Proportional Sharing at Network-level (PS-N)

• Congestion proportionality in severely restricted context• Per source-destination WFQ, total tenant weight = # VMs


WQA1A2= 1/NA1 + 1/NA2

A1 A2

NA2

NA1

Total WQA = #VMs A



WQB

WQA =

#VMs A#VMs B


Proportional Sharing on Proximate Links (PS-P)Min

Guarantee


High Utilization



Work Conservatio

n



Symmetry

• Assumes a tree-based topology: traditional, fat-tree, VL2(currently working on removing this assumption)

Proportional Sharing on Proximate Links (PS-P)


• Min guarantees– Hose model– Admission control


A1

BwA1

A2

BwA2

An

BwAn

…



• High Utilization– Per source fair sharing towards tree root




• High Utilization– Per source fair sharing towards tree root– Per destination fair sharing from tree root


Deploying PS-L, PS-N and PS-P• Full Switch Support

– All allocations can use hardware queues (per tenant, per VM or per source-destination)

• Partial Switch Support– PS-N and PS-P can be deployed using CSFQ [Sigcomm’98]

• No Switch Support– PS-N can be deployed using only hypervisors– PS-P could be deployed using only hypervisors, we are currently

working on it

Evaluation

• Small Testbed + Click Modular Router– 15 servers, 1Gbps links

• Simulation + Real Traces– 3200 nodes, flow level simulator, Facebook

MapReduce traces

Many to one

BwBBwA

A B

N

One link, testbedPS-P offers guarantees

BwA

N

MapReduce

One link, testbed

BwBBwA

5

R 5

M

M+R = 10 M

BwB

(Mbp

s)

PS-L offers link proportionality

MapReduce

Network, simulation, Facebook trace

MapReduce


PS-N is close to network proportionality

MapReduce


PS-N and PS-P reduce shuffle time of small jobs

by 10-15X

Summary• Sharing cloud networks is not trivial• First step towards a framework to analyze network

sharing in cloud computing – Key goals (min guarantees, high utilization and proportionality),

tradeoffs and properties• New allocation policies, superset properties from past work

– PS-L: link proportionality + high utilization– PS-N: restricted network proportional– PS-P: min guarantees + high utilization

• What are the assumptions, drawbacks?

Network Sharing Issues

Documents

network existing policies

specific sharing policies

design of sharing policies

motivation network

vm network share

exact sharing

frameworknew policies

vms a1a2bwab1b3b2bwbbwbbwa