ElasticSwitch: Practical Work- Conserving Bandwidth Guarantees for Cloud Computing HP Labs * Avi Networks + Google Lucian Popa Praveen Yalagandula * Sujata Banerjee Jeffrey C. Mogul + Yoshio Turner Jose Renato Santos
Feb 23, 2016
ElasticSwitch: Practical Work-Conserving Bandwidth Guarantees for Cloud
Computing
HP Labs* Avi Networks+ Google
Lucian Popa Praveen Yalagandula* Sujata Banerjee Jeffrey C. Mogul+ Yoshio Turner Jose Renato Santos
Goals
1. Provide Minimum Bandwidth Guarantees in Clouds
Tenants can affect each other’s traffic MapReduce jobs can affect performance of user-facing
applications Large MapReduce jobs can delay the completion of small
jobs Bandwidth guarantees offer predictable
performance
Goals
1. Provide Minimum Bandwidth Guarantees in Clouds
VMs of one tenant
Hose model
Bandwidth Guarantees
Virtual (imaginary) Switch
X Y
BX BYZ
BZ
VS
Other models based on hose model such as TAG [HotCloud’13]
Goals
1. Provide Minimum Bandwidth Guarantees in Clouds
2. Work-Conserving Allocation Tenants can use spare bandwidth from unallocated or
underutilized guarantees
Goals
1. Provide Minimum Bandwidth Guarantees in Clouds
2. Work-Conserving Allocation Tenants can use spare bandwidth from unallocated or
underutilized guarantees Significantly increases performance
Average traffic is low [IMC09,IMC10] Traffic is bursty
Goals
1. Provide Minimum Bandwidth Guarantees in Clouds
2. Work-Conserving Allocation
ElasticSwitch
X Y
Bmin
BminX
Y ba
ndwi
dth
Free capacity
Everything reserved & used
Time
Bmin
Goals
1. Provide Minimum Bandwidth Guarantees in Clouds
2. Work-Conserving Allocation3. Be Practical
Topology independent: work with oversubscribed topologies
Inexpensive: per VM/per tenant queues are expensive work with commodity switches
Scalable: centralized controller can be bottleneck distributed solution Hard to partition: VMs can cause bottlenecks anywhere in
the network
Goals
1. Provide Minimum Bandwidth Guarantees in Clouds2. Work-Conserving Allocation3. Be Practical
Prior WorkGuarante
esWork-
conserving
Practical
Seawall [NDSI’11], NetShare [TR], FairCloud (PS-L/N) [SIGCOMM’12]
X (fair
sharing)√ √
SecondNet [CoNEXT’10] √ X √
Oktopus [SIGCOMM’10] √ X ~X (centralized)
Gatekeeper [WIOV’11], EyeQ [NSDI’13] √ √ X (congestion-
free core)
FairCloud (PS-P) [SIGCOMM’12] √ √ X (queue/VM)
Hadrian [NSDI’13] √ √ X (weighted RCP)
ElasticSwitch √ √ √
Outline Motivation and Goals Overview More Details
Guarantee Partitioning Rate Allocation
Evaluation
ElasticSwitch Overview: Operates At Runtime
Tenant selects bandwidth guarantees. Models: Hose,
TAG, etc.
VMs placed, Admission Control ensures all
guarantees can be met
Enforce bandwidth guarantees
& Provide work-conservation
VM setup
RuntimeElasticSwitch
Oktopus [SIGCOMM’10]Hadrian [NSDI’10]CloudMirror [HotCLoud’13]
Network
ElasticSwitch Overview: Runs In Hypervisors
VM
VM
VM
ElasticSwitchHypervisor
VM
ElasticSwitchHypervisor
VM
ElasticSwitchHypervisor
VM
• Resides in the hypervisor of each host• Distributed: Communicates pairwise following
data flows
ElasticSwitch Overview: Two Layers
Guarantee Partitioning
Rate Allocation
Hypervisor
Provides Work-conservation
Provides Guarantees
ElasticSwitch Overview: Guarantee Partitioning1. Guarantee Partitioning: turns hose model into VM-to-VM pipe guarantees
VM-to-VM control is necessary, coarser granularity is not enough
X Y Z
VS
BX BY BZ
ElasticSwitch Overview: Guarantee Partitioning1. Guarantee Partitioning: turns hose model into VM-to-VM pipe guarantees
BXY BXZVM-to-VM guarantees bandwidths as if tenant communicates on a physical
hose network
Intra-tenant
ElasticSwitch Overview: Rate Allocation1. Guarantee Partitioning: turns hose model into VM-to-VM pipe guarantees
RateXY ≥
2. Rate Allocation: uses rate limiters, increases rate between X-Y above BXY when there is no congestion between X and Y
YX
Unreserved/Unused Capacity
Work-conserving allocation
X Y Z
VS
BX BY BZ
BXY
Inter-tenant
Hypervisor Limiter Hypervisor
ElasticSwitch Overview: Periodic Application
Guarantee Partitioning
Rate Allocation
Hypervisor
VM-to-VMguarantees Demand
estimates
Applied periodically, more often
Applied periodically and onnew VM-to-VM pairs
Outline Motivation and Goals Overview More Details
Guarantee Partitioning Rate Allocation
Evaluation
Guarantee Partitioning – Overview
BXY
BXZ BTY
X Y
Z T
QBQY
X Y
BX
Z
VS1
Q
BQ
T
Goals:A. Safety – don’t violate hose
modelB. Efficiency – don’t waste
guaranteeC. No Starvation – don’t block
traffic
Max-min allocation
Guarantee Partitioning – Overview
X Y
Z T
Q
X Y
BX
Z
VS1
Q
BQ
TBX = … = BQ = 100Mbps
33Mbps66Mbps 33Mbps
33Mbps
Max-min allocationGoals:
A. Safety – don’t violate hose model
B. Efficiency – don’t waste guarantee
C. No Starvation – don’t block traffic
Guarantee Partitioning – Operation
X Y
Z T
Q
X Y
BX
Z
VS1
Q
BQ
T
BX BY
BXBY
TYXZ
XY XY
BYQYHypervisor divides guarantee of each
hosted VM between VM-to-VM pairs in each direction
Source hypervisor uses the minimum between the source and destination
guarantees
X YBXY = min( BX , BY )
XY XY
Guarantee Partitioning – Safety
X Y
Z T
Q
X Y
BX
Z
VS1
Q
BQ
T
BX BY
BXBY
TYXZ
XY XY
BYQY
BXY = min( BX , BY )
XY XY
X Y
Safety: hose-model guarantees are not exceeded
Guarantee Partitioning – Operation
X Y
Z T
Q
X Y
BX
Z
VS1
Q
BQ
TBX = … = BQ = 100Mbps
BXY = min( BX , BY )
XY XY
Guarantee Partitioning – Operation
X Y
Z T
Q
X Y
BX
Z
VS1
Q
BQ
TBX = … = BQ = 100Mbps
BXY = min( BX , BY )
XY XY
BX = 50 BY = 33BXY = 33BX =
50BY = 33
TYXZ
XY XY
BY = 33QY
X Y
Guarantee Partitioning – Efficiency
What happens when flows have low demands?1
X Y
Z T
Q
BX = 50 BY = 33BXY = 33BX =
50BY = 33
TYXZ
XY XY
BY = 33QY
X Y
BX
Z
VS1
Q
BQ
TBX = … = BQ = 100Mbps
Hypervisor divides guarantees max-min based on demands
(future demands estimated based on history)
Guarantee Partitioning – Efficiency
2How to avoid unallocated guarantees?
X Y
Z T
Q
BX = 50 BY = 33BXY = 33BX =
50BY = 33
TYXZ
XY XY
BY = 33QY
X Y
BX
Z
VS1
Q
BQ
TBX = … = BQ = 100Mbps
What happens when flows have low demands?1
66
33
Guarantee Partitioning – Efficiency
X Y
Z T
Q
BX = 50 BY = 33BXY = 33BX =
50BY = 33
TYXZ
XY XY
BY = 33QY
X Y
BX
Z
VS1
Q
BQ
TBX = … = BQ = 100Mbps 6
6
33Source considers destination’s
allocation when destination is bottleneck
Guarantee Partitioning converges
Outline Motivation and Goals Overview More Details
Guarantee Partitioning Rate Allocation
Evaluation
Rate Allocation
Rate Allocation
BXY
Guarantee Partitioning
X
Y
Congestion data
Rate RXY
Spare bandwidth
Time
RXY
BXY
Fully used
Limiter
Rate Allocation
Rate Allocation
BXY
Guarantee Partitioning
X
Y
Congestion data
Rate RXYLimite
r
RXY = max(BXY , RTCP-like)
Rate AllocationRXY = max(BXY , RTCP-like)
144.178 144.678 145.178 145.678 146.178 146.678 147.178 147.678 148.1780100200300400500600700800900
1000Rate Limiter Rate
Seconds
Mbp
s
Guarantee
Another Tenant
X Y
Rate AllocationRXY = max(BXY , Rweighted-TCP-
like)
Weight is the BXY guarantee
BXY = 100Mbps
Z TBZT = 200Mbps
L = 1GbpsRXY = 333Mbps
RXT = 666Mbps
Rate Allocation – Congestion Detection
Detect congestion through dropped packets Hypervisors add/monitor sequence numbers in
packet headers
Use ECN, if available
Rate Allocation – Adaptive Algorithm
Use Seawall [NSDI’11] as rate-allocation algorithm TCP-Cubic like
Essential improvements (for when using dropped packets)
Many flows probing for spare bandwidth affect guarantees of others
Rate Allocation – Adaptive Algorithm
Use Seawall [NSDI’11] as rate-allocation algorithm TCP-Cubic like
Essential improvements (for when using dropped packets) Hold-increase: hold probing for free bandwidth after a
congestion event. Holding time is inversely proportional to guarantee.
GuaranteeRate increasing
Holding time
Outline Motivation and Goals Overview More Details
Guarantee Partitioning Rate Allocation
Evaluation
Evaluation Setup
Implementation in Linux Logic in user-space: controls rate limiters, sends
control packets Modified kernel OVS
Testbed ~100 servers 1Gbps tree network
Evaluation – Many-to-one
Z
XL = 1Gbps
…Z
VS2
X
VS1
450Mbps
450Mbps
UDP
TCP
Edge or core
0 1 2 10 100 200 3000
100200300400500600700800900
1000
Evaluation – Many-to-oneTh
roug
hput
(Mbp
s)
Senders to Z
XZ
0 1 2 10 100 200 3000
100200300400500600700800900
1000
Evaluation – Many-to-oneTh
roug
hput
(Mbp
s)
Senders to Z
VM Z takes all the bandwidth
X
…
No Protection
Z
0 1 2 10 100 200 3000
100200300400500600700800900
1000
Evaluation – Many-to-oneTh
roug
hput
(Mbp
s)
Senders to Z
No ProtectionStatic Reservation (e.g., Oktopus)
Wasted bandwidth
XZ
0 1 2 10 100 200 3000
100200300400500600700800900
1000
Evaluation – Many-to-oneTh
roug
hput
(Mbp
s)
Senders to Z
No ProtectionStatic Reservation (e.g., Oktopus)
ElasticSwitch
Work-conserving
XZ
0 1 2 10 100 200 3000
100200300400500600700800900
1000
Evaluation – Many-to-oneTh
roug
hput
(Mbp
s)
Senders to Z
No ProtectionStatic Reservation (e.g., Oktopus)
ElasticSwitch
Provides guarantees
XZ
0 1 2 10 100 200 3000
100200300400500600700800900
1000
Evaluation – Many-to-oneTh
roug
hput
(Mbp
s)
Senders to Z
ElasticSwitch Ideal behavior
XZ
Evaluation – MapReduce Setup
44 servers, 4x oversubscribed topology, 4 VMs/server Each tenant runs one job, all VMs of all tenants same
guarantee
Two scenarios: Light
10% of VM slots are either a mapper or a reducer Randomly placed
Heavy 100% of VM slots are either a mapper or a reducer Mappers are placed in one half of the datacenter
0.2 0.5 1 50
0.2
0.4
0.6
0.8
1
Evaluation – MapReduceCD
F
Worst case shuffle completion time / static reservation
0.2 0.5 1 50
0.2
0.4
0.6
0.8
1
Evaluation – MapReduceCD
FNo ProtectionElasticSwitch
Light Setup
Work-conserving pays off: finish faster than static reservation
Worst case shuffle completion time / static reservation
Longest completion reduced from No Protection
0.2 0.5 1 50
0.2
0.4
0.6
0.8
1
Evaluation – MapReduceCD
F
Heavy Setup
ElasticSwitch enforces guarantees in worst case
Worst case shuffle completion time / static reservation
No Protection
ElasticSwitch
Guarantees are useful in reducing worst-case shuffle completion
up to 160X
ElasticSwitch Summary Properties
1. Bandwidth Guarantees: hose model or derivatives2. Work-conserving3. Practical: oversubscribed topologies, commodity switches,
decentralized
Design: two layers Guarantee Partitioning: provides guarantees by transforming
hose-model guarantees into VM-to-VM guarantees Rate Allocation: enables work conservation by increasing
rate limits above guarantees when no congestion
HP Labs is hiring!
Backup Slides
Evaluation – OverheadNon-linear in number of limiters
Splitting control per hosted VM should keep operation point here
Can be significantly improved!
0 50 100 150 200 250 3000
20
40
60
80
100
Number of VM-to-VM flows
Over
head
of o
ne C
PU c
ore
(%)
Guarantee Partitioning – Dynamic Behavior Optimize for bimodal distribution flows
Most flows short, a few flows carry most bytes Short flows care about latency, long flows care
about throughput
Start with a small guarantee for a new VM-to-VM flow
If demand not satisfied, increase guarantee exponentially
Short Flows
Short Flows/s0 20 40 60 80 100
Thro
ughp
utLo
ng-F
low
(Mbp
s)
050
100150200250300350400450500
1 new VM-to-VM pair/sec10 new VM-to-VM pairs/sec20 new VM-to-VM pairs/sec30 new VM-to-VM pairs/sec50 new VM-to-VM pairs/sec
Latency
# VM-to-VM flows in background
0 1 5 20 23(Tight Guar.)
Com
plet
ion
time
(ms)
00.20.4
0.60.8
1
1.21.41.6
1.82
No ProtectionOktopus-like Res.ElasticSwitch
Future Work
Reduce Overhead ElasticSwitch: average 1 core / 15 VMs ,worst case
1 core /VM
Multi-path solution Single-path reservations are inefficient No existing solution works on multi-path networks
Guarantee Partitioning – Efficiency: Reallocation
21 3 4
Intuition: propagates from smaller to larger guarantee shares
Guarantee Partitioning converges
Rate AllocationRXY = max(BXY , Rweighted-TCP-like)
Weight is the BXY guarantee
Order of magnitude longer timescale than RTT – not affects TCP