Optimizing Cost and Performance for Content Multihoming

Yale LANS

Optimizing Cost and Performance for Content Multihoming

Hongqiang Harry LiuYe Wang

Yang Richard YangHao WangChen Tian

Aug. 16, 2012

Yale LANS

Content Multihoming is Widely Used

Content Publisher

Content Viewers

CDN-2 CDN-3CDN-1

Yale LANS

Why Content Multihoming: Performance Diversity

Yale LANS

Why Content Multihoming: Performance Diversity

Table: The fraction of successful deliveries for objects with streaming rate of 1Mbps | 2Mbps | 3Mbps.

Diversity in different areas

Diversity in different streaming rates

Yale LANS

Why Content Multihoming: Cost Diversity

Concave Function Region Based

Amazon CloudFrontVolume in a charging period

MaxCDN LiquidWeb

Yale LANS

Our Goal

• Design algorithms and protocols for content publishers to fully take advantage of content multihoming to optimize – publisher cost and – content viewer performance.

Yale LANS

Key Question

• A content object can be delivered from multiple CDNs, which CDN(s) should a content viewer use?

Yale LANS

Key Challenges

• Online vs statistical CDN performance– e.g., real-time network congestions or server

overloading

• Complex CDN cost functions– e.g., the cost of assigning one object to CDN(s)

depends on other assignments => coupling

Yale LANS

Our Approach: Two-Level Approach

Efficient Optimal Object Assignment Algorithm

Local, Adaptive Clients

Guidance from Content Publishers

Statistical Performance

OnlinePerformance

Yale LANS

Roadmap

• Motivations• Global optimization

– Problem definition– CMO: An efficient optimization algorithm

• Local active client adaptation• Evaluations

Yale LANS

Roadmap

• Motivations• Global optimization

Problem definition– CMO: An efficient optimization algorithm

• Local active client adaptation• Evaluations

Yale LANS

Problem Definition: Network Partition

Global Network Location Area

Object-i

it

1ait 2a

it 4ait

5ait 7a

it 8ait

3 0ait

6 0ait

Location Object Exclusionai

a

Yale LANS

Problem Definition:CDN Statistical Performance

Global Network

CDN-k

99%

92%

Target Performance: 90%

1ai 2ai

7aj

80%

71{ , }aakF i j Statistical Performance: e.g.,

probability of successful deliveries in an area

Yale LANS

,,

{ }

,

,

,

min

. . , , 0 : 1

, , , : 0

, , : 0

ai k

r a ak i k i

x k r a r

a ai i k

k

a ak i k

ai k

C x t

s t i a n x

i k a i F x

i k a x

Problem Definition:Optimization Formulation

Problem Q

is the fraction of traffic put into CDN-k for location object,ai kx

Charging function in region r of CDN-k

Traffic volume in charging region-r of CDN-k

ai

All requests are served

Performance constraints

Yale LANS

Solving Problem Q: Why not Standard Convex Programming or LP

• To minimize a concave objective function• Problem scale is too large to be tractable:

– N objects, A locations and K CDNs => N*A*K variables, and N*K constraints

– For example, given N=500K, A=200 and K=3 =>300M variables and 100M constraints

Yale LANS

Roadmap

• MotivationsGlobal optimization

– Problem definitionCMO: An efficient optimization algorithm

• Local active client adaptation• Evaluations

Yale LANS

Developing the CMO Algorithm: Base

• Problem Q has an optimal solution which assigns a location object into a single CDN.

• The object assignment problem is still hard:Assignment Space K CDNs and

N location objects => KN assignment possibilities

Yale LANS

CMO Key Idea: Reduction in the Outcome Space

Assignment Space Outcome (CDN Usage) Space

Infeasible Assignments

There are only vertices points, where is the # of charging regions (a small #)

There are up to points with CDNs and location objects

NKK N

KRN

Yale LANS

Mapping From Object Assignment to Outcome

CDNs CDN-1 CDN-2Traffic in Region-1Traffic in Region-2

Location ObjectsTraffic in Area-1Traffic in Area-2

11v

21v

12v

22v

11t 0 1

2t 0

0 21t 0 2

2t

21,1 1x 1

2,1 1x 22,2 1x

1 11 2t t 0

22t

21t

Example assumption: Area-i is in charging Region-i

11,1 1x

Yale LANS

Extensions

• CDN subscription levels (e.g. monthly plan)– Introducing CDN capacity constraints

• Per-request cost– Adding a row which indicates the #request in outcome

• Multiple streaming rates– Considering each video at each encoding rate as an

independent object

Yale LANS

Extension Example: Per-request Cost

CDNs CDN-1 CDN-2Traffic in Region-1Traffic in Region-2

#Request

Location ObjectsTraffic in Area-1Traffic in Area-2

#Request

11v

21v

12v

22v

11t 0 1

2t 0

0 21t 0 2

2t

1 2 11 1 2n n n 2

2n

1 11 2t t 0

22t

21t

11,1 1x 2

1,1 1x 12,1 1x

22,2 1x

21n

22n1

1n12n

Example assumption: Area-i is in charging Region-i

Yale LANS

From Algorithm to System

Optimizer

CDN1CloudFront

CDN2MaxCDNPassive Client

CPDNS

Resolve obj-i.cp.com

CNAMEd3ng4btfd31619.cloudfront.net

Client IP area

Long-term scale statistics

Fast-scale fluctuations

Yale LANS

Roadmap

• Motivations• Global optimization

– Problem definition– CMO: An efficient optimization algorithm

Local active client adaptation• Evaluations

Yale LANS

Active Clients

h11h12

h21

Primary CDNBackup CDN

Yale LANS

Informing Active Client

Optimizer

CDN1CloudFront

CDN2MaxCDN Active Client

Manifestation Server

Requestcp.com/sample.flv

Passive Client

Resolve obj-i.cp.com

Get CNAMEd3ng4btfd31619.cloudfront.net

Multiple CDNs with priorities

Yale LANS

How to Select Multiple CDNs?

• The same CMO algorithm, where input CDNs are virtual CDNs (ranked CDN combinations)

• Example: Select 2 CDNs (primary + backup) for an active client:– Each pair of CDNs is a “virtual CDN”: k’ = (k, j) – Fk’ : the set of location objects that CDN k and CDN j

together can achieve performance requirement• Each with 90% statistics => together > 90%

– Objective function: for each location object ia , primary CDN k delivers the normal amount of traffic and backup j incurs backup amount of traffic.

Yale LANS

Active Clients: Adaptation Goals

• QoE protection (feasibility): – Achieve target QoE through combined available

resources of multiple CDN servers

• Prioritized guidance: – Utilize the available bandwidth of a higher priority

server before that of a lower priority server

• Low session overhead (stability): – No redistributing load among same-priority servers

unless it reduces concurrent connections

Yale LANS

Active Clients: Control Diagram

h11 h12

h11+h12

Primary CDN Backup CDNh11<R and h12>R

h11>R and h12<R

h11<R and h12<R?

h11<R and h12<Rh11+ h12>R

h12>Rh11>R

h11+h12+h21

h11 + h12<R

h11 + h12>R

Yale LANS

Realizing Control Diagram: Key Ideas

• Controlling the windows– AIMD– Total load control

• Using the sliding windows– Priority assignment

h11

h12

h21

Pieces to request in T

# pieces can be downloaded from the server in a period T

Yale LANS

Roadmap

• Motivations• Global optimization

– Problem definition– CMO: An efficient optimization algorithm

• Local active client adaptationEvaluations

Yale LANS

CMO Evaluation Setting

• 6-month traces from two VoD sites (CP1 and CP2):– Video size– #request in each area (learned from clients’ IP)

• Three CDNs– Amazon CloudFront– MaxCDN– CDN3 (private)

• #request prediction– Directly using #request last month in each area

• Compare 5 CDN selection strategies:– Cost-only, Perf-only, Round-robin, Greedy, CMO

Yale LANS

Cost Savings of CMO

Avg Saving: ~35% compared with Greedy

Yale LANS

Cost Savings of CMO

All three CDNs have good performance in US/EU

Avg Saving: ~40% compared with Greedy

Yale LANS

Active Client Evaluation Setting

• Clients– 500+ Planetlab nodes with Firefox 8.0 + Adobe Flash

10.1

• Two CDNs– Amazon CloudFront– CDN3

Yale LANS

Active Client Test Cases

• Stress test– CDN3 as primary; CloudFront as backup

• Two servers in two CDNs: primary1, backup1• Two servers in the primary CDN: primary1, primary2

– Control primary1’s capacity• Step-down -> recover• Ramp-down -> recover• Oscillation

• Large scale performance test:– CloudFront as primary, CDN3 as backup– We saw real performance degradations

Yale LANS

Stress Tests (Step-down)

Step-down Recovery

Different Priority

Same Priority

Yale LANS

Stress Tests (Ramp-down)

Ramp-down

Recovery

Different Priority

Same Priority

Yale LANS

Stress Tests (Oscillation)

Different Priority

Same Priority

Yale LANS

Active Client QoE Gain (CloudFront + CDN3)

Yale LANS

Active Client: Cost Overhead

Yale LANS

Conclusions

• We develop and implement a two-level approach to optimize cost and performance for content multihoming: – CMO: an efficient algorithm to minimize publisher cost

and satisfy statistical performance constraints– Active client: an online QoE protection algorithm to

follow CMO guidance and locally handle network congestions or server overloading.

Yale LANS

Q&A

Yale LANS

Related Work and Conclusions• CDN switchers: seamless switch from one CDN to another

– One Pica Image• CDN Load Balancers: executing traffic split rules among CDNs

– Cotendo CDN– LimeLight traffic load balancer– Level 3 intelligent traffic management

• CDN Agent: CDN business on top of multiple CDNs– XDN– MetaCDN

• CDN Interconnection (CDNi)– Content multihoming problem still exists in the CDN delegations.

Yale LANS

Backup Slides

Yale LANS

Searching Extremal Assignments

Space of V

(1) Separation Lemma: ** *, : , 0is extremal P P V V

(2) Recall: vv

V v e (3) We prove: *

* *, , : , ,k vis extremal P v k v P v e P v e

P

*V

V

(4) With a proper , we can find an extremal assignment:• For each object , there is a unique minimum element in set

Pv { , | }kP v e k

How to find a proper ?

How to enumerate all possible extremal assignments?

P

** *, : , ,v v

v v

is extremal P P v e P v e

Yale LANS

Picking Proper P

A special subset of P (S’): all elements in are distinct , : , ,k jv k j P v e P v e

, 0k jP v e e 1 k jv e e

2 k jv e e

P

We prove:• Each extremal assignment can be found by an element in S’• Two interior points from the same cell find the same extremal assignment

Cell Enumeration of Hyperplane Arrangements

Conclusion:• All possible extremal assignments are exhausted by S’.• The number of extremal assignments is no more than the #cell (polynamial

with #object).

A Proper P:• there is a unique minimum element in set v { , | }kP v e k

{ , | }kP v e k

Yale LANS

Realizing Control Diagram: Key Ideas

• Yry (revise next slide) Draw a figure w/–An active client–3 cdn servers–Label a sliding window to conn. to each CDN

–Say 3 key techniques to control and use the sliding window: total