On Large-Scale Peer-to-Peer Streaming Systems with Network Coding Chen Feng, Baochun Li Dept. of Electrical and Computer Engineering University of Toronto.

On Large-Scale Peer-to-Peer Streaming Systems with Network

Coding

Chen Feng, Baochun LiDept. of Electrical and Computer

Engineering University of Toronto

Presentation by: Shabnam Mirshokraie

ACM Multimedia 2008

Peer to Peer Streaming Challenges How do we maintain a high playback

quality at all participating peers? How do we improve user experience

with the shortest initial buffering delay? How do we minimize server bandwidth

costs? How do we design a system that is

resilient to peer dynamics?

Related Work On achieving minimum delay

A centralized algorithm [Y. Liu ACM Multimedia 07]

On achieving maximum streaming rate A decentralized algorithm [Massoulie et al.

INFOCOM 07] On achieving near optimal steaming

rate and delay Several decentralized algorithms [Bonald et

al. SIGMETRICS 08]

Related Work (Cont.)

None of them is actually implemented by system designers No performance guarantees on

resilience Centralized algorithms go nowhere Complexity and overhead issues

Practical Implementation

Mesh based pull streaming strategies A live stream is divided into segments Segments arrive at a peer in roughly

sequential order

Mesh Based Pull Streaming

Cool Streaming (INFOCOM 05)

Mesh Based Pull Streaming (Cont.)

Advantages Simplicity of implementation Better resilience to peer dynamics

Disadvantages Significant overhead of requests and

buffer availability exchanges Longer initial buffering delays

Designing a Good P2P Streaming System

Simple to implement Low protocol overhead With theoretical guarantees on

Smooth playback Short initial buffering delay Low server bandwidth costs Resilience to peer dynamics

Streaming With Network Coding

segmenteach in blocks ofnumber total the

)(far so receivedbeen has which blocks coded ofnumber the

field Galois in the tscoefficien coding random ofset a:

block coded one :

],...,,[

.

21

1

m

mkk

c

x

bbbb

bcx

i

n

ii

k

i

Random Push on a Random Mesh

Random Push on a Random Mesh (Cont.) Traditional pull based strategies

Large buffer map size Frequently buffer map exchanges Explicit requests messages

Random push with random Network Coding Smaller buffer map Less frequent buffer map exchanges No need for explicit request messages

Synchronized Playback Synchronized playback buffers on all peers

All peers play the same segment at approximately the same time

Playback buffers overlap as much as possible The new peer skips a few segments

Receiving segments that are D seconds after the current point of playback

The duration of D seconds corresponds to the initial buffering delay

Performance Analysis of Coding Quantitative answers to the following

questions What are the sufficient conditions for Coding

to achieve good overall performance? How far from optimality is the performance

of Coding? Exploring the performance gap between

Coding and optimal streaming scheme Motivation for more elaborated designs

System Model and Notations

21

and 1 is sizeBlock

)21( as denoted ispeer class a ofcapacity Upload

)(strength Server

).(capacity server Relative

).(capacity peer average Relative

segment. ain blocks ofNumber

coding.network by induced blocksredundant ofFraction

crowd.flash a of Scale

seconds).(in delay buffering Initial

second).per blocks(in rate Streaming

second).per blocks(in capacity uploadServer

peers. ofcapacity upload Average

second).per blocks(in peer class a ofcapacity Upload

U U

,iUi

pNU

sU

Rs

U

su

Rp

U

pu

m

N

D

Rs

U

pU

ii

U

i

System Model and Notations (Cont.) Flash crowd scenario

most of the peers join the system in a short time period

Highly dynamic scenario peers join and leave the system in a

highly volatile Fashion (peer churn)

Flash Crowd Scenarios

capacity. uploadserver theis and

peers, ingparticipat ofcapacity upload average theis

as defined is ,by denoted strength,server The 2. DEFINITION

crowd.flash theduring

system in the peers ofnumber maximum theas defined is

,by denoted crowd,flash a of scale The 1. DEFINITION

sU

pU

pNU

sU

N

Flash Crowd Scenarios (Cont.)

. scale with crowdflash any for rate streaming aat quality

playbackperfect achieve toable is CodingThen coding.network

by induced blocks codeddependent linearly offraction thedenotes

and segment,each in blocks coded ofnumber theis where

(2) ln)1ln(

(1) )1(

:hold conditions following that theAssume 1. THEOREM

NR

mm

NRNUU ps

Flash Crowd Scenarios (Cont.) Theorem 1 establishes sufficient

conditions on smooth playback Heterogeneity in upload capacity in not an

issue in Coding

High bandwidth utilization

Flash Crowd Scenarios (Cont.)

1)1( :1 Theorem From

)(by bounded is :Proof

%1.0 oforder in the typicallyis and

ln)1ln(

bygiven is where

scheme, streaming optimal theof 1 offactor a within is

Coding rate, streaming esustainabl theof In terms 2. THEOREM

max

max

RR

N

NUUR

N

NUUR

m

ps

ps

Flash Crowd Scenarios (Cont.) Apply Theorem 1 to understand the gap

between Coding and optimal streaming

Theorem 2 demonstrates that Coding is near optimal in terms of sustainable streaming rate during a flash crowd.

Flash Crowd Scenarios (Cont.)

min

min

)1(2 1) Theorem from (replace 2segments 2 skippeer Each

)(

)(least at takes

playbacksmooth for segment one buffering of Process :Proof

ln)1ln(

bygiven is where

scheme, streaming optimal theof )1(2 offactor a within is

Coding delay, buffering initial theof In terms 3. THEOREM

DDRR

mD

NUU

NmD

NUU

Nm

m

ps

ps

Flash Crowd Scenarios (Cont.) Theorem 3 shows that Coding manages

to guarantee very short initial buffering delays during a flash crowd.

Theorem 2 and Theorem 3 suggest that the performance gap between Coding and optimal streaming scheme is small.

Flash Crowd Scenarios (Cont.)

1. Theorem ofCorollary :Proof

capacity.peer average relative theis and

ln1ln11

such that minimum the

is where, then , scale with crowdflash a during

rate streaming aat quality playback perfect supportsthat

capacityserver relative required thedenote usLet 4. THEOREM

**

p

p

ps

u

mu

NuuN

R

Validation of the required relative server capacity in several different flash crowd

scenarios

Impact of restricted neighborhoods on the playback quality

Highly Dynamic Scenarios

The arrivals of new peers in current time No effect on the playback quality of the

most urgent segment

The departures of existing peers Central role in the playback quality

Highly Dynamic Scenarios (Cont.)

slot. mecurrent ti of end

at theoccur peersbandwidth low of departures all whileslot,

mecurrent ti of beginning at thehappen peersbandwidth high of

departures all :follows as isslot mecurrent tiin case worst The

slot. mecurrent tiin peers

class of departures and arrivals ofnumber theare and

slot. mecurrent ti

of beginning at the system in the peers class ofnumber the

2

1

W

W

iWA

iN

ii

i

Assumptions:

Highly Dynamic Scenarios (Cont.)

RW ε UW UU

R N N U N U -WN U

R N Nε U N U N U

R.W ε UW

ss

s

s

111

2122111

212211

111

)1(obtain we and (4) , (3) From

(4) ))(1()(

:dynamicspeer case worst thehandle To

(3) ))(1(

:1 Theorem from slot, mecurrent tiin dynamicspeer no of caseIn :Proof

)1(

thanlessstrictly is slot, mecurrent tiin dynamicspeer handle to

required is which capacity,server additional The 5. THEOREM

0.5 set to is 2

ratio theand 2, set to is 1

ratio the000, 10, is scale System uuN

Theoretical and simulation results for relative additional server capacity to handle peer dynamics

in the worst case

The theoretical bound is tight when bandwidth supply barely exceeds bandwidth demand, while the bound is loose when supply outstrips demand.

Simulation results for relative additional server capacityto handle peer dynamics in the average case

Only a small amount of additional server capacity is required, even when 50% peers leave the system.

000 10, is scale System N

Formal Proof of Sufficient Conditions-Theorem 1.

Ni

N

in

in

iz

jt

jz

jn

jut

izt

iz

Ns

u

dt

(t)i

dzt

iZN

ti

Zst

ti

X

jj

Zj

Nj

us

uNi

Zkk

iZ

jj

Xj

Nj

us

uN-W

iX

kki

X

WNi

Xi

sW

ii

X

NuNuNp

ui

Ni

N

and )0(:conditions initial

))(()()( ofsolution the toconverges )(1

)()(arguments coupling Standard

rateat 1: withdeal tohard is process random This

rateat 1:

)/(peer gany workinby chosen been has class ofpeer idlean y that Probabilit

segment received completely peers ofnumber theis

state idlein peers class ofnumber theis

/)2211

(

mNNNmp

uN

NNmp

uNs

ti

Xi

mdtt

iX

iN

ium

sus

mN

tp

ue

δ

δ

δ

iN

ti

XN

ti

Z

tp

ue

δ

δ

δ

in

ti

z

)1()1ln(ln)1( : 1 Theoremin conditions Sufficient

)1ln(ln)1( than less no is segment within blocks

coded ofsupply maximum Thebound achieved previously by the )( Replacing0

segment within blocks coded ofsupply maximum The

)1( isplayback smooth achieve toblocks coded ofnumber expected The

)1(

11

1)(

)(

)1(

11

1)(

Formal Proof of Sufficient Conditions-Theorem 1. (Cont.)

Fraction of Redundant Blocks

Linearly dependent coded blocks from upstream peers Waste of bandwidth resources

Estimation of the fraction of redundant blocks Bandwidth utilization of Coding

Fraction of Redundant Blocks (Cont.)

field. Galois theof size theis where

11useful isblock x codedPr

Then, .peer topeer fromsent block coded

aConsider .namely peer, upstream s'peer of oneon spanned space thedenote

and peer on blocks coded by the spanned space thedenote Let . 1LEMMA

q

qdS

vS

dvx

vdv

Sdd

S

For the proof of Lemma 1. refer to “S. Deb, et al., Algebraic gossip: a network coding approach to optimal multiple rumor, IEEE Trans. on information theory

Fraction of Redundant Blocks (Cont.)

With high probability any coded block from an upstream peer is useful to its downstream peer The space spanned by the coded blocks on

the upstream peer is not a subspace of the space spanned on downstream peer.

Fraction of Redundant Blocks (Cont.)

qd

-pdv

pd

Sv

S

qp

11}peer block to coded useful a sendspeer upstream helpful aPr{

1}peer tohelpful is peer upstreamPr{

}Pr{, Where

1)

11)(1(

1

:follows asgiven is blocks codedredundant of

fraction expected themodel, simple In the 1. NPROPOSITIO

Fraction of Redundant Blocks (Cont.)

1)

11)(1(

1][

)1

1)(1(][][equality sWald'

...21

segment. original the

decodely successful topeer for needed blocks coded ofnumber thedenote Let

)1

1)(1( )1Pr(

.peer touseful isblock coded

th theevent that theoffunction indicator thebe Let :Proof

qpm

mME

qp

m

iYE

mME

mM

YYY

dM

q

- - p i

Y

d

ii

Y

Fraction of Redundant Blocks (Cont.) The randomized encoding algorithm on an

upstream peer does not take into account the coded blocks accumulated on its downstream peers

Producing some redundant coded blocks Size of the Galois field q

Upstream peer has no innovative coded blocks for its downstream peers

The probability of such event is small The random push operations naturally create

sufficient diversity

Simulation results for fraction of redundant coded blocks

The fraction of redundancy induced by network coding is in the order of 0.001, even when the field size q is as small as 64

Comparison with PullA comparison of playback quality between Coding and Pull

under different peer dynamic scenarios

The change of playback quality over time in Coding and Pull under a typical flash crowd

scenario and a highly dynamic scenario

Pull than dynamicspeer under

stabilitybetter much has Coding that shows comparison The

Summary Analytically investigation of the

performance of streaming systems with network coding Simple and effective streaming

Extensive large scale simulations The analytical results have been validated

Demonstrating the advantages of network coding based protocols over traditional pull based streaming protocols.