Page 1
June 13, 2013 Amir H. Payberah ([email protected] ) 1
Live Streaming in P2P and Hybrid P2P-Cloud Environments for the Open Internet
Doctoral Thesis in Information and Communication TechnologyStockholm, Sweden, June 13, 2013, 13:00
Amir H. Payberah
Advisors:Prof. Seif HaridiDr. Jim Dowling
Page 2
June 13, 2013 Amir H. Payberah ([email protected] ) 2
Outline
● Introduction
● Contribution 1 (Sepidar and GLive)
Problem description
Solution
● Contribution 2 (CLive)
Problem description
Solution
● Contribution 3 (Gozar and Croupier)
Problem description
Solution
● Wrap Up
1
2
3
Page 3
June 13, 2013 Amir H. Payberah ([email protected] ) 3
Introduction
Page 4
June 13, 2013 Amir H. Payberah ([email protected] ) 4
Media Streaming
● Media streaming is a multimedia that is sent over a network and played as it is being received by end users.
● Users do not need to wait to download all the media.
● It could be
Live
On Demand
Page 5
June 13, 2013 Amir H. Payberah ([email protected] ) 5
Solutions for Media Streaming
Client-Server
Page 6
June 13, 2013 Amir H. Payberah ([email protected] ) 6
Solutions for Media Streaming
Client-Server Peer-to-Peer
Page 7
June 13, 2013 Amir H. Payberah ([email protected] ) 7
QoS in P2P Media Streaming Systems
● High playback continuity: Smooth playback
● Short playback latency (only for Live Streaming)
Page 8
June 13, 2013 Amir H. Payberah ([email protected] ) 8
P2P Media Streaming Challenges
● Churn in the system
● Free-riding problem
● Bottleneck in the overlay network
● Connectivity Problem (NAT)
Page 9
June 13, 2013 Amir H. Payberah ([email protected] ) 9
Thesis Contribution
● Churn in the system
● Free-riding problem
● Bottleneck in the overlay network
● Connectivity Problem (NAT)
Sepidar and GLive
CLive
Gozar and Croupier
1
2
3
Page 10
June 13, 2013 Amir H. Payberah ([email protected] ) 10
Sepidar / GLiveP2P Solutions for Live Media Streaming
1
Page 11
June 13, 2013 Amir H. Payberah ([email protected] ) 11
Problem Description
Page 12
June 13, 2013 Amir H. Payberah ([email protected] ) 12
● Building and optimizing a P2P overlay for live media streaming.
High QoS
● Bounded number of download connections and upload connections.
Download slot
Upload slot
Problem Description (1/5)
source
Page 13
June 13, 2013 Amir H. Payberah ([email protected] ) 13
● The media stream is split into a number of sub-streams.
● To provide the full media to all the nodes: complete assignment (A)
Assign all download-slots in a node.
Download distinct sub-stream.
Child
Parent
Problem Description (2/5)
1 2 3 4 5 6
1 3 5
2 4 6
Page 14
June 13, 2013 Amir H. Payberah ([email protected] ) 14
● Cost cijk: the number of hops from the owner of the upload-slots j, to the source for the sub-stream k.
Problem Description (3/5)
j
i
cijk
source
Page 15
June 13, 2013 Amir H. Payberah ([email protected] ) 15
● An optimization problem:
● Objective function
Find a complete assignment that minimizes the total cost:
● Subject to
Each download-slot is assigned to exactly one upload-slot.
Each upload-slot is assigned to at most one download-slot.
The download-slots owned by the same node download distinct sub-streams.
Problem Description (4/5)
j
i
cijk
Page 16
June 13, 2013 Amir H. Payberah ([email protected] ) 16
● Centralized solution:
Needs global knowledge.
Possible for small system sizes.
● Distributed solution:
No global knowledge.
Inspired by auction algorithms.
Problem Description (5/5)
Page 17
June 13, 2013 Amir H. Payberah ([email protected] ) 17
Solution
Page 18
June 13, 2013 Amir H. Payberah ([email protected] ) 18
● What overlay topology is built for data dissemination?
Tree
Multiple-tree
Mesh
● What algorithm is used for data dissemination?
Push
Pull
Push-Pull
● How to construct and maintain this overlay?
Centralized
DHT
Gossip-based
...
P2P Streaming Overlay Construction Design Space
Page 19
June 13, 2013 Amir H. Payberah ([email protected] ) 19
● What overlay topology is built for data dissemination?
Tree
Multiple-tree
Mesh
● What algorithm is used for data dissemination?
Push
Pull
Push-Pull
● How to construct and maintain this overlay?
Centralized
DHT
Gossip-based
...
Sepidar vs. GLive
Sepidar
Multiple-treePush
Gossip
Page 20
June 13, 2013 Amir H. Payberah ([email protected] ) 20
● What overlay topology is built for data dissemination?
Tree
Multiple-tree
Mesh
● What algorithm is used for data dissemination?
Push
Pull
Push-Pull
● How to construct and maintain this overlay?
Centralized
DHT
Gossip-based
...
Sepidar vs. GLive
Sepidar
Multiple-treePush
Gossip
GLive
MeshPull
Gossip
Page 21
June 13, 2013 Amir H. Payberah ([email protected] ) 21
● Complete assignment that minimizes the costs.
● Each node knows only a small number of nodes in the system.
● Putting the nodes with higher number of upload slots closer to the source.
Streaming Overlay Construction (1/2)
Page 22
June 13, 2013 Amir H. Payberah ([email protected] ) 22
● Child nodes bid for better parents.
Closer to the source.
Use their number of upload-slots as their money.
● Parent nodes accept the highest bid.
Streaming Overlay Construction – Auction Model (2/2)
source
Candidate Parents
Page 23
June 13, 2013 Amir H. Payberah ([email protected] ) 23
Two More Questions?
● How to make the partial view at each node?
● How to handle free-riding nodes?
Page 24
June 13, 2013 Amir H. Payberah ([email protected] ) 24
Similar-view
Finger
● The Gradient overlay.
● Limit exploration to the set of nodes with a similar number of upload-slots.
Random view
How to Build Partial View at Each Node?
QN
MX
P
Page 25
June 13, 2013 Amir H. Payberah ([email protected] ) 25
● Freeriders are nodes that supply less upload bandwidth than claimed.
● Nodes identify freeriders through transitive auditing using their children’s children.
● Punish free-riders.
How to Prevent Free-riding?
Page 26
June 13, 2013 Amir H. Payberah ([email protected] ) 26
All Nodes are Equal, but Some Nodes are More Equal
Page 27
June 13, 2013 Amir H. Payberah ([email protected] ) 27
● P2P overlays for live media streaming.
● Distributed market model to construct the streaming overlay.
● The Gradient overlay to speed up the overlay construction.
● Transitive auditing to detect the free-riders.
Sepidar/GLive Summary
Page 28
Amir H. Payberah ([email protected] )June 13, 2013 28
CLiveA Hybrid P2P-Cloud Solution for Live Media Streaming
2
Page 29
June 13, 2013 Amir H. Payberah ([email protected] ) 29
Problem Description
Page 30
June 13, 2013 Amir H. Payberah ([email protected] ) 30
Problem Description (1/3)
● Bottlenecks in P2P video streaming systems: upload bandwidth
● A potential solution: P2P network is assisted by a cloud computing.
Page 31
June 13, 2013 Amir H. Payberah ([email protected] ) 31
Problem Description (2/3)
● Bottlenecks in P2P video streaming systems: upload bandwidth
● A potential solution: P2P network is assisted by a cloud computing.
Cloud is not free
Page 32
June 13, 2013 Amir H. Payberah ([email protected] ) 32
Problem Description (3/3)
● Bottlenecks in P2P video streaming systems: upload bandwidth
● A potential solution: P2P network is assisted by a cloud computing.
Cloud is not freeP2P vs. Cloud
Page 33
June 13, 2013 Amir H. Payberah ([email protected] ) 33
P2P vs. Cloud
● P2P P2P resources are cheap
Churn may compromise availability
● Cloud Superior availability
Cloud resources are not free
Page 34
June 13, 2013 Amir H. Payberah ([email protected] ) 34
We Cannot Beat Them, Let's Restrain Them
● The cloud as a support group for P2P.
● Reduce the number of cloud interactions as much as possible.
Occupy Wallstreet
Page 35
June 13, 2013 Amir H. Payberah ([email protected] ) 35
Solution
Page 36
June 13, 2013 Amir H. Payberah ([email protected] ) 36
Baseline Solution
SourcePH
● Rent passive helpers (PH), e.g., storage, from a cloud provider.
● Nodes pull the data from the PH if they cannot receive them from other nodes on time.
Page 37
June 13, 2013 Amir H. Payberah ([email protected] ) 37
CLive Solution
SourcePH
AHAH
AH
● Rent active helpers (AH), e.g., VM, in addition to PH.
Page 38
June 13, 2013 Amir H. Payberah ([email protected] ) 38
So?
Given that the costs of AHs and PHs are different, the goalis to minimize the total cost while delivering the desired QoS.
Page 39
June 13, 2013 Amir H. Payberah ([email protected] ) 39
Two Main Questions?
● How to estimate the extra load in the overlay?
● How many AH to add?
Page 40
June 13, 2013 Amir H. Payberah ([email protected] ) 40
How to Estimate the Extra Load?
● Load = swarm size – infected nodes
● The swarm size estimation: gossip-based aggregation
● Infected nodes:
The number of nodes that can be served by with the existing resources in the system, without the help of PH.
Tree-based diffusion pattern.
Estimate the tree depth.
Estimate the upload slot distribution: gossip-based aggregation
Page 41
June 13, 2013 Amir H. Payberah ([email protected] ) 41
How Many AH to Add?
● Calculate AH and PH cost in each round
● If load > ∂: add AH
● If load < ∂ - H: remove AH
H: number of peers served by one AH.
● Otherwise don't change AHs
Page 42
June 13, 2013 Amir H. Payberah ([email protected] ) 42
● Hybrid P2P-Cloud solution for for live media streaming.
● A combination of AHs and PH.
● Estimate the amount of extra load.
● Relay the extra load to the cloud.
● Add/remove AHs to minimise the cost.
CLive Summary
Page 43
Amir H. Payberah ([email protected] )June 13, 2013 43
Gozar/CroupierNAT-aware Peer Sampling and Distributed
NAT Traversal
3
Page 44
June 13, 2013 Amir H. Payberah ([email protected] ) 44
Problem Description
Page 45
June 13, 2013 Amir H. Payberah ([email protected] ) 45
Problem Description (1/10)
n1n2
n3
n4n5
n6n7
n8
n9
n10
n11
Page 46
June 13, 2013 Amir H. Payberah ([email protected] ) 46
Problem Description (2/10)
n1n2
n3
n4n5
n6n7
n8
n9
n10
n11
n8n7n10n5
n1n6n3n11
Page 47
June 13, 2013 Amir H. Payberah ([email protected] ) 47
Problem Description (3/10)
n1n2
n3
n4n5
n6n7
n8
n9
n10
n11
n8n7n10n5
n1n6n3n11
shuffle request
n8n7
Page 48
June 13, 2013 Amir H. Payberah ([email protected] ) 48
Problem Description (4/10)
n1n2
n3
n4n5
n6n7
n8
n9
n10
n11
n8n7n10n5
n1n6n3n11
shuffle response
n1n6
n8n7
Page 49
June 13, 2013 Amir H. Payberah ([email protected] ) 49
Problem Description (5/10)
n1n2
n3
n4n5
n6n7
n8
n9
n10
n11
n8n7n10n5
n1n6n3n11
n1n6
n8n7
Page 50
June 13, 2013 Amir H. Payberah ([email protected] ) 50
Problem Description (6/10)
n1n2
n3
n4n5
n6n7
n8
n9
n10
n11
n8n7n10n5
n1n3n6n11
n1n6
n8n7
UpdateState
UpdateState
Page 51
June 13, 2013 Amir H. Payberah ([email protected] ) 51
Problem Description (7/10)
n1n2
n3
n4n5
n6n7
n8
n9
n10
n11
n8n7n10n5
n1n3n3n11
n1n6
n8n7
Page 52
June 13, 2013 Amir H. Payberah ([email protected] ) 52
Problem Description (8/10)
n1
n2n3
n4
n5
n6
n7
n8
n9
n10
n11
Private node
Public node
shuffle request
Page 53
June 13, 2013 Amir H. Payberah ([email protected] ) 53
Problem Description (9/10)
n1
n2n3
n4
n5
n6
n7
n8
n9
n10
n11
Private node
Public node
shuffle response
UpdateState
UpdateState
Page 54
June 13, 2013 Amir H. Payberah ([email protected] ) 54
Problem Description (10/10)
n1
n2n3
n4
n5
n6
n7
n8
n9
n10
n11
Private node
Public node
shuffle request
Page 55
June 13, 2013 Amir H. Payberah ([email protected] ) 55
Solution
Page 56
June 13, 2013 Amir H. Payberah ([email protected] ) 56
● Node selection
Random
Tail
● View propagation
Push
Push-Pull
● View Selection
Blind
Healer
Swapper
PSS Design Space
Page 57
June 13, 2013 Amir H. Payberah ([email protected] ) 57
● Node selection
Random
Tail
● View propagation
Push
Push-Pull
● View Selection
Blind
Healer
Swapper
Gozar vs. Croupier
Gozar
Tail, Push-Pull, SwapperOne hop PSS
Built-in NAT Traversal
Page 58
June 13, 2013 Amir H. Payberah ([email protected] ) 58
● Node selection
Random
Tail
● View propagation
Push
Push-Pull
● View Selection
Blind
Healer
Swapper
Gozar vs. Croupier
Gozar
Tail, Push-Pull, SwapperOne hop PSS
Built-in NAT Traversal
Croupier
Tail, Push-Pull, SwapperPSS without Relaying
NAT Traversal can be added
Page 59
June 13, 2013 Amir H. Payberah ([email protected] ) 59
Croupier as a PSS (1/4)
● Private nodes vs. Public nodes
● All nodes only exchange their views with only public nodes.
● Public nodes, as croupiers and on be half of the private nodes, update their views.
Page 60
June 13, 2013 Amir H. Payberah ([email protected] ) 60
Croupier as a PSS (2/4)
● Uniform random selection
● Two views at each node: Public view Private view
● How to generate a random sample from from the public/private views?
public
private
Public view
Private view
8 red nodes, and 12 green nodes
X%?
Y%?
Page 61
June 13, 2013 Amir H. Payberah ([email protected] ) 61
Croupier as a PSS (3/4)
● Uniform random selection
● Two views at each node: Public view Private view
● How to generate a random sample from from the public/private views? Estimating the ratio of public to private nodes Gossip-based aggregation
public
private
Public view
Private view
8 red nodes, and 12 green nodes
60%
40%
Page 62
June 13, 2013 Amir H. Payberah ([email protected] ) 62
Croupier as a PSS (4/4)
● Public nodes: Counting the number of received shuffle requests in each round from
public and private nodes. Keeping track of the γ recent received estimation from public nodes.
● Private nodes: Keeping track of the γ recent received estimation from public nodes.
Page 63
June 13, 2013 Amir H. Payberah ([email protected] ) 63
Croupier as a NAT Traversal Middleware
● Each private node connects to one or more public nodes, called parents.
● A node's descriptor consists of its own address, its NAT type, and its parents addresses.
● Communicate with a private node through its parent.
Page 64
June 13, 2013 Amir H. Payberah ([email protected] ) 64
● One-hop relying vs. without relaying PSS.
● Public nodes behave as croupiers.
● Two views at each node: public and private views.
● Ratio of public/private nodes.
● Partnering private nodes with public nodes for NAT traversal.
Gozar/Croupier Summary
Page 65
Amir H. Payberah ([email protected] )June 13, 2013 65
Wrap Up
Page 66
June 13, 2013 Amir H. Payberah ([email protected] ) 66
● Sepidar and GLive
P2P solution for live streaming
Distributed market model
Gradient overlay
Transitive auditing
● CLive
Hybrid P2P-Cloud
Renting AH/PH from a cloud
● Gozar and Croupier
NAT-aware PSS with one-hop relaying and without relaying
Distributed NAT traversal
Summary
Page 67
June 13, 2013 Amir H. Payberah ([email protected] ) 67
● Handling the collusion attack.
● Putting all the components together and building a real system.
Future Work
Page 68
Amir H. Payberah ([email protected] )June 13, 2013 68
Thank You :)