Slide 1

NISSAN LEVTOV, NIKLAS CARLSSON, ZONGPENG LI , CAREY WILLIAMSON, SONG

ZHANG

DEPARTMENT OF COMPUTER SCIENCE, UNIVERSITY OF CALGARY.

Dynamic File-selection Policies for Bundling in BitTorrent-like

Systems

BitTorrent Systems

Files are split into many small pieces/ blocks A user interested in a file can join a “swarm” of users downloading the same file, and exchange blocks .Uses a Tit-for-tat mechanism to encourage cooperation and avoid free-riding .Offers high scalability and tolerance to flash crowdsHigh download rates for popular contents and large Torrents

Problem: unpopular files

However, there is no collaboration between peers downloading different files;

Peers downloading unpopular files join very small swarms (torrents) and often find the file unavailable or encounter very slow download rates.

Unpopular file availability is one of the main problems in BitTorrent –like systems

Solution: File Bundling

Content providers can bundle/group unpopular files into a single content

Leads to higher aggregate popularity for the bundled content.

Shown to increase availability for unpopular files in many cases (Menasche et al, 2009)

Bundling in economy was proposed decades ago as a mechanism for smoothing demands across multiple goods and extending monopoly power.

Current Bundling in BitTorrent

Bundling can be done by publishers in two manners:In pure static bundling peers are forced to

download the entire set of files, including the files they don’t need. Resulting in increased download times.

In mixed static bundling peers can choose to participate in downloading only a subset of files in the bundle. (often not supported by publishers)

Both methods do not offer any collaboration between peers downloading different files of various popularities and are not adjustable during download

Our Solution: Dynamic bundling

Bundling is done by tracker or publisher.Piece selection policies in the BitTorrent client

are modified to enable collaboration between peers interested in different files with homogenous popularities, while considering also the average and worst case download times.

The result is an aggregate torrent composed of smaller groups of file interests with cross group collaborations.

Our proposed mechanism is dynamic and adjusts to the torrent conditions during its lifetime.

Design Methods

Unlike most protocols for complex systems that are based on mathematical models and/or experimental results, we base our mechanisms on mathematical optimization and game-theoretic models, with their actual solutions for basic small systems. Namely: Markov Decision Process (MDP), and Stochastic games

The effectiveness of our solution is verified by simulations and experiments.

The Markov Decision Process Model(MDP)

MDP is a discrete time stochastic control process providing a natural framework for modeling dynamic decision making when outcomes are partly random.

Given a current system state and an action of the decision maker, the process randomly moves into a new state and the decision maker is given a reward.

Process possesses the Markov Property, i.e., independent of previous states and actions.

Goal: finding a policy that specifies which action to take in each state, s.t. a certain function of rewards is optimized.

MDP Model: Ideas

Decision makers model the downloader policies in the BitTorrent client, choosing a block of which file to download next.

The system state models the current number of blocks of each file that users have downloaded. (peers interested in the same file are aggregated into a single user type)

Transition into a new state depends on the downloading decisions and the current state, taking into account the unchoking mechanism in the uploader part.

MDP model: Notations and assumptions

We assume a system with two files f1 and f2.Each file contains k blocks.Three user types u1, u2, u3, interested in files f1,

f2 and both files, respectively. nu

t = the number of peers of type u at time t.S(u)f = the number of blocks of file f that u has .Peers have similar upload and download

capacitiesBlocks of each file are uniformly distributed; if u

currently has m blocks then each m-subset of 1,..,k has the same likelihood to appear.

MDP model goal: Social optimum

Users act as a single agent attempting to optimize the aggregate reward .

We consider three reward functions for different performance measures: Discounted average download rates: Maximize

#received blocks of desired files during a specified time, multiplied by a discount factor.

Average bounded download time: Minimize the average download time of desired files during a specified time.

Average availability: Peers gain 1 if finish downloading all the desired files, and 0 otherwise.

System file state: Number of blocks of each file that each type has downloaded.

Type1

Type2

Type3

Type actions (decisions)

At each time, each type u chooses a download actions d(u,v) against each type v, among the following: d1: u downloads only blocks of f1 from v d2: u downloads only blocks of f2 from v d3: u treats f1 and f2 a single combined file and

downloads a block independently of its file origin d4: u downloads an unwanted block from v only if v

does not have a block that u needs. This implies downloading blocks of the two files with priority to the wanted file.

MDP: State transition probabilities

Probabilities for moving to a new state are calculated based on the following:

The Stochastic Game Model

A stochastic game is a repeated multi-stage game with probabilistic state transitions.

Much like the Markov decision process only here players act as independent selfish agents attempting to optimize their own profit.

A strategy for each player is a choice of action to take in each state.

Solution concept- Nash Equilibrium: a choice of actions in which no players can benefit from deviating to another strategy.

Dynamic Programming Solution

Given the state transition function and reward function of MDP , we use dynamic programming to obtain an optimal solution, i.e., an optimal policy of file selection in each system state.

Due to complexity issues we assume each file is divided into a small number of blocks (up to 6 blocks per file)

For the stochastic game model we use dynamic programming to obtain a Nash Equilibrium when fixing the decisions made by type u3 to be d3 (we thus assume a two player game)

Solutions and Optimal Strategies

We analyzed the results obtained for the MDP model to gain insights about the optimal behavior of peers when seeking to minimize the overall average download times.

Results suggest that around 90% of the time, types choose to download only the file they want. (No collaboration).

Collaboration occurs mostly in cases where blocks of the desired file are dense but blocks of the other files are rare.

Collaboration occurs mostly in the end or beginning of the download process.

Piece selection Method Modification

We modify the piece selection method of the BitTorrent client to allow collaboration as implied by MDP model results in the following situations. Uploader is at the end of its download process: (If

uploader has many rare blocks then downloader will replicate them before uploader exits the system.)

Downloader is at the beginning of its download: (A new downloader benefits from quickly getting rare blocks to share.)

Downloader is at the end of its download: (We use a dynamic threshold for determining the number of blocks left to download, according to the number of are blocks. These blocks will be replicated before exiting the system).

Performance evaluation

We use discrete event simulation of BitTorrent. Example scenarios with a single seeder that leaves the system before all peers are completed.

Scenario 1: 200 peers , two files with 400 pieces each. Ratio between number of peers seeking the low and high popularities varies from 0 to 1.

Scenario 2: 200 peers, two files with 400 pieces each. Low/high popularity ratio is fixed to 0.1

Scenario 3: 250 peers, tow files with 500 pieces each. Low/high popularity ratio fixed to 0.05

Scenario 1 simulation results

Median, quartiles and extreme observations

Median, quartiles and extreme observations

Experimental results