Evolution of P2P file sharing · Freenet uses semantic-free references to make the keys independent of the le content; this is achieved by using hash-based keys Keys are created using

Evolution of P2P file sharing

Dmitri Moltchanov

Department of Communications EngineeringTampere University of Technology

[email protected]

September 10, 2014

Based on slides provided by R. Dunaytsev http://utopia.duth.gr/rdunayts/

Outline

1 Freenet

2 Statistics

Dmitri Moltchanov (TUT) TLT-53206, Addon to lecture 2 September 10, 2014 2 / 27

Outline

1 Freenet

2 Statistics


Freenet

The shutdown of the original Napster inspired the creation of pureP2P systems such as Gnutella v0.4, which unlike Napster haveno central control

Freenet was proposed by Ian Clarke in 1999 as a distributed P2Pfile sharing and data storage system

MSc thesis ”A Distributed Decentralized Information Storage andRetrieval System”, University of Edinburgh, 1999

The Freenet protocol is an open-source one and has been undercontinuous development since 2000

freenetproject.org


http://freenetproject.org

Freenet (cont’d)

Motivation – Governments around the world undertake efforts toforce ISPs to block access to content deemed unsuitable or subversive,or to make them liable for such material hosted on their servers

Objective – Privacy for information producers/consumers/holdersand resistance to information censorship

Key features – Internet-wide information storage and anonymousinformation publication and retrieval


Freenet (cont’d)

How Freenet works:1 Each Freenet participant runs a node (peer) that provides the network

with some storage space2 A file is inserted into the network with an associated key3 After ”insertion” is finished, the publisher is free to shut down his

node, since the file is stored in the network4 During a file’s lifetime, it can migrate to or be replicated on other nodes5 To retrieve a file, a user sends out a request message containing the key6 When the request finds a node containing a copy of the file, the file is

returned through the search path


Freenet (cont’d)

Freenet basic messages:

Insert – allows a node to insert new data into the network; themessage includes the data file and the keyRequest – a request for a certain file; the request contains the key ofthe fileReply – sent by the node that has the requested file; the actual file isincluded in the reply messageFailed – denotes a failure to insert or locate a file; the messagecontains the location of the node where the failure occurred and thereason

For more information, see FCPv2 (Freenet Client Protocol version 2):

new-wiki.freenetproject.org/FCPv2


http://new-wiki.freenetproject.org/FCPv2

Freenet (cont’d)

The Freenet routing algorithm issteepest ascent hill climbing with backtracking

Limited by a preset Time To Live (TTL) value

Simple hill climbing – the first closer node is chosen

Steepest ascent hill climbing – all available nodes are comparedand the closest to the solution is chosen

Backtracking – when the search reaches a dead end (e.g.,”plateau”), it simply returns back to the previous node and triesalternative paths


Freenet (cont’d)

Bootstrapping:

The process starts when a new node is announced in the networkA node new in Freenet needs to have a public/private key pair and toobtain an identifier (ID) for itselfThe node ID is a number between 0 and 1, and it is derived throughthe announcement processThe announcement message contains the public key and an address ofsome existing nodeThis announcement message is propagated by Freenet nodesThe message has a TTL value, which determines when the messagepropagation is stoppedWhen the message propagation stops, the nodes in the chaincollectively assign a new ID for the new node and some subspace of thekeyspace


Freenet (cont’d)


Freenet (cont’d)

Keys:

Each file that exists on Freenet has a key associated with itFreenet keys are somewhat analogous to URLs on the WWW, exceptunlike URLs, they do not point to the physical location of the dataFreenet uses semantic-free references to make the keys independentof the file content; this is achieved by using hash-based keysKeys are created using SHA (Secure Hash Algorithm)

Content Hash Keys (CHK) – usually for nonmutable files

E.g., mp3 files

Signed Subspace Keys (SSK) – usually for mutable files

E.g., sites (aka freesites)

In Freenet, shared files are encrypted and the encryption keys areseparated from the actual data


Freenet (cont’d)

CHK@e3uDsrpIf0ShvjOFECKYvv2cpLv-ca1L9Thr7jHRLjg,LyTnto9tMEjo6B8gCBVYHaIKZhPqWhLgJqHJ5Dr8OZ8,AAIC–8/TLT-2626.txt

wiki.freenetproject.org/FreenetCHKPages


http://wiki.freenetproject.org/FreenetCHKPages

Freenet (cont’d)

SSK@qjYwosBCuWZEiqcnAWD~jsoHlzqTIYnW~GdiFY63244,BcIeuEaaPFhGNjPIrJwaNBgE6Y9-i1VlJyyCcatRB2Y,AQACAAE/TLT-2626.txt

wiki.freenetproject.org/FreenetSSKPages


http://wiki.freenetproject.org/FreenetSSKPages

Freenet (cont’d)

Inserting data:

For every piece of data, there is an associated keyEach node has a routing table, which with respect to the key, gives anordering of neighbors after their desirability (closeness) as thedestination of the queryThe route is then constructed by going from node to node andselecting the most suitable neighborWhen this is not possible, the request backtracks and the route isrestarted from the previous nodeThe routing process is terminated either due to the request achievingits purpose or the TTL field is found to be 0

Thus, a request is routed to the node that has the numerically closestID value to the key


Freenet (cont’d)

Retrieving data:

To search for a file in the network, matching a key, one establishes aroute for the keyAt each step, the node checks its cache to see if a file associated withthat key is presentIf such a file is found, the search terminates and the file is returned tothe previous node in the route, which relays it back towards the nodewhich initiated the requestOtherwise, the request is forwarded to the node with the closestmatching IDThis routing process is repeated until either the file is found or theTTL field becomes 0


Freenet (cont’d)

Storing data:

During the proxying of data at both the insert and request steps, nodesstore the data in their cacheEach node has 2 separate cachesOne is a short-term cache where all data that the node transfers arestored temporarily until they are pushed out by other dataThe other a long-term cache for storing only inserted data that matchthe node ID

The network can be viewed as a large grid of caching proxy hosts,each proxying and caching for one another


Freenet (cont’d)

Key-based routing

The request moves through the network from node to node, backingout of a dead end (Step 3) and a loop (Step 7) before locating thedesired file

Requestor

A B C

F E D

Data holder

1 2

3

45

6

7

8 9

10

12

11

= request

= failed

= reply


Freenet (cont’d)

Summary of Freenet :

Developer(s) – Ian Clarke

Launched – 2000

Protocol – open source (Freenet)

Clients – freeware (Freenet, Frost, . . . )

Architecture – pure P2P

Search facility – keys / steepest ascent hill climbing search

Shared content – any (including freesites)


Outline

1 Freenet

2 Statistics


Statistics: traffic in Tbs

”Internet Study 2008/2009” and ”Internet Study 2007”by H. Schulze, K. Mochalski (ipoque GmbH)

8 regions1.1 million users represented1.3 petabytes of user traffic monitored (1 petabyte = 1024 terabytes)8 ISPs and 3 universities


Statistics: applications

Protocol class proportions 2008/2009

P2P generates most traffic in all regions


Statistics: yearly evolution

Protocol class proportions 2008/2009 vs. 2007

Lower percentage of P2P than in 2007Higher percentage of Web traffic mainly due to file hosting services(such as RapidShare, Megaupload, FileFactory, etc.)


Statistics: popular apps

Most popular protocols

BitTorrent is number oneHTTP is number two in most regionsAt least 2 P2P protocols are among top-5 in each region


Statistics: P2P protocols

Most popular P2P protocols

BitTorrent is the most popular P2P protocoleDonkey2000 is still the second, but much lower than in 2007Significant amount of Gnutella, iMesh, Thunder and DirectConnecttraffic in some regions


Statistics: NetFlow

Internet2 NetFlow statistics: netflow.internet2.edu

Aggregated application types (April 26, 2010):

File sharing 1.96% (that’s all???)Unidentified 46.83% (may be also P2P?)


http://netflow.internet2.edu

Statistics: Cisco

”Cisco Visual Networking Index: Forecast and Methodology,2009 - 2014”, June 2, 2010

In 2014, the Internet will be 4 times larger than it was in 2009

By year-end 2014, the equivalent of 12 billion DVDs will cross theInternet each month

P2P is growing in volume, but declining as a percentage ofoverall IP traffic

P2P file sharing networks are now carrying 3.5 exabytes per month andwill continue to grow at a moderate pace1 exabyte = 1024 petabytes, 1 petabyte = 1024 terabytesDespite this growth, P2P as a percentage of overall Internet traffic willdrop to 17% by 2014, down from 39% at the end of 2009

Internet video is now over 1/3 of all consumer Internet traffic andwill approach 40% of overall Internet traffic by the end of 2010

This is not including the amount of video exchanged through P2P!


Statistics: video

Global Internet traffic:

CAGR = compound annual growth rate1 petabyte = 1024 terabytes

The sum of all forms of video (IPTV, video on demand, P2P, etc.)will exceed 91% of overall Internet traffic in 2014

Internet video alone will account for 57% in 2014


Evolution of P2P file sharing · Freenet uses semantic-free references to make the keys independent of the le content; this is achieved by using hash-based keys Keys are created using

Documents