Peer2Peer

Distributed SystemsDistributed SystemsPeerPeer--toto--PeerPeer

Prof. Dr.-Ing. Torben WeisUniversität Duisburg-Essen

The History of P2P

USENET

1979

ICQ

1996

1998

Jabber

2000

GnutellaeDonkeyFreenet

2002

Kademlia

2005

WASTEAvalanche

2006

PNRP

Domains of P2P

4

Why PeerWhy Peer--toto--Peer?Peer?

Earlier days: ∙ Client-server architecture in area of distributed systems

∙ Problems:◦ High load on servers◦ CS systems not indefinitely scalable◦ Server acts as Single Point of Failure (SPOF)

Excessive use of internet required alternatives

Major goal: ∙ Distribute load fairly on all nodes participating in network

Paradigm ShiftParadigm Shift

End of ´90s Peer-to-Peer (P2P) systems begin to displace client-server systems

Benefits

Today P2P traffic sums up to 60% of all internet traffic

Self‐organizing

decentral fair (costs, bandwidth,…)

scalable dynamic

autonomous

anonymous

Institut für Informationstechnik

Torben Weis 6

Where does a peer live ?

∙ Peers live in an overlay above regular IP network∙ Connections between peers pose virtual links

◦ May correspond to a path consisting of several physical links∙ Overlay allows routing

messages to destinations not specified by IP address

What is a peer ?

P2P Attributes

Generations

1. Generation: Centralized P2P

ServerServer

PeerPeer

ConnectConnectQueryQuery

ReplyReply

TransferTransfer

e.g. Napster

•Server stores index•File transfer using P2P

•Scalability problems•Single point of failure

2. Generation: Pure P2P

PeerPeer

ConnectConnect

QueryQuery

ReplyReply

TransferTransfer

e.g. Gnutella

•All peers are the same•Queries/Pings are forwarded•No global knowledge•Very robust

•Performance problems•Message flooding•Colissions

•No hit guarantee

2. Generation: Hybrid P2P

PeerPeer

ConnectConnect

Super-PeerSuper-Peer

QueryQueryReplyReplyTransferTransfer

e.g. Fasttrack

•Different Roles•Regular Peers•Super-Peers

•Structure uses hierarchy•Super-Peers use localknowledge for queries•On miss, forward query

3. Generation: Distributed Hash Tables

PeerPeer

ConnectConnect

Key spaceKey space

DataData

QueryQueryReplyReplyTransferTransfer

e.g. Chord, Pastry…

•All peers are the same•No hierarchy•Fair load balancing•Nodes and objects are mapped in the same key space

Institut für Informationstechnik

Torben Weis 14

Napster - Development

∙ Developed by Shawn Fanning ◦ Born 1980 in Brockton, Massachusetts◦ Started studying Computer Science in Boston 1999◦ Foundation Napster Inc. 05/1999◦ Roxio buys remains of Napster in 2002 after bankruptcy

∙ Friends came up with the idea of Napster∙ No equivalent software for direct transfer available

Goal: Exchange music within circle of friends

Institut für Informationstechnik

Torben Weis 15

Napster – Network Structure

∙ Star-like Structure∙ Central server

◦ Farm consists of ~200 servers

∙ Servers store indices∙ User connect with server:

◦ Server-Client communication while searching

◦ Client-Client communication while transfering

Institut für Informationstechnik

Torben Weis 16

Napster - Tasks

∙ Portal for exchanging MP3 files∙ Distinct roles for clients and server

∙ Server◦ Indexes all .mp3 files in network◦ Relays communication between peers

∙ Clients◦ Goal: Download music◦ Upload list of shared files

∙ Mp3 Search◦ Client sends query to server◦ Server searches database◦ Server sends result set to client

∙ Mp3 Download1. Peer A sends query for song XY2. Server sends address for

peer B to A3. Client sends request to peer B4. Download commences

Institut für Informationstechnik

Torben Weis 17

Napster – File Transfer

∙ MP3 Download behind Firewall1. Peer A sends query for song2. Server tells connection data

of peer A to peer B3. Peer B opens connection to peer A4. Peer A starts download from peer B

Institut für Informationstechnik

Torben Weis 18

Napster – File Transfer

Institut für Informationstechnik

Torben Weis 19

Napster - Conclusion

∙ Pros:◦ Recent view on network due to central database◦ Support of MP3s only low risk on virus download

∙ Cons:◦ Scalability: bottleneck server farm ◦ Server poses Single Point of Failure ◦ No security, file transfers not encrypted◦ Censorship of database possible (using filters)◦ Freerider problem◦ No chunking possible, download only from single peer/file

high dependency on a single peer

Institut für Informationstechnik

Torben Weis 20

Napster - Summary

∙ Napster pioneered peer-to-peer systems∙ Only small use of P2P technology

◦ File download is p2p-based◦ File search is client-server-based

∙ Protocol is closed-source, reverse-engineering enabled development of OpenNap

∙ Napster got sued (4/2000), finally turned off (7/2001)∙ Napster now acts as legal, commercial music provider

Institut für Informationstechnik

Torben Weis 21

Gnutella - Development

• Started after prohibition of Napster (1999-2001) • Justin Frankel (Nullsoft) publishes V0.4 in March 2000• Mother company AOL stops distribution • Already downloaded thousand-fold• Reverse-engineering revealed protocol

Goal:• Simple exchange of music in company network• No usage of central components

Institut für Informationstechnik

Torben Weis 22

Gnutella – Properties

∙ Fully decentralized P2P network∙ Allows for download of all file types∙ No role allocation pure P2P∙ Each node is server and client Servent∙ Members are autonomous∙ Robust network, mainly 3-4 open connections

Problem:∙ Finding entry point (Bootstrapping)

◦ Host-Cache Server◦ List with known hosts from former sessions

Institut für Informationstechnik

Torben Weis 23

Gnutella - Messages

∙ Message ID◦ Distinct Identifier for messages in the network

∙ Payload Descriptor◦ Ping, Pong, Query, Query Hit, Push

∙ Time to Live (TTL)◦ Hops to go until packet is dropped, common value: TTL=7

∙ Hops◦ Hops packet has already taken

∙ Payload Length

Institut für Informationstechnik

Torben Weis 24

Gnutella – Network Structure

∙ No central index peers have to probe neighbors∙ Regularly broadcasting ping messages∙ Peer receives pong on same path it was sent, contains

◦ Information about address: IP, Port, Servent ID…◦ Amount and size of shared files

∙ Loss of a node may lead tonetwork partitioning

∙ Ping frequency vs.Up-to-dateness◦ Ping size = 22 bytes◦ 1000 peers/ 3 connections each

~64 MB/sec

Institut für Informationstechnik

Torben Weis 25

Gnutella - Search

∙ Broadcast query to neighbors∙ QueryHit contains servent ID, address, speed…∙ Search runtime equals breadth-first search (O(|V|+|E|)∙ Search only limited through TTL∙ Client sends request

GET /get/4356/foo.mp3 HTTP/1.0User-Agent: Gnutella

Connection: Keep-AliveRange: bytes=0-

Institut für Informationstechnik

Torben Weis 26

Gnutella – File Transfer

∙ Download using HTTP

∙ Use of push messages to bypass firewalls◦ Common query ends with timeout◦ Client sends push message (ID + address) to server◦ Server opens connection to client

∙ Download complicated if◦ Both peers behind Firewall◦ IP-Masquerading is used

HTTP 200 OKServer: Gnutella

Content-type: application/binary

Content-length: 3457827

Institut für Informationstechnik

Torben Weis 27

Gnutella – Problems

∙ Scalability◦ Massive traffic for keeping network up-to-date

∙ Reliability◦ In dense networks packets drop after 3 hops◦ Long paths reduce success rate

∙ Security ◦ No use of hash values◦ Similar to DDoS-attacks

∙ Privacy◦ Packets not encrypted

Institut für Informationstechnik

Torben Weis 28

Gnutella - Conclusion

Pros:∙ Very robust, connection init using TCP∙ Autonomous peers∙ Communication using UDP

Cons:∙ No guaranteed hits∙ Massive traffic and high latency∙ Massive scalability problems

Institut für Informationstechnik

Torben Weis 29

Chord – DHT-based P2P

∙ First 3rd Generation P2P system∙ Developed by Ion Stoica @ MIT in 2001∙ Scientific approach due to drawbacks of 2nd generation∙ Complete decentralization while

◦ Offering efficient and correct searches◦ Providing good scalability◦ Relying on flat network structure (no hierarchy)◦ Balancing load fairly on all nodes

∙ First use of distributed hash tables in P2P

Institut für Informationstechnik

Torben Weis 30

Chord – Use of DHTs

∙ Cryptographic function SHA-1◦ 160bit allow for addressing 2160 peers and objects◦ Collisions highly unlikely

◦ SHA-1 guarantees major variation even on minor changes• SHA-1(Franz)=b259d15d278969d8c6cc682bc5fb8c032a5a43de• SHA-1(Frank)=0df02da8548eeef2174c97c2ade67b4c5adc3160

◦ Keys in key space are • Equally distributed• Avoid collisions• Distinct

Institut für Informationstechnik

Torben Weis 31

Chord – Data Mapping

Nodes

Data

Example for 4 bit-space•f(x) = 3 * x mod 16•f(47) = 3 * 47 mod 16 = 141 mod 16 = 13

Institut für Informationstechnik

Torben Weis 32

Chord – Search

Example∙ 4 Nodes, 5 Object∙ Nodes responsible for

all keys between itselfand its predecessor

Search:∙ Nodes aware of both neighbors∙ Query direct neighbor∙ Runtime: O(n)

Institut für Informationstechnik

Torben Weis 33

Chord – Finger Tables

∙ Use of finger tables for abbreviations◦ N nodes, m entries◦ n=2m ⇔ m=log2n◦ Finger[k]

first node on circle that suceeds(n+2k-1)mod 2m, 1≤k ≤m

◦ Successor = finger [1]◦ Predecessor = previous node

// search the local table for the highest predecessor of id

n:closest preceding node(id)for i = m downto 1

if (finger[i] є (n; id))return finger[i];

return n;

Institut für Informationstechnik

Torben Weis 34

Chord – Improved Search

∙ Example: finger table for 22

∙ 22 searches 38◦ Query sent to node known to be closest lower then 38 30◦ 30 sends query to successor asking for responsibility yes, found data

i Address Node

1 22+20=23 262 22+21=24 263 22+22=26 264 22+23=30 305 22+24=38 396 22+25=54 55

Institut für Informationstechnik

Torben Weis 35

Chord – Adding new nodes

∙ New node q uses hash function to generate its ID:=55∙ Search for this ID delivers successor(55):=56∙ Correction steps:

◦ Predecessor of 56 (46) becomes predecessor of 55

◦ 55 becomes predecessor of 56◦ 55 becomes successor of 46◦ Copy finger table from 46 and

update all entries◦ All fingers from 46 have to check

their finger table too◦ Move data to new node if necessary

Institut für Informationstechnik

Torben Weis 36

Chord – Node leaves network

∙ Successor of 22 does not respond any more∙ Search next living finger (39)∙ Go backwards to last

functioning node∙ Last node becomes new

successor of 22

Institut für Informationstechnik

Torben Weis 37

Chord – Node leaves network

Problem: several nodes leave concurrently◦ Successor(22) = ?◦ Going backwards ends in 39

as 37 is not reachable◦ Successor(22)=39

though L is alive◦ Data from L not accessible

Solution: successor list◦ Nodes store

r=O(log n) successors22 knows of L and integrates it

Institut für Informationstechnik

Torben Weis 38

Chord - Conclusion

Pros:∙ Fully decentralized architecture∙ Equity among nodes, no role allocation∙ Improved scalability∙ Efficient and correct search methods: O(log n)

Cons:∙ Huge efforts to keep finger and neighbors up-to-date∙ Join and leave operations costly∙ No support for security, anonymity or firewalled users

Institut für Informationstechnik

Torben Weis 39

Summary

∙ P2P has shown new ways in exchanging data◦ Fairness regarding disk space, bandwidth…◦ Scalability allows for huge amounts of users◦ Improved robustness due to decentralization

∙ Still P2P is mainly found in prototypes◦ Especially 3rd generation applications only in scientific areas◦ Popular applications (Edonkey…) use 2nd generation protocols

∙ Future work◦ Use of P2P technology in Vista (PNRP for distributed DNS service)◦ OceanStore works on distributed data archives◦ Applications you build on your own…