This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
■ Decentralization enables deployment of applications that are:■ Highly available■ Fault-tolerant■ Self organizing■ Scalable■ Difficult or impossible to shut down
■ This results in a “grassroots” approach and “democratization” of the Internet
■ Problem: find the set of nodes S that store a copy of object O
■ Flooding: forward the search message to all neighbors (first Gnutella protocol)■ A search message contains either keywords or an object id■ Advantages:
▴ simplicity▴ no topology constraints
■ Disadvantages:▴ high network overhead (huge traffic generated by each search request)▴ flooding stopped by TTL (which produces search horizon)▴ only applicable to small number of nodes
■ Two-level overlay: use superpeers to track the locations of an object [Gnutella 2, BitTorrent]■ Each node connects to a superpeer and advertises the list of
objects it stores■ Search requests are sent to the supernode, which forwards them
to other supernodes■ Advantages:
▴ highly scalable■ Disadvantages:
▴ superpeers must be reliable, powerful and well connected to the Internet (expensive)
▴ superpeers must maintain large state▴ the system relies on a small number of superpeers
Data location in structured networks:Key-Based Routing
■ Structured networks: use a routing algorithm that implements Key-Based Routing (KBR) [Chord, Pastry, Overnet, Kad, eMule]
■ KBR (also known as Distributed Hash Tables) works as follows:■ nodes are (randomly) assigned unique node identifiers (nodeId)■ given a key k, the node whose nodeId is numerically closest to k
among all nodes in the network is known as the root of key k■ given a key k, a KBR algorithm can route a message to the root
of k in a small number of hops, usually O(log n)■ the location of an object with id objectId is tracked by the root of
k = objectId■ thus, one can find the location of an object by routing a message
to the root of k = objectId and querying the root for the location of the object
30
Structured overlay network: Chord
Basics
Each peer is assigned a unique m-bit identifier id .Every peer is assumed to store data contained in a file.Each file has a unique m-bit key k .Peer with smallest identifier id � k is responsible forstoring file with key k .succ(k): The peer (i.e., node) with the smallest identifierp � k .
NoteAll arithmetic is done modulo M = 2m. In other words, ifx = k ·M +y , then x mod M = y .
22 / 53
Structured overlay network: Chord
Basics
Each peer is assigned a unique m-bit identifier id .Every peer is assumed to store data contained in a file.Each file has a unique m-bit key k .Peer with smallest identifier id � k is responsible forstoring file with key k .succ(k): The peer (i.e., node) with the smallest identifierp � k .
NoteAll arithmetic is done modulo M = 2m. In other words, ifx = k ·M +y , then x mod M = y .
22 / 53
Example
0 12
3
4
5
6
7
8
9
10
11
12
13
14151617
1819
20
21
22
23
24
25
26
27
28
29
3031
Peer�9�storesfiles�with�keys5,�6,�7,�8,�9
Peer�1�storesfiles�with�keys29,�30,�31,�0,�1
Peer�20�storesfile�with�key�21
23 / 53
21
Efficient lookups
Partial view = finger table
Each node p maintains a finger table FTp[] with at most mentries:
FTp[i] = succ(p +2i�1)
Note: FTp[i] points to the first node succeeding p by atleast 2i�1.To look up a key k , node p forwards the request to nodewith index j satisfying
q = FTp[j] k < FTp[j +1]
If p < k < FTp[1], the request is also forwarded to FTp[1]
24 / 53
Example finger tables
succ(p�+�2����)i-10 12
3
4
5
6
7
8
9
10
11
12
13
14151617
1819
20
21
22
23
24
25
26
27
28
29
3031
1 42 43 94 95 18
1 92 93 94 145 20
1 112 113 144 185 28
1 142 143 184 205 28
1 182 183 184 285 1
1 202 203 284 285 4
1 212 283 284 285 41 28
2 283 284 15 9
1 12 13 14 45 14
i
25 / 53
Example lookup: 15@4
0 12
3
4
5
6
7
8
9
10
11
12
13
14151617
18
19
20
21
22
23
24
25
26
27
28
29
3031
1 4
2 4
3 9
4 9
5 18
1 9
2 9
3 9
4 14
5 20
1 11
2 11
3 14
4 18
5 28
1 14
2 14
3 18
4 20
5 28
1 18
2 18
3 18
4 28
5 1
1 20
2 20
3 28
4 28
5 4
1 21
2 28
3 28
4 28
5 4
1 28
2 28
3 28
4 1
5 9
1 1
2 1
3 1
4 4
5 14
1 FT4[4] 15 < FT4[5]) 4! 14
2 p = 14 < 15 < FTp[1]) 14! 18
26 / 53
Example lookup: 15@4
0 12
3
4
5
6
7
8
9
10
11
12
13
14151617
18
19
20
21
22
23
24
25
26
27
28
29
3031
1 4
2 4
3 9
4 9
5 18
1 9
2 9
3 9
4 14
5 20
1 11
2 11
3 14
4 18
5 28
1 14
2 14
3 18
4 20
5 28
1 18
2 18
3 18
4 28
5 1
1 20
2 20
3 28
4 28
5 4
1 21
2 28
3 28
4 28
5 4
1 28
2 28
3 28
4 1
5 9
1 1
2 1
3 1
4 4
5 14
1 FT4[4] 15 < FT4[5]) 4! 14
2 p = 14 < 15 < FTp[1]) 14! 18
26 / 53
Example lookup: 15@4
0 12
3
4
5
6
7
8
9
10
11
12
13
14151617
18
19
20
21
22
23
24
25
26
27
28
29
3031
1 4
2 4
3 9
4 9
5 18
1 9
2 9
3 9
4 14
5 20
1 11
2 11
3 14
4 18
5 28
1 14
2 14
3 18
4 20
5 28
1 18
2 18
3 18
4 28
5 1
1 20
2 20
3 28
4 28
5 4
1 21
2 28
3 28
4 28
5 4
1 28
2 28
3 28
4 1
5 9
1 1
2 1
3 1
4 4
5 14
1 FT4[4] 15 < FT4[5]) 4! 14
2 p = 14 < 15 < FTp[1]) 14! 18
26 / 53
Example lookup: 22@4
0 12
3
4
5
6
7
8
9
10
11
12
13
14151617
18
19
20
21
22
23
24
25
26
27
28
29
3031
1 4
2 4
3 9
4 9
5 18
1 9
2 9
3 9
4 14
5 20
1 11
2 11
3 14
4 18
5 28
1 14
2 14
3 18
4 20
5 28
1 18
2 18
3 18
4 28
5 1
1 20
2 20
3 28
4 28
5 4
1 21
2 28
3 28
4 28
5 4
1 28
2 28
3 28
4 1
5 9
1 1
2 1
3 1
4 4
5 14
1 FT4[5] 22) 4! 20
2 FT20[1] 22 < FT20[2]) 20! 21
3 p = 21 < 22 < FT21[1]) 21! 28
27 / 53
Example lookup: 22@4
0 12
3
4
5
6
7
8
9
10
11
12
13
14151617
18
19
20
21
22
23
24
25
26
27
28
29
3031
1 4
2 4
3 9
4 9
5 18
1 9
2 9
3 9
4 14
5 20
1 11
2 11
3 14
4 18
5 28
1 14
2 14
3 18
4 20
5 28
1 18
2 18
3 18
4 28
5 1
1 20
2 20
3 28
4 28
5 4
1 21
2 28
3 28
4 28
5 4
1 28
2 28
3 28
4 1
5 9
1 1
2 1
3 1
4 4
5 14
1 FT4[5] 22) 4! 20
2 FT20[1] 22 < FT20[2]) 20! 21
3 p = 21 < 22 < FT21[1]) 21! 28
27 / 53
Example lookup: 22@4
0 12
3
4
5
6
7
8
9
10
11
12
13
14151617
18
19
20
21
22
23
24
25
26
27
28
29
3031
1 4
2 4
3 9
4 9
5 18
1 9
2 9
3 9
4 14
5 20
1 11
2 11
3 14
4 18
5 28
1 14
2 14
3 18
4 20
5 28
1 18
2 18
3 18
4 28
5 1
1 20
2 20
3 28
4 28
5 4
1 21
2 28
3 28
4 28
5 4
1 28
2 28
3 28
4 1
5 9
1 1
2 1
3 1
4 4
5 14
1 FT4[5] 22) 4! 20
2 FT20[1] 22 < FT20[2]) 20! 21
3 p = 21 < 22 < FT21[1]) 21! 28
27 / 53
Example lookup: 22@4
0 12
3
4
5
6
7
8
9
10
11
12
13
14151617
18
19
20
21
22
23
24
25
26
27
28
29
3031
1 4
2 4
3 9
4 9
5 18
1 9
2 9
3 9
4 14
5 20
1 11
2 11
3 14
4 18
5 28
1 14
2 14
3 18
4 20
5 28
1 18
2 18
3 18
4 28
5 1
1 20
2 20
3 28
4 28
5 4
1 21
2 28
3 28
4 28
5 4
1 28
2 28
3 28
4 1
5 9
1 1
2 1
3 1
4 4
5 14
1 FT4[5] 22) 4! 20
2 FT20[1] 22 < FT20[2]) 20! 21
3 p = 21 < 22 < FT21[1]) 21! 28
27 / 53
Example lookup: 18@20
0 12
3
4
5
6
7
8
9
10
11
12
13
14151617
18
19
20
21
22
23
24
25
26
27
28
29
3031
1 4
2 4
3 9
4 9
5 18
1 9
2 9
3 9
4 14
5 20
1 11
2 11
3 14
4 18
5 28
1 14
2 14
3 18
4 20
5 28
1 18
2 18
3 18
4 28
5 1
1 20
2 20
3 28
4 28
5 4
1 21
2 28
3 28
4 28
5 4
1 28
2 28
3 28
4 1
5 9
1 1
2 1
3 1
4 4
5 14 1 p = 20 ⌅ 18 < FTp[1]6) 20! 21
2 FT20[5] < 18) 20! 4
3 FT4[4] 18 < FT4[5]) 4! 14
4 p = 14 < 18 < FTp[1]) 14! 18
28 / 53
Example lookup: 18@20
0 12
3
4
5
6
7
8
9
10
11
12
13
14151617
18
19
20
21
22
23
24
25
26
27
28
29
3031
1 4
2 4
3 9
4 9
5 18
1 9
2 9
3 9
4 14
5 20
1 11
2 11
3 14
4 18
5 28
1 14
2 14
3 18
4 20
5 28
1 18
2 18
3 18
4 28
5 1
1 20
2 20
3 28
4 28
5 4
1 21
2 28
3 28
4 28
5 4
1 28
2 28
3 28
4 1
5 9
1 1
2 1
3 1
4 4
5 14 1 p = 20 ⌅ 18 < FTp[1]6) 20! 21
2 FT20[5] < 18) 20! 4
3 FT4[4] 18 < FT4[5]) 4! 14
4 p = 14 < 18 < FTp[1]) 14! 18
28 / 53
Example lookup: 18@20
0 12
3
4
5
6
7
8
9
10
11
12
13
14151617
18
19
20
21
22
23
24
25
26
27
28
29
3031
1 4
2 4
3 9
4 9
5 18
1 9
2 9
3 9
4 14
5 20
1 11
2 11
3 14
4 18
5 28
1 14
2 14
3 18
4 20
5 28
1 18
2 18
3 18
4 28
5 1
1 20
2 20
3 28
4 28
5 4
1 21
2 28
3 28
4 28
5 4
1 28
2 28
3 28
4 1
5 9
1 1
2 1
3 1
4 4
5 14 1 p = 20 ⌅ 18 < FTp[1]6) 20! 21
2 FT20[5] < 18) 20! 4
3 FT4[4] 18 < FT4[5]) 4! 14
4 p = 14 < 18 < FTp[1]) 14! 18
28 / 53
Example lookup: 18@20
0 12
3
4
5
6
7
8
9
10
11
12
13
14151617
18
19
20
21
22
23
24
25
26
27
28
29
3031
1 4
2 4
3 9
4 9
5 18
1 9
2 9
3 9
4 14
5 20
1 11
2 11
3 14
4 18
5 28
1 14
2 14
3 18
4 20
5 28
1 18
2 18
3 18
4 28
5 1
1 20
2 20
3 28
4 28
5 4
1 21
2 28
3 28
4 28
5 4
1 28
2 28
3 28
4 1
5 9
1 1
2 1
3 1
4 4
5 14 1 p = 20 ⌅ 18 < FTp[1]6) 20! 21
2 FT20[5] < 18) 20! 4
3 FT4[4] 18 < FT4[5]) 4! 14
4 p = 14 < 18 < FTp[1]) 14! 18
28 / 53
Example lookup: 18@20
0 12
3
4
5
6
7
8
9
10
11
12
13
14151617
18
19
20
21
22
23
24
25
26
27
28
29
3031
1 4
2 4
3 9
4 9
5 18
1 9
2 9
3 9
4 14
5 20
1 11
2 11
3 14
4 18
5 28
1 14
2 14
3 18
4 20
5 28
1 18
2 18
3 18
4 28
5 1
1 20
2 20
3 28
4 28
5 4
1 21
2 28
3 28
4 28
5 4
1 28
2 28
3 28
4 1
5 9
1 1
2 1
3 1
4 4
5 14 1 p = 20 ⌅ 18 < FTp[1]6) 20! 21
2 FT20[5] < 18) 20! 4
3 FT4[4] 18 < FT4[5]) 4! 14
4 p = 14 < 18 < FTp[1]) 14! 18
28 / 53
The Chord graph
EssenceEach peer represented by a vertex; if FTp[i] = j , add arc h
�!i , ji,
but keep directed graph strict.
1
4
9
11
14
18
20
21
28
29 / 53
Chord: path lengths
ObservationWith dn
2 (i , j) = min{|i� j |,n� |i� j |}, we can see that every peeris joined with another peer at distance 1
2n, 14n, 1
8n, . . . ,1.
7.5
7.0
6.5
6.0
5.5
2 4 6 8 10 12 14 16 18 20
Ave
rage�p
ath
�length
Network�size�(x�1000)
30 / 53
Chord: degree distribution
600
500
400
300
200
100
20 40 60 80 100
Indegree
Occ
urr
ence
s
11 12 13 14 15 16 17
3500
2500
1500
500
Outdegree
Occ
urr
ence
s
31 / 53
Chord: clustering coefficient
0.12
0.11
0.10
0.09
201 5 10 15
NoteCC is computed over undirected Chord graph; x-axis showsnumber of 1000 nodes.
Data location in structured networks:Key-Based Routing
52
■ Structured networks■ Advantages:
▴ completely decentralized (no need for superpeers)▴ routing algorithm achieves low hop count for large network sizes
■ Disadvantages:▴ each object must be tracked by a different node▴ objects are tracked by unreliable nodes (which may disconnect)▴ keyword-based searches are more difficult to implement than with
superpeers (because objects are located by their objectid)▴ the overlay must be structured according to a given topology in order to
achieve a low hop count▴ routing tables must be updated every time a node joins or leaves the
■ Churn can have several effects on a P2P system:■ data objects may be become unavailable if all replicas disconnect■ routing tables may become inconsistent
(e.g., entries may point to disconnected nodes)■ the overlay may become partitioned if many nodes suddenly
■ Security in P2P systems is hard to enforce:■ Users have full control of their computers■ Modified clients may not follow the standard protocol■ Data may be corrupted■ Private data stored on remote computers may disclosed
■ The user cannot change its identity■ Solution: use a centralized, trusted Certification Authority
(CA)■ Each new user must obtain an identity certificate■ The certificate is digitally signed by the CA, whose public key is
known by all users■ A certificate cannot be forged (require the CA’s private key)■ To prove his identity, a user signs a message with his private key,
and attaches the corresponding certificate signed by the CA■ Strong identities prevent Sybil Attacks■ If an attacker is caught, it cannot easily rejoin the system