Peer-to-Peer Systems · Preferred implementation approach Hook into p2p overlay routing • Add columns for statistic information ... On Unbiased Sampling for Unstructured Peer-to-Peer

Peer-to-Peer Systems

Winter semester 2014

Jun.-Prof. Dr.-Ing. Kalman Graffi

Heinrich Heine University Düsseldorf


Monitoring the Quality of Peer-to-Peer Systems

– Introduction to P2P Quality Monitoring

– Gossip-based solutions

– Tree-based solutions

3 HHU – Technology of Social Networks – JProf. Dr. Kalman Graffi – Peer-to-Peer Systems – http://tsn.hhu.de/teaching/lectures/2014ws/p2p.html

Quality of Service in P2P Networks

Quality of service (QoS)

“The well-defined and controllable behavior of a system with

respect to quantitative parameters”

Challenges for providing (constant) quality in p2p systems:

Various scenarios

• Distributed storage

• Content delivery

• Discovery and contacting of users

Dynamics over time

• Network size

• Churn

Peer heterogeneity

• Peer capacities

• Connectivity

User

Overlay

Application

Devices

Network

Manage-ment


Monitoring and Controlling System Metrics

Examples: Statistics: minimum, maximum, average, standard deviation

P2P system: peer count, online time,

Overlay “metrics”: Hop count, routing delay

Overhead: bandwidth consumption and traffic (per message type)

Resources: Free / used CPU, memory, storage space, bandwidth

Monitoring Obtain (global) knowledge on the system statistics

• Aggregatable: Size of statistics for 10 or 1000 peers equal

• Statistics must be fresh, monitoring mechanism of low overhead

Management / control Modify system to meet desired QoS properties

• Example: User defines valid interval for lookup delay

• Control mechanism to be fast responding and stable


Motivation for Monitoring and Management

Monitoring:

Information on system status can be used for optimized decisions

• E.g. peer count defines size of time-to-live

• E.g. churn pattern defines stabilization frequency

Necessary to identify (bad) quality of mechanisms

• Too much overhead

• Too slow routing

• Efficiency leaks

Helps in designing better mechanisms

Management / Controlling

Allows automated adaptation of system performance

Maintains quality level under uncontrollable dynamics

Adaptation only on-the-fly, no re-deployment possible

Both: Essential for commercial use


Deployment and Running a P2P System

Is everything running fine?

How to debug and to gain insight?

How to improve the running system?

Underlay:

The Internet

Structured

Overlay: DHT

H(„my data“)

= 3107

2207

7.31.10.25

peer-to-peer.info

12.5.7.31

95.7.6.10

86.8.10.18

planet-lab.orgberkeley.edu

29063485

201116221008

709

611

89.11.20.15

?


P2P Monitoring

Goal: Obtaining statistics on the global system Input: local states 𝑥𝑖(𝑡) of peers 𝑝𝑖 at time 𝑡 Output: global status (aggregate) at time t: 𝐹 𝑡 = 𝑓(𝑥1 𝑡 , … , 𝑥𝑛 𝑡 )

Statistical information: Aggregatable: average, minimum, maximum, sum, standard deviation

More difficult: median, top 1000 …

Aggregate function: 𝑓 𝑑𝑜𝑢𝑏𝑙𝑒∗ → 𝑑𝑜𝑢𝑏𝑙𝑒 Matches non-empty set of values to a single value

Hierarchical computational property Associative: 𝑓 𝑣1, 𝑣2, 𝑣3 = 𝑓 𝑓(𝑣1, 𝑣2), 𝑣3 = 𝑓 𝑣1, 𝑓(𝑣2, 𝑣3)

Commutative: 𝑓 𝑣1, 𝑣2 = 𝑓 𝑣2, 𝑣1

Operations: Add local statistics

Get global statistics


Aggregation Functions


General Challenges for Monitoring

Robustness Handling Churn

Coping with Link-Losses

Scalability Scaling in terms of participating peers

Scaling in terms of exchanged information

Performance High precision, low outliers

Efficiency Lightweight solution

Minimize complexity: easier to use, more robust

Applicability Applicable on every (structured) p2p overlay

Independent of any application


Design Decisions for P2P Monitoring

Topology: distributed vs. centralized

Layering: new vs. integrated

Monitoring topology: mesh, tree, ring, star … If tree, tree node position assignment: deterministic vs. dynamic

If dynamic roles, role assignement: heterogeneous vs. homogeneous

Monitoring scope: complete vs. interpolated

Monitoring view generation: proactive vs. reactive

Monitoring view update: push vs. pull


Approaches for P2P Monitoring

Centralized monitoring

SNMP, Nagios

Overlay-specific solutions

DASIS

Uniform Sampling

Gossiping

Tree-based structure

SkyEye.KOM

Variant: many trees

Variant: push-based protocol

Variant: reactive update pattern



– Selected Monitoring Approaches


► Centralized Monitoring

Idea Server receives, compiles and presents statistics

All relevant network devices attached

Example RFC 1988: SNMP – simple network monitor protocol

Nagios: open source solution • „Industry standard in IT infrastructure monitoring“

AMT: Intel‘s Active Management Technology • System-on-a-chip: device discovery, monitoring, remote shut-

down/restart

Pros: Works well for small networks (<10000)

Monitoring server becomes bandwidth/CPU/memory bottleneck

Cons: Not scalable to millions, single-point-of-failure

More a “reverse-multicast”


► Overlay-specific Solutions

Idea: extend existing overlays with monitoring abilities

Strong interleaving with a given overlay

Optimized and bound for specific overlay

DASIS (2004)

Distributed Approximative System Information Service (DASIS)

Assumes structured overlay with prefix-based routing

Modifies

• Routing tables

• Stabilization protocols

Goal

Every node with peerID = 1234567…n should know the statistics

for his prefixes 1,12,123,1234… K. Albrecht, R. Arnold, M. Gähwiler, and R. Wattenhofer: Aggregating Information in

Peer-to-Peer Systems for Improved Join and Leave. In IEEE P2P ’04: Proceedings

of the International Conference on Peer-to-Peer Computing, 2004


DASIS: Protocol Details

Monitoring protocol: per induction

To obtain status for prefix 1234…i

Aggregate own status for 1234…i(i+1)

With status information of “neighbor” 1234…i NOT(i+1)

Preferred implementation approach

Hook into p2p overlay routing

• Add columns for statistic information

of specific neighbor

Piggyback monitoring information

while stabilizing with neighbors

K. Albrecht, R. Arnold, M. Gähwiler, and R. Wattenhofer: Aggregating Information in

Peer-to-Peer Systems for Improved Join and Leave. In IEEE P2P ’04: Proceedings

of the International Conference on Peer-to-Peer Computing, 2004


DASIS Evaluation

Some notes Piggybacking has no influence of traffic overhead

• But reduces amount of packages relevant?

Strong interleave with p2p overlay limits applicability

• Strong assumptions on p2p overlay applicability?

• Both cannot be improved independently

Pro Monitoring topology relies on p2p overlay topology

• Avoids redundant (costy) stabilization protocols

Every node is informed about statistics

Cons Strong interleave with p2p overlay not extendable, applicable

Inefficient: exchanging monitoring information with O(logN) peers


► Uniform Sampling

Idea

Only as a representative set of nodes and interpolate results

Representative set: obtained by random walk

• With correction to not be biased to high-degree nodes

Random walk

Start at random node x

Probability to visit neighbor y:

Pos(x): Probability for being at specific node x

• Convergence of stationary distribution

Observation:

• Stationary distribution (Pos(x)) is biased to peers with high degree

• Random walks may visit nodes twice D. Stutzbach, R. Rejaie, N. Duffield, S. Sen, W. Willinger:

On Unbiased Sampling for Unstructured Peer-to-Peer Networks.

In: Proceedings of the 6th Internet Measurement Conference, IMC (2006)


Uniform Sampling in Undirected Graphs

Metropolized Random Walk with Backtracking (MRWB)

Modified transition matrix:

Protocol: Selecting the next step from peer x Select a neighbor y of x uniformly at random

Query y for a list of its neighbors, to determine its degree

Generate a random number, p, uniformly between 0 and 1

• If then y is the next step

• Otherwise, remain at x as the next step D.Stutzbach, R.Rejaie, N.Duffield, S.Sen, W.Willinger: On unbiased

sampling for unstructured peer-to-peer networks. In: Proceedings of the 6th

Internet Measurement Conference, IMC (2006)

𝑄 𝑥, 𝑦 =

𝑃 𝑥, 𝑦 ∙ min𝑑𝑒𝑔𝑟𝑒𝑒 𝑥

𝑑𝑒𝑔𝑟𝑒𝑒 𝑦, 1 𝑖𝑓 𝑥 ≠ 𝑦

1 − 𝑄 𝑥, 𝑦 𝑖𝑓 𝑥 = 𝑦𝑥≠𝑦

𝑝 ≤𝑑𝑒𝑔𝑟𝑒𝑒 𝑥

𝑑𝑒𝑔𝑟𝑒𝑒 𝑦


Uniform Sampling in Undirected Graphs

Protocol continued

Reactive protocol: interested peer initiates a sample query

Sample query uses random walk:

• Time-to-live is decreased

• Peers attach (aggregate) their statistics

• Once time-to-live vanished:

– send results to querying peer

Pros

Wide applicability

Cons

Limited scope: obtained results questionable

Not all statistics supported: sum, minimum, maximum

High overhead: every node queries for himself



– Gossip-based Approaches


► Gossiping Protocols

Idea:

Communicate only with neighbors (gossip)

• Assumes no specific overlay topology

Exchange and aggregate information

• E.g. calculate averages, minimum, maximum

Characteristics

Gossip protocols are round-based (epochs)

For every round

• Each node selects a subset of nodes to interact with (pairwise)

• The selection function is often probabilistic;

• Nodes interact via “small” messages

• Local state changes due to new information

In general: “quick” convergence

D. Kempe, A. Dobra,J. Gehrke, “Gossip-Based Computation of Aggregate

Information,” IEEE Symposium on Foundations of Computer Science (FOCS’03)


Example Average Calculation

Example: 12 nodes

Initial state

After 1 round

• With communication links

After 5 rounds

After 10 rounds


Calculation of the Sum

One nodes starts with 1, all others with 0

Create average, once converged: 𝑠𝑢𝑚 =1

𝑎𝑣𝑒𝑟𝑎𝑔𝑒

Example:


Assumptions

Input: local states 𝑥𝑖(𝑡) of peers 𝑝𝑖 at time 𝑡

Initialization defines aggregation function

round(0){

1. 𝑠0,𝑖 = 𝑥𝑖 (for average calculation)

2. 𝑤0,𝑖 = 1 (for average calculation)

3. 𝑠𝑒𝑛𝑑 𝑠𝑖 , 𝑤𝑖 𝑡𝑜 𝑠𝑒𝑙𝑓 }

Round (r>0){ 1. Let {(𝑠𝑟−1

∗ , 𝑥𝑟−1∗ )} be all pairs sent to 𝑖 during round r-1

2. 𝑠𝑟,𝑖 = 𝑠𝑟−1∗

∗ ; 𝑤𝑟,𝑖 = 𝑤𝑟−1∗

∗

3. Choose a target node 𝑗 ≠ 𝑖 uniformly at random

4. Send the pair (1

2𝑠𝑟,𝑖 ,1

2𝑤𝑟,𝑖) to j and self

5. 𝑠𝑟,𝑖

𝑤𝑟,𝑖 is the estimate of aggregate in round r }

Gossip-Protocol: PushSum


Initialization of PushSum

Result of gossiping: Input: 𝑠𝑖 , 𝑤𝑖

Output: F t =𝑎𝑣𝑒𝑟𝑎𝑔𝑒(𝑠𝑖)

𝑎𝑣𝑒𝑟𝑎𝑔𝑒(𝑤𝑖)

Calculating the average: For all nodes: 𝑠𝑖 = 𝑥𝑖; wi = 1

Output:F t =𝑎𝑣𝑒𝑟𝑎𝑔𝑒(𝑥𝑖)

1

Node count: One single node: 𝑠𝑖 = 1;𝑤𝑖 = 1 All other nodes: 𝑠𝑖 = 1;𝑤𝑖 = 0

Output: F t =1

1/𝑛 with 1/n being the average share of 1 among n peers

Calculating the sum: One single node: 𝑠𝑖 = 𝑥𝑖; 𝑤𝑖 = 1 All other nodes: 𝑠𝑖 = 𝑥𝑖; 𝑤𝑖 = 0

Output: F t =𝑎𝑣𝑒𝑟𝑎𝑔𝑒(𝑥𝑖)

1/𝑛


Observation:

During the gossiping: local input 𝑥𝑖 cannot be changed

Aggregations round-based (epochs)

Initializing an Epoch

Centralized

• Several leader algorithms exist

– E.g. node with maximum specific value

• Epoch may be restarted periodically by leader

Decentralized

• Besides local measure (𝑥𝑖) and weight (𝑤𝑖): add version number

• Every node may start new epoch concurrently

– Larger version numbers dominate smaller ones

– Epoch runs out after convergence and minimal value variations (< 𝜀)

Initializing an Epoch and Peer Election


Performance and Complexity of Push-Sum

Performance: precision

Simulations with 1M nodes

• Gossip every 5 second

For most time:

• False values

• Although convergence exist

Problem

• Peer count starts always at 0

Convergence time • n = number of nodes

• ε = accepted relative error

Push-Sum converges quickly

Problem:

• Huge message overhead per node

W. Terpstra, C. Leng, A. Buchmann: Brief Announcement: Practical Summation via

Gossip, ACM Symposium on Principles of Distributed Computing (PODC 2007)

In-

pre-

cise


Pros:

General applicability in (expander) graphs

• No assumptions on the overlay topology

Converges to final values

Cons:

Slow procedure, epochs are long

Lots of redundant communication

High traffic overhead

Gossip-based Monitoring Evaluation



– Tree-based Approach: SkyEye.KOM


► Tree-based Monitoring Mechanism

Idea:

Create (additional) tree topology on top of DHT

Protocol:

• Periodically

– Calculate aggregate of own local view and received from child nodes

– Send aggregate to parent node

• Root calculates global view

– And passes global view to all peers

Example: SkyEye.KOM (2009)

Assumes structured p2p overlay

Aims at high precision with low overhead


Design decisions in SkyEye.KOM

New layer (vs. integrated) New layer allows wider applicability

Set on top of KBR-compatible structured p2p overlays

Proactive (vs. reactive) System state information is continuously interesting for all users

Monitoring topology: tree (vs. bus, ring, star, mesh) Fixed out and in degree

Position assignment: dynamic and deterministic Deterministic IDs used in topology, dynamically resolved with DHT

For all structured P2P overlays Covered by DHT-function: route(msg, key), lookup(key)


Assumptions

Reliable structured p2p overlay

“Key-based Routing” – operations

• boolean isMyKey(Key K)

• void route(key K, Message M, node hint)

Building a tree topology

Introduce new overlay layer

• With own ID space ([0,1[)

• Practical infinite granularity

Create tree topology in new overlay

• Using routing of p2p structured overlay

Concept of new layer

Decouples from specific p2p overlay

Unified ID space [0,1]

1 10

50

20 30

40

45 15

P2P Overlay

0 1 0.09 0.2 0.31 0,4 0.5 0.6 0.75 0.9

Internet


SkyEye.KOM: Tree Topology

Tree of information domains Domain: ID interval

• E.g. [0, 0.5[ or [0.75, 0.875[

• Largest domain, level 0: [0,1[

Domain ID: “middle value” in interval

Domain size split in β parts per level

Domain IDs build tree topology Node degree: β child nodes

Tree topology of Domain IDs does not change over time!

• Only the peers responsible for the Domain IDs might change

Assignment of peers to domains dynamic

1 10

50

20 30

40

45 15

P2P Overlay

0 1 0.09 0.2 0.31 0,4 0.5 0.6 0.75 0.9

Internet

0.5

0.25

0.375

0,3125

0.75

0.875 0.625 0.125

Domain Domain ID

0.3125

0.375

0.25

0.5


SkyEye.KOM: Node Placement in Tree

Tree-overlay p2p overlay Reconvert to

• E.g. = /

Coordinator: • Responsible for Domain ID

• Check via DHT function – isMyKey(Key K)

Peers to Domain ID assignment Peer p calculates for each

level l the Domain in which its own ID is located

• (note: sometime adaptation needed)

For those domains, they calculate the Domain IDs (

If peer is responsible for : position defined

Example:

Node ID ∈ {0,… , 2160 − 1}

Tree ID = Node ID /

(2160 − 1)

Example: 0.63

• Responsibility range: 0.61-

0.64

Check for resp. ID (β = 2)

• Level 0: 0.5 no

• Level 1: 0.75 no

• Level 2: 0.625 yes

Position of peer: 0.625

Parent node: 0.75 (tree ID)

• Route messages to node

ID 0.75 * (2160)


Detailed Algorithm to Identify Position in the Tree


SkyEye.KOM: Communication

Example tree

Tree degree (β) = 2

• Results in logarithmic tree

size

Balanced, if ID space

balanced

Not always β children

• Peers may be

Coordinators at various

levels


SkyEye.KOM: Communication Protocol

Gathering global view All peers measure local

status

Periodically sent to parent peer

• Update Interval (UI)

Aggregation Direct: count, sum, minimum,

maximum, sum of squares

Derived: mean, variance, std. deviation

Dissemination of global view Global view in root

Every update message is acknowledged

Contains global view from level above

Global view

Local measures, (synchronized signal in simulations)

Aggregated

view

β child nodes

… 1a 1b

1β

1. Independent updates

in UI intervals per node

2b 2a

2β

2. ACKs with view of parent

peer for every update


Time and Overhead Estimation

Convergence time: Aggregation up the tree and pushing global view down in periodic update intervals (UI)

Tree height dominated by β

Overhead: Per node: 1+ β messages

Very small, independent of N

Typical size of β : 4, 8, 16

Convergence:

Gather and disseminate time 𝑂 logβ 𝑁 ∙ 𝑈𝐼


SkyEye.KOM – Variant: Synchronized Communication

Idea:

Loosely synchronize updates messages

Push results once global view is created

Approach:

Push with ACKs: synchronization delay

Peers adapt their update offset

Updates processed in a row, from leafs to root

Push global view with “special-ACK”

Gather and disseminate time:

𝑂(logβ 𝑁 + 𝑈𝐼)

Before: 𝑂 logβ 𝑁 ∙ 𝑈𝐼


SkyEye.KOM – Variant: Synchronized Communication

Protocol

Assumption:

• Transmission delay < 2 sec

Global view message from root

• Contains offset, when to send next update

– E.g. UI = 60 seconds

• Is decreased every step

– by time 𝑡∆

– E.g. by 3 seconds

Resulting transmission offset

• Current time in root 𝑡0

• Average transmission time 𝑡𝑠𝑒𝑛𝑑

• Level 𝑖 sends at time: 𝑡𝑖 = 𝑡0 + (𝑈𝐼 − 𝑖 ∙ 𝑡∆ + 𝑖 ∙𝑡𝑠𝑒𝑛𝑑)

Example: Root – sends out message

• Offset 60 sec

Level 1 • Transmission time: 1s

• Offset 57 sec


• Offset 54 sec


• Offset 51 sec

…

Level i • Transmission time: 1s

• Offset 𝑈𝐼 − 𝑖 ∙ 𝑡∆

Sending times Level i: 𝑈𝐼 − 𝑖 ∙ 𝑡∆ + 𝑖 ∙ 𝑡𝑠𝑒𝑛𝑑 …

Level 3: 54 sec

Level 2: 56 sec

Level 1: 58 sec

Root receives next global view in 59 sec


Evaluation of SkyEye.KOM

Simulation Setup

Node count 5000

Churn: Join, KAD churn

Tree degree = 4

Update interval = 60sec

Observation:

Time delayed, precise

monitor

Very low overhead

• <100 bytes / s

• Overhead is precisely

monitored


Some Remarks on SkyEye.KOM

Robustness against churn If peer fails: automatically replaced in the DHT

Updates are routed to new peer for aggregation

Costs One update: ~1kb,

Out + in degree = 1 + tree degree (2, 4 or 8)

Independent of position in the tree!

Age of information: Limited by tree depth, O(log (N))

Influenced by update period

Pro: Low overhead, good precision

Continuous monitoring

Cons: Churn on higher located nodes shows impact

Applicability limited to DHTs



– Comparison of various Approaches


► Benchmarking of Monitoring Solutions

Structured / tree-based solutions Limited in applicability key-based routing needed

Unstructured / gossip-based solutions Needs only connected graph not limited in application

Question: Why do not we use only gossip-based solutions?

Answer: They are inefficient

Imprecise OR costy

Comparative Evaluation Node count: 1000, churn

SkyEye.KOM: UI=15 sec, β = 8, synchronized (every 20min)

PushSum: 30 messages per epoch

Centralized for comparison, UI = 60 sec

Same overhead allowed for all monitoring approaches


Churn – Reference: Node Count

Simulation setup Churn with joining and

instantly leaving nodes

Both decentralized solutions

• Use ca. 200 bytes/s per node

• For better comparability

Aggregation time

Of global view

Performance of tree similar

to centralized solution

Gossip-approach slower

• Sawtooth: epochs


Reference Signals: Steps, Sawtooth and Sine

PushSum Imprecise monitoring

Epochs are visible

Although same traffic overhead

Centralized and tree-based Precise

Tree become imprecise with too much churn


Impact of Parameters

In SkyEye.KOM:

Increase in branchingfactor

Tree less deep

Shorter paths

More precise monitoring

In Gossip-based approach

Increase Rounds per Epoch

Better convergence of

nodes

Information becomes

outdated


Summary on P2P Monitoring Solutions

Several approaches exist

Gossip-based solutions

• Pro: applicable in any graph

• Cons: expensive and imprecise

Tree-based solutions

• Pro: precise and of low overhead

• Con: rely on DHT substrate (which also induces costs)

Centralized solutions

• Pro: quick, fast and cheap

• Cons: do not scale

Next Questions:

How to use monitored system status

To improve the quality of the system?

Peer-to-Peer Systems · Preferred implementation approach Hook into p2p overlay routing • Add columns for statistic information ... On Unbiased Sampling for Unstructured Peer-to-Peer

Documents