1 PEER TO PEER GROUP FORMATION AND COLLABORATION IN A REMOTE LABORATORY Program Committee : By : Prof. Kanchana Kanchanasut Prithula Dhungel Dr. Yasuo.

1

PEER TO PEER GROUP FORMATION PEER TO PEER GROUP FORMATION AND COLLABORATION IN A AND COLLABORATION IN A

REMOTE LABORATORYREMOTE LABORATORY

Program Committee : By :

Prof. Kanchana Kanchanasut Prithula DhungelDr. Yasuo Tsuchimoto Department of Computer Dr. Paul Janecek Science and Information Management

School of Engineering and TechnologyAsian Institute of Technology

20th April 2006, SOI Asia/AI3 joint Meeting 2006, Spring18-20 April 2006, Hua Hin, Thailand

2

• Background • Problem Definition• Objectives• Methodology

– Design– Implementation

• System Evaluation• Conclusion

Outline

3

Consider a scenario in a University

Many laboratories inside the university desiring to provide remote laboratory access to remote users

In each of the laboratories, experiment equipment interfaced to a computer

Thermodynamics Control System Microprocessor

Given Scenario

4

Remote StudentRemote Student

Different students interested in performing different laboratory experiments inside university (a single university)

Remote Student

Scenario ….

5

Remote Student

Remote Student

Remote Student

• For each laboratory, they want to perform experiments in group and collaboratively control remote equipment

Scenario ….

6

• Direct communication with each other and node providing device resources, without going through any central entity = > Peer to Peer environment

• Role based access right to control the experiment equipment (teacher, student)

• Teacher = > more privileges• Student => less privileges

Scenario ….

7

Remote Student

Remote Student

Remote Teacher

Student

Student

Teacher

Scenario …. Inside Each Group

8

• Provide remote collaborative access to all laboratories in the University

• Such a way that multiple laboratories in University are integrated into a single system

Requirements

9

Remote Laboratory – The Past [1]

Remote Student

No concept of group formation and collaboration

Figure : NUS Remote Laboratory System

Server

10

Remote Laboratory – The Past [2]

Figure : Web-based Environment for Collaborative Remote Experimentation (University of Hagen)

Remote Student

•Allows group formation and collaboration among users•Allows various roles to group members (teacher and student)

ServerRemote Equipment

Remote Teacher

Remote Student

11

• Different laboratories inside a single University not integrated into a single software system– Users desiring to connect to different systems one by

one should connect to each system separately (possibly using different user interfaces)

Problem

12

1) Form multiple simultaneous groups of people based on their interest (people with same interest brought together in same group)

2) Provide access right assignment based on roles (teacher, student ) to members in each group

3) Allow collaborative conduction of experiment in each experiment group controlling actual hardware equipment

4) Allow member exit from group without effecting the working of the system

Steps to Fulfill Requirements

13

• Step 3 requires design and implementation specific to hardware devices in each laboratory

• Outside scope according to time and device constraints

Steps to Fulfill Requirements

14

Hence .. The Objectives

1) Design architecture to form multiple simultaneous groups of nodes based on interests (integrate laboratories inside University to single system)

2) Design an architecture to provide role bases access right assignment to members in each group

3) Provide collaboration among members (text messages)

4) Allow members to exit from respective groups

5) Implement the designed system

6) Evaluate system performance using performance metric like delay, traffic flow, scalability, etc.

15

• Peer to Peer Technology in Application Layer• An overlay (logical ) network in Application layer• Each peer has knowledge of the identities of only

a certain number of peers in the network (neighbors)

Design : Group Formation

overlay edge(logical connection)

16

• Nodes wanting to participate in laboratory experiments enter laboratory network as peers: nodes hosting equipment and students (and teachers) wanting to perform experiment

• Peers enter into Peer to Peer Network and announce their interests to search for other peers that share the same interest with them, using the help of neighbors

Peer to Peer Network – What?

17

• Nodes arranged in a logical circle

• Concept of Nodes and Group Names

• Node : Node wanting to be part of one of the groups• Group Name : Name of the laboratory experiment group (Control System, Thermodynamics, Microprocessor)

• Node ----- > UNIQUE identifier (Hash IP, 128 –bit)• Group Name ------ > UNIQUE key (Hash Group Name, 128 – bit)

Group Formation– Pastry [3]

18

• A Group Name with key K is mapped to a certain node in nodeID space (node that has nodeID numerically closest to the value of key K)

• Suppose,

Key(Control System) = K

Node responsible for key K Node X

• Key K Node XMAPPED TO

Rendezvous Point for Key

K

Responsible for storing information about group named Control System (key K)

Group Formation ….

19

Group Formation – How ?

Rendezvous Point for the group Control System

Message, Key = K

Key(Control System) = K

Send message tagged with key

K

Node X, responsible for key

K

•Any message, sent by any node in network, tagged with key K will be routed to Node X (i.e. to node that has nodeID numerically closest to key K)

20

Control System RP

Microprocessor RP

Thermodynamics RP Register

Register

Register

Node Interested in Microprocessor

Node Interested in Thermodynamics

Node Interested in Control System

Group ListGroup List

Group List

Register

Group List

Group Formation – How ?

21

• Our Scenario:– One or more hardware devices connected to each

other and all interfaced to a single node (Resource Provider)

– Performing experiment : changing parameters of one or more devices, that changes the output of the device and entire experiment

Design : Role Based Access Right Assignment

22

• Each node plays one of the following roles :• Resource Provider : one per group (Analogous to

the file system for Unix)• Teacher: one per group (Analogous to the super

user for Unix)• Student: one or more per group (Analogous to the

normal user for Unix)

• Access Rights :• Control (Send control signals to a device)

– Changing the parameters of the device– 1 controller per device possible

• Observe (Observe the output of a device)– Numerous observers per device possible

Role Based Access Right Assignment ….

23

ABC

Resource Provider

Controller and Observer details for each device

Device List Request

XYZ

Access Right Request (Control, Observe, Revoke)

CHANGES in Controller and Observer details for the device

Check Request validity

CHANGES in Controller/Observer

details for each device

CHANGES in Controller/Observer

details for each device

Device Role User User

Signal Generator

Controller XYZ <NP>

Observer ---- ----

Function Generator

Controller ---- ----

Observer XYZ ----

Device Role User User

Signal Generator

Controller XYZ <NP>

Observer ---- ----

Function Generator

Controller ---- <NP>

Observer XYZ ----

Device Role User User

Signal Generator

Controller XYZ <NP>

Observer ---- ----

Function Generator

Controller ABC <NP>

Observer XYZ ABC

Access Right Assignment

24

• Each group formed is a mesh– Each member knows of all other existing members

of the group– Collaboration (text messages) is direct among

members without going through any central entity– Collaboration

• One to one• One to many

Design : Collaboration Among Members

25

ABC

Resource Provider

XYZ

Exit from Group

BYE

BYE

BYE

Inform existing group members

Design : Exit from Group

26Rendezvous Point for the group Control System

BYE Message, Key = K

Key(Control System) = K

Send BYE message tagged

with key K

Inform RP

Exit from Group ….

27

• FreePastry 1.4.2 API [4] using Java in Application Layer

• Xcast6 API [5] for using Xcast in the Network Layer (using C) (Bandwidth saving)

Application

Transport

Network

Data Link

Physical

Free Pastry 1.4.2

XCAST6

• Used JNI (Java Native Interface) to call C functions from Java• FreeBSD platform

Implementation

28

• Traffic Flow• Compare performance of our system with other

system using Unicast • Stress on RP, stress on exiting node, stress on

Resource Peer• Total traffic flow in network for Group Formation,

Group Leave, Access Right Update

• Scalability • Ability of system to perform well in presence of large

number of nodes• Maximum members in each group, maximum

simultaneous groups

System Evaluation : Criteria

29

• Traffic Flow– Analytically for 3 Topology Scenarios

System Evaluation - Scenario

30

Topology 1 Topology 2

Topology 3

Scenario : Traffic Comparison

31

Variation of Stress on RP with respect to Group Size

0500

1000150020002500300035004000

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Group Size

Str

ess

(byt

es)

Xcast

Multi Unicast

Stress – Traffic flow in RP when new node arrives

Explanation - Sender has to send 1 packet instead of n individual packets to n receivers

Result : Traffic (Stress)

32

Group Formation Traffic Flow in Network for Various Topologies and Group Sizes

0

1000

2000

3000

4000

5000

6000

7000

2 3 4 5

Group Size

Tra

ffic

Flo

w (

byt

es)

Topology 1 (Xcast)

Topology 1 (Unicast)

Topology 2 (Xcast)

Topoogy 2 (Unicast)

Topology 3 (Xcast)

Topology 3 (Unicast)

Group Formation Traffic – Traffic that flows in whole network when a new node arrives to a group

Explanation - Traffic gain in sender side is overwhelmed by the price receivers have to pay in terms of increased header size of XCAST

- Data being transmitted is less in size

Result : Traffic (Network)

33

Traffic Flow for Various Topologies and Group Sizes (1024 bytes of data)

0

5000

10000

15000

20000

2 3 4 5

Group Size

Tra

ffic

Flo

w (

byt

es)

Topology 1 (Xcast)

Topology 1 (Unicast)

Topology 2 (Xcast)

Topology 2 (Unicast)

Topology 3 (Xcast)

Topology 3 (Unicast)

Explanation - Traffic gain in sender side is more in comparison to price receivers have to pay in terms of increased header size of XCAST

- Data being transmitted is large enough in size

Result : Traffic (Network)

34

Limit in the Group Size

1250

1300

1350

1400

1450

1500

1550

1600

66 67 68 69 70 71 72 73 74 75 76

Group Size

New

Mem

ber

Not

ific

atio

n

Pac

ket

Siz

e (b

ytes

)

Explanation - XCAST6 does not support packet fragmentation - XCAST6 packet size limited to MTU of 1500 bytes

• Maximum Number of Members in a Group

Result : Scalability

Figure : Limit in the Group Size due to New Member Notification Message

35

Result : Scalability

• Maximum number of simultaneous groups– Depends on the scalability of Pastry ring in terms of

maximum number of groups– Balanced Load property [6] of uniform hashing technique

used in Pastry ensures even distribution of keys among all nodes

– No one node is overloaded by having to be RP for unreasonably high number of group names

– Each member can register for one group at a time– If n members present in ring, maximum possible number of

registrations for separate groups in the ring is n => n simultaneous groups can exist

– Theoretically maximum 2 128 nodes possible in ring– Theoretically, maximum 2 128 simultaneous groups possible

36

• Addressed the problem of integrating numerous laboratories into single system

• Designed and implemented P2P based architecture to form multiple simultaneous groups (theoretically 2 128 groups possible); role based access right assignment inside each group

• Our design is efficient : failure of any one node does not affect the process of group formation and collaboration in entire network.

• Use of XCAST6 technology decreases the stress on sender side but increases the traffic flow in whole network

• System proposed will be able efficient in terms of traffic flow when later phases of remote laboratory will be implemented

• Results obtained in terms of maximum number of members and devices in each group are well above the requirement for practical collaborative remote laboratory groups

Conclusion

37

Future Work

• Provide user authentication and security • Implement the design proposed to provide

robustness in presence of silent departure of nodes from the system

• Design and implement each laboratory specific system in the Resource Peer for interfacing and controlling respective experiment equipment

• Provide key word based searching for groups such that it obviates the need for any member to know the exact group name

38

1) Ko, C.C. et al. (2001). A Webcast Virtual Laboratory on a Frequency Modulation Experiment. IEEE Conference on Decision and Control, Orlando, FL.

2) Röhrig, C., & Bischoff, A. (2003). Web-based Environment for Collaborative Remote Experimentation. Proceedings of the 42nd IEEE Conference on Decision and Control Maui, Hawaii USA.

3) Rowstron, A., & Druschel, P. (2001). Pastry: Scalable, Decentralized Object Location and Routing for Large-Scale Peer-to-Peer Systems. In the proceedings of the 18th IFIP/ACM International Conference on Distributed Systems Platforms, Heidelberg, Germany.

4) FreePastry (2005). The FreePastry homepage. Retrieved November 2005, from:http://freepastry.org/FreePastry/

5) XCAST over IPv6. Retrieved November 2005, from the SourceForge website:http://sourceforge.net/projects/xcast6/

6) Karger, D. et al. (1997). Consistent Hashing and Random Trees: Distributed Caching Protocols for Relieving Hot Spots on the World Wide Web. STOC 97, EI Paso, Texas, USA.

        References

39

THANK YOU

(QUESTIONS ?)

40

Pastry

• Based on the concept of Distributed Hash Tables (DHTs)

• Hash Table– Given a key, finds the location in the table

where the key belongs

• DHT– The tables are distributed– Given a key, finds the node where the key

belongs

41

DHT for P2P Systems

• Organize DHTs as nodes in an overlay

• Every node in DHT knows about few other nodes in DHT

• Every node has a unique ID

• When node receives query for key K:– Forwards the query to neighbor whose ID is

closest to K

42

Circular DHT

• Each node has two neighbors: successor and predecessor

• Information of a key is stored in closest successor

43

Circular DHT

0001

0011

0100

0101

10001010

1100

1111

information relatedto key 1110 stored here

Who’s resp For key 1110

I am

O(N) messageson avg to resolvequery

44

Pastry

• Each node is assigned a 128-bit nodeID

• The nodeID is viewed in base 16 – e.g., 65a1fc04

• nodeID indicates node’s position in a circular nodeID space

• Each node knows its predecessor and successor nodes as well as few other nodes

• Node forwards message to node whose nodeID shares with K a prefix that is at least one digit longer than that the key shares with the present node’s nodeID

45

Pastry ….

• Each node maintains a leaf set and a routing table

• LeafSet– Contains nodeIDs and IP Addresses of L

closest nodes (closest in terms of nodeId value)

• Routing Table– ith entry of table contains nodeIDs and IP

Addresses of nodes sharing i prefixes with the nodeID of the node (O(log N))

• Given a message tagged with key K, Pastry scheme routes the message to node that has a nodeID closest to K in value (O(log N) steps

46

Pastry Routing Table

0x

1x

2x

3x

4x

5x

7x

8x

9x

ax

bx

cx

dx

ex

fx

60x

61x

62x

63x

64x

66x

67x

68x

69x

6ax

6bx

6cx

6dx

6ex

6fx

650x

651x

652x

653x

654x

655x

656x

657x

658x

659x

65bx

65cx

65dx

65ex

65fx

65a0x

65a2x

65a3x

65a4x

65a5x

65a6x

65a7x

65a8x

65a9x

65aax

65abx

65acx

65adx

65aex

65afxlog16 N

rows

Row 0

Row 1

Row 2

Row 3

47

Pastry Routing Procedure

if (destination is within range of our leaf set) forward to numerically closest member inleaf set

elseif (there’s a longer prefix match in routing table)

forward to node with longest match

else forward to node in table (a) shares at least as long a prefix(b) is numerically closer than this node

48

Pastry ….

• New node join– Suppose new node with ID X joins the network– Should know the IP Address of at least one node

already in the ring => BootStrap node– New node sends join message to the ring (via the

BootStrap node) with a key X– Join message will be routed to node Z (say) that is

currently responsible for key X– LeafSet of Z is the LeafSet for X– Routing Table for X:

• All nodes encountered by join message on the way to node Z, send one row of their routing tables to X

• ith node encountered sends ith row

49

Multi unicast

A

B

C

D

R1 R2 R3

R4

R5 R6 R7

R8

R9

Figure : Multi unicast

n packets for n receivers

Multi unicast wastes bandwidth since a packet per receiver is to be generated and transmitted (for n receivers, n packets from the source)

50

Multicast

A

B

C

D

R1 R2 R3

R4

R5 R6 R7

R8

R9Figure : Multicast

1 packet

Only 1 packet coming out from the sender, but

Overheads :

•Multicast routing entry per group in all intermediate routers

•Multicast routing protocols required

51

Xcast

A

B

C

D

R1 R2 R3

R4

R5 R6 R7

R8

R9Xcast header:

< src = A >

< dest = B C D >Xcast header:

< src = A >

< dest = C D >

IP header:

< src = A >

< dest = D >

Address partitioning

Address partitioning

Figure : Working of an Xcast network

1 packet

• Only 1 packet coming out of the source

• No state information required in the intermediate routers

•No additional routing protocols required

52

IPv6 Hop By Hop IPv6 Routing Extension Destination Option UDP Data

40 8 40 32+ 16 * n F(n) 8 74

Table : New Member Arrival Information Message Size (XCAST6)

destinationHeaderLength = 8for n = 3 to Number of Destinations do

destOpt = 1count = 1while count <= 4 do

variablePart = destOpt * 8destinationHeaderLength = destinationHeaderLength + variabl

ePartdestOpt = 0count ++

Figure : Calculation of Destination Option Header Size

53

IPv6 Header (bytes) UDP Header (bytes) Data (bytes)

40 8 74

Table : New Member Arrival Information Message Size (Unicast)

IPv6 Header (bytes) UDP Header (bytes) Data (bytes)

40 8 8 + 65*n+1

Table : Existing Members Information Message Size

Hence, Total packet size = 122 bytesTotal bytes transferred to n receivers = 122 * n bytes

54

No.of Destinations BYE Message Packet Size

1 98

2 218

3 242

4 258

5 274

6 290

Table : BYE Message Size (XCAST6)

55

Stress in RP = Register message traffic from new node to RP + New Member Arrival message(s) traffic from RP to existing group members + Existing Group Members message traffic from RP to new node

Group Leave Traffic = BYE message traffic from exiting node to remaining members of group + BYE message traffic from exiting node to RP