PSIRP Architectural Components Part 1

PSIRP Architectural PSIRP Architectural Components Part 1Components Part 1

Walter WongNomadicLab & HIIT

08.02.2010

OutlineOutlineIdentifiersAlgorithmic IDsNode Internal ArchitectureHelper FunctionsRendezvous System

Background – IP-based Background – IP-based IdentifierIdentifierNetwork Layer Identifier

◦IP address◦Topological identifier◦Refers to a specific location in the

networkTransport Layer Identifier

◦Also IP address (IPsrc/dst, portsrc/dst, protocol)

◦Identifies end-hosts in the Internet

IP Semantic Overload IP Semantic Overload ProblemProblem

Application

Transport

Network

Link

Physical

IP

Socket

200.201.11/24

174.180.23/24

128.17.11.22

IP-based Identification – IP-based Identification – ProblemsProblemsEntangle host location with

identification◦Host-centric approach

Content can’t be addressed alone◦Sub-part of a URL◦Ex.

www.unicamp.br/main/courses/index.html◦ Identified as part of connection parameters◦Ex. TCP sequence number

Limited naming system◦Hinders new features, ex., mobility,

multicast◦Users can’t express their interest in data

regardless of location

PSIRP Identifiers – PSIRP Identifiers – MotivationMotivationExpress user interests

◦WWW users are interested in documents

regardless of their location

◦Currently users express an interest (what) and it is translated to a place (where)

Publish/subscribe◦Users express their interest on data◦Data is asynchronously delivered to

users

PSIRP IdentifiersPSIRP IdentifiersApplication Level Identifier (AId)Rendezvous Identifier (RId)Scope Identifier (SId)Forwarding Identifier (FId)Algorithmic Identifier (AlgId)

Application Level Identifier Application Level Identifier (AId)(AId)Application level identificationBased application requirements

◦Easiness to route, easiness to resolve, randomness, etc

Can be any namespace◦Flat labels◦Structured names (FQDN)◦Global Unique Identifier (GUID)◦Electronic Numbering (ENUM)

Rendezvous Identifier Rendezvous Identifier (RId)(RId)PSIRP network level identifierUniquely identifies a piece of

information256-bit identifier (SHA-256 hash

over the data)Goal: identify the interest

between publishers and subscribers in the rendezvous system

Rendezvous Identifier Rendezvous Identifier (Rid)(Rid)

Publish RIdnut

Subscribe RIdnut

Rendezvous Point

Interest!Doesn’t subscribe RIdnut

Deliver data

Bingo

Scrat

Nut

Scope Identifier (Sid)Scope Identifier (Sid)Also PSIRP network identifier with

256 bits lengthGoal: aggregate a set of Rids into

one group (scope)Represents a collection of whatever

information that Rids representsScope = implicit context of a set of

Rids◦Photo album has a Sid and each photo

has a RidAlso

◦Access control◦Metadata control operations

Scope Identifier (Sid)Scope Identifier (Sid)

RidC

RidB

RidA

Sidfamily_album Sidpublic_album

Forwarding IDForwarding ID256-bit long network identifierGoal: identify path segments in

the forwarding pathDifference:

◦Sid/Rid used in the rendezvous system (slow path)

◦Fid used in the forwarding fabric (fast path)

Can be aggregated in Bloom filters

Forwarding IDForwarding ID

Rendezvous 2

FId21 FId22

FIdA

Rendezvous 1

FId11

FId12

FId13

FId11FId21

FId12

Bloom-filter

Bob Clark

David

FId31

FId22

Forwarding IDForwarding IDSource controlled packet soft state

◦Topology manager creates the Bloom filter

◦+ stateless◦- requires constant updates from the

rendezvous to identify new subscribers◦- recycle Fids after un-subscription

In-network state◦Fid switching state in the forwarding

nodes◦+ Identifies just initial Fid◦- state in the switches

Algorithmic Identifiers Algorithmic Identifiers (AIgIds)(AIgIds)Special class of identifiersInformation collection concept

◦group semantically similar information items

Goal: automatic generation of IDs for different pieces of information

Allow elaborate tests with the identifier

Algorithmic IdentifiersAlgorithmic IdentifiersAllows relationship tests on each

ID:◦Ordering: precedence (does an

identifier precede another one?)◦Composition: does an ID belong to

another ID (e.g. a chunk ID belongs to a file ID)

◦Completeness: do we have all the identifiers composing another information object?

Algorithmic IDsAlgorithmic IDsApplications

◦Automatic deriving identifiers for information fragments

◦Calculating the next identifier for a sequence

◦Deducing the identifier

Subscription ManagementSubscription Management

Subscribecomics

Comics part 1

Comics Part 2

Comics Part 3

Comics Part 6

Comics Part 4

Comics Part 5

Billy

Forwarding State Forwarding State AggregationAggregation

FId11

FId12

FId13

FId21

FId31

FId22

FId32

FIdA

FIdB

FIdC

FId11

FIdA

FId13

FId31 FId32

FIdC

Bloom-filter

RVS

RVS

RVS

Bob Clark

David

Re-encoding the same Re-encoding the same publicationpublication

Publisher

ID1 ID2

IDA IDB IDC

AlgIDhigh_resolution

AlgIDlow_resolution

Publication

Return Path Return Path (legacy client-server model)(legacy client-server model)

Bob Clark

Publish(algIdA)

Subscribe(hash(algIdA))

Subscribe(algIdA)

Publish(hash(algIdA))

RVS

Hey Clark, let’s have a beer after work!Okee dokee, Bob!

Sequence NumberingSequence Numbering

RVS

Publish(algId) Subscribe(algId)

ID2 = hash(ID1)ID3 = hash(ID2)

ID1 = hash(algId)

Alice Beth

Algorithmic IDs – Other Algorithmic IDs – Other scenariosscenariosError control

◦generate error messages identified with IDs derived from the error ID

Reliability◦different algIds can be generated to

receive error messagesAnnouncements

◦prior to a data publication, publishers announce in the algId channel, informing possible subscribers that data will be published in the related algId

Algorithmic ID – General Algorithmic ID – General designdesignDirect Acyclic GraphIdentifiers are related to each other by

functionsSorting algorithms

◦Ordering the identifiersTrees are generated by consecutive

application of one-way functionsSequences can be expressed as the

recursive application of a singular function

Reverse function to walk in both directions

Algorithmic IDs – ExampleAlgorithmic IDs – Example

B = f(A,1)

C = f(A,2)

D = f(C,1)

C = f-1(F,3)A = f-1(C,2)

Node Internal Architecture Node Internal Architecture Blackboard Approach

◦Communication model for sharing memory objects through the network

Uses Memory Object Service◦Simple unreliable page service◦Simple memory object service

Unreliable Page ServiceUnreliable Page ServiceSimple unreliable

page service◦ Pages can be

mapped to packet level transmissions using PageIDs

◦ Allows for identifying and caching individual packets within the network

◦ There is one PageID for each Fid

Memory Pages

Memory Object ServiceMemory Object ServiceWorks on the page

serviceSupports larger

data files (over one memory page)

Supports update on memory objects

Memory PagesMemory Object

Memory Object Creation & Memory Object Creation & PublicationPublication

1. Publisher requests a meta-publication and a memory object2. Memory service returns a memory object with the meta attached

3. Publisher fills the data in the memory object

Blackboard

4. Publisher creates a Rid for the memory object. It can be republished as many times as he needs with equal or different tags

Publisher

Memory Object Memory Object SubscriptionSubscriptionSubscriber subscribers to the MO-RidSubscriber receives an event stream

◦File descriptor which can be read with select

Whenever the publisher publishes some data, the subscriber will be notified and will receive a new memory object representing the corresponding version of the object

Node Internals – Node Internals – NetworkingNetworkingNodes

◦Local input queues, with one Fid for each queue

◦Local output queues, with one Fid for each queue

◦Sender-helper sends the packet identified with the interfaces Fid

◦Network-receiver receives packets from the network

Node Internal ArchitectureNode Internal Architecture

Node Internal - Networking

Publish call

Node Internal ArchitectureNode Internal Architecture

Node Internal - Networking

Subscribe call

MemoryObject

Memory Object Memory Object SegmentationSegmentationPacketiser helper

◦handles memory objects larger than a packet size

◦Partitions objects into a set of packetsAt each local output places, whenever

there is a new object, the packetiser chunks the data and places each of them in the correct output queue scope

There is no memory copy, just a new mapping of the object in the scope directory

Memory Object Memory Object SegmentationSegmentation

Local Blackboard

Helper FunctionsHelper FunctionsNot core network serviceNot traditional applications

◦Actually, it is a set of functions providing additional features for the system

◦“Helps the system!”PSIRP has three categories:

◦Network Management Functions◦Remote Service Functions◦Host Centric Functions

Network Management Network Management FunctionsFunctionsNetwork Management &

Information gatheringInformation gathering for better

planning and management◦Increase performance

In PSIRP, network management functions can subscribe to the link state, receiving information as it becomes available

Network Management Network Management Functions – ExampleFunctions – ExampleCurrent Internet

◦SNMP/ICMP◦GET/SET/TRAP directives◦Managed systems generate

asynchronous messages to notify new events

Network Management Network Management Functions – Unified Link Functions – Unified Link Layer API (ULLA)Layer API (ULLA)Closely resembles publish/subscribeIntroduces an interface to collect

information from the link layerLink aware applications express

their interests by specifying link conditions

Notifications can be asynchronous or periodic

ULLA can be the base for the helper function which subscribes to the link conditions and wait for updates

Remote Service FunctionsRemote Service FunctionsSegmentation function

◦Helps in the segmentation decision◦Extracts data from link layer, such as

MTU, and provides parameters for the fragmentation

Merger function◦Merges partitioned data to get the

original publication

Remote Service FunctionsRemote Service FunctionsForwarding Error

Correction/Content protection◦Error correction in the transmission◦Recover received data without

retransmissions◦Send redundant data together◦Ex. Reed Solomon Block Code (N, K)◦N-K symbol redundancy◦After receiving any K symbols out of

N, the receiver is able to decode the original data

FEC ParametersFEC Parameters◦Currently, the FEC parameters are

statically set, degrading the performance

◦Inaccuracy in the estimation of the underlying channel state, resulting in inefficient bandwidth usage

◦FEC helper can subscribe to the channel conditions in order to generate better parameters

Remote Service FunctionsRemote Service FunctionsContent re-encoding

◦Scenario: heterogeneous subscribers requesting the same video but with different resolution

◦Content provider can’t attend the demand

◦Content provider asks for remote helper functions located closer to the subscribers to perform the translation

Application-specific Application-specific CachesCachesContent caching

◦Offers the same interface as the original content provider (transparent)

◦Applications can select from which cache to retrieve data

◦content providers may instruct caches to serve content on their behalf

◦subscribers can receive data from the closest caches, saving network resources

Host Service FunctionsHost Service FunctionsTransport and other middle network

servicesPSIRP offers shared blackboard for

distributed applications using the network

Analysis and debugging purposes◦Ping, traceroute, tcpdump, wireshark

PSIRP context◦Transport component for rate control and

reliability algorithm for point-to-point delivery

Rendezvous SystemRendezvous SystemMeeting point of interests

between publishers and subscribers

Requirements◦Scalability (Internet scale)◦Efficiency in operation◦Deployability (gradual deployment

and stakeholders incentives)

Rendezvous NetworkRendezvous NetworkComposed of Rendezvous Nodes

(RNs)Organized as a BGP-like inter-domain

hierarchyCollection of RNs from multiple

cooperative ASesRendezvous points (RPs)

◦logical meeting points of interests between publishers and subscribers

◦one rendezvous point for each Sid/Rid pair

Rendezvous Network Rendezvous Network InterconnectionInterconnectionSome approaches

◦Central entity managing signaling reachability: lack of incentives, trust, competition

◦Multiple entities: providers compete for global reachability coverage

◦Fully distributed mechanism: rendezvous networks self-organize to interconnect without any third party infrastructure

Rendezvous Network Rendezvous Network ConnectionConnectionCanon DHT

◦Hierarchical DHT◦Maintains local communication

within its domain◦Caching in the sub-domain exit

between levels in the DHT

Canon Hierarchical DHTCanon Hierarchical DHT

Domain A Domain B

Top-level Domain

Rendezvous System Rendezvous System ExampleExample

Rendezvous System Rendezvous System BootstrappingBootstrapping

Publish OperationPublish Operation

Subscribe OperationSubscribe Operation

SummarizingSummarizingIdentifiers

◦ Many types, special semanticsAlgorithmic ID

◦ New use casesNode Internal Architecture

◦ Blackboard, Input/output queues, memory objects segmentation

Helper functions◦ Adding new functionalities in the

architectureRendezvous System

◦ Matching interests between publishers and subscribers

Questions?Questions?Comments?Thanks!