Chapter 2 Communication

7/28/2019 Chapter 2 Communication

1/59

Chapter 2 - Communication


2/59

2

Introduction

interprocess communication is at the heart of all distributed

systems communication in distributed systems is based on message

passing as offered by the underlying network as opposed to

using shared memory

modern distributed systems consist of thousands of

processes scattered across an unreliable network such as

the Internet

unless the primitive communication facilities of the network

are replaced by more advanced ones, development of large

scale Distributed Systems becomes extremely difficult


3/59

3

2.1 Layered Protocols

two computers, possibly from different manufacturers, must

be able to talk to each other

for such a communication, there has to be a standard

The ISO OSI (Open Systems Interconnection) Reference

Model is one of such standards - 7 layers

TCP/IP protocol suite is the other; has 4 or 5 layers

OSI Open to connect open systems or systems that are open

for communication with other open systems using standard

rules that govern the format, contents, and meaning of the

messages sent and received

these rules are called protocols

two types of transport layer protocols: connection-oriented

and connectionless

a conversation occurs between a sender and a receiver at

each layer


4/59

4

Middleware Protocols

a middleware is an application that contains general-purpose

protocols to provide services

example of middleware services authentication and authorization services

distributed transactions (commit protocols; locking

mechanisms)

middleware communication protocols (calling a procedure

or invoking an object remotely, synchronizing streams for

real-time data, multicast services) - see later in this Chapter

hence an adapted reference model for networked

communications is required


5/59

5

an adapted reference model for netwo rked communic at ion


6/59

6

2.2 Remote Procedure Call

the first distributed systems were based on explicit message

exchange between processes through the use of explicit

send and receive procedures; but do not allow accesstransparency

in 1984, Birrel and Nelson introduced a different way of

handling communication: RPC

it allows a program to call a procedure located on another

machine

simple and elegant, but there are implementation problems

the calling and called procedures run in different address

spaces

parameters and results have to be exchanged; what if themachines are not identical?

what happens if both machines crash?


7/597

pr inc iple of RPC between a cl ient and server program

Client and Server Stubs

RPC would like to make a remote procedure call look the

same as a local one; it should be transparent, i.e., the calling

procedure should not know that the called procedure is

executing on a different machine or vice versa

when a program is compiled, it uses different versions of

library functions called client stubs

a serverstub is the server-side equivalent of a clientstub


8/598

Steps of a Remote Procedure Call

1. Client procedure calls client stub in the normal way

2. Client stub builds a message and calls the local OS

(packing parameters into a message is called parametermarshaling)

3. Client's OS sends the message to the remote OS

4. Remote OS gives the message to the server stub

5. Server stub unpacks the parameters and calls the server

6. Server does the work and returns the result to the stub

7. Server stub packs it in a message and calls the local OS

8. Server's OS sends the message to the client's OS

9. Client's OS gives the message to the client stub

10. Stub unpacks the result and returns to client

hence, for the client remote services are accessed by making

ordinary (local) procedure calls; not by calling send and

receive

server machine vs server proc ess; cl ient machine vs cl ient process


9/599

steps involved in doing remote computation through RPC

Parameter Passing

1. Passing Value Parameters

e.g., consider a remote procedure add(i, j), where i andj areinteger parameters


10/5910

the above discussion applies if the server and the client

machines are identical

but that is not the case in large distributed systems, there can

be differences in:

data representation (e.g., IBM mainframes use EBCDIC

whereas IBM PCs use ASCII)

representing integers(1s complement or 2s complement)and floating-point numbers

byte numbering (from right to left in Pentium called little

endian and left to right in SPARC, big endian)

e.g. consider a procedure with two parameters, an integer and a

four-character string; each one 32-bit word (5, JILL)

the sender is Intel and the receiver is SPARC


11/5911

or iginal message on the Pentium

(the numbers in boxes indicate the add ress of each b yte)

the message after receipt on the SPARC; w rong integer (224+226=

83886080), but co rrect str in g


12/5912

the message after being inverted (co rrect integer but w rong str in g)

one approach is to invert the bytes of each word afterreceipt

additional information is required to tell which is an

integer and which is a string


13/5913

2. Passing Reference Parameters assume the parameter is a pointer to an array

copy the array into the message and send it to the server

the server stub can then call the server with a pointer to this

array the server then makes any changes to the array and sends it

back to the client stub which copies it to the client

this is in effect call-by-copy/restore

optimization of the method

one of the copy operations can be eliminated if the stubknows whether the parameter is input or output to theserver

if it is an input to the server (e.g., in a call to wri te), it need

not be copied back if it is an output, it need not be sent over in the first place;

only send the size

the above procedure can handle pointers to simple arraysand structures, but difficult to generalize it to an arbitrary

data structure


14/59

14

Parameter Specification and Stub Generation

the caller and the callee need to use the same protocol

(format of messages) and the same steps; with such rules the

client and server stubs can assemble, communicate, and

interpret messages correctly other issues that need the agreement of the client and the

server

how are simple data structures like integers (e.g. 2s

complement), characters (e.g. 16-bit Unicode), Booleans, ...represented?

endianess

which transport protocol to use - the connection-oriented

TCP or the unreliable connectionless UDP


15/59

15

a dooris a generic name for a procedure in the address

space of a server process that can be called by a processcolocated with the server

support from the local OS is required

Extended RPC Models

to solve some of the shortcomings of the original model

no need of network communication if server and client are

on the same machine no need of blocking for the client in some cases

a. Doors

the original RPC model assumes that the caller and thecallee can communicate only by means of passing

messages over a network; what if they are colocated on

the same machine?


16/59

16

1

2

3

the pr incip le of u sing d oors as IPC mechanism

1. the server process registers a door before it can be called

(door_create) and a name is attached to it

2. a client calls a door by a system call (door_call) including

all parameters

3. results are returned by the system call door_return


17/59

17

benefit: they allow the use of a single mechanism (procedure

calls) for communication

disadv: application developers have to be aware of where a

procedure is located; is it

local within the current process

local to a different process on the same machine

a remote process


18/59

18

b. Asynchronous RPC

if there is no need to block the client until it gets a reply

two cases

1. if there is no result to be returned e.g., adding entries in a database, ...

the server immediately sends an ack promising that itwill carryout the request

the client can now proceed without blocking

a)the interconnect ion between cl ient and server in a tradi t ional RPC

b)the interact ion us ing asynch ronou s RPC


19/59

19

2. if the result can be collected later

e.g., prefetching network addresses of a set of hosts, ...

the server immediately sends an ack promising that it

will carryout the request the client can now proceed without blocking

the server later sends the result

a cl ient and server interact ing th rough two asyn chronous RPCs


20/59

20

the above method combines two asynchronous RPCs

and is sometimes called deferred synchronous RPC

variants of asynchronous RPC

let the client continue without waiting even for an ack,called one-way RPC

problem: if reliability of communication is not guaranteed


21/59

21

DCE (Distributed Computing Environment) RPC

a middleware and an example RPC system developed by

OSF (Open Software Foundation), now The Open Group

it is designed to execute as a layer of abstraction between

existing OSs and distributed applications

the Open Group sells the source code and vendors integrate

it into their systems

it uses the client-server programming model and

communication is by means of RPCs services

distributed file service: a worldwide file system that

provides a transparent way of accessing files

directoryservice: to keep track of the location of allresources in the system (machines, printers, data,

servers, ...); a process can ask for a resource without

knowing its location

securityservice: for protecting resources; access is only

through authorization


22/59

22

distributedtimeservice: to maintain clocks on different

machines synchronized

Steps in writing a Client and a Server in DCE RPC

the system consists of languages, libraries, daemons,

utility programs, ... for writing clients and servers

IDL (Interface Definition Language) is the interface

language - the glue that holds everything together

it contains type definitions, constant declarations

and what the procedures do (only their syntax)


23/59

23

Uuidgen generates a prototype IDL file with a globally uniqueinterfaceidentifier

the IDL file is edited (filling the names of procedures and parameters) andthe IDL compileris called to generate 3 files

the application writer writes the client and server codes and are thencom iled and linked to ether with the stubs

Edit file

Bi di Cli t t S i DCE RPC


24/59

24

Binding a Client to a Server in DCE RPC for a client to call a server, the server must be registered (1

& 2)

the registration allows the client to locate the server and

bind to it the DCE daemon maintains a table (server, endpoint) and the

protocols the server uses

the directory server maintains the locations of all resourcesin the system (machines, servers, data,, ...)


25/59

25

two steps for server location

locate the servers machine (3)

locate the server process on that machine (which has

what is called an endpoint orport) (4)


26/59

26

resulted from object-based technology that has proven its

value in developing nondistributed applications

it is an expansion of the RPC mechanisms

it enhances distribution transparency as a consequence of

an object that hides its internal from the outside world by

means of a well-defined interface

Distributed Objects

an object encapsulates data, called the state, and theoperations on those data, called methods

methods are made available through interfaces

the state of an object can be manipulated only by invoking

methods this allows an interface to be placed on one machine while

the object itself resides on another machine; such an

organization is referred to as a distributed object

the state of an object is not distributed, only the interfaces

are; such objects are also referred to as remoteobjects

2.3 Remote Object (Method) Invocation (RMI)


27/59

27

the implementation of an objects interface is called a proxy

(analogous to a client stub in RPC systems)

it is loaded into the clients address space when a client

binds to a distributed object tasks: a proxy marshals method invocation into messages

and unmarshals reply messages to return the result of the

method invocation to the client

a server stub, called a skeleton, unmarshals messages andmarshals replies


28/59

28

common organization of a remo te object w i th cl ient-s ide proxy

Binding a Client to an Object


29/59

29

Binding a Client to an Object

a process must first bind to an object before invoking its

methods, which results in a proxy being placed in the

processs address space

binding can be implicit (directly invoke methods usingonly a reference to an object) orexplicit (by calling a

special function)

an object reference could contain

network address of the machine where the objectresides

endpoint of the server

an identification of which object

the protocol used

...


30/59

30

a)an example withimpl ic i t b indingus ing on ly global references

b)an example withexpl ic i t b indingus ing g lobal and local references

Distr_object* obj_ref; // Declare a systemwide object reference

obj_ref = ; // Initialize the reference to a distributed object

obj_refdo_something(); // Implicitly bind and invoke a method(a)

Distr_object obj_ref; // Declare a systemwide object reference

Local_object* obj_ptr; // Declare a pointer to local objects

obj_ref = ; // Initialize the reference to a distributed object

obj_ptr = bind(obj_ref); // Explicitly bind and obtain a pointer to the local proxy

obj_ptrdo_something(); // Invoke a method on the local proxy(b)

Parameter Passing


31/59

31

Parameter Passing

there are two situations when invoking a method with

object reference as a parameter: the object can be local or

remote to the client

local object: a copy of the object is passed; this means theobject is passed by value

remote object: copy and pass the reference of the object

as a value parameter; this means the object is passed by

reference


32/59

32

the si tuat ion when passing an object by reference or by value

two examples:

DCE Remote Objects

Java RMI


33/59

33

RPCs and RMIs are not adequate for all distributed system

applications

the provision of access transparency may be good but

they have semantics that is not adequate for all

applications

example problems

they assume that the receiving side is not running at thetime of communication

a client is blocked until its request has been processed

2.4 Message Oriented Communication


34/59

34

communication can be

persistent or transient

asynchronous or synchronous

persistent: a message that has been submitted fortransmission is stored by the communication system as longas it takes to deliver it to the receiver

e.g., email delivery, snail mail delivery

persistent communicat ion of let ters back in the days

of the Pony Express


35/59

35

transient: a message that has been submitted fortransmission is stored by the communication system only aslong as the sending and receiving applications are executing

Persistent Transient

Asynchronous

Synchronous message-oriented; three forms

asynchronous: a sender continues immediately after it has

submitted its message for transmission synchronous: the sender is blocked until its message is

stored in a local buffer at the receiving host or delivered to thereceiver

the different types of communication can be combined

persistent asynchronous: e.g., email

transient asynchronous: e.g., UDP, asynchronous RPC

in general there are six possibilities


36/59

36

persistent asynchron ous

communica t ion

persistent synchronous

communica t ion


37/59

37

receipt-based trans ient

synchronous communica tion

transient async hronous

communica t ion

weakest form; the sender isblocked until the message isstored in a local buffer at thereceiving host


38/59

38

response-based transient

synchronous communica t ion

delivery-based trans ient

synchronous commun icat ion

at message delivery

the sender is blocked until themessage is delivered to thereceiver for further processing

strongest form; the sender isblocked until it receives a replymessage from the receiver

M O i t d T i t C i ti


39/59

39

Message-Oriented Transient Communication

many applications are built on top of the simple message-

oriented model offered by the transport layer

standardizing the interface of the transport layer by

providing a set of primitives allows programmers to use

messaging protocols

they also allow porting applications

Examples of message-oriented transient communicationsystems: Berkley sockets and the Message-Passing

Interface (MPI)

M O i t d P i t t C i ti


40/59

40

Message-Oriented Persistent Communication

there are message-oriented middleware services, called

message-queuingsystems orMessage-OrientedMiddleware

(MOM)

they support persistent asynchronous communication

they have intermediate-term storage capacity for messages,

without requiring the sender or the receiver to be active

during message transmission

message transfer may take minutes instead of seconds ormilliseconds

Message-oriented persistent communication depends on

the message queuing model in which applications

communicate by inserting messages in specific queues

Example: IBM MQSeries

2 5 Stream Oriented Communication


41/59

41

until now, we focused on exchanging independent and

complete units of information

time has no effect on correctness; a system can be slow or fast

however, there are communications where time has a critical

role

Multimedia

media

storage, transmission, interchange, presentation,

representation and perception of different data types:

text, graphics, images, voice, audio, video, animation, ...

movie: video + audio +

multimedia: handling of a variety of representation media

2.5 Stream Oriented Communication


42/59

42

The Challenge

new applications

multimedia will be pervasive in few years (as graphics)

storage and transmission

e.g., 2 hours uncompressed HDTV (19201080) movie:

1.12 TB (19201080x3x25x60x60x2)

videos are extremely large, even after compressed continuous delivery

e.g., 30 frames/s (NTSC), 25 frames/s (PAL) for video

guaranteed Quality of Service

admission control search

can we look at 100 videos to find the proper one?


43/59

43

Types of Media

two types

discrete media: text, executable code, graphics, images;temporal relationships between data items are not

fundamental to correctly interpret the data

continuous media: video, audio, animation; temporal

relationships between data items are fundamental tocorrectly interpret the data

a datastream is a sequence of data units and can be applied

to discrete as well as continuous media

stream-oriented communication provides facilities for the

exchange oftime-dependent information (continuous media)

such as audio and video streams

timing in transmission modes


44/59

44

timing in transmission modes

asynchronous transmission mode: data items are

transmitted one after the other, but no timing constraints;

e.g. text transfer

synchronous transmission mode: a maximum end-to-end

delay defined for each data unit; it is possible that data can

be transmitted faster than the maximum delay, but not

slower

isochronous transmission mode: maximum and minimumend-to-end delay are defined; also called boundeddelay

jitter; applicable for distributed multimedia systems

a continuous data stream can be simple or complex

simple stream: consists of a single sequence of data; e.g.,

mono audio, video only (only visual frames)

complex stream: consists of several related simple streams

that must be synchronized; e.g., stereo audio, video

consisting of audio and video (may also contain subtitles,

translation to other languages, ...)


45/59

45

mov ie as a set of simple streams

a stream can be considered as a virtual connection between a


46/59

46

set t ing u p a stream between two p rocesses across a network

source and a sink

the source or the sink could be a process or a device

setting up a stream directly between two devices

th d t t l b lti t d t l i


47/59

47

an example of m ult icast ing a stream to several receivers

the data stream can also be multicasted to several receivers

if devices and the underlying networks have different

capabilities, the stream may be filtered, generally called

adaptation

Q lit f S i (Q S)


48/59

48

Quality of Service (QoS)

QoS requirements describe what is needed from the

underlying distributed system and network to ensure

acceptable delivery; e.g. viewing experience of a user for continuous data, the concerns are

timeliness: data must be delivered in time

volume: the required throughput must be met

reliability: a given level of loss of data must not beexceeded

qualityofperception; highly subjective

QoS Dimensions


49/59

49

timeliness dimensions

latency (maximum delay between consecutive frames)

start-up latency (maximum delay before starting apresentation)

jitter (delay variance)

volume dimensions

throughput in frames/sec or bits/sec or bytes/sec reliability dimensions

MTBF (Mean Time Between Failure) of disks

MTTR (Mean Time To Repair)

error rates on the telecommunication lines

QoS Dimensions

QoS Requirements


50/59

50

deterministic

precise values or ranges

e.g., latency must be between 45 and 55 ms probabilistic

probability of the required QoS

e.g., latency should be < 50 ms for 95% of the frames

stochasticdistributions e.g., frame arrival should follow normal distribution with

mean interval-time of 40 ms and 5 ms variance

classes

e.g., guaranteed and best effort

QoS Requirements

QoS Management


51/59

51

can be static or dynamic

Static QoS Management Functions

specification e.g., deterministic range for timeliness, volume and

reliability categories

negotiation

the application may accept lower level of QoS forlower cost

admissioncontrol

if this test is passed, the system has to guarantee the

promised QoS

resourcereservation

may be necessary to provide guaranteed QoS

QoS Management

Dynamic QoS Management Functions


52/59

52

monitoring

notices deviation from QoS level

at a certain level of granularity (e.g., every 100 ms) policing

detect participants not keeping themselves to the contract

e.g., source sends faster than negotiated (e.g., 25 fps)

maintenance sustaining the negotiated QoS

e.g., the system requires more resources

renegotiation

client tries to adapt may be can accept lower QoS

Dynamic QoS Management Functions

QoS requirements can be specified using flow specification


53/59

53

q p g p

containing bandwidth requirements, transmission rates,

delays, ...

e.g. by Partridge (1992)

it uses the token bucket algorithm which specifies how thestream will shape its network traffic (in fact the leaky

bucket, as used in networking)

the idea is to shape bursty traffic into fixed-rate traffic by

averaging the data rate packets may be dropped if the bucket is full

the input rate may vary, but the output rate remains

constant


54/59

54

the pr incip le of a token bucket algor i thm

Specifying QoS


55/59

55a flow s pecif icat ion

Characteristics of the Input Service Required

Maximum data unit size (bytes)

Token bucket rate (bytes/sec) Token bucket size (bytes)

Maximum transmission rate

(bytes/sec)

Loss sensitivity (bytes)

Loss interval (sec)

the above two specify the

maximum acceptable loss rate,

e.g. 1 byte per minute

Burst loss sensitivity (data units);

how many consecutive data units

may be lost

Minimum delay noticed (sec); how

long the network can delay the

delivery

Maximum delay variation (sec);

maximum tolerated jitter

Quality of guarantee; continue or

dont establish a stream if the

communication system can not

provide the required service

problem in flow specification


56/59

56

an application may not know its requirements

how can a user (human) specify quality using the various

parameters? usually very difficult

may be provide defaults for various streams as high,

medium, low quality

Setting up a Stream

resources such as bandwidth, buffers, processing powermust be reserved once a flow specification is made

one such protocol is RSVP - Resource reSerVation Protocol

it is a transport-level protocol for enabling resource

reservation in network routers


57/59

57

the basic o rganizat ion o f RSVP for resou rce reservat ion in a

dist r ibuted sys tem

Stream Synchronization


58/59

58the pr inc iple of exp l ic i t synchron izat ion on the level of data un its

how to maintain temporal relations between streams, e.g., lip

synchronization

two approaches

1. explicitly by operating on the data units of simple

streams; the responsibility of the application

2. through a multimedia middleware that offers a collection of


59/59

the pr incip le of sync hron izat ion as su ppo rted by high -level inter faces

g

interfaces for controlling audio and video streams as well as

devices such as monitors, cameras, microphones, ...

Chapter 2 Communication

Documents