1 Interprocess Communication CE 513 Computer Engineering, IEU.

1

Interprocess CommunicationInterprocess Communication

CE 513CE 513

Computer Engineering, IEUComputer Engineering, IEU

Cooperating Processes

• An independent process is one that cannot affect or be affected by the execution of another process

• A cooperating process can affect or be affected by the execution of another process in the system

• Advantages of process cooperation– Information sharing (of the same piece of data) – Computation speed-up (break a task into smaller subtasks)– Modularity (dividing up the system functions)– Convenience (to do multiple tasks simultaneously)

• Two fundamental models of interprocess communication– Shared memory (a region of memory is shared)– Message passing (exchange of messages between

processes)

Communications Models

Message passing Shared memory

Shared Memory Systems

• Shared memory requires communicating processes to establish a region of shared memory

• Information is exchanged by reading and writing data in the shared memory

• A common paradigm for cooperating processes is the producer-consumer problem

• A producer process produces information that is consumed by a consumer process– unbounded-buffer places no practical limit on

the size of the buffer– bounded-buffer assumes that there is a fixed

buffer size

Message-Passing Systems• Mechanism to allow processes to communicate and to synchronize their

actions

• No address space needs to be shared; this is particularly useful in a distributed processing environment (e.g., a chat program)

• Message-passing facility provides two operations:

– send(message) – message size can be fixed or variable

– receive(message)

• If P and Q wish to communicate, they need to:

– establish a communication link between them

– exchange messages via send/receive

• Logical implementation of communication link

– Direct or indirect communication

– Synchronous or asynchronous communication

– Automatic or explicit buffering

Direct Communication• Processes must name each other explicitly:

– send (P, message) – send a message to process P– receive(Q, message) – receive a message from process Q

• Properties of communication link– Links are established automatically between every pair of

processes that want to communicate– A link is associated with exactly one pair of communicating

processes– Between each pair there exists exactly one link– The link may be unidirectional, but is usually bi-directional

• Disadvantages– Limited modularity of the resulting process definitions– Hard-coding of identifiers are less desirable than indirection

techniques

Indirect Communication• Messages are directed and received from mailboxes (also

referred to as ports)– Each mailbox has a unique id– Processes can communicate only if they share a mailbox

• send(A, message) – send message to mailbox A• receive(A, message) – receive message from

mailbox A• Properties of communication link

– Link is established between a pair of processes only if both have a shared mailbox

– A link may be associated with more than two processes– Between each pair of processes, there may be many

different links, with each link corresponding to one mailbox

Indirect Communication (continued)

• For a shared mailbox, messages are received based on the following methods:– Allow a link to be associated with at most two

processes– Allow at most one process at a time to execute a

receive() operation– Allow the system to select arbitrarily which process will

receive the message (e.g., a round robin approach)• Mechanisms provided by the operating system

– Create a new mailbox– Send and receive messages through the mailbox– Destroy a mailbox

Synchronization

• Message passing may be either blocking or non-blocking• Blocking is considered synchronous

– Blocking send has the sender block until the message is received

– Blocking receive has the receiver block until a message is available

• Non-blocking is considered asynchronous– Non-blocking send has the sender send the message

and continue– Non-blocking receive has the receiver receive a valid

message or null• When both send() and receive() are blocking, we have a

rendezvous between the sender and the receiver

Buffering

• Whether communication is direct or indirect, messages exchanged by communicating processes reside in a temporary queue

• These queues can be implemented in three ways:

1. Zero capacity – the queue has a maximum length of zero- Sender must block until the recipient receives the message

2. Bounded capacity – the queue has a finite length of n - Sender must wait if queue is full

3. Unbounded capacity – the queue length is unlimited Sender never blocks

11

Interprocess Communications

- Exchange of data between two or more separate, independent processes/threads.

- Operating systems provide facilities/resources for inter-process communications (IPC), such as message queues, semaphores, and shared memory.

- Distributed computing systems make use of these facilities/resources to provide application programming interface (API) which allows IPC to be programmed at a higher level of abstraction. (e.g., send and receive)

- Distributed computing requires information to be exchanged among independent processes.

12

IPC – unicast and multicast• In distributed computing, two or more processes engage

in IPC using a protocol agreed upon by the processes. A process may be a sender at some points during a protocol, a receiver at other points.

• When communication is from one process to a single other process, the IPC is said to be a unicast, e.g., Socket communication. When communication is from one process to a group of processes, the IPC is said to be a multicast, e.g., Publish/Subscribe Message model.

13

Unicast vs. Multicast

P 2

P 1 P 1

P 2 P 3 P 4...

uni c as t m ul t i c as t

mm m m

14

P r o c es s 1 P r o c es s 2

d ata

s en d er r ec e iv er

Interprocess Communications in Distributed Computing

15

Operations provided in an IPC API

• Receive ( [sender], message storage object)• Connect (sender address, receiver address), for

connection-oriented communication.• Send ( [receiver], message)• Disconnect (connection identifier), for connection-

oriented communication.

16

Interprocess Communication in basic HTTP

C 1 C 2

S 3 S 4

C 4

W e b s e rv e r

W e b bro ws e r

a p r o c e s s

a n o p e r a t i o n

d a t a f l o w

o p e r a t i o n s :S 1 : a c c e p t c o n n e c t i o nS 2 : r e c e i ve ( r e q u e s t )S 3 : s e n d ( r e s p o n s e )S 3 : d i s c o n n e c tC 1 : m a k e c o n n e c t i o nC 2 : s e n d ( r e q u e s t )C 3 : r e c e i ve ( r e s p o n s e )C 4 : d i s c o n n e c t

S 2

C 3

S 1

H TTPr e q u e s t

H TTPr e s p o n s e

Processing order: C1, S1, C2, S2, S3, C3, C4, S4

s4

17

Event Synchronization• Interprocess communication may require

that the two processes synchronize their operations: one side sends, then the other receives until all data has been sent and received.

• Ideally, the send operation starts before the receive operation starts.

• In practice, the synchronization requires system support.

18

Synchronous vs. Asynchronous Communication

• The IPC operations may provide the synchronization necessary using blocking. A blocking operation issued by a process will block further processing of the process until the operation is fulfilled.

• Alternatively, IPC operations may be asynchronous or nonblocking. An asynchronous operation issued by a process will not block further processing of the process. Instead, the process is free to proceed with its processing, and may optionally be notified by the system when the operation is fulfilled.

19

Synchronous send and receive

pr o c e s s 1r unni ng o n ho s t 1

b lo c k in g s en d s ta r ts

b lo c k in g s en d r e tu r n s

b lo c k in g r ec e iv e s ta r ts

b lo c k in g r ec e iv e en d s

ex ec u tio n f lo w

s u s p en d ed p er io d

Sync hr o no us Se nd and R e c e i ve

an o p er a t io n

ac k n o w led g em en t o f d a ta r ec e iv edp r o v id ed b y th e I P C f ac ility

pr o c e s s 2r unni ng o n ho s t 2

Client ServerSender Receiver

Event Diagram

20

Asynchronous send and synchronous receive

P r o c e s s 1

P r o c e s s 2

b lo c k in g r ec e iv e s ta r ts

b lo c k in g r ec e iv e r e tu r n s



As ync hr o no us Se nd and Sync hr o no us R e c e i ve

n o n b lo c k in g s en d

o p er a tio n

Event Diagram

Client ServerSender Receiver

21

Synchronous send and Async. Receive - 1

P r o c e s s 1

P r o c e s s 2

n o n b lo c k in g r ec e iv e is s u ed



Sync hr o no us Se nd and As ync hr o no us R e c e i ve

b lo c k in g s en d is s u ed

S ce n a rio A

t r an s p ar en t ac k n o w led g em en tp r o v id ed b y th e I P C f ac ility

Data from P1 was received by P2 before issuing a non-blocking receive op in P2

22


in d ef in iteb lo c k in g

P r o c e s s 1

P r o c e s s 2

n o n b lo c k in g r ec e iv e is s u edan d r e tu r n ed im m ed ia te ly





S ce n a rio B

P r o c e s s 1

P r o c e s s 2

Data from P1 arrived to P2 after P2 issued a non-blocking receive op

23


P r o c e s s 1

P r o c e s s 2

n o n b lo c k in g r ec e iv e is s u edan d r e tu rn ed im m ed ia te ly





S ce n a rio C

p ro c es s is n o tif iedo f th e a r r iv a l o fd ata

tr an s p ar en t ac k n o w led g em en tp ro v id ed b y th e I P C f ac ility

Data from P1 arrived to P2 before P2 issues a non-blocking receive op. P2 is notified of the arrival of data

24

Asynchronous send and Asynchronous receive P r o c e s s 1

P r o c e s s 2

n o n b lo c k in g r ec e iv e is s u edan d r e tu r n ed im m ed ia te ly



As ync hr o no us Se nd and As ync hr o no us R e c e i ve


S ce n a rio C

p r o c es s is n o tif iedo f th e a r r iv a l o fd a ta

Does P1 need an acknowledgement from P2?

25

Event diagramPro ce s s A

Pro ce s s B

in te rp r o c es s c o m m u n ic a tio n


p ro c es s b lo c k ed

E ve nt di ag r am fo r a pr o to c o l

r eq u es t 1

r es p o n s e 1

r es p o n s e2

r eq u es t 2

t im e

Synchronous send and receive

26

Blocking, deadlock, and timeouts

• Blocking operations issued in the wrong sequence can cause deadlocks.

• Deadlocks should be avoided. Alternatively, timeout can be used to detect deadlocks.

r ec e iv e f r o m p r o c es s 2 is s u ed

r ec e iv ed f r o m p r o c es s 1 is s u ed

p r o c es s 1 b lo c k ed p en d in g d a taf r o m p r o c es s 2 .

p r o c es s 2 b lo c k ed p en d in g d a taf r o m p r o c es s 1 .

P r o c es s 1 P r o c es s 2

P1 is waiting for P2’s data; P2 is waiting for P1’s data.

27

Using threads for asynchronous IPC

• When using an IPC programming interface, it is important to note whether the operations are synchronous or asynchronous.

• If only blocking operation is provided for send and/or receive, then it is the programmer’s responsibility to using child processes or threads if asynchronous operations are desired.

pro ce s s

m ain th r ead

n ew th r ead is s u es a b lo c k in g I P C o p er a tio n

th r ead is b lo c k ed

th r ead is u n b lo c k ed af te r th e o p er a tio n is f u lf illed

m ain th r ead c o n tin u es w itho th er p r o c es s in g

28

Deadlocks and Timeouts

• Connect and receive operations can result in indefinite blocking

• For example, a blocking connect request can result in the requesting process to be suspended indefinitely if the connection is unfulfilled or cannot be fulfilled, perhaps as a result of a breakdown in the network .

• It is generally unacceptable for a requesting process to “hang” indefinitely. Indefinite blocking can be avoided by using timeout.

• Indefinite blocking may also be caused by a deadlock

29

Indefinite blocking due to a deadlock

" re ce iv e f ro m pro ce s s 2 " is s u e d;

" re ce iv e f ro m pro ce s s 1 " is s u e d;

pro ce s s 1 blo ck e d pe n din g da tafro m pro ce s s 2 .

pro ce s s 2 blo ck e d pe n din g da tafro m pro ce s s 1 .

Pro ce s s 1 Pro ce s s 2

p r o c e s se x e c u t i n g

p r o c e s sb l o c k e d

a n o p e r a t i o n

P1 is waiting for P2’s data; P2 is waiting for P1’s data.

30

Data Representation• Data transmitted on the network is a binary stream.• An interprocess communication system may provide the

capability to allow data representation to be imposed on the raw data.

• Because different computers may have different internal storage format for the same data type, an external representation in standard format may be required.

• Data marshalling is the process of (I) flattening a data structure, and (ii) converting the data to an external representation.

• Some well known external data representation schemes are: Sun XDR (External Data Representation) ASN.1 (Abstract Syntax Notation One) XML (Extensible Markup Language)

31

Data Encoding Protocols

ap p lic a tio n s p ec if ic d a ta en c o d in g lan g u ag e

g en era l d a ta en c o d in g lan g u ag e

n etw o rk d a ta en c o d in g s tan d ard

da ta e n co din g s ch e m e s S a m ple S ta n da rdsle v e l o f a bs tra ct io n

X M L :( Ex ten s ib le M ark u p L an g u ag e)

AS N .1 (Ab s tr ac t S y n tax N o ta tio n )

S u n X D R (E x ter n a l D ata R ep res en ta tio n )

32

Sample XML filehttp://java.sun.com/xml/docs/tutorial/overview/1_xml.html#intro

• XML is a text-based markup language that is fast becoming the standard for data interchange on the Web.

• XML has syntax analogous to HTML.• Unlike HTML, XML tags tell you what the data

means (meta data), rather than how to display it.

• Example:<message>

<to>[email protected]</to> <from>[email protected]</from>

<subject>XML Is Really Cool</subject> <text> How many ways is XML cool? Let me count the

ways... </text>

</message>

33

Data Marshalling"T h is is a tes t . "

"T h is is a tes t . "

1 .2 7 .3 - 1 .5

1 .2

7 .3

- 1 .5

1 1 0 0 1 1 . . . 1 0 0 0 0 1 0 0 . . .

m arsh a llin g

u n m arsh a llin g

1 . f la t ten in g o f s tr u c tu r ed d a ta item s2 . c o n v er tin g d a ta to ex ter n a l ( n e tw o r k ) r ep r es en ta tio n

1 . c o n v er t d a ta to in te r n a l r ep r es en ta tio n2 . r eb u ild d a ta s tr u c tu r es .

h o s t A

h o s t B

E x ter n a l to in te r n a l r ep r es en ta tio n an d v ic e v er s ais n o t r eq u ir ed - if th e tw o s id es a r e o f th e s am e h o s t ty p e ; - if th e tw o s id es n eg o tia tes a t c o n n ec tio n .

34

The OSI (Open System Interconnection ) Seven-layer network architecture

ap p lic a tio n lay er

p r es en ta tio n lay er

s es s io n lay er

tr an s p o r t lay er

n e tw o r k lay er

d a ta lin k lay er

p h y s ic a l lay er

ap p lic a tio n lay er

p r es en ta tio n lay er

s es s io n lay er

tr an s p o r t lay er

n e tw o r k lay er

d a ta lin k lay er

p h y s ic a l lay er

Message

Segment

Datagram

Frame

0/1

http://www.webopedia.com/quick_ref/OSI_Layers.asp

35

Text-based protocols

• Data marshalling is at its simplest when the data exchanged is a stream of characters, or text.

• Exchanging data in text has the additional advantage that the data can be easily parsed in a program and displayed for human perusal. Hence it is a popular practice for protocols to exchange requests and responses in the form of character-strings. Such protocols are said to be text-based.

• Many popular network protocols, including FTP (File Transfer Protocol), HTTP, and SMTP (Simple Mail Transfer Protocol), are text-based.

36

Event diagram

Pro ce s s 1Pro ce s s 2

in te rp r o c es s c o m m u n ic a tio n


p ro c es s b lo c k ed

E ve nt di ag r am fo r a pr o to c o l

r eq u es t 1

r es p o n s e 1

r es p o n s e2

r eq u es t 2

t im e

37

Sequence Diagram

Pro ce s s A Pro ce s s B

in te r p r o c es s c o m m u n ic a tio n

r eq u es t 1

r es p o n s e 1

r eq u es t 2

r es p o n s e 2

38

Protocol

• In a distributed application, two processes perform interprocess communication in a mutually agreed upon protocol.

• The specification of a protocol should include

(i) the sequence of data exchange, which can be described using a time event diagram.

(ii) the format of the data exchange at each step.

39

HTTP: A sample protocol

• The HyperText Transfer Protocol is a protocol for a process (the browser) to obtain a document from a web server process.

• It is a request/response protocol: a browser sends a request to a web server process, which replies with a response.

40

The Basic HTTP protocol

w eb s er v er w eb b r o w s er

r eq u es t

r es p o n s e

r eq u es t is a m es s ag e in 3 p ar ts : - < c o m m an d > < d o c u m en t ad d d r es s > < HT T P v er s io n > - an o p tio n a l h ead er - o p tio n a l d a ta f o r C G I d a ta u s in g p o s t m eth o d

r es p o n s e is a m es s ag e c o n s is t in g o f 3 p ar ts : - a s ta tu s lin e o f th e f o r m at < p r o to c o l> < s ta tu s c o d e> < d es c r ip tio n > - h ead er in f o r m atio n , w h ic h m ay s p an s ev er a l lin es ; - th e d o c u m en t its e lf .

W e w ill ex p lo r e HT T P in d e ta ils la te r th is q u ar te r .

41

A sample HTTP session

Script started on Tue Oct 10 21:49:28 2000 9:49pm telnet www.csc.calpoly.edu 80 Trying 129.65.241.20... Connected to tiedye2-srv.csc.calpoly.edu. Escape character is '̂ ]'. GET /~mliu/ HTTP/1.0 HTTP Request HTTP/1.1 200 OK HTTP response status line Date: Wed, 11 Oct 2000 04:51:18 GMT HTTP response header Server: Apache/1.3.9 (Unix) ApacheJ Serv/1.0 Last-Modified: Tue, 10 Oct 2000 16:51:54 GMT ETag: "1dd1e-e27-39e3492a" Accept-Ranges: bytes Content-Length: 3623 Connection: close Content-Type: text/html <HTML> document content <HEAD> <TITLE> Mei-Ling L. Liu's Home Page </TITLE> </HEAD> <BODY bgcolor=#ffffff> …

42

IPC paradigms and implementations

Paradigms of IPC of different levels of abstraction have evolved, with corresponding implementations.

re mo te p ro ce d u re /me th o d

so cke t API

d a ta tra n smiss io n s er ia l/p ar a lle l c o m m u n ic a tio n

Un ix s o c k e t AP I , W in s o c k

R em o te P r o c ed u r e C all ( R P C ) , J av a R M I

le v e l o fa bs tra ct io n

I PC pa ra dig m s Ex a m ple I PC I m ple m e n ta t io n s

UNIX Socket: http://www.ecst.csuchico.edu/~beej/guide/ipc/usock.html

43

Inter-process communication (IPC)• We have seen IPC via shared data in main memory.• Processes in separate address spaces also need to communicate.• Consider system architecture – both shared memory and cross-address-space IPC is needed• Recall that the OS runs in every process address space:

user thread user thread

shareddata

plusshared memory IPC

between user threads of a process

shared memory IPCIn OS

cross-address-space IPCe.g. pipes, message passing

OS

userspace

userspace

an address spaceof a process

an address spaceof a process

plusshared memory IPC

between user threads of a process

Cross address-space IPC

44

Concurrent programming paradigms – overview

IPC via shared data – processes share an address space – we have covered: 1. shared data is a passive object accessed via concurrency-controlled operations:

conditional critical regions, monitors, pthreads, Java 2. active objects (shared data has a managing process/thread) Ada select/accept and rendezvous 3. lock-free programming

We now consider: Cross-address-space IPC

Message passing – asynchronous – supported by all modern OS programming language example: ErlangKilim – a Java extension for shared memory or cross-address-space message passing.

Message passing – synchronous e.g. occam

Consider which of these might be used for distributed programming.


45

UNIX pipes outline - revision

A UNIX pipe is a synchronised, inter-process byte-streamA process attempting to read bytes from an empty pipe is blocked.

There is also an implementation-specific notion of a “full” pipe - a process is blocked on attempting to write to a full pipe.a pipe is implemented as an in-memory buffer in the file buffer-cache.The UNIX designers attempted to unify file, device and inter-process I/O.

To set up a pipe a process makes a pipe system call and is returned two file descriptorsin its open file table. It then creates, using fork two children who inherit all the parent’s

open files, including the pipe’s two descriptors.

Typically, one child process uses one descriptor to write bytes into the pipe and the other child process uses the other descriptor to read bytes from the pipe.Hence: pipes can only be used between processes with a common ancestor.

Later schemes used “named pipes” to avoid this restriction.UNIX originated in the late 1960s, and IPC came to be seen as a major deficiency.Later UNIX systems also offered inter-process message-passing, a more general scheme.


46

Asynchronous message passing - 1

A B

address space of a process

OS implementation of message passing

process A’sconceptual

message queue(actually in shared

data area,“waiting messages”)

waitMess ( ) may call OS_block_thread (tID)sendMess ( ) may call OS_unblock_thread (tID)

waitingmessages

waitMess (ptr)sendMess (ptr)

implementation of waitMess and sendMess

waitMess ( )

sendMess ( )

address space of a process

. = potential delay

waitMess ( ) before

sendMess ( )avoids buffering


47

Asynchronous message passing -2

Note no delay on sendMess in asynchronous message passing (OS buffers if no-one waiting)

Note cross-address-space IPC implemented by shared memory IPC in OS

Details of message header and body are system and language-specific e.g. typed messages.At OS-level message transport probably sees a header plus unstructured bytes.

Need to be able to wait for a message from “anyone” as well as from specific sourcese.g. server with many clients

Client-server interaction easily set up e.g. needed for system services in microkernel - structured OS.

A B

waitMess (ptr)sendMess (ptr) .

waitMess (ptr) sendMess (ptr).

.= potential delay


48

Programming language example: Erlang

Erlang is a functional language with the following properties: 1. single assignment – a value can be assigned to a variable only once, after which the variable is immutable

2. Erlang processes are lightweight (language-level, not OS) and share no common resources. New processes can be forked (spawned), and execute in parallel with the creator: Pid = spawn ( Module, FunctionName, ArgumentList ) returns immediately – doesn’t wait for function to be evaluated process terminates when function evaluation completes Pid returned is known only to calling process (basis of security) Pid is a first class value that can be put into data structures and passed in messages

3. asynchronous message passing is the only supported communication between processes. Pid ! Message ! means send Pid is the identifier of the destination process Message can be any valid Erlang term

Erlang came from Ericsson and is used for telecommunications applications.


49

Erlang continued – receiving messages

The syntax for receiving messages is (recall guarded commands and Ada active objects): receive Message1 ( when Guard1) -> Actions1 ; Message2 ( when Guard2 ) -> Actions2 ; .......... endEach process has a mailbox – messages are stored in it in arrival order.Message1 and Message2 above are patterns that are matched against messages in the process mailbox. A process executing receive is blocked until a message is matched.When a matching MessageN is found and the corresponding GuardN succeeds, the message isremoved from the mailbox, the corresponding ActionsN are evaluated and receive returns the value of the last expression evaluated in ActionsN.Programmers are responsible for making sure that the system does not fill up with unmatched messages.Messages can be received from a specific process if the sender includes its Pid in the pattern to be matched: Pid ! {self( ), abc} receive {Pid, Msg}


50

Erlang further information and examples

Part 1 of Concurrent Programming in Erlang is available for download fromhttp://erlang.org/download/erlang-book-part1.pdf

The first part develops the language and includes many small programs, including distributed programs, e.g. page 89 (page 100 in pdf) has the server and client code, with discussion, for an ATM machine.

The second part contains worked examples of applications, not available free.

A free version of Erlang can be obtained fromhttp://www.ericsson.com/technology/opensource/erlang


51

Kilim – shared address-space message passingKilim extends Java via annotations and static checking. A mailbox paradigm is used.

concurrent process

call write-message ( message)

• = potential delay

write-message

read-message

mailbox

•

…….. wait ( )

concurrent process

call read-message ( )

- after writing a message into a mailbox the sending process/thread loses all rights to that data- the receiver/reading-process gains rights. Based on linear type theory.- in the current shared memory implementation, threads share an address space messages are not physically copied but pointers are used i.e. mailboxes contain pointers- good performance demonstrated for large numbers of threads cf. Erlang - the motivation for Kilim is that in classical concurrency control, methods that execute under exclusion may make library calls, making it impossible for a compiler to carry out static checking of exclusive access to data.

messages


52

Kilim further information

Sriram Srinivasan’s PhD researchAfter some 12 years’ experience of developing web servers e.g. WebLogic, Ram wanted more efficient, but safe, concurrent programming with large numbers of threads.The work is inspired by Ada and Erlang, but extending Java gives a better chance ofacceptance and use.

For an overview, publications and download, see:http://www.malhar.net/sriram/kilim/

Ben Roberts’ PhD is to distribute Kilim, .... among other things ...


http://www.malhar.net/sriram/kilim/

53

Synchronous message passing -1

Delay on both sendMess and waitMess in synchronous message passing

Sender and receiver “hand-shake” - OS copies message cross-address-space

Note no message buffering in OS

How to avoid busy servers being delayed by non-waiting clients (on sending answer)? buffers could be built at application-level but synchronous message passing is not appropriate for client-server programming.

A B

waitMess (ptr)sendMess (ptr) .

waitMess (ptr) sendMess (ptr).

.

.


.= potential delay

1 Interprocess Communication CE 513 Computer Engineering, IEU.

Documents