CMP320 Operating Systems Lecture 07, 08 Operating System Concepts October 05 2009 Arif Butt Note: Some slides and/or pictures are adapted from Lecture.

CMP320Operating Systems

Lecture 07, 08

Operating System Concepts

October 05 2009Arif Butt

Note: Some slides and/or pictures are adapted from Lecture slides / Books of• Dr Mansoor Sarwar.• Dr Kubiatowicz.• Dr P. Bhat.• Dr Hank Levy.• Dr Indranil Gupta.• Text Book - OS Concepts by Silberschatz, Galvin, and Gagne.• Ref Books

• Operating Systems Design and Implementation by Tanenbaum.• Operating Systems by William Stalling.• Operating Systems by Colin Ritchie.

Today’s Agenda• Review of previous lecture• Cooperating Processes• Producer Consumer Problem• IPC–Message Passing System• Direct Communication• Indirect Communication

– Shared Memory System• Using pipes and FIFOS on the shell• Using pipe() system call in UNIX programs• Using mknod() system call in UNIX programs• Using shmget() system call in UNIX programs2CMP320 PUCIT Arif Butt04/10/23

Cooperating Processes• Independent process is a process that

cannot affect or cannot be affected by the execution of another process. A process that does not share data with another process is independent

• Cooperating process is a process that can affect or can be affected by the execution of another process. A process that share data with other process is a cooperating process

• Advantages of Cooperating processes:– Information sharing– Computation speed up– Modularity– Convenience 3CMP320 PUCIT Arif Butt04/10/23

Inter Process Communication

4CMP320 PUCIT Arif Butt04/10/23

• IPC is a mechanism for processes to communicate and to synchronize their actions. There are two fundamental models of IPC:

• Shared Memory – A region of memory that is shared by cooperating processes is established. Processes can then exchange information by reading and writing data to the shared region

• Messaging system – Communication takes place by means of messages exchanged between the cooperating processes. Processes communicate with each other without resorting to shared variables. (pipes, FIFO, sockets, message queues, semaphores, signals)

Inter Process Communication


Producer-Consumer Problem


ProducerProducer ConsumerConsumer

Empty Pool

Full Pool

Producer-Consumer Problem (cont…)• A producer process produces information that is

consumed by a consumer process• To allow producer and consumer run concurrently we

must have a buffer that can be filled by the producer and emptied by the consumer. Buffer can be bounded or unbounded

• Unbounded Buffer– Places no practical limit on the size of the buffer– The consumer may have to wait for new items (if the

buffer is empty), but the producer can always produce new items

• Bounded Buffer– Assumes a fixed size buffer– The consumer must wait if the buffer is empty, and

the producer must wait if the buffer is full 7CMP320 PUCIT Arif Butt04/10/23

Solution 1-Bounded Buffer Problem

8CMP320 PUCIT Arif Butt

#define BUFFER_SIZE 5

typedef struct{ ---- } item;

item buffer[BUFFER_SIZE];

int in = 0; //points to location where next item will be placed, will be used by producer process

int out = 0; //points to location from where item is to be consumed, will be used by consumer process

04/10/23

Producer Process

item nextProduced;

while(1)

{

nextProduced = getItem();

while(((in + 1)%BUFFER_SIZE) == out)

; //do nothing

buffer[in] = nextProduced;

in = (in + 1) % BUFFER_SIZE;

}

Consumer Process

item nextConsumed;

while(1)

{

while(in == out)

; //do nothing

nextConsumed = buffer[out];

out = (out + 1) % BUFFER_SIZE;

}

What is the problem with this solution?

Solution 2 - Bounded Buffer Problem


#define BUFFER_SIZE 5

typedef struct{ ---- } item;

item buffer[BUFFER_SIZE];

int in = 0; //points to location where next item will be placed, will be used by producer process

int out = 0; //points to location from where item is to be consumed, will be used by consumer process

int ctr = 0;

04/10/23

Producer Process

item nextProduced;

while(1)

{

nextProduced = getItem();

while(ctr == BUFFER_SIZE)

; //do nothing

buffer[in] = nextProduced;

in = (in + 1) % BUFFER_SIZE;

ctr++; }

Consumer Process

item nextConsumed;

while(1)

{

while(ctr == 0)

; //do nothing

nextConsumed = buffer[out];

out = (out + 1) % BUFFER_SIZE;

ctr--;

}

Can you do it withoutusing the ctr variable?

Messaging System


• Message passing provides a mechanism to allow processes to communicate and to synchronize their actions without sharing the same address space

• It is particularly useful in a distributed environment, where the communicating processes may reside on different computers connected by a network

• For example; a chat program used on the www

• Message Passing is one solution to the communication requirement. The added bonus is: It works with shared memory and with distributed systems

Messaging System (cont…)


• A message passing facility provides at least two operations:

– send(message) – message size fixed or variable

– receive(message)

• If P and Q wish to communicate, they need to:

– establish a communication link between them

– exchange messages via send/receive



Direct Communication• Processes must name each other explicitly:

– send (P, message) – send a message to process P– receive(Q, message) – receive a message from process Q

• Send primitive includes a specific identifier of the destination process

• Receive primitive could know ahead of time which process a message is expected

• Receive primitive could use source parameter to return a value when the receive operation has been performed

• Example is a letter mailed via Pakistan Postal Service• Properties of communication link in Direct Comm.:

– A Link is established automatically– A link is associated with exactly one pair of communicating processes– Between each pair there exists exactly one link



Indirect Communication• Messages are directed and received from mailboxes

(also referred to as ports)– Each mailbox has a unique id– Processes can communicate only if they share a mailbox– We can think of a mailbox as an abstract object into

which a message can be placed in or removed from

• Properties of communication link in Indirect Comm.:– Link established only if processes share a common

mailbox.– A link may be associated with many processes– Each pair of processes may share several communication

links



Indirect Communication (cont…)

• Operations– create a new mailbox.– send and receive messages through mailbox.– destroy a mailbox.

• Primitives are defined as:send(A, message) – send a message to mailbox Areceive(A, message) – receive a message from mailbox A

• Mailbox SharingP1, P2, and P3 share mailbox AP1, sends; P2 and P3 receiveWho gets the message?

• Solutions1. Allow a link to be associated with at most two processes.2. Allow only one process at a time to execute a receive

operation.3. Allow the system to select arbitrarily the receiver. Sender is

notified who the receiver was.

Indirect Process Communication (…)

04/10/23 15CMP320 PUCIT Arif Butt

General Message Format




Synchronization

• Message passing may be either blocking or non-blocking

• Blocking is considered synchronous

– Blocking send. The sending process is blocked until the message is received by the receiving process or by the mailbox. (Producer process blocks

when the buffer is full)

– Blocking receive. The receiver blocks until a message is available. (Consumer process blocks when the

buffer is empty)

• Non-blocking is considered asynchronous

– Non-blocking send. The sending process sends the message and resumes operation

– Non-blocking receive. The receiver receives either a valid message or a null



Buffering• Whether communication is direct or indirect, messages

exchanged by communicating processes reside in a temporary queue. Such queues can be implemented in three ways:

• Zero Capacity. The queue has a length of zero. Thus the link cannot have any messages waiting in it. The sender must block until the recipient receives the message

• Bounded Capacity. The queue has finite length n. Sender must wait if link is full

• Unbounded Capacity. The queue is of infinite length. So any number of messages can wait in it. Thus the sender never blocks

UNIX / LINUX IPC Tools


Pipes are used for communication between related processes (parent-child-sibling) on the same UNIX system.

P1 P2

UNIX/Linux System

Pipe



Named pipes (FIFO) are used for communication between related or unrelated processes on the same UNIX system.

P1 P2

UNIX/Linux System

FIFO

p1 p2



Sockets are used for communication between related or unrelated processes on the same or different UNIX systems.

Computer 1

Computer 2

P1 P2

Network ConnectionSocket Socket

USING PIPESIN

CMD LINE & PROGRAMMING


Cmd Line use of Pipes


• The UNIX system allows stdout of a command

to be connected to stdin of another command

using the pipe operator |.

cmd1 | cmd2 | cmd3 | … | cmdN

• Stdout of cmd1 is connected to stdin of cmd2,

stdout of cmd2 is connected to stdin of cmd3,

… and stdout of cmdN-1 is connected to stdin

of cmdN.

cmd2

cmdNpipe pipe pipecmd1

Cmd Line Use of Pipes


• Example 1. Write a command that displays the contents of /etc/ directory, one page at a time.

$ ls –l /etc/ 1> temp

$ less 0< temp

This will display the contents of file temp on screen one page at a time. After this we need to remove the temp file as well.

$ rm temp

So we needed three commands to accomplish the task, and the command sequence is also extremely slow because file read and write operations are involved.

Lets use the pipe symbol

$ ls –l /etc/ | less

The net effect of this command is the same. It does not use a disk to connect standard output of ls –l to standard input of less because pipe is implemented in the main memory.

Cmd Line Use of Pipes (cont…)


• Example 2. Write a command that will sort the

contents of file students.txt and displays those

contents on screen after removing duplication if any.

$ sort 0< students.txt 1> stds_sorted

$ uniq stds_sorted

$ rm stds_sorted

Lets use the pipe symbol

$ sort students.txt | uniq

Cmd Line Use of Pipes (cont…)


• Pipes and redirection operators alone cannot be used

to redirect stdout of a command to a file as well as

another command in a single command

• tee command copies stdin to stdout and at the same

time copies it to one or more files

$ tee tout1.txt tout2.txt

• tee command is used to redirect the stdout of a

command to one or more files, as well as to another

command

cmd1 | tee file1 … fileN | cmd2

• Stdout of cmd1 is connected to stdin of tee, and tee

sends its input to files file1, … fileN and as stdin of

cmd2

$ who | tee who.txt

System Call - pipe()

Call Fails when:

• Opening a pipe is equivalent to opening two files, so at least two

slots need to be empty in the PPFDT, otherwise, the call will fail.

• Buffer space not available in the kernel.

• File table is full.

int pipe(int pipefd[2]);

• pipefd[0] and pipefd[1] are read and write descriptors respectively.

• Will return -1 on failure• Bounded buffer, maximum data written is PIPE_BUF <linux/param.h>, 4096 bytes


System Call - pipe()


P1 P2fork

parent child

Write endRead end

• Process P1 will create a pipe (having two ends, a read end and a write end).

• Then the Parent Process P1 will create a child process P2.

• There are two descriptors associated with the pipe. The writer process (child) will use the write descriptor of the pipe. The reader process (parent) will use the read descriptor of the pipe.

• The child process will write data onto the pipe and the parent process will read that data at the read end and will display it on the screen.

Parent read from the pipe. It blocks if the pipe is empty.

Child writes in the pipe. It blocks if the pipe is full.pipe

Child Process will send a string to Parent Processand Parent Process will print that on screen.

Sample Program using pipe()


/* pipe1.c (Parent creates pipe, forks a child, child writes into pipe, and parent reads from pipe).*/

#include <stdio.h> #include <stdlib.h>#include <string.h> #include <sys/types.h>#include <sys/wait.h>int main(){ int pipefd[2], cpid, n, rc, nr; char *msg = "Hello, world!\n“; char * buf[1024]; rc = pipe(pipefd); //create a pipe if (rc == -1) {//if pipe call fails printf(“\n Pipe failed"); exit(1);} cpid = fork(); //if pipe call is successful we create a child if (cpid == -1) { printf(“\n Fork failed"); //if fork fails exit(1); }


if (cpid == 0) { // Child’s Code (child is the writer process)

write(pipefd[1], msg, strlen(msg));

exit(0);

}

else { // Parent’s Code (parent is the reader process)

n = strlen(msg);

nr = read(pipefd[0], buf, n);

rc = write(1, buf, nr);//use write sc to display the data //read from the pipe on the screen

printf("Good work child!\n");

return(0);

}

}

Sample Program using pipe() (cont…)

USING FIFOSIN

CMD LINE & PROGRAMMING


Named Pipes / FIFOs


• First In First Out are special file types in UNIX used for communication between related or unrelated processes on a computer.

• Processes communicating with pipes must be related to each other through a common ancestor process that you execute, processes communicating with FIFOs do not have to have this kind of relationship. They can be independently executed programs on a system.

• A pipe is a main memory buffer and has no name, while FIFO is created on disk and has a name like a file name.

• So like files and unlike a pipe, a FIFO has to be created and opened before using it for communication.

• We can use mkfifo command to create a FIFO file in a shell session. We can use mknod() system call or mkfifo() library call to create a FIFO in a process.

• UNIX system calls read(), write(), close(), etc. can be used with FIFOs.

Named Pipes / FIFOs


• Two common uses of FIFOs are:

• Used by shell commands to pass data from

one shell pipeline to another, without

creating temporary files

• In client server applications, FIFOs are used

to pass data between a server process and

client processes

Cmd Line Use of FIFOs


• Example. Write a command that will count the number

of lines in the man page of ls.

$ man ls | wc –l

239

$

• Example. Write a command that will count only the

number of lines containing string “ls” in the man page

of ls.$ man ls | grep ls | wc –l

9

$

Cmd Line Use of FIFOs (cont…)


$ mkfifo fifo1

$ prog3 0< fifo1 &

$ prog1 0< infile | tee fifo1 | prog2$ mkfifo fifo1

man ls 1> ls.dat

$ cat 0< fifo1 | grep ls | wc –l &

[1] 21108

$ sort 0< ls.dat | tee fifo1 | wc –l

9

239

prog1

prog2

prog3

infile

wc -l

infile

fifo1

wc -l Pipe

sort tee

grep Pipe

• First line will create an empty FIFO named fifo1.• In second line, prog3 is trying to read its input from fifo1. Since fifo1 is empty therefore prog3 will block at the read operation. More so the & shows that it will run in the back ground.• In third line, prog1 will read its input from infile and send its o/p to tee utility and tee will do two things:i.Write the o/p to fifo1 as well as to the stdout which will be sent to prog3ii.Through the pipe will give the o/p to prog2

Client Server Comm with FIFOs


client-1 client-K

server

well-known FIFO

client FIFO

. . .

read request

send reply

read response read response

send reply

send request

send request

client FIFO

Server will open a well known FIFO, which it will useto read the request of client.

Each client will open a client FIFO and send its name to the server process,so now the server knows via which client FIFO it can send reply to a particular client

System Call - mknod()

Call Fails when:

• File with the given name already exists

• Pathname too long

• A component in the pathname not searchable or non existent

• Destination directory is read only

• Not enough memory space available

#include <sys/types.h>

#include <sys/stat.h>

int mknod(const char * pathname, mode_t mode, dev_t dev);

• mode should be permission mode ORed with S_IFIFO flag.• dev is set to 0 for a FIFO, i.e. we are not making a device file.


Library Call - mkfifo()

Call Fails when:

• File with the given name already exists

• Pathname too long

• A component in the pathname not searchable or non existent

• Destination directory is read only

• Not enough memory space available


#include <sys/stat.h>int mkfifo(const char * pathname, mode_t mode);

• mode sets permission on FIFO• Calls mknod()


Example - FIFO


FIFO1

Server Client

Display Screen

FIFO2

Reading

Writing Reading

Writing

Client - Server Code


//myheader.h

#include <stdio.h>

#include <stdlib.h>

#include <string.h>



#include <sys/errno.h>

#define FIFO1 "/tmp/fifo1"

#define FIFO2 "/tmp/fifo2"

#define PERMS 0666

#define MESSAGE1 "Hello, world!\n"

#define MESSAGE2 "Hello, class!\n“

extern int errno;

Server Code - 1


#include “myheader.h”main(){ char buff[1024]; int readfd, writefd, n, size;/*create fifo1 using mknod system call*/ if (mknod (FIFO1, S_IFIFO | PERMS, 0) < 0) { perror ("mknod FIFO1"); exit (1); }/*create fifo2 using mkfifo library call. if mkfifo() fails in creating FIFO2, before exiting we must remove FIFO1 from file system*/ if (mkfifo(FIFO2, PERMS) < 0) { unlink (FIFO1); perror("mknod FIFO2"); exit (1); }

Server Code - 2


if ((readfd = open(FIFO1, 0)) < 0) {//open FIFO1 for reading perror ("open FIFO1"); exit (1); } if ((writefd = open(FIFO2, 1)) < 0){//open FIFO2 for writing perror ("open FIFO2"); exit (1); } size = strlen(MESSAGE1) + 1;/*if client has yet not run, the server process will block on following read. As soon as some client executes and writes some msg in FIFO1, the server process will automatically unblock.*/ if ((n = read(readfd, buff, size)) < 0) { perror ("server read"); exit (1); } if (write (1, buff, n) < n) {//write on screen perror("server write1"); exit (1); } size = strlen(MESSAGE2) + 1; if(write(writefd,MESSAGE2,size) != size){//write on FIFO2 perror ("server write2"); exit (1); } close (readfd); close (writefd);}

Client Code - 1


#include “myheader.h”main(){ char buff[BUFSIZ]; int readfd, writefd, n, size; // open FIFO1 for writing

if ((writefd = open(FIFO1, 1)) < 0) { perror ("client open FIFO1"); exit (1); } // open FIFO2 for reading if ((readfd = open(FIFO2, 0)) < 0) { perror ("client open FIFO2"); exit (1); } size = strlen(MESSAGE1) + 1; // write message in FIFO1 if (write(writefd, MESSAGE1, size) != size) { perror ("client write1"); exit (1); }

Client Code - 2


/*if server has yet not run, the client process will block on following read. As soon as the server writes some msg in FIFO2, the client process will automatically unblock.*/

if ((n = read(readfd, buff, size)) < 0) { perror ("client read"); exit (1); } else if (write(1, buff, n) != n) { perror ("client write2"); exit (1); } close(readfd); close(writefd); /* Remove FIFOs now that we are done using them */ if (unlink (FIFO1) < 0) { perror("client unlink FIFO1"); exit (1); } if (unlink (FIFO2) < 0) { perror("client unlink FIFO2"); exit (1); } exit (0);}

FIFO-Running the Code


Use any editor to type your program and then to compile use gcc compiler:

$ gcc server.c –o server

$ gcc client.c –o client

$ server &

[1] 432056

$ client

Hello, World!

Hello, Class!

$

Shared Memory


Shared Memory System


• Shared memory allows two or more processes to share a given region of memory.

• This is the fastest form of IPC because the data does not need to be copied between the client and server.

• The only trick in using shared memory is synchronizing access to a given region among multiple processes.

• If the server is placing data into a shared memory region, the client shouldn't try to access the data until the sever is done.

• Often semaphores are used to synchronize shared memory access. We can use record locking as well.

Prog 1LogicalAddressSpace 1

Prog 2LogicalAddressSpace 2

Data 2Stack 1Heap 1Code 1Stack 2Data 1Heap 2Code 2Shared

• In shared memory we declare a given section in the memory as one that will be used simultaneously by several processes.

• Typically a shared memory region resides in the address space of the process creating the shared memory segment.

• Other processes that wish to communicate using this shared memory segment must attach it to their address space.

CodeDataHeapStack

Shared

CodeDataHeapStack

Shared

Shared Memory System (cont…)


System Call shmget()

49

• A process must first create a shared memory segment using shmget() system call (SHared Memory GET).

• On success, shmget() returns a shared memory identifier that is used in subsequent shared memory functions.

• On failure, it returns -1.int segment_id = shmget(IPC_PRIVATE, 4096, S_IRUSR | S_IWUSR);#include <sys/types.h>

#include <sys/ipc.h>

#include <sys/shm.h>

int shmget(key_t key, int size, int flag);• There is a special key value, IPC_PRIVATE, that creates

shared memory private to the process.

• The second parameter, size, specifies the amount of memory required in bytes.

• The third parameter, flag consists of nine permission flags. Normally it is IPC_CREAT | 066604/10/23 CMP320 PUCIT Arif Butt

System Call shmat()


• Processes that wish to access a shared memory segment must attach it to their address space using the shmat() system call. (SHared Memory ATach)

• On success, shmat() returns a pointer to the first byte of shared memory.

• On failure, it returns -1.char * shared_memory = (char*) shmat(segment_id, NULL, 0);

void* shmat(int shmid, const void *shmaddr, int flag);• First parameter, shm_id, is the shared memory

identifier returned from shmget.

• Second parameter is a pointer location in memory indicating where the shared memory will be attached. NULL pointer allows the system to choose the address at which the memory appears.

• Third parameter identifies the mode, a 0 means read-write.04/10/23

System Call shdt()


• When a process no longer requires access to the shared memory segment, it detaches the segment form its address space.

• This does not remove the identifier and its associated data structure from the system. The identifier remains in existence until some process specifically removes it by calling shmctl() with a command of IPC_RMID.

• To detach a region of shared memory, the process can pass the pointer of the shared memory region to the shmdt() system call. (SHared Memory DeTach)

shmdt(shared_memory);

int shmdt(void * addr);• The addr argument is the value that was returned by a

previous call to shmat.

04/10/23

System Call shctl()


• Shared memory segment can be removed from the system with the shmctl() system call. (SHared Memory ConTroL)

• Returns 0 if OK, -1 on error.

shmctl(segment_id, IPC_RMID, NULL);

void* shmctl(int shmid, int cmd, struct shmid_ds * buf);• First parameter, shmid, is the shared memory identifier

returned from shmget().

• Second parameter can take five different values. IPC_RMID

removes the shared memory segment set from the system.

• Third parameter is normally a NULL.

04/10/23

Sample Program-Shared Memory


/* Program creates a shared memory area, attaches it to its address space, writes a string in it. Reads and print the contents of shared memory. Detach and finally remove shared memory segment.*/

#include <stdio.h>

#include <sys/shm.h>

#include <sys/ipc.h>


int main(){

const int size = 4096;

int segment_id = shmget(IPC_PRIVATE, size, IPC_CREAT | 0666);

char * shared_memory = (char*) shmat(segment_id, NULL, 0);

sprintf(shared_memory, “Hello World”);

printf(“%s\n”,shared_memory);

shmdt(shared_memory);

shmctl(segment_id, IPC_RMID, NULL);

return 0;}

SUMMARY


We’re done for now, but Todo’s for you after this lecture…

55CMP320 PUCIT Arif ButtIf you have problems visit me in counseling hours. .

. .

• Go through the slides and Book Sections: 3.1 to 3.4,

3.51

• Solution 2 to the bounded buffer problem is using an

additional variable ctr. Suggest another solution that

do not use any additional variable (do not mention

the one discussed in class).

• Write down the programs discussed in class in vim

editor and execute them. Make variations in the

programs and understand the underlying details.04/10/23

CMP320 Operating Systems Lecture 07, 08 Operating System Concepts October 05 2009 Arif Butt Note: Some slides and/or pictures are adapted from Lecture.

Documents

producer process int

fixed size buffer

consumer process int

size int

producerconsumer problem

item item bufferbuffer

system communication

size ctr