B. Prabhakaran 1 Advanced Operating Systems Main Text: “Advanced Concepts in Operating Systems: Distributed, Database, & Multiprocessor Operating Systems”

B. Prabhakaran 1

Advanced Operating Systems Main Text:

“Advanced Concepts in Operating Systems: Distributed, Database, & Multiprocessor Operating Systems” by Mukesh Singhal and Niranjan G. Shivaratri. McGraw Hill publishers.

Project Reference Text: Unix Network programming: Networking APIs Sockets and

XTI (2nd edition), Prentice Hall.

Reference Papers: Recommend reading appropriate reference papers given in

each chapter end.

B. Prabhakaran 2

Contact InformationB. PrabhakaranDepartment of Computer ScienceUniversity of Texas at DallasMail Station EC 31, PO Box 830688Richardson, TX 75083Email: [email protected]: 972 883 2349URL: http://www.utdallas.edu/~praba/cs6378.htmlPhone: 972 883 4680Office: ES 3.706Office Hours: 11.45am-12.45pm, Mon/Wednesdays

Other times by appointments through emailAnnouncements: Made in class and on course web page.TA: TBA.

B. Prabhakaran 3

Course Outline

Introduction to operating systems, inter-process communication. (Chapters 1 & 2).

Distributed Operating Systems Architecture (Chapter 4) Clock Synchronization, Ordering (Chapter 5) Distributed Mutual Exclusion (Chapter 6) Distributed Deadlock Detection (Chapter 7) Agreement Protocols (Chapter 8)

Distributed Resource Management Distributed File Systems (Chapter 9) Distributed Shared Memory (Chapter 10) Distributed Scheduling (Chapter 11)

Proposed Outline. Might be modified based on time availability:

B. Prabhakaran 4

Recovery & Fault Tolerance Chapters 12 and 13

Concurrency Control/ Security Depending on time availability

Course Outline ...

Discussions will generally follow the main text. However,additional/modified topics might be introduced from other texts and/or papers. References to those materials will begiven at appropriate time.

B. Prabhakaran 5

Evaluation 1 Mid-term: in class. 75 minutes. Mix of MCQs (Multiple

Choice Questions) & Short Questions. 1 Final Exam: 75 minutes or 2 hours (depending on class

room availability). Mix of MCQs and Short Questions. 2 - 3 Quizzes: in class. 5-6 MCQs or very short questions.

15-20 minutes each. Homeworks/assignments: 3 or 4 spread over the semester. Programming Projects: 3 planned over the semester

B. Prabhakaran 6

Grading Home works: 5% Quizzes: 10% (15% - if 3rd Quiz) Mid-term & Final: 55% Project 1: 5% Projects 2 & 3: 25%

B. Prabhakaran 7

Schedule Quizzes: Dates announced in class & web, a week ahead.

Mostly just before midterm and final. Mid-term: February 26, 2007 Final Exam: Last day of class (April 23rd) OR 8.30am,

April 30, 2007 (As per UTD schedule) Subject to minor changes Quiz, projects, and homework schedules will be announced

in class and course web page, giving sufficient time for submission.

Likely project deadlines: end January, Middle of March, End of semester (April).

B. Prabhakaran 8

Programming Projects No copying/sharing of code/results will be tolerated. Any instance of

cheating in projects/homeworks/exams will be reported to the University.

No copying code from the Internet. 2 individual students copying code from Internet independently: still

considered copying in the project !! Individual projects. Projects might involve Unix, C/C++/Java programming, network

programming. Deadlines will be strictly followed for projects and homeworks

submissions. Projects submissions through Web CT. Demo may be needed

B. Prabhakaran 9

Web CT Go to: http://webct6.utdallas.edu Web CT has a discussion group that can be used for

project and other course discussions.

B. Prabhakaran 10

Cheating Academic dishonesty will be taken seriously. Cheating students will be handed over to

Head/Dean for further action. Remember: home works/projects (exams too !)

are to be done individually. Any kind of cheating in home works/ projects/

exams will be dealt with as per UTD guidelines. Cheating in any stage of projects will result in 0

for the entire set of projects.

B. Prabhakaran 11

Projects 3 projects planned. Involves exercises such as ordering, deadlock detection, load

balancing, message passing, and implementing distributed algorithms (e.g., for scheduling, etc.).

Platform: Linux, C/C++/Java. Network programming will be needed. Multiple systems might be used.

Specific details and deadlines will be announced in class and course webpage.

Suggestion: Learn network socket programming and threads, if you do not know already. Try simple programs for file transfer, talk, etc.

Sample programs and tutorials available at: http://www.utdallas.edu/~praba/projects.html

B. Prabhakaran 12

Homeworks Each homework will be for 10 marks. Homeworks Submission:

Submit on paper to TA/Instructor.

B. Prabhakaran 13

Basic Computer Organization Input Unit Output Unit CPU Memory ALU (Arithmetic &

Logic Unit) Secondary Storage

CPU

ALUMemory Disk

I/O

Keyboard

Display

B. Prabhakaran 14

Simplified View of OS

Memory Space

PhysicalMemory

VirtualMemory

OS Kernel

OS Tools

User Processes

Tools ++

User Processes ..

Data

Data

Data

Data

CodeCode Code

Code

User iProcess j

B. Prabhakaran 15

Distributed View of the System

hardware hardware

hardwarehardware

hardware

Process

B. Prabhakaran 16

Inter-Process Communication Need for exchanging data/messages among processes belonging

to the same or different group. IPC Mechanisms:

Shared Memory: Designate and use some data/memory as shared. Use the shared memory to exchange data. Requires facilities to control access to shared data.

Message Passing: Use “higher” level primitives to “send” and “receive” data. Requires system support for sending and receiving messages.

Operation oriented language constructs Request-response action Similar to message passing with mandatory response Can be implemented using shared memory too.

B. Prabhakaran 17

IPC Examples Parallel/distributed computation such as sorting: shared

memory is more apt. Using message passing/RPC might need an array/data manager

of some sort. Client-server type: message passing or RPC may suit

better. Shared memory may be useful, but the program is more clear

with the other types of IPCs. RPC vs. Message Passing: if response is not a must,

atleast immediately, simple message passing should suffice.

B. Prabhakaran 18

Shared Memory

Shared Memory

Only one process can write at anypoint in time. Noaccess to readers.

Multiple readers canaccess. No access toWriters.

Writers/Producers

Readers/Consumers

B. Prabhakaran 19

Shared Memory: Possibilities Locks (unlocks) Semaphores Monitors Serializers Path expressions

B. Prabhakaran 20

Message Passing Blocked Send/Receive: Both sending and receiving

process get blocked till the message is completely received. Synchronous.

Unblocked Send/Receive: Both sender and receiver are not blocked. Asynchronous.

Unblocked Send/Blocked Receive: Sender is not blocked. Receiver waits till message is received.

Blocked Send/Unblocked Receive: Useful ? Can be implemented using shared memory. Message

passing: a language paradigm for human ease.

B. Prabhakaran 21

Un/blocked Blocked message exchange

Easy to: understand, implement, verify correctness Less powerful, may be inefficient as sender/receiver might

waste time waiting

Unblocked message exchange More efficient, no waste on waiting Needs queues, i.e., memory to store messages Difficult to verify correctness of programs

B. Prabhakaran 22

Message Passing: Possibilities

sender

receiver i

receiver j

receiver k

sender

receiver i

receiver j

receiver k

B. Prabhakaran 23

Message Passing: Possibilities...

receiver

sender i

sender j

sender k

receiver

sender i

sender j

sender k

B. Prabhakaran 24

Naming Direct Naming

Specify explicitly the receiver process-id. Simple but less powerful as it needs the sender/receiver to know the

actual process-id to/from which a message is to be sent/received. Not suitable for generic client-server models

Port Naming receiver uses a single port for getting all messages, good for client-

server. more complex in terms of language structure, verification

Global Naming (mailbox) suitable for client-server, difficult to implement on a distributed network. complex for language structure and verification

B. Prabhakaran 25

Communicating Sequential Processes (CSP)

process reader-writer OKtoread, OKtowrite: integer (initially = value); busy: boolean (initially = 0);

*[ busy = 0; writer?request() ->busy := 1; writer!OKtowrite;

busy = 0; reader?request() ->

busy := 1; reader!OKtoread;busy = 1; reader?readfin() -> busy := 0;busy = 1; writer?writefn() -> busy := 0;

]

B. Prabhakaran 26

CSP: Drawbacks Requires explicit naming of processes in I/O commands. No message buffering; input/output command gets

blocked (or the guards become false) -> Can introduce delay and inefficiency.

B. Prabhakaran 27

Operation oriented constructs

Service declarations

Task A Task B

……RPC: abc(xyz, ijk);

…….

Service declarationssend xyz; wait for result

return ijk

Remote Procedure Call (RPC):

• Service declaration: describes in and out parameters• Can be implemented using message passing• Caller: gets blocked when RPC is invoked.• Callee implementation possibilities:

• Can loop “accepting” calls• Can get “interrupted” on getting a call• Can fork a process/thread for calls

B. Prabhakaran 28

RPC: Issues Pointer passing, global variables passing can be difficult. If processes on different machines, data size (number of

bits for a data type) variations need to be addressed. Abstract Data Types (ADTs) are generally used to take care of

these variations. ADTs are language like structures that specify how many bits

are being used for integer, etc… What does this imply?

Multiple processes can provide the same service? Naming needs to be solved.

Synchronous/blocked message passing is equivalent to RPC.

B. Prabhakaran 29

Ada

Task body <name> isDeclaration of local variablesbegin list of statements ….. accept <entry id> (<formal parameters> do body of the accept statement end<entry id> exceptions Exception handlersend;

task [type] <name> isentry specificationsend

task proc-buffer is entry store(x:buffer); remove(y:buffer);end;

task body proc-buffer is temp: buffer;begin loop when flag accept store(x: buffer);

temp := x; flag :=0; end store; When !flag accept remove(y:buffer); y := temp; flag :=1; end remove; end loopend proc-buffer.

B. Prabhakaran 30

Ada Message Passing

….accept Store(.)…...

….Store(xyz);…..

entry store(...)xyz is sent fromTask B to A

Task A Task B

Somewhat similar to executing procedure call. Parameter valuefor the entry procedure is supplied by the calling task. Value of Result, if any, is returned to the caller.

result

B. Prabhakaran 31

RPC Design Structure

Caller: local call + stub Callee: stub + actual procedure

Binding Where to execute? Name/address of the server that offers a

service Name server with inputs from service specifications of a task.

Parameter & results Packing: Convert to remote machine format Unpacking: Convert to local machine format

B. Prabhakaran 32

RPC Execution

Local call

Return

Querybindingserver

Paramspacking

Wait

Unpackresult

Unpack

Localcall

Packresults

Executeprocedure

Return

ReceiveQuery

ReturnServer Address

RegisterServices

BindingServer

Caller Callee

Stub StubLocal Proc. Remote Proc.

B. Prabhakaran 33

RPC Semantics At least once

A RPC results in zero or more invocation. Partial call, i.e., unsuccessful call: zero, partial, one or more

executions.

Exactly once Only one call maximum Unsuccessful? : zero, partial, or one execution

At most once Zero or one. No partial executions.

B. Prabhakaran 34

RPC Implementation Sending/receiving parameters:

Use reliable communication? : Use datagrams/unreliable? Implies the choice of semantics: how many times a RPC may be

invoked.

B. Prabhakaran 35

RPC Disadvantage

Incremental results communication not possible: (e.g.,) response from a database cannot return first few matches immediately. Got to wait till all responses are decided.

B. Prabhakaran 36

Distributed Operating Systems

Global Knowledge Naming Scalability Compatibility Process Synchronization Resource Management Security Structuring Client-Server Model

Issues :

B. Prabhakaran 37

DOS: Issues .. Global Knowledge

Lack of global shared memory, global clock, unpredictable message delays

Lead to unpredictable global state, difficult to order events (A sends to B, C sends to D: may be related)

Naming Need for a name service: to identify objects (files, databases),

users, services (RPCs). Replicated directories? : Updates may be a problem. Need for name to (IP) address resolution. Distributed directory: algorithms for update, search, ...

B. Prabhakaran 38

DOS: Issues .. Scalability

System requirements should (ideally) increase linearly with the number of computer systems

Includes: overheads for message exchange in algorithms used for file system updates, directory management...

Compatibility Binary level: Processor instruction level compatibility Execution level: same source code can be compiled and

executed Protocol level: Mechanisms for exchanging messages,

information (e.g., directories) understandable.

B. Prabhakaran 39

DOS: Issues .. Process Synchronization

Distributed shared memory: difficult.

Resource Management Data/object management: Handling migration of files, memory

values. To achieve a transparent view of the distributed system. Main issues: consistency, minimization of delays, ..

Security Authentication and authorization

B. Prabhakaran 40

DOS: Issues .. Structuring

Monolithic Kernel: Not needed (e.g.,) file management not needed fully on diskless workstations.

Collective kernel: distributed functionality on all systems. Micro kernel + set of OS processes Micro kernel: functionality for task, memory, processor

management. Runs on all systems. OS processes: set of tools. Executed as needed.

Object-oriented system: services as objects. Object types: process, directory, file, … Operations on the objects: encapsulated data can be manipulated.

B. Prabhakaran 41

DOS: CommunicationComputer

Switch

B. Prabhakaran 42

ISO-OSI Reference Model

Network

Application

Presentation

Session

Transport

Datalink

Physical

Network

Application

Presentation

Session

Transport

Datalink

Physical

Network

Datalink

Physical

Network

Datalink

Physical

Communication Network

B. Prabhakaran 43

Un/reliable Communication Reliable Communication

Virtual circuit: one path between sender & receiver. All packets sent through the path.

Data received in the same order as it is sent. TCP (Transmission Control Protocol) provides reliable

communication.

Unreliable communication Datagrams: Different packets are sent through different paths. Data might be lost or out of sequence. UDP (User datagram Protocol) provides unreliable

communication.