Plan 9 from Bell Labs: Real World Distributed Operating System Case Study Plan 99Grid Bryan Kinney March 22, 2005 CPSC450/550 Distributed Operating Systems.

Plan 9 from Bell Labs:Real World Distributed Operating System Case Study

Plan 9 9Grid

Bryan KinneyMarch 22, 2005CPSC450/550 Distributed Operating SystemsDepartment of Physics, Computer Science and EngineeringChristopher Newport University

Plan 9 Case Study

Introduction History Goals Definitions Features Using Plan 9 Applications Significance Summary References

Plan 9: Introduction

Distributed SystemA distributed system can be defined as a

system in which its components are computers connected using a network that communicate and coordinate their actions only by passing messages [1].

Plan 9: Introduction

Major Design Considerations of Distributed Systems [1] Heterogeneity

The system must be constructed from a variety of different networks, operating systems, computer hardware and programming languages.

Openness Distributed systems should be extensible and provide published interfaces.

Security Provide adequate protection of shared resources and keep sensitive information safe

when transmitting messages. Scalability

Provide a system in which the cost to support adding a user is consistent in terms of the resources that must be added.

Failure handling Provide a system that can deal with the failure of any one of its components.

Concurrency Provide safe concurrent requests to shared resources.

Transparency Provide a single system image that allows the user to only be concerned with their

application and not aspects of the system such as networks topography and CPUs.

Plan 9: Introduction

Case Study How does Plan 9 compare to the definition

of a Distributed System? Does Plan 9 take into account the major

design considerations of:

Heterogeneity, Openness, Security, Scalability, Failure handling, Concurrency, Transparency

Plan 9: History

Bell Labs began development in late 1980’s

4 Releases Last Release was April of 2002

Updated June 2003 4th Edition available from Bell Labs site

http://plan9.bell-labs.com/plan9dist/index.html

Plan 9: History

Paradigm shift in mid 1980’sFrom: Large centralized timeshared

computersTo: Networks of smaller personal machines

Typically UNIX ‘workstations’

Plan 9 wanted it both waysCentral AdministrationCost effective

Plan 9: Goals

Build a time sharing system out of workstations

“Build a UNIX out of a lot of little systems, not out of a lot of little UNIXs”

Plan 9: Goals

Specialization of hardwareDifferent computers for different tasksSmall cheap machines in peoples offices

Terminals providing access to shared resources File servers, computing servers

File Servers and CPU Servers in Server Room

Plan 9: Goals

Personal Workstation Systems

Centralized Time shared Systems

Mainframe

Terminals

CPU ServersFile Servers

High Speed Network

Centrally located multiprocessor machines with large memories and high bandwidth point to point

interconnects.

Terminal

Medium Speed Network

Plan 9 Distributed Operating System

Plan 9: Definitions

Plan 9: Plan 9 is the name of the Distributed Operating System designed and

implemented by researchers at Bell Lab in Murray Hill New Jersey. It is available for download from the Plan 9 website [7].

9Grid: 9Grid is the name of a Plan 9 installation that provides Grid Style

Computing. More information can be found at the 9 Grid website [8].

8½: The Plan 9 window system. It provides textual I/O and bitmap graphic

services to both local and remote client programs by offering a multiplexed file service to those clients. It serves traditional UNIX files like /dev/tty as well as more unusual ones that provide access to the mouse and the raw screen. Bitmap graphics operations are provided by serving a file called/dev/bitblt that interprets client messages to perform raster operations [9].

Plan 9: Definitions

Rc: Rc is a command interpreter for Plan 9 that provides similar facilities to

UNIXs Bourne shell, with some small additions and less idiosyncratic syntax [10].

9P Protocol: The 9P protocol is the Plan 9 file system protocol. It is structured as a

set of transactions that send a request from a client to a (local or remote) server and return the result [2].

IL Protocol: IL is a custom implemented network protocol to transport the remote

procedure call messages 9P. It is a connection-based, lightweight transport protocol that carries datagrams encapsulated by IP. IL provides retransmission of lost messages and in-sequence delivery, but has no flow control and no blind retransmission [6].

Plan 9: Definitions

Factotum: Factotum is the central component of the security architecture.

Factotum securely holds a copy of the user’s keys and negotiates authentication protocols, on behalf of the user, with secure services around the network [5]

Plan 9: Features

Built upon 3 principles. 1. Resources as files

Named and accessed like files in a hierarchical file system.

2. Standard protocol9P, for accessing these resources.

3. Hierarchical file name spaceThe disjoint hierarchies provided by different services are joined together into a single private hierarchical file name space.

Plan 9: Features

1. Resources as files All resources in Plan 9 look like file systems. File Oriented access

Hierarchical name tree Accessible by name Access contents by read and write calls

2. 9P Protocol Standard Protocol

for accessing resources

Bell labs implemented IL Protocol for network transport of 9P messages

Plan 9: Features

Plan 9: Features

3. Hierarchical File namespace Every resource in the system,

local or remote

User or process assembles a private view of the system by constructing a file name space that connects the resources.

Allows the user to access files that are local or remote in the same manner. When writing a program the user does not need to create code that handles for cases in which the file is not local. Those details are abstracted to the system.

Plan 9: Features

Security Factotum is the central component of the security architecture.

Factotum securely holds a copy of the user’s keys and negotiates authentication protocols, on behalf of the user, with secure services around the network [5]

Plan 9: Features

Security Each box is a (typically)

separate machine; Ellipse is a process. FX are factotum processes PX are the pieces and proxies

of a distributed program. The authentication server is

one of several repositories for users security information that factotum processes consult as required.

Secstore is a shared resource for storing private information such as keys; factotum consults it for the user during bootstrap

Plan 9: Features

Capabilities for Parallel Programming Kernel provides a simple process model and a few

carefully designed system calls for synchronization and sharing.

Parallel programming language called Alef supports concurrent programming.

Although it is possible to write parallel programs in C, Alef is the parallel language of choice.

Alef uses a system call called rendezvous to provides a way for processes to synchronize.

Plan 9: Structure

Main Parts File servers CPU servers Terminals (user

access points).

Typically centrally located file servers and CPU servers

CPU ServersFile Servers

High Speed Network

Centrally located multiprocessor machines with large memories and high bandwidth point to point

interconnects.

Terminal

Medium Speed Network

Plan 9 Distributed Operating System

Plan 9: Using Plan 9 Example: Echo Server

This Code implements a typical TCP listener.

It announces itself listens for connections, and

forks a new process for each. The new process echoes data

on the connection until the remote end closes it.

The "*" in the symbolic name means the announcement is valid for any addresses bound to the machine the program is run on.

Plan 9: Using Plan 9 Example: Echo Server

1.Announce()

2.Listen()

3.Accept() || Reject()

4.Process()….

5.Close()…

Plan 9: Using Plan 9 Example: Echo Server

1. Announce() Returns open file descriptor for

the ctl file of a connection and fills dir with the path of the protocol directory for the announcement.

int announce(char *addr, char *dir)

Addr is the symbolic name/address announced; if it does not contain a service, the announcement is for all services not explicitly announced.

Plan 9: Using Plan 9 Example: Echo Server

2. Listen() Listen returns an open file

descriptor for the ctl file and fills ldir with the path of the protocol directory for the received connection.

It is passed dir from the announcement.

int listen(char *dir, char *ldir)

Plan 9: Using Plan 9 Example: Echo Server

3. Accept() and Reject() Accept and reject are called

with the control file descriptor and ldir returned by listen.

Some networks such as Datakit accept a reason for a rejection; networks such as IP ignore the third argument.

int accept(int ctl, char *ldir)

int reject(int ctl, char *ldir, char *reason)

Plan 9: Applications

Developed in research environment Continue research news way to work with Distributed Operating

Systems 9Grid

The Advanced Computing Cluster Research Lab, at Los Alamos National Laboratory uses Plan 9 for a secure grid environment:

“… utilize the distributed features of the Plan 9 operating system to create a tightly-coupled Grid environment in which running applications can cross the boundaries of the local cluster or institution and utilize resources around the globe or even further away. A distinguishing feature of 9grid is its security model. Plan 9 is a far more secure system, from the ground up, than any Unix system ever built. There is no need for add-ons such as firewalls to make Plan 9 Grid-capable. As users attach to nodes on the 9grid, their entire file system name space is visible from all the nodes which they are using -- and invisible to anyone else.”

Plan 9: Significance

Fulfills definition of Distributed System 9P: Uses message passing to coordinate

actions Provides for the major design

considerations as outlined Heterogeneity, Openness, Security,

Scalability, Failure handling, Concurrency, Transparency

Plan 9: Summary

The Plan 9 Distributed Operating System is a Distributed System which gives its users a single system image of the system of networks and hardware it manages. Plan 9 gives users the tools for conventional programming and parallel programming. Plan 9 is a distributed system built from the ground up incorporating distributed concepts and encouraging future research of concepts developed through using distributed computing environments.

Plan 9: References

1. Coulouris,George. Jean Dollimore, Tim Kindeberg, Distributed Systems Concepts and Design. Pearson Education Limited, 2001

2. Pike, Rob. Dave Presotto, Sean Dorward, Bob Flandrena, Ken Thompson, Howard Trickey, and Phil Winterbottom, Plan 9 from Bell Labs,http://plan9.bell-labs.com/sys/doc/index.html

3. Presotto, Dave. Phil Winterbottom. The Organization of Networks in Plan 9, http://plan9.bell-labs.com/sys/doc/index.html

4. Pike, Rob. Dave Presotto, Sean Dorward, Bob Flandrena, Ken Thompson, Howard Trickey, and Phil Winterbottom, The Use of Namespaces in Plan 9,http://plan9.bell-labs.com/sys/doc/index.html

5. Cox, Russ. Eric Grosse, Rob Pike, Dave Presotto, Sean Quinlan. Security in Plan 9. http://plan9.bell-labs.com/sys/doc/index.html

6. Presotto, Dave. Phil Winterbottom. The IL Protocol. http://plan9.bell-labs.com/sys/doc/index.html

Plan 9: References

7. Plan 9 Website, http://plan9.bell-labs.com/plan9dist/index.html8. 9Grid Website, http://plan9.bell-labs.com/9grid/index.html9. Pike, Rob. 8½, The Plan 9 Window System. http://plan9.bell-

labs.com/sys/doc/index.html10. LDuff, Tom, Rc- The Plan 9 Shell.

http://plan9.bell-labs.com/sys/doc/index.html11. Advanced Computing Cluster Research Lab, Los Alamos

National Laboratory. http://public.lanl.gov/cluster/projects/index.html

12. 9Grid (LANL): http://www.9grid.net

Plan 9 from Bell Labs:Real World Distributed Operating System Case Study

Plan 9 9Grid

Bryan KinneyMarch 22, 2005CPSC450/550 Distributed Operating SystemsDepartment of Physics, Computer Science and EngineeringChristopher Newport University