D u k e S y s t e m s CPS 210 Software Architecture and OS Platforms Jeff Chase Duke University chase/cps210.

D u k e S y s t e m s

CPS 210Software Architecture and OS Platforms

Jeff ChaseDuke University

http://www.cs.duke.edu/~chase/cps210

Friday

• Recitations– Fridays: 3:05-4:20 PM

– Soc Sci 139: see maps.duke.edu

– TA: Vamsi Thummala

– This week: important info for P1A: heap manager

• Monday– Prof. Landon Cox on Unix

– Reading: flashback to 1968

“Software Architecture”

User Applications

Operating System(s)

Substrate / Architecture

Softwarearchitecture

Computerarchitecture

Comparative architecture: what worksReusable / recurring design patterns

• Used in OS• Supported by OS

Physics stops here.

Abstraction(s)

• A means to organize knowledge– Capture what is common and essential

– Generalize and abstract away the details

– Specialize as needed

– Concept hierarchy

• A design pattern or element– Templates for building blocks

– Instantiate as needed

• E.g.: class, subclass, and instance

Platform abstractions

• Platforms provide “building blocks”…

• …and APIs to use them.– Instantiate/create/allocate

– Manipulate/configure

– Attach/detach

– Combine in uniform ways

– Release/destroy

• Abstractions are layered.– What to expose? What to hide?

The choice of abstractions reflects a philosophy of how to build and organize software systems.

[Garlan/Shaw 94]Components and connectors

Component

Component

Connector

Garlan and Shaw, An Introduction to Software Architecture, 1994.

Abstract Data Type (ADT)object/class/module pipes/filters

objects / ADTseventslayeringrepositoryinterpreter

Program

Context(Domain)

Thread

“Components in context”

For our purposes, an operating system is a platform that supports protection and isolation: every component runs within a context.Program, context and thread are OS abstractions.

On most platforms, contexts are isolated. They cannot interfere with one another, even if they are running the same program.

I.e., each context is its own “sandbox”. The programs may fail independently.

In classical OS, a context is called a process.Each has a private virtual address space.

Isolated contexts (domains)

Butler Lampson’s definition: “I am isolated if anything that goes wrong is my fault (or my program’s fault).”

Isolation might not be absolute.E.g., the program instances may choose to interact, or may access shared data.

Program

Running a program

When a program launches, the OS platform allocates memory to store its code and data.It may establish a new context and/or thread.

data

codeconstants

initialized dataimports/exports

symbolstypes/interfaces

The Operating System

• An operating system:– sets up the contexts

– enforces isolation

– mediates interactions

• Applications trust the OS to do these things.

• Any software platform for running application programs can be called an OS.

The Trusted Computing Base (TCB)

• Programs may incorporate (link to) libraries packaged with the OS or written by third parties.

• These libraries may even define a system API that applications use. (e.g., heap manager malloc and free).

• But they run within the program’s context, and so are not isolated from the rest of the program at runtime. Any failure in the library can damage the program, and any failure of the rest of the program can damage the library.

• These libraries have no more privilege or power than any other part of the program.

• The part of a platform that is responsible for the integrity of application contexts is called its TCB.

• Trusted software is often called the kernel in classical OS.

Operating Systems: The Classical View

data dataPrograms

run asindependent processes.

Protected system calls

...and upcalls (e.g., signals)

Protected OS kernel

mediates access to

shared resources.

Threads enter the kernel for

OS services.

Each process has a private

virtual address space and one

or more threads.

The kernel code and data are protected from untrusted processes.

Key Concepts for Classical OS• kernel

• The software component that controls the hardware directly, and implements the core privileged OS functions.

• Modern hardware has features that allow the OS kernel to protect itself from untrusted user code.

• thread• An executing instruction path and its CPU register state.

• virtual address space• An execution context for thread(s) defining a name space

for executing instructions to address data and code.

• process• An execution of a program, consisting of a virtual address

space, one or more threads, and some OS kernel state.

Principles of Computer System Design Saltzer & Kaashoek 2009

Protection Systems 101

Reference monitorExample: OS platform

Isolation boundary

Post-note• This slide introduces some new terminology that we will hear more about later:

• Principal. Some entity that requests an action. It is often (but not always) associated with a real-world identity, such as a user or account. E.g., in Android, each application is a separate principal.

• Authentication. When data arrives through a channel, or a channel is established, we generally need to determine something about the principal controlling the other endpoint of the channel. I.e., “to whom am I speaking?”. (See slides on “channels” later in this deck.)

• Authorization. Checks to determine if a given authenticated caller (a subject or principal) is permitted to perform the requested operation, according to an access control policy or procedure (the guard).

• Reference monitor. Any component that has an API that is externally visible (through a channel) must check authorization for operations on that API before performing the op. The component must be isolated from its callers, so it must run in a context that is distinct from the callers. We can use the term reference monitor to refer to the context, or the component, or the guard.

Threads: a familiar metaphor

Page links and back button navigate a

“stack” of pages in each tab.

Each tab has its own stack.One tab is active at any given time.

You create/destroy tabs as needed.You switch between tabs at your whim.

Similarly, each thread has a separate stack.The OS switches between threads at its whim.

One thread is active per CPU core at any given time.

1

2

3

time

OS Platform: A Better Model

Platform: same for all applications

Libraries: shared by multiple applications

Applications

OS platform mediates access to shared resources.

[RAD Lab]

[Garlan/Shaw 94]Components and connectors

Component

Component

Connector

Garlan and Shaw, An Introduction to Software Architecture, 1994.

Abstract Data Type (ADT)object/class/module pipes/filters

objects / ADTseventslayeringrepositoryinterpreter

GS1. Pipes and filters

Garlan and Shaw, An Introduction to Software Architecture, 1994.

GS2. Objects and invocation

Garlan and Shaw, An Introduction to Software Architecture, 1994.

GS3. Events and implicit invocation

Event producers {publish/raise/announce/signal} events to notify {consumers/receivers/subscribers} of occurrences relating to a {subject/topic/category}.The system invokes a registered handler method/procedure in each receiver.Delivery is synchronous or asynchronous.

publishnotify

subscriberegister

Communication: endpoints and channels

channelpipe

bindingconnection

endpointport

data transferstream

flowmessages

request/reply RPC

events

operationsadvertise (bind)listenconnect (bind)close

write/sendread/receive

If one side advertises a named endpoint, we call it a server.

If one side initiates achannel to a named endpoint, we call it a client.

Pipes

pipe

operationsclosewriteread

parent

Unix supports IPC channels called pipes between pairs of processes. Unidirectional byte stream (upstream, downstream). Brokered by a common parent that creates the pair. The child processes need not create/initiate the pipe or even

know where it comes from or where it goes. A pipe may be the “standard” input or output.

Example: Unix Pipes

[http://www.bell-labs.com/history/unix/philosophy.html]

MapReduce/Hadoop

MapReduce is a filter-and-pipe model for data-intensive cluster computing

We stopped here

We discussed the need for protected contexts (“sandboxes”), even on single-user systems like your smartphone. The discussion touched on various dimensions of isolation for protected contexts (e.g., processes):

•Fault isolation. One app or app instance (context or process) can fail independently of others. If it fails, the OS can kill the process and reclaim all of its memory, etc.

•Performance isolation. The OS manages resources (“metal and glass”: computing power, memory, disk space, I/O throughput capacity, network capacity, etc.). Each instance needs the “right amount” of resources to run properly. The OS prevents apps from impacting the performance of other apps. E.g., the OS can prevent a program with an endless loop from monopolizing the processor.

•Security. An app may contain malware that tries to corrupt the system, steal data, or otherwise compromise the integrity of the system. The OS uses protected contexts and a reference monitor to check and authorize all accesses to data or objects outside of the context, including channel setup.)

OS protectionThe discussion touched on how a classical OS uses the hardware to protect itself. (More on this later.)

•In particular, machines support events. The OS kernel registers handlers for various machine events when it boots (starts up). These events include machine exceptions (faults), which may be caused by errant code, interrupts from the clock or external devices (e.g., network packet arrives), and deliberate kernel calls (traps) caused by programs requesting service from the kernel through its API.

•All of these machine events make safe control transfers into the registered kernel handler for the named event. Applications run in sandboxes that prevent them from calling procedures in the kernel or accessing kernel data directly (unless the kernel chooses to allow it).

•In fact, after the system is booted, these events are the only ways to ever enter the kernel, i.e., to run code in the kernel.

GS4. Layered systems

Garlan and Shaw, An Introduction to Software Architecture, 1994.

Post-note

• Note that an operating system is a layered system.– The platform API encapsulates all the functionality below.

– Other platforms may be layered on top (“nested dolls”)

– Applications above the platform API provide other useful functions.

– These application-layer programs include utilities (e.g., program development tools) and servers (e.g., window system).

– See the layering diagrams for Android and Unix.

• Also, an OS uses layering below the platform API. – e.g., network protocol stack (later)

– device driver software, etc.

Other application programs

cc

Other application programs

Hardware

Kernel

sh who a.out

date

wc

grepedvi

ld

as

comp

cppnroff

Unix defines uniform, modular ways to combine programs to build up more complex functionality.

GS5. Shared state repository

Garlan and Shaw, An Introduction to Software Architecture, 1994.

State

storefilesystemrepository

volumedatabase

target

accessattach/detach

mount/unmountmap/unmapopen/close

put/getfetch

read/writepush

commit

storage objectsegment

filetable

object

bit, byteblockpagerow

fragmentstripeobject

GS6. Interpreter

Garlan and Shaw, An Introduction to Software Architecture, 1994.

Interpreter: example

An interpreter controls how a program executes and what it sees.

An interpreter can “sandbox” a program for isolation.

ECMAscript

Some languages and interpreters are safer than others.

Architectural styles: tradeoffs

• constraints on composition– give rise to general properties of compositions

• generality of connectors– data type signatures

• coupling and control

• concurrency

• state (data) partitioning and sharing– less sharing better scalability

– standardizing interfaces vs. schema

The Birth of a Program (C/Ux)

int j;char* s = “hello\n”;

int p() { j = write(1, s, 6); return(j);}

myprogram.c

compiler

…..p: store this store that push jsr _write ret etc.

myprogram.s

assembler data

myprogram.o

linker

object file

data program

(executable file)myprogram

datadatadata

libraries and other objects