Final Review 01 Operating Systems Wenbo Shen Acknowledgement: Some slides are based on Yajin Zhou@ZJU slides.
Final Review 01
Operating SystemsWenbo Shen
Acknowledgement: Some slides are based on Yajin Zhou@ZJU slides.
01: Introduction
3
Four Components of a Computer System
4
Interrupts and Traps
• Interrupt transfers control to the interrupt service routine
• interrupt vector: a table containing addresses of all the service routines
• incoming interrupts are disabled while serving another interrupt to prevent a lost interrupt
• interrupt handler must save the (interrupted) execution states
• A trap is a software-generated interrupt, caused either by an error or a user request
• an interrupt is asynchronous; a trap is synchronous
• e.g., system call, divided-by-zero exception, general protection exception…
• Int 0x80
• Operating systems are usually interrupt-driven
5
Interrupt Handling
• Operating system preserves the execution state of the CPU• save registers and the program counter (PC)
• OS determines which device caused the interrupt• polling• vectored interrupt system
• OS handles the interrupt by calling the device’s driver• OS restores the CPU execution to the saved state
6
I/O: from System Call to Devices, and Back
• A program uses a system call to access system resources
• e.g., files, network
• Operating system converts it to device access and issues I/O requests
• I/O requests are sent to the device driver, then to the controller
• e.g., read disk blocks, send/receive packets…
• OS puts the program to wait (synchronous I/O) or returns to it without waiting (asynchronous I/O)
• OS may switches to another program when the requester is waiting
• I/O completes and the controller interrupts the OS
• OS processes the I/O, and then wakes up the program (synchronous I/O) or send its a signal (asynchronous I/O)
7
Operating System Operations: Multiprogramming
• Multiprogramming is necessary for efficiency
• single user cannot keep CPU and I/O devices busy at all times
• user’s computing tasks are organized as jobs (code and data)
• kernel schedules jobs (job scheduling) so CPU always has things to do
• a subset of total jobs in system is kept in memory
• when a job has to wait (e.g., for I/O), kernel switches to another job
• What are the problems here?
8
Operating System Operations: multitasking
• Timesharing (multitasking) extends the multiprogramming• OS switches jobs so frequently that users can interact
with each running job • response time should be < 1s • each user has at least one program executing in memory
(process)• if several jobs ready to run at the same time (CPU
scheduling)• It makes the programmer easier: virtual/physical memory
9
Dual-mode operation
• Operating system is usually interrupt-driven (why?)
• Efficiency, regain control (timer interrupt)
• Dual-mode operation allows OS to protect itself and other system components
• user mode and kernel mode (or other names)
• a mode bit distinguishes when CPU is running user code or kernel code
• some instructions designated as privileged, only executable in kernel and cannot executed in user mode (and certain memory cannot be accessed in user mode!)
• system call changes mode to kernel, return from call resets it to user
10
Transition between Modes
• System calls, exceptions, interrupts cause transitions between kernel/user modes
In user mode, certain instructions cannot be executed
11
How System Call Is Handled
12
How System Call Is Handled• System call number and parameters:
• on 32bit x86 architecture, the system call identifier is stored in the EAX register, while parameters in registers EBX, ECX, EDX, ESI, EDI, EBP
• Steps:
• The application is setting up the system call number (EAX for x86) and parameters and it issues a trap instruction (int 0x80 for x86)
• The execution mode switches from user to kernel; the CPU switches to a kernel stack; the user stack and the return address to user space is saved on the kernel stack
• The kernel entry point saves registers on the kernel stack (SAVE_ALL)
• Get system call number
• The system call dispatcher identifies the system call function using the system call number
• Execute the system call function and saves the return value (to EAX for x86)
• The user space registers/stack are restored and execution is switched back to user (e.g. calling IRET)
• The user space application resumes
13
Resource Management: Process Management
• A process is a program in execution• program is a passive entity, process is an active
entity• a system has many processes running
concurrently• Process needs resources to accomplish its task• OS reclaims all reusable resources upon process
termination• e.g., CPU, memory, I/O, files, initialization data
14
Process Management Activities
• Process creation and termination• Processes suspension and resumption• Process synchronization primitives• Process communication primitives• Deadlock handling
15
From Process to Thread
• Single-threaded process has one program counter
• program counter specifies location of next instruction to execute
• processor executes instructions sequentially, one at a time, until completion
• Multi-threaded process has one program counter per thread
• Process is the unit of resource allocation and protection, not thread!
Figures: https://medium.com/coderscorner/threads-vs-processes-a0b199f56c2d
16
Resource Management: Memory Management
• Memory is the main storage directly accessible to CPU
• data needs to be kept in memory before and after processing
• all instructions should be in memory in order to execute
• Memory management determines what is in memory to optimize CPUutilization and response time, provides a virtual view of memory for programmer
• Memory management activities:
• keeping track of which parts of memory are being used and by whom
• deciding which processes and data to move into and out of memory
• allocating and deallocating memory space as needed
17
Resource Management: File Systems
• OS provides a uniform, logical view of data storage
• file is a logical storage unit that abstracts physical properties
• files are usually organized into directories
• access control determines who can access the file
• File system management activities:
• creating and deleting files and directories
• primitives to manipulate files and directories
• mapping files onto secondary storage
• backup files onto stable (non-volatile) storage media
A special FS: /proc file system
02: Operating System Services & Structures
19
Operating System Services (User/Programmer-Visible)
• User interface
• most operating systems have a user interface (UI).
• e.g., command-Line (CLI), graphics user interface (GUI), or batch
• Program execution: from program to process
• load and execute an program in the memory
• end execution, either normally or abnormally
• I/O operations
• a running program may require I/O such as file or I/O device
• File-system manipulation
• r/w, create, delete files/directories
• search or list files and directories
• permission management
20
Operating System Services (User-Visible)
• Communications
• processes exchange information, on the same system or over a network
• via shared memory or through message passing
• Error detection
• OS needs to be constantly aware of possible errors
• errors in CPU, memory, I/O devices, programs
• it should take appropriate actions to ensure correctness and consistency
21
Operating System Services (System View)
• Resource allocation
• allocate resources for multiple users or multiple jobs running concurrently
• many types of resources: CPU, memory, file, I/O devices
• Accounting/Logging
• to keep track of which users use how much and what kinds of resources
• Protection and security
• protection provides a mechanism to control access to system resources
• access control: control access to resources
• isolation: processes should not interfere with each other
• security authenticates users and prevent invalid access to I/O devices
• a chain is only as strong as its weakest link
• protection is the mechanism, security towards the policy
22
User Interface
• GUI• Shell
• How to get a return value from in bash: $?• Operations: eq: for numeric one, == for string. “$?” -eq
“1”• How to chain multiple commands: cmd1;cmd2;cmd3• echo, touch, cat
23
System Calls
• System call is a programming interface to access the OS services• Typically written in a high-level language (C or C++)• Certain low level tasks are in assembly languages
24
Application Programming Interface
• Mostly accessed by programs via a high-level Application Programming Interface (API) rather than direct system call use• three most common APIs:
• Win32 API for Windows• POSIX API for POSIX-based systems (UNIX/Linux,
Mac OS X)• Java API for the Java virtual machine (JVM)• why use APIs rather than system calls?
– portability
25
API – System Call – OS Relationship
26
Standard C Library Example
• C program invoking printf() library call, which calls write() system call
27
System Call Parameter Passing
• Parameters are required besides the system call number• exact type and amount of information vary according to OS and call
• Three general methods to pass parameters to the OS• Register: • pass the parameters in registers• simple, but there may be more parameters than registers
• Block:• parameters stored in a memory block (or table)• address of the block passed as a parameter in a register • taken by Linux and Solaris
• Stack:• parameters placed, or pushed, onto the stack by the program • popped off the stack by the operating system
• Block and stack methods don’t limit number of parameters being passed
28
How System Call Is Handled• System call number and parameters:
• on 32bit x86 architecture, the system call identifier is stored in the EAX register, while parameters in registers EBX, ECX, EDX, ESI, EDI, EBP
• Steps:
• The application is setting up the system call number (EAX for x86) and parameters and it issues a trap instruction (int 0x80 for x86)
• The execution mode switches from user to kernel; the CPU switches to a kernel stack; the user stack and the return address to user space is saved on the kernel stack
• The kernel entry point saves registers on the kernel stack (SAVE_ALL)
• Get system call number
• The system call dispatcher identifies the system call function using the system call number
• Execute the system call function and saves the return value (to EAX for x86)
• The user space registers/stack are restored and execution is switched back to user (e.g. calling IRET)
• The user space application resumes
29
Operating System Design and Implementation
• Much variation
• Early OSes in assembly language
• Then system programming languages like Algol, PL/1
• Now C, C++
• Actually usually a mix of languages
• Lowest levels in assembly
• Main body in C
• Systems programs in C, C++, scripting languages like PERL, Python, shell scripts
• More high-level language easier to port to other hardware
• But slower
30
Operating System Structure
• Many structures:• simple structure - MS-DOS• more complex -- UNIX• layered structure - an abstraction• microkernel system structure - L4• hybrid: Mach, Minix• research system: exokernel
31
Simple Structure: MS-DOS
• No structure at all!: (1981~1994)
• written to provide the most functionality in the least space
• A typical example: MS-DOS
• Has some structures:
• its interfaces and levels of functionality are not well separated
• the kernel is not divided into modules
32
Monolithic Structure – Original UNIX
• Limited by hardware functionality, the original UNIX had limited structure
• UNIX OS consists of two separable layers• systems programs• the kernel: everything below the system-call interface and
above physical hardware• a large number of functions for one level: file systems,
CPU scheduling, memory management …
33
Microkernel System Structure
• Microkernel moves as much from the kernel (e.g., file systems) into “user” space
• Communication between user modules uses message passing
• Benefits:
• easier to extend a microkernel
• easier to port the operating system to new architectures
• more reliable (less code is running in kernel mode)
• more secure
• Detriments:
• performance overhead of user space to kernel space communication
• Examples: Minix, Mach, QNX, L4…
34
Exokernel: Motivation
• In traditional operating systems, only privileged servers and the kernel can manage system resources
• Un-trusted applications are required to interact with the hardware via some abstraction model• File systems for disk storage, virtual address spaces for
memory, etc.• But application demands vary widely!!
• An interface designed to accommodate every application must anticipate all possible needs
03: Process
36
Process Concept
• Process is a program in execution, its execution must progress in sequential fashion
• a program is static and passive, process is dynamic and active
• one program can be several processes (e.g., multiple instances of browser, or even on instance of the program)
• process can be started via GUI or command line entry of its name
• through system calls
• Process is the basic unit for resource allocation and protection
37
Process Concept
• A process has multiple parts:
• the program code, also called text section
• runtime CPU states, including program counter, registers, etc
• various types of memory:
• stack: temporary data
– e.g., function parameters, local variables, and return addresses
• data section: global variables
• heap: memory dynamically allocated during runtime
38
Process State
• As a process executes, it changes state• new: the process is being created• running: instructions are being executed• waiting/blocking: the process is waiting for
some event to occur• ready: the process is waiting to be assigned to a
processor• terminated: the process has finished execution
39
Diagram of Process State
40
Process Control Block (PCB)
• In the kernel, each process is associated with a process control block
• process number (pid)
• process state
• program counter (PC)
• CPU registers
• CPU scheduling information
• memory-management data
• accounting data
• I/O status
• Linux’s PCB is defined in struct task_struct: https://elixir.bootlin.com/linux/v5.5-rc3/source/include/linux/sched.h#L629
41
Kernel stack
• 2.4
• 2.6
42
Threads
• So far, process has a single thread of execution• Consider having multiple program counters per process
• Multiple locations can execute at once• Multiple threads of control -> threads
• Must then have storage for thread details, multiple program counters in PCB
43
Context Switch
• Context switch: the kernel switches to another process for execution
• save the state of the old process
• load the saved state for the new process
• Context-switch is overhead; CPU does no useful work while switching
• the more complex the OS and the PCB, longer the context switch
• Context-switch time depends on hardware support
• some hardware provides multiple sets of registers per CPU: multiple contexts loaded at once
44
Context Switch
45
Process Creation
• Parent process creates children processes, which, in turn create other processes, forming a tree of processes• process identified and managed via a process identifier (pid)
• Design choices:
• three possible levels of resource sharing: all, subset, none• parent and children’s address spaces• child duplicates parent address space (e.g., Linux)• child has a new program loaded into it (e.g., Windows)
• execution of parent and children• parent and children execute concurrently• parent waits until children terminate
46
Process Creation
• UNIX/Linux system calls for process creation• fork creates a new process• exec overwrites the process’ address space with
a new program• wait waits for the child(ren) to terminate
47
C Program Forking Separate Process#include <sys/types.h>#include <studio.h>#include <unistd.h>int main(){
pid_t pid;pid = fork(); /* fork another process */if (pid < 0) { /* error occurred while forking */
fprintf(stderr, "Fork Failed");return -1;
} else if (pid == 0) { /* child process */execlp("/bin/ls", "ls", NULL);
} else { /* parent process */wait (NULL);printf ("Child Complete");
}return 0;
}
48
fork() multiple times#include <sys/types.h>#include <studio.h>#include <unistd.h>int main(){
pid_t pid;pid = fork(); pid = fork();pid = fork();printf(“fork done!”);return 0;
}
49
Interprocess Communication
• Processes within a system may be independent or cooperating
• independent process: process that cannot affect or be affected by the execution of another process
• cooperating process: processes that can affect or be affected by other processes, including sharing data
• reasons for cooperating processes: information sharing, computation speedup, modularity, convenience, Security
• Cooperating processes need interprocess communication (IPC)
• Two models of IPC
• Shared memory
• Message passing
50
Communications Models
Very fast!
51
POSIX Shared Memory
• POSIX Shared Memory
• Process first creates shared memory segment
• shm_fd = shm_open(name, O CREAT | O RDWR, 0666);
• Also used to open an existing segment
• Set the size of the object: ftruncate(shm_fd, 4096);
• Use mmap() to memory-map a file pointer to the shared memory object
• Reading and writing to shared memory is done by using the pointer returned by mmap().
52
A Simple Kernel Module
Further reading: https://blog.sourcerer.io/writing-a-simple-linux-kernel-module-d9dc3762c234
insmod xxx.ko
53
Takeaway
• This whole slides are takeaways