Case Study – The Linux OS Part II Mostly taken from: Silberschatz – Chapter 20 Nutt – Chapter 20
Case Study – The Linux OSPart II
Mostly taken from: Silberschatz – Chapter 20
Nutt – Chapter 20
Linux Operating System - Part -II - Slide 2 Fakhar Lodhi
Virtual Memory
The VM system maintains the address space visible to each process: It creates pages of virtual memory on demand, and manages the loading of those pages from disk or their swapping back out to disk as required
The VM manager maintains two separate views of a process’s address space: A logical view describing instructions concerning the layout of
the address space The address space consists of a set of nonoverlapping
regions, each representing a continuous, page-aligned subset of the address space
A physical view of each address space which is stored in the hardware page tables for the process
Linux Operating System - Part -II - Slide 3 Fakhar Lodhi
Virtual Memory (Cont.)
On executing a new program, the process is given a new, completely empty virtual-address space The program-loading routines populate the address space
Creating a new process with fork involves creating a complete copy of the existing process’s virtual address space The kernel copies the parent process’s VMA descriptors, then
creates a new set of page tables for the child The parent’s page tables are copied directly into the child’s,
with the reference count of each page covered being incremented
After the fork, the parent and child share the same physical pages of memory in their address spaces
Copy-on-write mechanism
Linux Operating System - Part -II - Slide 4 Fakhar Lodhi
Virtual Memory (Cont.)
The VM paging system relocates pages of memory from physical memory out to disk when the memory is needed for something else
The VM paging system can be divided into two sections: The pageout-policy algorithm decides which pages to
write out to disk, and when The paging mechanism actually carries out the transfer,
and pages data back into physical memory as needed
Linux Operating System - Part -II - Slide 5 Fakhar Lodhi
Virtual Memory (Cont.)
The Linux kernel reserves a constant, architecture-dependent region of the virtual address space of every process for its own internal use
This kernel virtual-memory area contains two regions: A static area that contains page table references to every
available physical page of memory in the system, so that there is a simple translation from physical to virtual addresses when running kernel code
The reminder of the reserved section is not reserved for any specific purpose; its page-table entries can be modified to point to any other areas of memory
Linux Operating System - Part -II - Slide 6 Fakhar Lodhi
Executing and Loading User Programs
Linux maintains a table of functions for loading programs; it gives each function the opportunity to try loading the given file when an exec system call is made
The registration of multiple loader routines allows Linux to support both the Executable and Linking Format (ELF) and a.out binary formats
An ELF-format binary file consists of a header followed by several page-aligned sections The ELF loader works by reading the header and
mapping the sections of the file into separate regions of virtual memory
Initially, binary-file pages are mapped into virtual memory Only when a program tries to access a given page will a
page fault result in that page being loaded into physical memory
Linux Operating System - Part -II - Slide 7 Fakhar Lodhi
Mapping of Programs into Memory
Points to the current extent
Top of user mode
Write protected
Variable size region
Fixed size region
Linux Operating System - Part -II - Slide 8 Fakhar Lodhi
Static and Dynamic Linking
A program whose necessary library functions are embedded directly in the program’s executable binary file is statically linked to its libraries
The main disadvantage of static linkage is that every program generated must contain copies of exactly the same common system library functions
Dynamic linking is more efficient in terms of both physical memory and disk-space usage because it loads the system libraries into memory only once
Linux Operating System - Part -II - Slide 9 Fakhar Lodhi
Linux File Manager
Defines a single internal view of files that can be used to read/write files onto a storage device using a variety of file managers
A file is viewed as a named byte stream that can be saved in the secondary storage
Supports directory and file operations
Implements asynchronous read/write operations
I/O Buffers
Device Driver
StorageDevice
File
Manager
Write Read
Linux Operating System - Part -II - Slide 10 Fakhar Lodhi
Linux File Manager – VFS Virtual File System (VFS) Implements FS independent operations VFS switch – provides the API to user space programs and
establishes an internal interface used by different file system translators
The VFS inode contains details about the file.
Linux File Manager System Call Interface
VFS Switch
MS-DOS MINIX Ext2 /proc
Linux Operating System - Part -II - Slide 11 Fakhar Lodhi
File Systems
To the user, Linux’s file system appears as a hierarchical directory tree obeying UNIX semantics
Internally, the kernel hides implementation details and manages the multiple different file systems via an abstraction layer, that is, the virtual file system (VFS)
The Linux VFS is designed around object-oriented principles and is composed of two components: A set of definitions that define what a file object is allowed to
look like The inode-object and the file-object structures represent
individual files the file system object represents an entire file system
A layer of software to manipulate those objects
Linux Operating System - Part -II - Slide 12 Fakhar Lodhi
The Linux Ext2 File System
Ext2 performs its allocations in smaller units The default block size on ext2fs is 1Kb, although 2Kb and
4Kb blocks are also supported Ext2 uses allocation policies designed to place
logically adjacent blocks of a file into physically adjacent blocks on disk, so that it can submit an I/O request for several disk blocks as a single operation
Linux Operating System - Part -II - Slide 13 Fakhar Lodhi
Two parts Partition into multiple block groups Each group belongs to a single cylinder of the physical disk May be Different sizes of groups
ext2 selects the block group for the file tries to keep the related information in the same group block for data block it tries to choose the same block group in which
the file’s inode has been allocated for inodes it selects the same group as the file’s parent
directory directory files are not kept together to spread out disk load within the block groups, it tries to keep the allocation
physically contiguous maintains a bitmap of all free blocks in a block group
The Linux Ext2 File System
Linux Operating System - Part -II - Slide 14 Fakhar Lodhi
when allocating the first block for a new file, it starts searching for a free block
for extending the file it continues the search from the block most recently used
the search is performed in two stages first it searches for an entire free byte in the bitmap – aims to allocate
disk blocks in chunks (at least 8 where possible). if it fails then it searches for any free bit
once a free block has been identified, the search is extended backwards until an allocated block is encountered
when a free byte is found in the bitmap. this backward extension prevents leaving a hole between the most recently allocated block in the previous non-zero byte and the zero byte found.
extends the allocation forward for up to eight blocks and preallocates these extra blocks to the file to reduce fragmentation and save disk allocation time.
preallocated blocks are returned to free space when the file is closed.
The Linux Ext2 File System
Linux Operating System - Part -II - Slide 15 Fakhar Lodhi
Ext2 Block-Allocation Policies
if we can find any free blocks sufficiently near the start of the search then we allocate them no matter how fragmented
Linux Operating System - Part -II - Slide 16 Fakhar Lodhi
Ext2 internals divided into group Super block, Group descriptor, Block descriptor, inode
bitmap, inode table, data blocks Different block size length
1KB, 2KB, 4KB, 8KB
Root has 5% of space reserved Enable filesystem recovery
The Linux Ext2 File System (cont.)
Linux Operating System - Part -II - Slide 17 Fakhar Lodhi
The inode contains important information regarding the file Kernel keeps the inodes of open files in memory A structure that contains file’s description:
Type Access rights Owners Timestamps Size 15 Pointers to data blocks
First twelve points to data block. Thirteenth is called indirect pointer Fourteenth is called doubly indirect pointer Fifteenth is called triple indirect pointer
The Linux Ext2 File System - inode
Linux Operating System - Part -II - Slide 18 Fakhar Lodhi
inode structure
File Info
INDDINDTIND
pointers to data blocks
indirect pointer
double indirect pointertriple indirect pointer
Linux Operating System - Part -II - Slide 19 Fakhar Lodhi
Directories
A directory is just a special type of file These are structured in a tree hierarchy Each can contain both files and directories Special user-functions for directory access Each dentry contains filename + inode-no Kernel searches the direrctory tree translates a pathname to an inode-number
Linux Operating System - Part -II - Slide 20 Fakhar Lodhi
Directory diagram
Inode TableDirectory
i1
i2
i3
i4
name1
name2
name3
name4
Linux Operating System - Part -II - Slide 21 Fakhar Lodhi
Links
Multiple names can point to same inode The inode keeps track of how many links If a file gets deleted, the inode’s link-count gets
decremented by the kernel File is deallocated if link-count reaches 0 This type of linkage is called a ‘hard’ link Hard links may exist only within a single FS Hard links cannot point to directories (cycles)
Linux Operating System - Part -II - Slide 22 Fakhar Lodhi
The Linux /proc File System
The /proc file system does not store data, rather, its contents are computed on demand according to user file I/O requests
proc must implement a directory structure, and the file contents within; it must then define a unique and persistent inode number for each directory and files it contains It uses this inode number to identify just what operation is
required when a user tries to read from a particular file inode or perform a lookup in a particular directory inode
When data is read from one of these files, proc collects the appropriate information, formats it into text form and places it into the requesting process’s read buffer
Linux Operating System - Part -II - Slide 23 Fakhar Lodhi
The /proc file system is a direct reflection of the system kept in memory and represented in a hierarchal manner.
The effort of the /proc file system is to provide an easy way to view kernel and information about currently running processes.
As a result, some commands (ps for example) read /proc directly to get information about the state of the system.
The premise behind /proc is to provide such information in a readable manner instead of having to invoke difficult to understand system calls.
The Linux /proc File System
Linux Operating System - Part -II - Slide 24 Fakhar Lodhi
Input and Output Linux splits all devices into three classes:
block devices allow random access to completely independent, fixed size blocks of data
character devices include most other devices; they don’t need to support the functionality of regular files
network devices are interfaced via the kernel’s networking subsystem User cannot directly transfer data to network devices
All device drivers appear as normal files
The Linux device-oriented file system accesses disk storage through two caches:
Data is cached in the page cache, which is unified with the virtual memory system
Metadata is cached in the buffer cache, a separate cache indexed by the physical disk block
Linux Operating System - Part -II - Slide 25 Fakhar Lodhi
Device-Driver Block Structure
Provide the main interface to all disk devices in a system The block buffer cache
serves two main purposes: it acts as a pool of buffers for active I/O it serves as a cache for completed I/O
The request manager manages the reading and writing of buffer contents to and from a block device driver
Block Devices
Linux Operating System - Part -II - Slide 26 Fakhar Lodhi
Device-Driver Block Structure
A device driver which does not offer random access to fixed blocks of data A character device driver must register a set of functions which implement the
driver’s various file I/O operations The kernel performs almost no preprocessing of a file read or write request to
a character device, but simply passes on the request to the device The main exception to this rule is the special subset of character device
drivers which implement terminal devices, for which the kernel maintains a standard interface
Character Devices
Linux Operating System - Part -II - Slide 27 Fakhar Lodhi
Device-Driver Block Structure
Networking is a key area of functionality for Linux. It supports the standard Internet protocols for UNIX to UNIX
communications It also implements protocols native to nonUNIX operating
systems, in particular, protocols used on PC networks, such as Appletalk and IPX
Network Devices
Linux Operating System - Part -II - Slide 28 Fakhar Lodhi
Device-Driver Block Structure
Internally, networking in the Linux kernel is implemented by three layers of software: The socket interface Protocol drivers Network device drivers
Network Devices
Linux Operating System - Part -II - Slide 29 Fakhar Lodhi
Device-Driver Block Structure
The most important set of protocols in the Linux networking system is the internet protocol suite It implements routing between different hosts anywhere
on the network On top of the routing protocol are built the UDP, TCP
and ICMP protocols
Network Devices
Linux Operating System - Part -II - Slide 30 Fakhar Lodhi
Interprocess Communication
Linux informs processes that an event has occurred via signals
There is a limited number of signals, and they cannot carry information: Only the fact that a signal occurred is available to a process
Linux Operating System - Part -II - Slide 31 Fakhar Lodhi
Passing Data Between Processes
The pipe mechanism allows a child process to inherit a communication channel to its parent, data written to one end of the pipe can be read a the other
Each pipe has a pair of wait queues to synchronize readers and writers
Linux Operating System - Part -II - Slide 32 Fakhar Lodhi
Shared Memory Object
Shared memory offers an extremely fast way of communicating; any data written by one process to a shared memory region can be read immediately by any other process that has mapped that region into its address space
Linux Operating System - Part -II - Slide 33 Fakhar Lodhi
Security – Pluggable Authentication Module (PAM)
PAM provides a way to develop programs that are independent of authentication scheme.
By using PAM, multiple authentication technologies, such as RSA, DCE, Kerberos, smart card and S/Key, can be added without changing any of the login services, thereby preserving existing system environments.
Which authentication module is to be attached is dependent upon the local system setup and is at the discretion of the local system administrator.
PAM is based on a shared library that can be used by any system component that needs to authenticate users
Linux Operating System - Part -II - Slide 34 Fakhar Lodhi
Security – Pluggable Authentication Module (PAM)
Access control under UNIX systems, including Linux, is performed through the use of unique numeric identifiers (uid and gid)
Access control is performed by assigning objects a protections mask, which specifies which access modes—read, write, or execute—are to be granted to processes with owner, group, or world access
End of Linux Overview