Phones OFF Please File Systems Parminder Singh Kang Home: pkang Email: [email protected].

Phones OFF Please

File Systems

Parminder Singh Kang

Home: www.cse.dmu.ac.uk/~pkang

Email: [email protected]

Introduction

 • File systems are needed:

Because main memory is not big enough To maintain permanent copies of information

 • File system should be device independent to allow programs to use the same commands with different devices,

• Disks provide the bulk of secondary storage on which a file system is maintained.

• The advantages of using disks are;

Ability to rewrite at same place. User can access directly given block of information And it improves efficiency and performance of operations by transferring data in unit of blocks instead of byte by byte.

1 File Subsystem 

• Provides users/applications with a logical interface

Impose uniform structure on storage,

i.e. typically hierarchical directory structure

Refer to elements by meaningful names

Specify operations on storage in application terms, e.g. read a real number

•Maps logical organisation to a physical storage media

• Hides details of the physical organisation – Using device drivers allows all I/O to be treated alike

1.1 Device drivers

• Transfer from/to peripheral requires series of steps :

Check current status

Initiate status change

Request transfer

Receive notification complete

1.2 File Subsystem requirement

• Specifying Logical File characteristics

• Cataloguing/locating files

• Mapping physical to logical

• Supporting file operation

• Controlling access

1.3 File System Structure 

• I/O Control:

Consist of device drivers and interrupt handlers to transfer information between main memory and disk system. The basic function of I/O controller is to access specific location on device.

• Basic File System: Main function is to issue generic commands to appropriate device driver to read and write physical block on disk. Basic file system uses concept of physical address space. Each block is identified by numeric disk address (e.g. drive, cylinder, track, sector)

•File Organization Module:

track of file allocation used and mapping between logical and physical blocks. By knowing type of file allocation and physical address; file organization module translates logical block address to physical address.

• Logical File System:

Contains metadata (file structure information, Inode information and file control block information). All information is managed by file control block (FCB). FCB includes information about file name, inodes, permissions, location of file content etc.

Application programs

Logical File System

File-Organization module

Basic File System

I/O Control

Devices

1.4 File System Structure Implementation

• File system implementation refers to disk and memory.

• Implementation varies with operating system and file system use.

• On Disk Implementation:

Disk Label (VTOC)

Boot Control Block

Primary Super Block

Backup Super Block

Inode

• In Memory Implementation:

Contains information about each mounted partition.

In memory directory structure holds information about recently accessed directories.

Open file table contains copy if Inodes for each open file.

disk and memory file system implementation and why they needed?

2. UNIX file characteristics

• Structure

File is flat sequence of bytes UNIX imposes no structure Other O/S sometimes do

• Naming conventions

Char sequence as name Max length can vary

• Type

UNIX does not infer type from name UNIX has different file type Regular Directories Symbolic Links (Hard or Soft) Device

• Organisation

Defines whether file is accessed sequentially or randomly Both supported by current offset pointer

• Access

Define who can do what to file Record when it was last done

3. Physical Storage

• Physical storage is mainly

hard disks CD-Rom/DVD Floppy disks Magnetic tape (backup)

4. UNIX File System

• btree structured• only one tree (one root)• may be multiple disks

 i.e. uses a device independent hierarchical which is regarded as a tree:  root

user

user files bin etc dev usr

Device file systems can be attached to the tree using the program /etc/mount

• Once device is mounted files can be accessed using directory name; • the device does not have to be known, e.g.

cp /user/test.dat mytest.dat

to copy a file to the current directory.

This has the advantage that a file system can be moved to a different device without the programs, which use it needing modification.

• MS-DOS and Windows on the other hand, are not device independent, i.e. one has to use device names, e.g.

copy a:test.dat c:\user.

4.1 Types of file

• Unix has:

regular files - users programs and data, etc. directories special files - I/O devices, e.g. /dev/tty and /dev/hd1

•MS-DOS/Windows has

regular files - users programs and data, etc. Directories special files – prn: con:, com1:, etc.

4.2 File names

• Standard UNIX allows up to 14 characters in a filename with combinations of name and extension as required, e.g. test.data.

• In UNIX the extension is solely for the programmers convenience, • i.e. to identify types of files at a logical level, •e.g. a user may end all data files with. data

4.3 Using files

• two basic operation needed; read and write.• At a program level one usually has a set of language or library or systems calls to access files

4.4 Directories (only OS can write into Directory, Justify?)

• a directory is a special system file which contains details of other files• provide a logical interface to user to keep track of files.

• Simple file systems, e.g. CP/M, have one directory per device:

These become large with many users one can have name conflicts.

• Alternatives are to have one directory per user (RSX) or

many directories per user (UNIX, MS-DOS, Windows). absolute path names (from the root) e.g. on UNIX /usr1/stf/bb/public/opsys/progs/pipe1.cc path names relative to the current directory, e.g. network/programs/terminal or ../../test_data/test.data

4.5 File structure

• At the lowest (physical) level one can read/write blocks from/to a device.• At a logical level one reads/writes records; where a record would be anything from a byte (read a character) to N bytes (read a large structure).

4.6 Disk space management

• The surface of a disk is divided into a number of cylindrical tracks each of which is divided up into a number of sectors.The unit reading/writing from/to a disk is a block. (where a block may be anything from a sector to a track).

• Disk I/O transfer speed is made up from (depends upon rotation speed of disk):

start-up time (for floppies) ~ 0.5sec track seek time (time to move head to require track) ~ 5-20msec for hard disk latency (time for required sector to appear under head) ~ 0.5-5msec transfer time ~ 0.05 - 0.1 mSec for 1Kblock

 • larger the block size the faster the transfer rate will be but the more space is wasted.• a disk cache can improve transfer rates significantly.

4.7 Disk Partitions

• To assist in the overall organisation of file system large disks can be split into logical disks by partitions.

Boot block (Master boot record) Partition 1 Partition 2 etc

• Each partition can be either “raw” containing no file system or “cooked” containing file system.

• Raw disk is used when no file system is appropriate.

For example; UNIX swap space uses raw partition, as it uses its own format and does not use a file system.

• Boot information can be stored in a separate partition. (Why?)

• Root Partition, Contains operating system kernel and system files are mounted at boot time.

5 Unix File storage

 • A file system needs to keep track of

where every file is stored details of each file where next block of file is which blocks are free which blocks are in use

5.1 Allocating Space to Files

• allocate a contiguous sequence of blocks

unused small areas problem with file growth

• Linked list of blocks

problem with random access

• index blocks (UNIX uses a version of this)

5.2 UNIX file storage structure

• only O/S can write to a directory• each entry has inode number and name 

┌───────────────┬────────── ─┐ │ i-node number │ file name │ └───────────────┴──────── ───┘

•command df (disk free) will tell you how many i-nodes are free.

• An i-node contains the following information on the file:

file mode (indicates type of file - normal file, special file, etc.) number of links to file (e.g. from other directories) owners user id owners group id access permissions for each user type, e.g. read, write, and execute file size in characters time created, last accessed, last modified location of first 10 blocks (if file < 10 blocks contains address of the file) single indexed, double indexed, triple indexed

5.3 Locating a file

• To locate a file’s data requires the following loop

Inode-data-inode-data….

• Note:

Root i-node is always i-node 2 Each directory has entries for . (current directory) and . . (parent directory)

root dir user dir staff dir sam dir ┌───────────┐ ┌────►┌───────────┐ ┌────►┌───────────┐ ┌────►┌───────────┐ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ ├───────────┤ │ ├───────────┤ │ ├───────────┤ │ ├───────────┤ ┌─┤ user │ │ ┌─┤ staff │ │ ┌─┤ sam │ │ ┌─┤ x.data │ │ ├───────────┤ │ │ ├───────────┤ │ │ ├───────────┤ │ │ ├───────────┤ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ └───────────┘ │ │ └───────────┘ │ │ └───────────┘ │ │ └───────────┘ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ user i-node │ │ staff i-node │ │ sam i-node │ │ x.data i-node │ ┌───────────┐ │ │ ┌───────────┐ │ │ ┌───────────┐ │ │ ┌───────────┐ └►│ │ │ └►│ │ │ └►│ │ │ └►│ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ location ├──┘ │ location ├──┘ │ location ├──┘ │ location ├──► file │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ └─────────┘ └─────────┘ └─────────┘ └─────────┘

• The sequence of events is:

the root directory is searched for the user directory file entry

the i-node number is extracted and the location of the user directory found from the i-node

the user directory is searched for the staff directory file entry

the i-node number is extracted and the location of the staff directory found from the i-node

the staff directory is searched for the sam directory file entry

the i-node number is extracted and the location of the sam directory found from the i-node

the sam directory is searched for the x.data file entry

the i-node number is extracted and the locations of the x.data file found from the i-node

5.4 The i-node and data addressing

• Addresses of data blocks of file stored in inode:

10 direct pointers 1 single indirect pointer to a block of addresses 1 double indirect pointer to a block, which contains pointers to blocks of addresses. Some systems have a third level pointer.

 5.4.1 how big should the pointers be?

• Early versions of Unix used 16 bit pointers.

• With 1K blocks this meant you were limited to 65Mbyte as the largest disk.

• Later versions of Unix used 32 bit pointers and 4K (or 8K blocks)

• This gives a maximum file/disk size of 4K x 4Gbytes = 16Tbytes.

• Modern versions of Unix are using 64 bit pointers. 

5.4.2 File Operations

• Opening an existing file • Creating a file

• Reading from a file

• Writing to a file

• Closing a file

• Deleting a file

• Changing access permissions

• Renaming a file

5.4.3 Links in Unix

• Each file has one i-node but may have many directory entries

• Each name entry is a link to an i-node

• links may be hard or soft 

• Hard links

each directory entry points directly at same i-node the i-node maintains count of links to it this only operates on a single device

• Soft Links

Special file containing the path to the target file Separate i-node Can span devices

5.5 Efficiency and Performance

• Unix uses a buffer cache to hold large block of memory

• As blocks are read they are stored in the cache, Reading next block can go on while current block is being processed

• If cache is sufficiently large or not?

• further improvement using Delayed write (can be problem if system crashes) 

i-nodes written back immediately Written data blocks are flushed after a few sec’s written to disk but marked delayed write. Block can be modified further before it reaches the head of the list when it is then written. Useful if file is deleted before block written.

 • Eventually cache fills

Block that was accessed longest ago is flushed Read ahead improves efficiency

6 Log Structured File Systems

• CPU’s are getting faster• Memory is getting faster• Disks are getting bigger, but not much faster

 This creates bottleneck in the file system - especially for large file servers (Solution?) Log structured file systems

• Most accesses are to the cache

• writes slow the system (small quantity of data)

• Disks operate most efficiently with large writes (one or more tracks) therefore;

Collect writes together and write them all at once as a log record. If record is big (~ 1Mbyte) disk will operate efficiently. Record contains i-nodes, directories, data mixed up. Need a table to keep track of where every i-node is. Keep this in memory and on disk.

Note:• Much more complex to administer.• Eventually disk fills. • Have a garbage collection process which goes through log records and compacts them.Disk operates like a very large circular buffer.

7 DOS/Windows

• The file system has

Boot sector FAT Root directory and Data blocks

• The directory entry contains all the details about the file including the name. • It has a pointer to the first block.• To find the next block the system uses a FAT (File Allocation table).• The FAT is a large one dimensional array. There is an entry for each block which contains either

The address of the next blockor End Of File marker