File Systems - Cornell CS

FileSystems

CS4410,Opera4ngSystems

Fall2016CornellUniversity

RachitAgarwalAnneBracy

See:Ch13inOSPPtextbook

TheslidesaretheproductofmanyroundsofteachingCS4410byProfessorsSirer,Bracy,Agarwal,George,andVanRenesse.

2

Long-termInformaLonStorageNeeds•largeamountsofinforma4on•informa4onmustsurviveprocesses•needconcurrentaccesstomul4pleprocesses

SoluLon•Storeinforma4onondisksinunitscalledfiles•persistent,onlyownercandelete•managedbytheOS

FileSystems:HowtheOSmanagesfiles!

FileSystems101

ChallengesforFileSystemDesigners

3

• Performance:despitelimita4onsofdisks‣leveragespa4allocality

• Flexibility:needjacks-of-all-trades,notjustFSforX

• Persistence:maintain/updateuserdata+internaldatastructuresonpersistentstoragedevices

•Reliability:muststoredataforlongperiodsof4me,despitecrashesormalfunc4ons

Firstthingsfirst:NametheFile!

4

1.Filesareabstractedunitofinforma4on2.Don’tcareexactlywhereondiskthefileis

➜Fileshavehumanreadablenames•filegivennameuponcrea4on•usethenametoaccessthefile

Name+Extension

5

NamingConven:ons• OSdependentWindowsnotcasesensi4ve,UNIXis• Usuallyokupto255characters

FileExtensions•AlsoOSdependentWindows:aZachesmeaningtoextensions

associatesapplica4onstoextensionsUNIX:extensionsnotenforcedbyOS

- Someapplica4onsmightinsistuponthem(e.g.,.c,.h,.o,.s,etc.forCcompiler)

Directory

6

Mapshumanreadablenametofilenumber

directory indexstructure

StorageBlock

FileNumber871

FileName

“foo.txt”music 320work 219foo.txt 871

PathNames

7

•Absolute:pathoffilefromtherootdirectorye.g.,/home/pat/projects/test.c•Rela4ve:pathfromthecurrentworkingdirectory(currentworkdirstoredinprocess’PCB)2specialentriesineachUNIXdirectory:

“.”currentdir“..”forparent

Toaccessafile:• Gotothefolderwherefileresides—OR—• Specifythepathwherethefileis

PathsinacLon!

8

music 320work 219foo.txt 871

File 830 ˝/home/tom˝

mike 682ada 818tom 830

File 158 ˝/home˝

File 871 ˝/home/tom/foo.txt˝

bin 737usr 924home 158

File 2 ˝/˝

The quick brown fox jumped over the lazy dog.

Example:/home/tom/foo.txt

allfiles

ImplemenLngDirectories

9

Whenafileisopened,OSusespathnametofinddir•Directoryhasinforma4onaboutthefile’sdiskblocks•DirectoryalsohasaZributesofeachfileDirectory:mapASCIIfilenametofileaZributes&loca4on2op4ons:entrieshaveallaZributes,orpointtofileI-node

FileSystemLayout

10

FileSystemisstoredondisks• diskisdividedinto1ormoreparLLons• Sector0ofdiskcalledMasterBootRecord• endofMBR:par44ontable(par44ons’start&endaddrs)

Firstblockofeachpar44onhasbootblock• loadedbyMBRandexecutedonboot

enLrediskPARTITION #4PARTITION #2PARTITION #1 PARTITION #3

PARTITION TABLE

MBR

Root DirFree Space MgmtBOOT BLOCK I-NodesSUPERBLOCK Files & Directories

StoringFiles

11

Filescanbeallocatedindifferentways:•Con4guousalloca4on

Allbytestogether,inorder•LinkedStructure

Eachblockpointstothenextblock•IndexedStructure

Indexblockpointstomanyotherblocks

Whichisbest?Forsequen4alaccess?Randomaccess?Largefiles?Smallfiles?Mixed?

ConLguousAllocaLon

12

Allbytestogether,inorder+Simple:

staterequiredperfile:startblock&size+Performance:

en4refilecanbereadwithoneseek–Fragmenta4on

externalisbiggerproblem–Usability:

userneedstoknowsizeoffile

UsedinCD-ROMs,DVDs

file1 file2 file3 file4 file5

13

MicrosoYFileAllocaLonTable•originally:MS-DOS,earlyversionofWindows• today:s4llwidelyused(e.g.,CD-ROMs,thumbdrives,cameracards)

Filetable:• Linearmapofallblocksondisk• Eachfilealinkedlistofblocks

CaseStudy#1:FileAlloca6onTable(FAT)

[late70’s]

data

next

data

next

data

next

decoupled

physically

32bitentries

FAT BLOCKS

FATFileSystem

14

Data BlocksFAT

0123456789

1011121314151617181920

File 9 Block 3

File 9

File 12

File 12 Block 1File 9 Block 4

File 9 Block 0File 9 Block 1File 9 Block 2File 12 Block 0

•1entryperblock•EOFforlastblock•0indicatesfreeblock•usuallyusesasimplealloca4onstrategy(e.g.next-fit)•directoryentrymapsnametoFATindex

Directorybart 9

maggie 12

0

0

0

EOFEOF

}

FATDirectoryStructure

15

Folder:afilewith32-byteentriesEachEntry:• 8bytename+3byteextension(ASCII)• crea4ondateand4me• lastmodifica4ondateand4me• firstblockinthefile(indexintoFAT)• sizeofthefile• LongandUnicodefilenamestakeupmul4pleentries

music 320work 219foo.txt 871

HowGoodisFAT?

16

+Simple• staterequiredperfile:startblockonly

+Widelysupported+Noexternalfragmenta4on+allofblockusedfordata

-Poorlocality-Manyfileseeksunlessen4reFATinmemory-Poorrandomaccess-Limitedmetadata-Limitedaccesscontrol-Nosupportforhardlinks-Limita4onsonvolumeandfilesize-Nosupportforreliabilitytechniques

17

UNIXFastFileSystem

•inodetable- AnalogoustoFATtable

• inode- Metadata- 12datapointers- 3indirectpointers

CaseStudy#2:FastFileSystem(FSS)[early80’s]Inode Array

File Metadata

Indirect PointerDbl. Indirect Ptr.Tripl. Indirect Ptr.

InodeData

BlocksIndirectBlocks

DoubleIndirectBlocks

TripleIndirectBlocks

DPDirect Pointer

DPDPDPDPDPDPDPDPDP

Direct Pointer

FFSSuperblock

18

Iden4fiesfilesystem’skeyparameters:•type•blocksize•inodearrayloca4onandsize(oranalogousstructureforotherFSs)

blocknumber 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

blocks:

Remainingblocksi-nodeblocks

superblock

FFS:IndexStructures

19

Inode Array

File Metadata

Indirect PointerDbl. Indirect Ptr.Tripl. Indirect Ptr.

InodeData

BlocksIndirectBlocks

DoubleIndirectBlocks

TripleIndirectBlocks

DPDirect Pointer

DPDPDPDPDPDPDPDPDP

Direct Pointer

• Type- ordinaryfile- directory- symboliclink- specialdevice

• Sizeofthefile(in#bytes)• #linkstothei-node• Owner(useridandgroupid)• Protec4onbits• Times- crea4on,lastaccessed,last

modified

WhatelseisinanInode?

20

Inode Array

File Metadata

Indirect PointerDbl. Indirect Ptr.Tripl. Indirect Ptr.

InodeData

BlocksIndirectBlocks

DoubleIndirectBlocks

TripleIndirectBlocks

DPDirect Pointer

DPDPDPDPDPDPDPDPDP

Direct Pointer

File Metadata

AKA file control block (FCB)

FFS:IndexStructures

21

Inode Array

File Metadata

Indirect PointerDbl. Indirect Ptr.Tripl. Indirect Ptr.

InodeData

BlocksIndirectBlocks

DoubleIndirectBlocks

TripleIndirectBlocks

DPDirect Pointer

DPDPDPDPDPDPDPDPDP

Direct Pointer

12

Assumeblocksare4K&blockreferences4bytes

12x4K=48Kdirectlyreachablefromtheinode

2(nx10)x4K=withnlevelsofindirecLon

1K

1K

1K

1K 1K

1K

1K

1K

1K

1K n=1:4MB

n=2:4GB

n=3:4TB

4CharacterisLcsofFFS

22

1.TreeStructure•efficientlyfindanyblockofafile

2.HighDegree(orfanout)•minimizesnumberofseeks•supportssequen4alreads&writes

3.FixedStructure•implementa4onsimplicity

4.Asymmetric•notalldatablocksareatthesamelevel•supportslarge•smallfilesdon’tpaylargeoverheads

SmallFilesinFFS

23

Inode Array

File Metadata

NILNILNILNILNILNILNILNILNILNILNIL

InodeData

Blocks

DPDP

Direct Pointer

Direct Pointer

Whatiffixed3levelsinstead?•4KBfileconsumes~16KB(4KBdata+3levelsof4KBindirectblocks+inode)

•readingfilewouldrequirereading5blockstotraversetree

allblocksreachedvia

directpointers

SparseFilesinFFS

24

File Metadata

Dbl. Indirect Ptr.

Inode

DataBlocks

IndirectBlocks

DoubleIndirectBlocks

TripleIndirectBlocks

NILNILNILNILNIL

Direct PointerNILNILNIL

NILNILNILNIL

NIL

Filesize(ls-lgGh):1.1GBSpaceconsumed(du-hs):16KB

Readfromhole:0-filledbuffercreatedWritetohole:storageblocksfordata+requiredindirectblocksallocated

2x4KBbocks:1@offset01@offset230

FFSDirectoryStructure

25

Originally:arrayof16byteentries•14bytefilename•2bytei-nodenumber

Now:linkedlists.Eachentrycontains:•4-byteinodenumber• Lengthofname•Name(UTF8orsomeotherUnicodeencoding)

Firstentryis“.”,pointstoselfSecondentryis“..”,pointstoparentinode

music 320work 219foo.txt 871

FFS:Stepstoreading/foo/bar/baz

26

(1) inode#2(rootalwayshasinumber2),findroot’sblocknum(912)(2) rootdirectory(inblock912),findfoo’sinumber(31)(3) inode#31,findfoo’sblocknum(194)(4) foo(inblock194),findbar’sinumber(73)(5) inode#73,findbar’sblocknum(991)(6) bar(inblock991),findbaz’sinumber(40)(7) inode#40,finddatablocks(302,913,301)(8) datablocks(302,913,301)

194

…

301 302

…

912 913

…

991

baz40ni80nit87

n d Iremember.I doandI

bin47foo31usr98

fie23far81bar73

understand.

I heara n d Iforget.Iseea

912 194 302913301

991

2 31 40 73inodes blocks

1 23 4

Read&Open:

8 8857 7

Cachingallowsfirstfewstepsto

beskipped