On Windows File Access Modes : A Performance Study Jalil Boukhobza & Claude Timsit [email protected] laboratory Versailles Saint Quentin University.

On Windows File Access Modes :

A Performance Study

Jalil Boukhobza & Claude Timsit

[email protected]

laboratorylaboratoryVersailles Saint Quentin University Versailles Saint Quentin University (France)(France)

01/04/2005 PRiSM / UVSQ 2

Outline

1. Motivations

2. Windows file access strategies

3. Read performance analysis

4. Write performance analysis

5. Conclusions and guidelines

6. Perspectives

01/04/2005 PRiSM / UVSQ 3

Motivations

Windows I/O systems very used BUT very poorly studied !

CreateFile() function Many file access modes different caching algorithms big performance variations

Disk subsystems built independently from OS Interaction with the OS are not easily predictable

How does the performance vary with the access strategy and the storage subsystem and why ?

01/04/2005 PRiSM / UVSQ 4

Previous work

Riedel et al., Microsoft (1998) : global performance measurements for some architectures (operating systems, disks, RAID configurations, etc.)

Chung et al., Microsoft (2000) : continuation of the first one with more recent architectures.

Not all the access modes have been studied and

performance fluctuations have not been explained

01/04/2005 PRiSM / UVSQ 5

Windows file access strategies

Different file access modes in the CreateFile() function: Without using the file system

cache: no buffer mode Using the file system cache:

sequential, normal, write through modes.

Achieved experiments : accessing sequential blocks with different I/O strategies (access modes and request sizes)

File System DriverFile System DriverFile System DriverFile System Driver

Storage Device Storage Device DriverDriver

Storage Device Storage Device DriverDriver

Storage Storage DeviceDevice

Storage Storage DeviceDevice

Cache ManagerCache ManagerCache ManagerCache Manager

Virtual Memory Virtual Memory ManagerManager

Virtual Memory Virtual Memory ManagerManager

I/O requestI/O requestFastIOFastIO

Page faultPage fault

01/04/2005 PRiSM / UVSQ 6

Read performance analysis

Read transfer rates for different modes

0

5

10

15

20

25

30

35

40

45

50

0 100 200 300 400 500 600

Request sizes (KB)

Tra

nsfe

r ra

tes (

MB

/sec)

1- No buffer 2- Normal 3- Sequential

1

2

3

Read transfer rates for different modes

0

1

2

3

4

5

6

7

8

9

10

0 100 200 300 400 500 600 700Request sizes (KB)

Read

tra

nsfe

r ra

tes (

MB

/sec)

normal mode sequential mode no buffer mode

Pentium IV 1.8GHz / 256MB RAM

Win 2K professional SP3 / FAT32&NTFS

Disk: Fujitsu Man3184mp / Ultra SCSI / 18.4 GB / 8MB cache / 10025 Rpm

seek: min: 0.4ms, avg 4.5 ms, max : 11ms track size : 221KB ~ 370KB

Transfer rate int: 84MB/sec,

ext : 160MB/sec

Pentium III 500 MHz / 192MB RAM

Win 2K professional SP2 / FAT32&NTFS

Disk: Toshiba MK6012MAP/ ATA 4 / 6GB / 1MB cache / 4200 rpm

seek: min: 3ms, avg 13 ms, max : 24ms Track size : unknown

Transfer rate int: 24Mo/sec,

ext : 66.7Mo/sec

01/04/2005 PRiSM / UVSQ 7

No buffer mode response time for 128kb requests

0

5

10

15

20

25

30

0 20 40 60 80 100

Request number

Resp

on

se t

ime (

ms)

KB

Response time measurements"no buffer" mode read of 64KB

0

0,5

1

1,5

2

2,5

0 5 10 15 20 25 30

Request number

Res

po

nse

tim

e (m

s)

2

1

"No buffer" mode read

0

2

4

6

8

10

12

14

0 10 20 30 40

Request number

Res

po

nse

tim

e(m

s) 192kb

256kb

320kb

448kb

512kb

384kb

s

Sequential mode response times

0

2

4

6

8

10

12

14

0 10 20 30 40Request number

Res

po

nse

tim

e (m

s)512kb

448kb

384kb

256kb

128kb

192kb

320ko

01/04/2005 PRiSM / UVSQ 8

Normal mode response times

Normal mode read

0

5

10

15

20

25

30

0 10 20 30 40 50Request number

Resp

on

se t

imes (

ms)

64kb

320kb

Periodic response times They depend on request

sizes. They are due to :

File system prefetching algorithms

Disk prefetching algorithms Interactions between them

Normal mode read

0

5

10

15

20

25

30

35

0 10 20 30 40 50Request number

Re

sp

on

se

tim

e (

ms

)

128kb

512kb

01/04/2005 PRiSM / UVSQ 9

Normal mode read

0

2

4

6

8

10

12

14

0 10 20 30 40 50Request number

Res

pons

e tim

es (m

s)

64kb

Example of normal mode read(Fujitsu disk)

Response time (ms)

B1 B3

B2 B5

B4 +B5,B6,B7,B8,B9,B10,B11

B1 B3

B2

B6

B5

B7

B8

B9

B1

0 B1

1

B1

0

B9

B8

B7

B6

B1

1B4 +B5,B6,B7,B8,B9,B10,B11

Block1

Block2

Block3

1

2

3

Optimal access

Normal mode access

Block1

Block2

Block3

1

2

3

Disk prefetching

4

System process

Application process

Wait

System prefetching

Disk prefetching

7.3 7.2 0.09 12.7 0.8

1.5

0.12 0.7

0.8

1.3

1.1

Application

01/04/2005 PRiSM / UVSQ 10

Normal mode read operation 64KB requests

0

5

10

15

20

25

0 10 20 30 40 50 60 70

Request number

Res

pons

e tim

e (m

s)

Example of normal mode read(Toshiba)

Response times for one period:t1 = 8,5mst2 = 15,6mst3 = 0,16mst4 = 16,9ms

The tested architecture characteristics:

Disk cache throughput ~30MB/sec

Memory throughput : ~400MB/sec

Track revolution time: 14,28ms

Seek time track/track: 3.5ms

Track size : 182KB

Block1

Block2

Block3

3Optimal access Normal mode access

Block1

Block2

Block3

1

2

3

Disk prefetching

t1:The application requests the block 1 t1 = Reading of a 64KB block + track/track seek = 64 * 14.28/182 + 3.5 = 8.5ms

t2:The system asks for the block 3 (prefetching), then the application asks for the block 2:t2 = reading of the block 2 on the disk (local prefetching) + reading of the block 3 on the disk (system prefetching of block 3) + reading of block 2 from the disk cache (disk prefetching)

= (64 * 14.28/182) + (64 * 14.28/182+ 3.5) + (64*1000/(1024*30)) = ~ 5 + 8,5 + 2,1 = 15,6 ms

t3:Memory copy of block 3 prefetched by the system.t3 = 64*1000/(1024*400) = ~ 0.16ms

t4:Reading time of the block 4: repositioning of the disk head on the first sector of the block then reading :t4 = ((14,28 – (reading of the block 2 from the disk cache + memory copy of block 3)) + 5) = 14,28 – (2,1 + 0,16) + 5 = ~ 16,9 ms.

01/04/2005 PRiSM / UVSQ 11

Conclusions : read operations Periodic response times System prefetching does not depend on file systems (NTFS & FAT32) but

on access modes Interactions between system and disk prefetching influence very much the

performances No buffer mode :

Good performances / except some request sizes No file system cache level

Sequential mode : As good as no buffer mode for request sizes multiple of 64KB and less than

160KB File system cache level activated

Normal (default) mode : Performance drop by a factor 2-3 A different file system cache algorithm (compared to sequential mode) Different disk prefetching reactions

01/04/2005 PRiSM / UVSQ 12

Write operation analysis

"No buffer" Writing

0

2

4

6

8

10

12

0 5 10 15 20 25 30

Request number

Res

pons

e tim

e (m

s) 64KB

128KB

256KB

384KB

512KB

No buffer write operation / 64KB requests

0

1

2

3

4

5

6

7

0 10 20 30 40 50 60 70 80 90

Request number

Res

po

nse

tim

e (m

s)

Flush of data that were on the cache segment to be used

Copy of data on the disk cache segment

Once the segment is filled or/and the disk head positioned, the copy of data from the cache to the disk begins

Disk driver

File system driver

Win32 function

Disk cache

Lazy write algorithm

01/04/2005 PRiSM / UVSQ 13

normal mode write 320KB

0

5

10

15

20

25

0 5 10 15 20 25 30

Request number

Res

pons

e tim

e (m

s)

normal mode write of 384KB

0

50

100

150

200

250

300

0 20 40 60 80 100 120

Request number

Res

pons

e tim

e(m

s)

normal mode write 320KB

0

50

100

150

200

250

0 20 40 60 80 100 120

Request number

Resp

onse

tim

e (m

s)

Normal and sequential mode analysis

1

264KB block copied to the disk

64KB block copied to the file system cache and flushed later on to the disk

Example with a 320KB request size:

File system cache

Disk

flush

Req 4 Req 5

Req 6 Req 7 Req 8

Req 2

Req 1

Req 3

normal mode write 384KB

0

5

10

15

20

25

30

35

0 5 10 15 20 25 30

Request number

Res

pons

e tim

e (m

s)

01/04/2005 PRiSM / UVSQ 14

Write through mode analysis

Treponse = Td1 + Ttr1 + Td2 + Ttr2

Td1: time to position the head from the system block to the data block to write

Ttr1: writing time of the data block

Td2: positioning time from the written data block to the system block

Ttr2: writing time of the system block

Block to write

System block to modify

Departure and arrival position

Write through algorithms on both disk cache and file system cache

Response times are multiple of 6ms (one disk revolution)

"Write Through" mode writing

0102030405060708090

0 10 20 30 40

Request number

Res

po

nse

tim

e (m

s) 64KB

128KB

256KB

384KB

512KB

01/04/2005 PRiSM / UVSQ 15

Conclusions : write operations Periodic response times No buffer mode :

Generally good performances whatever the request size No copy of the written data on the file system cache

Normal and sequential modes : Performance drop compared to the no buffer mode Many passing over the same region of the disk Strong interactions between the disk cache and the file system cache One copy of some written data on the file system cache

Write through mode Poor performances as compared to the other modes Data are immediately copied on the disk

01/04/2005 PRiSM / UVSQ 16

Conclusions and guidelines To understand file access performance → response time analysis Read operations :

If one read → no buffer mode (exceptions) If many reads → sequential mode (request sizes)

Write operations : If one write (no modification of data) → no buffer mode If many writes → sequential or normal mode (request size) Security → write through mode (loss : till a factor 10 !!!)

OS behavior is the same (given an access mode) but storage systems employ different strategies

Choosing a good I/O strategy is crucial to obtain good performances

01/04/2005 PRiSM / UVSQ 17

Perspectives

Multi file access performances

Simulation of file accesses (validation phase)

Doing the same job on Linux systems

Testing RAIDs

01/04/2005 PRiSM / UVSQ 18

Thank you [email protected]