Windows io manager

Windows IO Manager and Windows IO Manager and Layered Driver FrameworkLayered Driver Framework

Sisimon SomanSisimon Soman

File System

Volume Manager

Disk Class Driver

Hardware Driver

IO Manager

App issue ReadFile

NtReadFile

IO Mgr create IRP Packet, send to driver stack

User Land

Kernel Land

IRP

What is IO Request Packet (IRP)What is IO Request Packet (IRP)

IO Operation passes thru, IO Operation passes thru, – Different stages.Different stages.– Different threads.Different threads.– Different drivers.Different drivers.

IRP Encapsulate the IO request.IRP Encapsulate the IO request.

IRP is thread independent.IRP is thread independent.

IRP Continued..IRP Continued..

Compare IRP with Windows Messages -Compare IRP with Windows Messages -MSG structure.MSG structure.

Each driver in the stack do its own task, Each driver in the stack do its own task, finally forward the IRP to the lower driver finally forward the IRP to the lower driver in the stack.in the stack.

IRP can be processed synchronously or IRP can be processed synchronously or asynchronously.asynchronously.

IRP Continued..IRP Continued..

Usually lower level hardware driver takes more Usually lower level hardware driver takes more time. H/W driver can mark the IRP for pending time. H/W driver can mark the IRP for pending and return.and return.

When H/W finish IO, H/W driver complete the When H/W finish IO, H/W driver complete the IRP by calling IoCompleteRequest().IRP by calling IoCompleteRequest().

IoCompleteRequest() call IO completion routine IoCompleteRequest() call IO completion routine set by drivers in stack and complete the IO.set by drivers in stack and complete the IO.

Structure of IRPStructure of IRP

Fixed IRP HeaderFixed IRP Header

Variable Stack locations –Variable Stack locations –– One sub stack per driverOne sub stack per driver

IRP Header

Stack Location 1

Stack Location 2

Stack Location 3

Stack Location N

Flow of IRPFlow of IRP

IRP Header

Stack Location 1

Stack Location 2

Stack Location 3

Stack Location 4

File System

Volume Manager

Disk Class Driver

Hardware Driver

Storage Stack

IRP for Storage Stack

Forward IRP to lower driver in the stack

Flow of IRP CompletionFlow of IRP Completion

IRP Header

Stack Location 1

Stack Location 2

Stack Location 3

Stack Location 4

File System – Completion Routine

Volume Manager – Completion Routine

Disk Class Driver – Completion Routine

Hardware Driver – Complete the IRP

Storage Stack

IRP for Storage Stack

Call the completion routine while completing the IRP

IRP HeaderIRP Header

IO buffer Information.IO buffer Information.

FlagsFlags– Page IO FlagPage IO Flag– No Caching IO flagNo Caching IO flag

IO Status – On Completion set this to IO IO Status – On Completion set this to IO Completed.Completed.

IRP cancel routineIRP cancel routine

IRP Stack LocationIRP Stack Location

IO Manager get the driver count in the IO Manager get the driver count in the stack from the top device in the stack.stack from the top device in the stack.

While creating IRP, IO manager allocate While creating IRP, IO manager allocate the IO stack locations equal to the device the IO stack locations equal to the device count from the top device object.count from the top device object.

Contents of IO Stack LocationContents of IO Stack Location

Major and minor function code,Major and minor function code,– IRP_MJ_PNPIRP_MJ_PNP

IRP_MN_START_DEVICEIRP_MN_START_DEVICEIRP_MN_QUERY_REMOVE_DEVICEIRP_MN_QUERY_REMOVE_DEVICE

Argument Specific to the function codeArgument Specific to the function code// System service parameters for: NtReadFile// System service parameters for: NtReadFilestruct {struct { ULONG Length;ULONG Length; ULONG POINTER_ALIGNMENT Key;ULONG POINTER_ALIGNMENT Key; LARGE_INTEGER ByteOffset;LARGE_INTEGER ByteOffset; } Read;} Read;// System service parameters for: NtWriteFile// System service parameters for: NtWriteFile

struct {struct { ULONG Length;ULONG Length; ULONG POINTER_ALIGNMENT Key;ULONG POINTER_ALIGNMENT Key; LARGE_INTEGER ByteOffset;LARGE_INTEGER ByteOffset; } Write;} Write;

Contents of IO Stack LocationContents of IO Stack Location

IO Completion routine specific to the IO Completion routine specific to the driver.driver.

File object specific to the request.File object specific to the request.

Asynchronous IOAsynchronous IO

CreateFile(…, CreateFile(…, FILE_FLAG_OVERLAPPEDFILE_FLAG_OVERLAPPED ,..), ,..), ReadFile(.., ReadFile(.., LPOVERLAPPEDLPOVERLAPPED))When complete the IO operation, IO Mgr When complete the IO operation, IO Mgr signal the signal the EVENTEVENT in in LPOVERLAPPED.LPOVERLAPPED.

How Async IO work in KernelHow Async IO work in Kernel

Lower layer driver complete IRP in arbitrary Lower layer driver complete IRP in arbitrary thread context.thread context.IO Mgr call IO Completion routine in reverse IO Mgr call IO Completion routine in reverse order.order.If operation is Async, IO Mgr queue an APC If operation is Async, IO Mgr queue an APC specific to the initiator thread.specific to the initiator thread.This APC has complete info of buffer, size info.This APC has complete info of buffer, size info.This APC get executed later in the context of This APC get executed later in the context of initiator thread, which copy buffer to user space, initiator thread, which copy buffer to user space, trigger the event set by App.trigger the event set by App.

Common issues related IRPCommon issues related IRP

After forward the IRP down, don’t touch it (except from After forward the IRP down, don’t touch it (except from IO completion routine).IO completion routine).If lower driver mark the IRP for pending, all top layer If lower driver mark the IRP for pending, all top layer driver should do the same.driver should do the same.If a middle level driver need to keep the IRP for further If a middle level driver need to keep the IRP for further processing after completing it by lower driver, it can processing after completing it by lower driver, it can return STATUS_MORE_PROCESSING REQUIRED return STATUS_MORE_PROCESSING REQUIRED from completion routine.from completion routine.Middle layer driver should complete it later.Middle layer driver should complete it later.See ReactOS source code (instead of reading 20 page See ReactOS source code (instead of reading 20 page doc)doc)FastIO - ConceptsFastIO - Concepts

Part 2Part 2

How device stack setupHow device stack setup

Bus driver, notice new deviceBus driver, notice new device

Bus driver, notice new deviceBus driver, notice new device

Create Physical Device ObjectCreate Physical Device Object

Bus driver, notice new deviceBus driver, notice new device

Create Physical Device ObjectCreate Physical Device Object

Inform PNP, something changedInform PNP, something changed

Bus driver, notice new deviceBus driver, notice new device

Create Physical Device ObjectCreate Physical Device Object

Inform PNP, something changedInform PNP, something changed

PNP query devicesPNP query devices(Device relations) in bus (Device relations) in bus

Bus driver, notice new deviceBus driver, notice new device

Create Physical Device ObjectCreate Physical Device Object

Inform PNP, something changedInform PNP, something changed

PNP query devicesPNP query devices(Device relations) in bus(Device relations) in bus

PNP MhrGet the H/W device ID PNP MhrGet the H/W device ID

Bus driver, notice new deviceBus driver, notice new device

Create Physical Device ObjectCreate Physical Device Object

Inform PNP, something changedInform PNP, something changed

PNP query devicesPNP query devices(Device relations) in bus(Device relations) in bus

PNP MhrGet the H/W device ID PNP MhrGet the H/W device ID

Get Functional Driver Info fromGet Functional Driver Info from registry using H/W Device IDregistry using H/W Device ID

Bus driver, notice new deviceBus driver, notice new device

Create Physical Device ObjectCreate Physical Device Object

Inform PNP, something changedInform PNP, something changed

PNP query devicesPNP query devices(Device relations) in bus(Device relations) in bus

PNP MhrGet the H/W device ID PNP MhrGet the H/W device ID

Get Functional Driver Info fromGet Functional Driver Info from registry using H/W Device IDregistry using H/W Device ID

Load Functional driver,call Load Functional driver,call AddDevice() AddDevice()

Pass PDO of device alsoPass PDO of device also

Bus driver, notice new deviceBus driver, notice new device

Create Physical Device ObjectCreate Physical Device Object

Inform PNP, something changedInform PNP, something changed

PNP query devicesPNP query devices(Device relations) in bus(Device relations) in bus

PNP MhrGet the H/W device ID PNP MhrGet the H/W device ID

Get Functional Driver Info fromGet Functional Driver Info from registry using H/W Device IDregistry using H/W Device ID

Load Functional driver,call Load Functional driver,call AddDevice() AddDevice()

Pass PDO of device alsoPass PDO of device also

Create FDO of device,Attach to Create FDO of device,Attach to PDO,Continue this for all devicesPDO,Continue this for all devices

Physical Device ObjectPhysical Device Object

Functional Device ObjectFunctional Device Object

Upper FilterUpper Filter

Lower FilterLower Filter

Physical Device ObjectPhysical Device Object

Functional Device ObjectFunctional Device Object

PDO-FDO pair for a Device

PDO-FDO pair with registered filter driver for a Device

Real world example Real world example

How storage stack build upHow storage stack build up

Questions ?Questions ?