-
Armin Bahramshahry
1501-2024 Fullerton Ave.
Vancouver, B.C. V7P 3G4
April 16, 2007
Dr. Matei Ripeanu
Electrical and Computer Engineering
The University of British Columbia
2332 Main Mall
Vancouver, BC V6T 1Z4
Dear Dr. Ripeanu,
Please find attached my final report for the EECE 496 course
project. The report contains
a description of the context, in which my work was performed,
and includes an
explanation of my research, design and implementation for this
project. I also have
mentioned a set of recommendations for future work.
I look forward to your feedback.
Thanks,
Armin Bahramshahry.
c.c. J. Pavelich
-
UBC Electrical & Computer Engineering Department
vanDisk: An Exploration in Peer-To-Peer Collaborative Back-up
Storage SystemEECE 496 Term Paper Submitted to: Ms. Jane Pavelich
and Dr. Matei Ripeanu
Armin Bahramshahry 4/16/2007
-
ii
ABSTRACT
This term paper discusses the research, initial setup, and
development of a new user interface
component for the currently existing reliable peer-to-peer
backup storage system developed by
EECE 496 students.
The objective of the project is to study the existing
user-interface of the vanDisk project for
compatibility with another project named “free-loader”. There
has been research done on other
technologies provided by Microsoft for Windows operating systems
to design and implement a
virtual drive with file system callbacks.
The requirement of such component is to provide common
“file-level” file system callback
interface for both the vanDisk project and the free-loader
project. It needs to have a reasonable
throughput and CPU consumption. It is also very important to
keep in mind the response time of
the system.
This report explains the research on four technologies offered
by Microsoft named, Kernel-Mode
Driver Framework (KMDF), User-Mode Driver Framework (UMDF),
Windows Shell
Namespace, and Installable File System (IFS). Later it explains
the design decisions that have
been made, and it will explain the initial design and
development. Following which it will
demonstrate the experiments that have performed on the developed
functionalities and their
results. Finally it will make a set of recommendations on the
findings and conclude on how this
project could be continued and how the findings are useful.
-
iii
Table of Contents
ABSTRACT .....................................................................................................................................ii LIST
OF
FIGURES ......................................................................................................................... v LIST
OF
TABLES .......................................................................................................................... vi GLOSSARY .................................................................................................................................. vii LIST
OF
ACRONYMS ................................................................................................................ viii 1.0
INTRODUCTION ............................................................................................................... 1
2.0
BACKGROUND ................................................................................................................. 3 3.0
RESEARCH ......................................................................................................................... 4
3.1 Windows Kernel-Mode Driver
Framework ..................................................................... 4 3.1.1
Framework
Overview ............................................................................................... 4 3.1.2
I/O Flow
Model......................................................................................................... 5 3.1.3
Build
Process ............................................................................................................ 7 3.1.4
Installation
Process ................................................................................................... 8 3.1.5
Debug
Process ........................................................................................................... 8
3.2 Windows User-Mode Driver
Framework ........................................................................ 9 3.2.1
Framework
Overview ............................................................................................... 9 3.2.2
I/O Flow
Model....................................................................................................... 10 3.2.3
UMDF
Objects ........................................................................................................ 11 3.2.4
UMDF Callback
Interfaces ..................................................................................... 13
3.3 Windows Shell
Namespace ............................................................................................ 14 3.4
Windows Installable File
System ................................................................................... 15
3.4.1
Overview ................................................................................................................. 15 3.4.2
Filter Manager
Benefits .......................................................................................... 16
3.4.3
Architecture ............................................................................................................. 17 3.4.4
I/O Flow
Model....................................................................................................... 18
4.0 DESIGN AND
IMPLEMENTATION .............................................................................. 20 4.1
Design............................................................................................................................. 20
4.1.1
Overview ................................................................................................................. 20 4.1.2
Minispy .................................................................................................................. 20 4.1.2
Initial
Design ........................................................................................................... 21
4.2
Implementation............................................................................................................... 22
-
iv
5.0
EXPERIMENTS ................................................................................................................ 24 5.1
Experiments .................................................................................................................... 24 5.2
Results and
Analysis ...................................................................................................... 24
6.0
RECOMMENDATION ..................................................................................................... 28
7.0
CONCLUSION .................................................................................................................. 29 8.0
APPENDIX
A .................................................................................................................... 30 9.0
APPENDIX
B .................................................................................................................... 32 10.0
APPENDIX
C .................................................................................................................... 34 11.0
APPENDIX
D .................................................................................................................... 40 12.0
REFERENCES .................................................................................................................. 42
-
v
LIST OF FIGURES
Figure 1 Lifecycle failure pattern for hard drives
[2]. .................................................................... 1 Figure
2 KMDF I/O Flow
[3] ......................................................................................................... 6 Figure
3 I/O Flow to UMDF Driver
[6] ........................................................................................ 11 Figure
4 Simplified I/O Stack with Filter Manager and Minifilter Drivers
[9] ............................ 18 Figure 5 File
System
Design ......................................................................................................... 22 Figure
6 Message Polling Stress Test
Graph ................................................................................ 25 Figure
7 Message Polling Stress Test Graph at 0
Intervals .......................................................... 26 Figure
8 Message Polling CPU Usage
Graph ............................................................................... 27
-
vi
LIST OF TABLES Table 1 Supporting Create, Cleanup, and
Close
Requests ............................................................ 10 Table
2 UMDF Object Types
[6] .................................................................................................. 12 Table
3 Driver Callback Objects and Interfaces
[6] ..................................................................... 14 Table
4 KM-UM Message Passing Stress Test
Result ................................................................. 31 Table
5 KM-UM Message Passing CPU
Consumption ................................................................ 33 Table
6 Mini-Filter Major Functions
[10] .................................................................................... 39 Table
7 Batch File
Description ..................................................................................................... 41
-
vii
GLOSSARY
Term or Phrase Definition
Block Level In data storage, a block is a group of records on a
storage device.
Blocks are manipulated as units. For example, disk drives
often
read and write data in 512- byte blocks.
Callback In computer science, a callback is executable code that
is passed as
a parameter to other code. It allows a low level software layer
to
call a function occurring in a higher level layer.
Device Driver A software module that manages access to a
hardware device,
taking the device in and out of service, setting hardware
parameters, transmitting data between memory and the device,
sometimes scheduling multiple uses of the device on behalf
of
multiple processes, and handling I/O errors.
Filter Driver A file system filter driver intercepts requests
targeted at a file
system or another file system filter driver. By intercepting
the
request before it reaches its intended target, the filter driver
can
extend or replace functionality provided by the original target
of the
request.
Kernel The fundamental part of a program, typically an operating
system,
that resides in memory at all times and provides the basic
services.
It is the part of the operating system that is closest to the
machine
and may activate the hardware directly or interface to
another
software layer that drives the hardware.
Polling A technique that continually interrogates a peripheral
device to see
if it has data to transfer.
-
viii
LIST OF ACRONYMS
CPU
.............................................................................................................
Central Processing Unit
DMA
............................................................................................................
Direct Memory Access
DDI
............................................................................................................
Device Driver Interfaces
FSFilter
.................................................................................................................
File System Filter
GUID.......................................................................................................
Globally Unique Identifier
IFS
.................................................................................................................
Installable File System
INF
..........................................................................................................................
Information File
IRP
......................................................................................................................
I/O Request Packet
KM
...............................................................................................................................
Kernel Mode
KMDF
...........................................................................................
Kernel-Mode Driver Framework
OS
.........................................................................................................................
Operating System
P2P
.................................................................................................................................
Peer-to-Peer
PC
........................................................................................................................
Personal Computer
UM
...................................................................................................................................
User Mode
UMDF
..............................................................................................
User-Mode Driver Framework
WDM
..........................................................................................................
Windows Driver Model
WMI
...................................................................................
Windows Management Instrumentation
-
1
1.0 INTRODUCTION
Today computers are used more often than ever in society,
corporations, and government
organizations. Other than many security risks involving loss or
manipulation of personal and
important data, there are many others less obvious but more
disastrous factors that are not
considered by most individuals and organizations. One of the
factors that are much related to the
topic of this project is the reliability of the storage systems.
Hard drives tend to fail and they
actually have a well known “useful-life period” pattern shown in
figure 1 and considering them
as infinitely reliable can be very costly.
Figure 1 Lifecycle failure pattern for hard drives [2].
The vanDisk project provides a storage system with a much higher
reliability, meaning lower
risks for the users of its system. VanDisk also provides higher
performance compare to local
hard drive since clients are able to perform read and write
operations in parallel. VanDisk
provides these functionalities by replicating data over multiple
servers, and reconstructing the
data and re-replicating it upon any failures. The core
components of this system are implemented
and are operating; however, there are some improvements and new
features that have to be
implemented. The user interface of this system is currently
providing information at “block
level” which doesn’t provide enough information for better
management on the client side than
the existing implementation; it is also not compatible with the
free-loader project. This project is
focused on providing such interface at “file system level”,
which provides enough information
for all sorts of management on the client side and it is also
satisfying the needs of the free-loader
project.
-
2
This report presents an investigation of a windows operating
system (OS) specific user interface
for the vanDisk project, considering that this component would
also be integrated to another
ongoing project called “Free-Loader” [1]. A major part of this
investigation was to research and
study different technologies that provide functionality to
server these purposes. This report
contains a summary of my exploration of such systems.
My objectives during this project are first to attempt in
changing the existing user interface to
provide the new functionalities required, part of which is to
check for compatibility with the free-
loader project. In case that the existing code does not satisfy
the project’s requirements, further
study on other technologies that would enable implementation of
such interface has to be done.
The next step after having a solid decision on which technology
to use, is to design and
implement initial functionalities based on the selected
technology.
Through my investigation of the existing system I realized that
it could not be modified for use
by free-loader since free loader requires information at file
system level. Afterwards I studied
four technologies offered by Microsoft to implement such
functionalities. Later I have found the
best fit solution for the specified requirements and began the
initial design and implementation of
this system. Later I measured the performance of the system that
I implemented by running few
stress tests, all of which are explained in detail through this
report.
This report divides into the following primary sections. The
section 2 provides background
information and system overview of the vanDisk project. The
section 3 is focused on my
research on the four technologies offered by Microsoft for such
problems. The section 4
explains the initial design choices and issues of implementing
the chosen technology. Section 5
is focused on implementation and experiments, I make few
recommendations in section 6 and
section 7 concludes my findings.
-
3
2.0 BACKGROUND
VanDisk provides different set of configurations, every user of
the system can be the client or the
owner of the data to be stored, and it also can be a peer-node
which provides storage space for
other clients. One type of configuration is where only selected
trusted nodes that are
administrated by an organization are the peer-nodes. Another
type of configuration would be
where every node is both the client and peer-node forming a
completely distributed backup
storage system.
The system provides a very reliable storage space for its
clients by replicating the client’s data
over multiple peer-nodes. It is also providing a very friendly
and easy to use interface by
mounting a new logical volume that operates just like other
local hard disk volumes. The
difference is that vanDisk logical volume operates over the
network with a number of selected
peer-nodes to perform read and write operations. These peer
nodes are responsible for securely
and consistently respond to client’s request.
The vanDisk operations are currently performed at “block level”
meaning there are no
information regarding the type of the operation nor there any
information on the directory
structures or file information. The vanDisk system encrypts the
data before writing it to a peer-
node and accordingly decrypts the block upon read operations;
these operations are handled by
TrueCrypt which also implements the user interface for
vanDisk.
There is a new feature under development to secure the integrity
of the data by appending the
hash code of the blocks at its end enabling the client to check
for modifications by peer-node.
There is also another ongoing project on providing caching
mechanism on the client side. To
have full control over all operations, and to improve the
caching mechanism as well as the
security level, we require information at file level and not
block level. We also like to remove
and later replace the encryption and decryption algorithms by
the TrueCrypt with our own
implemented security system. To satisfy all these needs while
considering the free-loader project
in mind, we need to implement a new user interface intercepting
file system calls at file level.
-
4
3.0 RESEARCH
The research is focused on analyzing different technologies that
provides functionalities to
mount a logical drive to the user’s work space, which also
provides functionality to intercept the
file system calls at file level. To do so first I studied the
implementation of the TrueCrypt, and I
realized that they develop a logical driver that is implemented
using the Kernel-Mode Driver
Framework (KMDF). Based on the information I have gathered I
began my research by looking
at the KMDF technology which later led me to look at the
User-Mode Driver Framework
(UMDF). Following my research on these two technologies I have
acted upon Dr. Ripeanu’s
suggestion to study the Windows Shell Namespace and the Windows
Installable File System
(IFS) as well. Bellow is a detail explanation of these four
different technologies, their
architecture and functionality.
3.1 Windows Kernel-Mode Driver Framework
The Kernel-Mode Driver Framework (KMDF) is a driver framework
developed by the Microsoft
to make the process of creating and maintaining Kernel mode
device drivers easier. KMDF is
one of the frameworks included in the Windows Driver Foundation
(WDF). It is worth
mentioning that KMDF was originally released to only support
Windows XP and Windows
Server 2003 [3], and later in KMDF version 1.1 Microsoft have
added the support for Windows
2000 as well. The current version of the KMDF is 1.5.
3.1.1 Framework Overview
The benefit of using KMDF is that all the functionalities of
kernel mode programming is
provided, while developers can take advantage of many provided
predefined functionalities by
the KMDF. Some of the fundamental features that are provided by
the KMDF are the followings:
• Plug and Play and power management
-
5
• I/O queues
• Direct memory access (DMA)
• Windows management instrumentation (WMI)
• Synchronization
In addition, KMDF offers many management functionalities. For
example, KMDF provides a
well-defined object model and controls the lifetime of the
objects as well as their memory
allocation and de-allocation.
Developers of Kernel device drivers need to develop a minimal
set of operations since KMDF
provides functionality for the default common operations;
through which Microsoft can ensure
that the drivers are compatible with each successive release of
Windows operating system. In
this design the functionalities that required to be developed
are device specific event responses.
3.1.2 I/O Flow Model
Understanding the I/O request and response model is one of the
very main steps in writing a file
system driver. Following figure shows the overall flow of the
I/O through the KMDF library into
and out of the device driver implemented on top of the KMDF.
-
6
Figure 2 KMDF I/O Flow [3]
In the first step the IRP is directed to one of the following
components by KMDF for processing:
• I/O request handler, responsible for handling request
involving device I/O
• Plug and Play/power request handler, responsible for requests
involving change in the
device status
• WMI handler, responsible for event-tracing requests or system
control requests
Based on the type of the request each of the components
mentioned above take one of the
following actions:
• Raise one or more events to the driver
• Forward the request to an internal handler or a target I/O
handler for further processing
• Complete the request based on its own action
• Complete the request based on the drivers completion of the
request
If any request is not processed when it reaches the end of the
framework processing, KMDF
either takes an action and responds instead of the driver
according to the driver’s type, or returns
-
7
the STATUS_INVALID_DEVICE_REQUEST status. In general KMDF
automatically forwards
the requests to the default I/O target for “filter drivers”.
3.1.3 Build Process
Even though KMDF supports new Device Driver Interfaces (DDI) and
programming models, it
still has a lot in common with WDM [4]. Based on this fact there
are some limitations on
development of KMDF device drivers. Following is a short list of
few main points:
• Drivers are normally written in C. C++ can be used for driver
development in only a very
limited way. Usage of some basic C++ features like inline
variable declaration would be safe,
but the object-oriented features of C++ may produce generated
code that is not guaranteed to
work correctly in kernel mode [4].
• The C++ compiler works fine with C code and provides better
error detection and type safety
than the C compiler [4].
• Drivers must be built with the WDK or DDK build tools.
Microsoft Visual Studio is not
designed to support driver development and can be used in only a
limited way [4].
KMDF provides two types of build tools, one is the checked build
tool and the other one is the
free build tool. Checked builds are similar to the debug builds
that are used in application
development. They generate detailed debugging information and
enable certain types of
debugging-related code such as ASSERT macros. Checked builds are
normally used during the
earlier stages of driver development because they are much
easier to debug than free builds.
Keep in mind that due to the support for debugging, checked
builds are slower compare to the
free builds.
Free builds are similar to the Release builds that are used in
application development. They do
not support detailed debugging information; however, they are
fully optimized. Free builds are
-
8
more difficult to debug, and thus they are used at the end of
development cycle where the final
testing and performance tuning is happening.
3.1.4 Installation Process
Drivers need to be installed before they can be used for
debugging or end user purposes.
Installing the Kernel Device Drivers that are developed on top
of KMDF differ greatly from the
installation of drivers which use installation applications. To
install a KMDF driver package, it
needs to contain at least the following three files:
• The KDMF co-installer dynamic-link library (DLL)
• The INF file
• The digitally signed catalog (.cat) file (not required for
testing and debugging)
Driver packages also can optionally have files such as icons,
property sheet providers, supporting
DLLS, and more.
3.1.5 Debug Process
One of the most effective factors in the speed and efficiency of
a development process is the
correct usage of available debugger tools. All programs under
development have bugs at some
point, especially in the early stages. Debuggers are also very
helpful in a sense that they can be
used as a learning tool, since the developers are able to step
through the code and understand
different structures and behaviors of the system.
-
9
Mainly the debuggers are used to determine where and why the
driver is failing, and the only
exception to this rule is the kernel debuggers which can be used
to analyze crash-dump files.
Debugging kernel code is a very complicated process, and it
requires three hardware
components. One is the host computer running the “WinDbg” [5]
debugger. This is usually the
computer used to develop and build the kernel mode device
driver. The second component is the
test computer running an appropriate build of Windows that also
has the driver installed and the
kernel debugging feature enabled. The third component is a
communication system between the
host and the test computer. There are three options for this
component:
• Serial ports on the host and test computers with a null-modem
cable
• USB
• IEEE 1394 cables
In conclusion, the setup of such environment requires two set of
computer or support for virtual
machines that is costly. Also even after the setup, this process
is not easy to use and not as
friendly as other debugging tools.
3.2 Windows User-Mode Driver Framework
3.2.1 Framework Overview
The User-Mode Driver Framework (UMDF) is first introduced for
development of certain
classes of device drivers for Microsoft’s Windows Vista
operating system, and later it was also
made available for Windows XP. The problem with development of
completely Kernel Mode
device driver is that a badly written driver can cause major
damages to a system since it has very
high privileges and direct access to the kernel. However,
development of a driver on top of
UMDF requires the developers to use a dedicated API instead of
direct access to the kernel. This
-
10
is important since upon any critical error the system can
restart the driver and prevent it from
damaging other components of the system.
UMDF is a COM based DLL but it doesn’t use the COM runtime
library and it only uses the
COM as a programming pattern. UMDF doesn’t even use COM for
loading, unloading or for
controlling concurrency [7].
UMDF drivers typically have to implement create, cleanup and
close operations. In cases where
these functionalities are not implemented by the driver, UMDF
automatically forwards the
request to the default I/O target (lower driver in the device
stack) [7]. To support create, cleanup
and close requests, drivers have to implement the following
interfaces:
Request type Interface Method
Create IQueueCallbackCreate OnCreateFile
Cleanup IFileCallbackCleanup OnCleanupFile
Close IFileCallbackClose OnCloseFile Table 1 Supporting Create,
Cleanup, and Close Requests
3.2.2 I/O Flow Model
UMDF I/O flow is relatively simpler compare to KMDF I/O flow.
Following figure shows the
path that an I/O request takes from an application to a UMDF
driver. Callers use the same system
calls and Win32 API to send I/O requests to the UMDF driver as
they use to send to any other
device. Based on the I/O routine, the Windows kernel I/O manager
creates an I/O request packer
(IRP) and sends this IRP to the driver at the top of the
kernel-mode device stack for the target
device.
-
11
Figure 3 I/O Flow to UMDF Driver [6]
It is worth mentioning that UMDF manages the I/O cancellation
and completion, and ensures
that the Plug and Play and power state of the device is
compatible with performing device I/O.
Based on the type of the I/O request, the UMDF either queues the
request or invokes a callback
provided by the driver. UMDF provides configurable I/O queue
objects, and driver has to specify
which types of requests to place in each queue and how they
should be dispatched. The driver
has to provide callback interfaces to handle I/O requests on its
queues.
3.2.3 UMDF Objects
UMDF manages a series of objects that are exposed to the
user-mode driver. UMDF creates
some of these objects in response to application-triggered
actions, such as an I/O request; the
driver creates other objects by calling methods on UMDF
interfaces.
-
12
For each type of object, UMDF defines one or more interfaces
through which to manipulate
instances of the object. The interfaces provide methods and
properties. Methods define actions
that can be taken on behalf of the object and return a status to
indicate whether they succeeded or
failed. Property operations set and get the attributes of the
object and cannot fail. Some interfaces
are implemented by UMDF, and others are implemented by the
driver. Following table list all
different objects supported by UMDF and their interfaces and
description.
Type of object Interfaces Description
Base object IWDFObject Exposes a base object for use as the
driver requires.
Device IWDFDevice Exposes an instance of a device object. A
driver typically has one device object for each device that it
controls.
Driver IWDFDriver Exposes the driver object itself. Every driver
has one driver object.
File IWDFFile
Exposes a framework file object that was opened by the Win32
CreateFile function, through which applications can access the
device.
IWDFDriverCreatedFile Exposes a framework file object that the
driver created.
I/O queue IWDFIoQueue Exposes an I/O queue, which controls the
flow of I/O in the driver. A driver can have any number of I/O
queues.
I/O request IWDFIoRequest Exposes a request for device I/O.
I/O target IWDFIoTarget Represents the next-lower driver in the
device stack, to which the driver sends I/O requests.
Memory IWDFMemory Exposes memory that the driver uses, typically
an input or output buffer that is associated with an I/O
request.
USB device IWDFUsbTargetDevice Exposes a USB device object that
is an I/O target. Inherits from IWdfIoTarget.
USB interface IWDFUsbInterface Exposes an interface on a USB
device.
USB pipe IWDFUsbTargetPipe Exposes a USB pipe that is an I/O
target. Inherits from IWdfIoTarget. Table 2 UMDF Object Types
[6]
-
13
3.2.4 UMDF Callback Interfaces
UMDF drivers have to implement callback interfaces to provide
specific responses to requests.
Each interface is implemented on a specific UMDF object type.
Bellow there is a table of
callback interfaces and the type of object they associate with
their brief descriptions.
Type of object Callback Interfaces Description
Base object, or any object that inherits from
the base object type
IObjectCleanup
Provides processing that is required before an object is
deleted, typically releasing any references held on the object.
Driver IDriverEntry Provides main entry point and methods to
initialize the driver and add its devices.
Device
IPnpCallback Handles device stop, removal, and power state
changes.
IPnpCallbackHardware Provides hardware-related operations before
device power-up and after device power-down.
IPnpCallbackSelfManagedIo
Provides driver, rather than framework, control over I/O
operations at specific Plug and Play and power management
states.
IFileCallbackCleanup Handles clean-up requests for file objects
on a specific device.
IFileCallbackClose Handles close requests for file objects on a
specific device.
I/O queue
IQueueCallbackCreate Handles file create requests.
IQueueCallbackDefaultIoHandler
Handles create, device I/O control, read, and write requests for
which no other interface has been implemented.
-
14
IQueueCallbackDeviceIoControl Handles device I/O control
requests.
IQueueCallbackIoResume Resumes processing an I/O request after
its queue has been stopped.
IQueueCallbackIoStop Stops processing an I/O request because its
queue is stopping.
IQueueCallbackRead Handles read requests.
IQueueCallbackWrite Handles write requests.
I/O request
IImpersonateCallback Provides impersonation for certain Win32
I/O operations.
IRequestCallbackCancel Handles cancellation of an I/O
request.
IRequestCallbackRequestCompletion Handles completion of an I/O
request. Table 3 Driver Callback Objects and Interfaces [6]
Drivers only need to implement callbacks for the interfaces that
are important to their operations.
Upon a request if the corresponding callback is introduced it
will be invoked by UMDF,
otherwise the default callback for that request will be
invoked.
3.3 Windows Shell Namespace
The Windows Shell namespace is the system of organization that
Windows Explorer uses to
present the file system and other objects to the user. It
enables the developers to implement
virtual objects like “My Network Places” and “Recycle Bin” and
represent them just like real file
system and folders. One of the main responsibilities of the
Shell is managing and providing
access to many objects that make up the system. The most common
set of these objects are the
folders and files that exist on computer disk drives.
The other set of objects are the virtual objects that do not
involve physical storage at all. The
printer object, for instance, contains a collection of links to
networked printers. Other virtual
-
15
objects, such as the Recycle Bin, may contain data that is
stored on a disk drive, but needs to be
handled differently than normal files. Virtual objects may even
be located on remote computers.
Similar to the file system the two primary types of objects for
the shell namespace are files and
folders. Folders are split into two categories, real folders and
virtual folders, and like file system
folders, the collection of virtual folders generally varies from
system to system. On the other
hand unlike file system folders, users are not able to create
new virtual folders, and they can only
be installed. In general there are three classes of virtual
folders:
• Standard virtual folders, such as the Recycle Bin, that are
found on all systems.
• Optional virtual folders that have standard names and
functionality, but may not be
present on all systems.
• Non-standard folders that are installed by the user.
Windows Shell namespace introduces three main interfaces named
IPersistFolder, IShellFolder
and IDataObject. The IPersistFolder and IShellFolder provide
enough functionality to create a
virtual folder on the user’s system, and the IDataObject
provides lots of functionalities two of
which are “SetData” and “GetData” that is enough for our
purposes. Windows shell namespace
provides less information compare to UMDF and KMDF, and it lacks
a major required property.
It only works on top of Windows Explorer. This prevents
applications and command line scripts
to perform operation on a set of files or folders through the
implemented virtual folders. To show
such limitation, I have implemented a virtual folder using the
IShellFolder interface, and I have
tried to perform operations from command line and failed to do
so. This is a major drawback for
our system and in conclusion it is not possible to implement the
vanDisk user interface using
windows shell namespace technology.
3.4 Windows Installable File System
3.4.1 Overview
-
16
The Installable File System (IFS) is a file system API in
Microsoft Windows NT and successors
that enables the operating system to recognize and load drivers
for file systems. IFS kit has
always supported the development and installation of file system
filter driver (legacy filter). A
file system filter driver intercepts requests targeted at a file
system or another file system filter
driver. By intercepting the request before it reaches its
intended target, the filter driver can
extend or replace functionality provided by the original target
of the request. Examples of file
system filter drivers include anti-virus filters, backup agents,
and encryption products.
Recently Microsoft has introduced a new feature to its IFS kit
named “Filter Manager”. The
Filter Manager is a file system filter driver provided by
Microsoft that simplifies the
development of third-party filter drivers and solves many of the
problems with the existing
legacy filter driver model.
3.4.2 Filter Manager Benefits
A filter driver developed based on the Filter Manager model is
called a minifilter. There are
many benefits in developing a file system driver using
minifilter model instead of legacy filter
concept, some of which are listed bellow:
• Better control over filter loads and unloads order: Unlike a
legacy filter driver, a
minifilter driver can be loaded or unloaded at any time, while
legacy filters can only be
loaded at startup.
• Ability to process only necessary operations: The filter
manager uses a callback model
in which a minifilter driver can choose which types of I/O
operations to filter. Appendix
C contains the list of major operation that can be chosen by a
minifilter.
• Reduced complexity: The filter manager simplifies filtering
I/O requests by providing
support routines for common functionality, such as naming,
context management,
communication between user mode and kernel mode, and masking
differences between
file systems [9].
-
17
• Easier addition of new operations: Because minifilter drivers
register only for the I/O
operations they will handle, support for new operations can be
added to the filter manager
without breaking existing minifilter drivers.
• Better support for multiple platforms: A minifilter driver can
run on any version of
Windows that supports the filter manager. If a minifilter driver
registers for an I/O
operation that isn't available at runtime, the filter manager
simply doesn't call the
minifilter driver for that operation. A minifilter driver can
determine programmatically
whether functions are available, and filter manager structures
are designed to be
extensible [9].
• Better support for user-mode applications: The filter manager
provides common
functionality for user-mode services and control programs that
work with minifilter
drivers. The filter manager user-mode library, Filterlib.dll,
enables communication
between a user-mode service or control program and a minifilter
driver. Filterlib.dll also
provides interfaces for management tools.
3.4.3 Architecture
The filter manager is installed with Windows, but it becomes
active only when a minifilter driver
is loaded. The filter manager attaches to the file system stack
for a target volume. A minifilter
driver attaches to the file system stack indirectly, by
registering with the filter manager for the
I/O operations the minifilter driver chooses to filter.
A legacy filter driver's position in the file system I/O stack
relative to other filter drivers is
determined at system startup by its load order group. For
example, an antivirus filter driver
should be higher in the stack than a replication filter driver,
so it can detect viruses and disinfect
files before they are replicated to remote servers. Therefore,
filter drivers in the file system filter
(FSFilter) Anti-Virus load order group are loaded before filter
drivers in the FSFilter Replication
group. Each load order group has a corresponding system-defined
class and class GUID used in
the INF file for the filter driver [9].
-
18
Like legacy filter drivers, minifilter drivers attach in a
particular order. However, the order of
attachment is determined by a unique identifier called an
altitude. The attachment of a minifilter
driver at a particular altitude on a particular volume is called
an instance of the minifilter driver.
A minifilter driver's altitude ensures that the instance of the
minifilter driver is always loaded at
the appropriate location relative to other minifilter driver
instances, and it determines the order in
which the filter manager calls the minifilter driver to handle
I/O. Altitudes are allocated and
managed by Microsoft [9].
The following figure shows a simplified I/O stack with the
filter manager and three minifilter
drivers:
Figure 4 Simplified I/O Stack with Filter Manager and Minifilter
Drivers [9]
3.4.4 I/O Flow Model
A minifilter driver can filter IRP-based I/O operations as well
as fast I/O and file system filter
callback operations. For each of the I/O operations it chooses
to filter, a minifilter driver can
-
19
register a pre-operation callback routine, a post-operation
callback routine, or both [9]. When
handling an I/O operation, the filter manager calls the
appropriate callback routine for each
minifilter driver that registered for that operation. When that
callback routine returns, the filter
manager calls the appropriate callback routine for the next
minifilter driver that registered for the
operation.
For example, assuming all three minifilter drivers in the above
figure registered for the same I/O
operation, the filter manager would call their pre-operation
callback routines in order of altitude
from highest to lowest (A, B, C), then forward the I/O request
to the next-lower driver for further
processing. When the filter manager receives the I/O request for
completion, it calls each
minifilter driver's post-operation callback routines in reverse
order, from lowest to highest (C, B,
A).
For interoperability with legacy filter drivers, the filter
manager can attach filter device objects to
a file system I/O stack in more than one location. Each of the
filter manager's filter device
objects is called a frame. From the perspective of a legacy
filter driver, each filter manager frame
is just another legacy filter driver.
-
20
4.0 DESIGN AND IMPLEMENTATION
4.1 Design
4.1.1 Overview
The major part of the design of this new user interface is to
choose the technology that would
properly support development of this component. After a complete
research on these four
different technologies I have concluded that Windows Shell
Namespace does not satisfy the
needs of this project. I also have concluded that both the KMDF
and UMDF technologies
support enough functionality to implement our required
functionalities. In terms of using KMDF
as it has been discussed above, it is not as safe as UMDF since
any small bug could cause critical
damages to the system, and it also very difficult to debug. UMDF
is the better choice between
the three technologies mentioned; however, after studying the
features of the IFS kit it is
completely clear that IFS is the better choice of all.
IFS contains the benefits of both UMDF and KMDF while providing
a safer environment
compare to KMDF and much more functionality compare to UMDF. It
also provides a reliable
and very fast communication system between kernel mode (KM)
application and the user mode
(UM) application, which can be very useful. Based on these facts
the design of the vanDisk user
interface is based on IFS technology.
In order to have a better understanding of how minifilters are
designed and implemented I have
studied one of the samples provided by the IFS kit named
“minispy”.
4.1.2 Minispy
Minispy is a minifilter provided as an example within the IFS
package. It is implemented to
simply spy on a user specified logical drive (can be a virtual
drive). This example shows how a
-
21
minifilter loads and unloads as well as a demonstration on how
it can attach itself to a drive and
later detach. This is really important since within this model
the two process of virtual drive
creation and the FSFilter development are completely separated
and independent.
Minispy registers itself for the complete set of operations for
the attached drive and provides
functionality to log the operations to screen or file based on
user’s desire. Minispy is consist of
two separate processes, one is the FSFilter driver which is
implemented in KM. The other one is
the UM application which is only responsible for interaction
with the user. Minispy is a really
useful example to study since it also creates a communication
layer between the KM application
and the UM application. Using this communication layer user
commands such as list, attach, and
detach are sent to the KM and the logging messages are passed to
the UM application from KM
driver.
4.1.2 Initial Design
Using IFS kit the process of development for virtual drive
creation is completely separated from
implementation of the vanDisk’s file system. This report doesn’t
focus on how the virtual drive
could be implemented and instead it is mostly focused on the
design and implementation of the
vanDisk’s file system.
The vanDisk’s file system can follow the same design as the
minispy example. This is a
relatively safe design in terms of development as well, because
it is minimizing the KM
development. In this design the FSFilter driver developed in KM
only intercepts the file system
calls and passes them to the UM application. The UM application
can share a common interface
with both the vanDisk project as well as the free-loader
project, and each of these projects can
simply extend the shared interface to service their needs.
Based on the proposed design, the user interface is consisting
of a separate project which creates
a virtual drive, and another project implementing the file
system. The main components of this
file system is the FSFilter driver, the UM file system, the
shared interface and a fast
-
22
communication layer between the FSFilter driver and the UM file
system. Following diagram
shows the initial design:
Figure 5 File System Design
4.2 Implementation
Taking advantage of the minispy code which is a very simple
FSFilter driver just observing the
file system calls and possibly logging them based on user’s
request, the next important
component of the design represented in figure 5 is the
communication layer. The communication
layer requires to be able to pass the IRP from KM filter driver
to the UM file system application.
It also requires a well defined structure in order to
consistently pass all messages back and forth.
Initially I tried to simply modify the existing minispy code to
pass specific messages from KM
driver to UM application. In doing so I have encountered an
“unknown bug”, I would like to
emphasize on the fact that this bug until today is unknown for
the fact that I did not have a
proper debugging environment setup, and due to this bug Windows
crashed after running the
program for few seconds. I do believe that one reason that this
attempt has failed is that I tried to
modify their fairly complicated communication system for the
vanDisk purposes, which most
probably have caused an unexpected situation.
-
23
Faced with two options to continue the implementation, I had to
choose between setting up the
debugging environment and attempting to implement the
communication layer from scratch
without any debugging tools. I decided to implement this layer
even though it was risky due to
the fact that with facing any bug my attempt would have been a
failure without debugging tools.
I managed to successfully implement this layer for a simple
message passing initially. This
system is based on a polling algorithm by the UM application.
The KM driver or the “server”
registers to a specific port name, to which the UM application
connects. Following that the UM
application can send messages to the KM driver through a buffer
passed as “input buffer”. In
cases where the UM application is expecting messages from KM
application in return it has to
provide a buffer for such messages as well.
Extending the initially very simple implementation of the
communication layer, I have
implemented a fixed size queue in the KM driver to be able to
keep track of messages for the
UM application until the next poll time.
-
24
5.0 EXPERIMENTS
5.1 Experiments
There are always few main concerns with using polling algorithms
and those are the throughput
of the system, its CPU consumption and the average response time
of the system. In order to
measure these values I performed a set of stress tests on the
communication layer implemented. I
measured the throughput of the system by passing different
buffer sizes to the KM application,
and varying the polling interval.
I gathered information on throughput and CPU consumption of the
system for polling intervals
varying from 0 milliseconds to 1000 milliseconds. I performed
this test for buffer sizes varying
from 256 bytes (B) to 4 mega byte (MB). I have included a
selective set of the result in appendix
A and appendix B. Results in appendix A demonstrates the
throughput of the system for passing
as many messages from KM to UM in one second. Results in
appendix B are related to the CPU
consumption of the system in percentage for different test
cases.
5.2 Results and Analysis
The interesting findings in the throughput results is how the
slope of the graph changes among
different buffer sizes focusing on the polling interval of ‘0’
to ‘1’ millisecond. The other
important finding is that there is almost no change in the
throughput of the system between the
‘1’ to ‘10’ millisecond polling intervals, I believe this is due
to the UM application waiting in the
ready queue of the operating system for CPU time. Figure 6
demonstrates the throughput of the
system in terms of number of 128B messages per second.
-
25
Figure 6 Message Polling Stress Test Graph
The other very interesting result is that the maximum throughput
of the system at ‘0’ polling
interval is reached with 64MB, and passing larger buffer sizes
with ‘0’ polling interval only
reduces the throughput. Following figure is over the same set of
data as figure 5 focused on the
‘0’ to ‘1’ millisecond polling interval.
1
10
100
1000
10000
100000
1000000
10000000
0.1 1 10 100 1000
# of M
essages Pe
r Second
Polling Interval (ms)
KM‐UM Message Polling Stress Test
256 Buffer Size
512 Buffer Size
1024 Buffer Size
2048 Buffer Size
4096 Buffer Size
8192 Buffer Size
16384 Buffer Size
32768 Buffer Size
65536 Buffer Size
131072 Buffer Size
262144 Buffer Size
524288 Buffer Size
1048576 Buffer Size
2097152 Buffer Size
4194304 Buffer Size
-
26
Figure 7 Message Polling Stress Test Graph at 0 Intervals
In terms of CPU consumption running the tests on a Pentium 4 on
a Dell INSPIRON 5100 with
512MB RAM, results in figure 8 are achieved. As shown below for
all test with less than 2MB
buffer size the CPU consumption is minimal and reasonable while
it grows to an undesired state
by reaching the 2MB buffer size and reaching even worst
situation with 4MB buffer size.
0
500000
1000000
1500000
2000000
2500000
3000000
3500000
0 0.2 0.4 0.6 0.8 1 1.2
# of M
essages Pe
r Second
Polling Interval (ms)
KM‐UM Message Polling Stress Test
256 Buffer Size
512 Buffer Size
1024 Buffer Size
2048 Buffer Size
4096 Buffer Size
8192 Buffer Size
16384 Buffer Size
32768 Buffer Size
65536 Buffer Size
131072 Buffer Size
262144 Buffer Size
524288 Buffer Size
1048576 Buffer Size
2097152 Buffer Size
4194304 Buffer Size
-
27
Figure 8 Message Polling CPU Usage Graph
In conclusion the polling algorithm does satisfy the throughput
and CPU requirements; however,
as mentioned the response time of the system is a really
important factor. As demonstrated by the
results, it is not possible to have any polling interval less
than 10 millisecond (5 millisecond on
average) which itself is not a reasonable response time for the
vanDisk project. This
communication layer should only be used for passing debugging
and non-file system call
messages.
0
20
40
60
80
100
120
0.1 1 10 100 1000
% of C
PU Usage
Polling Interval (ms)
CPU Usage Diagram
2MB Buffer
4MB Buffer
-
28
6.0 RECOMMENDATION
My main recommendation to student continuing this part of the
vanDisk project is to read my
findings and references to completely understand how KMDF, UMDF
and IFS differ. It is really
important to understand the functionalities offered by IFS and
take advantage of them.
Furthermore I recommend the future developer of this system to
completely read and understand
the minispy project, and later my implementation of the
communication layer. By the end of this
process the developer should be comfortable with implementing
other functionalities for the
system. I recommend the student continuing this project to read
appendix D which explains on
how to setup his environment for easy build and development of
the minifilter.
The next step is ideally design and implementation of the
communication based on the “push”
algorithm, where the KM driver sends messages to the UM
application as soon as they are ready
to be sent. This is done through a blocking send message call by
the KM driver. This call
requires the UM application to be expecting the message; thus,
it is required to use a separate
thread which is constantly waits for messages from the KM driver
and responds to them as
quickly as possible.
Following the completion of the design and implementation of the
communication layer, the
replacements for the major operations of the minifilter has to
be implemented one by one. One
could do that by implementing a very simple file system
initially which simply redirects system
calls at UM level instead of the KM level. Upon a successful
implementation of such system, the
common interface has to be implemented which converts the
received messages to proper type
for vanDisk and free-loader projects.
In terms of developing the creation of the virtual driver, I
recommend the developers to study the
TrueCrypt implementation which contains the complete code for
creating and installation of the
virtual drive.
-
29
7.0 CONCLUSION
It is very important to always consider the expansion of the
project and how certain parts of the
project could be re-used. In this experiment I have learned a
great deal about different
technologies that support implementation of virtual drives and
file system. I have learned that it
is very important to study different options before making any
decisions.
Through this report I have concluded that windows shell
namespace does not satisfy the needs of
vanDisk project for implementing the required user interface. I
have also realized that even
though UMDF and KMDF provide enough functionality; they are not
the best solutions as one is
very difficult to implement and debug while the other one is
easier but does not include features
offered by IFS.
In conclusion the initial design and implementation of the
system that I have placed helps in both
debugging the KM application and understanding and developing
the minifilter file system
easier. Finally a major section of my project was focused on
understanding the benefits of all
these technologies and their drawbacks. This can be both reused
for future project and it is also a
very good starting point for student continuing this
project.
-
30
8.0 APPENDIX A
POLLING INTERVAL
(ms)
256B Buffer Size
(Message/Sec)
64KB Buffer Size
(Message /Sec)
256KB Buffer Size
(Message /Sec)
2MB Buffer Size
(Message /Sec)
4MB Buffer Size
(Message /Sec)0 571363.023 3264616.27 2197973.657 1771971.684
1745715.499 1 199.5125018 51177.55557 204710.2223 1629493.369
1635994.168 2 199.9123265 51177.55557 204710.2223 1631131.051
1635994.168 3 199.7124141 51177.55557 204710.2223 1465725.192
1634356.537 4 199.9123265 51177.55557 204710.2223 1629493.369
1635994.168 5 199.9123265 51177.55557 204710.2223 1627855.688
1635994.168 6 199.9123265 51177.55557 204710.2223 1631131.051
1632722.18 7 199.9123265 51177.55557 204710.2223 1629493.369
1632722.18 8 199.9123265 51177.55557 204710.2223 1632768.733
1632722.18 9 199.9123265 51177.55557 204710.2223 1631131.051
1632722.18 10 199.9123265 51177.55557 204710.2223 1631131.051
1632722.18 20 100.0559195 25614.31539 102457.2615 818840.8891
1090662.779 30 66.77071703 17093.30356 68373.21424 545892.8354
819629.8129 40 50.12781595 12832.72088 51330.88353 409827.4134
656356.9418 50 40.14219524 10276.40198 41105.60792 328516.347
547504.0245 60 33.48471614 8572.087332 34288.34933 274033.3083
469757.0494 70 28.72985778 7354.843591 29419.37436 235120.1097
371150.9158 80 25.16376418 6441.92363 25767.69452 205935.6205
366096.5971 90 22.39018056 5731.886224 22927.5449 183053.8849
329811.1996 100 20.17095383 5163.76418 20655.05672 164910.6325
300122.5593 110 18.35518691 4698.92785 18795.7114 150215.6256
275365.3551 120 16.84082857 4311.252114 17245.00846 137823.3382
254451.7351 130 15.5620063 3983.873612 15935.49445 127356.7261
236490.2461 140 14.46329983 3702.604757 14810.41903 118365.9257
220948.5669 150 13.51287976 3459.297219 13837.18888 110587.4737
206924.1636 160 12.68015328 3246.119239 12984.47696 103772.8664
195342.5873 170 11.94690414 3058.407459 12233.62983 97771.55955
184656.7675 180 11.29326151 2891.074947 11564.29979 92422.70912
175111.3679 190 10.70950384 2741.632982 10966.53193 87645.21946
166538.3285 200 10.18533312 2607.445279 10429.78112 83354.89403
158740.0282 210 9.708242353 2485.310042 9941.24017 79451.10082
151660.2089 220 9.275181288 2374.44641 9497.785639 75907.27792
145206.583 230 8.880492723 2273.406137 9093.624548 72677.18099
139302.4737 240 8.519477983 2180.986364 8723.945455 69722.3946
133883.7543 250 8.188208979 2096.181499 8384.725995 66943.92012
128818.4385 260 7.878201741 2016.819646 8067.278582 64474.6442
124211.3455
-
31
270 7.596668405 1944.747112 7778.988447 62169.67567 119746.3585
280 7.330713614 1876.662685 7506.65074 59993.68838 115708.8213 290
7.083395969 1813.349368 7253.397472 57970.06641 112054.3951 300
6.852876955 1754.336501 7017.346002 56083.78392 108534.2473 350
5.902829974 1511.124473 6044.497893 48308.38868 93948.17997 400
5.192522767 1329.285828 5317.143314 42494.65186 82835.14777 450
4.631012501 1185.5392 4742.156801 37864.08709 74216.69797 500
4.193960697 1073.653938 4294.615753 34322.60342 67232.63706 550
3.82224716 978.495273 3913.981092 31281.91389 61552.25584 600
3.52433672 902.2302003 3608.920801 28843.08887 56685.07224 700
3.043236787 779.0686174 3116.27447 24906.4753 49064.4924 800
2.688436344 688.2397041 2752.958816 22002.51427 43419.75044 900
2.403945726 615.4101057 2461.640423 19674.90588 39146.42078 1000
2.196836555 562.3901582 2249.560633 17978.50655 35565.09062
Table 4 KM-UM Message Passing Stress Test Result
-
32
9.0 APPENDIX B
Polling Interval < 2MB Buffer 2MB Buffer 4MB Buffer 0 100 100
100 1 0 2 51 2 0 2 51 3 0 2 51 4 0 2 51 5 0 2 51 6 0 3 51 7 0 2 51
8 0 2 50 9 0 2 51 10 0 2 51 20 0 0 34 30 0 1 25 40 0 0 20 50 0 0 17
60 0 0 14 70 0 0 21 80 0 0 11 90 0 0 10
100 0 0 9 110 0 0 8 120 0 0 7 130 0 0 7 140 0 0 7 150 0 0 6 160
0 0 6 170 0 0 5 180 0 0 5 190 0 0 5 200 0 0 5 210 0 0 4 220 0 0 4
230 0 0 4 240 0 0 4 250 0 0 4 260 0 0 3 270 0 0 3
-
33
280 0 0 3 290 0 0 3 300 0 0 4 350 0 0 3 400 0 0 2 450 0 0 2 500
0 0 2 550 0 0 1 600 0 0 1 700 0 0 1 800 0 0 1 900 0 0 1
1000 0 0 1 Table 5 KM-UM Message Passing CPU Consumption
-
34
10.0 APPENDIX C
Major Function Description
IRP_MJ_CREATE The operating system sends an IRP_MJ_CREATE
request to open a handle to a file object or device object.
IRP_MJ_CLOSE Receipt of the IRP_MJ_CLOSE request indicates that
the reference count on a file object has reached zero
IRP_MJ_READ The IRP_MJ_READ request is sent by the I/O Manager
or by a file system driver. This request can be sent, for example,
when a user-mode application has called a Microsoft Win32 function
such as ReadFile, or when a kernel-mode component has called
ZwReadFile Minor Operations: IRP_MN_COMPLETE IRP_MN_COMPLETE_MDL
IRP_MN_COMPLETE_MDL_DPC IRP_MN_COMPRESSED IRP_MN_DPC IRP_MN_MDL
IRP_MN_MDL_DPC IRP_MN_NORMAL
IRP_MJ_WRITE The IRP_MJ_WRITE request is sent by the I/O Manager
or by a file system driver. This request can be sent, for example,
when a user-mode application has called a Microsoft Win32 function
such as WriteFile or when a kernel-mode component has called
ZwWriteFile Minor Operations: IRP_MN_COMPLETE IRP_MN_COMPLETE_MDL
IRP_MN_COMPLETE_MDL_DPC IRP_MN_COMPRESSED IRP_MN_DPC IRP_MN_MDL
IRP_MN_MDL_DPC IRP_MN_NORMAL
IRP_MJ_QUERY_INFORMATION
This request can be sent, for example, when a user-mode
application has called a Microsoft Win32 function such as
GetFileInformationByHandle or when a kernel-mode component has
called ZwQueryInformationFile
-
35
IRP_MJ_SET_INFORMATION
It can be sent, for example, when a user-mode application has
called a Microsoft Win32 function such as SetEndOfFile or when a
kernel-mode component has called ZwSetInformationFile
IRP_MJ_QUERY_EA When a user-mode application has requested
information about a file's extended attributes
IRP_MJ_SET_EA The I/O Manager sends the IRP_MJ_SET_EA request to
set a file's extended attributes.
IRP_MJ_FLUSH_BUFFERS The IRP_MJ_FLUSH_BUFFERS request is sent by
the I/O Manager and other operating system components, as well as
other kernel-mode drivers, when buffered data needs to be flushed
to disk. It can be sent, for example, when a user-mode application
has called a Microsoft Win32 function such as FlushFileBuffers.
(For file system drivers and file system filter drivers, calling
CcFlushCache is usually preferable to sending an IRP.)
IRP_MJ_QUERY_VOLUME_INFORMATION
The IRP_MJ_QUERY_VOLUME_INFORMATION request is sent by the I/O
Manager. It can be sent, for example, when a user-mode application
has called a Microsoft Win32 function such as GetDiskFreeSpace or
GetFileType.
IRP_MJ_SET_VOLUME_INFORMATION
It can be sent, for example, when a user-mode application has
called a Microsoft Win32 function such as SetVolumeLabel.
IRP_MJ_DIRECTORY_CONTROL
It can be sent, for example, when a user-mode application has
called a Microsoft Win32 function such as ReadDirectoryChangesW or
FindNextVolumeMountPoint or when a kernel-mode component has called
ZwQueryDirectoryFile Minor Functions:
IRP_MN_NOTIFY_CHANGE_DIRECTORY
Indicates a request for notification of changes to the
directory. Usually, instead of satisfying this request immediately,
the file system driver holds the IRP in a private queue. When a
change occurs to the directory, the file system driver performs the
notification, and dequeues and completes the IRP.
IRP_MN_QUERY_DIRECTORY Indicates a directory query request. The
types of information that can be queried are file-system-dependent,
but generally include the following: FileBothDirectoryInformation
FileDirectoryInformation FileFullDirectoryInformation
FileIdBothDirectoryInformation FileIdFullDirectoryInformation
FileNamesInformation
-
36
FileObjectIdInformation FileReparsePointInformation
IRP_MJ_FILE_SYSTEM_CONTROL
It can be sent, for example, when a user-mode application has
called the Microsoft Win32 DeviceIoControl function to send a file
system I/O control (FSCTL) request.
Minor Functions:
File system drivers should handle the following minor function
codes: IRP_MN_KERNEL_CALL
This request is the same as IRP_MN_USER_FS_REQUEST (described
following), except that the source of the request is a trusted
kernel component.
IRP_MN_MOUNT_VOLUME Indicates a volume mount request. If a file
system driver receives this IRP for a volume whose format does not
match that of the file system, the file system driver should return
STATUS_UNRECOGNIZED_VOLUME.
IRP_MN_USER_FS_REQUEST Indicates an FSCTL request, possibly on
behalf of a user-mode application that has called the Microsoft
Win32 DeviceIoControl function or on behalf of a kernel-mode
component that has called ZwDeviceIoControlFile or
IoBuildDeviceIoControlRequest. For detailed information about FSCTL
requests, see "Device Input and Output Control Codes" in the
Microsoft Windows SDK documentation.
IRP_MN_VERIFY_VOLUME Indicates a volume verification request.
For removable media, the file system must verify the volume when it
detects that the media has been removed and returned to ensure that
it is still the same volume that the file system was previously
working with. If the volume has changed, the file system should
invalidate all outstanding handles. It will also return an error if
the file system on this new media has changed. This request is most
often used for floppy drives.
File system recognizers must handle the following minor function
code: IRP_MN_LOAD_FILE_SYSTEM
Indicates a load-file system request.
IRP_MJ_DEVICE_CONTROL Normally this IRP is sent on behalf of a
user-mode application that has called the Microsoft Win32
DeviceIoControl function or on behalf of a kernel-mode component
that has called ZwDeviceIoControlFile
-
37
IRP_MJ_INTERNAL_DEVICE_CONTROL
When Sent The IRP_MJ_INTERNAL_DEVICE_CONTROL request is sent by
the I/O Manager and other operating system components, as well as
other kernel-mode drivers. Unlike IRP_MJ_DEVICE_CONTROL requests,
IRP_MJ_INTERNAL_DEVICE_CONTROL requests are used only for
communication among kernel-mode components. While an
IRP_MJ_DEVICE_CONTROL request usually originates with a call to
DeviceIoControl or ZwDeviceIoControlFile, these routines cannot
create IRP_MJ_INTERNAL_DEVICE_CONTROL requests. However, both types
of IRP can be created by calling IoBuildDeviceIoControlRequest.
Operation: File System Drivers The file system driver should
extract and decode the file object to determine whether the request
has been issued on a handle that represents a volume open. If this
is the case, the file system driver should pass the IRP to the
device driver for the storage device on which the volume is
mounted. If not, the driver should fail the IRP.
IRP_MJ_SHUTDOWN The IRP_MJ_SHUTDOWN request is sent by the I/O
Manager or by a file system driver when the system is being shut
down
IRP_MJ_LOCK_CONTROL The file system driver should extract and
decode the file object to determine whether the target device
object is the file system's control device object. If this is the
case, the file system driver should complete the IRP as appropriate
without processing the lock request. Otherwise, if the request has
been issued on a handle that represents a user file open, the file
system driver should perform the o peration indicated by the minor
function code and complete the IRP. If not, the driver should fail
the IRP. The following are the valid minor function codes:
IRP_MN_LOCK
Indicates a byte-range lock request, possibly on behalf of a
user-mode application that has called the Microsoft Win32 LockFile
function.
IRP_MN_UNLOCK_ALL Indicates a request to release all byte-range
locks for a file, usually because the last outstanding handle to a
file object is being closed.
IRP_MN_UNLOCK_ALL_BY_KEY Indicates a request to release all
byte-range locks with a specified key value.
IRP_MN_UNLOCK_SINGLE Indicates a request to release a single
byte-range lock, possibly on behalf of a user-mode application that
has called the Microsoft Win32 UnlockFile function.
-
38
IRP_MJ_CLEANUP When Sent Receipt of the IRP_MJ_CLEANUP request
indicates that the handle reference count on a file object has
reached zero. (In other words, all handles to the file object have
been closed.) Often it is sent when a user-mode application has
called the Microsoft Win32 CloseHandle function (or when a
kernel-mode driver has called ZwClose) on the last outstanding
handle to a file object. It is important to note that when all
handles to a file object have been closed, this does not
necessarily mean that the file object is no longer being used.
System components, such as the Cache Manager and the Memory
Manager, might hold outstanding references to the file object.
These components can still read to or write from a file, even after
an IRP_MJ_CLEANUP request is received.
IRP_MJ_QUERY_SECURITY The IRP_MJ_QUERY_SECURITY request is sent
by the I/O Manager. It can be sent, for example, when a user-mode
application has called a Microsoft Win32 function such as
GetSecurityInfo.
IRP_MJ_SET_SECURITY The IRP_MJ_SET_SECURITY request is sent by
the I/O Manager. This request can be sent, for example, when a
user-mode application has called a Microsoft Win32 function such as
SetSecurityInfo.
IRP_MJ_QUERY_QUOTA When Sent The IRP_MJ_QUERY_QUOTA request is
sent by the I/O Manager. This request can be sent, for example,
when a user-mode application has called a Microsoft Win32 method
such as IDiskQuotaControl::GetQuotaState. Operation: File System
Drivers If the file system supports disk quotas, the file system
driver should extract and decode the file object to determine
whether it represents a user open of a file or directory. If it
does, the driver should process the query and complete the IRP.
Otherwise, the driver should complete the IRP as appropriate
without processing the query.
IRP_MJ_SET_QUOTA When Sent The IRP_MJ_SET_QUOTA request is sent
by the I/O Manager. It can be sent, for example, when a user-mode
application has called a Microsoft Win32 method such as
IDiskQuotaControl::SetQuotaState. Operation: File System Drivers
IRP_MJ_SET_QUOTA and IRP_MJ_QUERY_QUOTA existed in Windows NT 4.0
but were not used by file systems. On Windows 2000 and later
systems, they are used for disk quota support in NTFS. Support for
these IRPs by new file systems is optional.
IRP_MJ_PNP When Sent The Plug and Play Manager sends the
IRP_MJ_PNP request
-
39
whenever Plug and Play activity occurs on the system. Other
operating system components, as well as other kernel-mode drivers,
can also send certain IRP_MJ_PNP requests, depending on the minor
function code. For more information about Plug and Play IRP
processing requirements for drivers, see Plug and Play. For
reference information about IRP_MJ_PNP minor function codes, see
Plug and Play Minor IRPs. Operation: File System Drivers The file
system should check the minor function code to determine which
operation is requested. File systems must handle the following
minor function codes: IRP_MN_CANCEL_REMOVE_DEVICE
Indicates that a previous query-remove device request was
canceled. This request is sent to alert the file system in case it
needs to perform any cleanup related to the cancellation.
IRP_MN_QUERY_REMOVE_DEVICE Indicates that a device is about to
be removed. If a file system is mounted on the device, the PnP
Manager sends this request to the file system and to any file
system filters. If there are open handles to the device, the file
system typically fails the query-remove request. If not, the file
system typically locks the volume to prevent future create requests
from succeeding. If a mounted file system does not support a
query-remove request, the PnP Manager fails the query-remove
request for the device.
IRP_MN_REMOVE_DEVICE Indicates that a device is about to be
removed. If a file system is mounted on the device, the PnP Manager
sends this IRP to the file system and to any file system filters.
The file system should immediately pass this IRP to the storage
driver for the device, setting a completion routine in which the
file system then dismounts the volume.
IRP_MN_START_DEVICE Indicates that a device is being started.
The file system should pass this IRP to the storage driver for the
device.
IRP_MN_SURPRISE_REMOVAL Indicates that a device has been
removed. If a file system was mounted on the device, the PnP
Manager sends this IRP to the file system and to any file system
filters. The file system should immediately pass this IRP to the
storage driver for the device, setting a completion routine in
which the file system then dismounts the volume.
Table 6 Mini-Filter Major Functions [10]
-
40
11.0 APPENDIX D There is a DVD containing the installation files
for WDK 6000-16386 that can be requested from
Dr. Ripeanu. In this DVD the files for the vanDisk user
interface and FSFilter driver is include in
a directory named “vanDisk”. Within this directory both the
report of the project exists and the
files for setting up the developing environment. It is required
to get access to Visual Studio .Net
2005 since the project files are stored in the 2005 version
format.
This directory contains few batch files in
“VanDisk\C\WINDDK\6000\bin” that has to be copied
to the “C:\WINDDK\6000\bin”, also the content of
“VanDisk\C\WINDDK\6000\src\filesys\miniFilter\vanDisk” has to be
copied to
“C:\WINDDK\6000\src\filesys\miniFilter\vanDisk”.
Upon acquiring Visual Studio .Net 2005, it is possible to simply
open the vanDisk solution file
located at “miniFilter\vanDisk\vanDisk\vanDisk.sln” and build
the “filter” that is the KM driver
and the “user”, which is the UM application projects. There is
also another project named
“RunMain” which simply runs
“vanDisk\vanDisk\RunMain\runbatch.bat” that installs the
vanDisk minifilter.which automatically copies the “minispy.sys”
and “minispy.exe” to proper
location and installs the FSFilter driver using the
“minispy.inf”. Keep in mind that this batch
doesn’t load the driver.
There are quiet a few batch files developed for easing the
process of build and running the
project, following table explains details of each of these batch
files and executables:
Batch File Description
vanDisk\copyback.bat Copies the “minispy.exe” and “minispy.sys”
to the correct location, and installs the filter using the
“minispy.inf” file.
vanDisk\load.bat Loads the minispy filter of the vanDisk
project.
vanDisk\unload.bat Unloads the minispy filter of the vanDisk
project.
-
41
vanDisk\show.bat Shows the list of minifilters loaded into the
system. This is a quick way to see if the minispy filter is loaded
or not.
vanDisk\vanDisk\RunMain\runbatch.bat Runs the
“vanDisk\copyback.bat”
vanDisk\vanDisk\Filter\FilterBuild.bat Builds the vanDisk filter
project, it requires typing “make” and pressing enter at the
end.
vanDisk\vanDisk\User\UserBuild.bat Builds the vanDisk user
project, it requires typing “make” and pressing enter at the
end.
bin\setenv-vanDisk-filter.bat Setups the build environment for
building the vanDisk filter project by only typing “make” in the
command line and pressing enter. To clean the project simply type
“make /clean” and press enter.
bin\setenv-vanDisk-user.bat Setups the build environment for
building the vanDisk user project by only typing “make” in the
command line and pressing enter. To clean the project simply type
“make /clean” and press enter.
Table 7 Batch File Description
-
42
12.0 REFERENCES
[1]
[2] J. Yang and F.-B. Sun. “A Comprehensive Review of Hard-Disk
Drive Reliability”. In
Proc. of the Annual Reliability and Maintainability Symposium,
1999.
[3]
[4]
[5]
[6]
[7]
[8]
[9]
[10]