Armin Bahramshahry 1501-2024 Fullerton Ave. Vancouver, B.C. V7P 3G4 April 16, 2007 Dr. Matei Ripeanu Electrical and Computer Engineering The University of British Columbia 2332 Main Mall Vancouver, BC V6T 1Z4 Dear Dr. Ripeanu, Please find attached my final report for the EECE 496 course project. The report contains a description of the context, in which my work was performed, and includes an explanation of my research, design and implementation for this project. I also have mentioned a set of recommendations for future work. I look forward to your feedback. Thanks, Armin Bahramshahry. c.c. J. Pavelich
51
Embed
Armin Bahramshahry Vancouver, B.C. V7P 3G4 April 16, 2007 ...arminb/PAPERS/2007.04-vanDisk-ArminBahramsh… · Vancouver, B.C. V7P 3G4 April 16, 2007 Dr. Matei Ripeanu Electrical
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Armin Bahramshahry
1501-2024 Fullerton Ave.
Vancouver, B.C. V7P 3G4
April 16, 2007
Dr. Matei Ripeanu
Electrical and Computer Engineering
The University of British Columbia
2332 Main Mall
Vancouver, BC V6T 1Z4
Dear Dr. Ripeanu,
Please find attached my final report for the EECE 496 course project. The report contains
a description of the context, in which my work was performed, and includes an
explanation of my research, design and implementation for this project. I also have
mentioned a set of recommendations for future work.
I look forward to your feedback.
Thanks,
Armin Bahramshahry.
c.c. J. Pavelich
UBC Electrical & Computer Engineering Department
vanDisk: An Exploration in Peer-To-Peer Collaborative Back-up Storage SystemEECE 496 Term Paper Submitted to: Ms. Jane Pavelich and Dr. Matei Ripeanu
Armin Bahramshahry 4/16/2007
ii
ABSTRACT
This term paper discusses the research, initial setup, and development of a new user interface
component for the currently existing reliable peer-to-peer backup storage system developed by
EECE 496 students.
The objective of the project is to study the existing user-interface of the vanDisk project for
compatibility with another project named “free-loader”. There has been research done on other
technologies provided by Microsoft for Windows operating systems to design and implement a
virtual drive with file system callbacks.
The requirement of such component is to provide common “file-level” file system callback
interface for both the vanDisk project and the free-loader project. It needs to have a reasonable
throughput and CPU consumption. It is also very important to keep in mind the response time of
the system.
This report explains the research on four technologies offered by Microsoft named, Kernel-Mode
Driver Framework (KMDF), User-Mode Driver Framework (UMDF), Windows Shell
Namespace, and Installable File System (IFS). Later it explains the design decisions that have
been made, and it will explain the initial design and development. Following which it will
demonstrate the experiments that have performed on the developed functionalities and their
results. Finally it will make a set of recommendations on the findings and conclude on how this
project could be continued and how the findings are useful.
iii
Table of Contents
ABSTRACT .....................................................................................................................................ii LIST OF FIGURES ......................................................................................................................... v LIST OF TABLES .......................................................................................................................... vi GLOSSARY .................................................................................................................................. vii
LIST OF ACRONYMS ................................................................................................................ viii 1.0 INTRODUCTION ............................................................................................................... 1
3.3 Windows Shell Namespace ............................................................................................ 14 3.4 Windows Installable File System ................................................................................... 15
8.0 APPENDIX A .................................................................................................................... 30
9.0 APPENDIX B .................................................................................................................... 32 10.0 APPENDIX C .................................................................................................................... 34 11.0 APPENDIX D .................................................................................................................... 40
IRP_MJ_CREATE The operating system sends an IRP_MJ_CREATE request to open a handle to a file object or device object.
IRP_MJ_CLOSE Receipt of the IRP_MJ_CLOSE request indicates that the reference count on a file object has reached zero
IRP_MJ_READ The IRP_MJ_READ request is sent by the I/O Manager or by a file system driver. This request can be sent, for example, when a user-mode application has called a Microsoft Win32 function such as ReadFile, or when a kernel-mode component has called ZwReadFile Minor Operations: IRP_MN_COMPLETE IRP_MN_COMPLETE_MDL IRP_MN_COMPLETE_MDL_DPC IRP_MN_COMPRESSED IRP_MN_DPC IRP_MN_MDL IRP_MN_MDL_DPC IRP_MN_NORMAL
IRP_MJ_WRITE The IRP_MJ_WRITE request is sent by the I/O Manager or by a file system driver. This request can be sent, for example, when a user-mode application has called a Microsoft Win32 function such as WriteFile or when a kernel-mode component has called ZwWriteFile Minor Operations: IRP_MN_COMPLETE IRP_MN_COMPLETE_MDL IRP_MN_COMPLETE_MDL_DPC IRP_MN_COMPRESSED IRP_MN_DPC IRP_MN_MDL IRP_MN_MDL_DPC IRP_MN_NORMAL
IRP_MJ_QUERY_INFORMATION
This request can be sent, for example, when a user-mode application has called a Microsoft Win32 function such as GetFileInformationByHandle or when a kernel-mode component has called ZwQueryInformationFile
35
IRP_MJ_SET_INFORMATION
It can be sent, for example, when a user-mode application has called a Microsoft Win32 function such as SetEndOfFile or when a kernel-mode component has called ZwSetInformationFile
IRP_MJ_QUERY_EA When a user-mode application has requested information about a file's extended attributes
IRP_MJ_SET_EA The I/O Manager sends the IRP_MJ_SET_EA request to set a file's extended attributes.
IRP_MJ_FLUSH_BUFFERS The IRP_MJ_FLUSH_BUFFERS request is sent by the I/O Manager and other operating system components, as well as other kernel-mode drivers, when buffered data needs to be flushed to disk. It can be sent, for example, when a user-mode application has called a Microsoft Win32 function such as FlushFileBuffers. (For file system drivers and file system filter drivers, calling CcFlushCache is usually preferable to sending an IRP.)
IRP_MJ_QUERY_VOLUME_INFORMATION
The IRP_MJ_QUERY_VOLUME_INFORMATION request is sent by the I/O Manager. It can be sent, for example, when a user-mode application has called a Microsoft Win32 function such as GetDiskFreeSpace or GetFileType.
IRP_MJ_SET_VOLUME_INFORMATION
It can be sent, for example, when a user-mode application has called a Microsoft Win32 function such as SetVolumeLabel.
IRP_MJ_DIRECTORY_CONTROL
It can be sent, for example, when a user-mode application has called a Microsoft Win32 function such as ReadDirectoryChangesW or FindNextVolumeMountPoint or when a kernel-mode component has called ZwQueryDirectoryFile Minor Functions: IRP_MN_NOTIFY_CHANGE_DIRECTORY
Indicates a request for notification of changes to the directory. Usually, instead of satisfying this request immediately, the file system driver holds the IRP in a private queue. When a change occurs to the directory, the file system driver performs the notification, and dequeues and completes the IRP.
IRP_MN_QUERY_DIRECTORY Indicates a directory query request. The types of information that can be queried are file-system-dependent, but generally include the following: FileBothDirectoryInformation FileDirectoryInformation FileFullDirectoryInformation FileIdBothDirectoryInformation FileIdFullDirectoryInformation FileNamesInformation
It can be sent, for example, when a user-mode application has called the Microsoft Win32 DeviceIoControl function to send a file system I/O control (FSCTL) request.
Minor Functions:
File system drivers should handle the following minor function codes: IRP_MN_KERNEL_CALL
This request is the same as IRP_MN_USER_FS_REQUEST (described following), except that the source of the request is a trusted kernel component.
IRP_MN_MOUNT_VOLUME Indicates a volume mount request. If a file system driver receives this IRP for a volume whose format does not match that of the file system, the file system driver should return STATUS_UNRECOGNIZED_VOLUME.
IRP_MN_USER_FS_REQUEST Indicates an FSCTL request, possibly on behalf of a user-mode application that has called the Microsoft Win32 DeviceIoControl function or on behalf of a kernel-mode component that has called ZwDeviceIoControlFile or IoBuildDeviceIoControlRequest. For detailed information about FSCTL requests, see "Device Input and Output Control Codes" in the Microsoft Windows SDK documentation.
IRP_MN_VERIFY_VOLUME Indicates a volume verification request. For removable media, the file system must verify the volume when it detects that the media has been removed and returned to ensure that it is still the same volume that the file system was previously working with. If the volume has changed, the file system should invalidate all outstanding handles. It will also return an error if the file system on this new media has changed. This request is most often used for floppy drives.
File system recognizers must handle the following minor function code: IRP_MN_LOAD_FILE_SYSTEM
Indicates a load-file system request.
IRP_MJ_DEVICE_CONTROL Normally this IRP is sent on behalf of a user-mode application that has called the Microsoft Win32 DeviceIoControl function or on behalf of a kernel-mode component that has called ZwDeviceIoControlFile
37
IRP_MJ_INTERNAL_DEVICE_CONTROL
When Sent The IRP_MJ_INTERNAL_DEVICE_CONTROL request is sent by the I/O Manager and other operating system components, as well as other kernel-mode drivers. Unlike IRP_MJ_DEVICE_CONTROL requests, IRP_MJ_INTERNAL_DEVICE_CONTROL requests are used only for communication among kernel-mode components. While an IRP_MJ_DEVICE_CONTROL request usually originates with a call to DeviceIoControl or ZwDeviceIoControlFile, these routines cannot create IRP_MJ_INTERNAL_DEVICE_CONTROL requests. However, both types of IRP can be created by calling IoBuildDeviceIoControlRequest. Operation: File System Drivers The file system driver should extract and decode the file object to determine whether the request has been issued on a handle that represents a volume open. If this is the case, the file system driver should pass the IRP to the device driver for the storage device on which the volume is mounted. If not, the driver should fail the IRP.
IRP_MJ_SHUTDOWN The IRP_MJ_SHUTDOWN request is sent by the I/O Manager or by a file system driver when the system is being shut down
IRP_MJ_LOCK_CONTROL The file system driver should extract and decode the file object to determine whether the target device object is the file system's control device object. If this is the case, the file system driver should complete the IRP as appropriate without processing the lock request. Otherwise, if the request has been issued on a handle that represents a user file open, the file system driver should perform the o peration indicated by the minor function code and complete the IRP. If not, the driver should fail the IRP. The following are the valid minor function codes: IRP_MN_LOCK
Indicates a byte-range lock request, possibly on behalf of a user-mode application that has called the Microsoft Win32 LockFile function.
IRP_MN_UNLOCK_ALL Indicates a request to release all byte-range locks for a file, usually because the last outstanding handle to a file object is being closed.
IRP_MN_UNLOCK_ALL_BY_KEY Indicates a request to release all byte-range locks with a specified key value.
IRP_MN_UNLOCK_SINGLE Indicates a request to release a single byte-range lock, possibly on behalf of a user-mode application that has called the Microsoft Win32 UnlockFile function.
38
IRP_MJ_CLEANUP When Sent Receipt of the IRP_MJ_CLEANUP request indicates that the handle reference count on a file object has reached zero. (In other words, all handles to the file object have been closed.) Often it is sent when a user-mode application has called the Microsoft Win32 CloseHandle function (or when a kernel-mode driver has called ZwClose) on the last outstanding handle to a file object. It is important to note that when all handles to a file object have been closed, this does not necessarily mean that the file object is no longer being used. System components, such as the Cache Manager and the Memory Manager, might hold outstanding references to the file object. These components can still read to or write from a file, even after an IRP_MJ_CLEANUP request is received.
IRP_MJ_QUERY_SECURITY The IRP_MJ_QUERY_SECURITY request is sent by the I/O Manager. It can be sent, for example, when a user-mode application has called a Microsoft Win32 function such as GetSecurityInfo.
IRP_MJ_SET_SECURITY The IRP_MJ_SET_SECURITY request is sent by the I/O Manager. This request can be sent, for example, when a user-mode application has called a Microsoft Win32 function such as SetSecurityInfo.
IRP_MJ_QUERY_QUOTA When Sent The IRP_MJ_QUERY_QUOTA request is sent by the I/O Manager. This request can be sent, for example, when a user-mode application has called a Microsoft Win32 method such as IDiskQuotaControl::GetQuotaState. Operation: File System Drivers If the file system supports disk quotas, the file system driver should extract and decode the file object to determine whether it represents a user open of a file or directory. If it does, the driver should process the query and complete the IRP. Otherwise, the driver should complete the IRP as appropriate without processing the query.
IRP_MJ_SET_QUOTA When Sent The IRP_MJ_SET_QUOTA request is sent by the I/O Manager. It can be sent, for example, when a user-mode application has called a Microsoft Win32 method such as IDiskQuotaControl::SetQuotaState. Operation: File System Drivers IRP_MJ_SET_QUOTA and IRP_MJ_QUERY_QUOTA existed in Windows NT 4.0 but were not used by file systems. On Windows 2000 and later systems, they are used for disk quota support in NTFS. Support for these IRPs by new file systems is optional.
IRP_MJ_PNP When Sent The Plug and Play Manager sends the IRP_MJ_PNP request
39
whenever Plug and Play activity occurs on the system. Other operating system components, as well as other kernel-mode drivers, can also send certain IRP_MJ_PNP requests, depending on the minor function code. For more information about Plug and Play IRP processing requirements for drivers, see Plug and Play. For reference information about IRP_MJ_PNP minor function codes, see Plug and Play Minor IRPs. Operation: File System Drivers The file system should check the minor function code to determine which operation is requested. File systems must handle the following minor function codes: IRP_MN_CANCEL_REMOVE_DEVICE
Indicates that a previous query-remove device request was canceled. This request is sent to alert the file system in case it needs to perform any cleanup related to the cancellation.
IRP_MN_QUERY_REMOVE_DEVICE Indicates that a device is about to be removed. If a file system is mounted on the device, the PnP Manager sends this request to the file system and to any file system filters. If there are open handles to the device, the file system typically fails the query-remove request. If not, the file system typically locks the volume to prevent future create requests from succeeding. If a mounted file system does not support a query-remove request, the PnP Manager fails the query-remove request for the device.
IRP_MN_REMOVE_DEVICE Indicates that a device is about to be removed. If a file system is mounted on the device, the PnP Manager sends this IRP to the file system and to any file system filters. The file system should immediately pass this IRP to the storage driver for the device, setting a completion routine in which the file system then dismounts the volume.
IRP_MN_START_DEVICE Indicates that a device is being started. The file system should pass this IRP to the storage driver for the device.
IRP_MN_SURPRISE_REMOVAL Indicates that a device has been removed. If a file system was mounted on the device, the PnP Manager sends this IRP to the file system and to any file system filters. The file system should immediately pass this IRP to the storage driver for the device, setting a completion routine in which the file system then dismounts the volume.
Table 6 Mini-Filter Major Functions [10]
40
11.0 APPENDIX D There is a DVD containing the installation files for WDK 6000-16386 that can be requested from
Dr. Ripeanu. In this DVD the files for the vanDisk user interface and FSFilter driver is include in
a directory named “vanDisk”. Within this directory both the report of the project exists and the
files for setting up the developing environment. It is required to get access to Visual Studio .Net
2005 since the project files are stored in the 2005 version format.
This directory contains few batch files in “VanDisk\C\WINDDK\6000\bin” that has to be copied
to the “C:\WINDDK\6000\bin”, also the content of
“VanDisk\C\WINDDK\6000\src\filesys\miniFilter\vanDisk” has to be copied to
“C:\WINDDK\6000\src\filesys\miniFilter\vanDisk”.
Upon acquiring Visual Studio .Net 2005, it is possible to simply open the vanDisk solution file
located at “miniFilter\vanDisk\vanDisk\vanDisk.sln” and build the “filter” that is the KM driver
and the “user”, which is the UM application projects. There is also another project named
“RunMain” which simply runs “vanDisk\vanDisk\RunMain\runbatch.bat” that installs the
vanDisk minifilter.which automatically copies the “minispy.sys” and “minispy.exe” to proper
location and installs the FSFilter driver using the “minispy.inf”. Keep in mind that this batch
doesn’t load the driver.
There are quiet a few batch files developed for easing the process of build and running the
project, following table explains details of each of these batch files and executables:
Batch File Description
vanDisk\copyback.bat Copies the “minispy.exe” and “minispy.sys” to the correct location, and installs the filter using the “minispy.inf” file.
vanDisk\load.bat Loads the minispy filter of the vanDisk project.
vanDisk\unload.bat Unloads the minispy filter of the vanDisk project.
41
vanDisk\show.bat Shows the list of minifilters loaded into the system. This is a quick way to see if the minispy filter is loaded or not.
vanDisk\vanDisk\RunMain\runbatch.bat Runs the “vanDisk\copyback.bat”
vanDisk\vanDisk\Filter\FilterBuild.bat Builds the vanDisk filter project, it requires typing “make” and pressing enter at the end.
vanDisk\vanDisk\User\UserBuild.bat Builds the vanDisk user project, it requires typing “make” and pressing enter at the end.
bin\setenv-vanDisk-filter.bat Setups the build environment for building the vanDisk filter project by only typing “make” in the command line and pressing enter. To clean the project simply type “make /clean” and press enter.
bin\setenv-vanDisk-user.bat Setups the build environment for building the vanDisk user project by only typing “make” in the command line and pressing enter. To clean the project simply type “make /clean” and press enter.
Table 7 Batch File Description
42
12.0 REFERENCES
[1] <http://www.csm.ornl.gov/~vazhkuda/Morsels/>
[2] J. Yang and F.-B. Sun. “A Comprehensive Review of Hard-Disk Drive Reliability”. In
Proc. of the Annual Reliability and Maintainability Symposium, 1999.