App-V Volume Format Specification

Microsoft Application Virtualization Volume Format Specification

MICROSOFT APPLICATION VIRTUALIZATION VOLUME FORMAT SPECIFICATION May 15th, 2010 - Version 1.0

Abstract This specification describes the file format, internal structure and layout of volume files used by the Microsoft Application Virtualization (hereafter App-V) product.

1 Contents2 Introduction.........................................................................................................................................2

3 Overview of App-V Volume File...........................................................................................................2

3.1 App-V Volume Format.................................................................................................................3

3.2 Container.....................................................................................................................................4

3.3 Permanent First 16 Containers....................................................................................................9

3.4 Remaining containers after the first 16.....................................................................................20

4 Appendix............................................................................................................................................31

Microsoft Corporation. All rights reserved. By using or providing feedback on these materials, you agree to by accepting the license agreement.


2 INTRODUCTIONThe reader should be familiar with the overall concept of application virtualization technology as well as concepts and terminology commonly used in conjunction with App-V.

The following terminology is used in this paper:

App-V Volume FileThe App-V volume file-system is a file based file-system that is all the data is stored in special volume files with any of the following file extensions: .fsd, .fsg, .pkg, and .tmp.

ContainerA container is a data structure which contains some metadata about a data structure (i.e. directory structure, volume information, allocation map, file structure etc.

Storage BlockStorage Blocks serve as fixed size data containers which are marked as either used or free by the Storage Block Allocation Bitmap.

Package A set of applications that have been sequenced by the App-V Sequencer. These applications share a common root and run under the same virtual environment.

3 OVERVIEW OF APP-V VOLUME FILE

The App-V volume file-system is a file based file-system that is all the data is stored in special volume files with any of the following file extensions: .fsd, .fsg, .pkg, and .tmp, rather than on the disk as individual files. A summary of each of the volume files is stated below:

FSD Cache Volume (.fsd): This volume file maintains read-only package data which is basically the content of the SFT file. Thus FSD Cache Volume file acts as a true cache, where any data, if lost, can be fully recovered from the App-V Server.

Global FSD Cache Volume (.fsg): This volume maintains any data that is created in the context of processes that are not registered within App-V virtual environment or are not related to any package. An example of such processes would be the App-V Listener process, where it is not registered within App-V environment, but it is still allowed to access the App-V file system. Its contents are not recoverable if lost. While it is important to maintain the integrity of this volume, it has to be noted that if this volume is lost, it is not going to affect the streaming of the packages since this volume does not maintain any package specific data.



Per System Package Volume (.pkg/.tmp): There are 2 types of volumes in this case: one type maintains any changes that were made to package specific files that are marked as APP_DATA/APP_CONFIG, and the other one maintains any changes that were made to package specific files that are marked as USER_DATA/USER_CONFIG. These changes are made by allowed system processes and by definition these are global to all users, and will affect all users logged on to the App-V Client. In multi-user environment all launches and shutdowns that occur on behalf of different users are maintained. These volumes are initialized when a user launches the first instance of the package, and will be closed, when the last instance of the package is closed.

Per User Package Volume (.pkg/.tmp): Again there are 2 types of volumes in this case: one type maintains any changes that were made to package specific files that are marked as USER_DATA/USER_CONFIG, and the other one maintains any changes that were made to package specific files that are marked as APP_DATA/APP_CONFIG. Such changes are made by the applications of a specific package running under a user context. The USER_DATA Package Volume file will reside in user specific profile, and in roaming and folder-redirection profile environments it will be accessible to a user no matter what server user is logged on to. Theses volumes will be initialized when the user launches the first instance of the package, and will be closed, when the user closes the last instance of the package.

For any volume other than FSD Cache Volume and Global FSD Cache Volume, while a volume is in-use we will be accessing its temporary copy (.tmp). The master copy will be updated only upon volume shutdown, which occurs when the last reference of the volume is closed. If any system failures occur while the volume is in-use that cause volume session shutdown or/and system reboot, the temporary copy of the volume will be discarded and any changes made to it will be lost. In case of FSD Cache Volume file, its contents will be analyzed upon reboot to determine if the previous abnormal shutdown could corrupt its contents, and if any or all data has to be discarded and re-fetched from the App-V Server.

3.1 APP-V VOLUME FORMATThe volume files provide a persistent data storage for all the data that has to be used by App-V File System between system shutdowns and reboots. Since all these different types of volumes share the same format, from now on in the rest of the document they will be treated as a single type of volume. The root of a volume file is populated with several well-known Containers (explained in the next section) that together describe all the remaining data stored in a volume file. At the beginning, a volume file is going to be divided into predetermined number of fixed size Storage Blocks, where a Block size is a configurable parameter. In present implementation, the default Storage Block size is 4 KB, and the default Container size is 512 bytes. Even though these 2 block sizes are configurable, it is advised not to change the default sizes which may produce undesired side effects in the File System. The number of Storage Blocks is determined by the size of the volume file, which is also a configurable parameter. Storage Blocks serve as fixed size data containers which are marked as either used or free by the Storage Block Allocation Bitmap. The primary layout of a volume file is shown in the figure 1:



Volume ContainerPrimary

Index: 0

Volume Container Secondary

Index: 1

Container Allocation

map

Index: 2

Beginning of Storage Blocks

Security Descriptor Container

Index: 3

Storage Allocation

Map

Index: 4

Reserved

Index: 5-14

Root Directory

Index: 15

Figure 1: Primary layout of an App-V volume

In order to completely process an App-V volume, one needs to process all the containers and the data structures that the containers point to. Figure 2 shows a simple flowchart of a parser which enumerates the files and directories of a volume file:

Process the first 16 containers

Build a container table by processing the

Container Allocation Map

Enumerate files and directories starting

from root

Figure 2: Flowchart of a simple volume parser

3.2 CONTAINERA container is a data structure which contains some metadata about a data structure (i.e. directory structure, volume information, allocation map, file structure etc.). So in order to process a data structure, we need to first process the container. The data structure that the container points to is either embedded within the container, or it is stored in a storage block outside the container block in which case the container will have some pointers to that storage block. The list of different type of containers is as follows:

Volume Container Primary: This container points to the volume information data structure which contains basic information about this volume.

Volume Container Secondary: This container points to the duplicate volume information data structure used as a backup copy of the Volume Container Primary (unused).

Container Allocation Map Container: This container points to the container allocation map structure.

Security Descriptor Container: This container points to the security descriptor table. Storage Allocation Map Container: This container points to the storage allocation map data

structure. Reserved: This container is currently unused to be used in the future.



Package Container: This container points to a package information data structure. Empty Container: This container is empty. It doesn’t point to any valid data structure. Package Directory Container: This container points to a directory structure. This directory was

created by App-V Sequencer and is part of a package. Run Directory Container: This container points to a directory structure which has been

generated during the package runtime. Package File Container: This container points to a directory and file data block structure. This

file was created by App-V Sequencer and is part of a package. Run File Container: This container points to a directory and file data block structure. This file has

been created during the package runtime. Package Deleted Container: The corresponding file or directory has been marked as deleted. Link Data Container: The container points to a symbolic link.

Each container can logically be divided into two parts, where part one consists of a common header, and part two consists of an opaque data, the content of which is determined by the type of a container. By default the total size of a container is 512 bytes. The actual size of containers within a volume can be retrieved from the Volume Container Primary described later. Figure 2 depicts the common header of the Container structure:

Common Header Fields: 90 bytes

TypeId: 4 bytesVersion: 4 bytesGenerationId: 4 bytesPrimaryDirectoryId: 4 bytesFileGUID: 16 bytesPackageID: 4 bytesTypeName: 16 bytesInfoDataLength: 4 bytesCheckSum: 4 bytesMetaDataIndex: 4 bytesPackageDataIndex: 4 bytesContainerIndex: 4 bytesContainerFlushIndex: 4 bytesSecurityDescriptorIndex: 4 bytesDataStorageType: 2 bytesReserved: 2 bytesFlags: 4 bytesSoftricityFileTypeFlags: 2 bytesData: Variable

Figure 3: Common Header of the Container Structure

Common Header Field DescriptionsThe following provides detailed definitions of the fields used in a container common header:

TypeId



This indicates the type of this container. The opaque data member of the container points to different data structures depending of this value. The following is the list of flags which are described in later sections:

Flag Value Purpose

PACKAGE_FILE 0x00000001 Points to a package file data structure

PACKAGE_DIRECTORY 0X00000002 Points to a package directory data structure

RUN_FILE 0X00000003 Points to a runtime generated file data structure

RUN_DIRECTORY 0X00000004 Points to a runtime generated directory data structure

PACKAGE_DELETED 0X00000005 Points to a file/directory which has been marked as deleted

SECURITY_DESCRIPTOR 0X00000007 Points to the security descriptor table

CONTAINER_EMPTY 0X00000008 Empty container

VOLUME_CONTAINER_P 0X00000009 Points to the volume information data structure

VOLUME_CONTAINER_S 0X0000000A Duplicate copy of VOLUME_CONTAINER_P

CONTAINER_RESERVED 0X0000000B Reserved for future used (Unused)

CONTAINER_PACKAGE 0X0000000C Points to a package information block

LINK_DATA 0X0000000D Points to a symbolic link

CONTAINER_ALLOCATION_MAP 0X0000000E Points to the container allocation map structure

STORE_ALLOCATION_MAP 0X0000000F Points to the storage allocation map structure

TRANSACTION_LOG 0X00000010 Unused

Version This value identifies the particular version number of the data within this container. Thus, the Version number and container TypeId uniquely identify the specific format usage information for the container data. This number is unique to App-V File System and is changed only when the format of the container is changed.

GenerationId This value stores the version id of a file. It is specific to the files that are created as part of the package initialization. It is used during the package upgrade only to determine if a file has been changed and needs to be upgraded.

PrimaryDirectoryId This indicates the parent directory container (the ContainerIndex of a container) where additional type specific information of this container is located. For example, if a container points to a directory with ContainerIndex 30 and the parent directory of this directory has ContainerIndex 15, then PrimaryDirectoryId of this container will have value 15. A value of 0 implies that there is no parent associated with this container.

FileGUIDThis value stores a unique file identifier (in GUID form). It is specific to files/directories that are created as part of package initialization, and used to find a specific Container by its GUID rather than by its name.

PackageID



This indicates a package container (the ContainerIndex of a package container) that this file or directory (pointed to by this container) belongs to. A value of 0 implies that this container is not associated with any package.

TypeName An ASCII type name buffer used for debugging purposes.

InfoDataLength This describes the number of bytes used for the container type-specific information which is referenced by the Data member of the container (please see the container Data member section).

CheckSum This is a checksum on the ContainerHdr which is an in-memory data structure. It does not cover data outside the container header and is present only to protect against certain types of runtime system failures.

MetaDataIndex This is an offset into directory entry table used for a quick lookup of the Container metadata structure within a parent container. (Please see the Root Directory section about the layout of a directory entry table).

PackageDataIndex This is an offset into Package File Table used for a quick lookup of the container reference within a package container (Please see the Package Container section about the layout of a directory entry table).

ContainerIndex This is an index within the Container Table which is an in-memory data structure. For an on-disk Container structure, this value represents the sequence number that has been assigned to this container. This value is unique per container and used as a primary identifier.

ContainerFlushIndex This is an index in volume file, where this container resides. It is used to locate/update on-disk container block.

SecurityDescriptorIndex A security descriptor offset in the Security Table (Please see Security Descriptor Container for additional information).

DataStorageType This indicates the format of the data storage region. The flags used are:



Flag Value Purpose

FORMAT_RAW 0x00000001 Raw data

FORMAT_INLINE 0X00000010 The data structure is inline within the container pointed to by the Data member

FORMAT_BLOCKMAP 0X00000020 The data structure is in a storage block outside of this container. The Data member points to the storage block containing the data.

A value of 0 in this field means “Not Applicable” (i.e. a container with TypeId CONTAINER_RESERVED). The format can be a bitmap of any of the following combinations (explained in a later section):

FORMAT_RAW & FORMAT_BLOCKMAPFORMAT_RAW & FORMAT_INLINE

Flags This is a bit mask of persistent flags that states additional information of the container:

Flag Value Purpose

FILE_VERIFIED 0x00000001 The file data has been downloaded from the server and verified

LOCKED 0X00000002 Used to lock the container while in use

NOTIFY_LAUNCH_DATA_REMOVED 0X00000008 The package launch data has been deleted; the App-V Client needs to steam the data again from the Server

PACKAGE_FILE_MODIFIED 0X00000010 Unused

PACKAGE_FILE 0X00000020 Package file

ALLOC_MASK_DIRTY 0X00000040 Allocation map has been updated and needs to be flushed in the disk

DATA_DIRTY 0X00000080 The container is updated and needs to be flushed

MODIFIED_ACL_PERSISTED 0X00001000 The corresponding security descriptor (of the FSD Cache) has been updated and is now in the package volume file.

A value of 0 implies “Not Applicable”.

SoftricityFileTypeFlags This is a bit mask of persistent flags that states additional information of the data structures pointed to by this container:

Flag Value Purpose

FILETYPE_APP_DATA 0x0001 Application data file (i.e. dll, exe etc.)

FILETYPE_APP_CONFIG 0X0002 Application configuration file

FILETYPE_USER_DATA 0X0004 User data file (i.e. txt file)

FILETYPE_USER_CONFIG 0X0008 User configuration file

FILETYPE_PERMANENT 0X0010 The corresponding file/directory cannot be deleted

FILETYPE_OVERRIDE 0X0020 The corresponding file/directory can be overridden

FILETYPE_DELETED 0X0040 The corresponding file/directory has been deleted

A value of 0 implies “Not Applicable”.

Data This member points to a data region within the container, the structure of which depends on the container TypeId. The data region can be further divided into 2 parts: Data Information Region, and Data Storage Region. The container type specific data information region is used to uniquely



describe a structure the container points to. Its format is based upon the Container TypeId. Its size is defined by InfoDataLength value. After the data information region, data storage region takes up the remainder of the container buffer, and it begins immediately after the data information region. The DataStorageType and TypeId values of the container determine the content of Data Storage Region.

3.3 PERMANENT FIRST 16 CONTAINERS Volume Container Primary This container contains information necessary to initialize the volume. The type of this container is VOLUME_CONTAINER_P. In addition to the common header, the container has the following volume specific information:

CSRInfo 20 bytes

VolumeVersion 4 bytesFlags 4 bytesCSRBlockSize 4 bytesSTBlockSize 4 bytesFileSize 4 bytes

Common Header 90 bytesData

CSRData 62 bytes

CreationTime 8 bytesVolumeLabelLength 4 bytesVolumeLabel 50 bytes

Unused Bytes

Figure 4: Volume Container Primary

Data Information RegionCSRInfo Structure: VolumeVersion - Volume file version number Flags - Stores one of the following flags:

Flag Value Purpose

CSRINFO_DIRTY_STATE 0x00000001 The volume is in use. If the volume is closed but this flag is set then the volume has not been shut down properly.

CSRINFO_ACTIVITY_STATE 0X00000002 The volume has reference opened

0X00000000 The volume is closed

CSRBlockSize - Size of a Container; Default is 512 bytes. STBlockSize - Size of a Storage Block, Default is 4096 bytes.



FileSize – The maximum size (in MB) that this volume can be grown without extending the storage allocation map.

Data Storage RegionCSRData Structure: CreationTime - Time (UTC) when this volume is created. VolumeLabelLength - Number of bytes in the VolumeLabel including the NULL double

byte character. VolumeLabel - Volume Name in WCHAR; Default is “AppVirt”. This is an array of fixed

size 50 bytes.

Volume Container Secondary This container is a duplicate of the Volume Container Primary to be used in the future for fail over of the Volume Container Primary. This container is of type VOLUME_CONTAINER_S.

Container Allocation Map ContainerThis container points to the Container Allocation Map structures. The TypeId of this container is CONTAINER_ALLOCATION_MAP. The Container Allocation Map is a permanent storage for a Container Bitmap that describes the layout of the CSR containers on-disk. The DataStorageType of this container should be a bitmap of FORMAT_RAW | FORMAT_BLOCKMAP.

AllocMapInfo: 8 bytes

BlockCount 4 bytesMapLen 4 bytes


StorageBlockInfo: 8 bytes

AllocationIndex 4 bytesBlockCount 4 bytes



Unused Bytes (if any)

Figure 5: Container Allocation Map ContainerData Information Region

AllocMapInfo Structure: BlockCount - Total number of Containers allocated for this volume



MapLen - Unused; currently set to 0.In order to completely decode an App-V volume, all the containers of the volume needs to be processed. The physical layout of all the containers of a volume is specified by one or more StorageBlockInfo which reside in the Data Storage Region of this container (Please see figure 6, 12 and 13 about how to process this bitmap and creating a container table).

Data Storage RegionThis region contains one or more StorageBlockInfo structures which describe the starting storage index of a contiguous run of containers. If more containers are added to the volume, one or more StorageBlockInfo structures are added to this region. If this region is completely full with the StorageBlockInfo structures, a new storage block (4096 bytes default) will be assigned to hold the rest of the StorageBlockInfo structures.

StorageBlockInfo Structure: AllocationIndex - The start index of a contiguous run of Containers. BlockCount - Indicates the number of CSR Containers referenced by AllocationIndex.

There are 2 steps in processing the Container Allocation Map:1. Process all the StorageBlockInfo structures. Figure 6 shows how to process all the

StorageBlockInfo structures starting from the first one. Here initial it is assumed that the pointer points to the first StorageBlockInfo structure with in the container.

2. Each StorageBlockInfo structures points to a contiguous run of containers. Process all those containers for each of the StorageBlockInfo (Figure 12). In this step also build an in-memory Container Table (Figure 13).



NoStep 3

AllocMapInfo: 8 bytesBlockCount 4 bytesMapLen 4 bytes


StorageBlockInfo: 8 bytesAllocationIndex 4 bytesBlockCount 4 bytes


NoStep 2

AllocationIndex = SOFTFS_END_OF_BLOCK

(0xFFFFFFFF)?

We have processed all the StorageBlockInfo entries of

this container

There are more StorageBlockInfo entries available for this container which reside in another storage block. The storage block index is specified here by BlockCount. The actual starting offset of the storage block within the volume

can be calculated using the following formula:Offset = 16 (# of first 16 containers) * size of a container(512 default) +

BlockCount * size of a storage block(4096 default)

AllocationIndex = 0 and BlockCount = 0?

BlockCount > 0?

Yes; Step 4

Yes; Step 6

No, Step 5

Yes

Step 1



Storage Block layout

Container Allocation Map Container


Storage Block containing a set of StorageBlockInfo structures


This is valid StorageBlockInfo structure; advance the current pointer

to point to the next StorageBlockInfo structure

and goto step 1

This is a storage block containing a set of StorageBlockInfo structures. Set the current

StorageBlockInfo pointer to the start offset of this block, and process the StorageBlockInfo structures by

repeating the previous steps starting with step 1

Figure 6: Flowchart of how to process all StorageBlockInfo of this container

Security Descriptor Container



This container points to an inline security descriptor table or a set of StorageBlockInfo structures which point to the storage blocks which contain the security descriptor table. The type of this container is SECURITY_DESCRIPTOR.

This container has no Data Information Region so the InfoDataLength will be set to 0. If DataStorageType is true for FORMAT_BLOCKMAP flag, then right after the common header, there can be one or more StorageBlockInfo structures which point to the actual storage blocks that contain the security descriptors. If DataStorageType is true for FORMAT_INLINE flag, the security descriptors are within this container, right after the common header.

Security Descriptor

_SECURITY_HDR 12 bytes

TypeId 4 bytesEntryLength 4 bytesDataLength 4 bytesData Variable


Security Descriptor



Security Descriptor ContainerDataStorageType is set to

FORMAT_BLOCKMAP







Figure 7: Security Descriptor Container (BlockMap format)

Data Storage RegionThis region contains either StorageBlockInfo structures or _SECURITY_HDR structures with Windows NT security descriptors.

_SECURITY_HDR Structure: TypeId - Indicates whether this SD is in use or free. This value can be one of the

following 2 flags:



Flag Value Purpose

SECURITY_ENTRY 0x00000001 This is a valid SD

SECURITY_EMPTY 0X00000002 A security entry was created before but is now empty in order to reuse the space

0X00000000 The corresponding space is empty

EntryLength - Stores the total size of the SD including _SECURITY_HDR. DataLength - Stores the size of the SD. Data - Stores a reference to the beginning of the SD.



Security Descriptor



Security Descriptor



Security Descriptor ContainerDataStorageType is set to FORMAT_INLINE

Figure 8: Security Descriptor Container (Inline format)

For the purpose of linking directory entries (either directories or files) to the correct security descriptor entry, each directory entry contains an index which is a relative offset to a security descriptor entry. This index is an offset relative to the beginning of the Security Descriptor Table + 1. For example, the first security entry header starts at byte offset 0; the index (from a container) to it contains 1, and so forth, as illustrated below. If the SecurityDescriptorIndex of a container is 0, then the container doesn’t have any corresponding security descriptor.



Security Descriptor

Security Descriptor

Security Descriptor

Security Descriptor

. . .S

ecu

rtiy

Desc

rip

tors

Sub

sect

ion

Directory Entry

Directory Entry

Directory Entry

Offset + 1

Directory entries refer to security descriptors using relative offsets + 1

Figure 9: Security Descriptor mapping for a file or directory

Ideally, the security descriptors should be contiguous from the beginning of the security descriptor table. However, because of the deletion and creation of the security descriptors, they become scattered throughout the security descriptor table. Hence, while decoding a volume, sometimes it becomes necessary to find a security descriptor by reverse mapping a container’s SecurityDescriptorIndex into the security descriptor table and then extract the SD from the table.

Storage Allocation Map Container This container points to the Storage Block Allocation Map bitmap where each bit describes whether a Storage Block is in use or free. It is not necessary to parse this structure if the volume is decoded only in the read-only mode. The type of this container is STORE_ALLOCATION_MAP. The structure of this container is same as that of Container Allocation Map Container with the following changes in interpretation of the AllocMapInfo structure:

BlockCount - Total number of Storage Blocks allocated to store the bitmap. Maplen - Total size of the Bitmap in bytes.

This bitmap can be represented in memory using RTL_BITMAP structure.

Root Directory Container The purpose of this container is to provide a permanent storage for the root directory of an App-V volume. It provides a starting point to enumerate an App-V volume. The name of this directory is always “\”. The following is a depiction of the tree structure of the file system:



(Root)\Office14.win7\

…VFS\…

VSTS2010.win7\…VFS\…

The TypeId of this container is PACKAGE_DIRECTORY. There is no Data Information Region for this container; that is the InfoDataLength field is set to 0. Based on the DataStorageType the container can have 2 different types of data storage region. If DataStorageType is true for FORMAT_BLOCKMAP flag, then right after the common header, there will be a _DIRECTORY_STRUCT_PRIMARY structure which describes the root directory. After the _DIRECTORY_STRUCT_PRIMARY structure there can be one or more StorageBlockInfo structures which point to the actual storage blocks that contain the rest of the _DIRECTORY_STRUCT_PRIMARY which are children to this directory. If DataStorageType is true for FORMAT_INLINE flag, any child _DIRECTORY_STRUCT_PRIMARY will be inline within this container after the root _DIRECTORY_STRUCT_PRIMARY.




_SOFTFS_STORAGE_BLK_INFO: 8 bytes


_SOFTFS_STORAGE_BLK_INFO: 8 bytes



_DIRECTORY_STRUCT_PRIMARY (Child 1)


Root Package Directory ContainerDataStorageType is set to

FORMAT_BLOCKMAP

_DIRECTORY_STRUCT_PRIMARY VariableStructureTypeId 4 bytesDirDataLen 4 bytesDirDataOffset 4 bytesContainerIndex 4 bytesOwnerId 4 bytesReserved (Unused) 4 bytesCreationTime 8 bytesModificationTime 8 bytesAccessTime 8 bytesAllocationLength 8 bytesValidDataLength 8 bytesSoftDataLength 8 bytesPrimaryGroupId 4 bytesFileAttributes 4 bytesSoftricityFileTypeFlags 2 bytesShortNameLength 2 bytesFileNameLength 2 bytesShortName 26 bytesFileName Variable


_DIRECTORY_STRUCT_PRIMARY (Child n)






Figure 10: Layout of Root Package Directory Container (Blockmap)



Common Header 90 bytes



Root Package Directory ContainerDataStorageType is set to FORMAT_INLINE

_DIRECTORY_STRUCT_PRIMARY (Root)



Figure 11: Layout of Root Package Directory Container (inline)

Data Storage Region_DIRECTORY_STRUCT_PRIMARY Structure:This structure contains the metadata of a file or a directory. The structure consists of the following information:

StructureTypeId – This value indicates the type of the DirectoryPrimary structure. The value can be one of the following values:



Flag Value Purpose

PACKAGE_FILE 0x00000001 Points to a package file data structure

PACKAGE_DIRECTORY 0X00000002 Points to a package directory data structure

RUN_FILE 0X00000003 Points to a runtime generated file data structure

RUN_DIRECTORY 0X00000004 Points to a runtime generated directory data structure

PACKAGE_DELETED 0X00000005 The corresponding file or directory has been marked as deleted (This actually does not delete the actual file or directory rather hides the file or directory from directory enumeration)

CONTAINER_EMPTY 0X00000008 Empty container

LINK_DATA 0X0000000D This structure points to a symbolic link. In this case we cannot use this structure any longer and we need to cast to _DIRECTORY_STRUCT_SECONDARY structure (described later)

DirDataLen – This is the total size of this structure.

DirDataOffset - This is the offset of this structure entry within the parent directory entry table (the children directory structures that reside within a parent as depicted in the figure 10 and 11).

ContainerIndex – This is the index of the corresponding Container that describes this structure.

OwnerId – Security entity of the owner of this file or directory.

Reserved – Unused.

CreationTime – The time when the file/directory was created.

ModificationTime – The time when the file/directory was last modified.

AccessTime – The time when the file/directory was last accessed.

AllocationLength - Current allocation length of this file.

ValidDataLength – This indicates the last byte written for the given file or directory; information beyond the Valid Data Length will read as zero values. For directory type, this value is always 0.

SoftDataLength – This indicates the actual size of the file’s valid data length, as it is stored on App-V Streaming Server.

PrimaryGroupId – The security identifier of the primary group for the file/directory (this is principally for POSIX support, where the group ID is mandatory.)

FileAttributes – These are the standard Windows file attributes.Microsoft Corporation. All rights reserved. By using or providing feedback on these materials, you agree to by accepting the license agreement.


SoftricityFileTypeFlags – Same as Container::SoftricityFileTypeFlags.

ShortNameLength – The length of the short file name (in bytes) not including the double byte terminating NULL character.

FileNameLength – The length of the long file name (in bytes) not including the double byte terminating NULL character.

ShortName – Short file name in WCHAR. This is an array of characters with fixed size 13 (26 bytes).

FileName – Long file name in WCHAR.

_DIRECTORY_STRUCT_SECONDARY Structure: This structure is only used if the structure type is LINK_DATA. This structure is just a shorter version of the _DIRECTORY_STRUCT_PRIMARY structure. The members are as follows:

StructureTypeId DirDataLenDirDataOffsetContainerIndexPrimaryContainerId FileNameLengthShortNameLength ShortNameFileName

Reserved Container This container is of type SOFTFS_CONTAINER_RESERVED. All the data fields of this container are 0 except for the TypeId. Currently these containers are unused. Reserved containers should only reside in the volume from container flush index 5 to 14.

3.4 REMAINING CONTAINERS AFTER THE FIRST 16After the first 16 containers, the rest of the containers need to be processed by decoding the Container Allocation Map. Figure 12 shows how to process containers following Container Allocation Map:



AllocMapInfo

Common Header

StorageBlockInfo

AllocationIndex = 40BlockCount = 23

Container Allocation Map Container

_CONTAINER: 1

_CONTAINER: 2

_CONTAINER: 8

_CONTAINER: 17

_CONTAINER: 18

_CONTAINER: 23

StorageBlockInfo

AllocationIndexBlockCount


Storage Block with index = 40

Storage Block with index = 42

The start offset of the storage block 40 can be calculated as follows:Offset = 16 (# of first 16 containers) * size of a container (512 default) + AllocationIndex * size of a storage block (4096 default)

Unused Byte

Figure 12: Processing containers following CSR Allocation Map.

Figure 13 depicts a simple algorithmic view of processing of all the containers. In the figure we created a Container Table in memory in order to enumerate all the files and folders starting from root.



Process the first 16 containers and create a container table in memory

where each cell in the table corresponds to a container

Container 0 (Volume Container)

Container 1 (Volume container backup) Container 2 (CSR Allocation Map)

…

Container 15 (Root Container)

Container 16 (Varies, normally a

package) …

…

Container n

_CONTAINER::ContainerIndex = 0





_CONTAINER::ContainerIndex = n

Parse the CSR Allocation Map and expand the Container Table

Process the remaining containers following the CSR bitmap and create

a complete Container Table

Container Table

Figure 13: A simple algorithmic view of processing all the containers

The different types of containers which can be processed by Container Allocation Map are described below:

Container type PACKAGE_DIRECTORY and RUN _DIRECTORY

As stated in the earlier section, these 2 types of containers reference directories. The structure of this container is similar to Root Package Directory Container with one exception. This container does not have a fixed _DIRECTORY_STRUCT_PRIMARY structure which describes itself as seen in the case of Root Package Directory Container. Since the Root Package Directory Container does not have any parent, the Root Package Directory Container itself contains the _DIRECTORY_STRUCT_PRIMARY structure which contains the metadata of the root directory. For the rest of the directories, the _DIRECTORY_STRUCT_PRIMARY containing the metadata of the directories reside within the parent directory entry table.













_DIRECTORY_STRUCT_PRIMARY (Child n+1)


_DIRECTORY_STRUCT_PRIMARY (Child n+2)

_DIRECTORY_STRUCT_PRIMARY (Child n+x)

Storage Block layoutContainer type: RUN_DIRECTORY,

PACKAGE_DIRECTORYDataStorageType is set to FORMAT_BLOCKMAP

Figure 14: Layout of a directory container when DataStorageType is blockmap

Common Header



Container type: RUN_DIRECTORY, PACKAGE_DIRECTORYDataStorageType is set to FORMAT_INLINE



Figure 15: Layout of a directory container when DataStorageType is inline



The following figure shows how to enumerate a volume starting from the root. Processing the data blocks of a file has been described in the later file container section.



PACKAGE_FILE

RUN_FILE

Common Header

Unused Bytes

Child 2 (RUN_FILE)

Find the corresponding container from Container

Table by _DIRECTORY_STRUCT_PRIMARY::

ContainerIndex

PACKAGE_FILE or RUN_FILE

Container Layout

Common Header



Root Directory with FORMAT_INLINE

_DIRECTORY_STRUCT_PRIMARY (Root)



Common Header



Child 1 (Directory)with FORMAT_INLINE



Find the corresponding container from Container

Table by _DIRECTORY_STRUCT_PRIMARY::

ContainerIndex

PACKAGE_DIRECTORY or RUN_DIRECTORY

File Data Index Table

Common Header

Unused Bytes

Child 2 (PACKAGE_FILE)

File Data Index Table

PackageFileInfo: 8 bytesFeatureSetBlockIndex: 4 bytesFeatureSetBlockCount: 4 bytes

Figure 16: Enumerating directories and files starting from root where the format of directory structure is INLINE



Container type CONTAINER_PACKAGEThe Package Container is used to track a set of files, directories, and other information specific to an App-V package. This container has both a Data Information Region and a Data Storage Region.

Data Information RegionData Information Region within the Container referenced by Data member contains the data of type _PACKAGE_INFO and InfoDataLength member of the container is initialized to the size of _PACKAGE_INFO. The structure _PACKAGE_INFO consists of the following information:

PackageGUID – A unique identifier of the package (GUID).

AccessTime – Indicates last time the package was accessed.

PackageSize – The sum of the sizes of all files that are part of the package.

LaunchSize – The actual size of the launch specific data required for the launch of the package.

CachedFileSize – The size of package file data resident in FSD Cache Volume File.

CachedLaunchSize - The size of package launch specific file data resident in FSD Cache Volume File.

PercentLaunch – The percentage of the launch data cached.

PercentTotal – The percentage of the total data cached.

PackVersion – Indicates a version of this package; used during package upgrade.

PackageName – Name of the package.

DirVersion – Indicates a version of package metadata; used during package upgrade.

PackVersionGUID - package version GUID; used during package upgrade.

Rank – Indicates package rank relative to other packages (Obsolete).

MetaDataLength – The length of the buffer that consists of metadata descriptors for all files and directories that belong to the package.

SecurityDataLength - The length of the buffer that consists of security descriptors for all files and directories that belong to the package.

ServerBlockSize – The size of the data storage block used by App-V streaming server.



Reference – Indicates how many instances of the package are opened.

Flags –Package specific flags; the value can be a bitmap of the of the following flags:

Flag Value

PACKAGE_META_DATA_INITIALIZED 0x00000001

PACKAGE_POPULATE_FAILED 0X00000002

PACKAGE_NAME_COLLISION 0X00000004

PACKAGE_PREALLOCATED 0X00000008

VersionHistoryCount - Used during package upgrade.

VersionHistoryBufferIndex - Used during package upgrade.









_PACKAGE_FILE_GUID_INFO 20 bytes

FileGUID 16 bytesIndex 4 bytes

Unused Bytes

Container type: CONTAINER_PACKAGE

_PACKAGE_INFO 248 bytes

PackageGUID 16 bytesAccessTime 8 bytesPackageSize 8 bytesLaunchSize 8 bytesCachedFileSize 8 bytesCachedLaunchSize 8 bytesPercentLaunch 4 bytesPercentTotal 4 bytesPackVersion 4 bytesPackageName 128 bytesDirVersion 4 bytesPackVersionGUID 16 bytesRank 4 bytesMetaDataLength 4 bytesSecurityDataLength 4 bytesServerBlockSize 4 bytesReference 4 bytesFlags 4 bytesVersionHistoryCount 4 bytesVersionHistoryBufferIndex 4 bytes








Unused Bytes

Figure 17: Layout of a package container

Data Storage RegionsPackage Container maintains a Package Table that stores information about all files and directories that belong to the package or were created in context of the package. The Package Table is stored using a Block Map format. The Data Storage Region is used as a placeholder for Storage Block Table, where each entry in Storage Block Table is of type StorageBlockInfo, as described earlier. Last entry in Storage Block Table, if necessary, points to a next Storage Block, which is used as a placeholder for the next Storage Block Table. Each entry in a Storage Block Table points to a Storage Block that is used as a placeholder for Package File Guid Table. Each entry in Package File Guid Table is of type _PACKAGE_FILE_GUID_INFO, where _PACKAGE_FILE_GUID_INFO structure consists of the following members:



FileGUID – A unique identifier of the file/directory that belongs to the package.Index – Index of the corresponding container of the package file/directory.

Container type RUN_FILERun-File container maintains a data index table which starts just after the container common header. Since there is no Data Information Region, the InfoDataLength member of the container is always set to 0. The format of the data index table (_SOFTFS_IND_FILE_BLOCK_ROOT_DATA) is as follows:

FileDataTable - An array (size 20) of storage indexes (4 bytes each) that reference storage blocks that describe the first segment of the file, up to 80KB by default. IndirectionTableIndex - This is an index (4 bytes) to a storage block which contains an array of storage indexes (maximum 1024) that describe 2nd segment, up to 4MB.DblIndirectionTableIndex - This is an index (4 bytes) to a storage block which contains an array of storage indexes (up to 1024). Each of the indexes points to another storage block of data indexes which point to the actual data blocks. Using DblIndirectionTableIndex the maximum addressable data size is 4GB.TrplIndirectionTableIndex - This is another indirection layer on top of DblIndirectionTableIndex by which the maximum addressable data size is 4TB.

In order to get the data of a file, we need to get each data index which points to an actual data block. Figure 18 portrays a file data layout and how to get the actual data blocks. For any volume other than FSD cache, if a data index contains 0, then the corresponding data block currently is not in this volume and we need to look at the corresponding data index table cell of FSD cache to get that data block. If the data index table cell of FSD cache is 0 then that data block is not in the cache and it must be streamed from App-V server into the FSD cache. Please note that Per User Package Volumes will only contain files of type SOFTFS_CONTAINER_RUN_FILE where as Per System Package Volumes can contain both SOFTFS_CONTAINER_RUN_FILE and SOFTFS_CONTAINER_PACKAGE _FILE types of files.




Unused Bytes

Container type: RUN_FILE

_SOFTFS_IND_FILE_BLOCK_ROOT_DATA 92 bytes

FileDataTable 4 * 20 bytesIndirectionTableIndex 4 bytesDblIndirectionTableIndex 4 bytesTrplIndirectionTableIndex 4 bytes

Data Block Index 1

Data Block Index 2

…..

Data Block Index 20

Data Block (4096 bytes default)


Data Block Index 1

Data Block Index 2

…..

Data Block Index 1023



Data Block Index 1

Data Block Index 2

…..




Data Block Index 1

Data Block Index 2

…..


Data Block Index 1

Data Block Index 2

…..




Data Block Index 1

Data Block Index 2

…..




Data Block Index 1

Data Block Index 2

…..


Data Block Index 1

Data Block Index 2

…..


Data Block Index 1

Data Block Index 2

…..


Data Block Index 1

Data Block Index 2

…..




Figure 18: Layout of a run file container

Container type SOFTFS_CONTAINER_PACKAGE_FILEThe format of this container is basically same as that of RUN_FILE container. The only difference between the two is that this container has a data information region which contains a PackageFileInfo (8 bytes) structure. The purpose of this structure is to store the reference of the storage block that maintains Feature Map Table type FeatureSetBlock. Basically it describes what feature data blocks that make up the file belong to. For example, feature block 1, 2, 3 ..., where feature block 1 is launch specific data. The structure has the following members:

FeatureSetBlockIndex - Index of feature set (4 bytes)FeatureSetBlockCount - Stores the number of storage blocks allocated for FeatureSetBlock.(4 bytes)

This structure is not needed in order to extract the data of a file.

Container type CONTAINER_EMPTYThis container was previously created but is no longer needed. In order to reuse it in the future it is marked as empty. It may or may not point to any valid data structure and will be reused when needed. This container should be ignored from processing.

4 APPENDIX

Special flagsINVALID_VOLUME_ID 0xFFFFFFFFINVALID_PACKAGE_ID 0xFFFFFFFF INVALID_CSR_INDEX 0xFFFFFFFF SOFTFS_END_OF_BLOCK 0xFFFFFFFF


App-V Volume Format Specification

Documents