FspFsctlTransactCreateKind, FspFsctlTransactOverwriteKind, FspFsctlTransactCleanupKind, FspFsctlTransactCloseKind, FspFsctlTransactReadKind, FspFsctlTransactWriteKind, FspFsctlTransactQueryInformationKind, FspFsctlTransactSetInformationKind, FspFsctlTransactQueryEaKind, FspFsctlTransactSetEaKind, FspFsctlTransactFlushBuffersKind, FspFsctlTransactQueryVolumeInformationKind, FspFsctlTransactSetVolumeInformationKind, FspFsctlTransactQueryDirectoryKind, FspFsctlTransactFileSystemControlKind, FspFsctlTransactDeviceControlKind, FspFsctlTransactShutdownKind, FspFsctlTransactLockControlKind, FspFsctlTransactQuerySecurityKind, FspFsctlTransactSetSecurityKind, FspFsctlTransactQueryStreamInformationKind,
WinFsp Kernel Mode File Systems
Since release 2019.3 WinFsp supports development of file systems in kernel mode. Such file systems are implemented as kernel drivers that use WinFsp as the primary FSD (File System Driver) as well as the software library that they interface with to retrieve and service file system requests. This document discusses how to write such file systems.
The primary goal of WinFsp is to enable the easy creation of user mode file systems using an easy to use API. There are however legitimate reasons for wanting to develop a file system as a kernel mode driver, but the difficulty of doing so often deters people because Windows kernel file system development is notoriously difficult and has many pitfalls.
Some of the reasons that a kernel mode file system may be preferrable to a user mode one:
A kernel driver may be able to achieve better performance than user mode processes.
A kernel driver may be able to access advanced OS features that are not easily available to user mode processes.
A kernel driver may be able to present an alternative interface than the one presented by WinFsp to user mode processes.
Since release 2019.3 WinFsp supports development of file systems as kernel mode drivers. The primary motivation for this work was to support the WinFuse project, which exposes the FUSE protocol from the Windows kernel in a way that allows FUSE file systems to interface with it, either directly or via libfuse. The support was added in such a way that it is generic enough to be used by other kernel mode file systems.
A WinFsp "volume" (file system) is typically created using the
FspFsctlCreateVolume API. This is simply a wrapper around a
CreateFileW call on one of the WinFsp control devices, either
CreateFileW call returns a
HANDLE to the newly created volume, which acts either as a "disk" or a "network" file system, depending on which control device was used to create it.
User mode file systems interact with the WinFsp FSD via
DeviceIoControl/FSP_FSCTL_TRANSACT messages on the volume
HANDLE. (This is usually done indirectly via the WinFsp DLL, which hides this detail behind an easy to use API.)
Since release 2019.3 (v1.5) WinFsp supports the following:
Registration and on-demand loading of a third party driver when a volume that is destined to be handled by the third party driver is created.
Forwarding of custom
DeviceIoControlmessages to a third party driver. This allows the third party driver to handle custom
FSCTLrequests directed to a WinFsp volume
Kernel-mode API’s (called DDI’s) that allow a third party driver to register itself with the FSD and interact with its I/O queues.
Such drivers are called within the WinFsp sources and header/library files by the name of "Fsext Providers" (File System Extension Providers). In the text below we will usually refer to them as simply "Providers".
A Provider is a kernel mode driver that uses the FSD as a frontend file system driver to interface with Windows and implement all the complex details that this entails. The Provider fetches I/O requests from the WinFsp I/O queues and may filter them, transform them into a different protocol (as is the case with WinFuse) or use them to fully implement a file system in kernel mode.
The WinFsp installer includes a "Kernel Developer" feature. When installed this feature adds header and library files in the
opt\fsext installation subdirectory.
The primary header file used for Provider development is
<winfsp/fsext.h> and can be found in
opt\fsext\inc. Additionally the file
<winfsp/fsctl.h> found in the
inc installation subdirectory must be in the compiler’s include path.
Providers must also be linked with one of the import libraries that can be found in
opt\fsext\lib. Use the
winfsp-x64.lib import library when linking the 64-bit version of a Provider and the
winfsp-x86.lib import library when linking the 32-bit version of a Provider.
<winfsp/fsext.h> header file includes definitions for the following important DDI’s:
FspFsextProviderRegister: This DDI is used by a Provider to register itself with the FSD. Typically the Provider prepares an
FSP_FSEXT_PROVIDERstructure and calls
DriverEntryroutine. The structure’s fields are as follows:
Version: Set to
DeviceIoControlcode that should be forwarded to the Provider.
The code must be defined as follows:
CTL_CODE(FILE_DEVICE_FILE_SYSTEM, 0xC00 + ProviderSpecific, METHOD_BUFFERED, FILE_ANY_ACCESS).
Please note that this scheme allows for up to 1024 codes. If you want to define a new Provider code I will appreciate it if you email me at
<billziss at navimatics.com>so that we can keep track of Provider codes, avoid conflicts and see when we need to extend this scheme.
DeviceExtensionSize: The size of private data that the Provider wants for itself in the
DeviceExtensionof the FSD volume’s device object.
DeviceInit: Called when a new volume is created. The
DeviceObjectparameter is the volume’s device object. The
VolumeParamsare the initial parameters passed to
FspFsctlCreateVolumeand can be modified by the Provider.
DeviceFini: Called when a volume is going away. This usually happens when the volume
DeviceExpirationRoutine: Called once a second as part of the FSD’s expiration timer. The FSD uses this timer to expire caches, long-pending IRP’s, etc. A Provider can use this call for a similar purpose. The
ExpirationTimeparameter contains the current interrupt time as determined by the FSD.
DeviceTransact: Called whenever the FSD receives a
DeviceIoControlrequest with the
Irpparameter contains the relevant
DeviceExtensionOffset: Set to
0on input. On successful return from
FspFsextProviderRegisterit will contain the offset to use for accessing the Provider’s private data in the
DeviceExtensionof the FSD volume’s device object. Given a
DeviceObject, the data can be accessed as
(PVOID)((PUINT8)DeviceObject→DeviceExtension + Provider.DeviceExtensionOffset).
FspFsextProviderTransact: This DDI is used by a Provider to interact with the FSD I/O queues. The
DeviceObjectis the FSD volume’s device object. The
FILE_OBJECTthat corresponds to the volume
Responseis the response to send and can be
Requestis a pointer that receives a pointer to a new request from the WinFsp I/O queue and can be
NULL; if the received pointer is not
NULLit must be freed with
When the FSD receives a file system IRP it is often able to handle and complete it without help from an external user mode or kernel mode file system. However in most cases the IRP has to be seen and acted upon by an external file system. For this reason the FSD preprocesses the IRP and places it in an I/O queue in the form of a "request". At a later time the external file system retrieves the request, processes it and sends back a "response". The FSD uses the response to locate the original IRP, perform any necessary postprocessing and finally complete the IRP.
Providers typically use the
FspFsextProviderTransact DDI to receive requests and send back responses. Requests are of type
FSP_FSCTL_TRANSACT_REQ and responses are of type
FSP_FSCTL_TRANSACT_RSP. Requests have a
Kind field which describes what kind of file system operation is being requested. The following request kinds are currently defined in
When request processing is complete the Provider must prepare a response and send it to the FSD using
FspFsextProviderTransact as mentioned above. It is particularly important that the Provider initializes the
Hint fields by copying the values from the corresponding request.
This document does not describe in detail how each request kind is supposed to be handled. For the full details refer to the implementation for the WinFsp DLL in the WinFsp sources:
src/dll/fsop.c. Although this implementation is for user mode file systems, similar logic and techniques should be used for Providers.
Providers are loaded on demand and must be properly registered:
A provider must be registered as a kernel driver. This can be achieved by using the command
sc create PROVIDER type=kernel binPath=X:\PATH\TO\PROVIDER.SYSor by using the Service Control Manager API’s (
CreateServiceW, etc.). You do not need an INF file or to use the Setup API in order to register a Provider driver.
A provider must be registered under the registry key
HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\WinFsp\Fsext. Create a string value with name the textual representation of the Provider’s transact code (see
"%08lx"format and value the Provider’s driver name.
For example the WinFuse Provider registers its driver under the name
WinFuse and adds a registry value of
Providers are loaded on demand by the FSD during volume creation. This process works as follows:
During volume creation (e.g. by using
FspFsctlCreateVolume) a non-zero
FsextControlCodemust be specified in
If the FSD sees the
FsextControlCodeas non-zero it attempts to find a corresponding Provider driver.
It first checks an internal mapping of codes to Provider drivers. If the code is found, the FSD proceeds to the
If the code is not found in the internal mapping, the FSD checks the registry under the registry key
HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\WinFsp\Fsext. If the code is not found the volume creation fails.
If the code is found the FSD loads the Provider driver using
ZwLoadDriver. The Provider is supposed to register itself with the FSD during
Finally the internal mapping of codes to Providers is rechecked. Assuming that everything worked as intended, the corresponding Provider driver is now loaded and we can proceed to the
The FSD proceeds to call the
DeviceInitcallback of the Provider. The Provider can use this call to initialize itself in relation to the new volume device object.
Assuming that the volume device object is created successfully, the FSD will do the following:
FsextControlCode==DeviceTransactCoderequests that it gets in its
IRP_MJ_FILE_SYSTEM_CONTROLto the Provider via
Call the Provider’s
DeviceExpirationRoutineonce a second as part of the FSD’s expiration process.
Eventually the volume device object will be torn down (e.g. because the corresponding
HANDLEis closed). In this case the FSD will call the Provider’s
Finally note that once loaded a Provider driver cannot be unloaded (without a reboot).