How to develop a virtual disk for Windows

List of contents
1. Introduction
1.1 Why to process requests in user mode?
1.2 Article structure
1.3 Technologies used
1.4 Project structure
1.5 Project directory structure
2. Windows and disks
3. Implementation
3.1 Stage 1. Initialization
3.2 Stage 2. Disk mounting
3.3 Stage 3. Processing requests
3.4 Stage 4. Unmounting
4. How to build this Solution
4.1 Testing
4.2 Supported Windows Versions
5. Conclusion

Introduction

This article is oriented to Windows-developers and considers how to create virtual disk in Windows system.

In Windows virtual disks are implemented by means of kernel mode drivers. Some necessary information concerning disk drivers for Windows is given in the section “Windows and disks”.

The implementation of virtual disk for Windows can also be found in the widely known project with open sources FileDisk (http://www.acc.umu.se/~bosse/ ). The main difference of the solution, which is given in this article, is that FileDisk processes requests to the virtual disk in kernel mode while the proposed solution processes them in user mode.

Why to process requests in user mode?

This approach can be useful when there is already developed code of user mode that provides access to the data source (there can be any data source, e.g. disk image in memory, remote disk, or cash register) and it’s hard to port this code to the kernel mode; or there is no source code at all, for example when performing network access or using specific encryption library.
As an example of such user mode library I used SparseFile (file-container with some data; as the data is accumulated it will size up to the maximal size).

Article structure

In section «Windows and disks» we will discuss the way the Windows interacts with the disks; possible variants of virtual disk creation; necessary information for disk driver development.

In section «Implementation» the solution architecture is considered as well as key implementation aspects and main stages of our disk’s life cycle.

The section «How to build this Solution» contains a couple of phrases about how to build and test the project.

Technologies used

• C++. The whole project with both kernel mode and user mode parts is developed in C++ with the utilizing of the exceptions. There are a lot of contradictory opinions concerning the utilizing of C++ for driver development. Here is the good article on this topic http://www.osronline.com/article.cfm?article=490.
• CppLib (http://www.acc.umu.se/~bosse/). This beautiful library made it possible to develop drivers in C++.
• STLPort (http://www.stlport.org/). A standard library.
• BOOST. (http://www.boost.org/). Really useful library.

Project structure

Solution includes several projects:

• CoreMntTest (user mode, executable) – creates disk image and mounts it using the code from CoreMnt_user.
• CoreMnt_user (user mode, library) – receives requests to virtual disks from the CoreMnt driver and serves them.
• UsrUtils (user mode, library) – contains auxiliary code of the interaction with drivers using DeviceIoControl.
• CoreMnt (kernel mode, executable) – implements OS requirements for the disks; performs request transformation; sends them to CoreMnt_user for serving.
• drvUtils (kernel mode, headers only library) – auxiliary code of kernel mode, for example synchronization tools.

The scheme below represents the projects’ relations.

Project directory structure

.\bin - folder with binary files
.\lib - folder with library files
.\obj - folder with object files
.\src - folder with source files
|
|-> .\CoreMnt       - Kernel mode driver.
|-> .\CoreMnt_user  - User mode mount library.
|-> .\CoreMntTest   - User mode mount test.
|-> .\drvCppLib     - Kernel Library to develop driver in C++.
|-> .\drvUtils      - Kernel Library with utils for kernel mode projects.
|-> .\mnt_cmn       - Files that are shared between projects.
|-> .\STLPort       - Directory with STLPort 4.6 ported for utilizing in windows drivers.
|-> .\usrUtils      - Win32 Library with utils for user mode projects.

Windows and disks

If you are acquainted with driver development for Windows and have developed at least one even the simplest driver then you can skip this section.

I’m glad to appease the rest – everything is very simple here. Windows sends a request “write” or “read” to the disk. The disk returns read data or error code. And that’s all.

Surely there are some nuances, how it can be without them.

Let’s consider a simplified scheme of processing of requests to the disk. So, what is going on after an application calls, for example, ReadFile function. First the read file request is received by the file system driver (for example ntfs.sys). The scheme illustrates this process:

File system driver detects where the requested file is exactly situated on the disk (with what offset) and forms the read disk request. A file can be divided into parts, which can be located at different places on the disk. In this situation several requests will be formed. Just those very requests will be received by our virtual disk driver from the file system driver. And by the way, virtual disk can be also implemented at the file system level, see details in the article fs-filter-driver-tutorial.aspx .

Terms listed below will be used in this article.

• IRP (I/O Request Packet) is a structure of Windows kernel that stores request parameters. For example, if we want to read data from device we will need to indicate request type, buffer to read data to, size and offset. With some reservations we can say that IRP is a request to some device. At least in this article when we speak about IRP we always speak about a request. More details can be found here http://www.microsoft.com/whdc/driver/kernel/IRPs.mspx.
• STATUS_PENDING is a special return code that notifies the request initiator about that IRP cannot be processed right now and will be processed later on. There is a termination event for this situation, device will set it when complete the request processing. Below we will consider the code that uses this return code.
• Device is an object of Windows kernel representing any device. It stress an information about this device, for example its name. It also contains DeviceExtension.
• DeviceExtension is a field in Device structure, which can be used by the device creator in his own way. Below we will consider the code that uses DeviceExtension.

Implementation

The solution itself is a driver (CoreMnt.sys) and application (CoreMntTest.exe). The general scheme is as follows.

Driver provides the service of disk mounting. Application creates data source and mounts it as a disk using the service. Driver receives IRP, process them in user mode and returns the result. General scheme of driver work is represented on the figure.

Application (CoreMntTest.exe) serves the requests from OS to the virtual disks. Structural scheme is represented on the figure.

Now let’s consider how it looks in the source code, stage by stage.

Stage 1. Initialization

On this stage we start CoreMnt driver by means of the command:

Collapse | Copy Code

c:\>sc start CoreMnt

We should create the management device in DriverEntry as an access point for CoreMntTest from user mode:

Collapse | Copy Code

NTSTATUS DriverEntry(IN PDRIVER_OBJECT DriverObject,
                     IN PUNICODE_STRING RegistryPath)
{
...
    NTSTATUS status;
    status = IoCreateDevice(DriverObject,     // pointer on DriverObject
                            0,                // additional size of memory
                            &gDeviceName,     // pointer to UNICODE_STRING
                            FILE_DEVICE_NULL, // Device type
                            0,                // Device characteristic
                            FALSE,            // "Exclusive" device
                            &gDeviceObject);  // pointer do device object
    if (status != STATUS_SUCCESS)
        return STATUS_FAILED_DRIVER_ENTRY;

    status = IoCreateSymbolicLink(&gSymbolicLinkName,&gDeviceName);
    if (status != STATUS_SUCCESS)
        return STATUS_FAILED_DRIVER_ENTRY;

Next step is the register the driver request handler. We will use a single handler for all request types:

Collapse | Copy Code

    for (size_t i = 0; i <= IRP_MJ_MAXIMUM_FUNCTION; ++i) 
        DriverObject->MajorFunction[i] = IrpHandler;

And finally let’s create MountManager:

Collapse | Copy Code

    gMountManager = new MountManager(DriverObject);
    return STATUS_SUCCESS;
}

Stage 2. Disk mounting

On this stage we start application CoreMntTest.exe. It sends management message CORE_MNT_MOUNT_IOCTL to the driver:

Collapse | Copy Code

    CORE_MNT_MOUNT_REQUEST request;
    request.totalLength = totalSize;
    request.mountPojnt = mountPoint;
    DWORD bytesWritten = 0;
    CORE_MNT_MOUNT_RESPONSE response;
    if(!m_coreControl.DeviceIoGet(CORE_MNT_MOUNT_IOCTL, 
                                  &request, 
                                  sizeof(request),
                                  &response,
                                  sizeof(response),
                                  &bytesWritten))
    {
        throw std::exception(__FUNCTION__" DeviceIoGet failed.&);
    }

Function DispatchMount deserialize request parameter and calls MountManager::Mount:

Collapse | Copy Code

        if(inputBufferLength < sizeof(CORE_MNT_MOUNT_REQUEST) || 
            outputBufferLength < sizeof(CORE_MNT_MOUNT_RESPONSE) )
        {
            throw std::exception(__FUNCTION__" buffer size mismatch");
        }
        DISK_PROPERTIES diskProperties;
        CORE_MNT_MOUNT_REQUEST * request = (CORE_MNT_MOUNT_REQUEST *)buffer;
        diskProperties.totalLength.QuadPart = request->totalLength;
        CORE_MNT_MOUNT_RESPONSE * response = 
            (CORE_MNT_MOUNT_RESPONSE *)buffer;
  response->deviceId = gMountManager->Mount(&diskProperties);

In MountManager::Mount we create an object of the class MountedDisk and save it. MountedDisk includes LogicIrpDispatcher. Its constructor creates disk device. OS will sends requests to this device:

Collapse | Copy Code

LogicIrpDispatcher::LogicIrpDispatcher(PDISK_PROPERTIES diskProperties, 
                                       PDRIVER_OBJECT   DriverObject,
                                       MountManager* mountManager)
{
...
    //create device
    status = IoCreateDevice(DriverObject,sizeof(InThreadDeviceExtension),
        &deviceName,FILE_DEVICE_DISK,
        0,
        FALSE,&deviceObject_);
    if (!NT_SUCCESS(status))
        throw std::exception(__FUNCTION__" can't create device.");

After the device is created we have to initialize DeviceExtension. We want to use it for storing device identifier. Thus when we get IRP we will easily find the corresponding MountedDisk:

Collapse | Copy Code

    InThreadDeviceExtension* devExt = 
        (InThreadDeviceExtension*)deviceObject_->DeviceExtension;
    memset(devExt, 0, sizeof(InThreadDeviceExtension));

    devExt->mountManager = mountManager;
    devExt->deviceId = diskProperties->deviceId;

So at the moment MountManager has created an instance of MountedDisk and saved it to the container. Initialization stage is finishing in the user mode. A thread is creates for each disk and all its requests are served there. A thread sends IOCTL RequestExchange to the driver and proceeds to the request awaiting mode:

Collapse | Copy Code

    while(true)
    {
        int type = 0;
        int size = 0;
        __int64 offset = 0;
        drvCtrl->RequestExchange(deviceId, lastType, lastStatus, lastSize, &dataBuf[0], dataBuf.size(),
                                 &type, &size, &offset);

        //do requested operation
        DispatchImageOperation(image, type, size, offset, &dataBuf[0], dataBuf.size(), &lastStatus);
        lastType = type;
        lastSize = size;
    }

Performance note: serving requests in one thread is surely the «bottle neck». There will be certainly a thread pool in the real project.

Stage 3. Processing requests

So, our virtual disk is ready to process requests. Let’s follow the complete sequence in request processing. It all begins with the function IrpHandler, which is registered by our driver as the procedure of IRP processing. Here we get the device identifier from DeviceExtension (we saved it there on initialization stage) and transmit IRP to MountManager:

Collapse | Copy Code

NTSTATUS IrpHandler( IN PDEVICE_OBJECT fdo, IN PIRP pIrp )
{
...
        InThreadDeviceExtension* devExt = 
            (InThreadDeviceExtension*)fdo->DeviceExtension;
        return gMountManager->DispatchIrp(devExt->deviceId, pIrp);

MountManager receives IRP, finds the corresponding MountedDisk by device identifier, and redirects IRP to it. The code below decides if it is possible to process this request at once or it should be processed in the user mode:

Collapse | Copy Code

NTSTATUS MountedDisk::DispatchIrp(PIRP irp)
{
    IrpParam irpParam(0,0,0,0);
    irpDispatcher_.getIrpParam(irp, &irpParam);
    if(irpParam.type == directOperationEmpty)
    {
...
            irpDispatcher_.dispatch(irp);
...
        NTSTATUS status = irp->IoStatus.Status;
        IoCompleteRequest(irp, IO_NO_INCREMENT);
        return status;
    }
    IoMarkIrpPending( irp );
    irpQueue_.push(irp);
    return STATUS_PENDING;
}

Decision making is simple: if it is IRP_MJ_READ or IRP_MJ_WRITE then it should be processed in user mode. Driver can process all other requests itself. For example IOCTL_DISK_GET_LENGTH_INFO: our driver knows the disk size, and it knows also that the disk size cannot be changed. The complete list of requests, which Windows can send to the disk, can be found in LogicIrpDispatcher::dispatchIoctl.

The thread that serves this disk selects requests from the list:

Collapse | Copy Code

void MountedDisk::RequestExchange(UINT32 lastType, UINT32 lastStatus, UINT32 lastSize, char* buf, UINT32 bufSize,
                     UINT32 * type, UINT32 * length, UINT64 * offset)
{
...
    NTSTATUS status = KeWaitForMultipleObjects(sizeof(eventsArray)/sizeof(PVOID), eventsArray, WaitAny, 
        Executive, KernelMode, FALSE, NULL, 0);
...
            IrpParam irpParam(0,0,0,0);
            irpDispatcher_.getIrpParam(lastIrp_, &irpParam);
            *type = irpParam.type;
            *length = irpParam.size;
            *offset = irpParam.offset;

If it is IRP_MJ_WRITE, data to write will be copied to the buffer. Then this buffer will be passed to the user mode code:

Collapse | Copy Code

        if(*type != directOperationEmpty 
            && opType2DirType(directOperationTypes(*type)) == directOperationWrite)
        {
            IrpParam irpParam(0,0,0,0);
            irpDispatcher_.getIrpParam(lastIrp_, &irpParam);

            if(irpParam.buffer)
                memcpy(buf, irpParam.buffer, *length);

After returning from RequestExchange function we will get to the cycle of request processing (DispatchImage) again:

Collapse | Copy Code

    while(true)
    {
        int type = 0;
        int size = 0;
        __int64 offset = 0;
        drvCtrl->RequestExchange(deviceId, lastType, lastStatus, lastSize, &dataBuf[0], dataBuf.size(),
                                 &type, &size, &offset);

        //do requested operation
        DispatchImageOperation(image, type, size, offset, &dataBuf[0], dataBuf.size(), &lastStatus);
        lastType = type;
        lastSize = size;
    }

The variables type, size, offset now contain the new request to be processed. It’s a task for DispatchImageOperation function:

Collapse | Copy Code

void DispatchImageOperation(IImage * image, 
                            int type, int size, __int64 in_offset, char* buf,
                            int bufsize,
                            int* status)
{
    switch(type)
    {
...
    case directOperationRead:
        {
           image->Read((char*)buf, in_offset, size);
           *status = 0;
            break;
        }
    case directOperationWrite:
        {    
            image->Write((const char*)buf, in_offset, size);
            *status = 0;
            break;
        }

After the request is served the function RequestExchange will be called again and the thread will proceed to the new request awaiting mode.

Stage 4. Unmounting

This stage starts in user mode with the call of UnmountImage function. Code below checks if the disk is being used at the moment:

Collapse | Copy Code

void UnmountImage(int devId, wchar_t mountPoint, DriverControl * drvCtrl)
{
...
        if (!DeviceIoControl(hVolume,FSCTL_LOCK_VOLUME,NULL,
                             0,NULL,0,&BytesReturned,NULL))
        {
            throw std::exception("Unable to lock logical drive");
        }
        else if (!DeviceIoControl(hVolume,FSCTL_DISMOUNT_VOLUME,
                                  NULL,0,NULL,0,&BytesReturned,NULL))
        {
            throw std::exception("Unable to dismount logical drive");
        }
        else if (!DeviceIoControl(hVolume,FSCTL_UNLOCK_VOLUME,NULL,
                                  0,NULL,0,&BytesReturned,NULL))
        {
            throw std::exception("Unable to unlock logical drive");
        }

Then we destroy the connection of mounting point with our device:

Collapse | Copy Code

    if (UndefineLogicDrive(mountPoint))
        throw std::exception("Unable to undefine logical drive");

And then we send a message for all those components that store the disk list in the system (for example explorer.exe or any other file manager):

Collapse | Copy Code

    SHChangeNotify(SHCNE_DRIVEREMOVED, SHCNF_PATH, root, NULL);

And finally we notify our driver that the device can be deleted:

Collapse | Copy Code

    drvCtrl->Unmount(devId);
  }

MountManager::Unmount simply deletes the corresponding MountedDisk form the container that causes its destructor call:

Collapse | Copy Code

MountedDisk::~MountedDisk()
{

We set the stop event for the thread of processing requests for this disk:

Collapse | Copy Code

    stopEvent_.set();

And we terminate all IRP, which were not served and are in a queue at the moment:

Collapse | Copy Code

    if(lastIrp_)
        CompleteLastIrp(STATUS_DEVICE_NOT_READY, 0);

    while(irpQueue_.pop(lastIrp_))
        CompleteLastIrp(STATUS_DEVICE_NOT_READY, 0);
}

The thread of request processing, which has been in the MountedDisk::RequestExchange in the awaiting state, reacts on stopEvent_ set and throws an exception:

Collapse | Copy Code

    NTSTATUS status = KeWaitForMultipleObjects(sizeof(eventsArray)/sizeof(PVOID), eventsArray, WaitAny, 
        Executive, KernelMode, FALSE, NULL, 0);
    if(status != STATUS_SUCCESS)
    {
        throw std::exception("MountedDisk::RequestExchange - mount stop.");
    }

We will get the thrown exception in catch block of DispatchException function and return STATUS_UNSUCCESSFUL to the user mode:

Collapse | Copy Code

NTSTATUS DispatchExchange(PVOID buffer, ULONG inputBufferLength, ULONG outputBufferLength)
{
    try
    {
...
        gMountManager->RequestExchange(request->deviceId, 
                                       request->lastType, 
                                       request->lastStatus, 
                                       request->lastSize, 
                                       request->data, 
                                       request->dataSize, 
                                       &response.type,
                                       &response.size, 
                                       &response.offset);
...
    }
    catch(const std::exception & ex)
    {
        KdPrint((__FUNCTION__" %s\n", ex.what()));
        return STATUS_UNSUCCESSFUL;
    }
}

Returned error state will be then processed by user mode code in DriverControl::RequestExchange function, and will also throw an exception in its turn:

Collapse | Copy Code

void DriverControl::RequestExchange(int deviceId, 
                                    int lastType, 
                                    int lastStatus, 
                                    int lastSize, 
                                    char * data, 
                                    int dataSize,
                                    int *type, 
                                    int *size, 
                                    __int64 * offset)
{
...
    if(!m_coreControl.DeviceIoGet(CORE_MNT_EXCHANGE_IOCTL, 
                                  &request, 
                                  sizeof(request), 
                                  &response, 
                                  sizeof(response), 
                                  &bytesWritten))
    {
        throw std::exception(__FUNCTION__" DeviceIoGet failed.");
    }
...
}

This exception in its turn will be caught by catch block in SyncMountmanager::mountDispatchThread:

Collapse | Copy Code

void SyncMountManager::mountDispatchThread(void* pContext)
{
...
    try
    {
        DispatchImage(dispContext->devId, 
                      image->GetMountPoint(),
                      dispContext->image, 
                      dispContext->mountManager->GetDriverControl());
    }
    catch(const std::exception& ex)
    {
        dispContext->mountManager->OnUnmount(dispContext->image, ex.what());
    }
...
}

It will cause the request processing thread termination and also the IImage destructor call.

How to build this Solution

1. Install Windows Driver Developer Kit 2003. (http://www.microsoft.com/whdc/devtools/ddk/default.mspx)
2. Set global environment variable "BASEDIR" to path of installed DDK.

Computer -> Properties -> Advanced -> Environment variables ->System Variables -> New
Like this: BASEDIR -> c:\winddk\3790

3. Download and install boost (tested with 1.40 version) (http://www.boost.org/users/download/ )
4. Set global environment variable "BOOST" to path of installed boost. (You have to restart your computer after this.)
5. Use Visual Studio 2008 to build the solution.

Testing

1. Build the solution using the instructions above.
2. Copy CoreMnt.sys to %windir%\system32\drivers.

3. Register the driver in the system by the command:

Collapse | Copy Code

sc create CoreMnt type=  kernel binPath= system32\drivers\CoreMnt.sys

4. Start driver by the command:

Collapse | Copy Code

sc start CoreMnt

5. Start CoreMntTest.exe.

If everything was ok then CoreMntTest.exe will display the message:

Collapse | Copy Code

  Image was mounted. Press  any key for unmount.

Disk Z will appear in the system.

Now we can format it.

A file «tst_img» will appear in the directory along with CoreMntTest.exe.