Note that creating virtual CD-ROM drives attached by SCSI and then loading them with virtual game CDs so that said game will play, was the entire basis of the drive emulator market for a while. :-)
I have also known people who use this technique to create an encrypted disk image which is distributed with their program which provides the 'secret bits'. Sort of like mount the image, decrypt it by giving it the key, and then launch the app which loads its .dll files from the newly decrypted attached disk. A bit of obfuscation which minimizes the impact on the application that you're trying to keep out of the hands of people without a license to it.
What is the advantage of your framework compared to implementing the iSCSI protocol (SCSI over TCP) and using standard tools like Microsoft iSCSI Initiator?
Good question. Unfortunately I do not know enough about iSCSI to answer this question fully.
In my case I wanted a simple way to create "SCSI disks" using a user mode process. Since I had already created WinFsp, which has a solid IPC (Inter-Process Communication) protocol, this felt like the best way forward.
This might sound like a dismissive question, so I should say upfront that I don't mean it to be. I'm truly curious and not overly familiar with this technology. With that said...
What is this for? What itch were you attempting to scratch which lead you to create this? How could it be useful in a project? / What sorts of projects would it it lend itself to?
I think it is a perfectly legitimate question. I had two reasons for creating this: a practical one and a theoretical one.
The practical one is that I wanted to be able to store disk data in a cloud storage bucket (i.e. "format" a bucket with NTFS and use it as a disk). For various reasons I did not want to build this as a WinFsp or FUSE file system.
The theoretical one is that I am exploring the possibility of writing a new OS where all/most IO/services/etc. could be implemented either in kernel mode or user mode. Not bolt it onto an existing OS like I have done with WinFsp and WinSpd, but build the OS like this from the ground up.
To finish here are some use cases for WinSpd as food for thought:
- Mounting virtual disks in a format other than VHD.
- Mounting a cloud storage bucket as a disk.
- Serving a read-only disk from an internet location for use by multiple users.
- Tracking disk I/O for backup, snapshotting or other purposes.
Great reply, thanks! I did wonder if this would be useful for integration with things like cloud storage, so good to hear that it was one of your primary driving factors.
MS is actually working on their own replacement for Dokan that is called Projected File System [0]. If I understand correctly, they need it internally for handling large git repositories, so it's probably there to stay.
As far as I can tell it is not the same thing. Projected File System overlays a part of a real file system, and files are cached to the local file system.
Agree. The Projected File System is very limited compared to WinFsp, Dokany or CBFS. I have looked at it quite extensively as my own WinFsp open source software is in the same space.
WinSpd is not a standalone application, but rather a set of libraries and components that allows others to create applications that can do what you want (serve a disk over a network).
The included sample, called rawdisk, memory maps a file and uses it as a disk image file. This could presumably be done over SMB or NFS, although I have not tested the performance.
Yeah. I thought of putting my Steam game library on my NAS so I can use the same library across multiple computers, but Steam doesn't like having its game library set on a network share.
Sorry for the second reply, but I read a bit about ImDisk.
I think the main difference is that WinSpd is a general tool for creating disks as user mode processes, whereas ImDisk is a tool for mounting image files only.
It looks like WinSpd is a more generic tool. But I know very little about ImDisk so I may be mis-representing it.
Just want to note, that ImDisk, mentioned here in the thread, while technically GPL too, can be stripped of non-GPL code easily, according to its author.
"ImDisk is a virtual disk driver for Windows NT/2000/XP/Vista/7/8/8.1/10 and Windows Server 2003/2003 R2/2008/2008 R2/2012/2012 R2, 32 and 64 bit editions. It can create virtual hard disk, floppy or CD/DVD drives using image files or system memory."
That still misses the point, that it provides a protocol dubbed "devio", that enables userspace programs to create these disks virtually, and handle access via own implementations of write/read primitives.
I have also known people who use this technique to create an encrypted disk image which is distributed with their program which provides the 'secret bits'. Sort of like mount the image, decrypt it by giving it the key, and then launch the app which loads its .dll files from the newly decrypted attached disk. A bit of obfuscation which minimizes the impact on the application that you're trying to keep out of the hands of people without a license to it.
In my case I wanted a simple way to create "SCSI disks" using a user mode process. Since I had already created WinFsp, which has a solid IPC (Inter-Process Communication) protocol, this felt like the best way forward.
What is this for? What itch were you attempting to scratch which lead you to create this? How could it be useful in a project? / What sorts of projects would it it lend itself to?
The practical one is that I wanted to be able to store disk data in a cloud storage bucket (i.e. "format" a bucket with NTFS and use it as a disk). For various reasons I did not want to build this as a WinFsp or FUSE file system.
The theoretical one is that I am exploring the possibility of writing a new OS where all/most IO/services/etc. could be implemented either in kernel mode or user mode. Not bolt it onto an existing OS like I have done with WinFsp and WinSpd, but build the OS like this from the ground up.
To finish here are some use cases for WinSpd as food for thought:
- Mounting virtual disks in a format other than VHD.
- Mounting a cloud storage bucket as a disk.
- Serving a read-only disk from an internet location for use by multiple users.
- Tracking disk I/O for backup, snapshotting or other purposes.
I tried Dokan and CBFS (Call Back File System) a long time ago. They were both great, but I ran into bugs fairly quickly with each.
[0] https://docs.microsoft.com/en-us/windows/desktop/projfs/proj...
I like to think that WinFsp is very stable software. I am working to bring WinSpd to the same levels of stability.
[1] http://www.secfs.net/winfsp/develop/tests/
Here is the most updated version of those documents:
https://github.com/billziss-gh/winfsp/wiki/WinFsp-Testing
The included sample, called rawdisk, memory maps a file and uses it as a disk image file. This could presumably be done over SMB or NFS, although I have not tested the performance.
Command queues are supported by default as well, as this is a Storport virtual miniport driver (which supports command queues).
They both have their uses but in general I think WinFsp is more useful for most.
I am not familiar with ImDisk.
I think the main difference is that WinSpd is a general tool for creating disks as user mode processes, whereas ImDisk is a tool for mounting image files only.
It looks like WinSpd is a more generic tool. But I know very little about ImDisk so I may be mis-representing it.
http://www.ltr-data.se/opencode.html/#ImDisk
They too have a .NET wrapper BTW.