Getting started with CBFS Connect


CBFS Connect consists of two components - a kernel-mode driver and a user-mode API. The driver works under the hood and you don't work with it directly. The API is linked to your user-mode application. You use the API to create, delete and manipulate the storage. The API also tells your application to perform certain actions which are required to actually implement the file systems.


Install and uninstall the drivers

Before the driver can be used, it must be installed. This can be done using Install() API method or using the Installer DLL. The API is used when the executable module of your application installs or uninstalls the driver. The Installer DLL was designed to be called from installation scripts, which are compiled into setup programs by special setup applications, such as Wise, InstallShield, InnoSetup, etc.

After the driver is installed, it might be necessary to restart the system in order for the new driver to be installed and used. The API method or installer DLL will tell you if you need to restart the system.

The driver must be installed once. You don't have to install it each time you start your application. If the driver is not installed correctly, you will get error 2 ("file not found") which means that the driver could not be found or loaded.

When driver installation is complete, you can use the API to create and manage the storage with the virtual file system.

When you don't need the driver anymore, you can uninstall it using Uninstall() API method or using the installer DLL.

Create and delete the storage

Before the CBFS Connect class can be used (and virtual disks can be created), you need to call Initialize() method to initialize the virtual file system.

The storage consists on the "drive" and the "media", similar to a CD-ROM drive and the actual compact disk which is inserted to the drive.

So, with CBFS Connect you first create a "drive" by calling CreateStorage() API method. Next, you assign a drive letter to the created "drive" using AddMountingPoint() API method. You can create a visible drive by assigning a drive letter or create an invisible drive by assigning a UNC path. The invisible drives (mounted via UNC paths), are not shown in Explorer or other file managers. They can, however, be accessed from the applications. Also, the virtual file system can be mounted as a folder on the existing NTFS drive.

Finally, you insert the "disk" to the "drive" by using MountMedia() API method. But wait. How will the system know how to work with the given disk? You need to implement the actual file system. This is done by handling the callbacks (events) of CBFS Connect class as described below.

When you don't need a disk anymore, you call the opposite sequence of methods - first you call UnmountMedia() method to remove the "disk", then delete the "drive" using DeleteStorage() method.

Handle the callbacks (events)

Callbacks (event handlers) are the places where you implement the functionality of the file system. While we tried to hide most of the complexities under the hood, quite a bit of work still needs to be done by users in order to implement the functionality of the file system.

All callback functions can be grouped by functionality. With the exception of named stream callbacks and security-related callbacks, all groups of callbacks are mandatory, meaning that your application must handle those callbacks.

Callback functions are called by the API when the Windows operating system (the OS) is notified about your "drive" and your "disk" and needs to work with them. Some requests are handled internally by CBFS Connect, but most of them (those which perform actual work) are passed to your application by means of callbacks.

Tell the OS about file system capabilities

First of all, the OS needs to know the size and the name of the disk. You need to handle OnGetVolumeId, OnGetVolumeLabel and OnGetVolumeSize events.

The OS can change the volume label. This type of request is passed to your application using OnSetVolumeLabel callback.

Let the OS get information about files and directories

The next important operation for any disk is to provide information about its contents to the OS. When the OS needs to read contents of some directory (starting from the root directory), it sends the requests which are translated into OnIsDirectoryEmpty, OnEnumerateDirectory and OnCloseEnumeration callbacks.

OnIsDirectoryEmpty is a simple callback which is used to find out if a directory is empty. If the directory is empty, it is not enumerated.

OnEnumerateDirectory callback is called when the OS starts or continues enumeration of some folder. Unlike Win32 interface, which has FindFirstFile and FindNextFile functions, in CBFS Connect OnEnumerateDirectory callback is called for both, first file and consequent entries in the directory. The API calls OnEnumerateDirectory callback in a loop until OnEnumerateDirectory returns false in FileFound parameter. The application must provide information about some file or directory, contained in the directory which is enumerated via parameters of OnEnumerateDirectory callback. Each time different file or directory must be reported.

To store information about which file or directory was reported the last time (and so to know which file or directory must be reported next), the application can use EnumerationContext parameter. The following scenario can be used: if EnumerationContext is not allocated, this is the first call in a row to enumerate the given directory. The application can open access to the data, which represents the directory or perform other operation needed to get access to the list of the entries. Also the application allocates the context and passes it back to the API. If the context is already allocated, it is used for enumeration.

Once the application tells the API that there are no more files or directories left to enumerate, the API calls OnCloseEnumeration callback and passes it the allocated enumeration context. You need to free the allocated resources and dispose of the enumeration context.

In addition to the above callbacks, two more callbacks are called during directory enumeration. They are OnOpenFile and OnCloseFile. Despite what the names of these callbacks imply about the files, the callbacks are also called for the directory (Windows doesn’t distinguish files and directories when it sends requests to the file system).

While the directory enumeration is used to get a list of entries in the directory, it's sometimes necessary for the OS to obtain information about some particular file. Such a request is passed to your application via OnGetFileInfo callback. This callback is very similar to OnEnumerateDirectory and requests the same information, but gives you the name of the file or directory of interest and is only called once.

Create and open files

The next important operation is to create new files and open existing files. CBFS Connect object has two callbacks for this: OnCreateFile and OnOpenFile. As the names suggest, these callbacks are called when the OS needs to create a new file or to open an existing file.

When your application handles these callbacks, it usually opens access to some resource (be it a file, some remote resource or just a reference to a memory block). It is necessary to keep the handle to the opened resource somewhere, in order to use this handle in consequent file operations. CBFS Connect offers FileHandleContext parameter. The application can store the above mentioned handled in the context. The context is passed to other callbacks, which are called when operations on open files (reading / writing / seeking / closing etc.) are performed. Your callback handlers will identify the opened file using this context.

To optimize multiple file opening operations which occur in parallel (for example, when you open two instances of some application which opens supplementary files or DLLs), CBFS Connect by default calls OnOpenFile callback, only the first time when the file is opened. If the file is opened successfully, the opened file is used for the next file open operations, which happen while the original file handle is opened. Usually this works correctly, but if the file is opened with different access mode, such scheme is not acceptable. To disable the described behavior and have OnOpenFile callback be called every time, set CallAllOpenCloseCallbacks property to true.

As mentioned above, OnCreateFile and OnOpenFile are called for files and also for directories. The directories are created via OnCreateFile callback and they are opened for contents enumeration using OnOpenFile callback.

Close the files

After the file is read or written, it is closed by the OS. Closing doesn't necessarily happen as soon as the application, that used the file, calls CloseFile() Win32 function. In some cases, the OS (it's cache manager) keeps the handle to the file open for some time.

CBFS Connect receives the request to close the file and calls OnCloseFile callback. Much like the OS, CBFS Connect driver delays closing of the files in order to save resources and time. This behavior can be disabled by calling AllowDelayedClose() method and passing False as parameter.

OnCloseFile callback includes the file handle context as a parameter, and this context must be disposed of by your application.

If CBFS Connect called OnOpenFile callback only the first time the file was opened, OnCloseFile will also be called only when the last handle to the file is closed by the OS (see description of this behavior above).

When the directory contents enumeration is performed by the OS, OnCloseFile is called for the directory being enumerated when the enumeration is completed.

Read from files and write to files

When the OS needs to read the data from the file or write the data to the file, CBFS Connect API uses OnReadFile and OnWriteFile respectively. Each of those callbacks includes parameter, which specifies the offset (position) in the file from which the application must read the data or to which the application must write the data. Also, each of the callbacks includes FileHandleContext parameter. The file, on which the operation is performed, is identified using this context.

Handle file size change requests

Before writing the data, the OS tells the file system that certain amount of space must be allocated for the file to accept more data. This is done using OnSetAllocationSize callback. If the size is not allocated, the OS won't be able to write the data. The allocation size should not be confused with the file size. The allocation size is the space which is occupied by the file. The file size is the actual size of the file contents. The allocation size is always equal or larger than file size. If the OS or some application needs to change the size of the files, it sends the request which is translated into OnSetEndOfFile callback. Each of the callbacks includes FileHandleContext parameter. The file, on which the operation is performed, is identified using this context.

Handle file renaming and deletion

File and directory renaming (including change of the file location in the directory tree) and file and directory deletion are atomic operations which must be handled via OnRenameOrMove and OnDeleteFile callbacks.

Create and delete directories

The directory creation must be performed in OnCreateFile callback (which is described above).

Deletion of the directories must be done in OnDeletFile callback.

Advanced operations

Advanced operations include manipulations with the named streams, security operations (retrieval and change of file and directory security attributes) and raw volume operations.

Work with named streams

Named (alternate) streams are the separate data chunks, stored in the file. They have their own names inside of the file. Support for the named streams is optional.

Work with named streams is very similar to work with regular files. The named streams are opened via OnOpenFile callback and are read / written / closed in the same way as files are. When the named stream is opened, the file name contains a semicolon (":"), which delimits the file name and the stream name.

Enumeration of the named streams is done via OnEnumerateNamedStreams callback in a way similar to enumeration of directory entries. Once enumeration is complete, OnCloseEnumeration callback is called.

If you don't want to support named streams, don't assign a handler for OnEnumerateNamedStreams callback. In this case CBFS Connect will tell the OS that the named streams are not supported.

Handle security operations

Security operations include setting and retrieving ACL (access control lists) for the files and directories. Security operations are implemented via OnGetFileSecurity and OnSetFileSecurity callbacks. These callbacks are optional. If your application doesn't assign handlers for them, CBFS Connect will tell the OS that the security operations are not supported.

What you may and may not do in callback handlers

The answer is short - the less calls (direct or indirect) to the OS API is done the better. First of all, many system calls can lead to a system-wide deadlock. Network operations are generally safe, but calls which involve the GUI can be unsafe. This is especially true when some third-party file system filter driver (such as antivirus or firewall application) comes into play.

Another reason is that the operation can take certain time and the OS was not designed in the way that expects the file system operations to be fast (milliseconds or a couple of seconds is an upper limit expected by the OS). So, if your callback takes a minute to handle the callback, you can be sure that the whole system will be waiting for your callback (in some cases).

If allowed by file system implementation design, try to cache the file system operations in memory and perform all disk, GUI and network operations asynchronously, i.e. in a separate worker thread and not in the callback handler.

Asynchronous operations

The idea of asynchronous operations is to have a worker thread (or several threads) which will help the callbacks do their job. Callback function works with the local cache and the worker thread transfers the data to and from the actual data location. Implementation of the worker threads and Inter-thread communication is outside of scope of this article.

Ready to get started?

Learn more about Callback Technologies or download a free trial.

Download Now