Getting Started with CBFS Connect

Introduction

CBFS Connect consists of two components — a kernel-mode driver and a user-mode API. The driver works under the hood; you don't work with it directly. The API is linked to your user-mode application. You use the API to create, delete, and manipulate the storage. The API also instructs your application to perform certain actions that are required to actually implement the file systems.


Contents


Install and Uninstall the Drivers

Before the driver can be used, it must be installed. This can be done using the Install() API method or using the Installer DLL. The API is used when the executable module of your application installs or uninstalls the driver. The Installer DLL was designed to be called from installation scripts, which are compiled into setup programs by special setup applications, such as Wise, InstallShield, InnoSetup, etc.

After the driver is installed, it might be necessary to restart the system in order for the new driver to be installed and used. The API method or installer DLL will notify you if you need to restart the system.

The driver needs to be installed only once. There is no need to install it each time you start your application. If the driver is not installed correctly, you will get error 2 ("file not found"), which means that the driver could not be found or loaded.

When the driver installation is complete, you can use the API to create and manage the storage with the virtual file system.

When you don't need the driver anymore, you can uninstall it using the Uninstall() API method or using the installer DLL.


Create and Delete the Storage

Before the CBFS Connect class can be used (and virtual disks created), you need to call the Initialize() method to initialize the virtual file system.

The storage consists of the "drive" and the "media," similar to a CD-ROM drive and the actual compact disk that is inserted to the drive.

To create the drive, call the CreateStorage() API method. Next, assign a drive letter to the created drive using the AddMountingPoint() API method. You can create a visible drive by assigning a drive letter or you can create an invisible drive by assigning a UNC path. The invisible drives (mounted via UNC paths), are not shown in Explorer or other file managers. They can, however, be accessed from applications. Also, the virtual file system can be mounted as a folder on the existing NTFS drive.

Finally, you insert the "disk" to the "drive" by calling the MountMedia() API method. r the system to know how to work with the given disk, you need to implement the actual file system. This is done by handling the callbacks (events) of the CBFS Connect class as described below.

When you don't need a disk anymore, you call the opposite sequence of methods — first you call the UnmountMedia() method to remove the disk, then delete the drive using the DeleteStorage() method.

Handle the Callbacks (Events)

In callbacks (event handlers), you implement the functionality of the file system. CBFS Connect hides most of the complexities involved; see the following for your main considerations when using CBFS Connect.

All callback functions can be grouped by functionality. With the exception of named stream callbacks and security-related callbacks, all groups of callbacks are mandatory, meaning that your application must handle those callbacks.

Callback functions are called by the API when the Windows operating system (the OS) is notified about your drive and your disk and needs to work with them. Some requests are handled internally by CBFS Connect, but most of them (those which perform actual work) are passed to your application by means of callbacks.


Provide Information on File System Capabilities to the OS

You need to handle the OnGetVolumeId, OnGetVolumeLabel, and OnGetVolumeSize events to provide the name of the name of the disk and its size to the OS.

The OS can change the volume label. This type of request is passed to your application using the OnSetVolumeLabel callback.


Get Information about Files and Directories using the OS

An important operation for any disk is to provide information about its contents to the OS. When the OS needs to read the contents of some directory (starting from the root directory), it sends requests that are translated into the OnIsDirectoryEmpty, OnEnumerateDirectory, and OnCloseEnumeration callbacks.

OnIsDirectoryEmpty is a simple callback that is used to find out if a directory is empty. If the directory is empty, it is not enumerated.

The OnEnumerateDirectory callback is called when the OS starts or continues the enumeration of some folder. Unlike the Win32 interface, which has the FindFirstFile and FindNextFile functions, in CBFS Connect the OnEnumerateDirectory callback is called for both the first file and subsequent entries in the directory. The API calls the OnEnumerateDirectory callback in a loop until OnEnumerateDirectory returns false in the FileFound parameter. The application must provide information about a file or directory contained in the directory that is enumerated via the parameters of the OnEnumerateDirectory callback. Each time, a different file or directory must be reported.

To store information about which file or directory was reported the last time (and so to know which file or directory must be reported next), the application can use the EnumerationContext parameter. The following scenario can be used: if EnumerationContext is not allocated, this is the first call in a row to enumerate the given directory. The application can open access to the data that represents the directory or perform other operations needed to get access to the list of the entries. Also, the application allocates the context and passes it back to the API. If the context is already allocated, it is used for enumeration.

Once the application tells the API that there are no more files or directories left to enumerate, the API calls the OnCloseEnumeration callback and passes it the allocated enumeration context. You need to free the allocated resources and dispose of the enumeration context.

In addition to the above callbacks, two more callbacks are called during directory enumeration: OnOpenFile and OnCloseFile. Despite what the names of these callbacks imply about the files, the callbacks are also called for the directory (Windows doesn’t distinguish files and directories when it sends requests to the file system).

Directory enumeration is used to get a list of entries in the directory, but it's sometimes necessary for the OS to obtain information about a particular file. Such a request is passed to your application via the OnGetFileInfo callback. This callback is very similar to OnEnumerateDirectory and requests the same information. But, it gives you the name of the file or directory of interest and is only called once.


Create and Open Files

To create new files and open existing files, the CBFS Connect object exposes two callbacks: OnCreateFile and OnOpenFile. As the names suggest, these callbacks are called when the OS needs to create a new file or to open an existing file.

When your application handles these callbacks, it usually opens access to some resource (be it a file, some remote resource, or just a reference to a memory block). It is necessary to keep the handle to the opened resource somewhere, in order to use this handle in consequent file operations. CBFS Connect offers the FileHandleContext parameter. The application can store the above-mentioned handle in the context. The context is passed to other callbacks, which are called when operations on open files (reading / writing / seeking / closing, etc.) are performed. Your callback handlers will identify the opened file using this context.

To optimize multiple file-opening operations that occur in parallel (for example, when you open two instances of some application that opens supplementary files or DLLs), CBFS Connect by default calls the OnOpenFile callback, only the first time that the file is opened. If the file is opened successfully, the opened file is used for the next file open operations, which happen while the original file handle is opened. Note that if the file is opened with a different access mode, such a scheme is not suitable. To disable the described behavior and have the OnOpenFile callback be called every time, set the CallAllOpenCloseCallbacks property to true.

As mentioned above, OnCreateFile and OnOpenFile are called for files and also for directories. The directories are created via the OnCreateFile callback and they are opened for enumerating the file contents using the OnOpenFile callback.


Close Files

After the file is read from or written to, it is closed by the OS. Closing doesn't necessarily happen as soon as the application that used the file calls the CloseFile() Win32 function. In some cases, the OS (its cache manager) keeps the handle to the file open for some time.

CBFS Connect receives the request to close the file and calls the OnCloseFile callback. Much like the OS, the CBFS Connect driver delays closing files in order to save resources and time. This behavior can be disabled by calling the AllowDelayedClose() method and passing False as a parameter.

The OnCloseFile callback includes the file handle context as a parameter, and this context must be disposed of by your application.

If CBFS Connect called the OnOpenFile callback only the first time the file was opened, OnCloseFile will also be called only when the last handle to the file is closed by the OS (see the description of this behavior above).

When the directory contents enumeration is performed by the OS, OnCloseFile is called for the directory being enumerated when the enumeration is completed.


Read from Files and Write to Files

When the OS needs to read data from the file or write data to the file, the CBFS Connect API uses OnReadFile and OnWriteFile. Each of those callbacks includes a parameter that specifies the offset (position) in the file from which the application must read the data or to which the application must write the data. Also, each of the callbacks includes the FileHandleContext parameter. The file on which the operation is performed is identified using this context.


Handle File Size Change Requests

Before writing the data, the OS notifies the file system that a certain amount of space must be allocated for the file to accept more data. This is done using the OnSetAllocationSize callback. If the size is not allocated, the OS won't be able to write the data. The allocation size should not be confused with the file size. The allocation size is the space that is occupied by the file. The file size is the actual size of the file contents. The allocation size is always equal to or larger than the file size. If the OS or some application needs to change the size of the files, it sends a request that is translated into the OnSetEndOfFile callback. Each of the callbacks includes the FileHandleContext parameter. The file on which the operation is performed is identified using this context.


Handle File Renaming and Deletion

File and directory renaming (including changing the location of the file in the directory tree) and file and directory deletion are atomic operations that must be handled via the OnRenameOrMove and OnDeleteFile callbacks.


Create and Delete Directories

Directory creation must be performed in the OnCreateFile callback. Deletion of the directories must be done in the OnDeletFile callback.

Advanced Operations

Advanced operations include manipulations with the named streams, security operations (retrieval and change of file and directory security attributes), and raw volume operations.


Working with Named Streams

Named (alternate) streams are the separate data chunks stored in the file. They have their own names inside of the file. Support for the named streams is optional.

Working with named streams is very similar to working with regular files. The named streams are opened via the OnOpenFile callback and are read / written / closed in the same way that files are. When the named stream is opened, the file name contains a semicolon (":") that delimits the file name and the stream name.

Named streams are enumerated via the OnEnumerateNamedStreams callback in a way similar to how directories are enumerated. Once enumeration is complete, the OnCloseEnumeration callback is called.

If you don't want to support named streams, don't assign a handler for the OnEnumerateNamedStreams callback. In this case, CBFS Connect will notify the OS that named streams are not supported.


Handling Security Operations

Security operations include setting and retrieving ACLs (access control lists) for the files and directories. Security operations are implemented via the OnGetFileSecurity and OnSetFileSecurity callbacks. These callbacks are optional. If your application doesn't assign handlers for them, CBFS Connect will notify the OS that the security operations are not supported.


What You May and May Not Do in Callback Handlers

The answer is short - the less calls (direct or indirect) to the OS API that are done, the better. First of all, many system calls can lead to a system-wide deadlock. Network operations are generally safe, but calls that involve the GUI can be unsafe. This is especially true when some third-party driver that filters the file system comes into play (such as an antivirus or firewall application).

Another reason is that the operation can take a certain amount of time and the OS was designed in a way that expects file system operations to be fast (milliseconds or a couple of seconds is an upper limit expected by the OS). So, if your callback takes a minute to handle the callback, you can be sure that the whole system will be waiting for your callback (in some cases).

If allowed by the design of the file system implementation, try to cache the file system operations in memory and perform all disk, GUI, and network operations asynchronously, i.e., in a separate worker thread and not in the callback handler.


Asynchronous Operations

The idea behind asynchronous operations is to have a worker thread (or several threads) that will help the callbacks do their job. The callback function works with the local cache and the worker thread transfers the data to and from the actual data location. The implementation of the worker threads and inter-thread communication is outside of the scope of this article.

Ready to get started?

Learn more about Callback Technologies or download a free trial.

Download Now