7 reasons to choose a file system (CBFS Storage) over the database for managing your application data

Introduction

Database Management Systems (DBMS) are popular as a uniform data storage, and they have become increasingly popular with proliferation of mobile and web development, where file systems are either not available at all, or have restricted access.

Such proliferation has caused certain bias to using DBMS (and specifically SQLite as a built-in DBMS on mobile platforms) as a universal solution to data storage needs.

However, file systems are not a thing of the past. The advancement of mobile software brings back the necessity of design templates similar to those on desktop systems. In particular, the need arises to use files and file-based approaches to management of heterogeneous information. So, files are still in demand and they are more suitable for many classes of data management tasks.

Here I would like to discuss the use cases where the file system is more suitable solution than DBMS


Content


Encryption

Data encryption is a complicated goal which is to some extent contradictory to the nature of DBMS. The database needs to know the size and in many cases the content of the data fields for operating efficiently. This limits the flexibility in data management. Modern DBMS (mostly client-server ones) provide means to encrypt table fields, but on mobile devices these capabilities are absent. Also, field encryption doesn't hide metadata and doesn't disguise the structure of data and the number of entries. So, each developer needs to invent his own encryption schemes, and proper implementation of encryption is a hard to achieve goal.

At the same time, encrypting files and the file system in whole is trivial. CBFS Storage and some other file systems implement on-the-fly encryption and decryption of files in streaming mode, so you (the developer) don't need to care about this. Moreover, certain file systems, such as CBFS Storage, support both whole-storage and per-file encryption and you can use different keys and encryption mechanisms for different files and the storage itself. And if you need to plug your own encryption (e.g. certificate-based), this is not hard to do as well via callback mechanisms offered by CBFS Storage.


Compression

Compression is very important on mobile devices, where the user pays for traffic and where the storage space is severely limited. Databases are space-hungry and have no mechanisms for data compression, which leads to excessive complication of the application that needs to deliver data in compressed form and then import them to the database (and even then uncompressed data takes precious space in device memory).

Modern file systems have built-in data compression implemented on file level. I.e. you can compress all or some files as needed.


Versioning

Quite often you have to keep several variants of the same named entity (be it a calendar record, a contact, a text note or anything more complicated). Is it a simple Undo mechanism or logging where you need to keep one or more previous versions of the data, you need a place to store those versions. The file system with Alternate Data Streams support lets you store all variants in one file, so file operations like copying and moving perform operations on the same data.

In DBMS to have versioning you need to maintain a separate table or two as there are no built-in mechanisms for versioning. With file systems version management can be almost transparent - to create a version you don't open the file itself, but its alternate stream. Just add a suffix to the filename and you have a version!


Large and heterogeneous files

Relational databases were designed for storing uniform data with predefined number and type of fields for each record. Such design is effective when processing large amounts of such uniform data, but doesn't work well with objects that have different set of properties. The necessity to manage such object sets lead to the creation of OODBMS (object-oriented DBMS) and later no-SQL DBMS. Still, SQL-based DBMS are mainstream, especially on mobile devices.

SQL-based relational DBMS don't handle large fields of undefined length (BLOBs): such fields break a sleek structure of the DB table. Various DBMS offer different ways to manage BLOBs. Frequently, BLOBs are kept in separate files on the disk (file system comes into play again) with references from the DB table. Other DBMS have a mini-file system built into their engine to manage BLOBs and keep them in one file (here comes a file system again).

The described way DBMS works with BLOBs is an extra level of code which slows down operations with large data and does not add stability and reliability.

With a file system you can have files as large as you want, and some file systems let you add custom properties (CBFS Storage calls them tags) to the files. Such properties let you organize object storage easily.


Remote and in-memory storage

The database in DBMS is almost always a set of files in local persistent storage (file system or memory storage). Some DBMS allow the creation of in-memory tables for temporary operations, but migrating data to and from such memory storage is cumbersome.

Virtual file system, such as CBFS Storage, allows you to keep the storage anywhere - in memory, in the database (if needed), on the network disk or in the cloud. Using callback mechanism CBFS Storage offers unprecedented flexibility in maintaining storage. Moreover, you can have on-the-fly mirroring of your storage, i.e. have a backup copy on the remote storage as soon as the storage is changed locally.


Easy backup

DBMS are not designed for an easy back up. You rarely can just grab files of the DBMS and copy them - the restoration of these files becomes a tricky operation. More often you need to export the data to [large and not easily maintainable] SQL file which is then packed and stored somewhere. If you need to recover the data, you need to execute this SQL file, potentially editing it.

In case of file systems backup and restore operations are trivial - you just copy the files or the container itself. In case of CBFS Storage, which stores all files in single container, you can copy the container, then put it back and it will have anything that's needed to continue its work.

If you use mirroring as mentioned above, you have backups created on-the-fly.


Exposure to other applications

The last but not least, is that you often need to expose the data contained in the DBMS, as files for processing by other applications. With DBMS there's no way to accomplish the task without external assistance. And while on desktop systems you can use third-party solutions (e.g. CBFS Connect on Windows or FUSE on Linux) to expose custom data as files, on mobile platforms this is not possible at all.

With "regular" (OS-provided) file systems such task does not appear. CBFS Storage also lets you expose CBFS Storage as virtual disks. CBFS Storage OS edition lets you create a virtual disk from CBFS Storage on Windows, MacOS X, Linux and FreeBSD systems (unfortunately the task cannot be accomplished on mobile devices due to limitations of the system architecture on those devices).


Conclusion

While DBMS does provide a handy and inexpensive way to store certain data, the expenses and shortcomings of DBMS are numerous and alternatives must be seriously considered before the application is designed. Virtual file system can be a way better alternative to DBMS in many cases.

Ready to get started?

Learn more about Callback Technologies or download a free trial.

Download Now