Alternatives++

paper/paper.ms

@@ -74,6 +74,9 @@ Document sync'ed with DODB \*[VERSION]
 .fi
 .br
 .po
+.B "Status of this document" .
+Although fairly advanced, this document lacks a few reviews, a bit of discussion about filesystems (section not entirely finished), to talk about alternatives to DODB and a final conclusion.
+
 .SECTION Introduction to DODB
 A database consists in managing data, enabling queries to add, to retrieve, to modify and to delete a piece of information.
 These actions are grouped under the acronym CRUD: creation, retrieval, update and deletion.
@@ -1098,14 +1101,43 @@ The next paragraphs will give an idea of how filesystems work, the implied limit
 Explaining the way filesystem work and their design is out of the scope of this document, so this part will be kept short for readability reasons.
 .FOOTNOTE2
 
-Beside filesystems designed for specific constraints, such as writing data on a compact disk\*[*] or providing a network filesystem, most
-.dq generic
-filesystems share a (loosely) common set of objectives.
+Filesystems designed for specific constraints, such as writing data on a compact disk\*[*] or providing a network filesystem, are out-of-scope of this document.
 .FOOTNOTE1
 A compact disk has specific constraints since the device will then only provide read-only access to the data, obviating the need for most of the complexity revolving around fragmentation, inode management and so on.
 All storage devices have their own particularities, but regular hard drives and solid-state drives are the important ones for this discussion since filesystems have mostly been designed for them.
 .FOOTNOTE2
-These features could be summarized in a few points.
+The rest of this section will address more
+.dq generic
+filesystems\*[*].
+.FOOTNOTE1
+Furthermore, the rich history behind filesystems is inherently related to the rich history of storage devices, this document is not supposed to be a survey on either of those.
+Let's keep it short and simple.
+.FOOTNOTE2
+
+Filesystems have similarities in their inner workings.
+For instance, they have some
+.dq "special files"
+called
+.I inodes
+(hidden from the user) to keep track of
+.I where
+the files are on the disk, their size, the last time they were modified or accessed and other metadata.
+A filesystem is split into a list of
+.I blocks
+of a certain size\*[*] (4 kilobytes by default).
+.FOOTNOTE1
+Working only with blocks (from 0 to x) is called
+.dq "Logical block addressing" .
+Before that, other schemes were used such as
+.I cylinder-head-sector
+(CHS) but this is fairly obsolete since even hard disks do not use this anymore.
+.FOOTNOTE2
+Since all files cannot be reasonably expected to be written in a continuous segment of data, inodes store the block numbers where the file has been written.
+Filesystems may enable to tweak the block size (related to the
+.I "sector"
+size of the storage device) either to reduce fragmentation and metadata (bigger block sizes to partitions with big files) or to avoid wasting space (smaller block sizes to partitions with a huge number of files under the size of a block).
+
+Filesystems share a (loosely) common set of objectives.
 
 .STARTBULLET
 .KS
@@ -1114,19 +1146,16 @@ These features could be summarized in a few points.
 Above all, as already established, filesystems enable CRUD operations on a storage device through the concepts of directories and files; this is how users have been directly interacting with their computer to store data for decades.
 .KE
 
-.BULLET
 .KS
+.BULLET
 .B "Reliability and safety" .
-.TBD
-Since computers do not run in a vacuum, many problems can occur during operation including the loss of the energy supply.
-Filesystems try to mitigate damage by keeping a journal of operations (journalized filesystems).
-Advanced filesystems may also detect file corruption with automated checksums.
+Since applications do not run in a vacuum, many problems can occur during normal computer operation including the loss of the energy supply, slow filesystem corruption due to a faulting storage device, etc.
+Filesystems try to mitigate damage in different ways, such as keeping a journal of operations (journalized filesystems) to avoid losing files upon energy loss, or by detecting file corruption with automated checksums.
 .KE
 
-.BULLET
 .KS
+.BULLET
 .B "Security" .
-.KE
 File access should be limited in a number of cases.
 For example, several applications with networking features might run on a computer.
 If one of these applications is successfully attacked, the attacker shouldn't be able to access other services data or user data.
@@ -1134,13 +1163,14 @@ Same thing for shared computers, one user shouldn't be able to see other users'
 Therefore, the most widespread form of security comes from filesystem permissions, enabling a user (or a group of users) to access (or to be denied from accessing) specific data (files and directories).
 Those permissions include the right to read, to modify or to execute a file, to list or to remove files from a directory, to create or to remove directories and a few other permissions.
 Extended permissions and attributes exist but are out-of-scope.
+.KE
 
 Beside permissions, encryption also brings some kind of security.
 In this case, the point is to prevent attackers from accessing protected data despite retrieving files.
 Some advanced filesystems can encrypt files individually, others provide the encryption of a whole partition, both methods having their pros and cons.
 
-.BULLET
 .KS
+.BULLET
 .B "Performance and capacity" .
 Many file systems were developed over the years to circumvent contemporary limitations on file or partition sizes, the number of possible files, the limitation on path name lengths, etc.
 While storage devices mostly impose physical limitations, a filesystem may be wasting resources because of a simplistic or inadequate design.
@@ -1153,16 +1183,47 @@ So, worst case scenario, data rate is
 (huge waste) meaning that a 1GB of data would require an entire 4TB hard drive\*[*] (without even taking the inodes' size into account).
 .FOOTNOTE1
 Ext4 can integrate up to 60 bytes of data into an extended inode.
+.TBD
 .FOOTNOTE2
 
-.BULLET
 .KS
+.BULLET
 .B "Miscealeneous and advanced features" .
 A few other features need to be mentionned, such as block suballocation, file content included in the inode, etc.
 Some filesystems added more than a decade ago then under-explored features such as snapshots, compression and transactions.
 .KE
 .ENDBULLET
 
+.ds OK \[OK]
+.ds NOK \[tmu]
+.nr total 16.0c
+.nr col1 3.0c
+.nr col2 (\n[total]-\n[col1])/3
+.nr col3 (\n[total]-\n[col1]-\n[col2])
+.\"total: \n[total]
+.\"col1: \n[col1]
+.\"col2: \n[col2]
+.\"col3: \n[col3]
+.TS
+allbox tab(:);
+c | c | c
+cw(\n[col1]u) | lw(\n[col2]u) | lw(\n[col3]u).
+Feature : Traditional databases : Filesystems
+CRUD operations : SQL : Files & directories
+Atomicity : \*[OK] :T{
+transactions are implemented in a few filesystems (ex: BTRFS)
+and there is a locking mechanism based on files
+T}
+Consistency : \*[OK] : \*[NOK]
+Isolation : \*[OK] :T{
+.dq "new file then mv"
+technique
+T}
+Durability : \*[OK] : yes (checksums)
+Access Time : 0.1 to 2ms : a few µs (cache) to a few ms (first access)
+Transactions : :
+.TE
+
 In conclusion, no current filesystem has been designed to be used the way DODB use them.
 However, having a few millions entries is fine on most filesystems.
 .
@@ -1196,13 +1257,39 @@ Sure, DODB doesn't have the same features, but are they worth multiplying the co
 .FOOTNOTE2
 
 .BULLET
-.B "Key-value stores."
-.B "Memcached"
+.B "Embedded SQL database" .
+Examples: SQLite and DuckDB.
 
+.BULLET
+.B "New types of SQL" .
+Example:
+.B EdgeDB
+(modern SQL with easy-of-use complex relations).
+
+.BULLET
+.B "Key-value stores."
+Examples:
+.B RocksDB
+(embedded),
+.B Memcached
+(application to store data cache, not to be used as a primary database system),
+.B CockroachDB
+(distributed, supports ACID transactional semantics)
+
+.BULLET
+.B "Document databases" .
+Such as DODB, many other document-oriented databases exist.
+For example,
+.B CouchDB
+(distributed, fault-tolerant, RESTful HTTP and JSON API, etc.).
+
+.BULLET
 .B "Redis"
 
+.BULLET
 .B "MongoDB"
 
+.BULLET
 .B "duckdb"
 .ENDBULLET