Baguette/dodb.cr
3
0
Fork 1
Code Issues Pull Requests Releases Wiki Activity

Massive class renaming.

Browse Source
This commit is contained in:
Philippe PITTOLI 2024-05-22 18:29:08 +02:00
parent 5320a6afaf
commit 633085f63b
16 changed files with 71 additions and 75 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

20
paper/paper.ms
View File

@ -42,7 +42,7 @@
. \" The document starts here. . \" The document starts here.
. .
.TITLE Document Oriented DataBase (DODB) .TITLE Document Oriented DataBase (DODB)
.AUTHOR Philippe P. .AUTHOR Philippe PITTOLI
.ABSTRACT1 .ABSTRACT1
DODB is a database-as-library, enabling a very simple way to store applications' data: storing serialized DODB is a database-as-library, enabling a very simple way to store applications' data: storing serialized
.I documents .I documents
@ -197,7 +197,7 @@ end
Let's create a DODB database for our cars. Let's create a DODB database for our cars.
.SOURCE Ruby ps=9 vs=10 .SOURCE Ruby ps=9 vs=10
# Database creation # Database creation
database = DODB::DataBase(Car).new "path/to/db-cars" database = DODB::Storage::Basic(Car).new "path/to/db-cars"
# Adding an element to the db # Adding an element to the db
database << Car.new "Corvet", "red", ["elegant", "fast"] database << Car.new "Corvet", "red", ["elegant", "fast"]
@ -461,12 +461,12 @@ A cached database has the same API as the other DODB databases.
.QP .QP
.SOURCE Ruby ps=9 vs=10 .SOURCE Ruby ps=9 vs=10
# Create a cached database # Create a cached database
database = DODB::CachedDataBase(Car).new "path/to/db-cars" database = DODB::Storage::Cached(Car).new "path/to/db-cars"
.SOURCE .SOURCE
All operations of the All operations of the
.CLASS DODB::DataBase .CLASS DODB::Storage::Basic
class are available for class are available for
.CLASS DODB::CachedDataBase . .CLASS DODB::Storage::Cached .
.QE .QE
. .
.SS Cached indexes .SS Cached indexes
@ -502,7 +502,7 @@ Moreover, this database has exactly the same API as the others, thus changing fr
.QP .QP
.SOURCE Ruby ps=9 vs=10 .SOURCE Ruby ps=9 vs=10
# RAM-only database creation # RAM-only database creation
database = DODB::RAMOnlyDataBase(Car).new "path/to/db-cars" database = DODB::Storage::RAMOnly(Car).new "path/to/db-cars"
.SOURCE .SOURCE
Yes, the path still is required which may be seen as a quirk but the rationale\*[*] is sound. Yes, the path still is required which may be seen as a quirk but the rationale\*[*] is sound.
.QE .QE
@ -544,7 +544,7 @@ See the "Future work" section.
. .
.SS Uncached database .SS Uncached database
By default, the database (provided by By default, the database (provided by
.CLASS "DODB::DataBase" ) .CLASS "DODB::Storage::Basic" )
isn't cached. isn't cached.
. .
.SS Uncached indexes .SS Uncached indexes
@ -584,9 +584,9 @@ command enables to browse the full documentation with a web browser.
.QP .QP
.SOURCE Ruby ps=9 vs=10 .SOURCE Ruby ps=9 vs=10
# Uncached, cached and RAM-only database creation. # Uncached, cached and RAM-only database creation.
database = DODB::DataBase(Car).new "path/to/db-cars" database = DODB::Storage::Basic(Car).new "path/to/db-cars"
database = DODB::CachedDataBase(Car).new "path/to/db-cars" database = DODB::Storage::Cached(Car).new "path/to/db-cars"
database = DODB::RAMOnlyDataBase(Car).new "path/to/db-cars" database = DODB::Storage::RAMOnly(Car).new "path/to/db-cars"
.SOURCE .SOURCE
.QE .QE
. .

6
spec/benchmark-cars.cr
View File

@ -99,9 +99,9 @@ def batch()
fn = ->(storage : DODB::Storage(Car), fn = ->(storage : DODB::Storage(Car),
current_db_size : Int32, current_db_size : Int32,
name : String, name : String,
search_name : DODB::Index(Car), search_name : DODB::Index::Index(Car),
search_color : DODB::Partition(Car), search_color : DODB::Index::Partition(Car),
search_keywords : DODB::Tags(Car)) { search_keywords : DODB::Index::Tags(Car)) {
name_to_search = ENV["CARNAME"] rescue "Corvet-#{(current_db_size/2).to_i}" name_to_search = ENV["CARNAME"] rescue "Corvet-#{(current_db_size/2).to_i}"
color_to_search = ENV["CARCOLOR"] rescue "red" color_to_search = ENV["CARCOLOR"] rescue "red"

4
spec/benchmark-todo.cr
View File

@ -1,7 +1,7 @@
require "benchmark" require "benchmark"
require "./db-ships.cr" require "./db-ships.cr"
class DODBCached < DODB::CachedDataBase(Ship) class DODBCached < DODB::Storage::Cached(Ship)
def initialize(storage_ext = "", remove_previous_data = true) def initialize(storage_ext = "", remove_previous_data = true)
storage_dir = "test-storage#{storage_ext}" storage_dir = "test-storage#{storage_ext}"
@ -13,7 +13,7 @@ class DODBCached < DODB::CachedDataBase(Ship)
end end
end end
class DODBUnCached < DODB::DataBase(Ship) class DODBUnCached < DODB::Storage::Basic(Ship)
def initialize(storage_ext = "", remove_previous_data = true) def initialize(storage_ext = "", remove_previous_data = true)
storage_dir = "test-storage#{storage_ext}" storage_dir = "test-storage#{storage_ext}"

6
spec/spec-database.cr
View File

@ -1,4 +1,4 @@
class DODB::SpecDataBase(V) < DODB::DataBase(V) class DODB::SpecDataBase(V) < DODB::Storage::Basic(V)
property storage_dir : String property storage_dir : String
def initialize(storage_ext = "", remove_previous_data = true) def initialize(storage_ext = "", remove_previous_data = true)
@storage_dir = "specdb-storage-uncached#{storage_ext}" @storage_dir = "specdb-storage-uncached#{storage_ext}"
@ -15,7 +15,7 @@ class DODB::SpecDataBase(V) < DODB::DataBase(V)
end end
end end
class DODB::CachedSpecDataBase(V) < DODB::CachedDataBase(V) class DODB::CachedSpecDataBase(V) < DODB::Storage::Cached(V)
property storage_dir : String property storage_dir : String
def initialize(storage_ext = "", remove_previous_data = true) def initialize(storage_ext = "", remove_previous_data = true)
@storage_dir = "specdb-storage-cached#{storage_ext}" @storage_dir = "specdb-storage-cached#{storage_ext}"
@ -32,7 +32,7 @@ class DODB::CachedSpecDataBase(V) < DODB::CachedDataBase(V)
end end
end end
class DODB::RAMOnlySpecDataBase(V) < DODB::RAMOnlyDataBase(V) class DODB::RAMOnlySpecDataBase(V) < DODB::Storage::RAMOnly(V)
property storage_dir : String property storage_dir : String
def initialize(storage_ext = "", remove_previous_data = true) def initialize(storage_ext = "", remove_previous_data = true)
@storage_dir = "specdb-storage-ram#{storage_ext}" @storage_dir = "specdb-storage-ram#{storage_ext}"

6
spec/test-ships.cr
View File

@ -6,7 +6,7 @@ def fork_process(&)
Process.new Crystal::System::Process.fork { yield } Process.new Crystal::System::Process.fork { yield }
end end
describe "DODB::DataBase" do describe "DODB::Storage::Basic" do
describe "basics" do describe "basics" do
it "store and get data" do it "store and get data" do
db = DODB::SpecDataBase(Ship).new db = DODB::SpecDataBase(Ship).new
@ -525,7 +525,7 @@ end
# Basically the same thing as before, with some slight # Basically the same thing as before, with some slight
# differences based on the fact that changing the on-disk data # differences based on the fact that changing the on-disk data
# won't change the cached one. # won't change the cached one.
describe "DODB::CachedDataBase" do describe "DODB::Storage::Cached" do
describe "basics" do describe "basics" do
it "store and get data" do it "store and get data" do
db = DODB::CachedSpecDataBase(Ship).new db = DODB::CachedSpecDataBase(Ship).new
@ -585,7 +585,7 @@ describe "DODB::CachedDataBase" do
db2 = DODB::CachedSpecDataBase(Ship).new remove_previous_data: false db2 = DODB::CachedSpecDataBase(Ship).new remove_previous_data: false
db2 << Ship.mutsuki db2 << Ship.mutsuki
# Only difference with DODB::DataBase: concurrent DB cannot coexists. # Only difference with DODB::Storage::Basic: concurrent DB cannot coexists.
db2.to_a.size.should eq(2) db2.to_a.size.should eq(2)
db1.rm_storage_dir db1.rm_storage_dir

12
src/dodb.cr
View File

@ -1,10 +1,10 @@
require "file_utils" require "file_utils"
require "json" require "json"
require "./dodb/*" # Indexes (basic indexes, partitions, tags, etc.). require "./dodb/*" # Databases and indexes (basic indexes, partitions, tags, etc.).
require "./storage.cr" # Database specs. # dodb/storage.cr # Database specs.
require "./uncached.cr" # Uncached database. # dodb/uncached.cr # Uncached database.
require "./cached.cr" # Cached database. # dodb/cached.cr # Cached database.
require "./stacked.cr" # Reasonably cached database. # dodb/stacked.cr # Reasonably cached database.
require "./ramonly.cr" # RAM-only database. # dodb/ramonly.cr # RAM-only database.

4
src/dodb/directed_graph.cr → src/dodb/index/directed_graph.cr
View File

@ -1,11 +1,9 @@
require "file_utils" require "file_utils"
require "json" require "json"
require "./indexer.cr"
# WARNING: this code hasn't been reviewed nor used in years. # WARNING: this code hasn't been reviewed nor used in years.
class DODB::DirectedGraph(V) < DODB::Indexer(V) class DODB::Index::DirectedGraph(V) < DODB::Indexer(V)
property name : String property name : String
property key_proc : Proc(V, Array(String)) property key_proc : Proc(V, Array(String))
getter storage_root : String getter storage_root : String

15
src/dodb/index.cr → src/dodb/index/index.cr
View File

@ -1,8 +1,5 @@
require "file_utils" require "file_utils"
require "./exceptions.cr"
require "./indexer.cr"
# Basic indexes for 1-to-1 relations. # Basic indexes for 1-to-1 relations.
# Uncached version. # Uncached version.
# #
@ -30,7 +27,7 @@ require "./indexer.cr"
# NOTE: no cache, thus considered as *slow* for creation, deletion **and retrieval**. # NOTE: no cache, thus considered as *slow* for creation, deletion **and retrieval**.
# NOTE: see `CachedIndex` for a cached version, faster for retrieval. # NOTE: see `CachedIndex` for a cached version, faster for retrieval.
# NOTE: for fast operations without fs representation, see `RAMOnlyIndex`. # NOTE: for fast operations without fs representation, see `RAMOnlyIndex`.
class DODB::Index(V) < DODB::Indexer(V) class DODB::Index::Index(V) < DODB::Indexer(V)
# Name of the index, such as *id* or *color* for example. # Name of the index, such as *id* or *color* for example.
# This is an arbitrary value, mostly to create the index directory. # This is an arbitrary value, mostly to create the index directory.
# #
@ -311,7 +308,7 @@ end
# NOTE: fast for retrieval, slow for index creation and deletion (fs operations). # NOTE: fast for retrieval, slow for index creation and deletion (fs operations).
# NOTE: see `Index` for an uncached version, even less memory-hungry. # NOTE: see `Index` for an uncached version, even less memory-hungry.
# NOTE: for fast operations without fs representation, see `RAMOnlyIndex`. # NOTE: for fast operations without fs representation, see `RAMOnlyIndex`.
class DODB::CachedIndex(V) < DODB::Index(V) class DODB::Index::CachedIndex(V) < DODB::Index::Index(V)
# This hash contains the relation between the index key and the data key, used for # This hash contains the relation between the index key and the data key, used for
# **internal operations**. # **internal operations**.
# #
@ -340,7 +337,7 @@ class DODB::CachedIndex(V) < DODB::Index(V)
data.clear data.clear
end end
# Indexes the value on the file-system as `DODB::Index#index` but also puts the index in a cache. # Indexes the value on the file-system as `DODB::Index::Index#index` but also puts the index in a cache.
# #
# NOTE: used for internal operations. # NOTE: used for internal operations.
def index(key, value) def index(key, value)
@ -351,7 +348,7 @@ class DODB::CachedIndex(V) < DODB::Index(V)
@data[index_key] = key.to_i @data[index_key] = key.to_i
end end
# Removes the index of a value on the file-system as `DODB::Index#deindex` but also from # Removes the index of a value on the file-system as `DODB::Index::Index#deindex` but also from
# the cache, used for **internal operations**. # the cache, used for **internal operations**.
# #
# NOTE: used for internal operations. # NOTE: used for internal operations.
@ -387,13 +384,13 @@ end
# ``` # ```
# #
# Since there is no file-system operations, all the operations are fast. # Since there is no file-system operations, all the operations are fast.
# `DODB::RAMOnlyIndex` enables the flexibility of indexes without a file-system representation # `DODB::Index::RAMOnlyIndex` enables the flexibility of indexes without a file-system representation
# for absolute efficiency. # for absolute efficiency.
# Exactly as easy to use as the other index implementations. # Exactly as easy to use as the other index implementations.
# #
# NOTE: reasonable amount of memory used since it's just an index. # NOTE: reasonable amount of memory used since it's just an index.
# NOTE: fast for all operations, but no file-system representation. # NOTE: fast for all operations, but no file-system representation.
class DODB::RAMOnlyIndex(V) < DODB::CachedIndex(V) class DODB::Index::RAMOnlyIndex(V) < DODB::Index::CachedIndex(V)
# Indexes a value in RAM, no file-system operation. # Indexes a value in RAM, no file-system operation.
# #
# NOTE: used for internal operations. # NOTE: used for internal operations.

16
src/dodb/partition.cr → src/dodb/index/partition.cr
View File

@ -1,7 +1,5 @@
require "file_utils" require "file_utils"
require "./indexer.cr"
# Partitions for 1-to-n relations. # Partitions for 1-to-n relations.
# Uncached version. # Uncached version.
# #
@ -32,7 +30,7 @@ require "./indexer.cr"
# NOTE: no cache, thus considered as *slow* for creation, deletion **and retrieval**. # NOTE: no cache, thus considered as *slow* for creation, deletion **and retrieval**.
# NOTE: see `CachedPartition` for a cached version, faster for retrieval. # NOTE: see `CachedPartition` for a cached version, faster for retrieval.
# NOTE: for fast operations without fs representation, see `RAMOnlyPartition`. # NOTE: for fast operations without fs representation, see `RAMOnlyPartition`.
class DODB::Partition(V) < DODB::Indexer(V) class DODB::Index::Partition(V) < DODB::Indexer(V)
# Name of the index, such as *color* for example. # Name of the index, such as *color* for example.
# This is an arbitrary value, mostly to create the index directory. # This is an arbitrary value, mostly to create the index directory.
# #
@ -225,14 +223,14 @@ end
# NOTE: fast for retrieval, slow for index creation and deletion (fs operations). # NOTE: fast for retrieval, slow for index creation and deletion (fs operations).
# NOTE: see `Partition` for an uncached version, even less memory-hungry. # NOTE: see `Partition` for an uncached version, even less memory-hungry.
# NOTE: for fast operations without fs representation, see `RAMOnlyPartition`. # NOTE: for fast operations without fs representation, see `RAMOnlyPartition`.
class DODB::CachedPartition(V) < DODB::Partition(V) class DODB::Index::CachedPartition(V) < DODB::Index::Partition(V)
# This hash contains the relation between the index key and the data key, used for # This hash contains the relation between the index key and the data key, used for
# **internal operations**. # **internal operations**.
# #
# WARNING: used for internal operations, do not change its content or access it directly. # WARNING: used for internal operations, do not change its content or access it directly.
property data = Hash(String, Array(Int32)).new property data = Hash(String, Array(Int32)).new
# Indexes the value on the file-system as `DODB::Partition#index` but also puts the index in a cache. # Indexes the value on the file-system as `DODB::Index::Partition#index` but also puts the index in a cache.
# #
# NOTE: used for internal operations. # NOTE: used for internal operations.
def index(key : String, value : V) def index(key : String, value : V)
@ -250,7 +248,7 @@ class DODB::CachedPartition(V) < DODB::Partition(V)
@data[partition] = array @data[partition] = array
end end
# Removes the index of a value on the file-system as `DODB::Partition#deindex` but also from # Removes the index of a value on the file-system as `DODB::Index::Partition#deindex` but also from
# the cache, used for **internal operations**. # the cache, used for **internal operations**.
# #
# NOTE: used for internal operations. # NOTE: used for internal operations.
@ -266,7 +264,7 @@ class DODB::CachedPartition(V) < DODB::Partition(V)
end end
# Gets partition entries and the database key for each entry. # Gets partition entries and the database key for each entry.
# In `DODB::CachedPartition`, `#get_with_keys(partition : String)` is modified to retrieve data keys from # In `DODB::Index::CachedPartition`, `#get_with_keys(partition : String)` is modified to retrieve data keys from
# the index cache. # the index cache.
# In case the data isn't already in the cache, it is retrieved from the file-system. # In case the data isn't already in the cache, it is retrieved from the file-system.
# #
@ -311,14 +309,14 @@ end
# ``` # ```
# #
# Since there is no file-system operations, all the operations are fast. # Since there is no file-system operations, all the operations are fast.
# `DODB::RAMOnlyPartition` enables the flexibility of partitions without a file-system representation. # `DODB::Index::RAMOnlyPartition` enables the flexibility of partitions without a file-system representation.
# Absolute efficiency, exactly as easy to use as the other partition implementations. # Absolute efficiency, exactly as easy to use as the other partition implementations.
# #
# NOTE: reasonable amount of memory used since it's just an index. # NOTE: reasonable amount of memory used since it's just an index.
# NOTE: fast for all operations, but no file-system representation. # NOTE: fast for all operations, but no file-system representation.
# NOTE: see `Partition` for an uncached version, even less memory-hungry. # NOTE: see `Partition` for an uncached version, even less memory-hungry.
# NOTE: for an fs representation but still fast for retrieval, see `CachedPartition`. # NOTE: for an fs representation but still fast for retrieval, see `CachedPartition`.
class DODB::RAMOnlyPartition(V) < DODB::CachedPartition(V) class DODB::Index::RAMOnlyPartition(V) < DODB::Index::CachedPartition(V)
def index(key : String, value : V) def index(key : String, value : V)
partition = key_proc.call value partition = key_proc.call value
return if partition.is_a? NoIndex return if partition.is_a? NoIndex

12
src/dodb/tags.cr → src/dodb/index/tags.cr
View File

@ -31,7 +31,7 @@ require "file_utils"
# NOTE: no cache, thus considered as *slow* for creation, deletion **and retrieval**. # NOTE: no cache, thus considered as *slow* for creation, deletion **and retrieval**.
# NOTE: see `CachedTags` for a cached version, faster for retrieval. # NOTE: see `CachedTags` for a cached version, faster for retrieval.
# NOTE: for fast operations without fs representation, see `RAMOnlyTags`. # NOTE: for fast operations without fs representation, see `RAMOnlyTags`.
class DODB::Tags(V) < DODB::Indexer(V) class DODB::Index::Tags(V) < DODB::Indexer(V)
# Name of the index, such as *keywords* for example. # Name of the index, such as *keywords* for example.
# This is an arbitrary value, mostly to create the index directory. # This is an arbitrary value, mostly to create the index directory.
# #
@ -127,7 +127,7 @@ class DODB::Tags(V) < DODB::Indexer(V)
# Nothing can beat custom implementations tailored with specific problems in mind, so in case this # Nothing can beat custom implementations tailored with specific problems in mind, so in case this
# algorithm isn't fine for you, feel free to override this function for your specific data-set. # algorithm isn't fine for you, feel free to override this function for your specific data-set.
# #
# NOTE: to seriously boost performance, use `DODB::CachedTags`. # NOTE: to seriously boost performance, use `DODB::Index::CachedTags`.
# WARNING: throws a MissingEntry exception on non-existing tag or no match. # WARNING: throws a MissingEntry exception on non-existing tag or no match.
def get_with_keys(keys : Array(String)) : Array(Tuple(V, Int32)) def get_with_keys(keys : Array(String)) : Array(Tuple(V, Int32))
r_value = Array(Tuple(V, Int32)).new r_value = Array(Tuple(V, Int32)).new
@ -266,7 +266,7 @@ end
# NOTE: fast for retrieval, slow for index creation and deletion (fs operations). # NOTE: fast for retrieval, slow for index creation and deletion (fs operations).
# NOTE: see `Tags` for an uncached version, even less memory-hungry. # NOTE: see `Tags` for an uncached version, even less memory-hungry.
# NOTE: for fast operations without fs representation, see `RAMOnlyTags`. # NOTE: for fast operations without fs representation, see `RAMOnlyTags`.
class DODB::CachedTags(V) < DODB::Tags(V) class DODB::Index::CachedTags(V) < DODB::Index::Tags(V)
# This hash contains the relation between the index key and the data keys. # This hash contains the relation between the index key and the data keys.
property data = Hash(String, Array(Int32)).new property data = Hash(String, Array(Int32)).new
@ -302,7 +302,7 @@ class DODB::CachedTags(V) < DODB::Tags(V)
end end
end end
# In `DODB::CachedTags`, `#get_with_keys(tag : String)` is modified to retrieve data keys from the index cache. # In `DODB::Index::CachedTags`, `#get_with_keys(tag : String)` is modified to retrieve data keys from the index cache.
# In case the data isn't already in the cache, it is retrieved from the file-system. # In case the data isn't already in the cache, it is retrieved from the file-system.
# #
# ``` # ```
@ -346,14 +346,14 @@ end
# ``` # ```
# #
# Since there is no file-system operations, all the operations are fast. # Since there is no file-system operations, all the operations are fast.
# `DODB::RAMOnlyTags` enables the flexibility of tags without a file-system representation. # `DODB::Index::RAMOnlyTags` enables the flexibility of tags without a file-system representation.
# Absolute efficiency, exactly as easy to use as the other tag implementations. # Absolute efficiency, exactly as easy to use as the other tag implementations.
# #
# NOTE: reasonable amount of memory used since it's just an index. # NOTE: reasonable amount of memory used since it's just an index.
# NOTE: fast for all operations, but no file-system representation. # NOTE: fast for all operations, but no file-system representation.
# NOTE: see `Tags` for an uncached version, even less memory-hungry. # NOTE: see `Tags` for an uncached version, even less memory-hungry.
# NOTE: for an fs representation but still fast for retrieval, see `CachedTags`. # NOTE: for an fs representation but still fast for retrieval, see `CachedTags`.
class DODB::RAMOnlyTags(V) < DODB::CachedTags(V) class DODB::Index::RAMOnlyTags(V) < DODB::Index::CachedTags(V)
def index(key : String, value : V) def index(key : String, value : V)
tags = key_proc.call value tags = key_proc.call value
return if tags.is_a? NoIndex return if tags.is_a? NoIndex

1
src/dodb/indexer.cr
View File

@ -37,3 +37,4 @@ abstract class DODB::Indexer(V)
end end
end end
require "./index/*"

22
src/storage.cr → src/dodb/storage.cr
View File

@ -169,7 +169,7 @@ abstract class DODB::Storage(V)
# Creates a new basic index **with a cache**. # Creates a new basic index **with a cache**.
# The *name* parameter is the name of the directory that will be created. # The *name* parameter is the name of the directory that will be created.
def new_index(name : String, &block : Proc(V, String | DODB::NoIndex)) def new_index(name : String, &block : Proc(V, String | DODB::NoIndex))
CachedIndex(V).new(self, @directory_name, name, block).tap do |indexer| Index::CachedIndex(V).new(self, @directory_name, name, block).tap do |indexer|
@indexers << indexer @indexers << indexer
end end
end end
@ -179,7 +179,7 @@ abstract class DODB::Storage(V)
# #
# NOTE: this will be a lot slower than the cached version. # NOTE: this will be a lot slower than the cached version.
def new_uncached_index(name : String, &block : Proc(V, String | DODB::NoIndex)) def new_uncached_index(name : String, &block : Proc(V, String | DODB::NoIndex))
Index(V).new(self, @directory_name, name, block).tap do |indexer| Index::Index(V).new(self, @directory_name, name, block).tap do |indexer|
@indexers << indexer @indexers << indexer
end end
end end
@ -189,7 +189,7 @@ abstract class DODB::Storage(V)
# #
# NOTE: this index is the fastest, but doesn't have a file-system representation. # NOTE: this index is the fastest, but doesn't have a file-system representation.
def new_RAM_index(name : String, &block : Proc(V, String | DODB::NoIndex)) def new_RAM_index(name : String, &block : Proc(V, String | DODB::NoIndex))
RAMOnlyIndex(V).new(self, @directory_name, name, block).tap do |indexer| Index::RAMOnlyIndex(V).new(self, @directory_name, name, block).tap do |indexer|
@indexers << indexer @indexers << indexer
end end
end end
@ -203,7 +203,7 @@ abstract class DODB::Storage(V)
# Creates a new partition **with a cache**. # Creates a new partition **with a cache**.
# The *name* parameter is the name of the directory that will be created. # The *name* parameter is the name of the directory that will be created.
def new_partition(name : String, &block : Proc(V, String | DODB::NoIndex)) def new_partition(name : String, &block : Proc(V, String | DODB::NoIndex))
CachedPartition(V).new(self, @directory_name, name, block).tap do |table| Index::CachedPartition(V).new(self, @directory_name, name, block).tap do |table|
@indexers << table @indexers << table
end end
end end
@ -213,7 +213,7 @@ abstract class DODB::Storage(V)
# #
# NOTE: this will be a lot slower than the cached version. # NOTE: this will be a lot slower than the cached version.
def new_uncached_partition(name : String, &block : Proc(V, String | DODB::NoIndex)) def new_uncached_partition(name : String, &block : Proc(V, String | DODB::NoIndex))
Partition(V).new(self, @directory_name, name, block).tap do |table| Index::Partition(V).new(self, @directory_name, name, block).tap do |table|
@indexers << table @indexers << table
end end
end end
@ -223,7 +223,7 @@ abstract class DODB::Storage(V)
# #
# NOTE: this partition index is the fastest but doesn't have a file-system representation. # NOTE: this partition index is the fastest but doesn't have a file-system representation.
def new_RAM_partition(name : String, &block : Proc(V, String | DODB::NoIndex)) def new_RAM_partition(name : String, &block : Proc(V, String | DODB::NoIndex))
RAMOnlyPartition(V).new(self, @directory_name, name, block).tap do |table| Index::RAMOnlyPartition(V).new(self, @directory_name, name, block).tap do |table|
@indexers << table @indexers << table
end end
end end
@ -237,7 +237,7 @@ abstract class DODB::Storage(V)
# Creates a new tag **with a cache**. # Creates a new tag **with a cache**.
# The *name* parameter is the name of the directory that will be created. # The *name* parameter is the name of the directory that will be created.
def new_tags(name : String, &block : Proc(V, Array(String) | DODB::NoIndex)) def new_tags(name : String, &block : Proc(V, Array(String) | DODB::NoIndex))
CachedTags(V).new(self, @directory_name, name, block).tap do |tags| Index::CachedTags(V).new(self, @directory_name, name, block).tap do |tags|
@indexers << tags @indexers << tags
end end
end end
@ -247,7 +247,7 @@ abstract class DODB::Storage(V)
# #
# NOTE: this will be a lot slower than the cached version. # NOTE: this will be a lot slower than the cached version.
def new_uncached_tags(name : String, &block : Proc(V, Array(String) | DODB::NoIndex)) def new_uncached_tags(name : String, &block : Proc(V, Array(String) | DODB::NoIndex))
Tags(V).new(self, @directory_name, name, block).tap do |tags| Index::Tags(V).new(self, @directory_name, name, block).tap do |tags|
@indexers << tags @indexers << tags
end end
end end
@ -257,7 +257,7 @@ abstract class DODB::Storage(V)
# #
# NOTE: this tag index is the fastest but doesn't have a file-system representation. # NOTE: this tag index is the fastest but doesn't have a file-system representation.
def new_RAM_tags(name : String, &block : Proc(V, Array(String) | DODB::NoIndex)) def new_RAM_tags(name : String, &block : Proc(V, Array(String) | DODB::NoIndex))
RAMOnlyTags(V).new(self, @directory_name, name, block).tap do |tags| Index::RAMOnlyTags(V).new(self, @directory_name, name, block).tap do |tags|
@indexers << tags @indexers << tags
end end
end end
@ -270,7 +270,7 @@ abstract class DODB::Storage(V)
# WARNING: directed graphs haven't been reviewed in YEARS, assume as dead code. # WARNING: directed graphs haven't been reviewed in YEARS, assume as dead code.
def new_directed_graph(name : String, index : DODB::Index(V), &block : Proc(V, Array(String))) : DirectedGraph(V) def new_directed_graph(name : String, index : DODB::Index(V), &block : Proc(V, Array(String))) : DirectedGraph(V)
DirectedGraph(V).new(self, @directory_name, index, name, block).tap do |table| Index::DirectedGraph(V).new(self, @directory_name, index, name, block).tap do |table|
@indexers << table @indexers << table
end end
end end
@ -465,3 +465,5 @@ abstract class DODB::Storage(V)
end end
end end
end end
require "./storage/*"

8
src/cached.cr → src/dodb/storage/cached.cr
View File

@ -17,7 +17,7 @@ end
# #
# ``` # ```
# # Creates a DODB cached database. # # Creates a DODB cached database.
# car_database = DODB::CachedDataBase.new "/path/to/db" # car_database = DODB::Storage::Cached.new "/path/to/db"
# #
# # Creates a (cached) index. # # Creates a (cached) index.
# cars_by_name = car_database.new_index "name", &.name # cars_by_name = car_database.new_index "name", &.name
@ -35,8 +35,8 @@ end
# │   └── Corvet -> ../../data/0000000000 # │   └── Corvet -> ../../data/0000000000
# ``` # ```
# #
# WARNING: beware of the RAM use, see `DODB::StackedDataBase` for a less memory-hungry option. # WARNING: beware of the RAM use, see `DODB::Storage::Stacked` for a less memory-hungry option.
class DODB::CachedDataBase(V) < DODB::Storage(V) class DODB::Storage::Cached(V) < DODB::Storage(V)
@indexers = [] of Indexer(V) @indexers = [] of Indexer(V)
property data = Hash(Int32, V).new property data = Hash(Int32, V).new
@ -51,7 +51,7 @@ class DODB::CachedDataBase(V) < DODB::Storage(V)
end end
# Load the database in RAM at start-up. # Load the database in RAM at start-up.
DODB::DataBase(V).new(@directory_name).each_with_key do |v, key| DODB::Storage::Basic(V).new(@directory_name).each_with_key do |v, key|
puts "\rloading data from #{@directory_name} at key #{key}" puts "\rloading data from #{@directory_name} at key #{key}"
self[key] = v self[key] = v
end end

8
src/ramonly.cr → src/dodb/storage/ramonly.cr
View File

@ -5,7 +5,7 @@
# Indexing (basic indexes, partitions, tags) will behave the same way. # Indexing (basic indexes, partitions, tags) will behave the same way.
# ``` # ```
# # Creates a DODB RAM-only database (yes, the path is still required). # # Creates a DODB RAM-only database (yes, the path is still required).
# car_database = DODB::RAMOnlyDataBase.new "/path/to/db" # car_database = DODB::Storage::RAMOnly.new "/path/to/db"
# #
# # Creates a (cached) index (same as for all other DODB database implementations). # # Creates a (cached) index (same as for all other DODB database implementations).
# cars_by_name = car_database.new_index "name", &.name # cars_by_name = car_database.new_index "name", &.name
@ -21,7 +21,7 @@
# │   └── by_name <- the "name" basic index # │   └── by_name <- the "name" basic index
# │   └── Corvet -> ../../data/0000000000 <- the index works despite not pointing to a real file # │   └── Corvet -> ../../data/0000000000 <- the index works despite not pointing to a real file
# ``` # ```
class DODB::RAMOnlyDataBase(V) < DODB::CachedDataBase(V) class DODB::Storage::RAMOnly(V) < DODB::Storage::Cached(V)
# Initialization still uses a directory name and creates a few paths. # Initialization still uses a directory name and creates a few paths.
# This is an implementation detail to re-use code of `DODB::Storage` and to get the indexers to work. # This is an implementation detail to re-use code of `DODB::Storage` and to get the indexers to work.
def initialize(@directory_name : String) def initialize(@directory_name : String)
@ -30,12 +30,12 @@ class DODB::RAMOnlyDataBase(V) < DODB::CachedDataBase(V)
@last_key = -1 @last_key = -1
end end
# The `last_key` function doesn't read a file in the `DODB::RAMOnlyDataBase` database. # The `last_key` function doesn't read a file in the `DODB::Storage::RAMOnly` database.
def last_key def last_key
@last_key @last_key
end end
# The `last_key=` function doesn't write to a file in the `DODB::RAMOnlyDataBase` database. # The `last_key=` function doesn't write to a file in the `DODB::Storage::RAMOnly` database.
def last_key=(key : Int32) def last_key=(key : Int32)
@last_key = key @last_key = key
end end

2
src/stacked.cr → src/dodb/storage/stacked.cr
View File

@ -4,7 +4,7 @@
# #
# NOTE: fast for frequently requested data and requires a stable (and configurable) amount of memory. # NOTE: fast for frequently requested data and requires a stable (and configurable) amount of memory.
# TODO: not yet implemented. # TODO: not yet implemented.
class DODB::StackedDataBase(V) < DODB::CachedDataBase(V) class DODB::Storage::Stacked(V) < DODB::Storage::Cached(V)
# The maximum number of accepted entries in the cache. # The maximum number of accepted entries in the cache.
property max_entries : Int32 property max_entries : Int32

4
src/uncached.cr → src/dodb/storage/uncached.cr
View File

@ -3,7 +3,7 @@
# #
# ``` # ```
# # Creates a DODB (uncached) database. # # Creates a DODB (uncached) database.
# car_database = DODB::DataBase.new "/path/to/db" # car_database = DODB::Storage::Basic.new "/path/to/db"
# #
# # Creates a (cached) index. # # Creates a (cached) index.
# cars_by_name = car_database.new_index "name", &.name # cars_by_name = car_database.new_index "name", &.name
@ -22,5 +22,5 @@
# ``` # ```
# #
# NOTE: slow but doesn't require much memory. # NOTE: slow but doesn't require much memory.
class DODB::DataBase(V) < DODB::Storage(V) class DODB::Storage::Basic(V) < DODB::Storage(V)
end end