Baguette
/
dodb.cr
3
0
Fork 1
Code Issues Pull Requests Releases Wiki Activity

Common

toying-with-ramdb
Philippe PITTOLI 2024-05-28 03:29:01 +02:00
parent c105b7fa33
commit f29fbfb850
2 changed files with 43 additions and 25 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

53
paper/paper.ms
View File

@ -197,7 +197,7 @@ end
Let's create a DODB database for our cars.
.SOURCE Ruby ps=9 vs=10
# Database creation
database = DODB::Storage::Basic(Car).new "path/to/db-cars"
database = DODB::Storage::Uncached(Car).new "path/to/db-cars"
# Adding an element to the db
database << Car.new "Corvet", "red", ["elegant", "fast"]
@ -485,7 +485,7 @@ A cached database has the same API as the other DODB databases and keeps a copy
database = DODB::Storage::Cached(Car).new "path/to/db-cars"
.SOURCE
All operations of the
.CLASS Storage::Basic
.CLASS Storage::Uncached
class are available for
.CLASS Storage::Cached .
.QE
@ -532,7 +532,7 @@ along with its related data from the cache.
.B "Implementation details" .
The implementation is time-efficient;
the duration of adding a value is constant, it doesn't change with the number of entries.
the duration of adding a value is almost constant, it doesn't change much with the number of entries.
This efficiency is a memory tradeoff.
All the entries are added to a
.B "double-linked list"
@ -549,6 +549,15 @@ Moreover,
.I "common database"
enables to adjust the number of stored entries.
.
.QP
.SOURCE Ruby ps=9 vs=10
# Create a database with a data cache limited to 100.000 entries
database = DODB::Storage::Common(Car).new "path/to/db-cars", 100000
.SOURCE
The
.CLASS Storage::Common
class has the same API as the other database classes.
.QE
.
.SECTION RAM-only database for short-lived data
Databases are built around the objective to actually
@ -610,23 +619,26 @@ This way, one can opt for a cached index and, after some time not using the file
.
.
.SECTION DODB and memory constraint
In contrast with the previous section, some environments have a memory constraint.
For example, in case the database is larger than the available memory, it won't be possible to use a data cache\*[*].
Some environments may have very peculiar constraints, where caching data would cause problems or would be inefficient anyway\*[*].
.FOOTNOTE1
Keep in mind that for the moment "cached database" means "all data in memory".
It is perfectly reasonable to have a cached database with a policy of keeping just a certain amount of values in memory, in order to limit the memory required by selecting the relevant values to keep in cache (the most recently used, for example).
But for now, the cached version keeps everything.
See the "Future work" section.
Caching would be inefficient for databases where the distribution of requests is homogeneous between the different entries, for example.
If the requests are random, without a small portion of the data receiving most requests (such as a Pareto distribution), caching becomes mostly irrelevant.
.FOOTNOTE2
.B "Uncached database" .
The
In these cases, the
.CLASS "DODB::Storage::Uncached"
database has no data cache at all and can be used in very constrained environments.
can be used\*[*].
.FOOTNOTE1
However, the
.CLASS DODB::Storage::Common
should (probably) be considered instead, even if the configured number of entries is low.
A small data cache is still better than no cache.
should be considered instead for most applications, even if the configured number of entries is low due to low RAM.
.FOOTNOTE2
.
.QP
.SOURCE Ruby ps=9 vs=10
# Uncached database creation
database = DODB::Storage::Uncached(Car).new "path/to/db-cars"
.SOURCE
.QE
.B "Uncached indexes" .
Cached indexes do not require a large amount of memory since the only stored data is an integer (the
@ -664,10 +676,11 @@ command enables to browse the full documentation with a web browser.
.SS Database creation
.QP
.SOURCE Ruby ps=9 vs=10
# Uncached, cached and RAM-only database creation.
database = DODB::Storage::Basic(Car).new "path/to/db-cars"
database = DODB::Storage::Cached(Car).new "path/to/db-cars"
database = DODB::Storage::RAMOnly(Car).new "path/to/db-cars"
# Uncached, cached, common and RAM-only database creation.
database = DODB::Storage::Uncached(Car).new "path/to/db"
database = DODB::Storage::Cached(Car).new "path/to/db"
database = DODB::Storage::Common(Car).new "path/to/db", 50000 # nb cache entries
database = DODB::Storage::RAMOnly(Car).new "path/to/db"
.SOURCE
.QE
.
@ -712,7 +725,7 @@ function.
.KE
.
.
.SS Triggers creation
.SS Trigger creation
.QP
.SOURCE Ruby ps=9 vs=10
# Uncached, cached and RAM-only basic indexes.

15
spec/benchmark-cars.cr
View File

@ -171,16 +171,19 @@ def bench_50_shades_of_fifo()
cars_fifo1 = SPECDB::Common(Car).new "-1k", 1_000
cars_fifo5 = SPECDB::Common(Car).new "-5k", 5_000
cars_fifo10 = SPECDB::Common(Car).new "-10k", 10_000
cars_fifo20 = SPECDB::Common(Car).new "-20k", 20_000
fifo_Sby_name1, fifo_Sby_color1, fifo_Sby_keywords1 = cached_indexes cars_fifo1
fifo_Sby_name5, fifo_Sby_color5, fifo_Sby_keywords5 = cached_indexes cars_fifo5
fifo_Sby_name10, fifo_Sby_color10, fifo_Sby_keywords10 = cached_indexes cars_fifo10
fifo_Sby_name20, fifo_Sby_color20, fifo_Sby_keywords20 = cached_indexes cars_fifo20
fn = ->search_benchmark(DODB::Storage(Car), Int32, String, DODB::Trigger::Index(Car), DODB::Trigger::Partition(Car), DODB::Trigger::Tags(Car))
prepare_env cars_fifo1, "fifo1", fifo_Sby_name1, fifo_Sby_color1, fifo_Sby_keywords1, &fn
prepare_env cars_fifo5, "fifo5", fifo_Sby_name5, fifo_Sby_color5, fifo_Sby_keywords5, &fn
prepare_env cars_fifo10, "fifo10", fifo_Sby_name10, fifo_Sby_color10, fifo_Sby_keywords10, &fn
prepare_env cars_fifo20, "fifo20", fifo_Sby_name20, fifo_Sby_color20, fifo_Sby_keywords20, &fn
end
ENV["REPORT_DIR"]?.try { |report_dir| Context.report_dir = report_dir }
@ -193,11 +196,13 @@ ENV["DBSIZE_START"]?.try { |it| Context.from = it.to_i }
ENV["DBSIZE_INCREMENT"]?.try { |it| Context.incr = it.to_i }
ENV["FIFO_SIZE"]?.try { |it| Context.fifo_size = it.to_u32 }
pp! Context.nb_run
pp! Context.from
pp! Context.to
pp! Context.incr
pp! Context.max_indexes
puts "REPORT_DIR: #{Context.report_dir}"
puts "MAXINDEXES: #{Context.max_indexes}"
puts "NBRUN: #{Context.nb_run}"
puts "DBSIZE: #{Context.to}"
puts "DBSIZE_START: #{Context.from}"
puts "DBSIZE_INCREMENT: #{Context.incr}"
puts "FIFO_SIZE: #{Context.fifo_size}"
if ARGV.size == 0
puts "Usage: benchmark-cars (fifo|search|add)"