From 71d8fa53d5bb33258d827d12a02251c1231e6d6c Mon Sep 17 00:00:00 2001 From: Philippe PITTOLI Date: Mon, 13 May 2024 02:24:59 +0200 Subject: [PATCH] Longer explanation of the experimental scenario. --- graphs/graphs.ms | 83 +++++- graphs/macros.roff | 621 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 696 insertions(+), 8 deletions(-) create mode 100644 graphs/macros.roff diff --git a/graphs/graphs.ms b/graphs/graphs.ms index 3c212d4..b55babf 100644 --- a/graphs/graphs.ms +++ b/graphs/graphs.ms @@ -1,7 +1,76 @@ -.NH -Indexes -.LP +.so macros.roff +.TITLE Brief performance analysis of Document Oriented DataBase (DODB) +.AUTHOR Philippe P. +.ABSTRACT1 +DODB is a database-as-library, enabling a very simple way to store applications' data: storing serialized +.I documents +(basically any data type) in plain files. +To speed-up searches, attributes of these documents can be used as indexes which leads to create a few symbolic links +.I symlinks ) ( +on the disk. +.br +See the \f[CW]README\f[] for a longer explanation. +This document briefly presents an experiment to understand the performances we can get with this approach. +.br +.UL Status : +WIP +.ABSTRACT2 +.SECTION Experimental scenario +.LP +The following experiment shows the performance of DODB based on quering durations. +Data can be searched via +.I indexes , +as for SQL databases. +Three possible indexes exist in DODB: +(a) basic indexes, representing 1 to 1 relations, the document's attribute is related to a value and each value of this attribute is unique, +(b) partitions, representing 1 to n relations, the attribute has a value and this value can be shared by other documents, +(c) tags, representing n to n relations, enabling the attribute to have multiple values whose are shared by other documents. + +The scenario is simple: adding values to a database with indexes (basic, partitions and tags) then query 100 times a value based on the different indexes. +Loop and repeat. + +Four instances of DODB are tested: +.BULLET \fIuncached database\f[] shows the achievable performance with a strong memory constraint (nothing can be kept in-memory) ; +.BULLET \fIuncached data but cached index\f[] shows the improvement you can expect by having a cache on indexes ; +.BULLET \fIcached database\f[] shows the most basic use of DODB\*[*] ; +.BULLET \fIRAM only\f[], the database doesn't have a representation on disk (no data is written on it). +The \fIRAM only\f[] instance shows a possible way to use DODB: to keep a consistent API to store data, including in-memory data with a lifetime related to the application's. +.ENDBULLET +.FOOTNOTE1 +Having a cached database will probably be the most widespread use of DODB. +When memory isn't scarce, there is no point not using it to achieve better performance. +.FOOTNOTE2 + +The computer on which this test is performed\*[*] is a AMD PRO A10-8770E R7 (4 cores), 2.8 GHz.When mentioned, the +.I disk +is actually a +.I "temporary file-system (tmpfs)" +to enable maximum efficiency. +.FOOTNOTE1 +A very simple $50 PC, buyed online. +Nothing fancy. +.FOOTNOTE2 + +The library is written in Crystal and so is the benchmark (\f[CW]spec/benchmark-cars.cr\f[]). +Nonetheless, despite a few technicalities, the objective of this document is to provide an insight on the approach used in DODB more than this particular implementation. + +The manipulated data type can be found in \f[CW]spec/db-cars.cr\f[]. +.SOURCE Ruby ps=9 vs=9p +class Car + property name : String # 1-1 relation + property color : String # 1-n relation + property keywords : Array(String) # n-n relation +end +.SOURCE +. +.SECTION Basic indexes (1 to 1 relations) +.LP +An index enables to match a single value based on a small string. +Since there is only one value to retrieve, the request is quick and time is almost constant. +When the value and the index are kept in memory (see \f[CW]RAM only\f[] and \f[CW]Cached db\f[]), the retrieval is almost instantaneous (about 50 to 120 ns). +In case the value is on the disk, deserialization takes about 15 µs (see \f[CW]Uncached db, cached index\f[]). +The request is a little longer when the index isn't cached, in this case DODB walks the file-system to find the right symlink to follow, thus slowing the process even more, by up to 20%. .G1 copy "legend.grap" frame invis ht 3 wid 4 left solid bot solid @@ -71,9 +140,8 @@ copy "../data/index.d" thru X .gcolor X .G2 - -.NH -Partitions +.bp +.SECTION Partitions (1 to n relations) .LP .G1 copy "legend.grap" @@ -142,8 +210,7 @@ copy "../data/partitions.d" thru X X .G2 .bp -.NH -Tags +.SECTION Tags (n to n relations) .LP .G1 copy "legend.grap" diff --git a/graphs/macros.roff b/graphs/macros.roff new file mode 100644 index 0000000..ff810a3 --- /dev/null +++ b/graphs/macros.roff @@ -0,0 +1,621 @@ +.\" .RP = report document +.nr PO 0.5i \" page offset default 1i +.nr LL 7.5i \" line length default 6i +.nr FM 0.8i \" page foot margin default 1i +.nr DI 0 +.nr FF 3 \" footnotes' type: numbered, with point, indented +. +.nr LIST_NUMBER 0 +1 +. +.R1 +no-label-in-reference +accumulate +.R2 +. +. \" COLORS +.defcolor darkgreen rgb 0.1 0.5 0.2 +.defcolor darkblue rgb 0.3 0.3 0.7 +.defcolor darkred rgb 0.7 0.3 0.3 +.defcolor black rgb 0 0 0 +.defcolor color_box rgb 1 1 .6 +. +. \" with semantic +.defcolor citation rgb 0.4 0.4 0.4 +.defcolor citationbar rgb 0.3 0.3 0.7 +.defcolor explanation rgb 0.7 0.4 0.4 +.defcolor explanationbar rgb 0.8 0.3 0.3 +. +.defcolor specialcolor_command rgb 0.7 0.3 0.3 +.defcolor specialcolor_type rgb 0.6 0.3 0.5 +.defcolor specialcolor_constructor rgb 0.1 0.5 0.2 +.defcolor specialcolor_module rgb 0.1 0.5 0.2 +.defcolor specialcolor_function rgb 0.4 0.4 0.7 +.defcolor specialcolor_question rgb 0.0 0.0 0.7 +.defcolor specialcolor_operator rgb 0.3 0.8 0.3 +.defcolor specialcolor_shine rgb 0.3 0.3 0.7 +. +. \" SIZES +.nr specialsize_command 10 +.nr specialsize_type 8 +.nr specialsize_constructor 8 +.nr specialsize_module 8 +.nr specialsize_function 8 +.nr specialsize_operator 9 +.nr specialsize_question 10 \" Current point size, no change. +.nr specialsize_shine 11 +. +. \" FONTS +.ds specialfont_command CW +.ds specialfont_type CW +.ds specialfont_constructor CW +.ds specialfont_module CW +.ds specialfont_function CW +.ds specialfont_operator CW +.ds specialfont_question I +.ds specialfont_shine B +. +. +.de BELLOWEXPLANATION1 +.sp 0.5 +.ps 7 \" point size (~= font size) +.vs 8p \" vertical spacing between lines +.. +.de BELLOWEXPLANATION2 +.br +.ps 9 +.vs 11p +.. +. +.\" BULLET and ENUM => do not add space when no parameter are provided +.de BULLET \" Bullet points +.IP \(bu 2 +.ie '\\$1'' \ +. +.el \\$* +.. +.de ENDBULLET +.in -2 \" indent +.. +. +.de ENUM \" Numbered list +.nr LIST_NUMBER +1 +.IP \\n[LIST_NUMBER] 2 +.ie '\\$1'' \ +. +.el \\$* +.. +.de ENDENUM +.nr LIST_NUMBER 0 +.in -2 \" indent +.. +. +.de b1 \" Begin code box +.B1 +.sp 0.2 +.ft CW +.. +.de b2 \" End code box +.sp 0.5 +.B2 +.ft +.. +. +.de CITATION1 +.KS \" start a keep +.ft I \" citation in italics +.mk C \" set a marker for line drawing +.in +1 \" indent a bit +.gcolor citation +.. +.ig + The CITATION2 macro closes the quote then draws a line + from current line to the start of the quote. +.. +.de CITATION2 +.mk D \" set second marker to come back here +.ft \" back to previous font +.in -1 \" remove indent +.gcolor \" remove previous color +.gcolor citationbar +.\" r = move upward +.\" Z D t = drawing thickness +.\" L = draw the line +\r\ +\Z'\D't 1p''\ +\L'|\\nCu' \" draw line +.gcolor black \" remove previous color +.sp -2 \" get two lines back +\Z'\D't 1'' \" get the previous drawing thickness back +.KE \" end of the keep +.. +. +.de NAMECITATION +.QP +.vs -\\n[legendps]p +.ps -\\n[legendps] +.in -1.2 +.ll +1.2 +\h'|-2'\(em\h'|-0.4' +\\$* +.br +.LP +.. +. +.de EXPLANATION1 +.KS \" start a keep +.ft B \" citation in italics +.mk C \" set a marker for line drawing +.in +1 \" indent a bit +.gcolor explanation +.. +.de EXPLANATION2 +.ft \" back to previous font +.in -1 \" remove indent +.gcolor \" remove previous color +.gcolor explanationbar +\r\L'|\\nCu' \" draw line (\r moves upward, \L draw the line, ...) +.gcolor \" remove previous color +.sp -1 \" get two lines back +.KE \" end of the keep +.. +. +.de METAINFO1 +.ft CW \" constant width font +.ps 8 \" small font +.vs 9p \" smaller vertical spacing between lines +.. +.de METAINFO2 +.sp 1 +.vs \" come back to the previous vertical spacing +.ps \" come back to the previous point size +.ft \" come back to the previous font +.sp -1 \" return one line above +.. +. +. +.de FRAC +.ie '\\$3'' \{\ +\v'-.7m\s[\\n(.s*6u/10u]+.7m'\\$1\v'-.7m\s0+.7m'\ +\(f/\s[\\n(.s*6u/10u]\\$2\s0 +\} +.el \{\ +\v'-.7m\s[\\n(.s*6u/10u]+.7m'\\$1\v'-.7m\s0+.7m'\ +\(f/\s[\\n(.s*6u/10u]\\$2\s0\\$3 +\} +.. +.de FOOTNOTE_TO_COLUMN_WIDTH +.nr pg@fn-colw \\n[pg@colw] \" footnotes' column width +.. +.de SINGLE_COLUMN +.1C +.FOOTNOTE_TO_COLUMN_WIDTH +.. +.de TWO_COLUMNS +.2C +.FOOTNOTE_TO_COLUMN_WIDTH +.. +.de HORIZONTALLINE +\l'15' +.FOOTNOTE_TO_COLUMN_WIDTH +.. +. +. \" Fonts and colors. +. +.de SPECIAL_WORDS +.ie !'\\$3'' \\$3\c +.nr current_size \\n[.s] \" Current point size. +.gcolor specialcolor_\\*[semantictoken] +. +.if !((\\n[current_size] == \\n[specialsize_\\*[semantictoken]]) \ +.ps \\n[specialsize_\\*[semantictoken]] +. +.ie '\\$2'' \{\ +\f[\\*[specialfont_\\*[semantictoken]]]\\$1\f[] +. ps \\n[current_size] +. gcolor black \" FIXME: should be the previous color +\} +.el \{\ +\f[\\*[specialfont_\\*[semantictoken]]]\\$1\f[]\c +. ps \\n[current_size] +. gcolor black \" FIXME: should be the previous color +\\$2 +\} +.. +.de SMALLFONT +.ps 8 +.vs 9p +.. +.de NORMALFONT +.vs +.ps +.. +.de COMMAND1 +.b1 +.. +.de COMMAND2 +.b2 +.. +.de COMMANDNAME +.ds semantictoken command +.SPECIAL_WORDS \\$@ +.. +.de FUNCTION +.ds semantictoken function +.SPECIAL_WORDS \\$@ +.. +.de TYPE +.ds semantictoken type +.SPECIAL_WORDS \\$@ +.. +.de TYPECLASS +.I "\\$1" "\\$2" +.. +.de OPERATOR +.ds semantictoken operator +.SPECIAL_WORDS \\$@ +.. +.de QUESTION +.ds semantictoken question +.SPECIAL_WORDS \\$@ +\h'5p' +.. +.de CONSTRUCTOR +.ds semantictoken constructor +.SPECIAL_WORDS \\$@ +.. +.de MODULE +.ds semantictoken module +.SPECIAL_WORDS \\$@ +.. +.de SHINE +.ds semantictoken shine +.SPECIAL_WORDS \\$@ +.. +.de MODULEX +.MODULE \\$1 , +.. +.de TBD +.ft B +To be defined or to finish. +.ft R +.. +.de ARROW +.br +\(->\h'5p' \\$* +.. +.af dy 00 +.af mo 00 +.ds CURRENT_DATE \\n(dy/\\n(mo/\\n[year] +.ds WEBSITE https://t.karchnu.fr/doc +.ds EMAIL karchnu@karchnu.fr +.de INFORMATIONS +Check out for newer versions: +.ft CW +.ps 8 +\h'2p' \\$1 +.ps +.ft +.br +And if you have questions: +.ft CW +\h'13p' \\$2 +.ft +.\" .DE + +.LP +Lastly compiled the +.SHINE \*[CURRENT_DATE] +(day/month/year, you know, like in any sane civilization). +.. +.de INFORMATIONS_FR +.LP +Nouvelles versions : +.ft CW +.ps 8 +\h'2p' \\$1 +.ps +.ft +.br +Questions : +.ft CW +\h'36p' \\$2 +.ft +.\" .DE + +.LP +Compilé pour la dernière fois le +.SHINE \*[CURRENT_DATE] +.. +. +.\" RENAMING REQUESTS +. +.de SECTION +.NH +.ps +3 +.fam H \" helvetica family +\\$* +.fam \" back to previous font family +.ps +.PARAGRAPH_INDENTED +.. +.de SUBSECTION +.NH 2 +.ps +1 +.fam H \" helvetica family +\\$* +.fam \" back to previous font family +.ps +.PARAGRAPH_INDENTED +.. +.de SUBSUBSECTION +.NH 3 +.fam H \" helvetica family +\\$* +.fam \" back to previous font family +.ps +.PARAGRAPH_INDENTED +.. +.de SUBSUBSUBSECTION +.NH 4 +.fam H \" helvetica family +\\$* +.fam \" back to previous font family +.PARAGRAPH_INDENTED +.. +.de SECTION_NO_NUMBER +.SH +.fam H \" helvetica family +\\$* +.fam \" back to previous font family +.PARAGRAPH_INDENTED +.. +.de SUBSECTION_NO_NUMBER +.SH 2 +.fam H \" helvetica family +\\$* +.fam \" back to previous font family +.PARAGRAPH_INDENTED +.. +.de PARAGRAPH_INDENTED +.PP +.. +.de PARAGRAPH_UNINDENTED +.LP +.. +.de NO_ABSTRACT +.AB no +.. +.de ABSTRACT1 +.AB +.. +.de ABSTRACT2 +.AE +.. +.ds CH Page % +.de TITLE +.TL +\\$* +.ds LH \\$* +.de HD .XX +.sp -2.8 +\l'7.5i' +.sp +1.5 +.br +..XX +.. +.de AUTHOR +. AU +. ie !'\\$1'' \\$* +.. +.de FOOTNOTE1 +. FS +.. +.de FOOTNOTE2 +. FE +.. +.de VOCABULARY1 +. KS +. BULLET +. UL "\\$*" : +.. +.de VOCABULARY2 +. KE +.. +. +. +.de HIGHLIGHT +. +. nr @wd \w'\\$1' +. nr x1 0 +. nr y1 (\\n[rst]u - \\n[rsb]u + .4m) +. nr x2 (\\n[@wd]u + .4m) +. nr y2 0 +. nr x3 0 +. nr y3 (\\n[rst]u - \\n[rsb]u + .4m) +. nr x4 (\\n[@wd]u + .4m) +. nr y4 0 +. +\h'.2m'\ +\h'-.2m'\v'(.2m - \\n[rsb]u)'\ +\M[color_box]\ +\D'P \\n[x1] -\\n[y1]u \\n[x2]u \\n[y2]u \\n[x3]u \\n[y3]u -\\n[x4]u \\n[y4]u '\ +\h'.2m'\v'-(.2m - \\n[rsb]u)'\ +\M[]\ +\\$1\ +\h'.2m' +.. +. +. +. +.ds SPACE_SS_NUMBER_TITLE 0.5\" not a number register because of leading 0 +.nr CURRENT_SECTION 0 +1 +.nr CURRENT_APPENDIX 0 +.af CURRENT_APPENDIX I +.nr CURRENT_SUBSECTION 0 +1 +.nr CURRENT_SSSECTION 0 +1 +.rm SECTION +.de SECTION +. nr CURRENT_SUBSECTION 0 \" reset current subsection numbering +. nr CURRENT_SSSECTION 0 \" reset current subsubsection numbering +. ie !(\\n[CURRENT_SECTION]=0) .sp +1 +. br +. ie (\\n[APPENDIX_TIME]=0) \ +. ds RH \\n+[CURRENT_SECTION].\h'\\*[SPACE_SS_NUMBER_TITLE]' \\$* +. el \{ +. ds RH \\n[CURRENT_APPENDIX].\h'\\*[SPACE_SS_NUMBER_TITLE]' \\$* +. bp \} +. ps +2 +. fam H \" helvetica family +. ft B +. ne 4 \" should be at least a few lines left at the bottom of the page +\\*[RH] +. ft +. fam \" back to previous font family +. ps -2 +. PARAGRAPH_INDENTED +.. +.nr APPENDIX_TIME 0 +.de APPENDIX +. nr CURRENT_APPENDIX +1 +. nr APPENDIX_TIME 1 +. SECTION \\$* +.. +.de SS +. nr CURRENT_SSSECTION 0 +. ie (\\n[APPENDIX_TIME]=0) \ +. SUBSECTION_NO_NUMBER \\n[CURRENT_SECTION].\ +\\n+[CURRENT_SUBSECTION]\h'\\*[SPACE_SS_NUMBER_TITLE]' \\$* +.el \ +. SUBSECTION_NO_NUMBER \\n[CURRENT_APPENDIX].\ +\\n+[CURRENT_SUBSECTION]\h'\\*[SPACE_SS_NUMBER_TITLE]' \\$* +.. +.de SSS +. br +. ps -2 +. fam H \" helvetica family +. ft B +. ie (\\n[APPENDIX_TIME]=0) \ +. SUBSECTION_NO_NUMBER \\n[CURRENT_SECTION].\ +\\n[CURRENT_SUBSECTION].\\n+[CURRENT_SSSECTION]\h'\ +\\*[SPACE_SS_NUMBER_TITLE]' \\$* +. el \ +\\n[CURRENT_APPENDIX].\ +\\n[CURRENT_SUBSECTION].\\n+[CURRENT_SSSECTION]\h'\ +\\*[SPACE_SS_NUMBER_TITLE]' \\$* +. ft +. fam \" back to previous font family +. ps +2 +. PARAGRAPH_INDENTED +.. +.de INNERBULLET +. in +1 +. br +\(bu +. in +1 +. sp -1 +\\$* +. in -2 +.. +.de EENUM \" Numbered list +. nr ENUM_INDENTATION 2 +. ie !(\\n[LIST_NUMBER]=0) .in -\\n[ENUM_INDENTATION] +. br +\\n+[LIST_NUMBER]. +. in +\\n[ENUM_INDENTATION] +. sp -1 +\\$* +.. +.de EENDENUM +. nr LIST_NUMBER 0 +. in -\\n[ENUM_INDENTATION] +.. +.nr legendps 2 +.de LEGEND1 +. QP +. vs -\\n[legendps]p +. ps -\\n[legendps] +. in -1.2 +. ll +1.2 +. br +.. +.de LEGEND2 +. br +. vs +\\n[legendps]p +. ps +\\n[legendps] +. br +. LP +.. +.de IEME +\\$1\u\s-4\\$2\s+4\d +.. +.de CENTERED +. ce +\\$* +. br +.. +.de GIVEEXAMPLE1 +. in +1 +. ll -1 +. KS \" start a keep +. \" .ft I \" citation in italics +. mk C \" set a marker for line drawing +. in +1 \" indent a bit +. gcolor citation +.. +.de GIVEEXAMPLE2 +. mk D \" set second marker to come back here +. \" .ft \" back to previous font +. in -1 \" remove indent +. gcolor black\" remove previous color +. gcolor citationbar +. \" r = move upward +. \" Z D t = drawing thickness +. \" L = draw the line +\r\ +\Z'\D't 1p''\ +\L'|\\nCu' \" draw line +. gcolor black \" remove previous color +. sp -2 \" get two lines back +\Z'\D't 0.5p'' \" get the previous drawing thickness back +. KE \" end of the keep +. ll +1 +. in -1 +.. +.de ST +.nr ww \w'\\$1' +\Z@\v'-.25m'\l'\\n[ww]u'@\\$1 +.. +.de INCREMENT +.br +.in \\*[PINCREMENT] +.br +\h'-\\*[DECALAGE]'\\*[CHARACTER]\h'|0'\\$* +.. +.de D +.ds DECALAGE 1.0 +.ds PINCREMENT 2 +.ds CHARACTER \(bu +.INCREMENT \\$* +.. +.de DD +.ds DECALAGE 1.0 +.ds PINCREMENT 3 +.ds CHARACTER \(bu +.INCREMENT \\$* +.. +.de AA +.ds DECALAGE 1.5 +.ds PINCREMENT 3 +.ds CHARACTER \(-> +.INCREMENT \\$* +.. +.de AAA +.ds DECALAGE 1.5 +.ds PINCREMENT 4 +.ds CHARACTER \(-> +.INCREMENT \\$* +.. +.de ED +.br +.in 0 +..