6 hours later

content/_index.md

@@ -2,12 +2,23 @@
 title = "Baguette - OS, tools and stuff"
 +++
 
-# BaguetteOS - concise overview
+- [Overview](#overview)
+- [Technical choices](#technical-choices)
+- [Custom tools](#custom-tools)
+- [Slotting](#slotting)
+- [Roadmap](#roadmap)
+
+<a name="overview"></a>
+# 1. Concise overview
 
 BaguetteOS status: Work In Progress.
 A beta will be available circa mid-2020 for advanced users.
 
 
+<red>TODO: explications partie Custom Tools, factorisation, exemples de code et de commandes, liens vers d'autres projets.</red>
+
+<red>Warning:</red> this documentation is currently oriented toward advanced unix users.
+
 ## Objectives, for simple users
 
 BaguetteOS aims at provide a simple unix-like system, with an **unified web interface**.
@@ -26,9 +37,10 @@ The web interface should cover online services, providing an unified experience
 **One-click management.** *service installs, updates, etc.*<br />
 The web interface should handle basic system and network configurations, such as adding users, dhcp, DNS, backups, etc.
 
-**Well-known, reliable software.**<br />
+**Well-known, reliable software.** *for real*<br />
 BaguetteOS relies on robust and independant software.
 At no point the user should be forced to reinstall, a borked configuration has to be easily fixed.
+Static compilation for system tools *(at least)*: there is almost no way to get a borked system with an update (yes, *almost*, people are creative these days).
 
 **Hardware support.** *new or old, fast or slow, it doesn't matter*<br />
 We provide support for RPi and other small cards: if our kernel runs on it, it has to work.
@@ -37,6 +49,11 @@ Minimal hardware requirement should be around 100 MB RAM, 50 MB disk.
 **Documentation.** *simple, reliable, useful, all-in-one-place*<br />
 Similar to the OpenBSD FAQ: updated, complete, concise and well-written.
 
+**Constrained devices use case.** *wanna try what small systems can do?*<br />
+By default, we try to provide the smallest memory footprint: we do not ship manuals, nor runtime libraries when not required.
+Our programs will never require you to download development libraries, nor alternative package managers.
+The kernel is compiled with size in mind, as our choice of tools.
+As a result, our system can be installed quickly even on slow machines.
 
 ## Objectives, for advanced users
 
@@ -48,9 +65,6 @@ This means having a very small and consistent set of tools, easy to learn, easy
 We provide a web interface that should handle basic system and network configurations, such as adding users, firewall management, dhcp, DNS, backups, etc.
 CLI tools are available to manage your services, they are design to be simple, consistent and reliable.
 
-**Robust system.** *for real*<br />
-Static compilation for system tools *(at least)*: there is almost no way to get a borked system with an update (yes, *almost*, people are creative these days).
-
 **Officially supported and documented services.** *so you are sure to get them working*<br />
 We use some services for our own personal usage, so we will provide support for them.
 For instance: gitea, postgresql, a building plateform and a continuous integration tool, etc.
@@ -71,12 +85,6 @@ Manual configuration is the last option.
 [Slotting](#slotting) by default helps to install many programs, even with peculiar library version requirements.
 The same program can be installed in several versions without conflicts.
 
-**Constrained devices use case.** *wanna try what small systems can do?*<br />
-By default, we try to provide the smallest memory footprint: we do not ship manuals, nor runtime libraries when not required.
-Our programs will never require you to download development libraries, nor alternative package managers.
-The kernel is compiled with size in mind, as our choice of tools.
-As a result, our system can be installed quickly even on slow machines.
-
 
 ## Objectives, for contributors
 
@@ -159,14 +167,15 @@ However, since we have to change all packages to get slotting, the service manag
 This approach of package management is interesting, having to learn a language to make recipes isn't.
 And that sums a lot about this technology.
 It's great to have, not to put everywhere.
-Every part of BaguetteOS is simple to understand, GUIX is not *enough*.
+Every part of BaguetteOS is simple to understand, GUIX is not simple *enough*.
 But keep it up guys, it's still awesome.
 
 **Plan9, Inferno, morpheus, etc.** *kinda abandoned systems* <br />
 That's it.
 
 
-# <a name="technical-choices"></a> Technical choices
+<a name="technical-choices"></a>
+# 2. Technical choices
 
 ## Base system
 
@@ -254,65 +263,151 @@ There is a strict separation between core system and third party software.<br />
 - `/usr/bad` for unslottable software
 - `/usr/third-party` for other software
 
-# <a name="custom-tools"></a> BaguetteOS - custom tools
+<a name="custom-tools"></a>
+# 3. BaguetteOS: custom tools
+
+All our tools are designed to be simple to use, to understand, to read.
+<u>Feel free to provide a feedback.</u>
+
+<red>TODO: spec files, the file format used in `package`, `packaging` and `service`.</red>
+
+---
+
+<red>TODO: explains why it's different / better than other package managers.</red>
 
 **[Package][package]: our package manager.**<br />
 Package covers the basics: install, remove, search and provide informations about a package, and it creates rootfs.
-It is used by `packaging` to create low-cost build environments.
 Package knows the minimal set of binaries and configuration required to build the target, so it only installs the minimal environment to perform compilation.
-This environment is low-cost since we hardlink binaries into the building rootfs, which is inspired by the [proot][proot] tool on OpenBSD.
 
 Package provides slotting by default: no need for custom environments for each software.
 
-Packages format ar simple `tar` archives containing a `meta.spec` file describing all meta-data about the package, and `files.tar.xz` with the files to install.
+Packages format is a simple `tar` archive containing a `meta.spec` file describing all meta-data about the package (hash, manifest, etc.) and `files.tar.xz` with the files to install.
 The database format contains `world`, `installed`, `[package-name]/[slot]/manifest` and `[package-name]/[slot]/meta.spec`.
 
 Package's configuration is a list of repositories, authorized package signing keys and packaging variables (cflags, makeflags, and so on).
 
-**[Packaging][packaging]: to create packages**<br />
+---
+
+<a name="packaging"></a>
+**[Packaging][packaging]: to create packages.**<br />
 Packaging uses simple, declarative recipe files, here some examples: [hello][recipe-hello], [dhcpcd][recipe-dhcpcd], [alsa-utils][recipe-alsautils].
 The only required parameters are `name` and `sources`.
 
+<a name="packaging-build-env"></a>
 Packaging creates build environments to test packages before validation.
+It works as follow:
+1. creation of a `/tmp/packaging/build-UUID/` directory
+2. sources are downloaded, extracted then compiled<br />
+   Recipes and [`packaging` host configuration](#packaging-host-config) may add parameters to it: adding steps before compilation, changing `configure` arguments, etc.
+   The package is compiled for a specific slot, by default `/usr/baguette`.
+3. compiled applications and libraries are put in `/tmp/packaging/build-UUID/root` which is used to create the final package
 
-Packaging currently creates 
 
-- [service][service]: service management
+`Packaging` uses `package` to create low-cost build environments since we hardlink binaries into the building rootfs, which is inspired by the [proot][proot] tool on OpenBSD.
-  - add an init script for a service, for a specified domain
+
-    - example: `service add wordpress example.com`
+<red>TODO.</red>
-  - the init script verifies if a configuration file is installed<br />
+<a name="packaging-host-config"></a>
-    The configuration file is created if not present.<br />
+Packaging configuration is simple.
-    Configuration templates are provided for all services.
+
-  - the service can be installed in a specific environment (read: a custom rootfs) ← NOT IMPLEMENTED (also, environments == domains atm)
+<red>TODO.</red>
-    - example: `service add wordpress example.com testingenv`
+<a name="packaging-host-config"></a>
-  - `service` provides an unified way to configure the system<br />
+Packaging recipes.
-    It alleviates the need for manual configuration. For example, adding a wordpress service will automatically change the `nginx` configuration, create a new database and a new user in `mariadb` for this specific service.<br />
+
-    If several `nginx` are required, ports will be registered and automatically managed for each instance, no need for user input.<br />
+---
-    Behind the scene, it's a simple token system with configuration templating!<br />
+
-    No heavy machinery here, and we'll keep it that way.
+<red>Factorisation.</red>
-- `build.cr` (temporary name): Makefile creation
+
-  - create makefiles from simple declarative configuration file
+<a name="service"></a>
-  - can replace most build systems
+**[Service][service]: service management.** *not just `kill` or start/stop/status wrapper*<br />
-  - FIXME: design something using .spec format
+Service management often comes with:
-- `tap-aggregator`: quality assurance & test results aggregation
+- default configuration files, users should learn how to configure them and do it manually
-- `webhooksd`
+- a single possible instance, otherwise the configuration has to be heavily changed
-  - automatic verification of the recipes on new application or library version
+- root-only management, simple users rarely run their own services (except on systemd, kuddos for once)
-  - automatic cross-compilation (x86_64, ARM, others will follow)
+- no domain management
-- `libipc`: an IPC communication library
+
-  - currently used for
+These shortcomings imply manual configuration, scripting to manage databases and users, specific tooling for each database and service: this is heavy machinery.
-    1. the administration dashboard
+Simple users:
-    2. the web interface for the services
+1. should be only required to provide absolute necessary information for their services
-    3. `todod` (a kanban) 4. several other tools we use for collaboration
+2. should be able to run as many services they want
-  - provides a way to communicate between clients and services
+3. shouldn't have to learn configuration syntax for their services
-  - uses simple unix sockets behind the scene
+4. shouldn't be afraid of updates
-  - transparent remote communications
+
+Here are a few functionnalities `service` brings.
+1. **domain-wise service configuration**<br />
+   Example: `service add wordpress domain=example.com`
+
+2. **templates** *configuration files are generated by templates and user data*<br />
+   When we want a software, for instance a blog, we want to provide the minimum informations it requires to work and that's it.
+   When `service` is invoked to start a service, it verifies if a configuration file is installed or create it.
+   Users shouldn't need to manually change the configuration.
+   <u>Syntax may change at any time without breaking a single service</u>, since the configuration will smoothly be regerenated with useful information at start-up.
+
+3. **environments**<br />
+   Each service can be installed in a specific environment (read: a custom rootfs).
+   Example: `service add wordpress example.com testing-env`
+
+4. **unified way to configure the system**<br />
+   It alleviates the need for manual configuration.
+   For example, adding a wordpress service will automatically change the `nginx` configuration, create a new database and a new user in `mariadb` for this specific service.
+   If several `nginx` are required, ports will be registered and automatically managed for each instance, no need for user input.<br />
+   Behind the scene, it's a simple token system with configuration templating!<br />
+   No heavy machinery here, and we'll keep it that way.
+
+
+---
+
+<red>Better introduction.</red>
+
+<a name="libipc"></a>
+**[LibIPC][libipc]: an IPC communication library** *nothing new, yet it still feels fresh*<br />
+
+
+1. Applications should talk to each other
+2. We need services, not libraries<br />
+   Therefore, languages are irrelevant: you can use *any* library in *any* language.
+
+LibIPC is currently used for the administration dashboard, the web interface for the services, [for a kanban][todod] and several other tools we use for collaboration.
+It provides a way to communicate between clients and services using simple unix sockets behind the scene.
+
+<red>Explain remote communications.</red>
+Remote remote communications are transparent.
     - clients and services do not need remote communication
     - any client can join remote services via any communication protocol
     - any service is implicitly accessible from anywhere, anyhow
-  - C library with Crystal bindings (other languages coming soon)
-  - we create services, not libraries<br />
-    Therefore, languages are irrelevant: you can use any *library* in any language.
 
+C library with Crystal bindings (other languages coming soon)
+
+```crystal
+require "ipc.cr"
+
+server = IPC::Service.new "MyService"
+server.loop do |message|
+# ...
+end
+```
+
+<red>TODO: show that's easy to write even in plain C.</red>
+
+<red>TODO: performances are crazy, we have to tell the world.</red>
+
+---
+
+<red>TODO.</red>
+**[Build.zsh][build.zsh]: makefile creation.** *for mere mortals*<br />
+Build.zsh creates a makefile from a simple declarative configuration file.
+It can replace most build systems.
+
+---
+
+<red>TODO.</red>
+**[tap-aggregator][tap-aggregator]: quality assurance & test results aggregation**<br />
+
+---
+
+<red>TODO: better explanation.</red>
+**[webhooksd][webhooksd]: verify recipes.**<br />
+Webhooksd provides an automatic verification of the recipes, based on new application or library version.
+Paired with a build system, new recipes received in the repository create packages for a couple of architectures (x86_64, ARM, others will follow).
 
 ## Still in discussion
 
@@ -328,18 +423,20 @@ So, we need a language for both administration dashboard and online services, he
 	  - useful compilation errors
 	  - no runtime error
   - Elm
-    - as Purescript but with way fewer documentation
+    - as Purescript but with way fewer documentation (but reading the code is sometimes enough here)
-    - less generic code
+    - less generic code (functions such as `fold` and `map` have hardcoded type), which feels a bit hacky
     - still very young
   - WASM
     - seems to be a very young tech, with no real good language or documentation
     - Zig has wasm as a Tier 1 support, we should investigate
 
-# <a name="slotting"></a> Slotting: providing software the right way
+<a name="slotting"></a>
+# 4. Slotting: providing software the right way
 
 The usual way to provide software is to maintain a version of a software or a library, package it into a distribution, then provide it as *the* OS version of the software.
 In the long run, software and libraries change, which is no big deal since maintainers verify the consistency of the different versions provided by the OS.
-TODO
+
+<red>TODO</red>
 
 **Problem:** what happens when two programs need a different version of a library?<br />
 **Problem:** what happens when two libraries are compatible but you want both on your system (see libressl and openssl)?<br />
@@ -350,39 +447,29 @@ Official OS packages are installed under `/usr/baguette/`, for non-essential pro
 Here, the slot is `baguette`.
 Any package outside the official ones are in another named slot.
 
-TODO
+<red>TODO</red>
 
 **This in nothing new, however not often used, and still maybe the best way to handle the problem.** 
 
 
-- usual directories under root: bin, sbin, lib, boot, dev, proc, sys, home, mnt, root, run, tmp
+### BaguetteOS file system hierarchy
-- etc
-  - rc
-    - services
-    - environments
-  - templates
-- var
-  - cache
-- srv
-  - "env-name" (see [service][service])
-    - etc
-    - data
-    - cache
-    - run
-- usr
-  - local: things that are installed by the local system administrator without using packages
-  - baguette: things provided by the system that are not necessary for it to run (and boot, and restart, and do system things)
-  - "repo"
-    - lib
-    - libexec: try to avoid using it whenever possible. May or may not stay.
-    - bin
-    - share
-    - man
-    - include
-  - bad: things that cannot be properly installed or slotted somewhere else
 
+- usual directories under root: bin, sbin, lib, boot, dev, proc, sys, home, mnt, root, run, tmp
+- `/etc/rc` with `services` and `environments` for running service instances
+- `/etc/templates` for local service configuration templates
+- `/var/cache`
+- under `/srv/"env-name"` (see [service](#service)), these subdirs when required: `/etc`, `/data`, `/cache`, `/run`
+- under `/usr`
+  - `/local`: things that are installed by the local system administrator without using packages
+  - `/baguette`: things provided by the system that are not necessary for it to run (and boot, and restart, and do system things)
+  - `/"repo"`: `/lib`, `/bin`, `/share`, `/man`, `/include` (`/libexec` but we try to avoid it whenever possible.)
+  - `/bad`: things that cannot be properly installed or slotted somewhere else
+
+<a name="roadmap"></a>
 # Roadmap
 
+<red>TODO</red>
+
 We currently aim at providing a rootfs with our tools, when we will have enough spare time to contribute.
 
 
@@ -391,6 +478,9 @@ We currently aim at providing a rootfs with our tools, when we will have enough
 [package]: https://git.baguette.netlib.re/Baguette/package
 [packaging]: https://git.baguette.netlib.re/Baguette/packaging
 [build.zsh]: https://git.baguette.netlib.re/Baguette/build.zsh
+[libipc]: https://git.baguette.netlib.re/Baguette/libipc
+[webhooksd]: https://git.baguette.netlib.re/Baguette/
+[tap-aggregator]: https://git.baguette.netlib.re/Baguette/tap-aggregator
 
 [baguette-rc]: https://git.baguette.netlib.re/Baguette/recipes/src/branch/master/rc
 [baguette-profile]: https://git.baguette.netlib.re/Baguette/recipes/src/branch/master/rc/profile

sass/custom.scss

@@ -13,6 +13,10 @@
 	// margin-bottom: -1cm;
 }
 
+red {
+	color: red;
+	font-weight: bold;
+}
 
 // less spacing in lists
 li > p {