The web interface should cover online services, providing an unified experience for main usages: mails, calendar, instant messaging, personal website, file sharing, etc.
At no point the user should be forced to reinstall, a misconfiguration has to be easily fixable.
We use static compilation for *(at least)* system tools: there is almost no chance an update break the system (yes, *almost*, people are creative these days).
This means having a very small and consistent set of tools, easy to learn, easy to remember.
**Basic system and network management.** *with the simplest tools ever*<br/>
We provide a web interface that should handle basic system and network configurations, such as adding users, firewall management, dhcp, DNS, backups, etc.
Packaging applications and libraries should be possible with the fewest and simplest tooling possible.
BaguetteOS provides [a simple tool to package applications](#packaging) (`packaging`) which allows simple recipes for your packages.
`packaging` handles [slotting](#slotting), compiling, stripping binaries and libraries, splitting the result into different packages (`-man`, `-lib`, `-doc`, etc.), authenticating, etc.
**OpenBSD.** *we will get there quick, but will focus on Linux a bit before*<br/>
We definitively want to use OpenBSD, currently we just focus on Linux for hardware compatibility reasons (and out of habits) but it's not set in stone.
**CRUX and Alpine.** *great source of inspiration*<br/>
We **do** use the CRUX's `rc` script, and as Alpine it is a source of inspiration for package recipes.
However, since we have to change all packages to get slotting, the service manager to have tokenized services, the packaging software to get declarative recipes (...), why even bother use those systems?
**Linux kernel**, but we are lurking on the OpenBSD one.<br/>
Linux is compatible with most hardware and software, it is fast and we can easily compile a custom version to remove most of the bloat for server usage.
Still, we don't want to rely on Linux-specific components.
At some point, our system will be kernel-agnostic and will be able to run on any BSD as well.
OpenBSD has `pledge` and `unveil` syscalls, which is an elegant way to provide a guarantee on the software behavior.
**Musl.** *reasonable libc for Linux*<br/>
It has a reasonable amount of features, it is efficient, provides reasonable binary sizes and static compilation.
Musl is simple, reliable and remove all glibc-specific functions.
Others can be added easily, which is useful for compatibility and comparisons, through [slotting](#slotting).
**Bootable system and rootfs available.**<br/>
A bootable system to install in virtual machines or bare metal, a rootfs to use BaguetteOS from any other OS, including non-Linux ones.
**SysV-style init + [CRUX-like /etc/{rc,mdev.conf,...}][baguette-rc]**. *easy to read, easy to adapt*<br/>
The init could come from toybox or another minimalist project.
**Default building tools** *every tool needed to bootstrap*<br/>
Clang (+ LLVM) is the default C (and C++) compiler.
[Libarchive][libarchive] is required for tarballs, packages, webhooks from `packaging`, and both [bsdcpio][bsdcpio] and [bsdtar][bsdtar] (sane implementations of `cpio` and `tar`).
[Auto-tools][autotools] are also required (for SysV init and libarchive).
[m4][m4] and [gnu-make][gmake] are required for compatibility reasons.
**Documentation.**<br/>
A full hand-book like the OpenBSD FAQ.
Our software man-pages are written with `scdoc` so anyone can contribute.
**[Packaging][packaging] for packaging software and libraries.** *dead simple, intuitive*<br/>
Yes, this is a language you have to learn to work with us on a couple of projects, but you can learn it in about a few days to a week and it increases our productivity like crazy.
We heard about `nim` and a ton of other languages, lots of them are great candidates, but **choices needed to be made**.
Keep reminding to yourself while reading that our tools are implemented within just a few hundred lines of code (up to 1500 lines for `service` and `libipc`).<br/>
Also, they can easily run on other systems: nothing here is OS specific.
Before presenting our tools, here a file format named `spec` that we tend to use when relevant.
It is *declarative*: we tend not to tell instructions how to do things (copy this file here, download this, etc.) but to describe something (url of the project is `https://example.com/xxx`).
Package provides slotting by default: no need for custom environments for each software.
Packages format is a simple `tar` archive containing a `meta.spec` file describing all meta-data about the package (hash, manifest, etc.) and `files.tar.xz` with the files to install.
The database format contains `world`, `installed`, `[package-name]/[slot]/manifest` and `[package-name]/[slot]/meta.spec`.
Package's configuration is a list of repositories, authorized package signing keys and packaging variables (cflags, makeflags, and so on).
Change the first `prefix` in your [packaging configuration](#packaging-host-config), compile your first package and you have your repository.
That is that simple.
<imgsrc="/shell-scripting-is-lava.png"alt="shell script is lava"/>
<aname="packaging-build-env"></a>
**`Packaging` build environments**<br/>
`Packages` creates build environments to test packages before validation.
It works as follow:
1. creation of a `/tmp/packaging/build-UUID/` directory
2. sources are downloaded, extracted then compiled<br/>
Recipes and [`packaging` host configuration](#packaging-host-config) may add parameters to it: adding steps before compilation, changing `configure` arguments, etc.
The package is compiled for a specific slot, by default `/usr/baguette`.
3. compiled applications and libraries are put in `/tmp/packaging/build-UUID/root` which is used to create the final package
`Packaging` uses `package` to create low-cost build environments since we hardlink binaries into the building rootfs, which is inspired by the [proot][proot] tool on OpenBSD.
In this case, `package` only installs the minimal set of binaries required by the package to build.
Besides the target application, a building environment size is <u>only a few kilobytes</u>.
<aname="packaging-host-config"></a>
**`Packaging` configuration**. *common configuration for your packages*<br/>
`Packaging` may be configured globally for your system with the file `/etc/packaging.conf` which contains the following:
```YAML
# Configuration file for `packaging`
# where to send built packages
packages-directory: /usr/local/pkg/
# where to download sources
sources-directory: /usr/local/src/
# prefixes for `packaging` running environment and child processes
prefixes:
# the first prefix is the default slot used for building application
- /usr/baguette/
- /
# list of environment variables we want to have when building
environment:
# you may choose another compiler, provide some CFLAGS, etc.
- CC: clang
- CFLAGS: -Os -Wall
# next three have special meaning
# to provide parameters to the `./configure` script when building
- configure: --disable-nls --without-gettext
# to provide parameters to the `make` command
- make:
# to provide parameters to the final `make install` command
That's it. You know all about `packaging` configuration.
These parameters may be override by recipes.
<aname="recipes"></a>
**`Packaging` recipes.** *we need to create packages*<br/>
A recipe is the way to reproduce something; here we want to create a package, the recipe should provide all data necessary to be able to reproduce the package.
This means at least having a name for the software and a version (they appear in the package name) and sources (software code).
Now you know how to deal with `@configure`, `@build` and `@install`: these are code blocks allowing you to fix this kind of problems.
Non standard build operations happen from time to time, and code blocks help you overcome this.
When a lot of packages have the same workarounds, we think about detection and integration into `packaging`, so we can keep only specifics into recipes.
**Future of `packaging`.** *let's be even more declarative*<br/>
As we saw, `packaging` allows maintainers to create very simple and **readable** recipes.
Sometimes we have to confront ourselves to poorly designed build systems, but we can hack a bit.
In the future, manual operations should be reduced even more:
- by adding a few other parameters to the environment<br/>
This imply to check for patterns in the recipes and to provide workarounds.
- by adding new code blocks *before* the `configuration`, `build` and `install` steps<br/>
This would allow performing a few hacks in the directory (a quick `sed` in a file for instance) and still keeping automatic operations: the best of the two worlds.
- a single possible instance, otherwise the configuration has to be heavily changed
- root-only management, simple users rarely run their own services (except on systemd, kuddos for once)
- no domain management
These shortcomings imply manual configuration, scripting to manage databases and users, specific tooling for each database and service: this is heavy machinery.
To overcome drawbacks of having simplistic tools, sys-admins developed all kind of monstrous architectures.
- **LXC** *chroot + network + software limits*
- **Qemu + KVM, Xen** *let's add software mimicking hardware's complexity to the mix, telling everyone it's for security and simplicity*
- **docker** *I don't know how to do simple applications nor packages, so I ship my whole dev environment*<br/>
Note: we have to admit, packaging on most OS is painful for absolute no good reason.
- **Chef and Puppet** *the 500 MB running Ruby's code on our virtual machines just to check for new configuration is okay, memory is cheap right?*<br/>
We talk about the importance of security from time to time, but running a software designed by people telling [it's okay not to free memory][ruby-memory-bp] is far from being wise.
- **Ansible** *templating your applications… from another machine*<br/>
As Chef and Puppet, ansible provides templating for applications, this time configuration propagation is **way** simpler since it uses well-known, trusted and loved ssh.
Still, templating is done for remote machines, as it is intended for server deployments: this is **a sys-admin tool**.
The introduction page already talks about `cloud provisionning` and `intra-service orchestration` on the first line, telling that you really need to setup SSH keys, etc.
As for many tools in IT: this is not for simple users.
Users shouldn't need to manually change the configuration.
<u>Syntax may change at any time without breaking a single service</u>, since the configuration will smoothly be regenerated with useful information at start-up.
For example, adding a Wordpress service will automatically change the `nginx` configuration, create a new database and a new user in `mariadb` for this specific service.
If several `nginx` are required, ports will be registered and automatically managed for each instance, no need for user input.<br/>
Behind the scene, it's a simple token system with configuration templating!<br/>
It [currently works][working-service-asciinema], and we just need to add more services!
Every new corner case will be thoroughly investigated to provide the smoothest experience possible under BaguetteOS.
We currently use it for a few services, such as the nginx providing this very website.
Currently, there are no fancy environments, just plain directories.
**Future of `service`**<br/>
First, we want to work on databases, to export data and get a simple backup service.
Then, we could let users manage their own services, but this is of little interest in practice since services are running under different users already.
The most useful thing to do right now is to provide new services.
Therefore, languages are irrelevant: you can use *any* library in *any* language.
LibIPC is currently used for the administration dashboard, the web interface for the services, [for a kanban][todod] and several other tools we use for collaboration.
It provides a way to communicate between clients and services using simple unix sockets behind the scene.
C library with Crystal bindings (other languages coming soon)
LibIPC explanation goes beyond the scope of this page… <red>and may even deserve a whole website of its own</red> but the tool is awesome and performances are crazy (we have to tell the world!).
**[Build.zsh][build.zsh]: makefile creation.** *for mere mortals*<br/>
Build.zsh creates a makefile from a simple declarative configuration file.
It can replace most build systems.
---
<red>TODO.</red>
**[tap-aggregator][tap-aggregator]: quality assurance & test results aggregation**<br/>
---
<red>TODO: better explanation.</red>
**[Webhooksd][webhooksd]: verify recipes.**<br/>
Webhooksd provides an automatic verification of the recipes, based on new application or library version.
Paired with a build system, new recipes received in the repository create packages for a couple of architectures (x86_64, ARM, others will follow).
## Still in discussion
For the simple users, we want to provide an unified web interface to manage the system and online services.
We currently use `Crystal` to work on service back-ends for a variety of projects, we are satisfied on a technical level and we are productive, it is highly probable we continue using it.
The front-end is still in discussion: we currently use `livescript` and it is way more usable than plain JS, but it is painful to debug.
So, we need a language for both administration dashboard and online services, here some examples we find interesting for the job:
- Purescript
- haskell-like syntax, with a smaller set of notions
- strongly typed, with type inference
- good documentation
- useful compilation errors
- no runtime error
- Elm
- as Purescript but with way fewer documentation (but reading the code is sometimes enough here)
- less generic code (functions such as `fold` and `map` have hardcoded type), which feels a bit hacky
- still very young
- WASM
- seems to be a very young tech, with no real good language or documentation
- Zig has wasm as a Tier 1 support, we should investigate
<aname="slotting"></a>
# 4. Slotting: providing software the right way
The usual way to provide software is to maintain a version of a software or a library, package it into a distribution, then provide it as *the* OS version of the software.
In the long run, software and libraries change, which is no big deal since maintainers verify the consistency of the different versions provided by the OS.
<red>TODO</red>
**Problem:** what happens when two programs need a different version of a library?<br/>
**Problem:** what happens when two libraries are compatible but you want both on your system (see libressl and openssl)?<br/>
**Problem:** what happens when you want to provide a **very** long term support for your users (see companies running decade-old databases)?
BaguetteOS has a simple and safe way to let users and maintainers provide packages: `slotting`.
Official OS packages are installed under `/usr/baguette/`, for non-essential programs.
Here, the slot is `baguette`.
Any package outside the official ones are in another named slot.
<red>TODO</red>
**This in nothing new, however not often used, and still maybe the best way to handle the problem.**
Do not hesitate to come on our [Mattermost][mattermost].
We will soon have bridges on XMPP and IRC, stay tuned!
# Just dropped things
80211d
networkctl
firewalld
ajouter des outils de gestion (suppression, modification) de préfixes dans l’environnement (PATH, LD_LIBRARY_PATH, PKGCONFIG_PATH, etc.). À faire a minima dans /etc/profile.
partitionnement et formatage des partitions
Avoir un programme qui permet de retrouver de quel paquet provient un fichier (un reverse apk manifest).