Baguette/website
3
1
Fork 0
Code Issues Pull Requests Releases Wiki Activity

current status: broken website

Browse Source
This commit is contained in:
Philippe PITTOLI 2020-04-23 17:51:38 +02:00
parent 96bbb76b99
commit 597acacf8d
10 changed files with 584 additions and 528 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
content/_common/header.fr.md Normal file
View File

@ -0,0 +1,14 @@
+++
+++
<h1 hidden>dnsmanager</h1>
<img src="/baguette.png" alt="Baguette" class="banner" />
<pre aria-hidden="true"><code>
____ _ _
| __ ) __ _ __ _ _ _ ___| |_| |_ ___
| _ \ / _` |/ _` | | | |/ _ \ __| __/ _ \
| |_) | (_| | (_| | |_| | __/ |_| || __/
|____/ \__,_|\__, |\__,_|\___|\__|\__\___|
|___/
</code></pre>

5
content/_common/header.md
View File

@ -2,6 +2,11 @@
+++
<h1 hidden>dnsmanager</h1>
<link rel="stylesheet" href="/styles/default.css">
<script src="/highlight.pack.js"></script>
<script>hljs.initHighlightingOnLoad();</script>
<img src="/baguette.png" alt="Baguette" class="banner" />
<pre aria-hidden="true"><code>

4
content/_common/menu.fr.md
View File

@ -1,7 +1,7 @@
+++
+++
[BaguetteOS](@/_index.fr.md)
[BaguetteOS](/fr/baguette/)
---
@ -9,6 +9,6 @@
---
[Blog](@/blog/_index.fr.md)
[Blog](/blog/)

4
content/_common/menu.md
View File

@ -1,7 +1,7 @@
+++
+++
[BaguetteOS](@/_index.md)
[BaguetteOS](/baguette/)
---
@ -9,5 +9,5 @@
---
[Blog](@/blog/_index.md)
[Blog](/blog/)

7
content/_index.fr.md
View File

@ -1,8 +1,5 @@
+++
title = "Baguette - un OS, des outils, des services"
paginate_by = 5
+++
# Baguette OS
La page n'a pas encore été traduite.

513
content/_index.md
View File

@ -1,514 +1,5 @@
+++
title = "Baguette - OS, tools and stuff"
paginate_by = 5
+++
- [Overview](#overview)
- [Technical choices](#technical-choices)
- [Custom tools](#custom-tools)
- [Slotting](#slotting)
- [Roadmap](#roadmap)
<a name="overview"></a>
# 1. Concise overview
BaguetteOS status: Work In Progress.
A beta will be available circa mid-2020 for advanced users.
<red>TODO: explications partie Custom Tools, factorisation, exemples de code et de commandes, liens vers d'autres projets.</red>
<red>Warning:</red> this documentation is currently oriented toward advanced unix users.
## Objectives, for simple users
BaguetteOS aims at provide a simple unix-like system, with an **unified web interface**.
**No command-line required for simple users.** *let users be just users*<br />
Simplicity should not only comes from an interface, but be inherent to the whole system.
If the OS is simple, there is no need to hack it.
**Unified interface is better than features.**<br />
We will provide the basic features then build up.
We do not want a patchwork of very different software, each of them having their own particularities.
**Online services.** *day-to-day use*<br />
The web interface should cover online services, providing an unified experience for main usages: mails, calendar, instant messaging, personal website, file sharing, etc.
**One-click management.** *service installs, updates, etc.*<br />
The web interface should handle basic system and network configurations, such as adding users, dhcp, DNS, backups, etc.
**Well-known, reliable software.** *for real*<br />
BaguetteOS relies on robust and independant software.
At no point the user should be forced to reinstall, a borked configuration has to be easily fixed.
Static compilation for system tools *(at least)*: there is almost no way to get a borked system with an update (yes, *almost*, people are creative these days).
**Hardware support.** *new or old, fast or slow, it doesn't matter*<br />
We provide support for RPi and other small cards: if our kernel runs on it, it has to work.
Minimal hardware requirement should be around 100 MB RAM, 50 MB disk.
**Documentation.** *simple, reliable, useful, all-in-one-place*<br />
Similar to the OpenBSD FAQ: updated, complete, concise and well-written.
**Constrained devices use case.** *wanna try what small systems can do?*<br />
By default, we try to provide the smallest memory footprint: we do not ship manuals, nor runtime libraries when not required.
Our programs will never require you to download development libraries, nor alternative package managers.
The kernel is compiled with size in mind, as our choice of tools.
As a result, our system can be installed quickly even on slow machines.
## Objectives, for advanced users
**A knowable OS.** *simplicity at (almost) all cost*<br />
Any interested user should be able understand the role of every part of the base system: no compromise.
This means having a very small and consistent set of tools, easy to learn, easy to remember.
**Basic system and network management.** *with the simpliest tools ever*<br />
We provide a web interface that should handle basic system and network configurations, such as adding users, firewall management, dhcp, DNS, backups, etc.
CLI tools are available to manage your services, they are design to be simple, consistent and reliable.
**Officially supported and documented services.** *so you are sure to get them working*<br />
We use some services for our own personal usage, so we will provide support for them.
For instance: gitea, postgresql, a building plateform and a continuous integration tool, etc.
**One need, one tool.** *this time for real*<br />
Installing an application or a library is done by [package][package].
Other methods are not supported **and the base system will never require them**.
We avoid to rely on `pip`, `cpanm`, or other third party package manager and dependency tree.
More on that in the [technical section](#technical-choices).
Starting, stopping, or configuring a service is done by [service][service].
This program alone is used to manage services on the OS.
Users should not be required to manually configure each software; instead, most of the configuration should be done upstream using templates.
Users should be able to change the default configuration through command-line options.
Manual configuration is the last option.
**Slotting.** *any software can be installed on any machine at any time, no extra tooling*<br />
[Slotting](#slotting) by default helps to install many programs, even with peculiar library version requirements.
The same program can be installed in several versions without conflicts.
## Objectives, for contributors
**Simple to contribute to.** *you can focus on providing recipes for packages, we handle the tooling*<br />
We want fewer and simpler tools as possible, even for packaging applications and libraries.
BaguetteOS ships [a simple tool to package applications][packaging] and it only requires you to create a very simple recipe for your package.
It handles [slotting](#slotting), compiling, stripping binaries and libraries, splitting the result into different packages (`-man`, `-lib`, `-doc`, etc.) then authenticate them and finally recreate the index.
All that, just by typing `packaging app`, nothing more.
<img src="/meanie-makefile.png" alt="meanie meanie dev" class="meanie-img" />
**No Makefile?** *no problem*<br />
Your application or your library lacks a build system? Here a [tool to create makefiles][build.zsh].
It works on any language, yes, even that one.
**Stable and development versions: same thing.** *slotting, again and again*<br />
One of the coolest aspect of slotting is: you don't need to change your system *at all* to be on the developement version of the OS.
The developement of the newest version of the OS is just slotted in `/usr/baguette-dev` and that's it.
At any point in time you can execute programs from your stable OS, or if you will, change a few environment variables and be in dev.
**New OS, open to explore**<br />
BaguetteOS do not suffer from cumbersome historical decisions: no overly engineered package format, no stupidly complex patchwork of mismatch tools.
BaguetteOS is easy enough to understand then even try to surpass it.
**Easy to write documentation.** *easy to write, hopefully less procrastination*<br />
Online documentation is written in Markdown (thanks Zola), and man pages too thanks to `scdoc`.
Every tool is shipped with a man page: no man page, no integration in base.
## Inspiration
- [CRUX][crux], [alpine][alpine]: simple to understand Linux systems
- [OpenBSD][openbsd]: security, therefore simplicity, no compromise
- [PFSense][pfsense]: system and (even advanced) networking administration, yet through a simple website
- [Plan9][plan9] and [Inferno][inferno]: everything is a file *no seriously guys*
- [suckless][suckless] and [cat-v][cat-v]: simplicity, code readability and reusability
- [morpheus][morpheus]: static compilation for the OS, demystified
## Why not use another system?
This section could be expanded.
**A few reasons why any of the candidate covers it all.**<br />
1. **we want slotting**<br />
So we could change the way we install and maintain packages.
2. **we want fast install and start-up on dumb devices**<br />
Coreutils will be shrinked to bare minimum, thanks to `toybox`.
We won't require a full-feature system with several hundred megabytes of used disk-space.
3. **documentation is important, but not for all installations**<br />
Your 42nd test install on a virtual machine doesn't need to provide manual for CLI tools you won't use since you do everything through the web interface.
Software and its documentation will be splited: manual pages won't be provided by default.
4. **we want automatic tests, and to allow people to test our system**<br />
We want easy chroot installs, on any system.
5. **we want to run on tiny ARM boxes, old laptops, top-notch servers**<br />
So we need to run on any available kernel.
6. **we want to control software distribution releases**<br />
We don't accept to follow updates from an upstream OS that could break our system at any time.
---
**Now, let's take a look at each candidate.**
**OpenBSD.** *we will get there quick, but focus on linux a bit before* <br />
We definitevely want to use OpenBSD, currently we just focused on Linux for hardware compatibility reasons (and out of habits) but it's not set in stone.
We love OpenBSD big time, some of us use it daily.
We aim at providing rootfs for running our system under an OpenBSD kernel and environment.<br />
*(also, snapshots could be great, guys)*
**PFSense.** *network focused* <br />
Way too focused on networking.
Great system, does the job very well, but won't fit our objectives.
**CRUX and Alpine.** *great source of inspiration* <br />
We **do** use the CRUX's `rc` script, and as Alpine it is a source of inspiration for package recipes.
However, since we have to change all packages to get slotting, the service manager to have tokenized services, the packaging software to get declarative recipes (...), why even bother use those systems?
**GUIX and coe.** *not simple enough*<br />
This approach of package management is interesting, having to learn a language to make recipes isn't.
And that sums a lot about this technology.
It's great to have, not to put everywhere.
Every part of BaguetteOS is simple to understand, GUIX is not simple *enough*.
But keep it up guys, it's still awesome.
**Plan9, Inferno, morpheus, etc.** *kinda abandoned systems* <br />
That's it.
<a name="technical-choices"></a>
# 2. Technical choices
## Base system
**Linux kernel**, but we are lurking on the OpenBSD one.<br />
Linux is compatible with most hardware and software, it is fast and we can easily compile a custom version to remove most of the bloat for server usage.
Still, we don't want to rely on Linux-specific components.
At some point, our system will be kernel-agnostic and will be able to run on any BSD as well.
OpenBSD has `pledge` and `unveil` syscalls, which is an elegant way to provide a guarantee on the software behavior.
**Musl.** *reasonable libc for Linux*<br />
It has a reasonable amount of features, it is efficient, provides reasonable binary sizes and static compilation.
Musl is simple, reliable and remove all glibc-specific functions.
Others can be added easily, which is useful for compatibility and comparisons, through [slotting](#slotting).
**Bootable system and rootfs available.**<br />
A bootable system to install in virtual machines or bare metal, a rootfs to use BaguetteOS from any other OS, including non-Linux ones.
**SysV-style init + [CRUX-like /etc/{rc,mdev.conf,...}][baguette-rc]**. *easy to read, easy to adapt*<br />
The init could come from toybox or another minimalist project.
The [rc script from CRUX][cruxinit] is simple to understand and to adapt to any requirement, so we used it.
We also added some other scripts, like [for profile][baguette-profile] so we can easily manage slotting.
No systemd BS.
**Toybox.** *the megabyte coreutils*<br />
[Toybox][toybox] combines common unix command line utilities together into a single BSD-licensed executable.
It is designed to be simple even to read, and is standards-compliant.
For the base system, that's all we need.
**ksh and zsh**. *the first for scripts and root, the other for users*<br />
[Ksh][ksh] is a very stable and reliable shell from AT&T, trusted by the paranoid people of OpenBSD.
That's a safe choice for a base system and the root account.
On the other hand, we do use [zsh][zsh] daily, as for many users, so we may use it for development or build scripts but not in base.
**[Service][service] for service management** *tokenized service description, templating and dumb cli tools for the win* <br />
[See custom tools.](#custom-tools)
**[Package][package] for package management** *simple, efficient, dead simple code* <br />
[See custom tools.](#custom-tools)
**OpenSSH.** *as we all know and love*<br />
This is required for almost all usages, and for debug.
Let's try not to shoot ourselves in the foot.
That's all you need for starters. Web administrative interface will be added to the base when ready.
## Development, build tools, contribution
**Default building tools** *every tool needed to bootstrap*<br />
Clang (+LLVM) is the default C (and C++) compiler.
[Libarchive][libarchive] is required for tarballs, packages, webhooks from `packaging`, and both [bsdcpio][bsdcpio] and [bsdtar][bsdtar] (sane implementations of `cpio` and `tar`).
autotools are also required (for sysv init and libarchive).
[m4][m4] and [gnu-make][gmake] are required for compatibility reasons.
**Documentation.**<br />
A full hand-book like the OpenBSD FAQ.
Our software manpages are written with `scdoc` so anyone can contribute.
**[Packaging][packaging] for packaging software and libraries.** *dead simple, intuitive*<br />
[See custom tools.](#custom-tools)
**Crystal language for system tools.** *syntax and productivity of Ruby, the speed of C*<br />
It is as simple to learn as a dynamic (oriented object) language, while at the same time being almost as fast as C.
Technically, Crystal is strongly typed which catches errors at compile-time, but with type inference so it is not cumbersome to use.
Applications are compiled in a simple binary, easy to deploy.
There is a good documentation, we used it for long enough to tell.
Technical choices are reasonable and documented.
Finally, Crystal has a large library with all we need for our system components.
There is not much of a drawback here.
Yes, this is a language you have to learn to work with us on a couple of projects (the web interface backend, the service manager, the package manager and packaging) but it increases our productivity like crazy.
We heard about `nim` and a ton of other languages, but this is the one which reach the sweet spot between these parameters:
- productivity (the package manager was mostly done in a few days, [and is just a few hundred lines long][package])
- easy learning (a developper with basic notions of oriented-object can read our code, no black magic here)
- good documentation
- reasonably deployable (no host dependencies)
- execution speed
**Slotting.** *custom file system hierarchy*<br />
Our FS is not FHS-compliant, partially because of the origin-based slotting.
There is a strict separation between core system and third party software.<br />
[More information on slotting.](#slotting)
- `/usr/baguette` for core system programs
- `/usr/bad` for unslottable software
- `/usr/third-party` for other software
<a name="custom-tools"></a>
# 3. BaguetteOS: custom tools
All our tools are designed to be simple to use, to understand, to read.
<u>Feel free to provide a feedback.</u>
<red>TODO: spec files, the file format used in `package`, `packaging` and `service`.</red>
---
<red>TODO: explains why it's different / better than other package managers.</red>
**[Package][package]: our package manager.**<br />
Package covers the basics: install, remove, search and provide informations about a package, and it creates rootfs.
Package knows the minimal set of binaries and configuration required to build the target, so it only installs the minimal environment to perform compilation.
Package provides slotting by default: no need for custom environments for each software.
Packages format is a simple `tar` archive containing a `meta.spec` file describing all meta-data about the package (hash, manifest, etc.) and `files.tar.xz` with the files to install.
The database format contains `world`, `installed`, `[package-name]/[slot]/manifest` and `[package-name]/[slot]/meta.spec`.
Package's configuration is a list of repositories, authorized package signing keys and packaging variables (cflags, makeflags, and so on).
---
<a name="packaging"></a>
**[Packaging][packaging]: to create packages.**<br />
Packaging uses simple, declarative recipe files, here some examples: [hello][recipe-hello], [dhcpcd][recipe-dhcpcd], [alsa-utils][recipe-alsautils].
The only required parameters are `name` and `sources`.
<a name="packaging-build-env"></a>
Packaging creates build environments to test packages before validation.
It works as follow:
1. creation of a `/tmp/packaging/build-UUID/` directory
2. sources are downloaded, extracted then compiled<br />
Recipes and [`packaging` host configuration](#packaging-host-config) may add parameters to it: adding steps before compilation, changing `configure` arguments, etc.
The package is compiled for a specific slot, by default `/usr/baguette`.
3. compiled applications and libraries are put in `/tmp/packaging/build-UUID/root` which is used to create the final package
`Packaging` uses `package` to create low-cost build environments since we hardlink binaries into the building rootfs, which is inspired by the [proot][proot] tool on OpenBSD.
<red>TODO.</red>
<a name="packaging-host-config"></a>
Packaging configuration is simple.
<red>TODO.</red>
<a name="packaging-host-config"></a>
Packaging recipes.
---
<red>Factorisation.</red>
<a name="service"></a>
**[Service][service]: service management.** *not just `kill` or start/stop/status wrapper*<br />
Service management often comes with:
- default configuration files, users should learn how to configure them and do it manually
- a single possible instance, otherwise the configuration has to be heavily changed
- root-only management, simple users rarely run their own services (except on systemd, kuddos for once)
- no domain management
These shortcomings imply manual configuration, scripting to manage databases and users, specific tooling for each database and service: this is heavy machinery.
Simple users:
1. should be only required to provide absolute necessary information for their services
2. should be able to run as many services they want
3. shouldn't have to learn configuration syntax for their services
4. shouldn't be afraid of updates
Here are a few functionnalities `service` brings.
1. **domain-wise service configuration**<br />
Example: `service add wordpress domain=example.com`
2. **templates** *configuration files are generated by templates and user data*<br />
When we want a software, for instance a blog, we want to provide the minimum informations it requires to work and that's it.
When `service` is invoked to start a service, it verifies if a configuration file is installed or create it.
Users shouldn't need to manually change the configuration.
<u>Syntax may change at any time without breaking a single service</u>, since the configuration will smoothly be regerenated with useful information at start-up.
3. **environments**<br />
Each service can be installed in a specific environment (read: a custom rootfs).
Example: `service add wordpress example.com testing-env`
4. **unified way to configure the system**<br />
It alleviates the need for manual configuration.
For example, adding a wordpress service will automatically change the `nginx` configuration, create a new database and a new user in `mariadb` for this specific service.
If several `nginx` are required, ports will be registered and automatically managed for each instance, no need for user input.<br />
Behind the scene, it's a simple token system with configuration templating!<br />
No heavy machinery here, and we'll keep it that way.
---
<red>Better introduction.</red>
<a name="libipc"></a>
**[LibIPC][libipc]: an IPC communication library** *nothing new, yet it still feels fresh*<br />
1. Applications should talk to each other
2. We need services, not libraries<br />
Therefore, languages are irrelevant: you can use *any* library in *any* language.
LibIPC is currently used for the administration dashboard, the web interface for the services, [for a kanban][todod] and several other tools we use for collaboration.
It provides a way to communicate between clients and services using simple unix sockets behind the scene.
<red>Explain remote communications.</red>
Remote remote communications are transparent.
- clients and services do not need remote communication
- any client can join remote services via any communication protocol
- any service is implicitly accessible from anywhere, anyhow
C library with Crystal bindings (other languages coming soon)
```crystal
require "ipc.cr"
server = IPC::Service.new "MyService"
server.loop do |message|
# ...
end
```
<red>TODO: show that's easy to write even in plain C.</red>
<red>TODO: performances are crazy, we have to tell the world.</red>
---
<red>TODO.</red>
**[Build.zsh][build.zsh]: makefile creation.** *for mere mortals*<br />
Build.zsh creates a makefile from a simple declarative configuration file.
It can replace most build systems.
---
<red>TODO.</red>
**[tap-aggregator][tap-aggregator]: quality assurance & test results aggregation**<br />
---
<red>TODO: better explanation.</red>
**[webhooksd][webhooksd]: verify recipes.**<br />
Webhooksd provides an automatic verification of the recipes, based on new application or library version.
Paired with a build system, new recipes received in the repository create packages for a couple of architectures (x86_64, ARM, others will follow).
## Still in discussion
For the simple users, we want to provide an unified web interface to manage the system and online services.
We currently use `Crystal` to work on service back-ends for a variety of projects, we are satisfied on a technical level and we are productive, it is highly probable we continue using it.
The front-end is still in discussion: we currently use `livescript` and it is way more usable than plain JS, but it is painful to debug.
So, we need a language for both administration dashboard and online services, here some examples we find interesting for the job:
- Purescript
- haskell-like syntax, with a smaller set of notions
- strongly typed, with type inference
- good documentation
- useful compilation errors
- no runtime error
- Elm
- as Purescript but with way fewer documentation (but reading the code is sometimes enough here)
- less generic code (functions such as `fold` and `map` have hardcoded type), which feels a bit hacky
- still very young
- WASM
- seems to be a very young tech, with no real good language or documentation
- Zig has wasm as a Tier 1 support, we should investigate
<a name="slotting"></a>
# 4. Slotting: providing software the right way
The usual way to provide software is to maintain a version of a software or a library, package it into a distribution, then provide it as *the* OS version of the software.
In the long run, software and libraries change, which is no big deal since maintainers verify the consistency of the different versions provided by the OS.
<red>TODO</red>
**Problem:** what happens when two programs need a different version of a library?<br />
**Problem:** what happens when two libraries are compatible but you want both on your system (see libressl and openssl)?<br />
**Problem:** what happens when you want to provide a **very** long term support for your users (see companies running decade-old databases)?
BaguetteOS has a simple and safe way to let users and maintainers provide packages: `slotting`.
Official OS packages are installed under `/usr/baguette/`, for non-essential programs.
Here, the slot is `baguette`.
Any package outside the official ones are in another named slot.
<red>TODO</red>
**This in nothing new, however not often used, and still maybe the best way to handle the problem.**
### BaguetteOS file system hierarchy
- usual directories under root: bin, sbin, lib, boot, dev, proc, sys, home, mnt, root, run, tmp
- `/etc/rc` with `services` and `environments` for running service instances
- `/etc/templates` for local service configuration templates
- `/var/cache`
- under `/srv/"env-name"` (see [service](#service)), these subdirs when required: `/etc`, `/data`, `/cache`, `/run`
- under `/usr`
- `/local`: things that are installed by the local system administrator without using packages
- `/baguette`: things provided by the system that are not necessary for it to run (and boot, and restart, and do system things)
- `/"repo"`: `/lib`, `/bin`, `/share`, `/man`, `/include` (`/libexec` but we try to avoid it whenever possible.)
- `/bad`: things that cannot be properly installed or slotted somewhere else
<a name="roadmap"></a>
# Roadmap
<red>TODO</red>
We currently aim at providing a rootfs with our tools, when we will have enough spare time to contribute.
[service]: https://git.baguette.netlib.re/Baguette/service
[package]: https://git.baguette.netlib.re/Baguette/package
[packaging]: https://git.baguette.netlib.re/Baguette/packaging
[build.zsh]: https://git.baguette.netlib.re/Baguette/build.zsh
[libipc]: https://git.baguette.netlib.re/Baguette/libipc
[webhooksd]: https://git.baguette.netlib.re/Baguette/
[tap-aggregator]: https://git.baguette.netlib.re/Baguette/tap-aggregator
[baguette-rc]: https://git.baguette.netlib.re/Baguette/recipes/src/branch/master/rc
[baguette-profile]: https://git.baguette.netlib.re/Baguette/recipes/src/branch/master/rc/profile
[recipe-hello]: https://git.baguette.netlib.re/Baguette/recipes/src/branch/master/hello/recipe.spec
[recipe-dhcpcd]: https://git.baguette.netlib.re/Baguette/recipes/src/branch/master/dhcpcd/recipe.spec
[recipe-alsautils]: https://git.baguette.netlib.re/Baguette/recipes/src/branch/master/alsa-utils/recipe.spec
[toybox]: http://www.landley.net/toybox/
[cruxinit]: https://crux.nu/gitweb/?p=ports/core.git;a=blob;f=rc/rc;h=26b8ca08d67208ceda4d4004c8333d362bcdc689;hb=HEAD
[ksh]: https://github.com/att/ast
[zsh]: https://www.zsh.org/
[libarchive]: https://libarchive.org/
[bsdcpio]: https://libarchive.org/
[bsdtar]: https://libarchive.org/
[m4]: https://www.gnu.org/software/m4/m4.html
[gmake]: https://www.gnu.org/software/make/
[proot]: https://man.openbsd.org/proot
[openbsd]: https://openbsd.org/
[pfsense]: https://www.pfsense.org/
[alpine]: https://alpinelinux.org/
[crux]: https://crux.nu/
[inferno]: http://www.vitanuova.com/inferno/index.html
[plan9]: https://9p.io/plan9/index.html
[morpheus]: https://morpheus.2f30.org/
[suckless]: https://suckless.org/
[cat-v]: http://cat-v.org/

10
content/baguette/_index.fr.md Normal file
View File

@ -0,0 +1,10 @@
+++
title = "Baguette - un OS, des outils, des services"
paginate_by = 5
+++
# Baguette OS
La page n'a pas encore été traduite.

539
content/baguette/_index.md Normal file
View File

@ -0,0 +1,539 @@
+++
title = "Baguette - OS, tools and stuff"
paginate_by = 5
+++
- [Overview](#overview)
- [Technical choices](#technical-choices)
- [Custom tools](#custom-tools)
- [Slotting](#slotting)
- [Roadmap](#roadmap)
<a name="overview"></a>
# 1. Concise overview
BaguetteOS status: Work In Progress.
A beta will be available circa mid-2020 for advanced users.
<red>TODO: explications partie Custom Tools, factorisation, exemples de code et de commandes, liens vers d'autres projets.</red>
<red>Warning:</red> this documentation is currently oriented toward advanced unix users.
## Objectives, for simple users
BaguetteOS aims at provide a simple unix-like system, with an **unified web interface**.
**No command-line required for simple users.** *let users be just users*<br />
Simplicity should not only comes from an interface, but be inherent to the whole system.
If the OS is simple, there is no need to hack it.
**Unified interface is better than features.**<br />
We will provide the basic features then build up.
We do not want a patchwork of very different software, each of them having their own particularities.
**Online services.** *day-to-day use*<br />
The web interface should cover online services, providing an unified experience for main usages: mails, calendar, instant messaging, personal website, file sharing, etc.
**One-click management.** *service installs, updates, etc.*<br />
The web interface should handle basic system and network configurations, such as adding users, dhcp, DNS, backups, etc.
**Well-known, reliable software.** *for real*<br />
BaguetteOS relies on robust and independent software.
At no point the user should be forced to reinstall, a misconfiguration has to be easily fixed.
Static compilation for system tools *(at least)*: there is almost no way to get a broken system with an update (yes, *almost*, people are creative these days).
**Hardware support.** *new or old, fast or slow, it doesn't matter*<br />
We provide support for RPi and other small cards: if our kernel runs on it, it has to work.
Minimal hardware requirement should be around 100 MB RAM, 50 MB disk.
**Documentation.** *simple, reliable, useful, all-in-one-place*<br />
Similar to the OpenBSD FAQ: updated, complete, concise and well-written.
**Constrained devices use case.** *wanna try what small systems can do?*<br />
By default, we try to provide the smallest memory footprint: we do not ship manuals, nor runtime libraries when not required.
Our programs will never require you to download development libraries, nor alternative package managers.
The kernel is compiled with size in mind, as our choice of tools.
As a result, our system can be installed quickly even on slow machines.
## Objectives, for advanced users
**A knowable OS.** *simplicity at (almost) all cost*<br />
Any interested user should be able understand the role of every part of the base system: no compromise.
This means having a very small and consistent set of tools, easy to learn, easy to remember.
**Basic system and network management.** *with the simplest tools ever*<br />
We provide a web interface that should handle basic system and network configurations, such as adding users, firewall management, dhcp, DNS, backups, etc.
CLI tools are available to manage your services, they are design to be simple, consistent and reliable.
**Officially supported and documented services.** *so you are sure to get them working*<br />
We use some services for our own personal usage, so we will provide support for them.
For instance: gitea, postgresql, a building plateform and a continuous integration tool, etc.
**One need, one tool.** *this time for real*<br />
Installing an application or a library is done by [package][package].
Other methods are not supported **and the base system will never require them**.
We avoid to rely on `pip`, `cpanm`, or other third party package manager and dependency tree.
More on that in the [technical section](#technical-choices).
Starting, stopping, or configuring a service is done by [service][service].
This program alone is used to manage services on the OS.
Users should not be required to manually configure each software; instead, most of the configuration should be done upstream using templates.
Users should be able to change the default configuration through command-line options.
Manual configuration is the last option.
**Slotting.** *any software can be installed on any machine at any time, no extra tooling*<br />
[Slotting](#slotting) by default helps to install many programs, even with peculiar library version requirements.
The same program can be installed in several versions without conflicts.
## Objectives, for contributors
**Simple to contribute to.** *you can focus on providing recipes for packages, we handle the tooling*<br />
We want fewer and simpler tools as possible, even for packaging applications and libraries.
BaguetteOS ships [a simple tool to package applications][packaging] and it only requires you to create a very simple recipe for your package.
It handles [slotting](#slotting), compiling, stripping binaries and libraries, splitting the result into different packages (`-man`, `-lib`, `-doc`, etc.) then authenticate them and finally recreate the index.
All that, just by typing `packaging app`, nothing more.
<img src="/meanie-makefile.png" alt="meanie meanie dev" class="meanie-img" />
**No Makefile?** *no problem*<br />
Your application or your library lacks a build system? Here a [tool to create makefiles][build.zsh].
It works on any language, yes, even that one.
**Stable and development versions: same thing.** *slotting, again and again*<br />
One of the coolest aspect of slotting is: you don't need to change your system *at all* to be on the development version of the OS.
The development of the newest version of the OS is just slotted in `/usr/baguette-dev` and that's it.
At any point in time you can execute programs from your stable OS, or if you will, change a few environment variables and be in dev.
**New OS, open to explore**<br />
BaguetteOS do not suffer from cumbersome historical decisions: no overly engineered package format, no stupidly complex patchwork of mismatch tools.
BaguetteOS is easy enough to understand then even try to surpass it.
**Easy to write documentation.** *easy to write, hopefully less procrastination*<br />
Online documentation is written in Markdown (thanks Zola), and man pages too thanks to `scdoc`.
Every tool is shipped with a man page: no man page, no integration in base.
## Inspiration
- [CRUX][crux], [alpine][alpine]: simple to understand Linux systems
- [OpenBSD][openbsd]: security, therefore simplicity, no compromise
- [PFsense][pfsense]: system and (even advanced) networking administration, yet through a simple website
- [Plan9][plan9] and [Inferno][inferno]: everything is a file *no seriously guys*
- [suckless][suckless] and [cat-v][cat-v]: simplicity, code readability and re-usability
- [morpheus][morpheus]: static compilation for the OS, demystified
## Why not use another system?
This section could be expanded.
**A few reasons why any of the candidate covers it all.**<br />
1. **we want slotting**<br />
So we could change the way we install and maintain packages.
2. **we want fast install and start-up on dumb devices**<br />
Coreutils shrank to bare minimum, thanks to `toybox`.
We won't require a full-feature system with several hundred megabytes of used disk-space.
3. **documentation is important, but not for all installations**<br />
Your 42nd test install on a virtual machine doesn't need to provide manual for CLI tools you won't use since you do everything through the web interface.
Software and its documentation will be split: manual pages won't be provided by default.
4. **we want automatic tests, and to allow people to test our system**<br />
We want easy chroot installs, on any system.
5. **we want to run on tiny ARM boxes, old laptops, top-notch servers**<br />
So we need to run on any available kernel.
6. **we want to control software distribution releases**<br />
We don't accept to follow updates from an upstream OS that could break our system at any time.
---
**Now, let's take a look at each candidate.**
**OpenBSD.** *we will get there quick, but focus on Linux a bit before* <br />
We definitively want to use OpenBSD, currently we just focused on Linux for hardware compatibility reasons (and out of habits) but it's not set in stone.
We love OpenBSD big time, some of us use it daily.
We aim at providing rootfs for running our system under an OpenBSD kernel and environment.<br />
*(also, snapshots could be great, guys)*
**PFsense.** *network focused* <br />
Way too focused on networking.
Great system, does the job very well, but won't fit our objectives.
**CRUX and Alpine.** *great source of inspiration* <br />
We **do** use the CRUX's `rc` script, and as Alpine it is a source of inspiration for package recipes.
However, since we have to change all packages to get slotting, the service manager to have tokenized services, the packaging software to get declarative recipes (...), why even bother use those systems?
**GUIX and coe.** *not simple enough*<br />
This approach of package management is interesting, having to learn a language to make recipes isn't.
And that sums a lot about this technology.
It's great to have, not to put everywhere.
Every part of BaguetteOS is simple to understand, GUIX is not simple *enough*.
But keep it up guys, it's still awesome.
**Plan9, Inferno, morpheus, etc.** *kinda abandoned systems* <br />
That's it.
<a name="technical-choices"></a>
# 2. Technical choices
## Base system
**Linux kernel**, but we are lurking on the OpenBSD one.<br />
Linux is compatible with most hardware and software, it is fast and we can easily compile a custom version to remove most of the bloat for server usage.
Still, we don't want to rely on Linux-specific components.
At some point, our system will be kernel-agnostic and will be able to run on any BSD as well.
OpenBSD has `pledge` and `unveil` syscalls, which is an elegant way to provide a guarantee on the software behavior.
**Musl.** *reasonable libc for Linux*<br />
It has a reasonable amount of features, it is efficient, provides reasonable binary sizes and static compilation.
Musl is simple, reliable and remove all glibc-specific functions.
Others can be added easily, which is useful for compatibility and comparisons, through [slotting](#slotting).
**Bootable system and rootfs available.**<br />
A bootable system to install in virtual machines or bare metal, a rootfs to use BaguetteOS from any other OS, including non-Linux ones.
**SysV-style init + [CRUX-like /etc/{rc,mdev.conf,...}][baguette-rc]**. *easy to read, easy to adapt*<br />
The init could come from toybox or another minimalist project.
The [rc script from CRUX][cruxinit] is simple to understand and to adapt to any requirement, so we used it.
We also added some other scripts, like [for profile][baguette-profile] so we can easily manage slotting.
No systemd BS.
**Toybox.** *the megabyte coreutils*<br />
[Toybox][toybox] combines common unix command line utilities together into a single BSD-licensed executable.
It is designed to be simple even to read, and is standards-compliant.
For the base system, that's all we need.
**ksh and zsh**. *the first for scripts and root, the other for users*<br />
[Ksh][ksh] is a very stable and reliable shell from AT&T, trusted by the paranoid people of OpenBSD.
That's a safe choice for a base system and the root account.
On the other hand, we do use [zsh][zsh] daily, as for many users, so we may use it for development or build scripts but not in base.
**[Service][service] for service management** *tokenized service description, templating and dumb cli tools for the win* <br />
[See custom tools.](#custom-tools)
**[Package][package] for package management** *simple, efficient, dead simple code* <br />
[See custom tools.](#custom-tools)
**OpenSSH.** *as we all know and love*<br />
This is required for almost all usages, and for debug.
Let's try not to shoot ourselves in the foot.
That's all you need for starters. Web administrative interface will be added to the base when ready.
## Development, build tools, contribution
**Default building tools** *every tool needed to bootstrap*<br />
Clang (+ LLVM) is the default C (and C++) compiler.
[Libarchive][libarchive] is required for tarballs, packages, webhooks from `packaging`, and both [bsdcpio][bsdcpio] and [bsdtar][bsdtar] (sane implementations of `cpio` and `tar`).
[Auto-tools][autotools] are also required (for SysV init and libarchive).
[m4][m4] and [gnu-make][gmake] are required for compatibility reasons.
**Documentation.**<br />
A full hand-book like the OpenBSD FAQ.
Our software man-pages are written with `scdoc` so anyone can contribute.
**[Packaging][packaging] for packaging software and libraries.** *dead simple, intuitive*<br />
[See custom tools.](#custom-tools)
**Crystal language for system tools.** *syntax and productivity of Ruby, the speed of C*<br />
It is as simple to learn as a dynamic (oriented object) language, while at the same time being almost as fast as C.
Technically, Crystal is strongly typed which catches errors at compile-time, but with type inference so it is not cumbersome to use.
Applications are compiled in a simple binary, easy to deploy.
There is a good documentation, we used it for long enough to tell.
Technical choices are reasonable and documented.
Finally, Crystal has a large library with all we need for our system components.
There is not much of a drawback here.
Yes, this is a language you have to learn to work with us on a couple of projects (the web interface back-end, the service manager, the package manager and packaging) but it increases our productivity like crazy.
We heard about `nim` and a ton of other languages, but this is the one which reach the sweet spot between these parameters:
- productivity (the package manager was mostly done in a few days, [and is just a few hundred lines long][package])
- easy learning (a developer with basic notions of oriented-object can read our code, no black magic here)
- good documentation
- reasonably deployable (no host dependencies)
- execution speed
**Slotting.** *custom file system hierarchy*<br />
Our FS is not FHS-compliant, partially because of the origin-based slotting.
There is a strict separation between core system and third party software.<br />
[More information on slotting.](#slotting)
- `/usr/baguette` for core system programs
- `/usr/bad` for non slot-able software
- `/usr/third-party` for other software
<a name="custom-tools"></a>
# 3. BaguetteOS: custom tools
All our tools are designed to be simple to use, to understand, to read.
<u>Feel free to provide a feedback.</u>
<red>TODO: spec files, the file format used in `package`, `packaging` and `service`.</red>
```yaml
name: hello
version: 2.10
release: 2
sources: https://ftp.gnu.org/gnu/hello/hello-%{version}.tar.gz
dependencies:
- gettext
build-dependencies:
- make
options:
- configure: --disable-nls
@watch
curl 'https://ftp.gnu.org/gnu/hello/' -o- 2>/dev/null |
sed -n "/hello-.*\.tar\.gz/{s/\.tar\.gz.*//;s/.*hello-//;p}" |
tail -1
```
---
<red>TODO: explains why it's different / better than other package managers.</red>
**[Package][package]: our package manager.**<br />
Package covers the basics: install, remove, search and provide informations about a package, and it creates rootfs.
Package knows the minimal set of binaries and configuration required to build the target, so it only installs the minimal environment to perform compilation.
Package provides slotting by default: no need for custom environments for each software.
Packages format is a simple `tar` archive containing a `meta.spec` file describing all meta-data about the package (hash, manifest, etc.) and `files.tar.xz` with the files to install.
The database format contains `world`, `installed`, `[package-name]/[slot]/manifest` and `[package-name]/[slot]/meta.spec`.
Package's configuration is a list of repositories, authorized package signing keys and packaging variables (cflags, makeflags, and so on).
---
<a name="packaging"></a>
**[Packaging][packaging]: to create packages.**<br />
Packaging uses simple, declarative recipe files, here some examples: [hello][recipe-hello], [dhcpcd][recipe-dhcpcd], [alsa-utils][recipe-alsautils].
The only required parameters are `name` and `sources`.
<a name="packaging-build-env"></a>
Packaging creates build environments to test packages before validation.
It works as follow:
1. creation of a `/tmp/packaging/build-UUID/` directory
2. sources are downloaded, extracted then compiled<br />
Recipes and [`packaging` host configuration](#packaging-host-config) may add parameters to it: adding steps before compilation, changing `configure` arguments, etc.
The package is compiled for a specific slot, by default `/usr/baguette`.
3. compiled applications and libraries are put in `/tmp/packaging/build-UUID/root` which is used to create the final package
`Packaging` uses `package` to create low-cost build environments since we hardlink binaries into the building rootfs, which is inspired by the [proot][proot] tool on OpenBSD.
<red>TODO.</red>
<a name="packaging-host-config"></a>
Packaging configuration is simple.
<red>TODO.</red>
<a name="packaging-host-config"></a>
Packaging recipes.
---
<red>Factorisation.</red>
<a name="service"></a>
**[Service][service]: service management.** *not just `kill` or start/stop/status wrapper*<br />
Service management often comes with:
- default configuration files, users should learn how to configure them and do it manually
- a single possible instance, otherwise the configuration has to be heavily changed
- root-only management, simple users rarely run their own services (except on systemd, kuddos for once)
- no domain management
These shortcomings imply manual configuration, scripting to manage databases and users, specific tooling for each database and service: this is heavy machinery.
Simple users:
1. should be only required to provide absolute necessary information for their services
2. should be able to run as many services they want
3. shouldn't have to learn configuration syntax for their services
4. shouldn't be afraid of updates
Here are a few functionalities `service` brings.
1. **domain-wise service configuration**<br />
Example: `service add wordpress domain=example.com`
2. **templates** *configuration files are generated by templates and user data*<br />
When we want a software, for instance a blog, we want to provide the minimum informations it requires to work and that's it.
When `service` is invoked to start a service, it verifies if a configuration file is installed or create it.
Users shouldn't need to manually change the configuration.
<u>Syntax may change at any time without breaking a single service</u>, since the configuration will smoothly be regenerated with useful information at start-up.
3. **environments**<br />
Each service can be installed in a specific environment (read: a custom rootfs).
Example: `service add wordpress example.com testing-env`
4. **unified way to configure the system**<br />
It alleviates the need for manual configuration.
For example, adding a Wordpress service will automatically change the `nginx` configuration, create a new database and a new user in `mariadb` for this specific service.
If several `nginx` are required, ports will be registered and automatically managed for each instance, no need for user input.<br />
Behind the scene, it's a simple token system with configuration templating!<br />
No heavy machinery here, and we'll keep it that way.
---
<red>Better introduction.</red>
<a name="libipc"></a>
**[LibIPC][libipc]: an IPC communication library** *nothing new, yet it still feels fresh*<br />
1. Applications should talk to each other
2. We need services, not libraries<br />
Therefore, languages are irrelevant: you can use *any* library in *any* language.
LibIPC is currently used for the administration dashboard, the web interface for the services, [for a kanban][todod] and several other tools we use for collaboration.
It provides a way to communicate between clients and services using simple unix sockets behind the scene.
<red>Explain remote communications.</red>
Remote remote communications are transparent.
- clients and services do not need remote communication
- any client can join remote services via any communication protocol
- any service is implicitly accessible from anywhere, anyhow
C library with Crystal bindings (other languages coming soon)
<pre><code class="Crystal">
require "ipc.cr"
server = IPC::Service.new "MyService"
server.loop do |message|
# ...
end
</code></pre>
<red>TODO: show that's easy to write even in plain C.</red>
<red>TODO: performances are crazy, we have to tell the world.</red>
---
<red>TODO.</red>
**[Build.zsh][build.zsh]: makefile creation.** *for mere mortals*<br />
Build.zsh creates a makefile from a simple declarative configuration file.
It can replace most build systems.
---
<red>TODO.</red>
**[tap-aggregator][tap-aggregator]: quality assurance & test results aggregation**<br />
---
<red>TODO: better explanation.</red>
**[Webhooksd][webhooksd]: verify recipes.**<br />
Webhooksd provides an automatic verification of the recipes, based on new application or library version.
Paired with a build system, new recipes received in the repository create packages for a couple of architectures (x86_64, ARM, others will follow).
## Still in discussion
For the simple users, we want to provide an unified web interface to manage the system and online services.
We currently use `Crystal` to work on service back-ends for a variety of projects, we are satisfied on a technical level and we are productive, it is highly probable we continue using it.
The front-end is still in discussion: we currently use `livescript` and it is way more usable than plain JS, but it is painful to debug.
So, we need a language for both administration dashboard and online services, here some examples we find interesting for the job:
- Purescript
- haskell-like syntax, with a smaller set of notions
- strongly typed, with type inference
- good documentation
- useful compilation errors
- no runtime error
- Elm
- as Purescript but with way fewer documentation (but reading the code is sometimes enough here)
- less generic code (functions such as `fold` and `map` have hardcoded type), which feels a bit hacky
- still very young
- WASM
- seems to be a very young tech, with no real good language or documentation
- Zig has wasm as a Tier 1 support, we should investigate
<a name="slotting"></a>
# 4. Slotting: providing software the right way
The usual way to provide software is to maintain a version of a software or a library, package it into a distribution, then provide it as *the* OS version of the software.
In the long run, software and libraries change, which is no big deal since maintainers verify the consistency of the different versions provided by the OS.
<red>TODO</red>
**Problem:** what happens when two programs need a different version of a library?<br />
**Problem:** what happens when two libraries are compatible but you want both on your system (see libressl and openssl)?<br />
**Problem:** what happens when you want to provide a **very** long term support for your users (see companies running decade-old databases)?
BaguetteOS has a simple and safe way to let users and maintainers provide packages: `slotting`.
Official OS packages are installed under `/usr/baguette/`, for non-essential programs.
Here, the slot is `baguette`.
Any package outside the official ones are in another named slot.
<red>TODO</red>
**This in nothing new, however not often used, and still maybe the best way to handle the problem.**
### BaguetteOS file system hierarchy
- usual directories under root: bin, sbin, lib, boot, dev, proc, sys, home, mnt, root, run, tmp
- `/etc/rc` with `services` and `environments` for running service instances
- `/etc/templates` for local service configuration templates
- `/var/cache`
- under `/srv/"env-name"` (see [service](#service)), these subdirs when required: `/etc`, `/data`, `/cache`, `/run`
- under `/usr`
- `/local`: things that are installed by the local system administrator without using packages
- `/baguette`: things provided by the system that are not necessary for it to run (and boot, and restart, and do system things)
- `/"repo"`: `/lib`, `/bin`, `/share`, `/man`, `/include` (`/libexec` but we try to avoid it whenever possible.)
- `/bad`: things that cannot be properly installed or slotted somewhere else
<a name="roadmap"></a>
# Roadmap
<red>TODO</red>
We currently aim at providing a rootfs with our tools, when we will have enough spare time to contribute.
[service]: https://git.baguette.netlib.re/Baguette/service
[package]: https://git.baguette.netlib.re/Baguette/package
[packaging]: https://git.baguette.netlib.re/Baguette/packaging
[build.zsh]: https://git.baguette.netlib.re/Baguette/build.zsh
[libipc]: https://git.baguette.netlib.re/Baguette/libipc
[webhooksd]: https://git.baguette.netlib.re/Baguette/
[tap-aggregator]: https://git.baguette.netlib.re/Baguette/tap-aggregator
[baguette-rc]: https://git.baguette.netlib.re/Baguette/recipes/src/branch/master/rc
[baguette-profile]: https://git.baguette.netlib.re/Baguette/recipes/src/branch/master/rc/profile
[recipe-hello]: https://git.baguette.netlib.re/Baguette/recipes/src/branch/master/hello/recipe.spec
[recipe-dhcpcd]: https://git.baguette.netlib.re/Baguette/recipes/src/branch/master/dhcpcd/recipe.spec
[recipe-alsautils]: https://git.baguette.netlib.re/Baguette/recipes/src/branch/master/alsa-utils/recipe.spec
[toybox]: http://www.landley.net/toybox/
[cruxinit]: https://crux.nu/gitweb/?p=ports/core.git;a=blob;f=rc/rc;h=26b8ca08d67208ceda4d4004c8333d362bcdc689;hb=HEAD
[ksh]: https://github.com/att/ast
[zsh]: https://www.zsh.org/
[libarchive]: https://libarchive.org/
[bsdcpio]: https://libarchive.org/
[bsdtar]: https://libarchive.org/
[m4]: https://www.gnu.org/software/m4/m4.html
[gmake]: https://www.gnu.org/software/make/
[proot]: https://man.openbsd.org/proot
[openbsd]: https://openbsd.org/
[pfsense]: https://www.pfsense.org/
[alpine]: https://alpinelinux.org/
[crux]: https://crux.nu/
[inferno]: http://www.vitanuova.com/inferno/index.html
[plan9]: https://9p.io/plan9/index.html
[morpheus]: https://morpheus.2f30.org/
[suckless]: https://suckless.org/
[cat-v]: http://cat-v.org/

16
sass/custom.scss
View File

@ -19,12 +19,12 @@ red {
}
// less spacing in lists
li > p {
padding: -1px;
padding-top: -1px;
padding-bottom: -1px;
margin: -1px;
margin-top: -1px;
margin-bottom: -1px;
}
//li > p {
// padding: -1px;
// padding-top: -1px;
// padding-bottom: -1px;
// margin: -1px;
// margin-top: -1px;
// margin-bottom: -1px;
//}

BIN
static/favicon.ico Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 3.9 KiB