Baguette
/
website
3
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Slotting and rant about Qemu/Xen.

master
Karchnu 2020-05-02 15:53:43 +02:00
parent 2cb2f0dc71
commit 7674e071a7
1 changed files with 111 additions and 40 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

143
content/baguette/index.md
View File

@ -10,6 +10,7 @@ paginate_by = 5
- [Custom tools](#custom-tools) - [Custom tools](#custom-tools)
- [Slotting](#slotting) - [Slotting](#slotting)
- [Roadmap](#roadmap) - [Roadmap](#roadmap)
- [BaguetteOS File-System Hierarchy](#baguetteos-fsh)
<a name="overview"></a> <a name="overview"></a>
# 1. Concise overview # 1. Concise overview
@ -627,8 +628,19 @@ Service management often comes with:
These shortcomings imply manual configuration, scripting to manage databases and users, specific tooling for each database and service: this is heavy machinery. These shortcomings imply manual configuration, scripting to manage databases and users, specific tooling for each database and service: this is heavy machinery.
To overcome drawbacks of having simplistic tools, sys-admins developed all kind of monstrous architectures. To overcome drawbacks of having simplistic tools, sys-admins developed all kind of monstrous architectures.
- **LXC** *it's basically a chroot with network and software limits*<br /> - **LXC** *it's basically a chroot with network and software limits*<br />
LXC is *kinda* reasonable, and may be useful in som cases, but it provides no simple way of configuring our services. LXC is *kinda* reasonable, and may be useful in some cases, but it provides no simple way of configuring our services.
- **Qemu + KVM, Xen** *let's add software mimicking hardware's complexity to the mix, telling everyone it's for security and simplicity* - **Qemu + KVM, Xen** *let's add software mimicking hardware's complexity to the mix, telling everyone it's for security and simplicity*<br />
These programs make the administration simple for sys-admins: no need to configure thoroughly users, groups, etc.
Everyone is root and handle its administration as (s)he wants.
They also helps big companies to have a large computing capacity paying for itself since they rent it when not needed.
At no point Qemu or Xen are helping you getting your services up and running, and [they are not made for security][cveqemu].
Yes, running broken programs within Qemu still is better than not.
But.
This is still <u>way less efficient than fixing the application</u>.
Running applications as simple users, compiling them with sane default options ([RETGUARD][retguard] for example) and providing a few syscalls (like [pledge][pledge] and [unveil][unveil]) to catch errors and most security holes is simple nowadays, let's use that.<br />
*But, I can't do that in my company 'cause reasons.*<br />
That's okay if you don't want to do that in your company, we are here for <u>**final users**</u> and advanced users, mostly.
We don't aim at fixing everybody's problems.
- **docker** *I don't know how to do simple applications nor packages, so I give to you my whole dev environment*<br /> - **docker** *I don't know how to do simple applications nor packages, so I give to you my whole dev environment*<br />
Note: we have to admit, packaging on most OS is painful for absolutely no good reason. Note: we have to admit, packaging on most OS is painful for absolutely no good reason.
- **Chef and Puppet** *the 500 MB running Ruby code on our virtual machines just to check for configuration updates is okay 'cause memory is cheap, right?*<br /> - **Chef and Puppet** *the 500 MB running Ruby code on our virtual machines just to check for configuration updates is okay 'cause memory is cheap, right?*<br />
@ -1094,55 +1106,61 @@ Maintainers' job is to verify that all programs have the right library versions
### Current set of problems ### Current set of problems
- what happens when two programs need a different version of a library?<br /> - What happens when two programs need a different version of a library?<br />
The installation of both may no be possible. The installation of both may no be possible.
See python from version 2 to 3 as an example: developers knew it will break OSs. See python from version 2 to 3 as an example: developers knew it will break OSs.
So, they provided by themselves new names for their binaries (`python-2.7`), and libraries are *by default* packaged into a directory specific for a python version, such as `/usr/lib/python3.8/`. So, they provided by themselves new names for their binaries (`python-2.7`), and libraries are *by default* packaged into a directory specific for a python version, such as `/usr/lib/python3.8/`.
This is mostly done for languages, but what about other packaged applications and libraries? This is mostly done for languages, but what about other packaged applications and libraries?
- what happens when two libraries are compatible but you want both on your system (see libressl and openssl)?<br /> - What happens when two libraries are compatible but you want both on your system (see libressl and openssl)?<br />
One of them could be provided in another path, such as `/usr/lib/libressl`. One of them could be provided in another path, such as `/usr/lib/libressl`.
- what happens when you want to provide a **very** long term support for your users? - What happens when you want to provide a **very** long term support for your users?
*see companies running decade-old OSs and databases* *see companies running decade-old OSs and databases*
BaguetteOS has a simple and safe way to let users and maintainers provide packages: `slotting`. BaguetteOS has a simple and safe way to let users and maintainers provide packages: `slotting`.
### What is slotting? ### What is slotting?
Slotting is a lot like repositories, except that repositories provide packages in the same prefixes than your base system. Slotting is a way to use prefixes (paths, directories) to separate execution environments: a program **A**, requiring libraries **B and C** can be installed this way:
```sh
/usr/slot-A/bin/A
/usr/slot-A/lib/B
/usr/slot-A/lib/C
```
In this example, the `slot` is named **slot-A**, providing an execution environment for A <u>no matter</u> the OS version of *B* and *C*.
**Without slotting** <side-note>*basically, your life sucks*</side-note><br /> **Without slotting** <side-note>*basically, your life sucks*</side-note><br />
Let's take an example with simple repositories. Let's take an example with software provided by a non-official repository on a Linux distribution.
You add a non-official repository for <blue>my-overly-awesome-game</blue> to your Debian system. You add a non-official repository for <blue>my-overly-awesome-game</blue> to your Debian system.
This newly installed program will be in `/usr/bin`, as every other program. This newly installed program will be in `/usr/bin`, as every other program.
1. what if the game requires libraries?<br /> 1. What if the game requires libraries?<br />
These libraries are installed in `/usr/lib`. These libraries are installed in `/usr/lib`.
2. what if the game requires libraries that are not in the official repository?<br /> 2. What if the game requires libraries that are not in the official repository?<br />
Either the repository for <blue>my-overly-awesome-game</blue> provides them directly, or you can find another repository providing them. Either the repository for <blue>my-overly-awesome-game</blue> provides them directly, or you can find another repository providing them.
In <u>both cases</u> these libraries will end-up in `/usr/lib`. In <u>both cases</u> these libraries will end-up in `/usr/lib`.
**With slotting** <side-note>*your're awesome*</side-note><br /> **With slotting** <side-note>*your're awesome*</side-note><br />
With slotting, the program will be in `/usr/`<blue>my-overly-awesome-game</blue>`/bin`. With slotting, the program will be in `/usr/`<blue>my-overly-awesome-game</blue>`/bin`.
1. What if requires libraries? 1. What if requires libraries?<br />
These libraries will be installed in your base system so any of your non-official slot can use them. These libraries will be installed in your base system so any of your non-official slot can use them.
2. What if the required libraries aren't available in the official `baguette` slot? 2. What if the required libraries aren't available in the official BaguetteOS repository?<br />
Either the game slot provides them, or they are in another slot. Either the game slot provides them, or they are in another slot.
In <u>both cases</u> the base system won't change a bit. In <u>both cases</u> the base system won't change a bit.
Official OS packages are installed under `/usr/baguette/`, for non-essential programs. Official OS packages are installed in the slot named `baguette` (`/usr/baguette/`), for non-essential programs.
Here, the slot is `baguette`. Any non-official package is in another slot.
Any package outside the official ones are in another named slot.
Wanna support Python 2.7 **for life**? Wanna support Python 2.7 **for life**?
Just maintain a `python-2.7` slot and tell the world! Just maintain a `python-2.7` slot and tell the world!
If BaguetteOS do not provide required libraries for your slot, just add them in your slot. If BaguetteOS do not provide required libraries for the continuous support of your application, just add them in your slot.
**This is nothing new, however not used directly in OSs, and still maybe the best way to handle the problem.** **This is nothing new, however not used directly in OSs, and still maybe the best way to handle the problem.**
### Why not use X? ### Why not use X?
Others are doing slotting too: snap, flatpak, cpanm, pip, go, stack, cabal, ... *the list seems endless*<br /> Others are doing slotting too: snap, flatpak, cpanm, pip, go, stack, cabal... <side-note>*the list seems endless*</side-note><br />
They all use slotting... *but*. They all use slotting... *but*.
Since they are *yet another package manager for your system*, you need to install all your software dependencies from their own dependency tree. Since they are *yet another package manager for your system*, you need to install all your software dependencies from their own dependency tree.
You have now a shit-ton of ways to get software updates, and for *almost* all of them, it's just giving up the idea of having people testing the software before providing it. You have now a shit-ton of ways to get software updates, and for *almost* all of them, it's just giving up the idea of having people testing the software before providing it.
@ -1167,20 +1185,82 @@ You have now a shit-ton of ways to get software updates, and for *almost* all of
<img src="/can-i-haz-my-firefox.png" alt="wanna the updates at all cost" class="meanie-img" /> <img src="/can-i-haz-my-firefox.png" alt="wanna the updates at all cost" class="meanie-img" />
<u>*But, BaguetteOS... I still need my last version of Blah!*</u> We gotcha buddy.<br /> <u>*But, BaguetteOS... I still need my last version of Blah!*</u> We gotcha buddy.<br />
You wanna go fast? Try sonic the good slot: `/usr/sonic`. You wanna go fast? Try sonic the fast slot: `/usr/sonic`.
With this slot, the BaguetteOS maintainers provide the last versions of a variety of applications and libraries. With this slot, the BaguetteOS maintainers provide the last versions of a variety of applications and libraries.
You will have bleeding-edge technologies and bugs. You're welcome! You will have bleeding-edge technologies and bugs. You're welcome!
### How slotting works in BaguetteOS ### How slotting works in BaguetteOS
<red>TODO: this documentation is not available right now. Sure the project is still awesome anyway!</red> **Applications and libraries provided by BaguetteOS.**<br />
For all official OS versions of the applications and libraries, `BaguetteOS` will provide them in `/usr/baguette`, the `baguette` slot.
In case several versions of a library are provided, they will be slotted.
For example, `LLVM` is provided in several versions (`llvm8`, `llvm9`, etc.), only the most recent is in `baguette`.
```zsh
$ ls /usr
/usr/baguette
/usr/llvm8
/usr/llvm9
...
```
**Applications and libraries provided by third parties.**<br />
`BaguetteOS` allows people providing their applications and libraries easily by creating repositories, but you have to slot them.
For example, if you want to provide a specific `nodejs` version, please use the following convention:
```zsh
/usr/$application-$version/
```
### How use and change slots used ### How use and change slots used
<red>TODO: this documentation is not available right now. Sure the project is still awesome anyway!</red> **Use a slot.**<br />
`BaguetteOS` comes with a `/etc/profile` script, adding the functions `prefix_add` and `prefix_del` to your shell.
So, if you want to use an application in the slot `my-awesome-app`:
### BaguetteOS file system hierarchy ```sh
$ prefix_add my-awesome-app
```
Your `$PATH` is then changed, allowing you to run applications in `/usr/my-awesome-app`:
```sh
$ echo $PATH
/bin:/usr/bin:/usr/local/bin:/usr/my-awesome-app/bin
```
**Install a new repository.**<br />
A new repository can be used after adding its address in `/etc/package.conf`:
```
# File `/etc/package.conf`
# Default repositories
https://repos.baguette.netlib.re/$arch/
# Add your repositories here
https://repos.my-awesome-app.com/$arch/
```
Then you can update your list of packages and install your application:
```zsh
# package update
# package install my-awesome-app
```
That's all folks!
<a name="roadmap"></a>
# 5. Roadmap
We currently aim at providing a rootfs with our tools, when we will have enough spare time to contribute.
**Web interface is for later**: we need more time to design its graphical components.
On the other hand, back-end should be straightforward.
<a name="baguetteos-fsh"></a>
# 6. BaguetteOS file system hierarchy
- usual directories under root: bin, sbin, lib, boot, dev, proc, sys, home, mnt, root, run, tmp - usual directories under root: bin, sbin, lib, boot, dev, proc, sys, home, mnt, root, run, tmp
- `/etc/rc` with `services` and `environments` for running service instances - `/etc/rc` with `services` and `environments` for running service instances
@ -1193,22 +1273,6 @@ You will have bleeding-edge technologies and bugs. You're welcome!
- `/"repo"`: `/lib`, `/bin`, `/share`, `/man`, `/include` (`/libexec` but we try to avoid it whenever possible.) - `/"repo"`: `/lib`, `/bin`, `/share`, `/man`, `/include` (`/libexec` but we try to avoid it whenever possible.)
- `/bad`: things that cannot be properly installed or slotted somewhere else - `/bad`: things that cannot be properly installed or slotted somewhere else
<a name="roadmap"></a>
# 5. Roadmap
We currently aim at providing a rootfs with our tools, when we will have enough spare time to contribute.
**Web interface is for later**: we need more time to design its graphical components.
On the other hand, back-end should be straightforward.
<a name="contact"></a>
# 6. Contact
Do not hesitate to come on our [Mattermost][mattermost].
We will soon have bridges on XMPP and IRC, stay tuned!
[mattermost]: https://team.baguette.netlib.re/
[service]: https://git.baguette.netlib.re/Baguette/service [service]: https://git.baguette.netlib.re/Baguette/service
[package]: https://git.baguette.netlib.re/Baguette/package [package]: https://git.baguette.netlib.re/Baguette/package
[packaging]: https://git.baguette.netlib.re/Baguette/packaging [packaging]: https://git.baguette.netlib.re/Baguette/packaging
@ -1259,3 +1323,10 @@ We will soon have bridges on XMPP and IRC, stay tuned!
[working-service-asciinema]: https://asciinema.org/a/0p2vGNA1TUmvq0s61Lu0r4TN6 [working-service-asciinema]: https://asciinema.org/a/0p2vGNA1TUmvq0s61Lu0r4TN6
[ruby-memory-bp]: https://web.archive.org/web/20160329122617/http://blog.rubybestpractices.com/posts/ewong/005-Avoiding-system-calls.html [ruby-memory-bp]: https://web.archive.org/web/20160329122617/http://blog.rubybestpractices.com/posts/ewong/005-Avoiding-system-calls.html
[pledge]: https://man.openbsd.org/pledge.2
[unveil]: https://man.openbsd.org/unveil.2
[retguard]: https://undeadly.org/cgi?action=article&sid=20170819230157
[cveqemu]: https://www.cvedetails.com/vulnerability-list.php?vendor_id=7506&product_id=0&version_id=0&page=1&hasexp=0&opdos=0&opec=0&opov=0&opcsrf=0&opgpriv=0&opsqli=0&opxss=0&opdirt=0&opmemc=0&ophttprs=0&opbyp=0&opfileinc=0&opginf=0&cvssscoremin=0&cvssscoremax=0&year=0&cweid=0&order=1&trc=276&sha=6055b0330a499f6aed7620adb79dc0cc143e50bc