website/content/projects/specification-file-format/index.md

+++ title = "Specification file format"

paginate_by = 5 +++

Configuration should be done with a simple file format. Here is the file format BaguetteOS uses for its tools: spec.

• Overview
• Real-life examples
  • In packaging
  • In service
• Rationale

Overview of the spec file format

# This is a comment

# This is a simple variable instanciation
my-variable: value

# This is an inlined list
my-list: a, b, c

# This is a multiline list
my-list:
  - a
  - b
  - c

# Interpolation is performed with %{variable}
url: https://example.com/
software-url: %{url}/my-software-v0.2.tar.gz


# "@an-example-of-code-block" is a code block.
@an-example-of-code-block
	# Code blocks allow inserting simple shell scripting in the configuration file.
	curl %{url} -o- | \
	sed "s/_my-software.tar.gz//" | \
	tail -1
	# The scope is the indentation.

# Not the same indentation: not in the block anymore.

# "%an-example-of-section" is a section, allowing to add metadata to a "target".
# "the-target" is the target, receiving metadata.
%an-example-of-section the-target
	# Within a section, variables are declared in the same way as outside the block.
	name: The name of the target.
	list-of-random-info-about-the-target:
	  - a
	  - b
	  - c

Real-life examples

In packaging

Most of the recipes used to build packages with packaging looks like this:

name: my-program
description: "my program is a tool to do blah"
version: 0.5.2
sources: https://example.com/software-v%{version}.tar.gz

From time to time, an application requires non-standard operation in its build-system, so a code block is used. @configure code blocks allow to change the execution of the configure script in a package build. Without it, packaging simply performs a ./configure with options provided in the recipe. In the next example, a file (some-file) needs to be changed before using the configure script.

# "@configure" is a code block in the recipes of `packaging`.
@configure
	sed -i "s/a/b/" some-file
	./configure

In this example, sed is called before ./configure, and this could have been as complex as required by the build system of the application. Having this code block allows to package applications with non-standard build system, while keeping recipes as simple as possible when applications do use standard build systems.

In service

A service description can be as simple as providing the name of the service, the command to run it and the list of tokens the service provides or consumes. For example, the pubsubd service:

name: pubsubd
environment-variables:
  - LD_LIBRARY_PATH=/usr/local/lib
command: pubsubd -s /srv/${ENVIRONMENT}/pubsub
provides: pubsub

Additionaly, before running the service, the service may require the creation of directories (or files). In this case, sections are used to describe the file to create. In the next example,

# "%configuration" is a section and its "target" ("nginx.conf" in this example ) is
# the name of both the template to use and the generated file.
%configuration nginx.conf
	# Within a section, variables are declared in the same way as outside the block.
	name: This is the configuration file for Nginx.
	permissions: 0700

Another example is the %directory section.

# "%directory" is a section, the target is the name of the directory.
%directory %{SERVICE_ROOT}/data
  name: Storage directory for pubsubd.
  # A command may be provided to know how to create the directory and its content.
  command: install -d -m6777 %{SERVICE_ROOT}/data   

Rationale

spec is declarative describe what is, not how

Configuration should be about describing what something is. For example, a web server should ask the domains it should serve.

spec can be imperative too for complex cases, where it is not possible to do differently

How things are done is an implementation detail. However, in complex cases the user may provide optional informations on how to do things.

Imperative configuration is useful for example in packaging: some applications require specific non-generic operations to be packaged. In those cases, it is not a reasonable solution to try to implement every possible - and changing - way of packaging an application directly in packaging. Instead, each application with a non-standard build system has a code block in its recipe where every non-standard operation is performed.

spec is concise and simple variables, lists, code blocks and sections

Variables and lists look like YAML (and markdown), which is one of the simpliest configuration possible. It is humanly readable, unlike XML and its bloated-beyond-repair syntax, and it is simpler to read than JSON.

XML (XML Schema could be used too, to create simpler lists after the definition of the schema, but seriously… if documentation is needed to create a simple fucking list, this is just fucked up)

<the-property>
	the value
</the-property>
<a-list>
	<element>
		value
	</element>
	<element>
		value
	</element>
	<element>
		value
	</element>
</a-list>

JSON

{
	"the-property": "the value",
	"a-list": [ "value", "value", "value" ]
}

YAML

the-property: the value
a-list: value, value, value

code blocks allow to insert shell scripting, because sometimes we do want to tell instructions and it is not available in YAML or JSON.

Finally, sections allow a more concise configuration by factoring simple patterns.

# Without sections
list-of-directories:
  this-directory:
    name: Storage directory.
    path: /path/to/dir

# With sections
%directory /path/to/dir
  name: Storage directory.

spec is simple for developers too

The spec file format can be read line-by-line, which is very simple to implement.

The specification file format is used in practice in packaging and service.

Back to top