Skip to content

Commit

Permalink
Improve documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
@amotl
amotl committed Jun 7, 2022
1 parent 3501076 commit 9a7b0ff
Show file tree
Hide file tree
Showing 8 changed files with 167 additions and 129 deletions.
1 change: 1 addition & 0 deletions CHANGES.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ in progress
Now, it gets 4 VCPUs and 4096 MB RAM.
- Improve platform guards and naming things
- Improve central command invocation function
- Improve documentation


2022-05-20 0.2.0
Expand Down
62 changes: 41 additions & 21 deletions README.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -40,9 +40,9 @@ Racker
:target: https://pypi.org/project/racker/
:alt: License

.. image:: https://img.shields.io/pypi/dm/racker.svg?label=PyPI%20downloads
.. image:: https://pepy.tech/badge/racker/month
:target: https://pypi.org/project/racker/
:alt: PyPI downloads
:alt: PyPI downloads / month


----
Expand All @@ -52,44 +52,61 @@ Racker
About
*****

An experimental harness tool based on `systemd`_ and `systemd-nspawn`_, to run
`operating system containers <OS containers_>`_, in the spirit of addressing some details of
`Docker Considered Harmful`_ and `Systemd vs. Docker`_.

At the same time, it is a tribute to the authors and contributors of GNU, Linux,
systemd, Python, VirtualBox, Vagrant, Docker, and more.
At a glance
===========

Most people running Linux probably want to use `Podman`_ these days. For more
background, enjoy reading `Container wars`_ and `Container Tools Guide`_.
Racker is an experimental harness tool for provisioning and running operating
system containers.

Racker is ...
=============
For launching Linux operating systems, it is based on `systemd`_ and
`systemd-nspawn`_, to run `operating system containers <OS containers_>`_,
in the spirit of addressing some details of `Docker Considered Harmful`_ and
`Systemd vs. Docker`_.

- A runtime harness for testing software packages and similar purposes, in

Details
=======

With Racker, you can ...

- Launch an interactive command prompt within a Linux environment or invoke
programs non-interactively.

- Use the runtime harness for testing software packages and similar purposes, in
different environments, mostly run headless and non-interactively.

Racker is ...

- A lightweight wrapper around ``systemd-nspawn`` to provide and launch
container environments for/with ``systemd``.

- A lightweight wrapper around ``vagrant`` to provide convenient access to all
things needing a full VM, like running Windows on Linux or macOS.

- A tribute to the authors and contributors of GNU, Linux, systemd, Python,
VirtualBox, Vagrant, Docker, Windows, Windows Docker Machine and countless
others.


Comparison with similar tools
=============================

The aims of Racker are very similar to `Distrobox`_ and `Toolbox`_. However,
there are also some differences.
The aims of Racker are very similar to `Docker`_, `Podman`_, `Distrobox`_ and
`Toolbox`_. However, there are also some differences.

- Racker is currently based on `systemd-nspawn`_ instead of `Docker`_ or
`Podman`_.
- Racker can invoke any kind of container payload, but strongly focuses on
running `OS containers`_ aka. `OS-level virtualization`_, using `systemd`_
as init process.
- Racker aims to provide concise usability by folding its usage into a single
command.
Most people running Linux probably want to use `Podman`_ these days. For more
background, enjoy reading `Container wars`_ and `Container Tools Guide`_.

- Racker is currently based on `systemd-nspawn`_ and `Vagrant`_ instead of
`Docker`_ or `Podman`_.
- Racker's focus is to provide easy provisioning and launching `OS containers`_
aka. `OS-level virtualization`_, using `systemd`_ as init process.
- The acquisition and provisioning of operating system images does not need any
special preparation steps, those are handled by Racker on the fly.
- Racker aims to provide concise usability by folding its usage into a single
command.
- Racker is written in Python instead of Golang or Bash.

See also `Comparison with similar tools - more details`_.

Expand Down Expand Up @@ -398,6 +415,7 @@ Troubleshooting

.. _autopkgtest: https://www.freedesktop.org/wiki/Software/systemd/autopkgtest/
.. _Changing Roots: http://0pointer.de/blog/projects/changing-roots.html
.. _Chocolatey: https://chocolatey.org/
.. _Comparison with similar tools - more details: https://github.com/cicerops/racker/blob/main/doc/comparison.rst
.. _Container Tools Guide: https://github.com/containers/buildah/tree/main/docs/containertools
.. _Container wars: https://github.com/cicerops/racker/blob/main/doc/research/container-wars.rst
Expand All @@ -421,6 +439,8 @@ Troubleshooting
.. _Systemd vs. Docker: https://lwn.net/Articles/676831/
.. _Toolbox: https://containertoolbx.org/
.. _umoci: https://github.com/opencontainers/umoci
.. _Vagrant: https://www.vagrantup.com/
.. _Windows Docker Machine: https://github.com/StefanScherer/windows-docker-machine

.. _Containers without a Container Manager, with systemd: https://invidious.fdn.fr/watch?v=sqhojVPr7xM
.. _Lennart Poettering und Kay Sievers über Systemd: https://invidious.fdn.fr/watch?v=6Q_iTG6_EF4
Expand Down
69 changes: 32 additions & 37 deletions doc/backlog.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,62 +7,54 @@ Racker backlog
Those are just random notes about ideas and more.


***********
Iteration 2
***********

- [x] Check if software tests can be invoked on CI/GHA.
- [x] Improve software tests.
- [x] Split functionality between ``racker`` and ``postroj``.

- ``racker {run,ps,pull,logs}``
- ``postroj {list-images,pkgprobe,selftest}``
- ``pronto opensuse/tumbleweed hostnamectl`` (``pronto.hexagon`` (hx), ``pronto.kaxon`` (kx))

- [x] Continue renaming to ``racker``.
- [x] Improve error messages, see "Compatibility" section
Compare with ``docker run --rm -it debian:bullseye-slim foo``.

- ``racker run -it --rm debian-stretch hostnamectl``
- ``racker run -it --rm debian-stretch /bin/hostnamectl``
- ``racker run -it --rm debian-stretch /usr/bin/hostnamectl``

- [x] Provide more advanced and generic image (label) resolution.
From docker.io, ghcr.io, registry.access.redhat.com, etc.
- [x] Improve documentation
- [x] Is it possible to run RHEL and SLES?

- registry.access.redhat.com/rhel7/rhel
- registry.suse.com/bci/bci-base
- https://registry.suse.com/


***********
Iteration 3
***********

- [o] Support for Windows. ``windows/nanoserver:1809`` and ``windows/servercore:ltsc2019``.
- [o] Support for Windows 11 and Windows Server 2022

- https://github.com/StefanScherer/packer-windows
- https://app.vagrantup.com/StefanScherer/boxes/windows_2022
- ``mcr.microsoft.com/windows/servercore:ltsc2022-amd64``
- [o] Provide curated image types per ``os://`` or ``vagrant://``.
- [o] Tests: Wait for container to properly shut down before moving on.

- Q: When running ``postroj selftest``, why is there more output from
containers shutting down, while the program is finished already?
- Q: Why are the tests failing when trying to subsequently spin up / tear down
the same container?
- [o] Use/integrate with ``mkosi``.

- https://github.com/systemd/mkosi
- http://0pointer.net/blog/mkosi-a-tool-for-generating-os-images.html
- https://lwn.net/Articles/726655/
- https://github.com/asiffer/netspot/search?q=mkosi


*************
Iteration 3.5
*************

- [o] Windows support leftovers

- --cpus=8 --memory=8192M
- https://github.com/docker/machine/issues/531#issuecomment-73938730
- --mount type=git,src=https://github.com/crate/crate,dst=C:/src
- --repository=https://github.com/crate/crate
- https://github.com/StefanScherer/packer-windows
- Make piping from STDIN possible
- Display machine resources (VCPUs, memory) before starting the command/job.

- [o] Address issues in ``bugs.rst``
- [o] When using ``apt-get``, use ``DEBIAN_FRONTEND=noninteractive``

- [o] Invoke arbitrary Docker containers, even when they don't contain an OS root directory.

- Directory ``/var/lib/postroj/archive/hello-world.img/rootfs`` doesn't look like an OS root directory (os-release file is missing). Refusing.
- Directory ``/var/lib/postroj/archive/hello-world.img/rootfs`` doesn't look like it has an OS root directory. Refusing.
- ``unshare --fork --pid --mount-proc --root=/var/lib/postroj/images/hello-world ./hello``

- [o] Use/integrate with ``mkosi``.

- https://github.com/systemd/mkosi
- http://0pointer.net/blog/mkosi-a-tool-for-generating-os-images.html
- https://lwn.net/Articles/726655/
- https://github.com/asiffer/netspot/search?q=mkosi

- [o] Improve timeout behaviors across the board,
using ``with stopit.ThreadingTimeout(timeout) as to_ctx_mgr``.
- [o] Tests: ``racker pull foo``. Also remove ``/var/lib/postroj/archive/foo`` again.
Expand All @@ -71,6 +63,9 @@ Iteration 3
- [o] ``racker --verbose run -it --rm https://cloud-images.ubuntu.com/minimal/daily/focal/current/focal-minimal-cloudimg-amd64-root.tar.xz /bin/bash``
- [o] TODO: Introduce appropriate exception classes.
- [o] Maybe use ``ubi8/ubi-init`` instead of ``ubi8/ubi``?
- [o] How to integrate with other tools from https://github.com/topics/systemd-nspawn
- [o] https://github.com/NixOS/nixops



***********
Expand Down
81 changes: 81 additions & 0 deletions doc/bugs.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,84 @@ See::
stdout/stderr redirection
=========================
Most probably, since c5266940/a721345, the progress bar of ``skopeo`` isn't displayed anymore.


VirtualBox on macOS with nested virtualization
==============================================
::

==> 2019-box: Booting VM...
There was an error while executing `VBoxManage`, a CLI used by Vagrant
for controlling VirtualBox. The command and stderr is shown below.

Command: ["startvm", "5f422042-d72f-46c5-9bcf-ec27eeee2e06", "--type", "headless"]

Stderr: VBoxManage: error: Cannot enable nested VT-x/AMD-V without nested-paging and unrestricted guest execution!
VBoxManage: error: (VERR_CPUM_INVALID_HWVIRT_CONFIG)
VBoxManage: error: Details: code NS_ERROR_FAILURE (0x80004005), component ConsoleWrap, interface IConsole
2022-06-07 15:58:59,573 [racker.cli ] CRITICAL: Launching container failed. Command 'vagrant up --provider=virtualbox 2019-box' returned non-zero exit status 1.



Using the CrateDB RPM package on openSUSE
=========================================

| A: Workaround applied by running ``rpm --install --nodeps``.
| TODO: Maybe adjust the RPM dependencies?
::

zypper --non-interactive install crate-4.7.2-1.x86_64.rpm
Loading repository data...
Reading installed packages...
Resolving package dependencies...

Problem: nothing provides 'systemd-units' needed by the to be installed crate-4.7.2-1.x86_64
Solution 1: do not install crate-4.7.2-1.x86_64
Solution 2: break crate-4.7.2-1.x86_64 by ignoring some of its dependencies

Choose from above solutions by number or cancel [1/2/c/d/?] (c): c

::

rpm -i crate-4.7.2-1.x86_64.rpm
warning: crate-4.7.2-1.x86_64.rpm: Header V4 RSA/SHA1 Signature, key ID 06f6eaeb: NOKEY
error: Failed dependencies:
shadow-utils is needed by crate-4.7.2-1.x86_64
systemd-units is needed by crate-4.7.2-1.x86_64


Spawning a container from the CrateDB Docker image
==================================================

Currently, as of 2022-06, the CrateDB Docker image is based on CentOS 7, which
has a systemd installation that is too old for being able to invoke nested
systemd environments.

::

racker run -it --rm crate/crate:nightly bash

::

Failed to start transient service unit: Cannot set property AddRef, or unknown property.

=> systemd too old.


Docker context on Windows VM not reachable
==========================================

https://github.com/docker/machine/issues/531

::

$ docker --context=2019-box ps
error during connect: Get "https://192.168.59.90:2376/v1.24/containers/json": x509: certificate is valid for 169.254.232.221, 172.30.112.1, 10.0.2.15, 127.0.0.1, not 192.168.59.90

$ docker --context=2019-box ps
error during connect: Get "https://192.168.59.90:2376/v1.24/containers/json": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "Docker TLS Root")

Solution::

docker context rm 2019-box
4 changes: 3 additions & 1 deletion doc/comparison.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,8 @@ invoking `systemd-nspawn`_ without using the ``--boot`` option.
Notes
*****

There is also `Boombox`_, `CoreOS toolbox`_, `microos-toolbox`_ and `tlbx`_.
There is also `Boombox`_, `CoreOS toolbox`_, `docker-coreos-toolbox-ubuntu`_,
`microos-toolbox`_ and `tlbx`_.


.. _Boombox: https://github.com/anthr76/boombox
Expand All @@ -72,6 +73,7 @@ There is also `Boombox`_, `CoreOS toolbox`_, `microos-toolbox`_ and `tlbx`_.
.. _Distrobox: https://github.com/89luca89/distrobox
.. _Distrobox - Containers distribution support: https://distrobox.privatedns.org/compatibility.html#containers-distros
.. _distrobox-init: https://distrobox.privatedns.org/usage/distrobox-init.html
.. _docker-coreos-toolbox-ubuntu: https://github.com/wallneradam/docker-coreos-toolbox-ubuntu
.. _microos-toolbox: https://github.com/openSUSE/microos-toolbox
.. _systemd-nspawn: https://www.freedesktop.org/software/systemd/man/systemd-nspawn.html
.. _tlbx: https://gitlab.com/uppercat/tlbx
Expand Down
69 changes: 0 additions & 69 deletions doc/cratedb.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -40,72 +40,3 @@ Test package on Fedora 36::
HTTP/1.1 503 Service Unavailable

Improve!


**************
postroj invoke
**************

Purpose: Invoke programs within a Windows/Java/OpenJDK environment.

::

# Basic usage.
postroj invoke \
--system=windows-1809 --cpus=8 --memory=8192M \
--mount type=git,src=https://github.com/crate/crate,dst=C:/src \
-- \
cmd /C "cd src && gradlew :server:test -Dtests.crate.run-windows-incompatible=false"

# Advanced usage.
postroj invoke \
--environment=windows-1809 --cpus=8 --memory=8192M \
--repository=https://github.com/crate/crate \
--command="gradlew :server:test -Dtests.crate.run-windows-incompatible=false"


********
Failures
********


Using the CrateDB RPM package on openSUSE
=========================================

| A: Workaround applied by running ``rpm --install --nodeps``.
| TODO: Maybe adjust the RPM dependencies?
::

zypper --non-interactive install crate-4.7.2-1.x86_64.rpm
Loading repository data...
Reading installed packages...
Resolving package dependencies...

Problem: nothing provides 'systemd-units' needed by the to be installed crate-4.7.2-1.x86_64
Solution 1: do not install crate-4.7.2-1.x86_64
Solution 2: break crate-4.7.2-1.x86_64 by ignoring some of its dependencies

Choose from above solutions by number or cancel [1/2/c/d/?] (c): c

::

rpm -i crate-4.7.2-1.x86_64.rpm
warning: crate-4.7.2-1.x86_64.rpm: Header V4 RSA/SHA1 Signature, key ID 06f6eaeb: NOKEY
error: Failed dependencies:
shadow-utils is needed by crate-4.7.2-1.x86_64
systemd-units is needed by crate-4.7.2-1.x86_64


Spawning a container from the CrateDB Docker image
==================================================

::

racker run -it --rm crate/crate:nightly bash

::

Failed to start transient service unit: Cannot set property AddRef, or unknown property.

=> systemd too old.
Loading

0 comments on commit 9a7b0ff

Please sign in to comment.