Bootstrap without docker #109
Conversation
| wget https://cloud.centos.org/centos/9-stream/${arch}/images/CentOS-Stream-Container-Base-9-20230501.0.${arch}.tar.xz \ | ||
| --output-document base.tar.xz | ||
|
|
||
| echo "$cksum base.tar.xz" | sha256sum -c |
There was a problem hiding this comment.
I found this construct to be more straightforward:
filename=base-${arch}.tar.xz
wget https://cloud.centos.org/centos/9-stream/${arch}/images/CentOS-Stream-Container-Base-9-20230501.0.${arch}.tar.xz \
--output-document ${filename}
grep $filename <<-SUMS | sha256sum -cw
63b7ddb444b23a07cb851398c338595e410fb3fac2dd72061d0292c653e5afe6 *base-x86_64.tar.xz
312a833dfe646ce5b41f362cae577df9797955b85ced96173be8e88e5ebd5990 *base-arm64.tar.xz
SUMSIt ends up being easier to extend since you don't need to remap the arch names for every arch, so adding new architectures is typically about adding a new line in the sum document
| arch=$(uname -m) | ||
| case "$arch" in | ||
| x86_64) | ||
| arch=x86_64 |
| internal/bootstrap/extract/1: | ||
| no-create-mountpoints: true | ||
| units: | ||
| - floor: .%internal/bootstrap/iso-extraction-floor | ||
| sources: [] | ||
| mappings: | ||
| /src/base: .%internal/bootstrap/base.tar.xz | ||
| build: | | ||
| tar --strip-components=1 -xf /src/base/base.tar.xz -C /dest | ||
|
|
||
| internal/bootstrap/extract/2: | ||
| description: | | ||
| Extract our bootstrapping environment and remove any pre-configured | ||
| yum repos. This bootstrapping environment will be centos 9 stream, | ||
| but because we will install el9 repos under /etc/yum.repos.d, the | ||
| environments that we boostrap will be el9. | ||
| no-create-mountpoints: true | ||
| units: | ||
| - floor: .%internal/bootstrap/iso-extraction-floor | ||
| sources: [] | ||
| mappings: | ||
| /src/layer: .%internal/bootstrap/extract/1 | ||
| build: | | ||
| tar -xf /src/layer/layer.tar -C /dest | ||
| rm /dest/etc/yum.repos.d/* |
There was a problem hiding this comment.
I don't see much value in separating these steps from the main download step. I guess the argument is that we don't need to redownload the tarball should any of the commands responsible for extracting and preparing that bootstrap image change, but redownloading a file is generally pretty fast for this to not really matter. This just ends up taking thrice the storage since we're individually storing the tarball, the layer tarball, and the extracted layer separately.
|
|
||
| internal/bootstrap/network: | ||
| entry: | ||
| share-net: true |
There was a problem hiding this comment.
not part of this change but this is definitely no longer accurate and should be using network.enabled: true
| yum repos. This bootstrapping environment will be centos 9 stream, | ||
| but because we will install el9 repos under /etc/yum.repos.d, the | ||
| environments that we boostrap will be el9. | ||
| no-create-mountpoints: true |
There was a problem hiding this comment.
You probably do want to ensure all of the mountpoints are created, because this image is getting entry settings that will attempt to mount /tmp, /dev, ... and while I'm sure the base centos image contains these, it doesn't hurt to assert that they're created
| Downloading a recent-ish centos container base from the upstream | ||
| centos registry. Note that we cache this step separately for quick | ||
| development. |
There was a problem hiding this comment.
Maybe do mention that this is used for bootstrapping alma linux, and just has to be a rpm-based system. I would also mention that the image should seldom change since it would invalidate all of the eext snapshots.
There was a problem hiding this comment.
Why would this image changing invalidate all eext-snapshots ?
It'd invalidate the eext base-image snapshot since the floor changed, but wouldn't the base-image snapshot be generated with the same content hash ?
There was a problem hiding this comment.
The snapshot hash isn't a content-hash --- it's actually a hash of all the inputs to the build
Since the definition of internal/bootstrap/base.tar.xz is an input to the next build, just changing the build script (and the version of alman that we fetch) is sufficient to change the hash of all downstream consumers
|
Sorry, I totally missed this PR. |
Ok, I understand why the alpine dependency is not a problem as it's only used in the iso-extraction-floor, and the bootstrap image's hash won't depend on it. However, I'm still confused about golang dependency. Is it minor go version updates that we roll in to eos-trunk ? Will this change the eext binary's hash which is built with the go generator ? Also, I have built on top of Andrew's commit here: #116 |
|
I've already pulled in Andrew's changes here: #116 |
With this change we bootstrap the workspace creation environment using a centos rootfs rather than pulling down a docker container. This has the following benefits:
It does add a bit of complexity that we start with a centos container to bootstrap an alma container. However I don't think this is really an issue as the content that we are installing comes from our internal alma repositories. The only thing we are using centos for is to run dnf and resolve/install dependencies into our alma images.