Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mention Openstack-S3, Harbor, PR vs nightly etc builds. #963

Merged
merged 1 commit into from
Jan 26, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 19 additions & 2 deletions for_developers/continuous_integration-testing/githubactions.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,21 @@ The ROOT project uses a GitHub Actions for continuous integration and delivery.

For building the Docker images we use GitHub's runners.
For everything else, [runners are hosted at CERN](https://github.com/root-project/root/settings/actions/runners).
Linux runners are Docker images; macOS and Windows are native (mac minis / OpenStack VMs).

## Nightly builds, on-merge builds, PR builds, building releases

All builds share the same infrastructure.
Nighly builds are triggered (`schedule:`); this only works for `master` and needs a wrapper (`.github/workflows/all-branches.yml`) to then trigger nightly builds for the relevant branches (to be implemented).
PR builds are incremental builds that grab the last on-merge build's build artifacts (source + build dir), apply the PR's source changes, build and test.

On-merge builds are full builds that update the build artifacts: before running roottest, the source and build directories are uploaded to [OpenStack's S3 storage](https://openstack.cern.ch/project/containers/container/ROOT-build-artifacts) (project `ROOT-build-artifacts`).
These build artifacts use the hash of the cmake invocation as key: if the on-merge build and the PR builds are using different CMake invocations, the PR build will be a full build, not an incremental one, because no build artifacts can be found for its CMake configuration.

Release builds can be [triggered by hand](https://github.com/root-project/root/actions/workflows/root-ci.yml) ("Run workflow").
They produce binary releases that get linked at the bottom of the Action's build page; GitHub stores them for 90 days.
Due to a [limitation in the GitHub artifact upload](https://github.com/actions/upload-artifact#zip-archives), each job can only upload a single file; multiple files will get combined and compressed.
This means that e.g. macOS artifacts (producing `.tar.gz` and `.pkg` binaries in one go) will need to be decompressed before uploading to `root.cern`.

## Registering a machine as a GitHub Actions Runner:

Expand Down Expand Up @@ -51,8 +65,8 @@ This section documents the configuration of these Docker containers and of their

Python packages or distro package dependencies are added not to runners but to the Docker images they run, see <https://github.com/root-project/root-ci-images>.
They can use a python3-venv / python3-virtualenv if set up at `/py-venv/ROOT-CI`.
We use podman, not docker; this and a bug in GH runners requires us to use a
[podman-docker-wrapper](https://gitlab.cern.ch/ai/it-puppet-hostgroup-lcgapp/-/blob/master/code/files/github_ci/wrapper.py?ref_type=heads) script.

These images are built on a daily basis; updated images are uploaded to CERN's [Harbor](https://registry.cern.ch/harbor/projects/3531/repositories) registry.

### Linux runners

Expand All @@ -67,6 +81,9 @@ documentation for configuration can be found on [ConfigDocs](https://configdocs.
The Puppet configuration of the runners are stored on GitLab at:
<https://gitlab.cern.ch/ai/it-puppet-hostgroup-lcgapp/>

We use podman, not docker; this and a bug in GH runners requires us to use a
[podman-docker-wrapper](https://gitlab.cern.ch/ai/it-puppet-hostgroup-lcgapp/-/blob/master/code/files/github_ci/wrapper.py?ref_type=heads) script.

Although using management tools is annoying (even awful), they simplify batch
operations and ensure uniformity. Here are some examples of common tasks to show
the strengths of Foreman/Puppet.
Expand Down
Loading