Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: standardize and optimize Check workflow #749

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

ci: standardize and optimize Check workflow #749

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
dafaaea
ci: standardize output redirection formatting
trueNAHO Jan 5, 2025
3d37c54
ci: optimize CI time by running only relevant checks
trueNAHO Jan 5, 2025
f99342b
ci: avoid downloading the repository twice
trueNAHO Jan 5, 2025
16b69eb
ci: consistently install repository with actions/checkout GitHub Action
trueNAHO Jan 5, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
128 changes: 91 additions & 37 deletions .github/workflows/check.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,43 +12,100 @@ jobs:
runs-on: ubuntu-24.04

steps:
- uses: actions/checkout@v4
with:
fetch-depth: ${{ github.event_name == 'pull_request' && 2 || 0 }}
danth marked this conversation as resolved.
Show resolved Hide resolved

danth marked this conversation as resolved.
Show resolved Hide resolved
- uses: DeterminateSystems/nix-installer-action@v16
- uses: DeterminateSystems/magic-nix-cache-action@v8

- id: get-derivations
run: |
nix flake show --json \
github:${{
github.repository
}}/${{
github.event.pull_request.head.sha || github.sha
}} |
jq --raw-output '
def format_output($arch; $type):
{
arch: $arch,
key: .,

os: (
if $arch == "x86_64-linux" then
"ubuntu-24.04"
else
"macos-14"
end
),

type: $type
};

[
["x86_64-linux", "x86_64-darwin"][] as $arch |
(.checks[$arch] | keys) as $checks |
(.packages[$arch] | keys) as $packages |
(($checks - $packages)[] | format_output($arch; "checks")),
($packages[] | format_output($arch; "packages"))
] |
"derivations=\(.)"
' >> $GITHUB_OUTPUT
nix flake show --json |
jq \
--argjson changed_files "$(
if ${{ github.event_name == 'pull_request' }}; then
git diff --name-only HEAD~1 HEAD
else
git diff --name-only \
${{ github.event.before }} ${{ github.event.after }}
fi |
jq --raw-input --raw-output --slurp 'split("\n")[:-1]'
)" \
--raw-output \
'
def format_output($arch; $type):
{
arch: $arch,
key: .,

os: (
if $arch == "x86_64-linux" then
"ubuntu-24.04"
else
"macos-14"
end
),

type: $type
};

[
["x86_64-linux", "x86_64-darwin"][] as $arch |
(.checks[$arch] | keys) as $checks |
(.packages[$arch] | keys) as $packages |
(($checks - $packages)[] | format_output($arch; "checks")),
($packages[] | format_output($arch; "packages"))
] as $derivations |

# Keep everything when touching critical files.
if (
$changed_files |
any(
. == "flake.lock" or
(. | startswith("stylix/")) or
(. | startswith(".github/workflows/"))
)
) then
$derivations

else
$derivations |
map(
select(
if .key == "nix-flake-check" then
($changed_files | any(. == "flake.nix"))

elif .key == "docs" then
($changed_files | any(startswith("docs/")))

elif .key == "palette-generator" then
($changed_files | any(startswith("palette-generator/")))

elif (.key | test("^testbed-[^-]+-")) then
(
.key | capture("^testbed-(?<module>[^-]+)") | .module
) as $module |
(
$changed_files |
any(startswith("modules/\($module)/"))
)

# Always keep the git-hooks derivation.
elif .key == "git-hooks" then
true

else
error(
"Derivation must be handled or explicitly ignored: " +
.key
)
Comment on lines +76 to +102
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This increases the maintenance burden compared to the previous workflow, as this will need to be updated whenever files are renamed or new outputs are introduced.

Copy link
Collaborator Author

@trueNAHO trueNAHO Jan 6, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The last branch will trigger an error when something unexpected happens. When a new derivation has to be ignored, the second-last branch can be extended:

# Always keep these derivations.
elif .key == "git-hooks" or .key == "<NEW_DERIVATION>" then
  true

However, this probably happens very rarely and is a very quick fix.

end
)
)
end |
"derivations=\(.)"
' >>$GITHUB_OUTPUT

outputs:
derivations: ${{ steps.get-derivations.outputs.derivations }}
Expand All @@ -65,16 +122,13 @@ jobs:
check: ${{ fromJSON(needs.get-derivations.outputs.derivations) }}

steps:
- uses: actions/checkout@v4
- uses: DeterminateSystems/nix-installer-action@v16
- uses: DeterminateSystems/magic-nix-cache-action@v8

- run: |
nix build --no-update-lock-file --print-build-logs \
github:${{
github.repository
}}/${{
github.event.pull_request.head.sha || github.sha
}}#${{
.#${{
matrix.check.type
}}.${{
matrix.check.arch
Expand Down