Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add http-network-relay dep and switch to our poetry2nix #265

Merged
merged 41 commits into from
Feb 5, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
3d4b0a7
add http-network-relay dep and switch to our poetry2nix
elikoga Jan 27, 2025
58bdb64
feat(nix): add http-network-relay dependency with poetry override
elikoga Jan 27, 2025
14cc212
refactor(nix): remove http-network-relay dependency from default.nix
elikoga Jan 27, 2025
fde40aa
it's alive!
elikoga Jan 28, 2025
8b4baf4
update relay
elikoga Jan 28, 2025
37ead57
refactor(network_relay): remove commented-out device notification logic
elikoga Jan 29, 2025
fd82c37
add vm starting feature
elikoga Jan 30, 2025
563d0c0
make it work
elikoga Jan 31, 2025
04f58c9
refactor(log_parse): improve log level handling and separate other me…
elikoga Feb 1, 2025
6e79a76
refactor(task): simplify SSHCommandTaskSubmission fields by removing …
elikoga Feb 1, 2025
0033293
add aarch64 build test
elikoga Feb 1, 2025
d594f55
fix(task): improve error logging for failed tasks in TaskWorkerPoolMa…
elikoga Feb 1, 2025
6ca6537
feat(device): add function to retrieve allowed image formats based on…
elikoga Feb 1, 2025
ca8b06f
refactor(dependencies): remove unnecessary logging from get_db_sessio…
elikoga Feb 1, 2025
2648f25
test: mark device_notify test as expected to fail
elikoga Feb 1, 2025
950e384
update screenshots
elikoga Feb 1, 2025
261cf96
Update e2e snapshots
thymis-github-app[bot] Feb 1, 2025
02caa2c
fix(task): conditionally include DBUS_SESSION_BUS_ADDRESS in environm…
elikoga Feb 1, 2025
e5ac1db
fix(task): dynamically generate unit name for nix-copy-closure command
elikoga Feb 1, 2025
878d4bc
fix(task): add PYTHONENV variable to shell application and update Pro…
elikoga Feb 1, 2025
0e4b0c4
fix(task): update task completion logic to check for child tasks befo…
elikoga Feb 1, 2025
d0edc51
fix(image): add fsck.vfat check and update FAT partition copying logic
elikoga Feb 2, 2025
f21439d
fix(api): update download_image to correctly set filename based on im…
elikoga Feb 2, 2025
ffcd2ac
fix(nixos-module): add time server to networking options for Thymis d…
elikoga Feb 2, 2025
2e2c85b
fix(kiosk): update nonce generation to use SHA-256 and add dry activa…
elikoga Feb 2, 2025
f5caf53
feat(notification): implement notification system with toast and inva…
elikoga Feb 2, 2025
9fd3c2a
fix(workflow): rename artifact upload identifiers for clarity and con…
elikoga Feb 3, 2025
bbd3037
fix(task): handle OSError when sending message to task and log error
elikoga Feb 3, 2025
ce537ca
refactor(dependencies): remove unnecessary logging in get_network_rel…
elikoga Feb 3, 2025
5bef9df
update screenshots to expected by hand
elikoga Feb 3, 2025
2f7a789
fix(workflow): update artifact upload ID and name in test workflow
elikoga Feb 3, 2025
188c082
fix: default agent controller url overriding tag configuration
MSchmoecker Feb 4, 2025
0a04533
fix: vnc connection over network relay
MSchmoecker Feb 4, 2025
67feee5
fix(migration): update downgrade function to use batch operation for …
elikoga Feb 4, 2025
85bcbe5
feat: add agent access URL configuration and improve task submission …
elikoga Feb 5, 2025
a017d52
feat: add agent access URL configuration to Thymis controller and ref…
elikoga Feb 5, 2025
1ee0f11
fix screenshot
elikoga Feb 5, 2025
5b5a605
fix: terminal connection edge cases
MSchmoecker Feb 5, 2025
78c9b14
chore: cleanup
MSchmoecker Feb 5, 2025
c5f3b86
fix: german download text
MSchmoecker Feb 5, 2025
226ff64
Update e2e snapshots
thymis-github-app[bot] Feb 5, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions .github/npm_test_integration.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
#!/usr/bin/env bash
OUR_NIX=$(readlink -f $(which nix))
OUR_NIX_PARENT=$(dirname $OUR_NIX)
TEE=$(which tee)
OLD_PATH=$PATH
export PATH=$OUR_NIX_PARENT
$OUR_NIX develop .#forNpmTesting --command npm run test:integration "$@" 2>&1 | $TEE output.log
export PATH=$OLD_PATH
PLAYWRIGHT_EXIT_CODE=$?
if grep -q -e "Error: A snapshot doesn't exist at" -e "Screenshot comparison failed" output.log; then
echo "Playwright tests failed due to a snapshot issue"
echo "SNAPSHOT_DIFFERENCES=true" >> $GITHUB_ENV
fi
if grep -q -E -e "npx playwright install" -e "error: attribute '\"[[:digit:]]+\.[[:digit:]]+\.[[:digit:]]+\"' missing" output.log; then
echo "Playwright tests failed due to missing browsers"
echo "MISSING_BROWSERS=true" >> $GITHUB_ENV
fi
if grep -q -E -e "[[:digit:]]+ failed" -e "was not able to start" output.log; then
echo "Playwright tests failed"
exit 1
fi
exit $PLAYWRIGHT_EXIT_CODE
2 changes: 1 addition & 1 deletion .github/workflows/approve-playwright-snapshots.yml
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ jobs:
with:
message: |
### 🎉 Successfully updated and committed Playwright snapshots! 🎉
comment-tag: playwright-snapshots-update-success
comment-tag: playwright-snapshots-update-success
mode: recreate
github-token: ${{ steps.generate-token.outputs.token }}

Expand Down
16 changes: 16 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,22 @@ jobs:
script: |
nix build .#thymis-agent --print-build-logs

build-agent-aarch64:
runs-on: ubuntu-22.04-arm
steps:
- uses: actions/checkout@v4
- uses: ./.github/actions/setup-nix
with:
attic_token: ${{ secrets.ATTIC_TOKEN }}
github_token: ${{ secrets.GITHUB_TOKEN }}
- name: build thymis-agent-aarch64
uses: ./.github/actions/run-command-with-nix-cache-upload
with:
attic_token: ${{ secrets.ATTIC_TOKEN }}
github_token: ${{ secrets.GITHUB_TOKEN }}
script: |
nix build .#thymis-agent --print-build-logs

build-thymis-controller:
runs-on: ubuntu-latest
steps:
Expand Down
152 changes: 108 additions & 44 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -90,28 +90,7 @@ jobs:
script: |
set +e # revert the default `set -e`
export THYMIS_FLAKE_ROOT='..'
cd frontend
OUR_NIX=$(readlink -f $(which nix))
OUR_NIX_PARENT=$(dirname $OUR_NIX)
TEE=$(which tee)
OLD_PATH=$PATH
export PATH=$OUR_NIX_PARENT
$OUR_NIX develop .#forNpmTesting --command npm run test 2>&1 | $TEE output.log
export PATH=$OLD_PATH
PLAYWRIGHT_EXIT_CODE=$?
if grep -q -e "Error: A snapshot doesn't exist at" -e "Screenshot comparison failed" output.log; then
echo "Playwright tests failed due to a snapshot issue"
echo "SNAPSHOT_DIFFERENCES=true" >> $GITHUB_ENV
fi
if grep -q -E -e "npx playwright install" -e "error: attribute '\"[[:digit:]]+\.[[:digit:]]+\.[[:digit:]]+\"' missing" output.log; then
echo "Playwright tests failed due to missing browsers"
echo "MISSING_BROWSERS=true" >> $GITHUB_ENV
fi
if grep -q -E -e "[[:digit:]]+ failed" -e "was not able to start" output.log; then
echo "Playwright tests failed"
exit 1
fi
exit $PLAYWRIGHT_EXIT_CODE
../.github/npm_test_integration.sh
working-directory: frontend
- uses: actions/upload-artifact@v4
id: artifact-upload
Expand Down Expand Up @@ -209,28 +188,7 @@ jobs:
script: |
set +e # revert the default `set -e`
# export THYMIS_FLAKE_ROOT='..' # stable input needs THYMIS_FLAKE_ROOT to be unset
cd frontend
OUR_NIX=$(readlink -f $(which nix))
OUR_NIX_PARENT=$(dirname $OUR_NIX)
TEE=$(which tee)
OLD_PATH=$PATH
export PATH=$OUR_NIX_PARENT
$OUR_NIX develop .#forNpmTesting --command npm run test 2>&1 | $TEE output.log
export PATH=$OLD_PATH
PLAYWRIGHT_EXIT_CODE=$?
if grep -q -e "Error: A snapshot doesn't exist at" -e "Screenshot comparison failed" output.log; then
echo "Playwright tests failed due to a snapshot issue"
echo "SNAPSHOT_DIFFERENCES=true" >> $GITHUB_ENV
fi
if grep -q -E -e "npx playwright install" -e "error: attribute '\"[[:digit:]]+\.[[:digit:]]+\.[[:digit:]]+\"' missing" output.log; then
echo "Playwright tests failed due to missing browsers"
echo "MISSING_BROWSERS=true" >> $GITHUB_ENV
fi
if grep -q -E -e "[[:digit:]]+ failed" -e "was not able to start" output.log; then
echo "Playwright tests failed"
exit 1
fi
exit $PLAYWRIGHT_EXIT_CODE
../.github/npm_test_integration.sh
working-directory: frontend
- uses: actions/upload-artifact@v4
id: artifact-upload
Expand All @@ -250,6 +208,112 @@ jobs:
comment-tag: playwright-snapshots
mode: recreate

test-frontend-integration-x64:
runs-on: ubuntu-latest
needs: changes
if: ${{ ! (needs.changes.outputs.package-lock-json == 'true' && github.actor == 'renovate[bot]' && github.event_name == 'pull_request') }}
steps:
- uses: actions/checkout@v4
- uses: ./.github/actions/setup-nix
with:
attic_token: ${{ secrets.ATTIC_TOKEN }}
github_token: ${{ secrets.GITHUB_TOKEN }}
- name: Install Node.js
uses: actions/setup-node@v4
with:
node-version: "20"
- name: Install dependencies
run: |
npm ci
working-directory: frontend
- name: Build application
uses: ./.github/actions/run-command-with-nix-cache-upload
with:
attic_token: ${{ secrets.ATTIC_TOKEN }}
github_token: ${{ secrets.GITHUB_TOKEN }}
script: |
nix build .#thymis-controller --print-build-logs
- name: Run tests
uses: ./.github/actions/run-command-with-nix-cache-upload
with:
attic_token: ${{ secrets.ATTIC_TOKEN }}
github_token: ${{ secrets.GITHUB_TOKEN }}
script: |
set +e # revert the default `set -e`
export THYMIS_FLAKE_ROOT='..'
../.github/npm_test_integration.sh -- tests/x86_vm.spec.ts
working-directory: frontend
- uses: actions/upload-artifact@v4
id: artifact-upload
if: always()
with:
name: playwright-report-x64
path: frontend/playwright-report/
retention-days: 30
- name: Comment on PR with report link
uses: thollander/actions-comment-pull-request@v3
if: ${{ always() && github.event_name == 'pull_request' && env.SNAPSHOT_DIFFERENCES == 'true' }}
with:
message: |
### Playwright visual snapshot differences were detected.
View the [Playwright report](${{ steps.artifact-upload.outputs.artifact-url }}) to review the visual differences.
**To approve the snapshot changes and update the snapshots, please comment:** /approve-snapshots
comment-tag: playwright-snapshots
mode: recreate

test-frontend-integration-stable-input-x64:
runs-on: ubuntu-latest
needs: changes
if: ${{ ! (needs.changes.outputs.package-lock-json == 'true' && github.actor == 'renovate[bot]' && github.event_name == 'pull_request') }}
steps:
- uses: actions/checkout@v4
- uses: ./.github/actions/setup-nix
with:
attic_token: ${{ secrets.ATTIC_TOKEN }}
github_token: ${{ secrets.GITHUB_TOKEN }}
- name: Install Node.js
uses: actions/setup-node@v4
with:
node-version: "20"
- name: Install dependencies
run: |
npm ci
working-directory: frontend
- name: Build application
uses: ./.github/actions/run-command-with-nix-cache-upload
with:
attic_token: ${{ secrets.ATTIC_TOKEN }}
github_token: ${{ secrets.GITHUB_TOKEN }}
script: |
nix build .#thymis-controller --print-build-logs
- name: Run tests
uses: ./.github/actions/run-command-with-nix-cache-upload
with:
attic_token: ${{ secrets.ATTIC_TOKEN }}
github_token: ${{ secrets.GITHUB_TOKEN }}
script: |
set +e # revert the default `set -e`
# export THYMIS_FLAKE_ROOT='..' # stable input needs THYMIS_FLAKE_ROOT to be unset
../.github/npm_test_integration.sh -- tests/x86_vm.spec.ts
working-directory: frontend
- uses: actions/upload-artifact@v4
id: artifact-upload
if: always()
with:
name: playwright-report-stable-input-x64
path: frontend/playwright-report/
retention-days: 30
- name: Comment on PR with report link
uses: thollander/actions-comment-pull-request@v3
if: ${{ always() && github.event_name == 'pull_request' && env.SNAPSHOT_DIFFERENCES == 'true' }}
with:
message: |
### Playwright visual snapshot differences were detected.
View the [Playwright report](${{ steps.artifact-upload.outputs.artifact-url }}) to review the visual differences.
**To approve the snapshot changes and update the snapshots, please comment:** /approve-snapshots
comment-tag: playwright-snapshots
mode: recreate

test-pre-commit:
runs-on: ubuntu-latest
steps:
Expand Down
5 changes: 5 additions & 0 deletions agent/default.nix
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,11 @@ let
preferWheels = true;
checkGroups = [ "test" ];
python = python313;
overrides = poetry2nix.overrides.withDefaults (final: prev: {
http-network-relay = prev.http-network-relay.overridePythonAttrs (oldAttrs: {
buildInputs = (oldAttrs.buildInputs or [ ]) ++ [ prev.poetry-core ];
});
});
}).dependencyEnv;
in
writeShellApplication {
Expand Down
Loading
Loading