[devtools] Enable the framework upgrade test again #15763

perryjrandall · 2025-01-16T23:21:23Z

Description

Removes a temporary fix that was disabling the framework upgrade test. This change re-enables the test by removing the code that was forcing relevant_changes_detected to false.

How Has This Been Tested?

The framework upgrade test will now run normally when relevant changes are detected, validating the intended behavior.

Key Areas to Review

Verify that removing this temporary fix doesn't reintroduce the original issues that led to disabling the test
Confirm that the framework upgrade test executes correctly when changes are detected

Type of Change

Bug fix
Tests

Which Components or Systems Does This Change Impact?

Developer Infrastructure

Checklist

I have read and followed the CONTRIBUTING doc
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I identified and added all stakeholders and component owners affected by this change as reviewers
I tested both happy and unhappy path of the functionality
I have made corresponding changes to the documentation

trunk-io · 2025-01-16T23:21:26Z

⏱️ 2h 47m total CI duration on this PR

Slowest 15 Jobs	Cumulative Duration	Recent Runs
forge-e2e-test / forge	33m	🟥
rust-smoke-tests	23m	⬜
test-target-determinator	19m	🟩 🟩 🟩 🟩
execution-performance / test-target-determinator	19m	🟩 🟩 🟩 🟩
check-dynamic-deps	15m	🟩 🟩 🟩 🟩 🟩 (+1 more)
forge-framework-upgrade-test / forge	14m	🟩
rust-cargo-deny	8m	⬜ 🟩 🟩 🟩 🟩
fetch-last-released-docker-image-tag	7m	🟩 🟩 🟩 🟩
rust-doc-tests	5m	🟩
rust-doc-tests	5m	🟩
rust-doc-tests	5m	🟩
rust-doc-tests	5m	🟩
semgrep/ci	2m	🟩 🟩 🟩 🟩 🟩 (+1 more)
general-lints	2m	🟩 🟩 🟩 🟩 🟩
file_change_determinator	55s	🟩 🟩 🟩 🟩 🟩

🚨 1 job on the last run was significantly faster/slower than expected

Job	Duration	vs 7d avg	Delta
execution-performance / single-node-performance	10s	16m

_{settings ⋅ feedback ⋅ docs ⋅ learn more about trunk.io}

perryjrandall · 2025-01-16T23:21:42Z

[devtools] Enable the framework upgrade test again #15763 👈 (View in Graphite)
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Previously we disabled this because it was broken on PR Now that it is fixed on PR enable again to make sure it doesnt break! Test Plan: add label, should be triggered on PR

github-actions · 2025-01-24T19:09:34Z

✅ Forge suite `realistic_env_max_load` success on `f6b53eb4ab226e435a626e6b0c1da0618681017a`

two traffics test: inner traffic : committed: 14669.12 txn/s, latency: 2701.84 ms, (p50: 2700 ms, p70: 2700, p90: 3000 ms, p99: 3300 ms), latency samples: 5577620
two traffics test : committed: 99.96 txn/s, latency: 1422.43 ms, (p50: 1400 ms, p70: 1400, p90: 1600 ms, p99: 2900 ms), latency samples: 1820
Latency breakdown for phase 0: ["MempoolToBlockCreation: max: 1.546, avg: 1.443", "ConsensusProposalToOrdered: max: 0.302, avg: 0.291", "ConsensusOrderedToCommit: max: 0.413, avg: 0.400", "ConsensusProposalToCommit: max: 0.705, avg: 0.691"]
Max non-epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 1.24s no progress at version 19226 (avg 0.20s) [limit 15].
Max epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 0.66s no progress at version 2390590 (avg 0.66s) [limit 16].
Test Ok

github-actions · 2025-01-24T19:09:46Z

✅ Forge suite `compat` success on `d15fc969c89551a1461d931d327b8d4dbfb2f814` ==> `f6b53eb4ab226e435a626e6b0c1da0618681017a`

Compatibility test results for d15fc969c89551a1461d931d327b8d4dbfb2f814 ==> f6b53eb4ab226e435a626e6b0c1da0618681017a (PR)
1. Check liveness of validators at old version: d15fc969c89551a1461d931d327b8d4dbfb2f814
compatibility::simple-validator-upgrade::liveness-check : committed: 9794.67 txn/s, latency: 3166.36 ms, (p50: 2800 ms, p70: 3200, p90: 4900 ms, p99: 8700 ms), latency samples: 351380
2. Upgrading first Validator to new version: f6b53eb4ab226e435a626e6b0c1da0618681017a
compatibility::simple-validator-upgrade::single-validator-upgrading : committed: 3630.71 txn/s, latency: 7566.81 ms, (p50: 8400 ms, p70: 10600, p90: 11100 ms, p99: 11200 ms), latency samples: 71320
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 3440.80 txn/s, latency: 10040.01 ms, (p50: 10600 ms, p70: 11500, p90: 11800 ms, p99: 12000 ms), latency samples: 127200
3. Upgrading rest of first batch to new version: f6b53eb4ab226e435a626e6b0c1da0618681017a
compatibility::simple-validator-upgrade::half-validator-upgrading : committed: 4140.38 txn/s, latency: 7311.47 ms, (p50: 8100 ms, p70: 8600, p90: 8900 ms, p99: 9000 ms), latency samples: 92160
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 4194.20 txn/s, latency: 8116.06 ms, (p50: 9100 ms, p70: 9300, p90: 9500 ms, p99: 9600 ms), latency samples: 147320
4. upgrading second batch to new version: f6b53eb4ab226e435a626e6b0c1da0618681017a
compatibility::simple-validator-upgrade::rest-validator-upgrading : committed: 7578.96 txn/s, latency: 4034.86 ms, (p50: 4600 ms, p70: 4900, p90: 5100 ms, p99: 5200 ms), latency samples: 141720
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 7494.55 txn/s, latency: 4562.22 ms, (p50: 4900 ms, p70: 5100, p90: 5300 ms, p99: 5600 ms), latency samples: 248920
5. check swarm health
Compatibility test for d15fc969c89551a1461d931d327b8d4dbfb2f814 ==> f6b53eb4ab226e435a626e6b0c1da0618681017a passed
Test Ok

github-actions · 2025-01-24T19:11:44Z

✅ Forge suite `framework_upgrade` success on `d15fc969c89551a1461d931d327b8d4dbfb2f814` ==> `f6b53eb4ab226e435a626e6b0c1da0618681017a`

Compatibility test results for d15fc969c89551a1461d931d327b8d4dbfb2f814 ==> f6b53eb4ab226e435a626e6b0c1da0618681017a (PR)
Upgrade the nodes to version: f6b53eb4ab226e435a626e6b0c1da0618681017a
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1185.28 txn/s, submitted: 1189.08 txn/s, failed submission: 3.79 txn/s, expired: 3.79 txn/s, latency: 3263.32 ms, (p50: 2100 ms, p70: 3100, p90: 6200 ms, p99: 12900 ms), latency samples: 106222
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1693.51 txn/s, submitted: 1697.93 txn/s, failed submission: 4.41 txn/s, expired: 4.41 txn/s, latency: 1755.36 ms, (p50: 1600 ms, p70: 1900, p90: 2500 ms, p99: 3600 ms), latency samples: 153460
5. check swarm health
Compatibility test for d15fc969c89551a1461d931d327b8d4dbfb2f814 ==> f6b53eb4ab226e435a626e6b0c1da0618681017a passed
Upgrade the remaining nodes to version: f6b53eb4ab226e435a626e6b0c1da0618681017a
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1001.93 txn/s, submitted: 1007.09 txn/s, failed submission: 5.17 txn/s, expired: 5.17 txn/s, latency: 2819.80 ms, (p50: 1600 ms, p70: 2100, p90: 6200 ms, p99: 14500 ms), latency samples: 89181
Test Ok

perryjrandall added CICD:run-framework-upgrade-test CICD:run-e2e-tests when this label is present github actions will run all land-blocking e2e tests from the PR labels Jan 16, 2025 — with Graphite App

perryjrandall marked this pull request as ready for review January 16, 2025 23:23

perryjrandall requested review from igor-aptos, JoshLind, rahxephon89, sionescu, vgao1996 and a team January 16, 2025 23:24

rahxephon89 approved these changes Jan 16, 2025

View reviewed changes