Hard failure in "rescue" test on Github runners

[dboreham]

Currently this test is disabled for stock runners because it fails:
https://github.com/0LNetworkCommunity/libra-framework/blob/main/.github/workflows/ci.yaml#L209

Same job passes in self-hosted runners, presumably due to some resource difference.

[dboreham]

Log tail from failed run:

Diem public info 000000000000000000000000000000000000000000000000000000000a550c18
Diem public info 000000000000000000000000000000000000000000000000000000000a550c18
Diem public info 000000000000000000000000000000000000000000000000000000000a550c18
1. generate framework upgrade writeset which should execute
2024-07-10T03:21:41.756910Z [test_framework_upgrade_writeset] INFO /home/runner/.cargo/git/checkouts/diem-7dee55d666bb1ba0/668b495/storage/schemadb/src/lib.rs:169 Opened RocksDB. {"rocksdb_name":"ledger_db"}
2024-07-10T03:21:41.756974Z [test_framework_upgrade_writeset] INFO /home/runner/.cargo/git/checkouts/diem-7dee55d666bb1ba0/668b495/storage/diemdb/src/ledger_db.rs:284 Opened ledger_db at "/tmp/.tmpW4J3zB/0/db/ledger_db"!
2024-07-10T03:21:41.756984Z [test_framework_upgrade_writeset] INFO /home/runner/.cargo/git/checkouts/diem-7dee55d666bb1ba0/668b495/storage/diemdb/src/ledger_db.rs:69 Opened ledger metadata db! {"ledger_metadata_db_path":"/tmp/.tmpW4J3zB/0/db/ledger_db"}
2024-07-10T03:21:41.756991Z [test_framework_upgrade_writeset] INFO /home/runner/.cargo/git/checkouts/diem-7dee55d666bb1ba0/668b495/storage/diemdb/src/ledger_db.rs:75 Individual ledger dbs are not enabled!
2024-07-10T03:21:41.756997Z [test_framework_upgrade_writeset] INFO /home/runner/.cargo/git/checkouts/diem-7dee55d666bb1ba0/668b495/storage/diemdb/src/state_kv_db.rs:43 State K/V DB is not enabled!
2024-07-10T03:21:41.759538Z [test_framework_upgrade_writeset] INFO /home/runner/.cargo/git/checkouts/diem-7dee55d666bb1ba0/668b495/storage/diemdb/src/state_merkle_db.rs:87 Sharded state merkle DB is not enabled!
2024-07-10T03:21:41.768119Z [test_framework_upgrade_writeset] INFO /home/runner/.cargo/git/checkouts/diem-7dee55d666bb1ba0/668b495/storage/schemadb/src/lib.rs:169 Opened RocksDB. {"rocksdb_name":"state_merkle_db"}
2024-07-10T03:21:41.768287Z [test_framework_upgrade_writeset] INFO /home/runner/.cargo/git/checkouts/diem-7dee55d666bb1ba0/668b495/storage/diemdb/src/state_store/mod.rs:380 Start syncing databases... {"overall_commit_progress":59}
2024-07-10T03:21:41.768317Z [test_framework_upgrade_writeset] INFO /home/runner/.cargo/git/checkouts/diem-7dee55d666bb1ba0/668b495/storage/diemdb/src/state_store/mod.rs:400 Start truncation... {"ledger_commit_progress":61}
thread 'test_framework_upgrade_writeset' panicked at /home/runner/.cargo/git/checkouts/diem-7dee55d666bb1ba0/668b495/storage/diemdb/src/state_store/mod.rs:415:18:
Failed to truncate ledger db.: Not implemented: Not supported operation in read only mode.
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
2024-07-10T03:21:41.769586Z [test_framework_upgrade_writeset] INFO /home/runner/.cargo/git/checkouts/diem-7dee55d666bb1ba0/668b495/storage/schemadb/src/lib.rs:304 Dropped RocksDB. {"rocksdb_name":"state_merkle_db"}
2024-07-10T03:21:41.770390Z [test_framework_upgrade_writeset] INFO /home/runner/.cargo/git/checkouts/diem-7dee55d666bb1ba0/668b495/storage/schemadb/src/lib.rs:304 Dropped RocksDB. {"rocksdb_name":"ledger_db"}
test dropped, running cleanup
Logs located at /tmp/.tmpW4J3zB

[dboreham]

Run time is a bit suspicious:

rescue 1h 0m 14s