[Genesis] Migration tools for genesis from legacy accounts (0LNetwork…

…Community#1237)

* For 0L genesis tools for hard forks, and migrations: create a MoveVm session from fake data to be able to apply migrations from Move system contracts.

* wip testing fakeexector for account migrations

* Scaffolding tests for v6 genesis fork. first cleanup deprecated modules.

* add fixtures for testing recovery genesis

* cleanup fixtures
implement functional tests for:
- exporting db backup to json
- creating genesis blob from json
- creating genesis blob in one shot from db backup

TODO: launch test node from from genesis.blob
TODO: e2e tests from a fixture file

* integration test using blob from fixture to start a diem-node in test mode

* patch github actions tests

* full e2e test from db backup to starting a node

* Add diem_logger as per @corythian plus comments.

* query nodes to check if onchain state is the same as recovery.

* - include db reader util to inspect a genesis.blob.
- refactor recovery tests

* test to check validator count. cleanup test utlis

* cleanup tests

* stub a "check" feature, to check a json file agains the output genesis.blob

* finish balance comparison with error list.

* clean

* test to create single validator and check output.

* skip comparing system zero address.

* cargo fix

* cargo fmt

* create function to start a vm executor, and uses a function pointer to transform legacy data to Move arguments.

* experimenting with alternative way of starting a vm genesis session

* runs

* cleanup

* need to explicitly set the validators at genesis

* test to confirm genesis validator set, is the same as defined set.

* tests for validator set and balance passing

* change fixtures, make node start from command line

* can start a test node from genesis blob

* can genesis tools test can start a node from a genesis and query for an accounts resources

* make ol version of swarm config generator in ol/smoke_tests

* wip

* move legacyRecovery to ol/types. So now we can run the user migrations in the vm_genesis::lib. First draft builds

* create GenesisMigration user migration test. Builds.

* ol/genesis-tools builds with refactor

* refactored with clean build

* minor refactors

* can migrate the balance of end users with new design

* migration successfully migrates balances of users, validators, and operators.

* cleanup unused files. clippy,

* tests passing

* whitespace to trigger ci

.github/workflows/forge.yaml

@@ -29,7 +29,7 @@ jobs:
           restore-keys: ${{ runner.os }}-cargo-
 
       - name: Build stdlib
-        run: make stdlib
+        run: sh ol/util/build-stdlib.sh
 
       - name: Run Forge Local Tests
         run: cargo r -p forge-cli -- test local-swarm

.github/workflows/smoke.yaml

@@ -28,8 +28,9 @@ jobs:
         run: sudo apt-get install -y lld
 
       - name: build stdlib
-        run: make stdlib
+        run: sh ol/util/build-stdlib.sh
 
       - name: smoke tests
-        run: CARGO_ARGS="--color never" make smoke-test
+        run: cargo t -p smoke-test -- --test-threads=1 test_basic_fault_tolerance --nocapture
+        # todo get the other smoke tests running
         continue-on-error: true

.github/workflows/tools.yaml

@@ -37,3 +37,6 @@ jobs:
       - name: cli tests
         run: cd ol/cli && NODE_ENV=test cargo test
         continue-on-error: true
+      - name: genesis tests
+        run: cd ol/genesis-tools && NODE_ENV=test cargo test
+        continue-on-error: true