Fast Recovery From App Hash Mismatch

Context

cronosd rollback provide a way to rollback and recover from app-hash mismatch failure, but it takes hours to run, here is a faster way to do it.

Fast Rollback

NOTICE: python-iavl only support rocksdb for now.

1. Disable iavl fastnode fastnode reindex itself is slow enough, to make it faster, disable it. Set iavl-disable-fastnode = true in app.toml
2. Rollback tendermint state

   $ tendermint rollback --home /chain/.cronosd/
   

3. Fast rollback app state, replace $VER to the version number shown in the above command.

   $ nix run github:yihuang/python-iavl/v0.0.1 -- fast-rollback --db /path/to/data/application.db --target $VER