localnet scripts: use remote Docker images by default. (#329)

Co-authored-by: Alfonso de la Rocha <[email protected]>

README.md

@@ -3,15 +3,27 @@
 Fendermint is an effort to implement [IPC with Tendermint Core](https://docs.google.com/document/d/1cFoTdoRuYgxmWJia6K-b5vmEj-4MvyHCNvShZpyconU/edit#). There is a preliminary [roadmap](https://docs.google.com/spreadsheets/d/1eVwkHEPGNg0js8DKRDIX7sugf5JqbI9zRBddIqzJFfI/edit#gid=0) that lays out the tasks towards implementing subnets that run IPLD and FVM under the Filecoin rootnet, sharing components with the Lotus/Eudico based implementation.
 
 ## Quick Start
+
 - [Local testnets](./docs/localnet.md)
 
 ## Docs
 
 Please have a look in the [docs](./docs/README.md) to see an overview of the project, how to run the components, and previous demos.
 
 ## IPC
+
 Fendermint is built with support for [IPC](https://github.com/consensus-shipyard/ipc) by design. If you are looking to deploy the infrastructure Fendermint-based IPC subnet, refer to the [IPC main repo](https://github.com/consensus-shipyard/ipc), or have a look at the [IPC infrastructure docs](./docs/ipc.md).
 
+## Building from source
+
+**Linux.** Install the following pre-requisites (instructions for Ubuntu):
+
+- Install system packages: `sudo apt install build-essential clang cmake pkg-config libssl-dev protobuf-compiler`.
+- Install Rust. See [instructions](https://www.rust-lang.org/tools/install).
+- Install cargo-make: `cargo install --force cargo-make`.
+- Install Foundry. See [instructions](https://book.getfoundry.sh/getting-started/installation).
+- Install Docker (if you intend to build Docker images). See [instructions](https://docs.docker.com/engine/install/ubuntu/).
+
 ## Testing
 
 The following command runs unit and integration tests:
@@ -28,7 +40,6 @@ while the next command builds docker images and runs an end-to-end test using th
 make e2e
 ```
 
-
 ## IPC Solidity Actors
 
 We generate Rust bindings for the Solidity actors we need to invoke from the [ipc-solidity-actors](https://github.com/consensus-shipyard/ipc-solidity-actors) library, some of which are deployed during the genesis process. The bindings live in [ipc_actors](./fendermint/vm/ipc_actors/), and are generated automatically during the build, or with the following command:
@@ -47,7 +58,6 @@ To test whether the genesis process works, we can run the following unit test:
 cargo test --release -p fendermint_vm_interpreter load_genesis
 ```
 
-
 ## Pre-built Docker Image
 
 The CI build publishes a [Docker image](https://github.com/consensus-shipyard/fendermint/pkgs/container/fendermint) to Github Container Registry upon a successful build on the `main` branch. This is the same image as the one used in the End-to-End tests; it contains the built-in actor bundle and IPC Solidity actors, ready to be deployed during genesis.

docs/ipc.md

@@ -49,7 +49,7 @@ Once a child subnet has been bootstrapped in its parent, its subnet actor has be
 In order to spawn a validator node in a child subnet, you need to run:
 ```bash
 cargo make --makefile infra/Makefile.toml \
-    -e VALIDATOR_PRIV_KEY=<VALIDATOR_PRIV_KEY> \
+    -e PRIVATE_KEY_PATH=<VALIDATOR_PRIV_KEY> \
     -e SUBNET_ID=<SUBNET_ID> \
     -e CMT_P2P_HOST_PORT=<COMETBFT_P2P_PORT> \
     -e CMT_RPC_HOST_PORT=<COMETBFT_RPC_PORT> \
@@ -66,7 +66,7 @@ This command will run the infrastructure for a Fendermint validator in the child
 - `CMT_RPC_HOST_PORT` (optional): Specifies the listening port in the localhost for CometBFT's RPC.
 - `ETHAPI_HOST_PORT` (optional): Specifies the listening port in the localhost for the ETH RPC of the node.
 - `NODE_NAME` (optional): Name for the node deployment. Along with `CMT_P2P_HOST_PORT`, `CMT_RPC_HOST_PORT` and `ETHAPI_HOST_PORT`, these variables come really handy for the deployment of several validator nodes over the same system.
-- `VALIDATOR_PRIV_KEY`: Path of the private key for your validator (it should be the corresponding one used to join the subnet in the parent).
+- `PRIVATE_KEY_PATH`: Path of the hex encoded private key for your validator (it should be the corresponding one used to join the subnet in the parent). This can be exported from the `ipc-cli` or any other wallet like Metamask.
 - `SUBNET_ID`: SubnetID for the child subnet.
 - `BOOTSTRAPS`: Comma separated list of bootstraps (or seeds in CometBFT parlance).
 - `PARENT_ENDPOINT`: Public endpoint that the validator should use to connect to the parent.

docs/localnet.md

@@ -1,16 +1,33 @@
 # Local Testnets
 
-Prerequisites:
-```bash
-make build docker-build
-```
+Setting up a Fendermint local testnet is a way to get started quickly with IPC.
+
+This guide offers two flavours:
+
+- A single node deployment: useful for developing smart contracts and testing the APIs.
+- A 4 node testnet: useful for testing consensus, checkpointing, and more.
+
+## Prerequisites
+
+On Linux (links and instructions for Ubuntu):
+
+- Install Docker. See [instructions](https://docs.docker.com/engine/install/ubuntu/).
+- Install Rust. See [instructions](https://www.rust-lang.org/tools/install).
+- Install cargo-make: `cargo install --force cargo-make`.
+
+## Docker images
+
+These commands will pull various Docker images from remote repositories, including `fendermint:latest`, by default.
+
+- To override which Fendermint Docker image to pull, set the `FM_DOCKER_TAG` env variable to the desired tag.
+- To use a local Fendermint image, set the `FM_PULL_SKIP` env variable to some value, e.g. `FM_PULL_SKIP=true`.
 
 ## Single node deployment
 
-To run IPC in the local rootnet just perform the following :
+To run IPC in the local rootnet just perform the following:
+
 ```bash
 cargo make --makefile ./infra/Makefile.toml testnode
-
 ```
 
 It will create three docker containers (cometbft, fendermint, and eth-api).
@@ -21,41 +38,48 @@ cargo make --makefile ./infra/Makefile.toml testnode-down
 ```
 
 ## Local 4-nodes deployment
-To run IPC in the local rootnet with 4 nodes perform the following command :
+
+To run IPC in the local rootnet with 4 nodes perform the following command:
+
 ```bash
 cargo make --makefile ./infra/Makefile.toml testnet
-
 ```
 
 To stop the network:
+
 ```bash
 cargo make --makefile ./infra/Makefile.toml testnet-down
 ```
 
 The testnet contains four logical nodes. Each node consists of cometbft, fendermint, and ethapi containers.
 The Docker internal network is `192.167.10.0/24`.
 
-ETH-API is accessible on the following interfaces on the Docker internal network:
+The Ethereum API is accessible on the following endpoints on the Docker internal network:
+
 - `192.167.10.10:8545` or `ethapi-node0:8545`
 - `192.167.10.11:8545` or `ethapi-node1:8545`
 - `192.167.10.12:8545` or `ethapi-node2:8545`
 - `192.167.10.13:8545` or `ethapi-node3:8545`
 
-and on the following interfaces from the host machine:
+And on the following endpoints from the host machine:
+
 - `127.0.0.1:8545`
 - `127.0.0.1:8546`
 - `127.0.0.1:8547`
 - `127.0.0.1:8548`
 
-## Deployment process
-
-The deployment process is as follows:
-- Remove all docker containers, files, networks, etc. from the previous deployment
-- Create all necessary directories
-- Initialize CometBFT testnet by creating `config` and `data` directories using `cometbft` tools
-- Read cometbft nodes private keys,derive node IDs and store in `config.toml` for each node
-- Create the `genesis` file for Fendermint
-- Share the genesis among all Fendermint nodes
-- Run Fendermint application in 4 containers
-- Run CometBFT in 4 containers
-- Run Eth API in 4 containers
+## What's happening behind the scenes
+
+> For a 4-node deployment.
+
+The deployment process performs the following steps:
+
+- Remove all Docker containers, files, networks, etc. from any previous deployments.
+- Create all necessary directories.
+- Initialize CometBFT testnet by creating `config` and `data` directories using `cometbft` tools.
+- Read CometBFT nodes private keys, derive node IDs and store them in `config.toml` for each node.
+- Create the `genesis` file for Fendermint.
+- Share the genesis among all Fendermint nodes.
+- Run Fendermint application in 4 containers.
+- Run CometBFT in 4 containers.
+- Run Eth API in 4 containers.

infra/Makefile.toml

@@ -3,18 +3,23 @@ extend = [
   { path = "scripts/cometbft.toml" },
   { path = "scripts/fendermint.toml" },
   { path = "scripts/ethapi.toml" },
+  { path = "scripts/genesis.toml"},
   { path = "scripts/node.toml" },
   { path = "scripts/testnet.toml" },
   { path = "scripts/testnode.toml" },
-  { path = "scripts/subnet.toml" }
+  { path = "scripts/subnet.toml" },
+  { path = "scripts/bootstrap.toml" },
 ]
 
 [config]
 default_to_workspace = false
 
 [env]
 # General network-specific parameters
-SUBNET_ID = { value = "/r0", condition = { env_not_set = ["SUBNET_ID"] }}
+SUBNET_ID = { value = "/r0", condition = { env_not_set = ["SUBNET_ID"] } }
+# The network name is derived from the SUBNET_ID, replacing slashes with dashes, and dropping the first dash if any.
+NETWORK_NAME = { script = ["echo $SUBNET_ID | sed -e 's/\\//-/g' -e 's/^-//1'"] }
+
 BALANCE = { value = "1000", condition = { env_not_set = ["BALANCE"] }}
 BASE_FEE = { value = "1000", condition = { env_not_set = ["BASE_FEE"] }}
 TIMESTAMP = { value = "1680101412", condition = { env_not_set = ["TIMESTAMP"] }}
@@ -29,38 +34,40 @@ NODE_NAME = { value = "ipc-node", condition = { env_not_set = ["NODE_NAME"] } }
 PARENT_ENDPOINT = { value = "https://api.calibration.node.glif.io/rpc/v1", condition = { env_not_set = ["PARENT_ENDPOINT"] } }
 PARENT_GATEWAY = { value = "0x56948d2CFaa2EF355B8C08Ac925202db212146D1", condition = { env_not_set = ["PARENT_GATEWAY"] } }
 PARENT_REGISTRY = { value = "0x6A4884D2B6A597792dC68014D4B7C117cca5668e", condition = { env_not_set = ["PARENT_REGISTRY"] } }
-NETWORK_TYPE = { value = "test", condition = { env_not_set = ["NETWORK_TYPE"] } }
-VALIDATOR_PRIV_KEY = { value = "~/.ipc/priv.key", condition = { env_not_set = ["VALIDATOR_PRIV_KEY"] } }
+FM_NETWORK = { value = "test", condition = { env_not_set = ["FM_NETWORK"] } }
 TOPDOWN_CHAIN_HEAD_DELAY = { value = "10", condition = { env_not_set = ["TOPDOWN_CHAIN_HEAD_DELAY"] } }
 # Comma-separated list of bootstrap nodes to be used by the CometBFT node.
 BOOTSTRAPS = { value = "", condition = { env_not_set = ["BOOTSTRAPS"] } }
+PRIVATE_KEY_PATH = { value = "", condition = { env_not_set = ["PRIVATE_KEY_PATH"] } }
 
 # Deployment-related
-BASE_DIR="${HOME}/.ipc/${SUBNET_ID}"
-FM_DIR="${BASE_DIR}/fendermint"
-CMT_DIR="${BASE_DIR}/cometbft"
+BASE_DIR="${HOME}/.ipc/${NETWORK_NAME}"
+FM_DIR="${BASE_DIR}/${NODE_NAME}/fendermint"
+CMT_DIR="${BASE_DIR}/${NODE_NAME}/cometbft"
 
-GENESIS_FILE="${FM_DIR}/genesis.json"
-KEYS_DIR="${FM_DIR}/keys"
+GENESIS_FILE="${BASE_DIR}/genesis.json"
+KEYS_SUBDIR="keys"
 KEY_NAME="validator_key"
-PUB_KEY_PATH="${KEYS_DIR}/${KEY_NAME}.pk"
-PRIV_KEY_PATH="${KEYS_DIR}/${KEY_NAME}.sk"
+PUB_KEY_PATH="${KEYS_SUBDIR}/${KEY_NAME}.pk"
+PRIV_KEY_PATH="${KEYS_SUBDIR}/${KEY_NAME}.sk"
 
-NETWORK_NAME = "${SUBNET_ID}"
+COMETBFT_SUBDIR = "cometbft"
 
 CMT_CONTAINER_NAME = "${NETWORK_NAME}-cometbft"
 FM_CONTAINER_NAME = "${NETWORK_NAME}-fendermint"
 ETHAPI_CONTAINER_NAME = "${NETWORK_NAME}-ethapi"
 
 CMT_DOCKER_IMAGE = "cometbft/cometbft:v0.37.x"
-FM_DOCKER_IMAGE = "fendermint:latest"
+FM_DOCKER_TAG = "latest"
+FM_DOCKER_IMAGE = "fendermint:${FM_DOCKER_TAG}"
+FM_REMOTE_DOCKER_IMAGE = "ghcr.io/consensus-shipyard/fendermint:${FM_DOCKER_TAG}"
 TEST_DATA_DIR = "${CARGO_MAKE_WORKSPACE_WORKING_DIRECTORY}/fendermint/testing/smoke-test/test-data"
 TEST_SCRIPTS_DIR = "${CARGO_MAKE_WORKSPACE_WORKING_DIRECTORY}/fendermint/testing/smoke-test/scripts"
 ACTORS_BUNDLE = "${CARGO_MAKE_WORKSPACE_WORKING_DIRECTORY}/../builtin-actors/output/bundle.car"
 # If this wasn't present, any wait task is skipped.
 CARGO_MAKE_WAIT_MILLISECONDS = 5000
 # This wait time seems to work locally.
-CMT_WAIT_MILLIS = 30000
+CMT_WAIT_MILLIS = 20000
 # Keep example logs to a minimum.
 VERBOSITY = ""
 

infra/docker-compose.yml

@@ -1,4 +1,20 @@
 services:
+  fendermint-node:
+    container_name: fendermint-node${NODE_ID}
+    user: ${UID}:${GID}
+    image: "fendermint:latest"
+    environment:
+      - FM_DATA_DIR=/data/fendermint/data
+      - FM_CHAIN_NAME=$NETWORK_NAME
+      - TENDERMINT_RPC_URL=http://cometbft-node${NODE_ID}:26657
+      - LOG_LEVEL=info
+      - FM_NETWORK=$FM_NETWORK
+    volumes:
+      - $BASE_DIR/node${NODE_ID}:/data
+    networks:
+      testnet:
+        ipv4_address: ${FMT_NODE_ADDR}
+
   cometbft-node:
     container_name: cometbft-node${NODE_ID}
     user: ${UID}:${GID}
@@ -22,20 +38,6 @@ services:
       testnet:
         ipv4_address: ${CMT_NODE_ADDR}
 
-  fendermint-node:
-    container_name: fendermint-node${NODE_ID}
-    user: ${UID}:${GID}
-    image: "fendermint:latest"
-    environment:
-      - FM_DATA_DIR=/data/fendermint/data
-      - FM_CHAIN_NAME=$SUBNET_ID
-      - LOG_LEVEL=info
-    volumes:
-      - $BASE_DIR/node${NODE_ID}:/data
-    networks:
-      testnet:
-        ipv4_address: ${FMT_NODE_ADDR}
-
   ethapi-node:
     container_name: ethapi-node${NODE_ID}
     user: ${UID}:${GID}

infra/scripts/bootstrap.toml

@@ -0,0 +1,11 @@
+[tasks.bootstrap]
+env = { "NETWORK_NAME"="${NETWORK_NAME}", "CMT_DIR" = "${BASE_DIR}/bootstrap/cometbft", "CMT_CONTAINER_NAME" = "cometbft-${NODE_NAME}-bootstrap" }
+run_task = "new-node"
+
+[tasks.bootstrap-id]
+env = { "CMT_CONTAINER_NAME" = "cometbft-${NODE_NAME}-bootstrap" }
+run_task = "cometbft-node-id"
+
+[tasks.bootstrap-down]
+env = { "CMT_CONTAINER_NAME" = "cometbft-${NODE_NAME}-bootstrap" }
+run_task = "node-down"

infra/scripts/fendermint.toml

@@ -2,8 +2,15 @@
 extend = "fendermint-run"
 env = { "ENTRY" = "fendermint", "CMD" = "run", "FLAGS" = "-d" }
 
-[tasks.fendermint-run]
+[tasks.fendermint-pull]
+condition = { env_not_set = ["FM_PULL_SKIP"], fail_message = "Skipped pulling fendermint Docker image." }
 script = """
+  docker pull ghcr.io/consensus-shipyard/fendermint:${FM_DOCKER_TAG}
+  docker tag ghcr.io/consensus-shipyard/fendermint:${FM_DOCKER_TAG} fendermint:${FM_DOCKER_TAG}
+"""
+
+[tasks.fendermint-run]
+script.main = """
 docker run \
   ${FLAGS} \
   --name ${FM_CONTAINER_NAME} \
@@ -15,9 +22,11 @@ docker run \
   --env FM_CHAIN_NAME=${NETWORK_NAME} \
   --env TENDERMINT_RPC_URL=http://${CMT_CONTAINER_NAME}:26657 \
   --env LOG_LEVEL=info \
+  --env RUST_BACKTRACE=1 \
   --entrypoint ${ENTRY} \
   ${FM_DOCKER_IMAGE} \
-  --network=${NETWORK_TYPE} ${CMD}
+  --network=${FM_NETWORK} \
+  ${CMD}
 """
 dependencies = ["docker-network-create", "fendermint-deps"]
 
@@ -36,7 +45,7 @@ docker run \
   --volume ${BASE_DIR}:/data \
   --env FM_DATA_DIR=/data/fendermint/data \
   --env FM_CHAIN_NAME=${NETWORK_NAME} \
-  --env FM_IPC__SUBNET_ID=${NETWORK_NAME} \
+  --env FM_IPC__SUBNET_ID=${SUBNET_ID} \
   --env FM_IPC__TOPDOWN__CHAIN_HEAD_DELAY=${TOPDOWN_CHAIN_HEAD_DELAY} \
   --env FM_IPC__TOPDOWN__PARENT_HTTP_ENDPOINT=${PARENT_ENDPOINT} \
   --env FM_IPC__TOPDOWN__PARENT_REGISTRY=${PARENT_REGISTRY} \
@@ -49,13 +58,30 @@ docker run \
   --env FM_TENDERMINT_RPC_URL=http://${CMT_CONTAINER_NAME}:26657 \
   --env TENDERMINT_RPC_URL=http://${CMT_CONTAINER_NAME}:26657 \
   --env LOG_LEVEL=info \
+  --env RUST_BACKTRACE=1 \
   --entrypoint ${ENTRY} \
   ${FM_DOCKER_IMAGE} \
-  --network=${NETWORK_TYPE} \
+  --network=${FM_NETWORK} \
   ${CMD}
 """
 dependencies = ["docker-network-create", "fendermint-deps"]
 
+[tasks.fendermint-tool]
+script.main = """
+docker run \
+  ${FLAGS} \
+  --init \
+  --user $(id -u) \
+  --volume ${BASE_DIR}:/data \
+  --env LOG_LEVEL=info \
+  --env RUST_BACKTRACE=1 \
+  --entrypoint ${ENTRY} \
+  ${FM_DOCKER_IMAGE} \
+  --network=${FM_NETWORK} \
+  ${CMD}
+"""
+dependencies = ["fendermint-deps"]
+
 [tasks.fendermint-deps]
 script = """
 # Check if the image exists