From 116a24f77e80e16adf24ac82c2ac7d878b9f1f2d Mon Sep 17 00:00:00 2001 From: Fridrik Asmundsson Date: Wed, 10 Apr 2024 10:30:29 +0000 Subject: [PATCH 1/5] Several improvents to deploy subnet script - Fix relayer not starting due to missing argument - Fix bug when default-key was first in the json array - Streamline local deployment by introducing a prepare_local.sh script - Improve documentation for local deployment and document more known issues --- README.md | 2 +- .../README.md | 8 +-- .../deploy.sh | 15 ++++-- .../prepare_local.sh | 50 +++++++++++++++++++ 4 files changed, 67 insertions(+), 8 deletions(-) mode change 100644 => 100755 scripts/deploy_subnet_under_calibration_net/deploy.sh create mode 100755 scripts/deploy_subnet_under_calibration_net/prepare_local.sh diff --git a/README.md b/README.md index e777843e9..8112d7450 100644 --- a/README.md +++ b/README.md @@ -107,4 +107,4 @@ To be able to interact with Calibration and run new subnets, some FIL should be ## Help -If you meet any obstacles join us in **#ipc-help** in the [Filecoin Slack workspace](https://filecoin.io/slack). +If you meet any obstacles join us in **#ipc** in the [Filecoin Slack workspace](https://filecoin.io/slack). diff --git a/scripts/deploy_subnet_under_calibration_net/README.md b/scripts/deploy_subnet_under_calibration_net/README.md index 0da252b41..3d5897d80 100644 --- a/scripts/deploy_subnet_under_calibration_net/README.md +++ b/scripts/deploy_subnet_under_calibration_net/README.md @@ -4,13 +4,13 @@ We currently have a [Github workflow](https://github.com/consensus-shipyard/ipc/actions/workflows/deploy-to-dedicated-host.yaml) to deploy IPC infra to a dedicated host. You can go to the workflow page and click `Run workflow` button on the top right corner to initiate a deployment. ## Deploy to local machine -The same `deploy.sh` script can also be used to deploy locally. This is more or less equivalent to following [quickstart-calibration.md](https://github.com/consensus-shipyard/ipc/blob/main/docs/ipc/quickstart-calibration.md), but much more automated. +The same `deploy.sh` script can also be used to deploy locally. This is more or less equivalent to following [Deploy a subnet](https://docs.ipc.space/quickstarts/deploy-a-subnet), but much more automated. To run this script locally, you need to first manually prepare the environment and files. -1. Make sure you have your ipc repo located at $HOME/ipc. -2. Follow Step 2 and Step 3 in [Github workflow](https://github.com/consensus-shipyard/ipc/actions/workflows/deploy-to-dedicated-host.yaml) to prepare ipc config file and wallets. Remember to go to [Calibration faucet](https://faucet.calibration.fildev.network/funds.html) to fund all of your addresses. -3. Run `bash deploy.sh local` to deploy IPC locally. +1. Run `prepare_local.sh` to generate a local ipc repo +2. Go to [Calibration faucet](https://faucet.calibration.fildev.network/funds.html) to fund all of your addresses. +3. Run `deploy.sh local` to deploy IPC locally. Please also notice that 1. The `deploy.sh` is only for running on Linux. If you are using a Mac, you need to disable all `apt` based dependency installation. You may also need to install bash (version >= 5) to run this script since the script isn't fully compatible with zsh (default shell on Mac). diff --git a/scripts/deploy_subnet_under_calibration_net/deploy.sh b/scripts/deploy_subnet_under_calibration_net/deploy.sh old mode 100644 new mode 100755 index 857c766db..6f678cc23 --- a/scripts/deploy_subnet_under_calibration_net/deploy.sh +++ b/scripts/deploy_subnet_under_calibration_net/deploy.sh @@ -1,12 +1,14 @@ #!/bin/bash # IPC Quick Start Script -# See also https://github.com/consensus-shipyard/ipc/blob/main/docs/ipc/quickstart-calibration.md +# See also https://docs.ipc.space/quickstarts/deploy-a-subnet # Known issues: # 1. Need to previously manual enable sudo without password on the host # 2. You may need to rerun the script after docker installation for the first time -# 2. You may need to manually install nodejs and npm on the host +# 3. You may need to manually install nodejs and npm on the host +# 4. If you get failures related to "deserializing fvm address", check if your +# FM_NETWORK env variable is set correctly (e.g it should either be testnet or mainnet) set -euxo pipefail @@ -34,6 +36,13 @@ else fi fi +# the script assumes there is no default-key, so lets check that early on. +# TODO: Update this script to handle having default-key set +if grep -q default-key ${IPC_CONFIG_FOLDER}/evm_keystore.json; then + echo "Please remove default-key from ${IPC_CONFIG_FOLDER}/evm_keystore.json before running this script" + exit 1 +fi + # Step 1: Prepare system for building and running IPC # Step 1.1: Install build dependencies @@ -272,7 +281,7 @@ done pkill -f "relayer" || true # Start relayer echo "$DASHES Start relayer process (in the background)" -nohup $IPC_CLI checkpoint relayer --subnet $subnet_id > nohup.out 2> nohup.err < /dev/null & +nohup $IPC_CLI checkpoint relayer --subnet $subnet_id --submitter $default_wallet_address > nohup.out 2> nohup.err < /dev/null & # Step 11: Print a summary of the deployment # Remove leading '/' and change middle '/' into '-' diff --git a/scripts/deploy_subnet_under_calibration_net/prepare_local.sh b/scripts/deploy_subnet_under_calibration_net/prepare_local.sh new file mode 100755 index 000000000..05c0842f7 --- /dev/null +++ b/scripts/deploy_subnet_under_calibration_net/prepare_local.sh @@ -0,0 +1,50 @@ +#!/bin/bash + +# A script which resets your local .ipc folder so we can run the deploy.sh script for a local deployment +# +# To run: +# 1. Run `./prepare_local.sh` +# 2. Run `./deploy.sh` +# +# Known issues: +# - The deploy.sh script does not run "make docker-build" which can cause errors if its outdated on your +# machine. To fix this, run `make docker-build` in ipc/fendermint folder and rerun the deploy.sh script + +set -eo pipefail + +SCRIPT_DIR=$( cd "$(dirname "${BASH_SOURCE[0]}")" ; pwd -P ) +echo "SCRIPT_DIR: $SCRIPT_DIR" + +echo "Installing dependencies..." +cargo install toml-cli + +echo "Removing existing .ipc folder" +rm -rf ~/.ipc + +echo "Copying new .ipc folder" +cp -r $SCRIPT_DIR/.ipc ~/.ipc + +echo "Fetching info about newest current deployment contracts" +hash=$(curl -s https://raw.githubusercontent.com/consensus-shipyard/ipc/cd/contracts/deployments/r314159.json) +echo $hash | jq + +echo "Parsing gateway and registry address..." +gateway_addr=$(echo $hash | jq -r '.gateway_addr') +echo "- Gateway address: $gateway_addr" +registry_addr=$(echo $hash | jq -r '.registry_addr') +echo "- Registry address: $registry_addr" + +echo "Updating config with new gateway and registry address" +toml set ~/.ipc/config.toml subnets[0].config.gateway_addr $gateway_addr > ~/.ipc/config.toml.tmp +toml set ~/.ipc/config.toml.tmp subnets[0].config.registry_addr $registry_addr > ~/.ipc/config.toml.tmp2 +cp ~/.ipc/config.toml.tmp2 ~/.ipc/config.toml + +echo "Setting up wallets" +wallet1=$(cargo run -q -p ipc-cli --release -- wallet new --wallet-type evm | tr -d '"') +echo "- Wallet1: $wallet1" +wallet2=$(cargo run -q -p ipc-cli --release -- wallet new --wallet-type evm | tr -d '"') +echo "- Wallet2: $wallet2" +wallet3=$(cargo run -q -p ipc-cli --release -- wallet new --wallet-type evm | tr -d '"') +echo "- Wallet3: $wallet3" + +echo "--- GO TO Calibration faucet (https://faucet.calibnet.chainsafe-fil.io/) and get some tokens for the wallets ---" From 2d98940a04e4ea07efedf11072f107a9134b271f Mon Sep 17 00:00:00 2001 From: Fridrik Asmundsson Date: Thu, 11 Apr 2024 16:10:15 +0000 Subject: [PATCH 2/5] Address review comments --- .../prepare_local.sh | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/scripts/deploy_subnet_under_calibration_net/prepare_local.sh b/scripts/deploy_subnet_under_calibration_net/prepare_local.sh index 05c0842f7..2fac63592 100755 --- a/scripts/deploy_subnet_under_calibration_net/prepare_local.sh +++ b/scripts/deploy_subnet_under_calibration_net/prepare_local.sh @@ -12,6 +12,11 @@ set -eo pipefail +read -p "Warning, this will erase your ~/.ipc folder, continue (y/n)?" +if [[ ! $REPLY =~ ^[Yy]$ ]]; then + exit 1 +fi + SCRIPT_DIR=$( cd "$(dirname "${BASH_SOURCE[0]}")" ; pwd -P ) echo "SCRIPT_DIR: $SCRIPT_DIR" @@ -25,19 +30,20 @@ echo "Copying new .ipc folder" cp -r $SCRIPT_DIR/.ipc ~/.ipc echo "Fetching info about newest current deployment contracts" -hash=$(curl -s https://raw.githubusercontent.com/consensus-shipyard/ipc/cd/contracts/deployments/r314159.json) -echo $hash | jq +deployment=$(curl -s https://raw.githubusercontent.com/consensus-shipyard/ipc/cd/contracts/deployments/r314159.json) +echo $deployment | jq echo "Parsing gateway and registry address..." -gateway_addr=$(echo $hash | jq -r '.gateway_addr') +gateway_addr=$(echo $deployment | jq -r '.gateway_addr') echo "- Gateway address: $gateway_addr" -registry_addr=$(echo $hash | jq -r '.registry_addr') +registry_addr=$(echo $deployment | jq -r '.registry_addr') echo "- Registry address: $registry_addr" echo "Updating config with new gateway and registry address" toml set ~/.ipc/config.toml subnets[0].config.gateway_addr $gateway_addr > ~/.ipc/config.toml.tmp toml set ~/.ipc/config.toml.tmp subnets[0].config.registry_addr $registry_addr > ~/.ipc/config.toml.tmp2 cp ~/.ipc/config.toml.tmp2 ~/.ipc/config.toml +rm ~/.ipc/config.toml.tmp* echo "Setting up wallets" wallet1=$(cargo run -q -p ipc-cli --release -- wallet new --wallet-type evm | tr -d '"') From cfe06388b463041ca5c825a8cbe116ae7015852f Mon Sep 17 00:00:00 2001 From: Fridrik Asmundsson Date: Thu, 18 Apr 2024 10:05:20 +0000 Subject: [PATCH 3/5] Fix deploy.sh script on Mac --- ipc/provider/src/manager/evm/manager.rs | 2 +- scripts/deploy_subnet_under_calibration_net/README.md | 2 +- scripts/deploy_subnet_under_calibration_net/deploy.sh | 8 +++++--- 3 files changed, 7 insertions(+), 5 deletions(-) diff --git a/ipc/provider/src/manager/evm/manager.rs b/ipc/provider/src/manager/evm/manager.rs index 4ed757769..4c67ea3bd 100644 --- a/ipc/provider/src/manager/evm/manager.rs +++ b/ipc/provider/src/manager/evm/manager.rs @@ -573,7 +573,7 @@ impl SubnetManager for EthSubnetManager { from: Address, amount: TokenAmount, ) -> Result { - log::debug!("approve token, subnet: {subnet}, amount: {amount}, from: {from}"); + tracing::debug!("approve token, subnet: {subnet}, amount: {amount}, from: {from}"); let value = fil_amount_to_eth_amount(&amount)?; diff --git a/scripts/deploy_subnet_under_calibration_net/README.md b/scripts/deploy_subnet_under_calibration_net/README.md index 3d5897d80..bb7fe79d1 100644 --- a/scripts/deploy_subnet_under_calibration_net/README.md +++ b/scripts/deploy_subnet_under_calibration_net/README.md @@ -13,7 +13,7 @@ To run this script locally, you need to first manually prepare the environment a 3. Run `deploy.sh local` to deploy IPC locally. Please also notice that -1. The `deploy.sh` is only for running on Linux. If you are using a Mac, you need to disable all `apt` based dependency installation. You may also need to install bash (version >= 5) to run this script since the script isn't fully compatible with zsh (default shell on Mac). +1. The `deploy.sh` is only for running on Linux. If you are using a Mac, you need to disable all `apt` based dependency installation. You may also need to install bash (version >= 5) to run this script since the script isn't fully compatible with zsh (default shell on Mac). Also, make sure you habe openssl packages installed (`brew install openssl@3`). 2. The automated dependency installation isn't guarantee to work 100% time. If you encountered any dependency installation issue, please refer to the script and retry. Usually you can resolve the issues by creating a new terminal, sourcing `~/.bash.rc`, etc. 3. Depends on the RPC endpoint's quality of service for the calibration net, your command may or may not succeed when interacting with the RPC endpoint. Sometimes you will get rate limited. In that case, you can choose a different calibration provider URL from [Chainlist](https://chainlist.org/?search=calibration&testnets=true) to replace the value of `RPC_URL` variable in the script, then retry it. 4. You need to manually install nodejs and npm. The reason is that we need to use very recent version of nodejs and it's usually not included with the Linux distribution. It's recommended that you use nvm (Node version manager) to manage your nodejs installation. diff --git a/scripts/deploy_subnet_under_calibration_net/deploy.sh b/scripts/deploy_subnet_under_calibration_net/deploy.sh index 6f678cc23..d5984bfee 100755 --- a/scripts/deploy_subnet_under_calibration_net/deploy.sh +++ b/scripts/deploy_subnet_under_calibration_net/deploy.sh @@ -109,9 +109,11 @@ else fi # Make sure we re-read the latest env before finishing dependency installation. -set +u -source ${HOME}/.bashrc -set -u +if test -f ${HOME}/.bashrc; then + set +u + source ${HOME}/.bashrc + set -u +fi # Step 2: Prepare code repo and build ipc-cli if ! $local_deploy ; then From 5387f19f3614b6f33c237adb3f634adaec8d161c Mon Sep 17 00:00:00 2001 From: Fridrik Asmundsson Date: Mon, 22 Apr 2024 16:20:53 +0000 Subject: [PATCH 4/5] Rename existing ipc folder with timestamp instead of deleting it --- .../prepare_local.sh | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/scripts/deploy_subnet_under_calibration_net/prepare_local.sh b/scripts/deploy_subnet_under_calibration_net/prepare_local.sh index 2fac63592..9451ff04e 100755 --- a/scripts/deploy_subnet_under_calibration_net/prepare_local.sh +++ b/scripts/deploy_subnet_under_calibration_net/prepare_local.sh @@ -12,9 +12,11 @@ set -eo pipefail -read -p "Warning, this will erase your ~/.ipc folder, continue (y/n)?" -if [[ ! $REPLY =~ ^[Yy]$ ]]; then - exit 1 +if [ -d ~/.ipc ]; then + TIMESTAMP=$(date +%s) + echo "Warning: an existing ~/.ipc folder already exists!" + echo "Renaming ~/.ipc to ~/.ipc.$TIMESTAMP for backup" + mv ~/.ipc ~/.ipc.$TIMESTAMP fi SCRIPT_DIR=$( cd "$(dirname "${BASH_SOURCE[0]}")" ; pwd -P ) @@ -23,9 +25,6 @@ echo "SCRIPT_DIR: $SCRIPT_DIR" echo "Installing dependencies..." cargo install toml-cli -echo "Removing existing .ipc folder" -rm -rf ~/.ipc - echo "Copying new .ipc folder" cp -r $SCRIPT_DIR/.ipc ~/.ipc From 76462787cde5f12f311da16c32b7be64ee36d177 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Fri=C3=B0rik=20=C3=81smundsson?= Date: Tue, 23 Apr 2024 14:34:16 +0000 Subject: [PATCH 5/5] Update scripts/deploy_subnet_under_calibration_net/README.md Co-authored-by: Jorge M. Soares <547492+jsoares@users.noreply.github.com> --- scripts/deploy_subnet_under_calibration_net/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/deploy_subnet_under_calibration_net/README.md b/scripts/deploy_subnet_under_calibration_net/README.md index bb7fe79d1..af05962d9 100644 --- a/scripts/deploy_subnet_under_calibration_net/README.md +++ b/scripts/deploy_subnet_under_calibration_net/README.md @@ -13,7 +13,7 @@ To run this script locally, you need to first manually prepare the environment a 3. Run `deploy.sh local` to deploy IPC locally. Please also notice that -1. The `deploy.sh` is only for running on Linux. If you are using a Mac, you need to disable all `apt` based dependency installation. You may also need to install bash (version >= 5) to run this script since the script isn't fully compatible with zsh (default shell on Mac). Also, make sure you habe openssl packages installed (`brew install openssl@3`). +1. The `deploy.sh` is only for running on Linux. If you are using a Mac, you need to disable all `apt` based dependency installation. You may also need to install bash (version >= 5) to run this script since the script isn't fully compatible with zsh (default shell on Mac). Also, make sure you have the openssl packages installed (`brew install openssl@3`). 2. The automated dependency installation isn't guarantee to work 100% time. If you encountered any dependency installation issue, please refer to the script and retry. Usually you can resolve the issues by creating a new terminal, sourcing `~/.bash.rc`, etc. 3. Depends on the RPC endpoint's quality of service for the calibration net, your command may or may not succeed when interacting with the RPC endpoint. Sometimes you will get rate limited. In that case, you can choose a different calibration provider URL from [Chainlist](https://chainlist.org/?search=calibration&testnets=true) to replace the value of `RPC_URL` variable in the script, then retry it. 4. You need to manually install nodejs and npm. The reason is that we need to use very recent version of nodejs and it's usually not included with the Linux distribution. It's recommended that you use nvm (Node version manager) to manage your nodejs installation.