feat: DEVOPS-1867 otlp-collector endpoint generation in the node configs

[pavlops]

• Generating the otlp_collector_endpoint config in the nodes config.toml - beginning of the file (currently enforcing the otlp is part of all environments and configurations in GCP).
• z2 deployer get-config-file not affected, as the otlp_collector_endpoint is only added at the install/update process.
• Fix on the otlp_collector_endpoint var name for z2 join.

Tests with z2 deployer install:

z2 deployer ssh --select zq2-infratest.yaml -- cat /config.toml
Recompiling z2, please wait …
🦆 Running SSH command for zq2-infratest.yaml' ..
Using the project ID prj-d-zq2-devnet-c83bkpsd
Create the instance list for zq2-infratest
◇  Select target nodes
│  zq2-infratest-api-ase1-0-5155
│
Running SSH command on zq2-infratest nodes
🦆 Running the SSH command: 'cat /config.toml' ..
---
zq2-infratest-api-ase1-0-5155
otlp_collector_endpoint = "http://localhost:4317"
p2p_port = 3333
bootstrap_address = [ "12D3KooWFsmJMnsRjFPsGi1EZfcK4N62ia9gTRfKDZHMpRKQWKeZ", "/dns/bootstrap.zq2-infratest.zilstg.dev/tcp/3333" ]

Tests with z2 join:

cat zq2-infratest.toml
p2p_port = 3333
bootstrap_address = ["12D3KooWLe4CnsLgeEZnfgS44gdDKdsMEayE96qfU2csB8Lijp54", "/dns/bootstrap.zq2-infratest.zilstg.dev/tcp/3333"]
otlp_collector_endpoint = "http://localhost:4317"

Tests with z2 deployer get-config-file:

z2 deployer get-config-file zq2-infratest.yaml
Recompiling z2, please wait …
🦆 Getting nodes config file for zq2-infratest.yaml ..
Using the project ID prj-d-zq2-devnet-c83bkpsd
Create the instance list for zq2-infratest
Bootstrap picked: zq2-infratest-bootstrap-ase1-2-334d
Using the project ID prj-d-zq2-devnet-c83bkpsd
Using the bootstrap endpoint bootstrap.zq2-infratest.zilstg.dev
Config file for a node role validator in zq2-infratest
---
p2p_port = 3333
bootstrap_address = [ "12D3KooWFsmJMnsRjFPsGi1EZfcK4N62ia9gTRfKDZHMpRKQWKeZ", "/dns/bootstrap.zq2-infratest.zilstg.dev/tcp/3333" ]

Tested in devnet:

[github-actions]

Bencher Report

Branch	devops-1867
Testbed	self-hosted

Click to view all benchmark results

Benchmark	Latency	Benchmark Result milliseconds (ms) (Result Δ%)	Upper Boundary milliseconds (ms) (Limit %)
full-blocks-erc20-transfers/full-blocks-erc20-transfers	📈 view plot 🚷 view threshold	1,190.90 (-3.62%)	1,633.88 (72.89%)
full-blocks-evm-transfers/full-blocks-evm-transfers	📈 view plot 🚷 view threshold	377.56 (-22.69%)	705.17 (53.54%)
full-blocks-zil-transfers/full-blocks-zil-transfers	📈 view plot 🚷 view threshold	4,885.20 (+24.31%)	5,166.30 (94.56%)
process-empty/process-empty	📈 view plot 🚷 view threshold	9.78 (+7.17%)	11.44 (85.45%)

🐰 View full continuous benchmarking report in Bencher