system/load/testing: load a full validator set with max block sizes for a day

[Bidon15]

This is a kick-start of load testing on system level. This part involves developing a test-suite that is executed every new release:

Tasks

Preview Give feedback

1. Prepare robusta k8s to deploy a 100 validator set
2. It should contain metrics/traces/logging possibilities for the team to have observability
3. Develop a test-client that will constantly flow with TXs making every block full
4. The scenario is active for 24hr
5. A GH action that notifies on test execution status

[evan-forbes]

this was done on mocha when it was spammed with large IBC txs. we should probably repeat similar tests after mocha increases block size.