Mina CI Performance Infrastructure

[dkijania]

Request for help in setup for performance infrastructure which our CI will utilize in below use cases:

1. Measuring Performance on PR against historical data:

a) New PR is pushed and CI is started.
b) All *Perf Jobs are downloading weighed mean from performance database and comparing against current measurement. If it exceeds threshold we are failing a job
c) If failure is acceptable then we adding approve flag and rerun bk pipeline

2. Adding new data or overriding alert

a) Nighty is scheduled on compatible/develop/master branches and performance jobs are ran
b) perform the same comparisons like in use case 1
c) Add actual measurement to performance db

3. Monitoring and Visualization and Alerting

Anyone can view historical data with some nice graphs. Additionally it would be great to receive alert on slack if Perf job failed. However, it is not required since nightly will fail anyhow

2. Components

Based on experience from sdk team we think that good spike for such infra would be to use Influx db and grafana

• influx db as performance data storage
• grafana for visualization and possible monitoring

In order to achieve such solution we need to

• install influx client on mina toolchain which should be able to send data to influx db (Update toolchain with influx db cli #16202)
• hosting influx db - we will use already setup o1labs instance
• hosting separate grafana instance (or reuse shared one) - we will use already setup o1labs instance
• enhance bk agent env vars with INFLUX_TOKEN, INFLUX_HOST, INFLUX_ORG
• unify all benchmarks in code
• add CI perf jobs
• introduce separate bk agent just for perf tests