You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This Issue address problem of visualizing test report in buildkite. Buildkite offers solution for storing test result in their premises. See test analytics page for more details. As a result of this issue all test results from test-executive should be stored there allowing anybody to track test cases results per PR. There are couple of challenges that need to be solved prior the upload:
1. Buildkite requires single upload of test result, while our test build are spread around buildkite jobs. This means that there is no single process to gather test results after its run. Therefore we need to first invest in test-executive logs and enhance them with sufficient information which allows to determine test result and gather sufficient information about possible failures post mortem ([Test-executive] Log test-result instead of just print it #12901)
1. a) if possible we could also tweak logger to introduce 'perf' type of logger and gather some performance statistics form test-executive run. Buildkite report also may contains information about duration of particular section or operation. Thanks to that we could also log some important operations (e.g. initialization of node/ transaction speed), which can be a foundation for performance tests analysis.
2. Once we finish with 1. there will be need to upload partial results (from single test-executive run) to some temporary but globally accessible storage. This step is necessary in order to build up merged report from all test-executive runs. First guess will be to use gcloud bucket. After test-executive run, we could have a very simple cli which will analyze log and convert it to buildkite compatible json and finally upload to gcloud bucket which uniquely identifies buildkite job and commit.
3. Finally there need to be a tear down job in buildkite pipeline which will be triggered after all test-executive tests will finish. It will read uploaded json file combine it into collection and upload to buildkite test analysis module
The text was updated successfully, but these errors were encountered:
This Issue address problem of visualizing test report in buildkite. Buildkite offers solution for storing test result in their premises. See test analytics page for more details. As a result of this issue all test results from test-executive should be stored there allowing anybody to track test cases results per PR. There are couple of challenges that need to be solved prior the upload:
The text was updated successfully, but these errors were encountered: