Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Test-executive] Upload test results to buildkite #12900

Open
4 tasks
dkijania opened this issue Mar 23, 2023 · 0 comments
Open
4 tasks

[Test-executive] Upload test results to buildkite #12900

dkijania opened this issue Mar 23, 2023 · 0 comments
Assignees

Comments

@dkijania
Copy link
Member

dkijania commented Mar 23, 2023

This Issue address problem of visualizing test report in buildkite. Buildkite offers solution for storing test result in their premises. See test analytics page for more details. As a result of this issue all test results from test-executive should be stored there allowing anybody to track test cases results per PR. There are couple of challenges that need to be solved prior the upload:

  • 1. Buildkite requires single upload of test result, while our test build are spread around buildkite jobs. This means that there is no single process to gather test results after its run. Therefore we need to first invest in test-executive logs and enhance them with sufficient information which allows to determine test result and gather sufficient information about possible failures post mortem ([Test-executive] Log test-result instead of just print it  #12901)
  • 1. a) if possible we could also tweak logger to introduce 'perf' type of logger and gather some performance statistics form test-executive run. Buildkite report also may contains information about duration of particular section or operation. Thanks to that we could also log some important operations (e.g. initialization of node/ transaction speed), which can be a foundation for performance tests analysis.
  • 2. Once we finish with 1. there will be need to upload partial results (from single test-executive run) to some temporary but globally accessible storage. This step is necessary in order to build up merged report from all test-executive runs. First guess will be to use gcloud bucket. After test-executive run, we could have a very simple cli which will analyze log and convert it to buildkite compatible json and finally upload to gcloud bucket which uniquely identifies buildkite job and commit.
  • 3. Finally there need to be a tear down job in buildkite pipeline which will be triggered after all test-executive tests will finish. It will read uploaded json file combine it into collection and upload to buildkite test analysis module
@dkijania dkijania self-assigned this Mar 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant