Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[T.1.2] Expanding Machine Configuration and Validation. #176

Open
smarr opened this issue Dec 4, 2021 · 0 comments
Open

[T.1.2] Expanding Machine Configuration and Validation. #176

smarr opened this issue Dec 4, 2021 · 0 comments
Labels
help-wanted We'd like help with this issue, and are happy to provide guidance Proposal Work proposed as part of a (rejected) funding application

Comments

@smarr
Copy link
Owner

smarr commented Dec 4, 2021

ReBench already configures benchmarking machines to minimize measurement error. The [Krun benchmark runner](https://github.com/softdevteam/krun} supports additional configuration settings and operating systems. This includes active thermal management, additional setting validation, and perturbation detection.

Additionally, we currently do no have any active setup validation based on basic and well understood benchmarks, which would be a recommended best practice.

For instance for a 20min benchmark job, it would be sensible to have a quick check at the beginning to see whether basic expectations of the machine can be replicated.

It would significantly increases confidence into the benchmarking setup.

@smarr smarr added help-wanted We'd like help with this issue, and are happy to provide guidance Proposal Work proposed as part of a (rejected) funding application labels Dec 4, 2021
@smarr smarr moved this to WP1. Improving Benchmarking Accuracy and Result Contextualization in ReBench for a wider community Mar 19, 2023
@smarr smarr moved this to 📋 Backlog in ReBench Summer 2024 May 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help-wanted We'd like help with this issue, and are happy to provide guidance Proposal Work proposed as part of a (rejected) funding application
Projects
Status: 📋 Backlog
Status: WP1. Improving Benchmarking Accuracy and Result Contextualization
Development

No branches or pull requests

1 participant