Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Mini-SBIBM #1396

Open
4 tasks
manuelgloeckler opened this issue Feb 20, 2025 · 0 comments
Open
4 tasks

Improve Mini-SBIBM #1396

manuelgloeckler opened this issue Feb 20, 2025 · 0 comments
Labels
enhancement New feature or request

Comments

@manuelgloeckler
Copy link
Contributor

Improve Mini-SBIBM

Mini-SBIBM should offer an easy and fast way to test if certain modifications impact the performance of the algorithms. For this, a small set of tasks are implemented on which different methods or configurations can be benchmarked against each other. This can be directly launched via pytest and is thus easily accessible for any developer.

It has a few modes to test different method classes, but here, one can improve quite a bit. There are the following areas of improvement:

  • Think about what level of customizability would be helpful. Mini-SBIBM should not be a full-baked benchmark to check if something in development works reasonably well quickly.
  • Think about if we want to allow this set of customizations through pytest API, without requiring to change the code directly. This is currently rather static and can be changed in the code. One can only select different "modes" that run a fixed set of i.e. NPE,SNPE, NRE configurations (depending on what is selected)
  • Properly organize configurations in some way.
  • Simple tasks for more "specialized" applications e.g. IID data ...
@manuelgloeckler manuelgloeckler added the enhancement New feature or request label Feb 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant