Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature request: mechanism for (non-CI) extended tests #51

Open
mikeingold opened this issue Oct 27, 2024 · 1 comment
Open

Feature request: mechanism for (non-CI) extended tests #51

mikeingold opened this issue Oct 27, 2024 · 1 comment

Comments

@mikeingold
Copy link

I recently implemented use of this package on MeshIntegrals.jl. It's great for PR's, but I try to keep limit CI runtimes. I'd really like to be able to define:

  • a core set of benchmarks that will run in CI for all PR's, targeting a <10 minutes runtime, which means SUITE must be able to complete in <5 minutes.
  • an extended set of benchmarks that I can trigger to be run on my local dev machine for a more thorough analysis.

I suppose this could currently be achieved by adding another benchmarks script and targeting it manually, but wasn't sure if there could be any broader interest in making a more explicit feature solution.

@MilesCranmer
Copy link
Owner

There is the --filter option which might help with this?

You could also set an environment variable and use that to conditionally add more or less tests to the SUITE group

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants