You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Allow the selection of which tests to run depending on how expensive (time consuming they are). This should take the form of a flag to the tester. For example, in Conjure, --limit_time=5 runs only those tests that have an expected run time of <=5 seconds.
CI should be modified to run both a quick version of the tests, and all the tests.
Background
In #572, I proposed using Rust's builtin #[ignore] attribute 1 to disable by default a test case that took too long to run.
However, we want a more fine grained approach to select which tests to run than whether a test is ignored or not.
Oz suggested the following:
I like the idea of having infrastructure to select which test to run depending on how expensive the test is though. In Conjure I optionally add a rough expected time (in seconds) for each test case. Then running --limit_time=5 runs only those tests that have an expected run time of <=5 seconds. Default value is 0.
Could be better than the coarse grained quick/all. Quick can then be an alias for <=1 or whatever and all can be actually all.
The text was updated successfully, but these errors were encountered:
Summary
Allow the selection of which tests to run depending on how expensive (time consuming they are). This should take the form of a flag to the tester. For example, in Conjure,
--limit_time=5
runs only those tests that have an expected run time of <=5 seconds.CI should be modified to run both a quick version of the tests, and all the tests.
Background
In #572, I proposed using Rust's builtin
#[ignore]
attribute 1 to disable by default a test case that took too long to run.However, we want a more fine grained approach to select which tests to run than whether a test is ignored or not.
Oz suggested the following:
The text was updated successfully, but these errors were encountered: