Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI invocations for non-captcha tests? #150

Closed
zstumgoren opened this issue Aug 26, 2021 · 2 comments
Closed

CI invocations for non-captcha tests? #150

zstumgoren opened this issue Aug 26, 2021 · 2 comments

Comments

@zstumgoren
Copy link
Member

Related to #149

@palewire raised the below questions about test invocations in a CI and non-CI context in #147

How would we handle the command-line options to create cases where:
A) Only CAPTCHA tests should be run
B) Only login-only based "slow" webtests should be run
C) All webtests, regardless of login method, should be run

For A, pyest --runslow -m captcha should work. This would not be run on CI, but is intended for manual runs by core devs and contributors (ideally prior to merging to master).

For B and C, one option would be to add yet another pyest marker such as noncaptcha to all webtests without a captcha.

We could then selectively target tests as below:

# Run all slow tests except for CAPTCHA-based
# For use on CI
pytest --runslow -m noncaptcha 

# For local dev
# Run all tests
pytest --runslow  

# Run only CAPTCHA
pytest --runslow -m captcha 

The above assumes that we devise a solution for passing in login credentials via environment variables, per #149.

@palewire
Copy link
Contributor

This sounds like a reasonable set of next steps to me. I'll mark the no-captcha tests and reconcile the test commands in the Makefile with what you've outlined above. We can deal with the credentials stuff for CI after that.

@palewire
Copy link
Contributor

This is done and can be closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants