Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix E2E test flakiness #873

Closed
sidvishnoi opened this issue Jan 29, 2025 · 2 comments · Fixed by #879
Closed

Fix E2E test flakiness #873

sidvishnoi opened this issue Jan 29, 2025 · 2 comments · Fixed by #879
Assignees
Labels
area: tests Improvements or additions to tests

Comments

@sidvishnoi
Copy link
Member

As discovered in #864 (comment), #864 (comment) and #864 (comment).

@sidvishnoi sidvishnoi added the area: tests Improvements or additions to tests label Jan 29, 2025
@sidvishnoi sidvishnoi self-assigned this Jan 29, 2025
@sidvishnoi sidvishnoi changed the title Fix test flakiness Fix E2E test flakiness Jan 29, 2025
@sidvishnoi sidvishnoi moved this from Backlog to In Progress in Web Monetization Extension Jan 29, 2025
@sidvishnoi
Copy link
Member Author

After numerous failed attempts today trying to clear extension state after each file, it's best to isolate tests for now, and create new browser for each test (not per file) until Playwright supports not reusing worker or cleanup per file.
This would make tests much slower, but at least they won't be flaky.

Another alternative would be to let go of fixtures and setup & teardown at top level manually, but that would be more trouble. Once Playwright has file level scope, we can simply replace scope in fixtures then.

@sidvishnoi
Copy link
Member Author

For performance with isolation, we can then increase parallelism with more workers and enabling fullyParallel.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: tests Improvements or additions to tests
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

1 participant