Tests Hints & Tips
This document contains some insight and examples of flaky tests and what is the reason.
Sometimes writing a test is straight forward logically but the test environment is setup to run as fast as possible and faster than the human interaction with the UI so sometimes one frame can be enough for a test to be failing, although correct. In other words - there are some specific cases or implementations that can lead to a passing test on a faster machine and failing test on a slower one.
- Some tests might be flaky and this can be more easily caught with clumsy (https://github.com/jagt/clumsy/releases/tag/0.3rc3). Use the options according to your needs but Throttle should be enough and also set the Chance to 100%. Remember to click on Start.
Note: You can specify the port for clumsy to throttle - check the Presets next to the Start button.
- A useful utility function is
retryIfFailswhich can be found in https://github.com/CoherentLabs/GameUIComponents/blob/master/tools/tests/actions.js. It is used to retry a function until it passes successfully or fails. It is useful, for example, if some responses should be completed before another action, like click to trigger another effect or a submit of a form.
Each example has references and if you are in a position of creating or finding one, please get back here and share.
Some time is needed for the form to give a response and set the textContent of the .response Element. Using retryIfFails to check the content of this element is recommended. Given enough retries, this will show if the expected response / value is incorrect or the test will succeed because the test case is correct.
A simple use of dispatchEvent can go sideways when the event isn't dispatched right away. Again, using retryIfFails is recommended because the application might not respond as fast as the following assert requires it to but when it does - the test passes.
The XHR might not be resolved before the assert is called to check if the callback was called as it should be. Here we have wrapped the "double" test xhr.onloads in a Promise and when the inner one is resolved, only then the assert will follow.
When a form submission is requested with a loadend event through the submitForm function, the adding of the new event listener should be awaited and the Submit button clicked. This is needed when the test checks the response content through the UI element that holds the response as text.