Skip to content

Tests Hints & Tips

Tony Gruev edited this page Sep 19, 2023 · 1 revision

Tests Hints & Tips

This document contains some insight and examples of flaky tests and what is the reason.

Sometimes writing a test is straight forward logically but the test environment is setup to run as fast as possible and faster than the human interaction with the UI so sometimes one frame can be enough for a test to be failing, although correct. In other words - there are some specific cases or implementations that can lead to a passing test on a faster machine and failing test on a slower one.

Useful Utilities

  • Some tests might be flaky and this can be more easily caught with clumsy (https://github.com/jagt/clumsy/releases/tag/0.3rc3). Use the options according to your needs but Throttle should be enough and also set the Chance to 100%. Remember to click on Start.

Note: You can specify the port for clumsy to throttle - check the Presets next to the Start button. image

Example Problematic Tests

Each example has references and if you are in a position of creating or finding one, please get back here and share.

Form Submit / Response Delay

Some time is needed for the form to give a response and set the textContent of the .response Element. Using retryIfFails to check the content of this element is recommended. Given enough retries, this will show if the expected response / value is incorrect or the test will succeed because the test case is correct.

Reference: https://github.com/CoherentLabs/GameUIComponents/commit/5a6ffe207ec0f41367b8227cf267de0e70532d2c#diff-b91e0d115615da073c2c5cb91b075f62f494df76cd47fb3811170538c9575d82R374

Triggering events with dispatchEvent

A simple use of dispatchEvent can go sideways when the event isn't dispatched right away. Again, using retryIfFails is recommended because the application might not respond as fast as the following assert requires it to but when it does - the test passes.

Reference: https://github.com/CoherentLabs/GameUIComponents/commit/5a6ffe207ec0f41367b8227cf267de0e70532d2c#diff-780baad32039d18eb3d6ffc9e6d1f001b5f2d47689e67358aeb1dec182203a48R99

XHR Response / Resolve

The XHR might not be resolved before the assert is called to check if the callback was called as it should be. Here we have wrapped the "double" test xhr.onloads in a Promise and when the inner one is resolved, only then the assert will follow.

Reference: https://github.com/CoherentLabs/GameUIComponents/commit/f96ea7c04afab02dd81c707219817eef8a384812#diff-34a5523aac522cffba1c8a78c6fc2a8f4cdc178ba155ba1f015d273c99ba79f5R35

Form with loadend event (utility fix)

When a form submission is requested with a loadend event through the submitForm function, the adding of the new event listener should be awaited and the Submit button clicked. This is needed when the test checks the response content through the UI element that holds the response as text.

Reference: https://github.com/CoherentLabs/GameUIComponents/commit/f96ea7c04afab02dd81c707219817eef8a384812#diff-15423e4c1a5e7a5b3734ed46a36901c0f870ce608b8424a8a3304ee87540bcf7R19

Fixed tests examples: