feat(test_framework): add an in-game lua testing framework #2515
Conversation
| local startZ = midZ - zStep * n / 2 | ||
|
|
||
| -- make two lines of units facing each other | ||
| SyncedRun(function(locals) |
There was a problem hiding this comment.
locals includes all local variables that the function would have access to. this is awkward, but pretty much the best we can do right now.
This function is serialized, sent to synced, deserialized, executed, return is serialized, sent to unsynced, and then return is deserialized.
Without debug in synced (beyond-all-reason/spring#867), we can't transmit upvalues; if we could, we could basically write these functions as we normally would any function.
| local badtype = {thread = true, userdata = true, cdata = true} | ||
| local getmetatable = debug and debug.getmetatable or getmetatable | ||
| local pairs = function(t) return next, t end -- avoid using __pairs in Lua 5.2+ | ||
| local keyword, globals, G = {}, {}, (_G or _ENV or widget or gadget) |
There was a problem hiding this comment.
this line is slightly modified to better handle the environments that it's running in
| @@ -0,0 +1,360 @@ | |||
| local serpent = serpent or VFS.Include('common/luaUtilities/serpent.lua') | |||
|
|
|||
There was a problem hiding this comment.
this file has most of the random functions I could split off from the main widget/gadget files. some could be split into further small files, and some could probably be integrated into existing util files.
| timeoutLeft = timeout, | ||
| } | ||
|
|
||
| local resumeOk, resumeResult = coroutine.yield() |
There was a problem hiding this comment.
coroutines are fun
salinecitrine
force-pushed
the
testing_framework
branch
2 times, most recently
from
6f9b141 to
4fef427
Compare
|
Would it make any sense (on headless) to listen on a socket and allow raw lua / return json comms to the framework? |
In the context of this testing framework, I think that it would at least make sense to allow external access to chat commands (so we could As for general lua code execution, I can't think of any use cases relevant to this PR/code, but I'm assuming there are plenty of cool things that could be done with it. |
This is currently just a demonstration, and is not ready for production use yet. Tests can be executed with `/runtests <pattern1> ... <patternN>`, where any test file (excluding file extension) that matches any pattern is included. `/runtests` by itself will run all tests. Features: * setup/cleanup functions can be defined in each test file. cleanup will always run, even if the test code fails. * nicely formatted test results, including errors with line information * two ways to call synced code: * `x = SyncedProxy.<Y>.<Z>()`, which is equivalent to `x = Y.Z()` executed in a synced context. * `x = SyncedRun(function() ... end)`, which allows execution of arbitrary code in a synced context. This is faster for things involving loops or many synced calls. * wait API to allow in-game actions to happen during a test. For example, `Test.waitFrames(5)` will wait 5 frames before resuming test execution. * `spy(parent, target)`, which allows tracking all calls to a spied function. * isolated environments for each test, while still having access to most globals accessible to widgets. Included are several sample widget tests, of a few different kinds: * balance tests, that set up fights between units * an example demonstrating different the wait API * a test for a user widget (gui_battle_resource_tracker) * several tests for a built-in widget (gui_selfd_icons) * some of these have sections testing bugs that are not fixed yet, and are thus commented out. The serpent library is included for straightforward serialization.
This allows tests to run pre-game.
right now it breaks wait timeouts
Co-authored-by: sprunk <[email protected]>
Calling `widgetHandler:EnableWidget(widgetName, true)` instead of `widgetHandler:EnableWidget(widgetName)` will now allow access to local variables as if they were globals, with `widget.localVariable`. Read and write are both supported.
This allows us to leave the widget file untouched and still test it.
This more closely matches how they would be restored with the debug library (when that becomes an option).
The previous behavior was between coroutine resumes, which led to callins occasionally being lost.
Previously, line information was lost because the distance was set too high.
Previously, sometimes the widget wouldn't have enough time to act before the test checked results.
It writes in mocha JSON format, for compatibility with other tools.
The trigger for running tests is disabled until we want to enable them. In order to run, these scripts (like other workflow scripts) must be in the main branch (master).
Just in case it was already on, this will disable it. It will then be re-enabled later if necessary.
Test.mock() works very similarly to Test.spy(). It takes a parent table, a target function name, and an optional replacement function. Whenever the target function is called, the call is recorded, and then if a replacement function was specified, that is called. The original is never called.
salinecitrine
force-pushed
the
testing_framework
branch
from
26e8ce3 to
9ec24fa
Compare
|
Closing, as this has now been refactored and split up into four PRs: |
This is currently just a demonstration, and is not ready for production use yet. Expect bugs and other incompleteness.
Tests can be executed with
/runtests <pattern1> ... <patternN>, where any test file (excluding file extension) that matches any pattern is included./runtestsby itself will run all tests.Features:
x = SyncedProxy.<Y>.<Z>(), which is equivalent tox = Y.Z()executed in a synced context.x = SyncedRun(function() ... end), which allows execution of arbitrary code in a synced context. This is faster for things involving loops or many synced calls.Test.waitFrames(5)will wait 5 frames before resuming test execution.spy(parent, target), which allows tracking all calls to a spied function.Included are several sample tests, of a few different kinds:
There are two primary components:
luaui/Widgets/dbg_test_framework.lua: the test runnerluarules/gadgets/dbg_test_framework_proxy.lua: for running synced codeThe serpent library is included for straightforward serialization.
Test steps