[contributing] Running only tests in specific files? Quickly identifying failing tests?

[donmccurdy]

A couple questions I've run into while making PRs on this codebase. Wanted to write them down somewhere, and perhaps add them to a CONTRIBUTING.md at some point when there are answers —

Running only tests from a specific file or files

I'm aware of Tape's test.only(...) and use that regularly. Often though I find it helpful to run all tests from a specific file, or files. In plain Tape I would write...

npx tape path/to/{file1,file2}.spec.ts

... or something a little more elaborate, if using ts-node or tsx for TypeScript execution. Is there an equivalent way to do this with Ocular Dev Tools?

Identifying failing test(s)

When running yarn test I see a lot of unformatted tap output, in which it can be difficult to spot the failing tests, or the failing tests can overflow the terminal's scrollback buffer. I can use a formatter to make that more readable...

yarn test | npx tap-spec

... but this only works when running tests in Node.js. When testing in a browser the unformatted tap output goes to the browser's console, which may also overflow the scrollback buffer, so I can't see which test is failing in the browser. Is there a way to narrow test failures down, or see a summary of the failing tests?

[ibgreen]

No, I usually use yarn test browser and use the console search for Expected: (with a colon!) to find the actual errors.

Open to ideas, would love to have nicer tap integration or perhaps even move to jest.

[donmccurdy]

Unorganized thoughts, no particular response needed –

I used Tape for a long time, but have moved my own open source libraries to Ava while switching to ESM. Ava handles ESM much better, has a very similar API to Tape (which I like), and has just been nice to use. However, like Tape, Ava is not really designed to run tests in a browser.

My understanding is that Jest is similarly designed with the assumption that you'll run tests in Node.js, and obtain any DOM APIs through jsdom. This is obviously not great for testing WebGL libraries. I haven't worked a lot with Jest though, perhaps running tests in a real browser (not just debugging a Node.js process in Chromium Dev Tools!) is not as hard as I assume.

I'm not aware of any test framework that supports both Node.js and Web in a satisfactory way, which is very frustrating.

I've also switched from ts-node to tsx. In my own projects, it has vastly reduced the execution time of unit tests, and supports ESM better. Ava (and probably tape?) requires one of the two when executing tests written in TypeScript. This might be worth considering regardless of the testing framework, at some point.

---

tl;dr – I don't think I have any better suggestion at the moment. Perhaps adding tap-spec formatting to ocular-dev-tools would be easy, and would benefit other contributors, but I don't know if that formatted output would appear when testing in a web browser.

[ibgreen]

My starting point is that I haven't heard much about ava but am often hearing about jest. But I am not too updated on that space. I suppose I am expressing some preference for going with a bigger framework.

In fact the team just did a big internal move from tape to jest here at Foursquare and people seem excited about it. Many new good features both in terms of use/ productivity and in terms of how to write test cases. And I believe it works well on browser when run in puppeteer.

One of the vis.gl repos (nebula.gl) has long had a jest setup, that could possibly be used as a reference.

tsx sounds promising, faster tests are always valuable.

[donmccurdy]

Understood! I have no objection to Jest, certainly. I think Jest is currently the best framework for testing web applications, and gets well-deserved attention for that reason. Sometimes I find tools for testing applications are not so good at testing libraries where — for example — we need to run tests in multiple runtime environments. But if the Puppeteer/Jest integration works well, I agree that might be a great help!