[DRUNX] Espressif QEMU runtime tests.

[cederom]

• @fdcavalcanti gave us a hint to extend DRUNX with Espressif's QEMU for NuttX firmare images runtime verification with no hardware. Espressif test that way internally.
• ESP QEMU srouce code [1] https://github.com/espressif/qemu.
• All tests are Pytest [3] based, but heavily use the pytest-embedded [2] plugin, more specifically the pytest-embedded-nuttx plugin (supports Nuttshell parsing, flashing, etc).
• Espressif use docker images for testing so it cannot be intergrates into GH CI at this point directly.

[1] https://github.com/espressif/qemu
[2] https://github.com/espressif/pytest-embedded
[3] https://docs.pytest.org/en/stable/

[cederom]

Copy-Paste from #15730 for context :-)

@fdcavalcanti: @cederom there are plenty of QEMU tests on Espressif side, all pytest based. If there is computing power available for running more QEMU tests, we could think of a way to integrate those tests to Nuttx. Had a brief talk about this with @lupyuen, so I'm tagging him here.

@cederom: Thank you @fdcavalcanti that would be great!!

Ultimately DRUNX will work on home computers of interested users, @lupyuen already has working prototype that simply run the CI scripts that we run on GitHub. There is also Dashboard that sums up the builds from various places. Ultimately there will be as many computing powers as tehere are users testing :-)

PyTest is a well standardized choice :-)

Are those QEMU images available to fetch from some public repo or even build by hand?

What platforms are supported?

Do you have some sort of bootstrap for them?

ESP QEMU tests would perfectly fit the build and runtime in a situation where no hardware is available or necessary.. maybe even GH CI if not too expensive to run :-)

How much place would it take to place them in nuttx-apps/testing?

I have created dedicated issue here: #15733 :-)

Thank you!! :-)

@fdcavalcanti: We use QEMU available for ESP32, ESP32S3 and ESP32C3. You can find them here.

All tests are Pytest based, but heavily use the pytest-embeded plugin, more specifically the pytest-embedded-nuttx plugin (supports Nuttshell parsing, flashing, etc).

Simple tests like building, booting and testing if nsh> is responsive exist. Which is enough to know if the build is at least working to this extent. This allows testing most defconfigs that do not require external hardware, WiFi and BLE.
We also test bootloader combinations: reuse all defconfigs and apply MCUBoot instead of simple boot.

However there are more complex tests that use some customized application similar to nuttx-apps. Most would be Espressif related since they target our boards and don't follow some rules (like directly calling GPIO instead of using gpio.h when testing motor driver, capture driver, etc). That's something to be analyzed internally first.

I'm afraid the current NuttX CI image is not compatible. We use a Docker image that has the build system and all plugins required for the tests.