[WEB Installer] Lolin S3 mini loads but does't boot wled

[marcobrianza]

Hello, just tested the web installer with release 0.14.2 on Lolin s3 mini
The installer does its thing but the board does not boot wled afterwards.
The serial loops with these messages

ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0x3 (RTC_SW_SYS_RST),boot:0x8 (SPI_FAST_FLASH_BOOT)
Saved PC:0x403cd98e
SPIWP:0xee
mode:DIO, clock div:1
load:0x3fce3808,len:0x43c
load:0x403c9700,len:0xbec
load:0x403cc700,len:0x2a3c
entry 0x403c98d8

FYI tasmota web installer works correctly on this board

[lost-hope]

Does it work, when flashed from PlatformIO?

[marcobrianza]

Using PIO I noticed that there is no profile for ESP32-S3 at 4MB Flash but only 8M and above. (The S3 mini is 4MB) so the S3 mini does not work also in PIO.

I tried an S3 PRO (16MB) and from PIO it works, but not from the web installer. I tried 0.14.1 and 0.14.2 and get different kind of errors

[blazoncek]

@lost-hope I think we may need to rethink the way web installer is constructed.

WLED binaries themselves are not dependent on the flash size (though they may not fit on every partition) so there can only be limited number of them for particular chip (i.e. 2 for ESP32, one with PSRAM and one without, excluding Ethernet).
So instead of hiding flash size form a user it could be part of the install process. Partition map binaries AFAIK are the same for every chip, their flash offset may vary though.

Classic ESP32 needs 2 files as PSRAM version requires GPIO16 and GPIO17.
C3 only ever needs 1 build as there are no HW variants requiring otherwise.
Even S2 could use only one build as there are no differences if using PSRAM or not.
I am not sure about S3 as I do not have one.

We can also integrate audio into every (compatible) build.

@marcobrianza there will never be every possible board configuration in platformio.ini. If you need particular configuration use platformio_override.ini and extend existing environment.

[lost-hope]

@blazoncek already thought of that cause of the esp8266 partitions.
the flash size selection needs to be done manually, so by the user, cause there is no automated detection.

and that will introduce the issue with users selecting the wrong size.

Partition map binaries AFAIK are the same for every chip, their flash offset may vary though.

So we only need 5 partition files: 1M 2M 4M 8M 16M

i think i have an idea on how to handle it, but i am not completely sure if that works out.

Then we have 6 Binaries for the chips (ESP8266, ESP32,ESP32 PSRAM, ESP32-S2 EPS32-S3 ESP32-C3) to cover all of the possible devices right.
plus variants for Ethernet and Audioreactive.
What about the new Wrover binary in the newest release?

[blazoncek]

and that will introduce the issue with users selecting the wrong size.

The user has the ESP still physically attached to the computer. I find this to be unproblematic as the user can flash again and again until correct flash size is selected.

ESP8266 is a different beast as the flash size is embedded in the binary and esptool extracts it.

WROVER needs separate binary since otherwise WLED defaults to GPIO16 for LEDs. PSRAM models of ESP32 use GPIO16 and GPIO17 for PSRAM and using regular ESP32 causes crashes upon boot. If we change default GPIO for LEDs (could also be dynamic) then only one binary would be needed.

[blazoncek]

@marcobrianza THANK YOU.

[blazoncek]

I now have all compatible ESP variants with various flash sizes and was able to determine and verify how flashing works (using PIO & esptool).

As mentioned above ESP8266 is a special case since everything is included within firmware binary (including flash size, CPU clock, flash clock, etc) so each variant needs its own binary file (currently 6 in total: 80MHz 1MB, 80MHz 2MB, 80MHz 4MB, 160MHz 1MB, 160MHz 2MB and 160MHz 4MB, but the future may bring 4 additional files for 8MB and 16MB flash). There is a possibility to change CPU clock speed at runtime but current firmware can't do that. Yet.

ESP32 and its variants S2, S3 and C3 use bootloader, partition map and firmware files to successfully flash a device.

In my testing the bootloader was identical regardless of features used on a particular board, so one bootloader per ESP variant seems reasonable choice. Bootloader needs to be flashed at address 0x00000 for ESP32, ESP32-S3 and ESP32-C3 but at address 0x01000 for ESP32-S2.

Partition map binary depends on the amount of flash memory a particular ESP device has. So far I've seen ESP32 (and variant) devices with 2MB, 4MB, 8MB and 16MB of flash. 2MB is unusable (even though it could run WLED) and will not be supported by WLED. Partition map needs to be flashed at address 0x08000 for all ESP32 variants.

Firmware binaries depend on the features selected. For classic ESP32 we can have 3 main binaries: plain, with PSRAM and with Ethernet (each of them can/may have up to 6 usermods). Ethernet needs its binary because it needs (allocates) quite a few commonly used GPIO pins, otherwise it could be included in plain binary. PSRAM support needs its own binary for chip revisions < rev.3 which have hardware bug, the fix for hardware bug increases firmware binary by 100kB. For newer chips plain will (or should) correctly select and use PSRAM if available (from 0.15.0-b2 onward). 0.15.0-b2 (and newer) will also correctly detect ESP32-PICO variant which uses GPIO16 for flash.
S2 and S3 may only need a single binary as PSRAM support can be detected and used without workarounds. The firmware binary file is the same regardless of flash size but may differ for S3 depending on the PSRAM type used (QSPI, OSPI, QPI, OPI, etc). These chips do not support Ethernet. (I do not own a S2 or S3 device without PSRAM and all devices I own have QSPI PSRAM.)
C3 has no PSRAM or Ethernet support so it only needs one firmware binary.

As I have no knowledge of how web installer works I am asking anyone who knows that, if an update to web installer could be made (and PRed) to include above information. I know that @softhack007 has a lot of experience with S3 so he may add something for devices with OPI.

References on PSRAM for S3 (we use Arduino framework so some configuration may be hidden and not available):
https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/api-guides/flash_psram_config.html
https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/api-guides/external-ram.html

[softhack007]

I know that @softhack007 has a lot of experience with S3 so he may add something for devices with OPI.

@blazoncek @lost-hope I remember that espressif docs say that -S3 with qspi and opi modes require a different bootloader. I have some devices here (no psram, qspi, opi) so I'll test them and report back.

How do you want to handle "USB CDC" vs "serial UART usb" devices? Improv (setting wifi password after install) needs to communicate with WLED over Serial, so it's important that the right serial driver is active.

[softhack007]

ESP32 and its variants S2, S3 and C3 use bootloader, partition map and firmware files to successfully flash a device.

Additionally, you need boot_app0.bin at 0xe000.

https://github.com/espressif/arduino-esp32/blob/49b94644c693df22f0a15ffb9454e2d7c3e6c679/tools/platformio-build.py#L212

[blazoncek]

I am leaning towards only supporting 2 or maximum 3 S3 boards as otherwise we'll get into thousand devices mess soon.
I recommend:

• 8MB/16MB OPI/OSPI flash with 8MB OPI/OSPI PSRAM (something @softhack007 has)
• 4MB QSPI flash with 2MB/4MB QSPI PSRAM (S3-Zero or Lolin S3 mini and some custom devices I received)
• optionally 4MB/8MB flash no PSRAM
  All with CDC serial only (as it is most common and easy to implement).

Unfortunately that would mean users with unsupported devices will need to compile themselves. Those that are unable to do so should be instructed to get supported device instead or seek help on the forum or Discord.

[lost-hope]

How do you want to handle "USB CDC" vs "serial UART usb" devices? Improv (setting wifi password after install) needs to communicate with WLED over Serial, so it's important that the right serial driver is active.

to be honest the Improv wifi setting is nice but can also be seen as a addition. If it woks: Nice. But as the possible list of binaries is already pretty long, i would decide against adding CDC and non CDC binaries...

[blazoncek]

Additionally, you need boot_app0.bin at 0xe000.

https://github.com/espressif/arduino-esp32/blob/49b94644c693df22f0a15ffb9454e2d7c3e6c679/tools/platformio-build.py#L212

That's interesting. Where did you find that info?
When I flashed S2 (and S3) there was no such file present in .pio/build/env/ folder.

I also did a binary diff on bootloaders for all 3 variants of S3 build environment and they were identical on my system.

Flash will be erased from 0x00000000 to 0x00003fff...
Flash will be erased from 0x00008000 to 0x00008fff...
Flash will be erased from 0x0000e000 to 0x0000ffff...
Flash will be erased from 0x00010000 to 0x0016dfff...
...
Wrote 14768 bytes (10149 compressed) at 0x00000000 in 0.2 seconds (effective 544.3 kbit/s)...
...
Wrote 3072 bytes (128 compressed) at 0x00008000 in 0.0 seconds (effective 541.7 kbit/s)...
...
Wrote 1431696 bytes (963320 compressed) at 0x00010000 in 10.8 seconds (effective 1061.2 kbit/s)...

The output of actual flashing of S3-Zero board.

[softhack007]

When I flashed S2 (and S3) there was no such file present in .pio/build/env/ folder.

I think this file is "generic" for all esp32 variants. It's not in the .pio/build folder because that one is directly taken from arduino-esp32. Its here https://github.com/espressif/arduino-esp32/blob/49b94644c693df22f0a15ffb9454e2d7c3e6c679/tools/partitions/boot_app0.bin

The info that this one is needed came from a discussion with Wladi (@wled-install) - after erase_flash, app0 must be uploaded together with bootloader and partitions, unless you are using a combined bootloader file which includes bootloader, app0 and partitions.

[blazoncek]

I'll be damned!

[lost-hope]

i added the boot_app0 for these 3 in a separate branch: https://lost-hope.github.io/WLED-WebInstaller/

[softhack007]

Just as additional info - found a nice summary of board variants: https://espeasy.readthedocs.io/en/latest/ESPEasy/ESPchips.html#quad-octal-spi-mode

[softhack007]

I have some devices here (no psram, qspi, opi) so I'll test them and report back.

Update: i've compared all three variants. Results:

• bootloader files are the same (observation from @blazoncek confirmed). It looks like only the basic flash mode (dio, qio) has influence on the bootloader.
• WLED firmware image is different for all three variants. I've successfully run the qspi PSRAM firmware on a no-PSRAM board, so it seems that we can indeed "get away" with two firmwares only: one for qspi PSRAM / no PSRAM, one for opi PSRAM. Flash size (4M/8M/16M) can be handled with different partition.bin files.

[blazoncek]

All of these boards support "Octal SPI" flash (per Espressif) but only in external implementation so Quad SPI is likely more common.

FYI: What I observed a while ago on my batch of 50 cheap ESP01 was that they were not able to boot when flashed with common dio mode. Unfortunately I had to revert to (slower) dout mode. Same might hold true for ESP32 variants. This is also most likely to be stored in bootloader as SPI can be configured (via SPI commands) to operate in dual/quad/octal modes.

I am wondering if Octal PSRAM can function in (slower) qspi mode?

These are excerpts from technical reference manuals.

[softhack007]

Hi,

FYI: What I observed a while ago on my batch of 50 cheap ESP01 was that they were not able to boot when flashed with common dio mode. Unfortunately I had to revert to (slower) dout mode. Same might hold true for ESP32 variants. This is also most likely to be stored in bootloader as SPI can be configured (via SPI commands) to operate in dual/quad/octal modes.

Partly "yes". Some "classic" ESP32 actually require dout bootloader - most likely due to old flash chips. I'm using a lot of newer "WROOM-E" boards, and these all work perfectly with "qio" quad mode.

But for S3/S2/C3, they always "worked for me" in qio mode, only one (-C3) became more stable with dio mode.

I am wondering if Octal PSRAM can function in (slower) qspi mode?

Espressif says "no" - and this is also my observation. It seems that you cannot talk to "octal PSRAM" in qspi mode.

[blazoncek]

A good read (with links) : https://hackaday.com/2017/10/01/trouble-flashing-your-esp8266-meet-dio-and-qio/

[softhack007]

About 8266 - I have some new "ESP01S" with 4MB here, and can run them at 80Mhz flash, qio mode, 160Mhz CPU.
So it seems that there are different generations of flash chips in the wild, and older ones want it slower....

[blazoncek]

About 8266 - I have some new "ESP01S" with 4MB here, and can run them at 80Mhz flash, qio mode, 160Mhz CPU. So it seems that there are different generations of flash chips in the wild, and older ones want it slower....

Yes, have those too, though they were 4 times as expensive as the other batch. 😄

[blazoncek]

i added the boot_app0 for these 3 in a separate branch: https://lost-hope.github.io/WLED-WebInstaller/

@lost-hope did you receive any feedback? Did you also manage to make any progress regarding the change to separate ESP32 into flash size + binary selection?

So far we know that we need:

• bootloader (determines flash mode: DIO, QIO, DOUT, etc); flashed at 0x00000, except S2 where it is at 0x01000
  • we could only support QIO, but some cheap clones may have flash not capable of QIO
• partition map (2M (uncommon, not supported), 4M, 8M, 16M); flashed at 0x08000
• boot_app0.bin (identical for all ESP32 variants); flashed at 0x0e000
• WLED firmware (determines PSRAM support: WROVER rel.1, QSPI, OPI)
  • ESP32: 2 binaries, WROVER rev.1 and all other variants
  • ESP32-S2: 1 binary
  • ESP32-S3: 2 binaries, QSPI and OPI
  • ESP32-C3: 1 binary

Firmware may also have embedded flash speed (40MHz or 80MHz). 80MHz may not work on every ESP32 board but seems ok on S2, S3 and C3.

[marcobrianza]

Hi, I just did some testing installing 0.15.0-b2

Lolin C3 mini: works!

Xiao C3: works!

Lolin S3 PRO: works!

Xiao S3: has this error:
09:48:21.184 -> ./components/esp_littlefs/src/littlefs/lfs.c:1229:error: CorruptE (105) I2S: i2s_driver_uninstall(2047): I2S port 0 has not instE (105) I2S: i2s_driver_uninstall(2047): I2S port 0 has not inst

Lolin S3 mini: boot loops with this message about flash size:
09:27:04.346 -> ESP-ROM:esp32s3-20210327
09:27:04.346 -> Build:Mar 27 2021
09:27:04.346 -> rst:0xc (RTC_SW_CPU_RST),boot:0x8 (SPI_FAST_FLASH_BOOT)
09:27:04.346 -> Saved PC:0x40379428
09:27:04.346 -> SPIWP:0xee
09:27:04.346 -> mode:DIO, clock div:1
09:27:04.384 -> load:0x3fce3808,len:0x44c
09:27:04.384 -> load:0x403c9700,len:0xbec
09:27:04.384 -> load:0x403cc700,len:0x2920
09:27:04.384 -> entry 0x403c98d8
09:27:04.638 -> E (270) spi_flash: Detected size(4096k) smaller than the size in the binary image header(8192k). Probe failed.
09:27:04.676 -> E (298) esp_core_dump_flash: Core dump flash config is corrupted! CRC=0x7bd5c66f instead of 0x0

Lolin S2 MINI: installs but does not work:
no serial port visible after boot so cannot get debug info

[blazoncek]

I'm dropping this here for reference:

https://espeasy.readthedocs.io/en/latest/ESPEasy/ESPchips.html#quad-octal-spi-mode

Quite an interesting read, confirming what was said above.

[paulwrath1223]

I'm having the same problem flashing ESP32-S3-WROOM-1U-N8 and ESP32-S3-WROOM-1-N8, works fine on PIO (except for a seemingly erroneous LittleFS 'failed to mount' error), but the web installer gets boot looped.

[lost-hope]

What version where you trying to install?

[SuperSandro2000]

Lolin S3 mini: boot loops with this message about flash size:

The pre compiled binaries do not work for me but compiling them on my machine with -D WLED_DEBUG works. No idea why tbh.

[shadowlab]

Lolin S3 mini: boot loops with this message about flash size:

The pre compiled binaries do not work for me but compiling them on my machine with -D WLED_DEBUG works. No idea why tbh.

I made my own board definition for an ESP32-C3 and ran into the same thing compiling and flashing v16.

CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/super_mini_esp32c3.html
PLATFORM: Espressif 32 (6.3.2) > ESP32C3-SuperMini
HARDWARE: ESP32C3 80MHz, 320KB RAM, 4MB Flash
DEBUG: Current (cmsis-dap) External (cmsis-dap, esp-bridge, esp-builtin, esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa)
PACKAGES:

• framework-arduinoespressif32 @ 3.20009.0 (2.0.9)
• tool-esptoolpy @ 1.40501.0 (4.5.1)
• toolchain-riscv32-esp @ 8.4.0+2021r2-patch5

Everything compiles successfully with or without debug. However, upon first reboot is where the issues lie. Both methods are using PlatformIO (esptool).

Without debug flag:

ESP-ROM:esp32c3-api1-20210207
E (290) esp_core_dump_flash: No core dump partition found!
Ada

When using the debug flag:

---WLED 0.16.0-dev 2412040 INIT---

esp32 v4.4.4
arduino-esp32 v2.0.9
CPU:   ESP32-C3 rev.4, 1 core(s), 80 MHz.
FLASH: 4 MB, Mode 3 (DOUT), speed 80 MHz.
heap 251872
JSON buffer allocated: 32767
TX power: 78/34
PIN ALLOC: Pin 21 successfully allocated bLoop took 1135ms.
Usermods took 0ms.
Strip took 1ms.
Network event: 4
(continues with regular boot) 

[blazoncek]

That's all good and well.

[shadowlab]

That's all good and well.

Yes, I'm glad adding the debug flag made it work, but I was mostly wondering why, seeing that it was the only build flag changed between tests.

Without it (first image), the ESP freezes there and doesn't proceed further in booting.

[softhack007]

Without it (first image), the ESP freezes there and doesn't proceed further in booting.

Can you explain what happens when you say it "freezes"?

Because you have "Ada" on serial monitor, it actually means that boot is completed, and you either have WLED-AP or it connects to your router (if configured).

[shadowlab]

Without it (first image), the ESP freezes there and doesn't proceed further in booting.

Can you explain what happens when you say it "freezes"?

Because you have "Ada" on serial monitor, it actually means that boot is completed, and you either have WLED-AP or it connects to your router (if configured).

That's where I'm a little confused too. Even though 'Ada' is shown, nothing more happens in the boot cycle; I get no further messages in the serial monitor and no AP is created or attempts connection to my home wifi.

However, only when -D WLED_DEBUG is added to the compile options do things actually proceed further and creates the WLED-AP as normal.

[blazoncek]

I get no further messages in the serial monitor

That is expected unless you use Improv or Adalight protocol.