Filesystem Guidance (IDFGH-14626)

[f-hoepfinger-hr-agrartechnik]

Answers checklist.

• I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
• I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
• I have searched the issue tracker for a similar issue and not found a similar issue.

General issue report

Hello,

we have now 4 Possibilities for File System on internal Flash

• spiffs
• fatfs
• fatfs + esp_jrnl
• littlefs

Links:

https://components.espressif.com/components/joltwallet/littlefs

https://components.espressif.com/components/espressif/esp_jrnl/versions/1.0.1

while Spiffs is slow,
and fatfs is really sensitive to power loss,

i am struggling between

• fatfs + esp_jrnl
• littlefs

what is really better in case of robustness.

any Opinions ?

https://stackoverflow.com/staging-ground/79432296

Thanks,
Franz

[f-hoepfinger-hr-agrartechnik]

@pacucha42

@BrianPugh

@Adam5Wu

[RathiSonika]

Hi @f-hoepfinger-hr-agrartechnik,
Both fatfs + esp_jrnl and LittleFS are designed to improve reliability, but they have different approaches:

1. esp_jrnl helps mitigate power loss issues by ensuring writes are logged before committing to FATFS, It’s a good option if you need to work with other systems. But on the other hand, FATFS still has fragmentation issues over time. Also I think Journaling adds overhead, increasing write latency.

2. littlefs has Power-loss resilient by design, Wear-leveling is built-in, increasing flash lifespan.
   Also efficient for small files and embedded applications. But not compatible with FATFS tools.

Thus if robustness is the key factor, LittleFS is the better choice due to its atomic file operations and wear-leveling.
If you need to use FATFS while making it more reliable, fatfs + esp_jrnl is a good option.

Yo can refer to this file-system-consideration

[BrianPugh]

(esp_littlefs author here, but not author of littlefs itself).

I would recommend esp_littlefs for environments where you are:

• mostly dealing with small files (KB to a few MB). I originally ported littlefs because I was downloading/loading assets that were 100~200KB in size and spiffs was too slow.
• Dealing with a filesystem on the order of KB or MB (not GB).
• Not an SD card (currently there may be performance limitations)
• Don't have to directly interact with the filesystem with a third party (FAT is easy to just plug into any computer).
• Not frequently doing file appends. I've seen a lot of github-issues relating to performance when people are appending log-files.

I haven't used esp_jrnl, but it certainly looks interesting to try!

A nice thing about esp-idf's virtual-filesystem abstraction is that it's (relatively) trivial to swap out filesystem backends. So I'd say just give one a try for your use-case, and if it doesn't meet requirements, try another.

I usually operate with fairly small filesystems with SPI flash, so I also like to compress data using my compression library.

[diplfranzhoepfinger]

https://github.com/espressif/esp-idf/tree/master/examples/storage/littlefs

[diplfranzhoepfinger]

@BrianPugh LittleFS now test-driving.