use of "errno" variable to indicate errors

[steverpalmer]

Firstly, thanks for this library. It looks like a good fit to my needs ... with perhaps one exception ...

You have obviously stuck to the libc/posix API, which is always a good starting point, but may be not in this case. I suggest that the libc interface includes some problems which have become apparent over time, but which libc can't break because of the massive impact it would have on existing code. In particular the use of the errno variable has become an increasing problem with multiple thread code. (See https://yarchive.net/comp/linux/errno.html). I might suggest that other libraries like littlefs (https://github.com/littlefs-project/littlefs) as an example of a library that has striven to break from the errno model. Might I gently suggest that there is some value is breaking the code into two parts: a low-level which does not use the errno variable, and a high-level wrapper which provides the more conventional libc interface?

Just a suggestion.

Cheers.

Background

I'm writing bare-metal code for a Pimoroni Pico Plus 2W (RPi Pico 2w but with 16MB Flash plus 8MB PSRAM). My code is running a Lua REPR loop in a little co-operative scheduler with a virtual file system. The VFS includes a ROMFS for various useful Lua libraries (e.g. Lust unit testing), and I am hoping to add an microSD card option and a ram file system. My aim is to be able to prototype application code in Lua on target with real sensors/actuators, before recording in C/Rust. Lua can also provide a useful framework for unit testing on target, or maybe for initial system configuration.

My "file system stack" looks something like:

Application (which mayor may not use Lua below. but may also make direct access to my VFS directory)
Lua
libc implemented by picolibc (https://github.com/picolibc/picolibc)
libgloss (handles mapping for to errno model from non-errno model below)
My Virtual File System (no errno)
| ROMFS | RamFS? | UART drivers | ...

Obviously, my use-case may not be your target audience, but I dare to suggest there are people who want a low-level ramfs library without errno.

Thanks again.

[jkent]

First off, sorry for not getting back to you sooner. I have not made enough time to address issues.

I'm welcoming discussion of this issue. I'll start. Is thread local storage not good enough for errno? I know picolibc supports TLS, with errno can live in the TLS space. I've used it with FreeRTOS before. If not, I can see separating this out.

Kind regards,
Jeff

[steverpalmer]

So some elaboration on my original comments.

For precision, I am using the following as the definition(s) of errno:

• https://en.cppreference.com/w/c/error/errno.html
• https://pubs.opengroup.org/onlinepubs/9799919799/

Also, for precision, I'm also looking at "main" branch of RamFS on github.

I did a quick search for "libc errno criticism" and found several relevant links:

• https://news.ycombinator.com/item?id=34770909
• https://lobste.rs/s/bejkkg/errno_libc
• https://lwn.net/Articles/655134/

Perhaps I've strayed into a "flame-war" zone - sorry.

I think that there are several issues I would raised around errno.

Multi-threaded Environments

errno in an multi-thread environment should be declared "thread local", as your comment suggests.
The definitions of errno above, indicate that this has has been true since C11. If RamFS continues with errno, then it perhaps needs to declare C11 as the minimum C standard it will work with? Or perhaps needs to include a warning if this is not the case?

(I guess I'm assuming that most modern embedded development environments will support Thread Local Storage.)

In the case of embedded software, the Multi-threaded concern should perhaps be extended to consideration of interrupts.
Perhaps an Interrupt Service Routine might be tempted to write to a RamFS file handle to store data for later processing by "main-thread" task. I suspect that this would not work, in part because of a shared errno.

I started wondering about use of RamFS functions inside signal handlers, but since functions like 'malloc' are not considered "safe", this should perhaps be discouraged.

Applying an errno-based API

The received wisdom for using errno is to set it to zero before the code segment of interest, and then checking for non-zero at the end of the code of interest. Since (from the above definitions) "library functions never store 0 in errno", this can be used to ensure that the code segment of interest has performed without error.

However, does RamFS itself do this?

To get started, I've focused on ramfs_vector.c, so if my comments are not true elsewhere, I apologise.

Consider function ramfs_create which assigns EINVAL to errno in several cases where incorrect arguments are used by its caller. Function ramfs_create calls the library function calloc to get some memory. The call to calloc is checked for a NULL results and, if so, ramfs_create returns another NULL back to the caller to indicate the error. Now calloc has at least 2 reasons for raising an error:

• ENOMEM - which I suspect works as expected;
• EINVAL - which might be caused by an error in RamFS itself.

As implemented, a caller to ramfs_create might see return NULL, check errno and see EINVAL and think that it has supplied the wrong arguments. But perhaps the error is internal to RamFS? This is a simple case - ramfs_create also calls a local function insert which in turn calls realloc which can return an EINVAL error...

Functions like malloc, realloc, calloc and strndup use errno and so RamFS, as client, should apply the received wisdom for errno to check for internal faults.

In a bigger picture, I worry that an errno-based API encourages an attitude that errors and exceptions are the callers problem, to be simply passed back up the call-chain. (Sorry if this is too critical.)

Consistency with other file system libraries

In my original post, I mentioned that littlefs does not use an errno-based API.

I scanned through other open source file systems listed on Awesome Embedded Resources for Developers (https://github.com/iDoka/awesome-embedded-software?tab=readme-ov-file#filesystems),
looking at approaches used by other similar software:

• lwext4 (https://github.com/gkostka/lwext4) seems to use functions that return an error code, rather than errno;
• the popular FatFs (https://elm-chan.org/fsw/ff/) uses functions that return error codes;
• Eclipse ThreadX LevelX (https://github.com/eclipse-threadx/levelx) uses functions that return error codes;
• ufat (https://github.com/dlbeer/ufat) uses functions that return error codes;
• fat_io_lib (https://github.com/ultraembedded/fat_io_lib) couldn't tell, but couldn't see any references to errno;
• Arduino's SdFat (https://github.com/greiman/SdFat) does not include the string "errno" (except in a linter program??);
• fat32 (https://github.com/strawberryhacker/fat32) uses functions that return error codes;
• OpenFAT (https://github.com/tmolteno/openfat) uses functions that return error codes;
• uC-FS (https://github.com/weston-embedded/uC-FS) couldn't tell, but couldn't see any references to errno. I did see some code that had a pointer paramater called p_err;
• littlefs already mentioned
• UFFS (https://github.com/rickyzheng/uffs) uses functions that return error codes;

If we assume that this is a credible sample of open-source file systems (I certainly didn't pick them), I would suggest that the majority (all-bar-two: fat_io_lin and uC-FS) are using functions that return error codes. Could this be interpreted as collective-wisdom? On the other hand, there maybe something special about RamFS (such as the use of malloc) that makes an errno-based API preferable.

Cheers,
Steve

[jkent]

Hi Steve,

Thank you for the insight and research. I don't consider this flame war territory as I'm open to change in a major release. Keep in mind when I developed this, I was thinking about ESP-IDF VFS compatibility (which is why there is the vfs.c code). Obviously I need to keep errno in some form for that vfs code. Also, this project is meant to mirror in many ways the way FrogFS works. I'm not sure if you've encountered it yet.

Kind regards,
Jeff

[steverpalmer]

Hi Jeff,

I appreciate that use case do, and should influence the code we write. (I spent a bunch of my working life trying to persuade programmers not to write the most general purpose motor driver - just write code for the motor hardware we've got to do the things that we need the motor to do!) Given that, I thank you for your consideration of my suggested change.

If I might make one more point: accepting that there are various styles of API that we want to accommodated, I've generally found it easy (even trivial) to add a errno-style wrapped on non-errno-code, but found it rather more tricky to go the other way. (Sorry, couldn't resist.)

I've not come across FrogFS - it looks cute. In my own code I went for a ROMFS, using genromfs to create it on host and bespoke code on target to unpack it. I liked the fact that the files were stored contiguously in memory so would allow a mmap-like API and so reduce further ram usage; another possible feature of RamFS, at least for file read.

Back to the original point, a significant aspect of RamFS is that it uses the libc memory management stuff (i.e. malloc et al), and therefore depends on libc. However, in my use case I am trying to use RamFS to supply the implementation of some 'libc' interfaces, at least in some special cases. I suspect that my trying to force RamFS into this kind of "Split Personality" might be the underlying problem I'm having. Hmm.

Thanks again,
Steve