State of low-level flash storage interfaces

[x3ro]

Currently there are several efforts to add a low-level flash interface to RIOT:

1. Flash interface #2239 by @d00616 proposes an interface with memcpy-semantics, which appears to be designed with memory-mapped NOR flash in mind (?). It also comes with an implementation of the interface for native (using mmap) and the nrf51822.
2. drivers/periph: added flashrom driver interface #5365 by @haukepetersen proposes an interface for page-aligned access which should work for both NOR and NAND flash. It comes with an implementation for nrf5x and stm32f1 CPUs.
3. mtd flash: add a generic low level flash interface [RFC] #5624 by @vincent-d proposes another low-level interface
4. Furthermore, an interface to access SD cards already exists in diskio.h, which is implemented for the lpc2387 (msba2). This one also provides page-aligned access.

I'd like to kick off a discussion on which kind of low-level interface we want for RIOT, since I think that, for simplicity, there ought to be only one.

I currently favor the interface proposed in #5365, since I feel that thinking of flash in terms of memory mapped I/O is dangerous: writing to the same address over and over is common in RAM, but destructive on flash. #5365 is relatively straight forward and can be extended with partial page writes/reads if necessary. We could even deprecate the diskio.h MCI interface after porting its functionality.

/cc @kYc0o @authmillenon @OlegHahm @kaspar030 @cgundogan

[jnohlgard]

I agree with your views about the danger of using flash devices as memory mapped storage. #5365 is a step in the right direction, but to me it seems like it only supports one flash device per build and it is designed for OTA/DFU use for accessing the CPU internal flash. I would like to see an interface that can be used as a block device for file systems as well, to keep the API complexity and number of user functions, low.

[LudwigKnuepfer]

I second @gebart - memory interfaces should allow their usage for internal and external memories.

[kaspar030]

but to me it seems like it only supports one flash device per build

I mentioned that in the PR, maybe you step in with your opinion there?

[jnohlgard]

@kaspar030 I guess I should have written it there as well, did now.

[kYc0o]

IMHO I think this low-level driver should only propose a very simple API to write into flash (MCU or external) like read/write block, read/write byte and some erase functions. However, I don't think this should be intended for a continuous use, thus reading/writing to any flash directly should be avoided in high level applications.
Instead, we need a file system to protect external flash storage from intensive write/erase process on the same blocks. A good filesystem would write new blocks until the memory is full and then restart from the beginning, without any user intervention.
For internal flash I think the only valid application should be firmware upgrades and dynamic module loading.

My 2 cents :)

[jnohlgard]

@kYc0o I agree, the flash API would be used by file system implementations, and not by the application developers.

Contiki CFS has some properties which make it useful for sensor data collection applications and it would probably not be too much work to port it if there is a flash API in RIOT.
https://github.com/contiki-os/contiki/wiki/File-systems

[jnohlgard]

Another PR on flash interfaces: #5624, with a spiffs implementation in #5625

[d00616]

I have designed #2239 to support accessing flash at the lowest possible level. That's the reason i work with pointers to flash memory. Reading a single word from internal flash doesn't need to copy a whole page into RAM. This helps to save some energy for ultra low power devices.

Next feature is the ability to write parts of a byte, if it is supported by underling flash type. My Plan was to create a Counter with this feature. By storing a start value + a bit map it's possible to increase a 64 bit counter 81,280,000 times in a 1K page while this page is only erased 10,000 times. My wish is to have the feature available via an API.

Bitwise writing helps to manage a map of defunct flash pages or sub pages for flash doesn’t support many erase cycles.

In my opinion the flash interface must be flexible enough to handle flash types which are more flexible in addressing but with very few erase cycles and flash which can erases often but with more restrictions in access.

I think the next layer, like a file system or counter object, needs to implement some optimized code for every flash type to optimize the life time of the MCU and battery.

At the moment my idea of an flash interface is to provide functions to:

• Open a page from flash for reading (pointer to RAM of flash)
• Close a page (free RAM if needed)
• Write a (part) of a page
• A function to give information how flash is accessible

My 2 cents.

[PeterKietzmann]

@x3ro can you check if all questions have been answered? If so please close, otherwise, please rephrase still open questions!

[x3ro]

Closing this as the discussions has stagnated. If I have the time, I shall review the topic again and perhaps kick off another discussions if necessary.