RNS_PSRAM_POOL_ALLOCATOR initializes before ESP32 PSRAM is ready and permanently disables the pool after failure

[p6zem0]

Description

On ESP32-S3, RNS_PSRAM_POOL_ALLOCATOR can attempt to initialize its TLSF
pool during C++ static initialization, before Arduino has initialized PSRAM.

The first pool allocation therefore fails, but the current pool_init()
implementation marks the pool as initialized before the backing PSRAM buffer
has actually been allocated.

As a result, the failed initialization is never retried and the PSRAM pool
remains permanently disabled (tlsf == nullptr, pool_size == 0).
Subsequent container allocations silently fall back to the internal heap.

I traced the issue to a namespace-scope allocating static in Persistence and
confirmed the initialization order through ELF .init_array analysis and a
non-allocating pre-setup RAM trace.

A minimal retry fix allows the PSRAM pool to initialize successfully later,
after Arduino has initialized PSRAM.

Environment

microReticulum revision tested:

40fa628

Hardware:

Heltec WiFi LoRa 32 V4-R8 / Expansion Kit V2
ESP32-S3R8
8 MB OPI PSRAM
16 MB flash

Build environment:

PlatformIO
platform = espressif32@7.0.1
Arduino-ESP32 2.0.17
ESP-IDF 4.4.7
Xtensa GCC 8.4.0

Relevant configuration:

-DBOARD_HAS_PSRAM=1
-DRNS_CONTAINER_ALLOCATOR=RNS_PSRAM_POOL_ALLOCATOR
-DRNS_PSRAM_POOL_BUFFER_SIZE=2097152

The PSRAM hardware itself is detected correctly and provides approximately
8 MB of usable memory after Arduino initialization.

Root cause

RNS::Persistence::_buffer is defined as a namespace-scope static:

static Bytes _buffer(Type::Persistence::BUFFER_MAXSIZE);

BUFFER_MAXSIZE is 1536 bytes.

ELF/disassembly analysis shows that this object is constructed from
.init_array before app_main() / initArduino().

The call path is:

RNS::Persistence::_buffer
  -> Bytes(1536)
  -> Bytes::newData()
  -> std::vector::reserve()
  -> ContainerAllocator<uint8_t>::allocate(1536)
  -> pool_malloc()
  -> pool_init()

At this point the PSRAM pool descriptor itself is valid. It is
constant-initialized in .data with:

type = RNS_PSRAM_POOL_ALLOCATOR
buffer_size = 2097152
pool_init = false
tlsf = nullptr

However, ESP32 PSRAM is not ready yet.

On Arduino-ESP32 2.0.17, ps_malloc() returns NULL while
spiramDetected == false. spiramDetected is only set later by
psramInit() from initArduino().

Therefore the early call effectively does:

pool_init()
  -> pool_info.pool_init = true
  -> ps_malloc(2097152)
  -> NULL
  -> tlsf remains nullptr

Because pool_init is already true, later pool_malloc() calls do not retry
initialization even after PSRAM becomes available.

The result is:

pool_size = 0
pool_free = 0
tlsf = nullptr

and allocations fall back to the internal heap.

Runtime verification

I added a tiny zero-initialized POD trace in .bss that only records primitive
values and performs no allocation or logging. This avoids relying on serial
output during static initialization.

It records:

allocate_seen=1 seq=1 n=1536 size=1536
pool_malloc_seen=1 seq=2 size=1536 init=0 tlsf=0x0 type=3 buffer=2097152
pool_init_seen=1 seq=3 type=3 buffer=2097152

This confirms that the 1536-byte container allocation and pool_init() happen
before normal Arduino setup logging.

With the unmodified pool initialization behavior, later diagnostics show:

pool_size=0
pool_free=0

despite PSRAM being available by then.

Minimal fix tested

I tested a minimal change to Memory::pool_init():

• do not set pool_info.pool_init = true at function entry;
• if the backing allocation fails, return with pool_init == false;
• if tlsf_create_with_pool() fails, free the backing buffer and leave
  pool_init == false;
• set pool_init = true only after both backing allocation and TLSF creation
  succeed.

Conceptually:

raw_buffer = ps_malloc(buffer_size);

if (raw_buffer == nullptr) {
    return;  // pool_init remains false, retry later
}

pool_info.tlsf = tlsf_create_with_pool(raw_buffer, buffer_size);

if (pool_info.tlsf == nullptr) {
    free(raw_buffer);
    return;  // retry later
}

pool_info.pool_init = true;

This allows an early pre-PSRAM attempt to fail harmlessly and a later call to
retry after psramInit().

Result with retry fix

The behavior is reproducible across multiple boots.

The initial pre-Arduino attempt still fails, as expected.

After PSRAM becomes available, initialization is retried. Runtime output then
shows a successful TLSF initialization and an actual 2 MiB PSRAM pool:

TLSF: PSRAM contiguous_size: 8257524
TLSF: contiguous_size: 8257524
TLSF: initialization with align=4, contiguous=8257524,
      size=2097152 SUCCESSFUL!!!

Before the NomadNet announce:

free_psram=6275699
largest_psram=6160372
pool_size=2097152
pool_free=2094452
pool_frag=0

The roughly 2 MiB reduction in free PSRAM confirms that this is a real PSRAM
pool and not the previous internal-heap fallback.

microReticulum then successfully proceeds through packet packing and reaches:

Packet::pack: packed packet of size 201 bytes
Packet::send: successfully sent packet!!!
RNS is READY!

No TLSF assertion, spinlock assertion or watchdog reset occurs with the active
PSRAM pool.

The same successful sequence was reproduced on a second boot.

Additional observation

After PSRAM becomes available, I consistently see one
tlsf_create_with_pool() attempt fail immediately before a subsequent retry
succeeds a few milliseconds later:

Initializing TLSF pool...
... size=2097152 FAILED!!!

Initializing TLSF pool...
... size=2097152 SUCCESSFUL!!!

I have not yet determined the cause of this first post-PSRAM TLSF failure, so
I am not treating it as part of the proven root cause above. The retry logic
handles it safely and the resulting pool is functional.

Expected behavior

A failed pool initialization before PSRAM is available should not permanently
mark the pool as initialized.

The allocator should either:

1. defer initialization until PSRAM is available, or
2. leave the pool uninitialized after failure so that a later allocation can
   retry initialization.

Actual behavior

The first failed initialization sets pool_init = true, preventing any future
retry. The configured PSRAM pool therefore remains disabled for the lifetime
of the process.

Possible fixes

The smallest fix appears to be changing pool_init() so pool_init = true
is set only after successful backing-buffer allocation and successful
tlsf_create_with_pool().

Another possible improvement would be to avoid dynamically allocating
namespace-scope Bytes objects such as RNS::Persistence::_buffer during
.init_array, for example by deferring their construction until first use.

I have only tested the retry fix on ESP32-S3 / Arduino-ESP32 2.0.17 so far.
I have not tested the shared allocator change on NRF52 or other supported
platforms.