Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 28 additions & 9 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ ZephCore is a LoRa mesh networking firmware running on Zephyr RTOS. It supports
- **Room Server**: Headless store-and-forward shared message room (BBS). Reuses the repeater's ACL/region/CLI; pushes new posts to logged-in clients (per-client sync cursor + ACK).
- **Observer** (ESP32): Listen-only node that publishes received LoRa packets to MQTT over WiFi.

Supported hardware: nRF52840, nRF54L15, ESP32 (classic PICO-D4 and C3/C6/S3), EFR32MG24, and STM32WL (LoRa-E5). Radios: SX126x family (SX1261/62/68, LLCC68, STM32WL sub-GHz), LR1110, SX127x (SX1272/76/78, loramac-node backend), and LR2021 (validated on the MeshTracker X1). A native Linux port runs the full stack on SBCs (Femtofox, Raspberry Pi) via Zephyr `native_sim` — see `LINUX_NATIVE.md`.
Supported hardware: nRF52840, nRF54L15, nRF54LM20A, ESP32 (classic PICO-D4 and C3/C6/S3), EFR32MG24, and STM32WL (LoRa-E5). Radios: SX126x family (SX1261/62/68, LLCC68, STM32WL sub-GHz), LR1110, SX127x (SX1272/76/78, loramac-node backend), and LR2021 (validated on the MeshTracker X1). A native Linux port runs the full stack on SBCs (Femtofox, Raspberry Pi) via Zephyr `native_sim` — see `LINUX_NATIVE.md`.

### Upstream Relationship

Expand Down Expand Up @@ -124,7 +124,7 @@ zephcore/
├── boards/ # Board definitions
│ ├── common/ # Shared configs, DTS includes, partition layouts
│ ├── nrf52840/ # RAK4631, T1000-E, ThinkNode M1/M3/M6, T-Echo, T114, ...
│ ├── nrf54l/ # XIAO nRF54L15
│ ├── nrf54l/ # XIAO nRF54L15, Seeed LR2021 EVK (nRF54L15 and nRF54LM20A), ME25LS02
│ ├── esp32/ # XIAO C3/C6/S3, Heltec V3/V4.x, Station G2, T-Beam, ...
│ ├── mg24/ # XIAO MG24
│ ├── stm32wl/ # Seeed LoRa-E5 mini
Expand Down Expand Up @@ -598,7 +598,8 @@ mesh::Mesh
Handles the binary BLE protocol with ~50 command opcodes. Key features:
- **Offline queue**: circular buffer with peek/confirm pattern (survives BLE drops); `CONFIG_ZEPHCORE_OFFLINE_QUEUE_SIZE`, default 256 frames (lowered on RAM-bound boards)
- **ACK tracking**: 8-slot table, computes expected ACK = SHA256(secret + hash)[0:4]
- **Contact iteration**: Streaming protocol with `lastmod` filtering for incremental sync
- **Contact iteration**: Streaming protocol with `lastmod` filtering for incremental sync. The dump advances only
while a client is connected and every connected transport has room; it is cancelled when the last client leaves
- **Lazy write batching**: Dirty contacts/channels flush after 5-second delay
- **Protocol versioning**: V2/V3 frame format negotiation with phone app
- **Ed25519 signing**: 3-phase flow (start→data→finish) for signing up to 8KB
Expand Down Expand Up @@ -721,6 +722,7 @@ Autonomous operation features:
- **Permission levels**: GUEST(0), READ_ONLY(1), READ_WRITE(2), ADMIN(3)
- **Region filtering**: `RegionMap` with transport key matching per flood packet
- **Rate limiting**: 4 requests per 120s (discovery), 4 per 180s (anonymous), 4 failed logins per 180s
- **Node discovery**: answers discovery requests only while forwarding is on. With `advert.interval` and `flood.advert.interval` both `0` the node is hidden and does not answer, as upstream (`isHiddenNode()`); the request still counts against the discovery rate limit
- **Neighbor tracking**: RSSI/SNR/name/timestamp table (`CONFIG_ZEPHCORE_MAX_NEIGHBOURS`, default 50 slots)
- **Temporary radio params**: `tempradio` command applies freq/bw/sf/cr via `LoRaRadio::setRadioOverride()` (does not mutate `_prefs`); auto-revert timer calls `clearRadioOverride()` to fall back to saved prefs
- **WiFi+MQTT uplink** (ESP32, `CONFIG_ZEPHCORE_REPEATER_UPLINK`): `RepeaterUplink.cpp` reports packets observer-style while still repeating; configured via `set uplink.*` CLI
Expand Down Expand Up @@ -763,15 +765,17 @@ Full command reference with constraints and remote-admin restrictions: `CLI_comm
- TX congestion control: queue (12 frames) + overflow buffer + retry + timeout watchdog
- Fast/slow advertising switching with post-disconnect flap prevention
- DLE (Data Length Extension) to 251 bytes
- Interface coexistence: BLE vs USB, one active at a time
- One of the companion transports served at the same time (BLE, USB/UART, TCP; [7.6](#76-wifi--mqtt--tcp-transports))
- Debug: build with `debug.conf` plus `-DCONFIG_ZEPHCORE_BLE_LOG_LEVEL_DBG=y` for adapter-level DBG logging

### 7.2 DataStore (`adapters/datastore/`)

- **Internal**: LittleFS on flash (`/lfs`), 256-byte cache for reduced flash I/O
- **External**: Optional LittleFS on QSPI (`/ext`) with auto-migration. The flash is `zephyr,deferred-init` and `/ext` is
not automounted (`boards/common/qspi-ext.dtsi`): `zephcore_fs_mount_ext()` brings it up on first use, because a
boot-time probe could beat a cold power rail (first boot after a UF2 update) and silently drop to internal flash
boot-time probe could beat a cold power rail (first boot after a UF2 update) and silently drop to internal flash.
Roles that keep nothing on `/ext` (repeater, room server, observer) put the flash into deep power-down at start-up
(`zephcore_fs_ext_power_down()`), with chip-select held high; a factory `erase` resumes it and puts it back
- **BLE bonds**: NVS (`storage_partition`, 0xD0000 on nRF52) via Zephyr settings backend (≥1.16.2)
- **Prefs**: `prefs.json` through upstream's `ConfigSerializer`, with upstream's key names for shared fields and ZephCore's own under `zc` (see §13); the older binary file is read once to migrate and kept
- **Contacts**: 152-byte records, stored on external flash if available
Expand Down Expand Up @@ -829,9 +833,20 @@ Repeaters and room servers default to `CONFIG_ZEPHCORE_REPEATER_GPS_INTERVAL_SEC

### 7.4 USB (`adapters/usb/`)

- **CompanionUSB**: V3-framed CDC (little-endian 16-bit length prefix + payload)
- **RepeaterUSB**: Minimal CDC with 1200-baud DFU touch detection
- Both share message queues with BLE adapter (transport-agnostic mesh layer)
- **`ZephyrUSBCDC`** (every role): the USB device and its CDC ACM class, the 1200-baud touch into the bootloader, DTR
tracking, and `zephcore_usbd_detach()` before a reset (a soft reset leaves the ESP32-S3's PHY attached). A host that
went away is reported as a DTR drop in three cases: DTR low, VBUS removed, and a bus reset. The reset case exists
because the CDC class keeps the old line state across a reset, so nothing else says that the port the host had open
is gone (a hub resetting upstream leaves VBUS up). Bus suspend, resume and reset also reach a second callback.
- **`ZephyrCompanionUSB`** (companion): the wired companion transport, over the CDC port or a plain UART
(`zephcore,companion-uart`). Framing: `<` (to the node) or `>` (from it), a little-endian 16-bit length, the payload;
a first byte that is printable and not `<` starts the text CLI instead. It has its own receive queue and an
interrupt-driven 2 KB TX ring whose "drained" callback paces the contact dump.
- **Session rule**: a session starts on the first inbound frame or CLI line and ends on a DTR drop as defined above.
While the bus is suspended (the computer asleep with the port open) the session is kept but is not a connected
client: nothing is queued for it, and it does not hold up the clean-up that runs when the last client leaves. Each
change reaches the main loop as a connect or a disconnect.
- Servers (repeater, room server, observer) use the CDC port as their serial console.

### 7.5 Board (`adapters/board/`)

Expand All @@ -845,7 +860,7 @@ Repeaters and room servers default to `CONFIG_ZEPHCORE_REPEATER_GPS_INTERVAL_SEC

**Telemetry** (`REQ_TYPE_GET_TELEMETRY_DATA`, every role, upstream's shape): battery on channel 1, then `sensors.querySensors()` (`adapters/sensors/ZephyrSensorManager.cpp`): GPS position on channel 1 while the GPS is on and has a fix, each environment sensor and power-monitor channel found at boot on its own channel from 2 up (probe order), board-local analog sensors (T1000-E) on channel 1; then the MCU temperature on channel 1. Servers give guests battery + MCU temperature only and honour the requester's inverse permission mask, as upstream. Encoded by `helpers/compat/CayenneLPP.h`: upstream's library API and wire format, but values round to the nearest step where the library truncates.

**Clock** (`adapters/clock/ZephyrRTCClock`): `setCurrentTime()` also writes the hardware RTC when the board has one (coalesced on the system work queue), as upstream's `AutoDiscoverRTCClock` writes its chip; `seedCurrentTime()` is the boot restore, not written back. Without a hardware RTC, the last time read is kept in no-init memory (RTC slow memory on ESP32) and restored after a reset that keeps RAM (reboot, crash, watchdog), as upstream's `ESP32RTCClock`; a power cycle starts from 1970 again.
**Clock** (`adapters/clock/ZephyrRTCClock`): `setCurrentTime()` also writes the hardware RTC when the board has one (coalesced on the system work queue; a time outside 2000-2099 is not written, and at an RV3028 with `rv3028-eeprom-config` its year is marked unset instead), as upstream's `AutoDiscoverRTCClock` writes its chip; `seedCurrentTime()` is the boot restore, not written back. A chip is adopted at boot only once it is identified (`adapters/clock/ZephyrRTCDiscover.c`), because other parts can answer at the same address: a device is passed over when two reads each show a bit set that the chip's data sheet fixes at zero, or a time field out of range, and a time is restored only from a read in which every field is valid. The time is written in each chip's own encoding (the RX8130CE takes a one-hot weekday, `weekday-one-hot` in its descriptor), and an RV-3028 whose descriptor carries `rv3028-eeprom-config` (the RAK12002 on a RAK4631) has that configuration, its backup switchover, stored in the chip's EEPROM. Without a hardware RTC, the last time read is kept in no-init memory (RTC slow memory on ESP32) and restored after a reset that keeps RAM (reboot, crash, watchdog), as upstream's `ESP32RTCClock`; a power cycle starts from 1970 again.

**LED master switch** (`helpers/led_gate.{c,h}`, `set leds on|off`, all roles): one process-wide
flag every LED driver consults — heartbeat and unread-message LEDs in `helpers/ui/ui_common.c`, the
Expand Down Expand Up @@ -984,6 +999,7 @@ prj.conf (base: console; production defaults — LOG=n, ASSERT=n)

- **nRF52840**: Zephyr open-source BLE controller, UF2 bootloader, partial flash erase for BLE coexistence
- **nRF54L15**: Same BLE controller as nRF52, CMSIS-DAP via SAMD11 bridge, no native USB
- **nRF54LM20A** (the LR2021 EVK with a XIAO nRF54LM20A, its own board directory): built and flashed like the nRF54L15 (`--no-sysbuild`, CMSIS-DAP); source-only, no published firmware
- **ESP32-C3/C6/S3**: Espressif proprietary BLE blob, 32KB heap, asserts disabled (blob IRQ false positives); simple-boot by default, MCUboot only with `--sysbuild` (WiFi OTA)
- **ESP32 classic (PICO-D4)**: much smaller DRAM — contact/queue caps shrunk in `board.conf`; console/CLI on `uart0` (no native USB); DIO flash mode required (QIO bootloops)
- **EFR32MG24**: SiLabs proprietary BLE blob, 32KB heap, SEMAILBOX enabled for hardware TRNG/crypto entropy, ADC disabled (no battery divider), CMSIS-DAP via onboard SAMD11
Expand All @@ -1005,6 +1021,8 @@ Applied automatically at CMake configure time; a failed patch aborts the configu
| 0008-flash-sim-per-node-file | LOW | Flash simulator defaults to per-node settings file (native Linux) |
| 0009-display-ssd16xx-fill-ram-white | LOW | E-paper full-refresh-to-white anti-ghosting helper |
| 0010-uarte-pm-suspend-bounded-rxto-wait | MEDIUM | Bounds the nRF UARTE STOPRX/RXTO spin on PM suspend; unbounded upstream, wedges the mesh thread |
| 0017-wifi-esp32-rx-never-block | MEDIUM | ESP32 Wi-Fi RX callback drops a frame instead of waiting 100 ms for a net buffer; the wait livelocked RX under load |
| 0019-usb-cdc-acm-poll-out-flush | MEDIUM | USB CDC ACM: a full TX buffer is handed to the stack from `poll_out()` and `fifo_fill()`, and only the current buffer counts as free space while the bus is suspended. Without it console bursts are cut at 64 bytes and a companion can hang on its first reply after boot |

**One patch per file.** No upstream file is touched by more than one patch, so
apply order is irrelevant and no patch can be anchored inside another's added
Expand Down Expand Up @@ -1054,6 +1072,7 @@ Build strings: `docs/supported_boards.md`. Flash methods: `docs/BUILDING.md`. Ad
| ProMicro SX1262 | nRF52840 | SX1262 (E22-900M30S) | Yes | - | Button, LED, battery ADC |
| muzi works R1 Neo | nRF52840 | SX1262 | Yes | - | Buzzer, button, RX8130CE RTC, latched-rail power-off |
| XIAO nRF54L15 | nRF54L15 | SX1262 | - | - | Contacts capped at 450 |
| Seeed LR2021 LoRa Plus EVK | nRF54L15 or nRF54LM20A | **LR2021** | - | - | Two board directories, one per XIAO; the nRF54LM20A one is source-only |
| XIAO ESP32-C3 | ESP32-C3 | SX1262 | - | - | Contacts capped at 300 |
| XIAO ESP32-C6 | ESP32-C6 | SX1262 | - | - | - |
| LilyGo TLoRa C6 | ESP32-C6 | SX1262 | - | - | - |
Expand Down
4 changes: 2 additions & 2 deletions docs/DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ multiple MCU families.
bounded recovery (§8.7).
3. **Upstream portability** — most new mesh features arrive from Arduino MeshCore, so code that mirrors upstream
stays structurally identical to it (§9).
4. **Hardware breadth** — one codebase across nRF52840, nRF54L15, the ESP32 family, EFR32MG24, STM32WL and native
4. **Hardware breadth** — one codebase across nRF52840, nRF54L15/LM20A, the ESP32 family, EFR32MG24, STM32WL and native
Linux, with four radio families.
5. **Resource discipline** — static allocation for the packet path; companion builds are RAM-bound.

Expand Down Expand Up @@ -104,7 +104,7 @@ flowchart LR

| Dimension | Values | Selected by |
|---|---|---|
| Platform | nRF52840, nRF54L15, ESP32 classic, ESP32-S3/C3/C6, EFR32MG24, STM32WL, native Linux | board directory |
| Platform | nRF52840, nRF54L15, nRF54LM20A, ESP32 classic, ESP32-S3/C3/C6, EFR32MG24, STM32WL, native Linux | board directory |
| Radio | SX126x family (default), LR1110, LR2021, SX127x | devicetree / board conf |
| Role | companion (default), repeater, room server, observer | user `EXTRA_CONF_FILE` |
| UI | none, button UI, joystick/keypad UI (companion) | board Kconfig select |
Expand Down
3 changes: 3 additions & 0 deletions docs/Repeater_CLI_commands.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Repeater CLI Commands

This page has moved: the CLI reference for every role is now [CLI_commands.md](CLI_commands.md).
44 changes: 40 additions & 4 deletions formatter/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
nRF formatter tools

- QSPI is formatted for all supported boards
- Erases the internal LittleFS and BLE-bond NVS partitions, and the external QSPI flash on
every supported board that has one (see [QSPI flash](#qspi-flash) below)
- Watch out for softdevice version! Flashing the wrong version can corrupt the node and you'll need a full bootloader reflash with adafruit-nrfutil!
- You can check what softdevice version you use if you open INFO_UF2.TXT on the storage drive when in DFU mode. Bootloader should say "sxxx 6.x.x" for v6 and "sxxx 7.x.x" for v7
- That warning applies to the **`.uf2`** files, which have no version guard. The **`.zip`** packages carry an `--sd-req` guard (v6 = `0x00B6`, v7 = `0x0123`), so the bootloader *rejects* a mismatched package instead of corrupting the node.
Expand All @@ -11,8 +12,8 @@ nRF formatter tools

| SoftDevice | Boards |
|------------|--------|
| **v6** | RAK4631, RAK3401 1W, ThinkNode M1/M3/M6, RAK WisMesh Tag, LilyGo T-Echo, LilyGo T-Impulse Plus, ProMicro SX1262, Heltec T114, Heltec T096, GAT562 30s, muzi works R1 Neo |
| **v7** | Wio Tracker L1, T1000-E, Ikoka Nano 30dBm, SenseCAP Solar, XIAO nRF52840 |
| **v6** | RAK4631, RAK3401 1W, ThinkNode M1/M3/M6, RAK WisMesh Tag, LilyGo T-Echo, LilyGo T-Impulse Plus, ProMicro SX1262, ProMicro LR2021, Heltec T114, Heltec T096, Heltec Mesh Node T1, GAT562 30s, muzi works R1 Neo |
| **v7** | Wio Tracker L1, Wio Tracker L1 Pro 1W, T1000-E, MeshTracker X1, Ikoka Nano 30dBm, SenseCAP Solar, XIAO nRF52840 |

- **`.uf2`** — manual drag-and-drop onto the UF2 mass-storage drive.
- **`.zip`** — Adafruit DFU package. Used by the Mesh America configurator as ZephCore's
Expand Down Expand Up @@ -46,5 +47,40 @@ prints `Formatter DFU zip: ENABLED (sd-req=...)` when wired up, and silently ski
zip otherwise. `--sd-req` is read automatically from the build board's `board.conf`
(`CONFIG_ZEPHCORE_SD_FWID`), so it always matches the SoftDevice being targeted.

## QSPI flash

The formatter does not know which board it is running on. It tries each known QSPI pin
set in turn, and the first one where a flash answers its JEDEC ID gets a full chip erase.

| Pin set | Boards | Flash rail pin |
|---------|--------|----------------|
| Wio/XIAO/Ikoka/SenseCap | Wio Tracker L1, Wio Tracker L1 Pro 1W, XIAO nRF52840, Ikoka Nano 30dBm, SenseCAP Solar | — |
| ThinkNode M1 / TEcho | ThinkNode M1, LilyGo T-Echo | P0.12 |
| ThinkNode M6 | ThinkNode M6 (units with the flash fitted) | P0.21 |
| RAK4631 / GAT562 | RAK base boards with QSPI flash | — |
| LilyGo TEcho Lite | LilyGo T-Echo Lite | — |
| Nano G2 Ultra | Nano G2 Ultra | — |
| LilyGo T-Impulse Plus | LilyGo T-Impulse Plus (v6 build only) | P0.14 |
| MeshTracker X1 | MeshTracker X1 (v7 build only) | P0.15 |

- A flash left in deep power-down is woken first. Repeaters, room servers and observers
put it to sleep, and the reset into the formatter does not wake it.
- Where the flash sits behind a switched supply rail, the formatter raises that pin and
gives the part time to start; nothing else powers it after the bootloader.
- The erase is checked: the log shows the first bytes of the flash before the erase, and
the erase is reported as failed unless they read back blank afterwards.
- The last lines of the log say what happened: `All partitions erased successfully
(internal flash + QSPI)`, or that the internal flash was erased and no QSPI flash
answered. The second is normal on a board without one (T1000-E, RAK4631 on a base
board without flash, ...). On a board from the table above it means the QSPI was
**not** erased.

Verified on hardware: Wio Tracker L1 (as companion and as repeater) and T1000-E. The
MeshTracker X1, T-Impulse Plus and ThinkNode M1 / T-Echo rail entries follow the board
devicetrees and have not been run on those boards.

To support a new board, add its QSPI pin mapping to `known_boards[]` in
`zephcore/tools/formatter/src/qspi_probe.c` — no DTS or Kconfig changes needed.
`zephcore/tools/formatter/src/qspi_probe.c` — no DTS or Kconfig changes needed. Set
`pwr_pin` if a GPIO switches the flash's supply, and `sd` if the board exists on one
SoftDevice only: every probed pin set drives its pins on every board that build runs on,
so a set limited to its own build stays off unrelated boards.
Binary file modified formatter/SoftDevice_v6_formatter.uf2
Binary file not shown.
Binary file modified formatter/SoftDevice_v6_formatter.zip
Binary file not shown.
Binary file modified formatter/SoftDevice_v7_formatter.uf2
Binary file not shown.
Binary file modified formatter/SoftDevice_v7_formatter.zip
Binary file not shown.
Loading
Loading