OTA tests + essential fixes for the key-store branch: devicetree-safe FlashWorker region, ComQueue arming fix, hardware-in-the-loop OTA CI

[Mikefly123]

Adds the OTA integration test suite and two essential fixes to hmac-to-storage (PR #472), all bench-validated on 2026-08-03/04 against exactly this content merged with the OTA topology (two V5e units).

Commits

1. fix(FlashWorker): resolve the update region from the devicetree — minimal port of PR OTA: fix the flash-area ID that bricks the board, add an integration test and a CI job #492's fix, and load-bearing on this branch specifically: adding keystore_partition renumbers Zephyr's flash-area IDs (assigned by devicetree dependency ordinal; the keystore node sorts first via the fstab reference):

   Flash area ID	main	this branch
   0	mcuboot	keystore
   1	slot0	mcuboot
   2	slot1 ✅	slot0 — the running image 💀
   3	slot2	slot1

   Verified in this branch's generated devicetree_generated.h. With the old hardcoded REGION_NUMBER = 2, the first PREPARE_UPDATE erases the running firmware — an instant brick, before any image is uplinked. Now resolved via PARTITION_ID(slot1_partition) plus a static_assert that the staging region can never overlap the running code partition.

2. fix(comms): fprime pin bump — ComQueue comStatusIn becomes block — fixes issue ComQueue permanently wedges in WAITING when the one-shot arming comStatus is dropped at boot (silent total downlink loss; exposed by PR #472's littlefs Kconfig) #494: the one-shot arming comStatus=SUCCESS shares ComQueue's IPC queue with the boot event burst; when dropped (invisibly — the FW_ASSERT reports to a disabled console and continues) the downlink wedges in WAITING forever, every boot, both UART and LoRa. This branch's littlefs mount shifts boot timing enough to lose that race deterministically — unfixed, this branch merged with an OTA-workload topology produced a 100%-reproducible mute board. Same fork commit (03c645123) as PR fix(comms): make ComQueue comStatusIn blocking so the one-shot arming status can't be dropped (#494) #495 (the standalone fix against main); whichever merges first makes the other's pin bump trivial.

3. test(ota): the OTA image-swap suite — full swap-and-revert lifecycle (uplink → on-board CRC verify → PREPARE/UPDATE/CONFIGURE_NEXT_BOOT[TEST] → reboot, assert new build-id → reboot without CONFIRM, assert MCUBoot auto-revert) plus negative paths, parametrized uart/lora. Includes make ota-test-image (unique build-id via ephemeral git tag, signing untouched), the --ota-image/--ota-build-id fixtures, an autouse post-reboot sequence-number resync (issue Svc::FileUplink emits FileReceived before the file is durably closed, causing spurious file-not-found #461 write-ahead persistence desyncs ground on every in-suite reboot), and UART-only downlink-repeater configuration inside the test (RAM-only setting, re-applied after each reboot, each application followed by a resync — an authenticated send between resync and COLD_RESET reliably desyncs otherwise).

4. ci: hardware-in-the-loop OTA — new ota-test-image job (parallel with build); integration-uart runs the UART OTA test last in the GDS phase; the slow LoRa variant runs on a new nightly cron only. With this, the FlashWorker regression class is caught by CI on the first PREPARE_UPDATE instead of on a bricked bench board.

Bench evidence

• Swap + revert + negative paths green multiple runs across 2026-08-03/04; full suite in under 5 minutes at --file-uplink-cooldown 0.05 (the 0.4 s default is LoRa pacing — pure dead air on USB CDC).
• Pre-fix vs post-fix comStatus behavior verified deterministically in both directions on two boards (see ComQueue permanently wedges in WAITING when the one-shot arming comStatus is dropped at boot (silent total downlink loss; exposed by PR #472's littlefs Kconfig) #494 for the GDB traces).

Caveats for reviewers

• One unreproduced anomaly: a single overnight bench session showed a dead-downlink wedge with issue Svc::ComAggregator: 10 Hz timeout signal FW_ASSERTs (queue FULL) whenever the com path pauses — board-killing on ground test #432's signature (aggregator IPC queue full) on a warm, idle board. A physical power cycle cleared it and it could not be re-armed despite three full suite runs, a 30-minute idle soak, and warm-reset cycles. Noted for awareness; GDB capture tooling is staged on the bench if it recurs. Svc::ComAggregator: 10 Hz timeout signal FW_ASSERTs (queue FULL) whenever the com path pauses — board-killing on ground test #432's droppable-signal pattern is the same disease class ComQueue permanently wedges in WAITING when the one-shot arming comStatus is dropped at boot (silent total downlink loss; exposed by PR #472's littlefs Kconfig) #494 fixes and likely deserves the same treatment.
• Known separate flake: TEST-image boots log SequenceNumberWriteFailed + PrmFileReadError and the board's sequence counter jumps ~3000 per reboot — the reason the resync choreography in the test is as careful as it is. Issue to be filed; it is pre-existing and not introduced by this PR.
• Build verification of this exact branch was static only (py_compile + pytest collection); the identical file content and fprime pin content built and ran on the bench branch. CI on this PR provides the compile proof.

🤖 Generated with Claude Code

https://claude.ai/code/session_01WpBURCutAx8281i59nj6fo

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 6f5225d0-c10d-4a4d-8416-3dd61204567a

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}