feat(sband) 3/3: topology re-enable — HWIL-validated, gated on fault-injection + 24h soak - #450
Draft
Mikefly123 wants to merge 15 commits into
Draft
feat(sband) 3/3: topology re-enable — HWIL-validated, gated on fault-injection + 24h soak#450Mikefly123 wants to merge 15 commits into
Mikefly123 wants to merge 15 commits into
Conversation
Un-comments the sband_nrst/sband_rx_en/sband_tx_en devicetree nodes that have sat dead-commented in the shared v5-family base since before this reintegration effort. Verified against every board's actual LoRa pin usage (not just the comments): v5/v5c/v5d's LoRa config never claims gpio0 21/22, so the raw-GPIO mapping is conflict-free there. v5e's SX126x LoRa module does claim those same pins for its own tx/rx-enable, but v5e already redeclares sband_rx_en/sband_tx_en on the MCP23017 in its own .dts; devicetree node-path merging (last property wins -- the same mechanism v5e's file already relies on for its &lora0 override) makes v5e's MCP-based mapping take precedence, while sband_nrst (never overridden by v5e) is inherited from this shared block. Confirmed by an actual build for both v5e and v5d, not just reasoned about. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…to the topology Slice 3.1 of the S-Band reintegration plan. In instances.fpp: adds SBAND_STACK_SIZE (8 KB, dedicated heap-allocated stack via CONFIG_DYNAMIC_THREAD_ALLOC pool-fallback, documented inline), uncomments the sband instance (now using the new stack constant instead of the shared 4 KB pool default) and its gpioSbandNrst/RxEn/TxEn/IRQ driver instances, and adds a new gpioSbandBusy instance to drive the getBusyLine port PR 2 added. In topology.fpp: uncomments the ComCcsdsSband.Subtopology import, all sband/comDelaySband instance declarations, the comSplitter/ authenticationRouter connections into ComCcsds_CdhCore, the full CommunicationsSBandRadio connections block, the file downlink/uplink sband hooks, and the previously-reserved rate-group slots (10Hz [3] aggregator, [11] sband.run, [12] comDelaySband.run; 1Hz [1] comQueue.run, [4] commsBufferManager.schedIn -- all verified free on this branch before uncommenting). Adds a new getBusyLine -> gpioSbandBusy.gpioRead connection alongside the existing MyConnectionGraph GPIO wiring. In ReferenceDeploymentPackets.fppi: uncomments every existing sband/ ComCcsdsSband packet assignment (Radio, Health, HealthWarnings, HealthAuxiliary, Authenticate, Beacon, omit block) and adds the three new PR 2 fault-policy channels (RadioFaulted, ConsecutiveRadioFailures, RadioResetCount) to the Health packet. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Uncomments the ComCcsdsSband PingEntries/SubtopologyTopologyDefs includes and TopologyState::comCcsdsSband field (subtopology state is empty -- no new plumbing needed beyond the include). In ReferenceDeploymentTopology.cpp: uncomments the sbandNrst/RxEn/TxEn gpio_dt_spec declarations and their .open() calls, the SPI cs_ctrl/config block, spiDriver.configure(), and sband.configureRadio() (discarded, matching SBand.cpp's own (void)configureRadio() convention -- a failed initial configure is not fatal at boot, SBandFaultPolicy surfaces it via events on first use instead). Fixes the pre-existing draft's misleading sbandTxEnIRQ name to sbandIrqGpio, and adds a new sbandBusyGpio spec for the PR 2 getBusyLine port. Both IRQ and BUSY reuse the existing rf2_io1/ rf2_io0 devicetree nodes (the two spare pins of the generic 4-pin RF header present on every board variant) rather than inventing new nodes; the BUSY assignment specifically is flagged inline as unverified against real hardware, to be bench-confirmed per SBAND-HWIL-PROCEDURE.md. In Main.cpp: uncomments the spi0 device binding (DEVICE_DT_GET(DT_NODELABEL(spi0))) and its TopologyState assignment, which SPand's SPI driver needs at boot. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Corrects the plan's "CI builds for v5c/v5d/v5e" claim (no such CI matrix exists; only v5e is CI-built today) and documents the actual per-board devicetree investigation: the gpio0 21/22 conflict between v5e's LoRa tx/rx-enable and the dead-commented sband rx_en/tx_en block, why v5e's MCP-based mapping avoids it, and why no board needed to be scoped out once the pin usage was checked against real hardware rather than assumed from the comments. Notes v5e and v5d were build-verified; v5/v5c were not (outside this task's gates) but are expected to build cleanly by the same reasoning. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
New PROVESFlightControllerReference/test/hwil/SBAND-HWIL-PROCEDURE.md, covering the functional pass, fault injection (including the boot-failure case from Slice 1.2), and 24 h dual-radio soak gate that this environment cannot run (no bench hardware). Every step references real channel/ event/command names pulled from the generated v5e dictionary, not assumed F Prime conventions. Flags the two open bench-only unknowns this PR could not resolve in software: the BUSY-line pin assignment (rf2_io0, inferred not confirmed) and the absence of any onboard S-Band frame/byte counters to directly evidence the soak's "RX/TX counters advance" gate (ground-side + authentication-router counters are the best available proxy). Explicitly marked NOT RUN throughout. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…nels Re-enabling the S-Band topology added ~18 telemetry channels, pushing the deployment's total to 214 — over the fixed TLMPACKETIZER_HASH_BUCKETS=202. TlmPacketizer::findBucket boot-asserts (bucket exhaustion) the moment the channel count exceeds the bucket pool, BusFaulting before any TM is emitted. Found on HWIL: main and PR1 boot clean, PR3 crashed 100% reproducibly at TlmPacketizer.cpp:180. 256 covers 214 with margin. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…gation Adds CONFIG_SYS_HEAP_RUNTIME_STATS=y (12 bytes/heap, negligible) so the libc malloc heap's free/allocated/max byte counters (z_malloc_heap.heap-> free_bytes/allocated_bytes/max_allocated_bytes) can be read live via GDB at any halt point, without needing console/printk (which stays disabled to avoid desyncing the GDS deframer). No functional change. This session used it plus static ELF analysis to establish the malloc arena total (164112 B, matching the toolchain's reported 69.18% RAM use) but could not complete live incremental-allocation tracing: boot on the bench today hits an early BusFault (before main()/topology setup, before any malloc() call, confirmed via z_malloc_heap showing zero allocations) that reproduces even on this exact pristine commit, unrelated to the S-Band topology changes. See session report for details and options. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… investigation" This reverts commit ce63c10.
CONFIG_DYNAMIC_THREAD_ALLOC=y (PR1) routes oversized stack requests to k_malloc, but the kernel system heap size was never set: it collapsed to the 1KB mqueue contribution, so the SBand thread's 8KB stack allocation returned NULL at task start. Confirmed on HWIL bench 2026-07-15/16. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…haust the malloc arena HWIL forensics (2026-07-16, vector-catch at first fault) proved the PR3 boot BusFault is heap exhaustion, not corruption: each ComCcsds subtopology costs ~35.6KB of arena (comQueue table 13.5K, buffer pool 11.2K, frame accumulator 1K, two OS queues 9.9K) and the third stack ran the arena dry during configComponents at ComCcsdsUart::commsBufferManager.setup. S-Band is a secondary downlink: give it smaller queues (comQueue 10, aggregator 8, events depth 15) and a 3+3 buffer pool (~-18.4KB), shrink the dynamic thread pool to the 24 tasks the deployment actually starts (sband's 8K stack uses the k_malloc fallback, so 23 slots + 1 spare, +4KB arena), and right-size the kernel heap to 12KB (+4KB arena). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
HWIL trace (port-invoke breakpoints, 2026-07-16) proved every downlink priming READY was being silently dropped: the comStatusIn async hop's 20-deep dispatch queue overflows during the bring-up event storm (this bench board is missing most I2C sensors, so bring-up emits dozens of warning events into all three comQueues via the splitters), and the overflow assert is non-halting. Lost priming = permanently deadlocked downlink: late statuses die in ComAggregator's assertNoStatus action, so nothing ever recovers. This deadlock reproduces on any current-main image; the last hardware-verified beaconing image predates the aggregator architecture. Add primeDownlinkQueues() and call it from main() three times at 1s intervals after setupTopology, paired with the lib/fprime ComQueue change that makes a redundant READY benign. Verified on HWIL: queued boot events flush immediately and Beacon frames flow at cadence. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…t overlay The ComQueue re-prime and CDC-ACM poll-mode fixes live on local submodule branches that cannot be pushed (submodules track upstream remotes), so export them under patches/ per repo convention. Also commit the direct-boot devicetree overlay used to flash the bench v5e (no mcuboot bootloader on that board). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 100 kHz SPI clock cost ~25 ms per 252 B buffer upload, a constant ~33.5 ms/packet driver overhead across all LoRa bandwidths (see proves-gs-project/spikes/REPORT-sband-goodput.md). Measured on the v5e bench at 4 MHz the per-packet overhead drops to ~13.3 ms: BW406.25: 1613.1 -> 1849.0 B/s (period 156.2 -> 136.3 ms) BW1625: 3933.7 -> 5719.3 B/s (period 64.1 -> 44.1 ms) Zero loss on the forward ladder. Flight image validated end-to-end on HWIL: clean boot, RadioFaulted=False, ConsecutiveRadioFailures=0 after a TX-enabled soak, and flight TM frames deframed over the air by the passthrough receiver. The SX1280 supports up to 18.18 MHz; the same spi0 bus already runs the SD card at 24 MHz. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
The submodule patches are byte-exact (trailing whitespace in diff context lines is significant); the trailing-whitespace and end-of-file-fixer hooks were rewriting them in CI, which both fails the lint job and would corrupt the patches. Verified: git apply --reverse --check passes for both live patches and sha256 hashes are unchanged after a full pre-commit run. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The integration jobs build from pristine submodules, but PR3's primeDownlinkQueues() re-prime hits stock ComQueue's FW_ASSERT on a READY-while-READY status — the exact behavior the exported fprime-comqueue-reprime-tolerance.patch fixes. Every CI board boot therefore asserted at runtime and the radio/uart integration jobs failed with gds_working=False. Wire both sband patches into the existing make-submodules patch step (same apply/already-applied/fail guard as the fprime-gds version patch) so CI builds the same tree the HWIL bench validated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Third of the stack (base: #449). Wires SBand + a third ComCcsds subtopology (ComCcsdsSband) into ReferenceDeployment. Draft: the merge gates from the reintegration plan (fault injection 3.3 + 24 h dual-radio soak 3.4) have not run yet. Everything below is bench-verified on a v5e.
What
sbandinstance (8 KB heap-backed stack via PR 1's DYNAMIC_THREAD_ALLOC), ComCcsdsSband subtopology, v5.dtsi S-Band GPIO revival (v5e MCP overrides win by DT merge), full fault-surface telemetry packeted.TLMPACKETIZER_HASH_BUCKETS202→256 (PR 3's channels overflow the table → boot assert).CONFIG_HEAP_MEM_POOL_SIZEwas effectively 1 KB → k_malloc NULL).test/hwil/SBAND-HWIL-PROCEDURE.md+ direct-boot overlay for benches without mcuboot.⚠ Submodule dependency (read before building)
Two required fixes live in submodule trees and are shipped as committed patch files under
patches/(bytes load-bearing — never reformat): lib/fprimefix/comqueue-reprime-toleranceand zephyrfix/cdc-acm-pollmode-enable-drain. Submodule pointers are deliberately NOT bumped (upstream PRs pending — the zephyr one should also cover the poll-mode RX twin bug found 2026-07-16). After clone/west update, re-apply perpatches/README.mdor the downlink deadlocks.HWIL evidence (bench v5e pair, 2026-07-16/17)
sband.m_configured=true, fault policy pristine.authenticatesband; NO_OP + SEND_PKT executed, acks over the S-Band downlink).Remaining before un-draft
Fault injection (CS/BUSY wire pull → bounded EVRs → latch → RESET_RADIO recovery; boot-failure stack variant), then the ≥24 h dual-radio soak with the stack/heap/reset gates from the plan doc.
🤖 Generated with Claude Code