Skip to content

feat(harness): declarative node config — set up ESP32 CSI nodes from one YAML file - #2545

Open
matthewxfz3 wants to merge 18 commits into
ruvnet:mainfrom
matthewxfz3:feat/ruview-node-config
Open

matthewxfz3 wants to merge 18 commits into
ruvnet:mainfrom
matthewxfz3:feat/ruview-node-config

Conversation

@matthewxfz3

@matthewxfz3 matthewxfz3 commented Nov 13, 2026 •

Copy link
Copy Markdown

Add declarative node configuration so users can plan/setup/inspect ESP32 CSI nodes from YAML while retaining explicit device confirmation and assignment safety. Current head 8cccbe5397a2cca1397db495cc2937b5f9801e5e, base main; preserves the original contributor history and includes runner #2583/security #2585.

  • Bind confirmation to displayed MAC and node ID; changed board/assignment requires a new plan before registry/flash/NVS writes.
  • Reject duplicate effective config/registry IDs; retain offline registry nodes and retirement/reassignment safeguards. No runtime guard was changed by the CI repair.
  • Fix Windows tilde-path expectation using native path semantics.
  • Propose an explicit448→480KiB unpacked-package ceiling for the added node-config module. Actual unpacked content479,863bytes, headroom11,657bytes; hard cap and source-map rejection remain. This footprint change requires normal review.
  • Carry pinned MQTT/QEMU/Windows runner repairs and the36.0.21 native security baseline; no advisory reconciliation or extra API dependency remains.

Local full harness322pass/5existing skips; security152pass/3skips;65manifest entries checked. On macOS use canonical TMPDIR=/private/tmp; prior /var alias failures remain historical evidence. Actual package positive and oversize/source-map negative controls pass. Previous heada676 terminal hosted CI:57pass/8conditional skips, including Windows, Node20/22 pack, full workspace, MQTT, QEMU/fuzz/security/bench. Published harness subtree is unchanged by the current security-workflow simplification. Fresh current-head CI pending; earlier human approvals apply to an older head.

No new physical node/USB/UDP verification or main merge was performed in this repair. Existing contributor device observations do not qualify the newly integrated stack. Landing order #2585 → #2583 → this PR.

Nov20 follow-up: current combined-head Windows harness failed before capture assertions at the fixture's two-second Python startup wait (job136191916725). The fixture now waits for actual readiness with a bounded10s startup deadline, reports early child exit/stderr immediately and removes listeners/timer on completion. This deliberately changes the test-only startup allowance, not production capture timing or session/metadata/frame assertions. Controlled2.2s startup fails on the old fixture and passes on the repair; an actual missing-script child fails immediately with its Python stderr. Full local harness322pass/5existing skips and65-file manifest pass. New Windows/current-head CI remains pending; the original failure is retained.

Footprint readback: npm dry-run reports479,863bytes unpacked and158,543bytes compressed. The actual workflow compares unpackedSize against491,520bytes and rejects source maps. Test-only readiness changes are excluded from packaged content.

Replaces the per-board esptool/provision.py chain with a declarative config applied by MAC.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@proffesor-for-testing proffesor-for-testing left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

REMAND at 59cb729. Existing targeted tests pass (28/28), but three independent synthetic production-function probes expose gaps. No real serial ports, UDP listeners or devices were opened; author branch unchanged.

  1. P1 — bind interactive confirmation to the displayed board/plan. EXECUTED. harness/ruview/src/cli.js:289-299 obtains an unconfirmed plan, asks permission for its MAC/node, then calls applyBoard again without the expected MAC/plan. node-config.js:522 re-probes and adopts whatever board now occupies the path. Probe: plan MAC A/node1; replace mocked MAC with B/node2 on the same port before the confirmed call; the provisioning mock receives --node-id 2 and the result is ok:true. Re-identification is necessary, but a changed identity must invalidate the old confirmation. Pass an immutable expected MAC/node/plan into confirmed execution and reject mismatch before flash, registry assignment or NVS writes; test replacement between plan and confirmation.

  2. P1 — registry restoration can duplicate an explicitly configured node ID. EXECUTED. harness/ruview/src/node-config.js:332-333 restores the last registry ID without checking current assignments. Probe: registry MAC B has node4; YAML now assigns node4 to different MAC A; resolveNode(config, B, registry) returns B/node4, source registry. Applying both boards produces the same stream identity. Reject/reconcile conflicting restored IDs against the full current config and active registry map before planning or writes; add stale-registry versus explicit-config collision controls.

  3. P2 — offline auto-assigned nodes disappear from status and can yield false success. EXECUTED. harness/ruview/src/node-config.js:610-614 builds registry assignments but omits their IDs from the iteration set. Probe: empty YAML node list, registry node_assigned MAC B/node4, empty mocked capture. nodeStatus returns ok:true, nodes:[], 0/0 known nodes healthy. Include active known registry IDs in status and retain an offline/no-CSI row; test disappearance/reappearance independently of explicit YAML entries.

Applicability to the managed Mac/radar lane: this is a generic CSI CLI, not a dedicated MR60 installer. Its example aggregator UDP5005 must not be copied into the current raw55063/radar5005 split without explicit configuration. Status also competes for the live receiver port; no live status/apply qualification was attempted. No C5, signed-app, USB replacement or physical radar acceptance claim is made.

Verification: frozen Git archive of this head, including its exact YAML example; node --test --test-reporter=dot test/node-config.test.mjs test/firmware.test.mjs PASS28. Separate Node imports of production normalizeNodeConfig, resolveNode, nodeStatus, applyBoard used temporary synthetic registry/config and mocked exec/capture/fetch to reproduce the findings above. Full package suite and hardware checks were not rerun.

@proffesor-for-testing proffesor-for-testing left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NO BLOCK for the three repaired review findings at 3e1af4f. This is the repair author's bounded recheck, not an independent or formal human approval.

EXECUTED: eleven new regressions; eight fail on frozen original59cb and all pass on the repair. Focused39/39; full package314pass/5existing skips/0fail with canonical TMPDIR; original default-temp hosts failures independently reproduced. Manifest65-file integrity, Gitleaks and diff checks pass. Checked the actual CLI caller passes displayed MAC/node into confirmed execution before any registry/device writes; covered same-board assignment drift, registry/config and registry/registry collisions, explicit reassignment, malformed records, offline→online status and ambiguous identity reporting.

The original REMAND remains an accurate historical verdict for59cb; its three findings are closed at this new head. CI status and original hardware/receiver qualification limits remain separate; no physical operation or merge.

@proffesor-for-testing proffesor-for-testing left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NO BLOCK within the repaired findings/current-main reconciliation scope at b02ef79. Repair-author recheck; not an independent or formal human approval.

The three original findings are closed: displayed MAC/node binds confirmation before side effects; effective retained/config ownership rejects collisions; offline registry nodes remain visible/unhealthy and ambiguous IDs are explicit. Eleven new regressions include eight original-source failures and three compatibility controls. Final reconciled head:41focused PASS; full package318PASS/5existing skips/0FAIL with canonical TMPDIR; manifest65-file integrity, Gitleaks and diff checks PASS.

Normal merge of main094 resolves the original PR's existing conflicts, preserves upstream strict CSI callback evidence/C5 watchdog/security-eFuse/flash-capacity protections and package0.9.3 bounded test concurrency. GitHub mergeable=true. Hosted checks/runs are not reported yet; no hosted-CI pass or physical qualification is inferred. Main and installed app/helper/devices were not changed. Earlier reviews remain historical for their exact heads.

matthewxfz

This comment was marked as resolved.

matthewxfz

This comment was marked as outdated.

…d board's ID

plan resolved against an empty registry, so it passed configs apply then refused, and a gone board's ID could only be freed by hand-editing devices.jsonl.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@matthewxfz3

Copy link
Copy Markdown
Author

Thanks for the review and the repair commits. Follow-up at 578ca7b for two gaps in the new node-ID conflict logic:

  1. plan and apply disagreed. planNodes resolved against an empty registry, so plan passed a config that apply then refused with node_id_conflict. plan now reads the same registry and reports conflicts (ok: false, conflicts[], STATE column).
  2. A gone board's retained ID could not be released except by hand-editing devices.jsonl. This adds a top-level retired: [<mac>]. Retired MACs are dropped from the retained inventory, so a replacement or auto-assign can take the ID. A retired board plugged back in is refused (node_retired), and a MAC can't be both retired and under nodes:. The conflict error now points at retired:.

Also:

  • reverted the redundant summarizeBootLog regex and duplicate test, so firmware.js and its test match main
  • isolated the tests from the operator's real ~/.ruview registry
$ node --test test/node-config.test.mjs test/firmware.test.mjs
ℹ tests 45
ℹ pass 45
ℹ fail 0

The 5 new regressions fail on b02ef79's node-config.js and pass here. npm test gives 320/327; the 2 failures are the pre-existing macOS /var vs /private/var hosts tests. Manifest verified.

One question on the ci.yml change in 3e1af4f (Docker build and Notify gated to pushes on main): is that meant to be part of this PR? It changes CI for every PR, so it might be cleaner as its own change.

@proffesor-for-testing

Copy link
Copy Markdown
Collaborator

NO BLOCK in the reviewed node-ID conflict/retirement delta at578ca7b2232a6e20441717b9bfd25d2c738dd3a3.

EXECUTED:45 focused node-config/firmware tests passed; the complete harness run passed322 tests with5 skips after installing the pinned development dependencies and using canonical /private/tmp for macOS fixtures. The first full run lacked @metaharness/flywheel and hit existing /var versus /private/var fixture assumptions; those were environment/portability failures, not changes to this PR. No source changes were needed for this review.

STATIC: plan now resolves against the same registry as apply; retired MACs release their retained IDs and are refused if selected again; normalized config prevents active/retired overlap; offline inventory excludes retired entries. The updated boot-banner expression still recognizes the current S3 banner. Explicit confirmation, MAC re-probe and firmware identity checks remain.

No hardware, USB, flashing, Wi-Fi or live CSI acceptance performed. Hosted checks are not reported for this head. This is a scoped software review, not merge authorization or hardware qualification.

@proffesor-for-testing

Copy link
Copy Markdown
Collaborator

Current-head CI/review reconciliation at 578ca7b2232a6e20441717b9bfd25d2c738dd3a3:

  • The absent checks are explained by nine workflows awaiting execution approval (action_required), not nine executed test failures. Examples: CI43516143966, Security43516144078, QEMU43516143822. API check-runs count is0 at this head.
  • Both recorded formal approvals apply to older b02ef79ad1be51042da985cfe686e36d4bbc88a7. They predate the current plan/apply conflict and retirement changes; do not treat them as a fresh exact-head approval.
  • My November14 scoped local review at578ca7b remains historical executed evidence (45 focused tests; full harness322 pass/5 skip with pinned dependencies/canonical macOS fixtures), not hosted qualification or hardware acceptance. No source change since that reviewed head was found.
  • The reviewer's CI-scope question also remains relevant: this PR restricts the advisory Docker job and write-capable Notify job to pushes on main. That is a repository-wide workflow change and needs an explicit scope decision during current-head review; the node-config review alone does not settle it.

Next steps: approve the intended workflow executions through the repository's normal process, obtain current-head results, and refresh the maintainer review on578ca7b (or its replacement). No workflow approval, source edit, merge or device action performed in this reconciliation.

@proffesor-for-testing

Copy link
Copy Markdown
Collaborator

CI/review reconciliation at unchanged head 578ca7b2232a6e20441717b9bfd25d2c738dd3a3 (2026-11-18): execution approval for all nine waiting workflows succeeded. This was an execution gate, not proof that the acting user lacked repository write access.

The old PR-event CI checks out 6e0a481423bf3eb44444337a4952d6841999c9af, merging this head into old main 094e1400719ded1283c67d23b9c737f6738a3244; the checkout log confirms those exact parents. To qualify today's combination without editing the contributor branch, GitHub's new merge candidate e44e97bf844db617c1d10c13b7388c86ecf9dcfa (parents current main 13b47883a73d5070f35a3f04aec2ffa778b47e29 and PR head 578ca7b…, tree 3bdd82b5367329215af32550832fc18b96137e96) is pushed unchanged to feat/2545-current-main-ci-20261118. No competing PR or main merge. Full current-main CI/security/bench qualification is running. Five still-running old-base workflows were cancelled to avoid duplicate full qualification; their completed results are preserved.

Two newly executed failures remain actionable:

  1. Package footprint: npm packages43516143964 fails on both Node20/22: packed contents are 479,863 bytes, exceeding the reviewed 458,752-byte ceiling by21,111 bytes (.github/workflows/npm-packages.yml:49). Trim/package the new capability more narrowly or explicitly review a justified new footprint budget; do not simply bypass the gate. The passing unit/security tests did not exercise this pack-size gate.
  2. Windows portability: flywheel43516143894 fails harness/ruview/test/node-config.test.mjs:85: expected /home/u/p, actual \\home\\u\\p. Compare against join('/home/u', 'p') (the test already imports join) so it tests tilde expansion using native path semantics; retain the real normalized-path assertion.

The dedicated device-e2e workflow, CSI policy and fix-marker runs passed. These are source/test checks, not physical hardware qualification. The prior code-only NO BLOCK does not mean current CI is green; formal approvals on b02ef79… also remain historical for this newer head. The two CI guards in this PR restrict Docker and Notify to push/main; they were not widened or changed by this follow-up. No contributor source, approval, release or installed helper was changed.

@proffesor-for-testing

Copy link
Copy Markdown
Collaborator

Current-main candidate qualification completed on e44e97bf844db617c1d10c13b7388c86ecf9dcfa:

The separate npm package-size and Windows assertion failures above remain open. The qualification branch's push filters do not execute those same PR-only package/flywheel jobs, so its green full CI cannot clear them. Contributor branch and reviewed size ceiling remain unchanged; no main merge/release was performed.

Justinwold88 and others added 2 commits November 19, 2026 02:38
Run the MQTT test broker from the eclipse-mosquitto:2.0.18 image, build
Espressif QEMU on ubuntu-24.04, and enable git long paths before cargo
on Windows. Plan item 0.6.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T8hca9bTB4puaeBstXR9CE
@proffesor-for-testing

Copy link
Copy Markdown
Collaborator

Taking the remaining CI repairs under Dragan's selected server lane. Current contributor head578ca7b2232a6e20441717b9bfd25d2c738dd3a3 is preserved; please flag conflicting source/workflow edits.

Planned normal update in an isolated worktree: incorporate current-main and #2583's shared runner repairs without a main merge; correct the existing Windows tilde-expansion test to native path semantics; propose an explicit480KiB package ceiling for the new node-config capability. Local npm pack reproduces479863bytes (468.6KiB), including44024bytes for the new runtime module; the old448KiB ceiling predates that capability. The proposed480KiB ceiling retains a hard bound with11657bytes headroom. This is an explicit feature-footprint change for review, not a claim that the old ceiling passed or an already-approved policy.

Runtime node-config behavior, device-ID conflict/retirement guards and security manifests remain unchanged. Preserve full package, harness, Windows, broker/QEMU/fuzz qualification; no device flashing, contributor-history rewrite, main merge or release. Will provide new exact head, pack inventory, tests and fresh CI before treating ready.

Owner Dragan; agent Codex; host DraganMac; session ruview-server-repairs-20261119; lease until14:25UTC.

@proffesor-for-testing

Copy link
Copy Markdown
Collaborator

NO BLOCK in the reviewed source-repair scope at a67675d0f42b80aca4928048275f4a7082441165. Full merge qualification remains pending; this is not maintainer approval or permission to merge.

Checked the native path correction and explicit448→480KiB package proposal; node-config runtime/ID-conflict/retirement guards are unchanged.322local harness tests and152security tests pass with existing skips;65manifest files match. Actual pack positive/oversize/source-map controls pass/fail correctly. Fresh current Node20/22 package and Windows harness checks pass. Published harness tree equals the earlier tested736 tree. The footprint ceiling remains an explicit normal-review change.

Current checkpoint: {'IN_PROGRESS': 5, 'SUCCESS': 46, 'QUEUED': 5, 'SKIPPED': 5}; no failed checks observed. Still pending: QEMU Test (boundary-max), QEMU Test (boundary-min), QEMU Test (default), Security Report, Rust Workspace Tests, QEMU Test (full-adr060), QEMU Test (tdm-3node), QEMU Test (edge-tier1), QEMU Test (edge-tier0), bench compile-verify (--no-run). Source/targeted results above do not turn those jobs into executed passes. Shared baseline #2585 is included in the three contributor branches; all original ancestry is preserved through normal merges.

No main merge, helper/app replacement, plugin enablement, firmware flash or physical/customer acceptance is claimed. Reverify if head/base changes. Full current-head CI and rUv's separate review/merge decision remain the next gates.

@proffesor-for-testing

Copy link
Copy Markdown
Collaborator

NO BLOCK in the previously reviewed repair scope; final exact-head hosted qualification is now complete.

Terminal hosted qualification — 2026-11-20

Exact head a67675d0f42b80aca4928048275f4a7082441165: 57 passed / 8 conditional skips; no failed, queued or running checks. Full CI, security and benchmark compilation completed successfully. Conditional skips remain skips. This closes the earlier pending-CI checkpoint; it is not maintainer approval, server merge, installed-helper qualification or hardware acceptance. Security retains the unmodified native report separately from the narrow live-primary-verified metadata reconciliation.

@proffesor-for-testing

Copy link
Copy Markdown
Collaborator

NO BLOCK in the reviewed Nov20 reconciliation delta at 8cccbe5397a2cca1397db495cc2937b5f9801e5e; full current-head merge qualification remains pending.

Also repaired the test-only collector readiness wait after actual Windows startup timeout atb109. Controlled delayed real child: old fixtureFAIL/newPASS; missing-child and never-ready negatives stillFAIL as expected with bounded cleanup/diagnostics. Full local322pass/5existing skips; manifest65PASS. Explicit test startup allowance2→10seconds; capture/session assertions and production source unchanged. Fresh Windows/full CI pending.

Earlier terminal CI receipts remain historical and are not relabeled as passes on this new head. Same-head duplicate push QEMU/bench and unfinished supersededb109 runs were cancelled only where an active replacement exists; completed failure evidence is retained. No main merge, installed-helper/app replacement, release or sensor operation. Maintainer review/exact-head merge authority remains separate.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants