dream(flywheel-promotion): verifyReplayBundle didn't verify the full diagnostic ledger (evaluated)

[ruvnet]

MetaHarness Dream Cycle, 2026-08-16. Deep surface: flywheel-promotion (slot 1). Full detail, receipts, candidate-selection scoring, adversarial-critique writeup, and evals-verticals/bench scan findings: #204.

Hypothesis

Given @metaharness/flywheel's verifyReplayBundle, which verified Ed25519 receipts and gate re-execution only across the promoted chain, when receipt verification (including an id+verdict cross-check against the signed payload) is extended to every entry in all_commits, and a PROMOTED commit missing sealed baselineScore/candidateScore is treated as a gateReExecutes failure instead of a silent skip, then a bundle carrying a tampered/spliced/unauditable ledger entry should FAIL replay (pass: false) where it previously passed, while every honestly-produced bundle continues to PASS unchanged, subject to: (a) zero regression in the existing suite, (b) purely additive ReplayVerdict.checks, (c) meetsPromotionRule itself untouched, (d) diff materially under 300 lines.

Candidate

verifyReplayBundle() (the function an external reviewer uses to establish trust in a promotion run "with no trust in the producer") only verified receipts on the promoted chain, never on all_commits — the "full diagnostic ledger" of every candidate tried, promoted and rejected, that analyzeBundle's rejection-reason/cost-per-win reporting reads — even though run.ts signs every commit it mints regardless of verdict. Separately, gateReExecutes silently skipped (rather than failed) a PROMOTED commit missing sealed scores. Fixed: added an allCommitsReceipts check; hardened gateReExecutes to fail on unauditable promotions. An independent adversarial critic then found the first-drafted fix was itself incomplete: a verified signature only proves a payload is internally consistent, not that it's attached to the right commit — neither the new check nor the pre-existing chain check cross-verified a commit's own id/verdict against its signed payload, so a REJECTED commit's outer verdict could be flipped to PROMOTED (receipt splicing). Fixed pre-ship with a receiptMatchesCommit() cross-check applied to both checks. 3 files, 132 insertions / 5 deletions (well under the 300-line target).

Evaluation Receipt

Deterministic reproduction (not sampled):

• packages/flywheel unit suite: baseline 47 → candidate 52/52 passed, 0 regressions.
• Before/after repro (fix stashed): the 4 new tests targeting the fix fail exactly as predicted (18/22 pass, 4 fail non-vacuously); fix restored → 52/52 green.
• tsc --noEmit: clean. Full repo npm run build: clean (no wasm/NAPI degradation tonight).
• Real-bundle regression check, beyond unit tests — every committed ReplayBundle in the repo re-verified against the hardened logic: packages/radio/.radio-flywheel/replay-bundle.json, kimi-k3-harness/.harness/flywheel/replay-bundle.json, packages/darwin-mode/bench/swebench/proof-bundle-swebench.json, experiments/signal-flywheel/bundle.json (via its own verify.mjs) — all four still PASS, allCommitsReceipts: true on every one.

Baseline Comparison

No behavioral change for any honestly-produced bundle — run.ts always signs {kind, id, verdict, ...} and always seals scores on non-root commits, so no legitimate producer output is affected. Confirmed directly against 4 real committed bundles, not just synthetic test fixtures.

Darwin Lineage

Not run — verification/security-correctness fix to library code, not a tunable routing/topology/prompt/tool/tier parameter; no Darwin mutation surface applies (ADR-071 mutation allowlist).

Flywheel Evidence

Not applicable in the "ran a flywheel evolution" sense — this candidate modifies @metaharness/flywheel's own verification code. Durable evidence is the deterministic test repro + real-bundle regression check above, committed to docs/dream-cycle/2026-08-16-gist.md.

Reward Hack Check

Purely additive diff (no existing assertion removed/weakened); no benchmark/gold-data touched; no hardcoded outputs, metric substitution, seed manipulation, or hidden preprocessing.

Security Review

This is itself a security-hardening change to a promotion-audit trust boundary — no credential/shell/network surface touched, no new dependency. Closes a receipt-splicing/verdict-flip forgery path present in both the pre-existing chain check and the newly-added all_commits check. Two related gaps are explicitly disclosed, not fixed tonight: (1) gateReExecutes still can't re-check the anchor (anti-Goodhart) clause — ReplayBundle doesn't carry rootAnchor at all; (2) failureReasons/Score fields beyond primaryDelta aren't part of the signed payload, so a bundle editor who cannot forge signatures could still alter a rejected candidate's displayed cost/reason without failing replay.

Regression Analysis

Zero regression: 52/52 flywheel tests (up from 47), tsc and full repo build clean, all 4 real committed replay bundles still verify PASS.

ADR

ADR-252 — extends ADR-235's replay-verification discipline to the full diagnostic ledger and to verdict/id authentication.

Research Gist

No gist-creation tool available in this session's toolset — committed instead: docs/dream-cycle/2026-08-16-gist.md (GIST=LOCAL).

Issue

#204 (full findings, candidate-selection scoring table, evals-verticals/bench scan results, competitor review).

Witness

session_commit : 5453c8c990824b54e05f289774e5a8b2cea0a32e
report_sha256  : dd68d96ffe115e205da78761d564563ef52b9feea0fca6ba0fec6614417fca8d
witness        : c28913c09f65f20e1a900bd0e55309bfe9fd06c26822f99eb7aeb0688c6f8936

Verifier procedure is in the gist's Witness section and was independently self-verified (round-trip reproduced) before this PR was opened.

Merge Policy

Human review required. Do not self-merge. Do not autonomously promote Flywheel state.

Recommendation: ACCEPT-WITH-CAVEATS — real, independently-critiqued, reproducible fix to a real (if narrow) trust-model gap. Two comparable-severity gaps (anchor re-verification, Score-field signing) are explicitly disclosed as still-open, natural next-night follow-ups — not silently fixed under review pressure, not silently left undisclosed.

---

Generated by Claude Code

[ruvnet]

Superseded by #258, merged as e1c92ab2 and published as @metaharness/flywheel@0.1.11.

I verified the subsumption rather than taking the consolidation claim at face value — by running this PR's own test file against #258's source:

• dream(flywheel-promotion): verifyReplayBundle didn't verify the full diagnostic ledger (evaluated) #205 — 22/22 pass unchanged.
• dream(flywheel-promotion): gateReExecutes never checked the anti-Goodhart anchor clause (evaluated) #231 — 4 tests failed at first look; dream(flywheel-promotion): bind sealed scores + receipt id to the commit they decorate (evaluated) #255 — 1 failed. All five were the same shape: cases expecting gateReExecutes === true returning false, i.e. failing closed.

Those five are fixture drift, not lost fixes. #258 adds parents/generation binding in receiptMatchesCommit(), so a signed receipt payload must match the commit's live fields; the fixtures in #231/#255 predate that binding and the stricter gate correctly refuses them. #258 carries the identical test names — a PROMOTED commit whose sealed scores RE-PASS the rule, an anchor-SURVIVING promotion, anchor EQUALS root exactly, no anchor suite used — and they all pass there with fixtures updated to carry the binding.

Net: 72 passing across 35 tests on main (up from 17), with every distinguishing concept from all three PRs present — VERDICT-FLIP, anchor, sealedFields, all_commits, cloning ONE genuine receipt. File coverage is the exact union of the three: {cli,replay,run}.ts.

Closing as superseded. If you think a specific assertion from this PR did not survive into #258, reopen and point at it — I checked by execution, but execution only covers what the tests actually assert.