dream(flywheel-promotion): bind sealed scores + receipt id to the commit they decorate (evaluated) - #255
Draft
ruvnet wants to merge 3 commits into
Draft
dream(flywheel-promotion): bind sealed scores + receipt id to the commit they decorate (evaluated)#255ruvnet wants to merge 3 commits into
ruvnet wants to merge 3 commits into
Conversation
…receipt payload
gateReExecutes (ADR-235) re-runs the frozen promotion rule on a commit's
baselineScore/candidateScore/anchorScore, but those fields lived outside
the signed receipt payload (signed payload was only
{kind,id,target,verdict,primaryDelta}). A bundle editor with no signing
key could splice favorable scores onto a PROMOTED commit and
gateReExecutes would re-run the gate on forged input and still pass.
- run.ts: sign baselineScore/candidateScore/anchorScore/failureReasons
into the receipt payload alongside the existing fields (no new
computation — reuses the same anchorScore/failureReasons already
derived for the commit object).
- replay.ts: new sealedFieldsAuthentic check — when a receipt's payload
carries any of the sealed keys, its signed copy must canon-match the
commit's live fields. Old-format receipts (no sealed keys present) are
left unchecked, so every pre-existing bundle keeps passing unchanged.
- units.test.ts: 3 new tests — honest new-format receipt passes; a
post-signing splice (signature itself still verifies) is caught;
old-format receipts stay backward-compatible.
Baseline (git-stashed source, tests kept): 2 of the 3 new tests fail
with "expected undefined" — sealedFieldsAuthentic doesn't exist yet,
confirming the gap is live on tonight's main. Candidate: 50/50 tests
pass (5 files, 0 regressions). Real-bundle regression check (not just
synthetic fixtures): packages/radio/.radio-flywheel/replay-bundle.json
and kimi-k3-harness/.harness/flywheel/replay-bundle.json both still
verify pass:true with all 7 checks green, including the new one.
Composes with (does not duplicate or require) #205 and #231, both
still open.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LMZYADjuHzPcuAdq9rJHH8
…arial critic found in the first commit
An independent critic (adversarial pass, not this session's candidate
author) found that binding sealed SCORE fields alone was insufficient:
nothing checked a receipt's signed 'id'/'target'/'verdict'/'primaryDelta'
against the LIVE commit it decorates. A bundle editor with no signing
key could clone ONE genuine PROMOTED receipt onto many fabricated
commit ids (copying the genuine scores onto each fabricated commit's
live fields too, so the score-binding check alone would pass) to
manufacture a fake multi-generation promoted lineage from a single real
promotion. verifyReceipt still reports true on every clone — it's a
genuine signature, just attached to the wrong commit.
Extends the same replay.ts check (still named sealedFieldsAuthentic) to
also bind id/target/verdict/primaryDelta whenever a payload carries
them — which real receipts always have, even pre-dating this diff, so
this closes the gap for every bundle produced going forward without
requiring anything else to change.
New test reproduces the critic's exact scenario: clone a genuine
receipt onto a fabricated 'c2-forged' commit id; verifyReceipt still
returns true for the clone, but sealedFieldsAuthentic now catches
payload.id ('c1') != forged commit id ('c2-forged') and the bundle
fails replay. 51/51 tests pass (up from 50), 0 regressions. Real-bundle
regression check re-run: both committed live bundles
(packages/radio/.radio-flywheel, kimi-k3-harness/.harness/flywheel)
still verify pass:true unchanged.
Disclosed, not fixed here (composes with #231 instead of duplicating
it): the critic separately noted gateReExecutes never passes `anchor`
into promotionRule() at all, so the anti-Goodhart clause is never
re-derived during replay regardless of this fix — #231 already targets
exactly that gap.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LMZYADjuHzPcuAdq9rJHH8
docs/dream-cycle/2026-08-31-gist.md — full SOTA research, hypothesis, evaluation receipt, and adversarial-critique writeup for tonight's candidate. LEDGER.md — appends tonight's row and a placeholder row noting the 2026-08-15 -> 2026-08-30 backfill is staged in the still- unmerged PR #240, not re-transcribed here to avoid drift between two copies of the same backfill. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LMZYADjuHzPcuAdq9rJHH8
This was referenced Sep 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
MetaHarness Dream Cycle, 2026-08-31. Rotation slot:
flywheel-promotion(slot 1). Full detail: #254.
Hypothesis
Given a
ReplayBundlewhere an attacker with no signing-key access either(a) edits a PROMOTED commit's unsigned
baselineScore/candidateScore/anchorScoreto values that passgateReExecutes(ADR-235), or (b) clonesone genuine receipt onto fabricated commit ids to manufacture a fake
multi-generation lineage from a single real promotion, when the receipt
payload is extended to bind these fields and its own
id/target/verdict/primaryDeltato the commit it decorates, and a new replaycheck verifies that binding, then
verifyReplayBundleshould FAIL such abundle where it previously passed, while every honestly-produced bundle —
including real bundles committed before this fix — continues to PASS
unchanged. Part (b) was added mid-session after an independent adversarial
critic found part (a) alone insufficient.
Candidate
Two commits,
packages/flywheel/src/run.ts+packages/flywheel/src/replay.tspackages/flywheel/__tests__/units.test.ts, ~150 lines total:run.ts: signbaselineScore/candidateScore/anchorScore/failureReasonsinto the receipt payload (reuses the same locals alreadycomputed for the commit object — no new logic, no drift risk).
replay.ts: newsealedFieldsAuthenticcheck — whichever ofid/target/verdict/primaryDelta/the 4 sealed fields a receipt'spayload carries must canon-match the commit's live fields. Additive:
old-format receipts (none of these keys present) are unchecked, so every
pre-existing bundle keeps passing.
Evaluation Receipt
evaluated: accepted. Evaluator:packages/flywheel/__tests__/units.test.ts(vitest, no LLM calls). Baseline (git-stashed source, new tests kept):
3/3 new tests fail — the check doesn't exist on
main. Candidate:51/51 tests pass (5 files), 0 regressions. Real-bundle regression check
(not just synthetic fixtures):
packages/radio/.radio-flywheel/replay-bundle.jsonand
kimi-k3-harness/.harness/flywheel/replay-bundle.jsonboth re-verifiedpass: true, all 7 checks green, unchanged.Baseline Comparison
See Evaluation Receipt — baseline is
mainat session start(
b611993d7088dff877f5713e41031a714e77bfc0), same evaluator, samefixtures, before/after diff isolated to the 3 touched files.
Darwin Lineage
Not run — direct correctness/security fix to fixed logic, not a
Darwin-scoped tunable parameter. N/A, not skipped.
Flywheel Evidence
N/A — this candidate modifies
@metaharness/flywheelitself; running itthrough its own gate would be circular. The vitest evaluation receipt
above is the evidence.
Reward Hack Check
Checked explicitly by an independent adversarial critic sub-agent against
the reward-hack checklist (weakened tests, altered gold data, cherry-picked
cases, hidden regressions, cost hiding): none found. No existing test
weakened; only additive.
Security Review
This candidate is the security fix — see #254's Deep Dive Findings. No
new attack surface introduced elsewhere; no MCP/tool-policy/credential
surface touched. Disclosed, not fixed here: the critic separately found
gateReExecutesnever passesanchorintopromotionRule()duringreplay, so the anti-Goodhart clause isn't actually re-derived — tracked by
already-open PR #231, not duplicated here.
Regression Analysis
51/51 tests pass vs. 50/50 pre-candidate (net +1 test file count unchanged,
+4 new tests, 0 removed/weakened). Both real committed replay bundles
unaffected. No perf concern — the new check is O(commits) per replay, same
order as existing checks.
ADR
None — this is a bug-level correctness/security fix to an existing,
already-ADR'd mechanism (ADR-235's sealed-score re-gate), not a new
architectural decision.
Research Gist
docs/dream-cycle/2026-08-31-gist.md(GIST=LOCAL— no gist-creation toolin this session's toolset, matching every prior night). Includes the full
Adversarial Critique section: the critic's exact finding, the fix, and an
explicit honesty-note disclosing that the critic-driven 2nd commit was
self-validated (via a test reproducing the critic's scenario) rather than
independently re-reviewed by a fresh critic pass.
Issue
#254 (full findings, tonight's ledger/backlog check, recommended human
actions).
Witness
Merge Policy
Human review required. Do not self-merge. Do not autonomously promote
Flywheel state.
Note on tonight's larger finding: this is dream-cycle PR #13 in an
unreviewed queue of 12 (now 13) — see #254 and issue #239 for the backlog
escalation. This PR is deliberately small (2 commits, ~150 lines, fully
test-backed) per this repo's own "bias toward small reviewable candidates
when merge signal is poor" guidance.
Generated by Claude Code