[vincentkoc]

Summary

An edit-handoff currently transfers to the cheaper worker as soon as an edit passes the original visible tests, which may already pass on broken source. Add an opt-in gate requiring the same bounded probe to fail with an assertion on frozen original source and pass on the current candidate in fresh isolated sandboxes. Recheck current visible and pinned regression tests before normal transfer.

Expose FUSION_HANDOFF_PROBE_GATE=1 for regression-gated edit/matched-handoff runs. Explicit turn limits still transfer unfinished work. Keep defaults, hidden acceptance and shared call/cost/time limits unchanged. Record the gate in receipts and reject mixed repeated protocols.

Verification

• 122 focused unit/runner checks passed; 40 optional Docker cases skipped in that invocation.
• The full 59-test agent integration suite passed with zero skips using the existing qualified image. This includes all six targeted sandbox checks: actual assertion transition, unchanged probe, runtime error, probe-induced visible regression, explicit forced transfer and invalid configuration.
• New qualified-transfer regression fails against original source with ERR_ASSERTION.
• Independent P0–P2 review with source context: scoped-clean.
• Public diff privacy check and whitespace check passed.

Deployment and risk

No deployment, release or default activation. Extra isolated checks consume the existing deadline and can reduce useful work. One discriminating probe does not prove full task correctness. A bounded matched development experiment stopped on an upstream HTTP 400 token-limit response before the lifecycle arms completed. Completed streaming-task arms passed for both matched Sol and mixed execution, but the incomplete block establishes no aggregate efficacy gain. The runner retained its uncertain charge and made no later calls or retry. No efficacy claim is part of this PR.

[clawsweeper]

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

[clawsweeper]

Codex review: needs maintainer review before merge.

What this changes

Add an optional Fusion handoff gate requiring an original-failing, candidate-passing assertion probe before normal worker transfer.

Example: An edit-handoff run enables both probe gates, makes a visible-test-passing edit, and runs console.log(1).

• Before: The passing edit triggers transfer to the successor.
• After: The first worker retains control because the probe also passes on original source, unless an explicit turn limit forces transfer.

Review scores

Measure	Result	What it means
Overall readiness	🦐 gold shrimp (3/6)	Focused implementation with meaningful native-sandbox coverage; inspected evidence establishes gate mechanics rather than end-to-end model efficacy.
Proof confidence	🦐 gold shrimp (3/6)	This benchmark-only change requires no shipped-product demonstration; the body reports native-sandbox integration coverage and an incomplete development experiment without claiming efficacy.
Patch quality	🐚 platinum hermit (4/6)	No actionable review findings were identified.

Product

Kind: Feature · Worth it: Yes · Fix scope: Complete
User problem: Benchmark operators cannot currently require evidence that a repair distinguishes broken original source before transferring work to a successor.
Reason: The maintainer-authored experiment adds a focused comparison capability while preserving default behavior, explicit allocation limits and independent grading.

Merge readiness

✅ Ready for maintainer review

Keep open: this maintainer-authored PR adds a distinct optional benchmark capability absent from current main, and no actionable patch defect was found.

Priority: P3
Reviewed head: dd4c29a74017ab5a618ad2e7e8ff0045d58fc4f6

Before merge

None.

Findings

None.

Agent review details

How this fits together

Fusion benchmark scripts coordinate model workers in isolated repair sandboxes, consume task definitions and qualification receipts, and produce independently graded results and accounting reports.

flowchart TD
  A[Task and qualification] --> B[Pair benchmark]
  B --> C[First repair worker]
  C --> D[Original and candidate probe comparison]
  D --> E[Successor worker]
  E --> F[Independent grading and reports]

Loading

Technical review

Best possible solution:

Land the optional gate using the existing comparison and regression owners while retaining independent acceptance and unchanged defaults.

Do we have a high-confidence way to reproduce the issue?

This is a feature request embodied in a patch; source inspection confirms the existing passing-edit handoff boundary, and no target code was executed.

Is this the best way to solve the issue?

The implementation is appropriately scoped: it reuses isolated candidate comparison and pinned regression replay instead of adding another execution path.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against 43d978e80dfa.

Provenance checked

• Edit-handoff boundary keeps the original intent (feat(fusion): test strong execution before cheap handoff #989: Transfer a visible-test-passing candidate to a cheaper verifier under shared budgets; the new gate changes that boundary only when explicitly enabled.)
• Forced transfer keeps the original intent (feat(fusion): bound first-worker turns before edit handoff #1031: Reserve successor calls by transferring unfinished work at an explicit first-worker turn cap.)
• Isolated probe execution keeps the original intent (feat(fusion): gate repairs on pinned regression probes #1023: Replay pinned probes in fresh sandboxes under bounded execution and cleanup deadlines without mutating the live candidate.)
• Repeated comparison protocol keeps the original intent (feat(fusion): validate repeated benchmark comparisons #965: Reject aggregation across differing experiment settings and retain explicit accounting and outcome checks.)

Testing

Proof path: in-process harness.

Security

None.

Evidence

What I checked:

• Maintainer direction: GitHub identifies the author as MEMBER; the PR explicitly limits the change to an optional benchmark experiment without deployment or efficacy claims. (dd4c29a74017)
• Distinct capability absent from main: Current main transfers after a passing edit or explicit turn limit; it contains neither handoffProbeGate nor FUSION_HANDOFF_PROBE_GATE. (scripts/fusion-agent-benchmark.mjs:306, 43d978e80dfa)
• Qualification boundary: The introduced patch compares the same probe against frozen original and current editable source in fresh sandboxes, then checks visible tests and pinned regressions; edits invalidate qualification and finish cannot bypass it. (scripts/fusion-agent-benchmark.mjs:393, dd4c29a74017)
• Existing allocation contract preserved: An explicit handoffMaxTurns still transfers unfinished work; the maintainer-authored PR acknowledges that additional sandbox checks consume the existing deadline. (scripts/fusion-agent-benchmark.mjs:313, dd4c29a74017)
• Receipt compatibility: The serialized handoffProbeGate field is optional when reading older reports, defaults to false in analysis, and participates in repeated-protocol matching without rewriting existing reports. (scripts/fusion-pair-analysis.mjs:70, dd4c29a74017)
• Focused behavioral coverage: Added native-sandbox scenarios cover discriminating assertions, unchanged probes, runtime failures, probe-induced visible regressions and forced transfer; runner and analyzer tests cover propagation and protocol separation. These tests were inspected, not executed during this read-only review. (test/fusion-agent-benchmark.test.mjs:177, dd4c29a74017)

Preflight source

Source unknown. Source comparison at the observed PR snapshot only. Completion is self-reported; body, policy, tests, execution, and report bytes are not verified.

Labels

Label changes:

• add P3: This is an optional benchmark experiment with no production routing or default change.
• add rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🦐 gold shrimp and patch quality is 🐚 platinum hermit.
• add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR.

Label justifications:

• P3: This is an optional benchmark experiment with no production routing or default change.
• rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🦐 gold shrimp and patch quality is 🐚 platinum hermit.
• status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR.

Rating scale

6/6 🦀 challenger crab · 5/6 🦞 diamond lobster · 4/6 🐚 platinum hermit · 3/6 🦐 gold shrimp · 2/6 🦪 silver shellfish · 1/6 🧂 unranked krab. Overall follows the weaker of proof and patch quality; ✨ marks media proof (a screenshot, video, or linked artifact) that directly shows the changed behavior.

Workflow

ClawSweeper edits this one comment on every review. Comment @clawsweeper re-review for a fresh review only. Repair and merge execution lanes are retired.

Reviewed November 13, 2026, 1:33 PM ET / 18:33 UTC.