[vincentkoc]

Summary

Add an opt-in first-worker turn cap to edit handoff benchmarks. If the first worker has not produced a passing edit by the cap, the successor receives the remaining shared allowance. Existing defaults and fixed controls stay unchanged.

Record the cap in pair receipts and reject pooling runs with different caps. Matched handoff controls use the same boundary. This establishes bounded allocation; it does not claim a measured quality or cost gain.

Verification

• 130 focused agent, pair, and analyzer tests passed, including native Docker regressions.
• Removing the cap caused the no-edit regression to fail.
• Independent full-context P0–P2 review: clean.

Deployment and risk

No deployment or default promotion. The setting can change model allocation, so compare against fixed-model controls before using it for a benchmark claim.

[clawsweeper]

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

[clawsweeper]

Codex review: needs maintainer review before merge.

What this changes

Adds an optional first-worker turn cap to edit-handoff benchmarks and keeps differently capped runs separate during analysis.

Example: An edit-handoff benchmark has four shared calls and FUSION_HANDOFF_MAX_TURNS=2; the first worker only reads files.

• Before: The first worker can consume all four calls without handing off.
• After: The successor takes over after two calls with the remaining allowance and current source.

Review scores

Measure	Result	What it means
Overall readiness	🦐 gold shrimp (3/6)	Focused and source-consistent, with meaningful harness coverage and reported native runs; no shipped-entry-point experiment was supplied or required.
Proof confidence	🦐 gold shrimp (3/6)	This benchmark-only allocation change requires no shipped-product demonstration; supplied validation exercises the real agent with sandboxed fixtures and scripted model responses.
Patch quality	🐚 platinum hermit (4/6)	No actionable review findings were identified.

Product

Kind: Feature · Worth it: Yes · Fix scope: Complete
User problem: Benchmark operators cannot bound first-worker call consumption when it never produces a passing edit.
Reason: A small, explicitly enabled allocation experiment supports controlled handoff comparisons without claiming an efficacy gain or changing existing defaults.

Merge readiness

✅ Ready for maintainer review

Keep open: this maintainer-authored PR adds a useful benchmark allocation control absent from main; no actionable patch defects were found.

Priority: P3
Reviewed head: 78f4bb6921d10d877b964f5e84b0161b8c716e31

Before merge

None.

Findings

None.

Agent review details

How this fits together

The offline Fusion benchmark runner sends qualified repository tasks through sandboxed repair agents, records model calls and acceptance results, and passes saved reports to the paired analyzer.

flowchart LR
  A[Qualified repair task] --> B[Pair benchmark runner]
  B --> C[First worker]
  C --> D[Existing handoff boundary]
  D --> E[Successor worker]
  E --> F[Independent acceptance]
  F --> G[Saved pair report]
  G --> H[Pair analyzer]

Loading

Technical review

Best possible solution:

Keep the cap opt-in within the existing handoff owner, preserving shared limits, fixed controls and legacy report compatibility.

Do we have a high-confidence way to reproduce the issue?

Main's edit-handoff loop waits for a passing edit or submission, so repeated reads can exhaust every turn without a successor call; this follows directly from the source and existing no-edit test.

Is this the best way to solve the issue?

The patch is a focused extension of the existing boundary, with matching control propagation and protocol separation; the maintainer-authored proposal establishes direction.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against 1e4104fb47f7.

Provenance checked

• Edit-handoff boundary keeps the original intent (b1ddf79: Transfer an attempted repair after a passing edit while sharing call, time and spend limits.)
• Matched strong-successor control keeps the original intent (d12e6c5: Separate model-selection effects from handoff protocol effects using equivalent execution boundaries.)
• Repeat protocol comparison keeps the original intent (a1772b0: Prevent aggregation of experiments with different settings or discontinuous spending ledgers.)

Testing

Proof path: in-process harness.

Security

None.

Evidence

What I checked:

• Pinned change and main behavior: The introduced diff adds cap validation and a turn-limit trigger to the existing handoff owner; uncapped behavior remains unchanged. (scripts/fusion-agent-benchmark.mjs:304, 78f4bb6921d1)
• Matched controls and shared limits: The cap reaches the mixed candidate and matched strong handoff control, while ordinary fixed controls retain their existing settings and shared budget enforcement. (scripts/fusion-pair-benchmark.mjs:198, 78f4bb6921d1)
• Saved-report compatibility: The additive pair.json field handoffMaxTurns defaults to null; the analyzer treats omitted legacy values identically and rejects repeat protocols with different caps. (scripts/fusion-pair-analysis.mjs:156, 78f4bb6921d1)
• Focused behavioral coverage: Added sandbox scenarios cover reads, invalid actions, failed edits, passing edits and submission; the PR reports 130 passing focused tests and a cap-removal regression failure, which this read-only review did not rerun. (test/fusion-agent-benchmark.test.mjs:743, 78f4bb6921d1)
• Feature history and original intent: Vincent Koc's merged work owns edit handoff, matched controls and repeat analysis: feat(fusion): test strong execution before cheap handoff #989, test(fusion): add a matched strong-successor control #993 and feat(fusion): validate repeated benchmark comparisons #965. Historical traversal encountered a missing older object; the relevant handoff and analyzer changes were available. (scripts/fusion-agent-benchmark.mjs:185, 1e4104fb47f7)
• Repository policy and discussion: Read the complete root policy; no nested AGENTS.md, maintainer notes or pr-simplify skill were present. Applied the read-only, sole-reviewer instruction instead of running autoreview. Both discussion comments were bot status messages. (AGENTS.md:1, 78f4bb6921d1)

Likely related people:

• Vincent Koc: Raw commit b1ddf79 adds scripts/fusion-agent-benchmark.mjs:151 relative to its recorded parents. This identifies author metadata, not feature responsibility or a PR merger. (role: source-line author; confidence: high; commits: b1ddf7900dcc; files: scripts/fusion-agent-benchmark.mjs)

Preflight source

Source unknown. Source comparison at the observed PR snapshot only. Completion is self-reported; body, policy, tests, execution, and report bytes are not verified.

Labels

Label changes:

• add P3: Opt-in benchmark experimentation with no production routing or default change.
• add rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🦐 gold shrimp and patch quality is 🐚 platinum hermit.
• add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR.

Label justifications:

• P3: Opt-in benchmark experimentation with no production routing or default change.
• rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🦐 gold shrimp and patch quality is 🐚 platinum hermit.
• status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR.

Rating scale

6/6 🦀 challenger crab · 5/6 🦞 diamond lobster · 4/6 🐚 platinum hermit · 3/6 🦐 gold shrimp · 2/6 🦪 silver shellfish · 1/6 🧂 unranked krab. Overall follows the weaker of proof and patch quality; ✨ marks media proof (a screenshot, video, or linked artifact) that directly shows the changed behavior.

Workflow

ClawSweeper edits this one comment on every review. Comment @clawsweeper re-review for a fresh review only. Repair and merge execution lanes are retired.

Reviewed November 13, 2026, 8:35 AM ET / 13:35 UTC.