fix(fleet): stop the dispatcher from re-publishing an already-open sweep PR - #691
Merged
Merged
Conversation
…eep PR Measured 2026-08-11: nine open sweep PRs, the last seven with byte-identical diffs (same 3 tags: Composite:Duration, DjVu:Note, XMP:ComponentsConfiguration). Because the fleet publishes but never merges, origin/main never advances past these mechanical fixes, so every round's gap detection (which only ever compares against origin/main) rediscovers the same gap, re-fixes it under a fresh sha, and run_sweep's own zero-delta check -- which only compares against origin_ref -- cannot catch it, since the branch really is different from origin/main, just not from a PR still open from an earlier round. Add a second idempotency gate in run_sweep, right beside the existing origin_ref one and for the same reason (skip the workspace suite, the fmt commit, the push and the PR before paying for any of them): before publishing, diff the assembled branch's tree against every currently open sweep/* PR. A tree-identical match short-circuits as "duplicate_of_open_pr" -- same durable-cursor-advance handling as zero_delta, so the stamps do not resurface next round either. Wired into the real dispatcher path via default_sweep_fn's existing list_open_sweep_prs helper. Additive and optional (open_sweep_prs_fn defaults to None), so every existing caller and test is unaffected.
This was referenced Aug 11, 2026
swackhamer
added a commit
that referenced
this pull request
Aug 11, 2026
…ouched Follow-up to #691. Measured 2026-08-11 in production: PR #692 duplicated #690 (byte-identical diffs, md5-verified) -- the #691 fix's own full-tree comparison missed it, because #690's branch was cut BEFORE #691 itself landed on origin/main, so it legitimately lacks the scripts/ changes #691 added. #692's branch, cut after #691 merged, carries those changes. A raw `diff --quiet <candidate>..HEAD` sees that unrelated drift as "different" and can never again match #690, no matter how many more times the same tag gap gets re-solved identically -- and the same defeat recurs for every future round whenever ANY unrelated commit lands on main while a sweep PR sits open, which given "the fleet publishes, it never merges" is routine. Fix: scope the comparison to `git diff --name-only origin_ref..HEAD` -- the paths THIS round's own squad merges actually touched -- instead of a full-tree diff. That isolates the tag-fix content from incidental history the two branches don't share. New regression test reproduces the exact #692-vs-#690 shape: an unrelated commit lands on main between when the open PR's branch was cut and when the new round's branch is cut, with the identical tag fix on both. Confirmed the test fails against the unversioned (pre-fix) code and passes with it. Full suite: 102 + 240 tests green.
swackhamer
added a commit
that referenced
this pull request
Aug 11, 2026
…ouched (#693) Follow-up to #691. Measured 2026-08-11 in production: PR #692 duplicated #690 (byte-identical diffs, md5-verified) -- the #691 fix's own full-tree comparison missed it, because #690's branch was cut BEFORE #691 itself landed on origin/main, so it legitimately lacks the scripts/ changes #691 added. #692's branch, cut after #691 merged, carries those changes. A raw `diff --quiet <candidate>..HEAD` sees that unrelated drift as "different" and can never again match #690, no matter how many more times the same tag gap gets re-solved identically -- and the same defeat recurs for every future round whenever ANY unrelated commit lands on main while a sweep PR sits open, which given "the fleet publishes, it never merges" is routine. Fix: scope the comparison to `git diff --name-only origin_ref..HEAD` -- the paths THIS round's own squad merges actually touched -- instead of a full-tree diff. That isolates the tag-fix content from incidental history the two branches don't share. New regression test reproduces the exact #692-vs-#690 shape: an unrelated commit lands on main between when the open PR's branch was cut and when the new round's branch is cut, with the identical tag fix on both. Confirmed the test fails against the unversioned (pre-fix) code and passes with it. Full suite: 102 + 240 tests green.
swackhamer
added a commit
that referenced
this pull request
Aug 11, 2026
…695) Second follow-up to #691/#693. Measured 2026-08-11 in production: PR #694 duplicated #692 (byte-identical diffs, md5-verified) despite #693's path-scoped duplicate check. The scoped diff DID find a difference -- but it was pure rustfmt whitespace (a 3-line match arm collapsed to one line), nothing semantic. #692's branch had already been through format_sweep_branch's fmt-and-commit step in its own round; this round's comparison ran on raw, pre-fmt worker output, because both idempotency checks (origin_ref and the open-PR one) are commit-to-commit diffs that only ever compare what THIS round has committed so far -- which, at that point in run_sweep, has never been through fmt. Unformatted content can never tree-match a PR that already went through fmt, no matter how many times the same gap gets re-solved with functionally identical code. Fix: run format_sweep_branch once, early -- right before the origin_ref zero-delta check -- so both idempotency checks compare already-formatted content. The later format_sweep_branch call (after the evidence table / judgment queue are built, where it has always lived) is usually now a no-op ("already cargo-fmt clean"); kept in place because commits_contributed resolves each squad's contribution from merge_infos boundaries recorded before any fmt call, so an early fmt commit can never leak into the evidence table regardless of when it happens -- but the evidence table's own construction still must not move earlier than it already is. New regression test reproduces the exact #694-vs-#692 shape: the already-open PR's branch carries the fmt-normalized form of a fix, this round's squad commit carries the identical fix pre-fmt. Confirmed the test fails against the pre-fix code and passes with this fix. One existing assertion (which of the two format_sweep_branch calls reports "committed") updated to match: the commit now happens at the earlier call, so the later one correctly reports false. Full suite: 103 tests green.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Measured 2026-08-11: nine open sweep PRs (#680, #682-#690), the last
seven with byte-identical diffs (md5-verified) fixing the exact
same three tags every round:
Composite:Duration,DjVu:Note,XMP:ComponentsConfiguration.Root cause: this repo's dispatcher publishes sweep PRs but never
merges them (by design — a human reviews every
[needs review]PR).That means
origin/mainnever advances past a mechanical fix sittingin an unmerged PR, so the next round's gap detection — which only
ever compares against
origin/main— rediscovers the exact same gap,re-fixes it under a fresh commit sha, and
run_sweep's own zero-deltacheck (which also only compares against
origin_ref) can't catch it:the branch genuinely is different from
origin/main, just not froma PR still open from an earlier round.
Net effect: every round burned a full workspace test suite, a push,
and a CI cycle to publish a PR that duplicated one already open,
spamming the PR queue and wasting fleet compute/API budget on already
-solved work.
Fix
Add a second idempotency gate in
run_sweep, right beside theexisting
origin_refzero-delta check and for the same reason (skipthe workspace suite, the fmt commit, the push and the PR before
paying for any of them): before publishing, diff the assembled
branch's tree against every currently-open
sweep/*PR's branch. Atree-identical match short-circuits with a new
duplicate_of_open_prstatus — same durable cursor-advance handling as
zero_delta, so thestamps don't resurface next round either.
Wired into the real dispatcher path via
default_sweep_fn's existinglist_open_sweep_prshelper (already used byadopt_open_sweep_prs).Additive and optional (
open_sweep_prs_fndefaults toNone), soevery existing caller and test is unaffected.
Testing
RunSweepIntegrationTests: acontent-identical-to-an-open-PR case (asserts
duplicate_of_open_pr,no workspace test/push/PR) and a content-different case (asserts
the duplicate gate never blocks genuinely new content).
test_overlord_sweep.py(101 tests) andtest_parallel_model_fix_loop.py(240 tests): all green.Instrument
md5sumongh pr diff <N>for #684/#686/#688/#690 — identicalhashes — is what confirmed this as a real duplicate-publish defect
rather than four coincidentally similar fixes.