Skip to content

guard(queue): recognize existing batches -- no duplicate batching + generated-file tolerance - #734

Merged
matt82198 merged 1 commit into
mainfrom
guard/queue-batch-dedupe
Aug 3, 2026
Merged

matt82198 merged 1 commit into
mainfrom
guard/queue-batch-dedupe

Conversation

@matt82198

Copy link
Copy Markdown
Owner

The defect (first real-world escape from the merge-queue advancer)

Two consecutive --advance passes each opened a new integration batch over the
same seven members:

batch PR branch members
#727 integrate/q-1785727927 #717 #689 #696 #702 #712 #716 #723
#728 (duplicate, closed by hand) integrate/q-1785729091 identical

Root cause

The merge-queue-batch label does not exist in this repository. So:

  1. build_batch called gh pr edit <n> --add-label merge-queue-batch, which failed,
    and ignored the result -- merge-queue batch q-1785727927 #727 shipped with "labels": [].
  2. Every later pass called gh pr list --state open --label merge-queue-batch, which
    returns [] at exit 0 when the label is undefined. Not an error -- an empty
    answer. Batch discovery was blind.
  3. The members still carried merge-queue, so they were re-admitted, found
    file-disjoint, and batched all over again.

Verified against live GitHub before writing the fix:

$ gh label list | grep -E 'merge-queue|batch'
merge-queue      Queued for the merge-queue advancer daemon
merge-priority   Jump the merge queue          # <- no merge-queue-batch, no queue-rejected

$ gh pr list --state open --label merge-queue-batch --json number ; echo "EXIT=$?"
[]
EXIT=0

$ gh pr view 727 --json labels
{"labels":[], ...}

Fix mechanism -- discovery no longer rests on a label

The daemon is stateless by design: labels and branches are its state. A label is a
write that can silently fail; the integration branch name is minted by build_batch
and pushed before the PR exists
, so it cannot. The branch is therefore authoritative
and the label is a convenience on top of it.

  • list_open_batches() unions the label filter with open PRs whose head matches
    integrate/q-<digits>, deduplicated on PR number.
  • resolve_batch_members(batch) reads the body Members: line (already written by
    build_batch, so the common path costs no git call), falling back to parsing the
    integrate #N into ... merge-commit subjects on the batch branch for any batch opened
    before that contract.
  • _advance resolves every open batch's member set before any queue work,
    records it in summary["batched_members"], excludes those numbers from the queue, and
    evaluates the existing batch (merge on green / dissolve on red, unchanged logic) first.
    A PR whose own head is an integrate/q-* branch can never enter the member queue.
  • Stale batch: no resolvable members and git ls-remote --heads origin <branch>
    empty -> batch_branch_missing exception row. Members are not unlabelled, the batch
    PR is left open for a human, and nothing is rebatched.
  • Closed-without-merging batch: not open, so not discovered; its members simply
    re-enter the queue on the next pass. Covered by a test.
  • apply_batch_label() now reads the label back after applying it, creates the
    label once if it is missing, retries, and rows batch_label_failed if it still did not
    stick -- the silent failure that started all of this can no longer be silent.

Second bug: dirty tree from a self-inflicted generated file

tools/verify_test_suite_count.py --check auto-corrects and writes tests/CLAUDE.md
([AUTO-CORRECT] Updated tests/CLAUDE.md and continuing). When that gate ran in the
scheduled task's project root, the advancer's next pass saw a dirty tree and refused to
build a batch -- stalled over output the repo itself generates.

worktree_is_safe() now:

  1. Checks the branch first, so a tree someone else has checked out is rejected before
    this function would modify anything in it.
  2. Restores only paths in the new tools/generated_paths.py registry, and only when
    every dirty path is registered. One unregistered edit poisons the tree and nothing
    is touched.
  3. Uses targeted git restore -- <path>, one path at a time. Never git stash (the
    stash stack is shared across worktrees and would eat another lane's WIP), never a
    blanket revert. Re-reads status afterward and still refuses if the tree did not come
    clean -- e.g. an untracked file, which git restore cannot undo.

tools/generated_paths.py is the single registry (tests/CLAUDE.md, tools/CLAUDE.md)
with the three membership criteria written down; tools/auto_merge.py already hard-codes
the same pair, so this is where that list should live.

Tests: 91 -> 114 (all green)

TDD: the 18 new assertions were written first and failed against origin/main. The
headline regression drives a full run_pass over the exact production gh responses
of that second pass (batch-label list [], label-less open list showing #727, seven
merge-queue members) and asserts zero pr create and zero branch construction. Before
the fix it failed with:

AssertionError: Lists differ: [('pr', 'create', '--base', 'main', '--hea[286 chars]n.')] != []
 : an already-open batch must never be rebatched

New coverage: branch-based discovery when the label is absent; label+branch dedupe;
member exclusion; closed batch requeues members; batch branch never requeued;
is_batch_branch boundaries; commit-subject member fallback; body-wins-over-fallback
(and costs no git call); deleted branch -> batch_branch_missing; missing label created
then reapplied; unlabelable batch rowed; label creation never forced; generated-file
restore proceeds / is targeted / refuses non-generated dirt / refuses mixed dirt /
refuses when restore did not clean; wrong branch rejected before any restore; porcelain
parsing (rename, untracked, quoted); registry shared not duplicated; module never stashes.

Three pre-existing fakes were tightened to model reality rather than stubbing every
git/gh call to a blanket success (pr edit --add-label now sticks and reads back;
ls-remote answers for a live branch; the pr list fake tolerates a label-less call).

Zero-sleep proof, the forbidden-lever scan (--admin/--auto/force-push), and the
gh/git-imported-not-duplicated check all still pass unchanged.

Verification

tests/test_merge_queue.py .................... 114 passed
tests/ (full python suite) ................... 4442 passed, 20 skipped
  (7 failures are pre-existing tests/fixtures/seam_s_sample_task sample files, excluded from CI)
claudemd_lint.py ............................. [OK] No issues found
claudemd_contract.py ......................... All 14 domain CLAUDE.md files passed
claudemd_sync_gate.py ........................ [OK] No changes detected
verify_test_suite_count.py ................... [OK] Test suite counts match
verify_gates_wired.py ........................ OK
secret_scan.py --staged ...................... CLEAN: 4 files scanned

tools/CLAUDE.md documents both behaviours in the existing merge_queue.py entry,
pipe-condensed to hold the file at its 149-line cap.

Labelled merge-queue + merge-priority: the daemon should heal itself first.

Not merged.

Generated with Claude Code

FIRST real-world defect in the merge-queue advancer. Two consecutive
--advance passes each opened a NEW integration batch over the SAME seven
members: #727 (integrate/q-1785727927) then duplicate #728
(integrate/q-1785729091), members #717 #689 #696 #702 #712 #716 #723.

Root cause: the `merge-queue-batch` LABEL does not exist in the
repository. `gh pr edit --add-label merge-queue-batch` therefore failed at
batch creation, and build_batch ignored the result -- #727 shipped with an
empty label set. On every later pass `gh pr list --label merge-queue-batch`
answered `[]` with EXIT 0 (gh reports an undefined label as an empty result,
not an error), so batch discovery was blind while the members still carried
`merge-queue` and were admitted and batched all over again.

Fix -- discovery no longer rests on a label:
  * list_open_batches() unions the label filter with open PRs whose head is
    an `integrate/q-*` branch. That branch name is minted by build_batch and
    pushed before the PR exists, so it cannot silently fail to be written.
  * resolve_batch_members() reads the body `Members:` line, falling back to
    the `integrate #N into` merge-commit subjects for a pre-contract batch.
  * _advance resolves every open batch's members BEFORE any queue work,
    excludes them from the queue, and evaluates the batch first.
  * A batch with no resolvable members whose branch is gone from origin is a
    `batch_branch_missing` row (stale, left open for a human), distinct from
    `batch_members_unparseable`.
  * apply_batch_label() reads the label back, creates it once if missing,
    retries, and rows `batch_label_failed` rather than passing silently.

Second bug, dirty tree: `verify_test_suite_count.py --check` auto-corrects
and WRITES tests/CLAUDE.md, so a gate run in the scheduled task's project
root left the tree dirty and worktree_is_safe refused to build a batch over
the repo's own generated output. worktree_is_safe now checks the branch
BEFORE touching anything, and `git restore`s only paths in the new
tools/generated_paths.py registry -- and only when EVERY dirty path is
registered. One unregistered edit poisons the tree and nothing is touched.
Never `git stash` (shared across worktrees), never a blanket restore of all.

Tests: 91 -> 114. The regression test drives a full run_pass over the exact
production gh responses of that second pass and asserts zero `pr create`.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@matt82198 matt82198 added merge-queue Queued for the merge-queue advancer daemon merge-priority Jump the merge queue labels Aug 3, 2026
@matt82198
matt82198 merged commit 1360d2b into main Aug 3, 2026
12 checks passed
@matt82198
matt82198 deleted the guard/queue-batch-dedupe branch August 3, 2026 04:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-priority Jump the merge queue merge-queue Queued for the merge-queue advancer daemon

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant