Repository navigation
π fix(mergelane): release a closed or merged front on the next sweep - #16056
Conversation
A serialized-lane front PR closed, merged outside the lane, made a draft or retargeted got no further round: the lane only reads its front when that PR's own merge path calls in, and every sweep lane lists open PRs only. It held every waiter until the front timeout (up to 4x that when its checks were last seen pending). Each sweep pass now sends the lane one repo-wide ReleaseGoneFronts request per hive-serialized repo it covers. Under the per-lane lock the lane reads each front PR once and, when prShape gives a reason, ends the front with "front check: <reason>" (fenced on the epoch it read) and promotes the oldest waiter. Open fronts, including ones waiting on CI, and unreadable PRs are left alone; nothing is written to GitHub. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BQYnPGFJx81c8JXR3o1sg1 Signed-off-by: Danathar <Danathar@users.noreply.github.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
There was a problem hiding this comment.
Read the change against the v6 tree at be3d228 (callers included). Looks correct to me; nothing blocking.
What I verified
src/pkg/mergelane/lane.go:313-347releaseGoneFronttakes the same per-lane mutex asAdvance(lockLane, lane.go:402), so it cannot interleave with a running round and will never observe a front the lane itself is mid-merging. The epoch re-check insideStore.update(lane.go:333) correctly drops the exit if the front changed between the snapshot and the write.- It reuses
prShape(lane.go:558), the same predicate the round already applies at lane.go:677 and :754, so "gone" means exactly what the lane already means by it β no new eligibility semantics. - An open front, or a front whose
PullRequestread fails, is left alone (lane.go:321-328) β the fix cannot evict a live front on a transient GitHub error. releaseGoneFronts(gate.go:100-128) iteratesStore.Lanes(repo)which is case-insensitive on repo (store.go:245), so theowner/nameform the sweep passes matches stored records.automerge_sweep.go:1856runs the check once per pass, gated bytargets.selectsRepoandserializedLane, so direct-strategy repos and untargeted repos never reach the lane βTestLaneSweep_PassReleasesGoneFrontspins both.- Builds clean (
go build ./pkg/mergelane/ ./pkg/github/...). Changelog fragment name follows thefixed-convention;operator-reference.mdupdated.
CI note for the maintainer: the red native-queue-policy check is not this PR. Its job checks out refs/heads/v6 and fails with Could not find '.github/scripts/native-queue-policy.test.cjs' β that file does not exist on v6 either (git ls-tree origin/v6 .github/scripts/), and this PR touches nothing under .github/. It will be red on every v6 PR until the base branch is fixed.
β hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88
|
@Danathar this PR is back in your court ( Failing checks:
Reply here or push a new head to this branch (same sign-off); CI and the review swarm re-run automatically and the PR returns to the normal path. Queued in your relay fix queue (1 waiting): fixes are handed out before new work as soon as you have capacity. |
|
The red Tracked as #16060; fix in #16061 (ports the two scripts to v6). Once that lands, re-running the check here should go green β nothing to change in this diff. All test shards and coverage are already green. |
Open PR overlap checkBase branch: Conflicts are checked with every PR merged onto the current tip of
|
|
Hive status β updated in place; edits do not notify. Full history is in the Activity Monitor. hive/awaiting-ci: all checks are green at be3d228 now that #16061 landed on v6. The hive merge relay refuses this head because its earlier attempts were exhausted while native-queue-policy was red (bootstrap deadlock, not this PR's fault). @Danathar a trivial push (or a maintainer merge) unsticks it; the sweep merges it on the next green head. π Hive Agent: β hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88 Evidence bundle `#16056be3d0cb` sha256:6421e1e744d1bb84c5c439462c91026ca29277f5b2b39b13048f68d7570da814 β download from the dashboard or `hivectl review evidence #16056`. |
There was a problem hiding this comment.
Read at be3d0cb against v6, including the callers. The diagnosis holds: Acquire/Advance only run when a PR's own path calls in, every sweep lane lists open PRs, and pruneLocked (lane.go:471) touches waiters only, so a closed/merged front could only leave via expireLocked.
The fix is sound as written:
releaseGoneFront(lane.go:315) takes the same per-lane mutexAdvancetakes (lane.go:359βlockLane), re-reads the record insideStore.updateand fences onFront.PRandFront.Epoch, so a round that replaced the front between the GitHub read and the write is a no-op rather than a wrong exit.TestGoneFrontCheckIsFencedOnTheFrontItReadpins that.- It reuses
prShape(lane.go:558) rather than a second classification, so the reasons are identical to what a live round would have produced, andexitLocked/promoteLockedare the existing exit path, soLastExitand theActionFrontExit/ActionFrontEnterevents are emitted the same way. - A read error leaves the front alone and surfaces as a warn in the sweep (
automerge_sweep.go:2375), never as a release β correct fail-safe direction. - Cost is one
PullRequestread per front per pass, only forhive-serializedrepos the pass targets (selectsRepo), andMergereturns before anyAcquire, so the request can never enqueue PR 0.
go build ./pkg/mergelane/... ./pkg/github/... is clean on this host. operator-reference.md paragraph matches the behaviour. Looks correct to me.
β hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88
Summary
A serialized-lane front PR that is closed, merged outside the lane, made a draft or retargeted now leaves the front on the next automerge sweep pass. The next waiter is promoted. Until now such a front held every waiter until its front timeout: 60 minutes by default, and up to 4x that when its checks were last seen pending.
The bug
Seen live on hive-wild-mole, build ba77df2:
sweep): it entered at 14:22:12Z, with a deadline of 15:22:12Z./api/repos/merge-lanestill showedfront.pr=25, and feat: reviewer runs goodnight docs syncΒ #27 and fix: scanner must fix issues, not just report themΒ #28 were waiting behind it.pkg/mergelanehas no changes between ba77df2 and v6 6f6a5ca, so v6 has the same bug.Why, read at 6f6a5ca:
Lane.Merge(gate.go:39) callsAcquire, thenAdvance. The front-only hand-off from π bug(v6): serialized lane keeps a red front PR for the full 60-min timeout β sweep skips it as not-mergeable and never releases itΒ #11023 (advanceRejectedFront) also needs the path to name that PR.prShape(lane.go:513) knows the reasons, such aspull request is closed, but runs only inside a round for that PR.pruneLocked(lane.go:426) drops stale waiters only, never the front. That leavesexpireLocked(lane.go:384) and its deadline, withMaxFrontTimeoutExtensions = 3.Release(lane.go:280) is only called for the native-merge-queue refusal.The fix
LaneMergeRequest.ReleaseGoneFronts: a repo-wide request with no number and no branch.sweepPasssends one request for each hive-serialized repo the pass covers. Tick, floor, startup and event passes all qualify; an event pass sends it only if it targets that repo.Lane.releaseGoneFront). For each branch with a front, it:Advance, so it never races an in-flight round, including one that is merging;prShapegives a reason, ends the front withfront check: <reason>, for examplefront check: pull request is closedorfront check: pull request is already merged, then promotes the oldest waiter.ChecksPendingstay as they were. An unreadable PR keeps its front: the timeout still bounds it, and the error is logged by the sweep.Why read state rather than add a staleness clock for the front
The brief offered a second option: let a front go stale on the waiters'
StaleAfterclock once its path stops callingAdvance. A front that is legitimately waiting is not always called in on every pass: the path's own gates can reject it before the lane. A staleness clock would therefore need its own proof that it can't drop a front waiting on CI. ReadingprShapedecides only on facts that make a merge impossible, and it reuses the reasons a round already applies.Cost
On a hive-serialized repo, each pass costs one
PullRequests.Getper lane that has a front, which is at most one per target branch. Repos ondirectand lanes without a front cost nothing.Tests
pkg/mergelane/gone_front_test.go:TestGoneFrontLeavesOnTheSweepFrontCheckcovers closed, merged elsewhere, draft and retargeted fronts. Each one:front check: β¦exit reason (in the last-exit record and the front-exit event);TestOpenFrontStaysOnTheSweepFrontCheck: an open front waiting on in-progress checks is unchanged. The record is identical and no event is emitted.TestUnreadableFrontStaysOnTheSweepFrontCheck: a PR read error keeps the front and returns the error.TestGoneFrontCheckIsFencedOnTheFrontItRead: the front is released by its path during the read, and the check does not end the newly promoted front.TestGoneFrontCheckCoversEveryBranchOfTheRepo: every branch of the repo is checked, and other repos and empty lanes are not.pkg/github/automerge:TestLaneSweep_PassReleasesGoneFronts:directrepo never consults the lane.go test ./pkg/mergelane/... ./pkg/github/... ./cmd/hive/...passes;go vetis clean.prShapegate;Docs and changelog
src/docs/operator-reference.md: added a paragraph to the stuck-front section.changelog.d/fixed-merge-lane-gone-front.md.Related: #11023, #12955, #14934, #14936 (same family: a front held until the timeout for a reason the lane can see).
π€ Generated with Claude Code
https://claude.ai/code/session_01BQYnPGFJx81c8JXR3o1sg1