Skip to content

πŸ› fix(mergelane): release a closed or merged front on the next sweep - #16056

Merged
hivecommons-hive[bot] merged 3 commits into
hivecommons:v6from
Danathar:fix/merge-lane-closed-front
Oct 31, 2026
Merged

hivecommons-hive[bot] merged 3 commits into
hivecommons:v6from
Danathar:fix/merge-lane-closed-front

Conversation

@Danathar

Copy link
Copy Markdown
Contributor

Summary

A serialized-lane front PR that is closed, merged outside the lane, made a draft or retargeted now leaves the front on the next automerge sweep pass. The next waiter is promoted. Until now such a front held every waiter until its front timeout: 60 minutes by default, and up to 4x that when its checks were last seen pending.

The bug

Seen live on hive-wild-mole, build ba77df2:

pkg/mergelane has no changes between ba77df2 and v6 6f6a5ca, so v6 has the same bug.

Why, read at 6f6a5ca:

  • The lane only looks at its front when that PR's own path calls in. Lane.Merge (gate.go:39) calls Acquire, then Advance. The front-only hand-off from πŸ› bug(v6): serialized lane keeps a red front PR for the full 60-min timeout β€” sweep skips it as not-mergeable and never releases itΒ #11023 (advanceRejectedFront) also needs the path to name that PR.
  • prShape (lane.go:513) knows the reasons, such as pull request is closed, but runs only inside a round for that PR.
  • Every sweep lane lists open PRs only, so a closed or externally merged front never gets another round.
  • pruneLocked (lane.go:426) drops stale waiters only, never the front. That leaves expireLocked (lane.go:384) and its deadline, with MaxFrontTimeoutExtensions = 3. Release (lane.go:280) is only called for the native-merge-queue refusal.

The fix

  • New LaneMergeRequest.ReleaseGoneFronts: a repo-wide request with no number and no branch.
  • The sweep sends it once per pass: sweepPass sends one request for each hive-serialized repo the pass covers. Tick, floor, startup and event passes all qualify; an event pass sends it only if it targets that repo.
  • The lane checks each front of the repo (Lane.releaseGoneFront). For each branch with a front, it:
    • takes the same per-lane lock as Advance, so it never races an in-flight round, including one that is merging;
    • reads the front PR once;
    • if prShape gives a reason, ends the front with front check: <reason>, for example front check: pull request is closed or front check: pull request is already merged, then promotes the oldest waiter.
  • The write is fenced on the front's PR and epoch as read. A front that changed while the PR was being read is left alone.
  • Never merges, enqueues, updates a branch or comments.
  • An open front is not touched. That includes one waiting on queued or running checks, whose deadline, extensions and ChecksPending stay as they were. An unreadable PR keeps its front: the timeout still bounds it, and the error is logged by the sweep.

Why read state rather than add a staleness clock for the front

The brief offered a second option: let a front go stale on the waiters' StaleAfter clock once its path stops calling Advance. A front that is legitimately waiting is not always called in on every pass: the path's own gates can reject it before the lane. A staleness clock would therefore need its own proof that it can't drop a front waiting on CI. Reading prShape decides only on facts that make a merge impossible, and it reuses the reasons a round already applies.

Cost

On a hive-serialized repo, each pass costs one PullRequests.Get per lane that has a front, which is at most one per target branch. Repos on direct and lanes without a front cost nothing.

Tests

  • pkg/mergelane/gone_front_test.go:
    • TestGoneFrontLeavesOnTheSweepFrontCheck covers closed, merged elsewhere, draft and retargeted fronts. Each one:
    • TestOpenFrontStaysOnTheSweepFrontCheck: an open front waiting on in-progress checks is unchanged. The record is identical and no event is emitted.
    • TestUnreadableFrontStaysOnTheSweepFrontCheck: a PR read error keeps the front and returns the error.
    • TestGoneFrontCheckIsFencedOnTheFrontItRead: the front is released by its path during the read, and the check does not end the newly promoted front.
    • TestGoneFrontCheckCoversEveryBranchOfTheRepo: every branch of the repo is checked, and other repos and empty lanes are not.
  • pkg/github/automerge: TestLaneSweep_PassReleasesGoneFronts:
    • a pass sends one repo-wide front check (no number, branch or authorization);
    • an event pass targeting the repo sends it too, and a lane error is only logged;
    • a pass targeting another repo sends nothing;
    • a direct repo never consults the lane.
  • Results:
    • go test ./pkg/mergelane/... ./pkg/github/... ./cmd/hive/... passes; go vet is clean.
    • Mutation run: 10/10 mutants killed. The mutants removed:
      • the sweep call;
      • the gate dispatch;
      • the fence;
      • the prShape gate;
      • the promotion;
      • the target and strategy filters;
      • the read-error path;
      • the per-branch loop;
      • the left outcome.

Docs and changelog

  • src/docs/operator-reference.md: added a paragraph to the stuck-front section.
  • changelog.d/fixed-merge-lane-gone-front.md.

Related: #11023, #12955, #14934, #14936 (same family: a front held until the timeout for a reason the lane can see).

πŸ€– Generated with Claude Code

https://claude.ai/code/session_01BQYnPGFJx81c8JXR3o1sg1

A serialized-lane front PR closed, merged outside the lane, made a draft
or retargeted got no further round: the lane only reads its front when
that PR's own merge path calls in, and every sweep lane lists open PRs
only. It held every waiter until the front timeout (up to 4x that when
its checks were last seen pending).

Each sweep pass now sends the lane one repo-wide ReleaseGoneFronts
request per hive-serialized repo it covers. Under the per-lane lock the
lane reads each front PR once and, when prShape gives a reason, ends the
front with "front check: <reason>" (fenced on the epoch it read) and
promotes the oldest waiter. Open fronts, including ones waiting on CI,
and unreadable PRs are left alone; nothing is written to GitHub.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BQYnPGFJx81c8JXR3o1sg1
Signed-off-by: Danathar <Danathar@users.noreply.github.com>
@kubestellar-prow kubestellar-prow Bot added the dco-signoff: yes Indicates the PR's author has signed the DCO. label Oct 31, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign kellyaa for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubestellar-prow kubestellar-prow Bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Oct 31, 2026

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read the change against the v6 tree at be3d228 (callers included). Looks correct to me; nothing blocking.

What I verified

  • src/pkg/mergelane/lane.go:313-347 releaseGoneFront takes the same per-lane mutex as Advance (lockLane, lane.go:402), so it cannot interleave with a running round and will never observe a front the lane itself is mid-merging. The epoch re-check inside Store.update (lane.go:333) correctly drops the exit if the front changed between the snapshot and the write.
  • It reuses prShape (lane.go:558), the same predicate the round already applies at lane.go:677 and :754, so "gone" means exactly what the lane already means by it β€” no new eligibility semantics.
  • An open front, or a front whose PullRequest read fails, is left alone (lane.go:321-328) β€” the fix cannot evict a live front on a transient GitHub error.
  • releaseGoneFronts (gate.go:100-128) iterates Store.Lanes(repo) which is case-insensitive on repo (store.go:245), so the owner/name form the sweep passes matches stored records.
  • automerge_sweep.go:1856 runs the check once per pass, gated by targets.selectsRepo and serializedLane, so direct-strategy repos and untargeted repos never reach the lane β€” TestLaneSweep_PassReleasesGoneFronts pins both.
  • Builds clean (go build ./pkg/mergelane/ ./pkg/github/...). Changelog fragment name follows the fixed- convention; operator-reference.md updated.

CI note for the maintainer: the red native-queue-policy check is not this PR. Its job checks out refs/heads/v6 and fails with Could not find '.github/scripts/native-queue-policy.test.cjs' β€” that file does not exist on v6 either (git ls-tree origin/v6 .github/scripts/), and this PR touches nothing under .github/. It will be red on every v6 PR until the base branch is fixed.

β€” hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive

Copy link
Copy Markdown
Contributor

@Danathar this PR is back in your court (stage/072-returned-to-contributor). The hive will not merge it until the head changes.

Failing checks:

  • check-failure

Reply here or push a new head to this branch (same sign-off); CI and the review swarm re-run automatically and the PR returns to the normal path.

Queued in your relay fix queue (1 waiting): fixes are handed out before new work as soon as you have capacity.

@hivecommons-hive

Copy link
Copy Markdown
Contributor

The red native-queue-policy check here is not this PR's fault. The workflow (on v5) checks out refs/heads/v6 by design and v6 has no .github/scripts/native-queue-policy*.cjs, so it fails on every v6-based PR before evaluating anything:

Could not find '.github/scripts/native-queue-policy.test.cjs'

Tracked as #16060; fix in #16061 (ports the two scripts to v6). Once that lands, re-running the check here should go green β€” nothing to change in this diff. All test shards and coverage are already green.

@github-actions

github-actions Bot commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Open PR overlap check

Base branch: v6

Conflicts are checked with every PR merged onto the current tip of v6, so a PR that only needs a rebase is not reported as conflicting with its siblings.

PR Title Relation Files
#16059 πŸ› fix(mergelane): keep a sweep PR whose writes are contained off the front same files src/docs/operator-reference.md
src/pkg/mergelane/gate.go
#16098 πŸ› automerge: name a standing change request in the review-required ask; no merge call on a conflicting head same files src/pkg/github/automerge/automerge_sweep.go

@hivecommons-hive

hivecommons-hive Bot commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Hive status β€” updated in place; edits do not notify. Full history is in the Activity Monitor.

hive/awaiting-ci: all checks are green at be3d228 now that #16061 landed on v6. The hive merge relay refuses this head because its earlier attempts were exhausted while native-queue-policy was red (bootstrap deadlock, not this PR's fault). @Danathar a trivial push (or a maintainer merge) unsticks it; the sweep merges it on the next green head.


🐝 Hive Agent: scanner | Instance: hosted-available-oke-11-placeholder-r05x | SHA: unknown

β€” hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88

Evidence bundle `#16056be3d0cb` sha256:6421e1e744d1bb84c5c439462c91026ca29277f5b2b39b13048f68d7570da814 β€” download from the dashboard or `hivectl review evidence #16056`.

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read at be3d0cb against v6, including the callers. The diagnosis holds: Acquire/Advance only run when a PR's own path calls in, every sweep lane lists open PRs, and pruneLocked (lane.go:471) touches waiters only, so a closed/merged front could only leave via expireLocked.

The fix is sound as written:

  • releaseGoneFront (lane.go:315) takes the same per-lane mutex Advance takes (lane.go:359 β†’ lockLane), re-reads the record inside Store.update and fences on Front.PR and Front.Epoch, so a round that replaced the front between the GitHub read and the write is a no-op rather than a wrong exit. TestGoneFrontCheckIsFencedOnTheFrontItRead pins that.
  • It reuses prShape (lane.go:558) rather than a second classification, so the reasons are identical to what a live round would have produced, and exitLocked/promoteLocked are the existing exit path, so LastExit and the ActionFrontExit/ActionFrontEnter events are emitted the same way.
  • A read error leaves the front alone and surfaces as a warn in the sweep (automerge_sweep.go:2375), never as a release β€” correct fail-safe direction.
  • Cost is one PullRequest read per front per pass, only for hive-serialized repos the pass targets (selectsRepo), and Merge returns before any Acquire, so the request can never enqueue PR 0.

go build ./pkg/mergelane/... ./pkg/github/... is clean on this host. operator-reference.md paragraph matches the behaviour. Looks correct to me.

β€” hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive
hivecommons-hive Bot merged commit eba4948 into hivecommons:v6 Oct 31, 2026
61 of 62 checks passed
@hivecommons-hive

Copy link
Copy Markdown
Contributor

merged by Hive trusted-author auto-merge: author @Danathar holds role owner and GitHub permission maintain on repo; CI green at be3d0cb.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dco-signoff: yes Indicates the PR's author has signed the DCO. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant