Skip to content

πŸ› fix(stage-labels): stop PR stages flipping between two sweeps' passes - #16819

Merged
hivecommons-hive[bot] merged 1 commit into
hivecommons:v6from
Danathar:fix/stage-label-flap
Nov 5, 2026
Merged

hivecommons-hive[bot] merged 1 commit into
hivecommons:v6from
Danathar:fix/stage-label-flap

Conversation

@Danathar

@Danathar Danathar commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

Executive summary

After stage labels started being written on contributor PRs (11-05 21:09Z), three projectbluefin/utah-packages fork PRs flipped stage on every pass for about 20 minutes, with head, checks and labels unchanged:

The hive log shows the cause. Two sweeps, the engine's own ticker and the main loop's queued sweep, feed one stage tracker and reconcile alternate passes, and each pass sees a different set of observations. This PR removes both ways that produced a different stage on alternate passes.

What

Shape Pass A (ticker) Pass B (queued sweep) Before After
#869, #873 contributor lane: base-refresh:merge-guard fork-guard:Canary / contributor-lane:no-linked-issue 080 ↔ 085/090 merge-guard is no opinion: 085 / 090 stays
#878 self-authored lane (trusted-bot author): 070 contributor lane only: 072 070 ↔ 072 070 stays while the self lane is absent, up to 30 min

Log evidence (build f0c02da, stage labels reconciled):

Why

  • base-refresh:merge-guard is what a lane records when it skips a PR because a merge to the same destination is in flight ([selfreport] unclassified_error_repeated: automerge sweep skipped PRΒ #16520), and here that merge belongs to the other sweep. ClassifyStage read it like real base preparation, an owned stage, so the skipped pass overwrote the stage the other sweep had set.
  • Lane precedence (self-authored > trusted-author > contributor) is resolved per pass. The self-authored lane runs only in the engine's ticker, so on queued-sweep passes the contributor lane's opinion for the same head won.

How

  • prstage.StageReasonBaseRefreshMergeGuard is added to stageHandoffReasons, next to fetch-pr, so it reads as "the lane did not look". With no other opinion the pass writes nothing; base preparation proper (base-refresh:preparing, queued-behind) still owns its stage.
  • StageTracker.owners remembers, per PR, the lane that set the stage just written, the head it was for and when it last reported; unlike the observations, it survives drain.
  • stageOwnerHolds skips the pass for that PR when all of these hold:
    • the recorded owner outranks the lane deciding now;
    • the owner did not report this pass;
    • the head is the same, and the stored stage is still the owner's;
    • the new stage differs;
    • the owner last reported within 30 minutes (twice selfAuthoredSweepMaxInterval).
  • A new head, a terminal outcome, or the owner staying away past the grace lets the other lane's stage through.

Files

File Change
src/pkg/stageengine/prstage/reasons.go StageReasonBaseRefreshMergeGuard
src/pkg/stageengine/derive_pr_classify.go merge-guard is a no-opinion hand-off reason
src/pkg/github/automerge/automerge_stage_labels.go StageTracker.owners, stageOwnerHolds, owner noted after the write
src/pkg/github/automerge/automerge_stage_flap_test.go New regression tests
src/docs/labels-and-control-signals/stage-labels.md, changelog Documents both

Tests

Test Covers
TestMergeGuardPassKeepsTheStage #869/#873 shape: 7 alternating passes (evaluated / merge-guard), zero writes after the first
TestAbsentOwningLaneKeepsTheStage #878 shape: 8 alternating ticker / queued passes stay 070; past the grace β†’ 072; new head β†’ 072
TestStageOwnerHolds 11-row table: every condition of the hold
TestMergeGuardIsNoStageOpinion the emitted string matches the constant; preparing still owns a stage
Before the fix all three behaviour tests failed on unchanged v6 (merge-guard wrote a stage; 070 β†’ 072)
Mutants 8/8 caught
go test automerge, stageengine, stagelabels, fleetreport pass
Writeback ratchet, gofmt, vet, check-docs-citations.py pass (2006 references)

Notes

πŸ€– Generated with Claude Code

https://claude.ai/code/session_01BQYnPGFJx81c8JXR3o1sg1

projectbluefin/utah-packages, 11-05 21:10-21:29Z: hivecommons#878 070<->072, hivecommons#873
080<->090 and hivecommons#869 080<->085 flipped every pass with head, checks and labels
unchanged. The engine's ticker and the queued sweep reconcile alternate
passes from one stage tracker:

- base-refresh:merge-guard (lane skipped the PR: a merge to its destination
  was in flight) classified as an owned base-refresh stage and overwrote the
  other sweep's stage. It is now a no-opinion hand-off reason.
- the self-authored lane runs only in the ticker, so the queued sweep's
  contributor lane overwrote its stage for the same head. The tracker now
  remembers the lane and head that set the stage; a lower lane's different
  stage waits while that lane is absent, up to 30 minutes, never across a
  new head or a terminal outcome.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BQYnPGFJx81c8JXR3o1sg1
Signed-off-by: Danathar <Danathar@users.noreply.github.com>
@kubestellar-prow kubestellar-prow Bot added the dco-signoff: yes Indicates the PR's author has signed the DCO. label Nov 5, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign clubanderson for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubestellar-prow kubestellar-prow Bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Nov 5, 2026
@github-actions

github-actions Bot commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

Open PR overlap check

Base branch: v6

Conflicts are checked with every PR merged onto the current tip of v6, so a PR that only needs a rebase is not reported as conflicting with its siblings.

PR Title Relation Files
#16816 πŸ› fix(adoption): refuse a fixer push that undoes the adopted PR's change same files src/docs/labels-and-control-signals/stage-labels.md
#16688 🌱 Forward-merge v5 into v6 (2 commits) same files src/docs/labels-and-control-signals/stage-labels.md
src/pkg/github/automerge/automerge_stage_labels.go

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read this against the tree at merge-base plus the callers; it looks correct to me. No defects found.

What I checked:

  • Owner timestamp cannot go stale while the owning lane keeps reporting. My first worry was that noteOwner runs only after a write (automerge_stage_labels.go:1471-1473), so an unchanged 070 every pass might never refresh At and the hold would lapse after stageOwnerAbsenceGrace (30 min). It does not: nothing between the hold check (:1045) and pending = append(...) (:1437) skips an unchanged stage β€” the only continues are the reporter-trust release and supersession-close paths β€” so an owned observation reaches ReconcileBatch every pass and At is refreshed whether or not a label moved.
  • The hold cannot pin a stage. stageOwnerHolds (:299-315) returns false on a new head, a terminal verdict, the same stage, a stage another writer moved (current != owner.Stage), an owner that reported this pass (list is this pass's drain, :373), an owner that does not outrank, and past the grace. stageLanePrecedence (:244) has only self/trusted/contributor, so unlisted lanes compare 0 <= 0 and never hold β€” covered by the "lanes without precedence never hold" row.
  • base-refresh:merge-guard as a hand-off reason. basePreparationPrefix + "merge-guard" (automerge_base_preparation.go:21, emitted at automerge_contributor_lane.go:1610, automerge_sweep_queued.go:25, automerge_sweep_trusted.go:26,648) matches prstage.StageReasonBaseRefreshMergeGuard exactly, and slogfeed.go:254 already treats the same string as "not an opinion". Adding it to stageHandoffReasons (derive_pr_classify.go:260) is consistent with how the other no-look reasons are handled.
  • The TestStageOwnerHolds table (11 rows) plus TestMergeGuardPassKeepsTheStage / TestAbsentOwningLaneKeepsTheStage cover the flap shape from utah-packages#878 and the grace expiry.

Not approving (that's a human signature); just confirming the read.

β€” hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive
hivecommons-hive Bot merged commit 694675a into hivecommons:v6 Nov 5, 2026
71 of 74 checks passed
@hivecommons-hive

Copy link
Copy Markdown
Contributor

Hive status β€” updated in place; edits do not notify. Full history is in the Activity Monitor.

Evidence bundle `#16819ff9b76e` sha256:a01ed9ad60caaf064e2893695c741ac88217baac6602008b40803d0731e24d8d β€” download from the dashboard or `hivectl review evidence #16819`.

hivecommons-hive Bot pushed a commit that referenced this pull request Nov 7, 2026
…85 PR to 080 (#17012)

projectbluefin/utah-packages#917 (fork PR, approve verdict, fork-guarded red
required Canary, head 7e52182c0 unchanged) went 085 -> 080 -> 085 about 15
times on 2026-11-07. Each 080 came from a contributor-lane pass refused the
destination-merge lock ("destination merge already in progress", another
sweep was evaluating a PR into the same base). The lane records that as
base-refresh:merge-guard with the error, so the observation is Err, and
ClassifyStage's errored-lane branch ran before the hand-off check that
#16819 added: approve verdict + no CI reason = a weak 080. The 080 seated
the red PR at a merge train's front until the next evaluated pass.

The refusal is now no stage opinion whether or not the lane reports an
error. The reversal guard (#16881) and the dampener (#16980) on v6 tip do
not stop this shape: the replay test fails on tip at the first refusal.

Also corrects two stale derive_pr_classify.go citations in the stage-engine
docs.


Claude-Session: https://claude.ai/code/session_01BQYnPGFJx81c8JXR3o1sg1

Signed-off-by: Danathar <Danathar@users.noreply.github.com>
Co-authored-by: Danathar <Danathar@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
hivecommons-hive Bot pushed a commit that referenced this pull request Nov 20, 2026
…19381)

GitHub reports mergeable_state "unknown" while it recomputes a merge,
and the contributor lane then stops before admission, review or CI with
reason mergeable-unknown. The stage engine read that as an approved head
with nothing left for its author (a weak 080), and another lane's weak
opinion could stand in for the silent contributor lane, so an unchanged
head dropped for one pass: projectbluefin/utah-packages#1222 went
090 -> 080 -> 090 on a no-linked-issue refusal, and a conflicting PR at
072 could read 090.

mergeable-unknown is now "no opinion", like base-refresh:merge-guard
(#16819): ClassifyStage abstains, ResolveStage drops other lanes' weak
opinions on such a pass (an owned opinion still applies), and the merge
train keeps the head's human-merge verdict instead of clearing it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BQYnPGFJx81c8JXR3o1sg1
Signed-off-by: Danathar <6772335+Danathar@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dco-signoff: yes Indicates the PR's author has signed the DCO. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant