Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/fixed-base-red-memory-and-new-head.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
- A PR whose only red checks are also red on its base branch no longer flips between 050/060 and 070/072 while the base stays red. The base's red checks are remembered across passes and drop only when the base tip's run of the check completes green, so a tip whose run is still in progress, or a failed base lookup (now logged at Warn), no longer sends the PR to its author, queues a relay fix and then withdraws the nudge. A new head the CI enrichment has not read yet is classified from that memory when every check red on it is the base's and it touches no CI-owning path.
1 change: 1 addition & 0 deletions src/cmd/hive/main_eval_cycle.go
Original file line number Diff line number Diff line change
Expand Up @@ -742,6 +742,7 @@ func runEvalCycle(
}
}
publishStageSharedCIIncidents(actionable)
publishStageBaseRedMemory(ghClient.BaseRedSnapshot())

// Record enumerated issues into the lifecycle timeline so the dashboard's
// lifecycle view has real data. Cheap and fully guarded: a nil dashboard or
Expand Down
1 change: 1 addition & 0 deletions src/cmd/hive/merge_eligibility.go
Original file line number Diff line number Diff line change
Expand Up @@ -165,6 +165,7 @@ func stageLabelPolicyFunc(cfg *config.Config) func() automerge.StageLabelPolicy
HiveAuthor: hiveIdentity(cfg).Matches,
SharedCIIncident: stageSharedCIIncident,
BaseBranchRed: stageBaseBranchRed,
BaseRedRemembered: stageBaseRedRemembered,
ForkGuard: stageForkGuard,
DCORequired: stage.DCO.RequiredFor,
SquashOnly: cfg.AutoMerge.SquashOnlyFor,
Expand Down
1 change: 1 addition & 0 deletions src/cmd/hive/repos_rescan.go
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,7 @@ func rescanRepos(

publishEnumeratedSnapshot(lastActionable, actionable, logger)
publishStageSharedCIIncidents(actionable)
publishStageBaseRedMemory(ghClient.BaseRedSnapshot())
if data, marshalErr := json.Marshal(actionable); marshalErr == nil {
atomicWrite(lastActionablePath, data)
}
Expand Down
27 changes: 27 additions & 0 deletions src/cmd/hive/stage_shared_ci.go
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,33 @@ func publishStageSharedCIIncidents(actionable *github.ActionableResult) {
stageSharedCIIndex.Store(&idx)
}

// stageBaseRedMemory holds the github client's remembered base-branch red
// checks (github.Client.BaseRedSnapshot), keyed by
// github.BaseRedRememberedKey, as of the last CI-enriched enumeration. The
// stage reconciler reads it through StageLabelPolicy.BaseRedRemembered so a
// PR head the index has not read yet, red only on its base's checks, is not
// returned to its author before the next enumeration
// (projectbluefin/chairlift#811).
var stageBaseRedMemory atomic.Pointer[map[string][]string]

// publishStageBaseRedMemory replaces the base-red memory snapshot.
func publishStageBaseRedMemory(snap map[string][]string) {
if snap == nil {
snap = map[string][]string{}
}
stageBaseRedMemory.Store(&snap)
}

// stageBaseRedRemembered answers StageLabelPolicy.BaseRedRemembered: the
// checks remembered red on repo's ("owner/name") base branch baseRef.
func stageBaseRedRemembered(repo, baseRef string) []string {
m := stageBaseRedMemory.Load()
if m == nil {
return nil
}
return append([]string(nil), (*m)[github.BaseRedRememberedKey(repo, baseRef)]...)
}

// stageSharedCIIncident answers StageLabelPolicy.SharedCIIncident. repo is
// "owner/name"; the snapshot may key a PR by either form. A head pushed
// since the snapshot is not covered: the deferral was decided for the old
Expand Down
18 changes: 18 additions & 0 deletions src/cmd/hive/stage_shared_ci_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -68,3 +68,21 @@ func TestStageBaseBranchRedIndex(t *testing.T) {
t.Fatalf("base-red without an incident reported incident %d", got)
}
}

func TestStageBaseRedRemembered(t *testing.T) {
t.Cleanup(func() { stageBaseRedMemory.Store(nil) })
if got := stageBaseRedRemembered("acme/widget", "main"); got != nil {
t.Fatalf("nothing published: %v", got)
}
publishStageBaseRedMemory(map[string][]string{github.BaseRedRememberedKey("acme/widget", "main"): {"Tests Passed", "Unit Tests"}})
if got := stageBaseRedRemembered("Acme/Widget", "main"); len(got) != 2 || got[0] != "Tests Passed" || got[1] != "Unit Tests" {
t.Fatalf("remembered = %v", got)
}
if got := stageBaseRedRemembered("acme/widget", "release"); len(got) != 0 {
t.Fatalf("another base: %v", got)
}
publishStageBaseRedMemory(nil)
if got := stageBaseRedRemembered("acme/widget", "main"); len(got) != 0 {
t.Fatalf("after an empty publish: %v", got)
}
}
2 changes: 1 addition & 1 deletion src/docs/labels-and-control-signals/stage-labels.md
Original file line number Diff line number Diff line change
Expand Up @@ -412,7 +412,7 @@ used to admit the PR:
| `contributor-lane:soaking(...)` β€” and no red check on the head | `095-pre-merge-soak` |
| `head-changed-since-eval`, `merge-failed`, `merge-queue-required` (the base branch's ruleset only accepts merges through GitHub's merge queue and the transport is a static token that cannot enqueue, so the direct REST merge is refused; with App auth on a repo opted in through `auto_merge.native_merge_queue` these bases are enqueued instead, see [native-merge-queue.md](../native-merge-queue.md); logged as a skip rather than a merge error, [#14018](https://github.com/hivecommons/hive/issues/14018); the self-authored sweep then skips the direct merge on that base for 30 minutes and logs the skip once per PR head, [#14068](https://github.com/hivecommons/hive/issues/14068); the trusted-author, queued and contributor-lane sweeps skip it the same way, sharing one memory of such bases; while a base is remembered the dashboard shows its otherwise-green PRs as a person's to merge, not Merge-eligible; with `auto_merge.require_current_base` on, a head behind such a base is not updated for 24 hours after the last refusal and holds here instead, so the base moving does not reset its review), `required-checks-expected`, `behind`, `updated-branch`, `behind-update-budget-exhausted` (a behind self-authored PR the sweep did not update only because this tick's branch-update budget was spent; retried next tick, waits on the hive, [#12607](https://github.com/hivecommons/hive/issues/12607)), `release-hold*`, `native-merge-queued: queued (position N, STATE)` (the PR was enqueued into GitHub's native merge queue) and the failures `native-merge-queue-read`, `native-merge-queue-enqueue`, `native-merge-queue-method` β€” and no red check on the head | `100-merge-queue` |
| CI red (required or optional check) while the PR is deferred to an open `[shared-ci]` incident (`<!-- hive-shared-ci-<n> -->`, the governor's `deferred_incident`) for the same head | `050-review-pending` / `060-reviewed`, reason `shared-ci-incident:<n>`: the base or a shared check is what is red, so the PR is never sent to 070/071/072, never nudged and never stalled. A 072 nudge already posted for the head (the nudge goes out on entering 072, and the incident marker can land minutes later) is edited in place to say it was withdrawn, and the head's nudge count is reset. Once the incident closes the PR stays deferred until the shared-CI rerun sweep has re-run its failed jobs (at most 30 minutes); a head still red after that is handed back as usual, with a fresh nudge. Conflicts and requested changes stay author-facing ([#12606](https://github.com/hivecommons/hive/issues/12606)) |
| CI red (required or optional) while **every** failing check on the head is also red on the base branch's tip β€” the governor's `base_red_checks` / `base_red`, set during CI enrichment when a same-named latest check run on the base ref (meta checks excluded) concluded failure / timed_out / cancelled / action_required / startup_failure / stale **and** the PR's changed files include none of `.github/workflows/*`, `Dockerfile` / `Dockerfile.*` / `*.Dockerfile`, `go.mod` / `go.sum` (a PR touching those plausibly owns the check and stays author-facing; a lookup error or a diff beyond the file-list page budget also stays author-facing) | `050-review-pending` / `060-reviewed`, reason `base-branch-red:<check>,<check>` ("waiting for the base branch's CI to recover (…) β€” not this PR's fault"): never 070/071/072, never nudged (a 072 nudge already posted for the head is edited in place to say it was withdrawn), never stalled, still not merged (the lane's skip reason is the same token). The fix loop counts no attempt, kicks no fixer, and posts one `<!-- hive-base-branch-red -->` hold comment per head; an escalation the hive itself raised (ledger `label_applied`) over such a head has its `needs-human` removed with the hold wording. The next evaluation after the base goes green re-evaluates the head: a still-red PR is handed back as usual. Conflicts and requested changes stay author-facing (seen on [#13270](https://github.com/hivecommons/hive/pull/13270) and [#13258](https://github.com/hivecommons/hive/pull/13258), 2026-10-20) |
| CI red (required or optional) while **every** failing check on the head is also red on the base branch's tip β€” the governor's `base_red_checks` / `base_red`, set during CI enrichment when a same-named latest check run on the base ref (meta checks excluded) concluded failure / timed_out / cancelled / action_required / startup_failure / stale **and** the PR's changed files include none of `.github/workflows/*`, `Dockerfile` / `Dockerfile.*` / `*.Dockerfile`, `go.mod` / `go.sum` (a PR touching those plausibly owns the check and stays author-facing; a diff beyond the file-list page budget also stays author-facing). The governor remembers each base's red checks across passes: a check stays red on the base until a read sees the tip's run of it complete green, so a tip whose run is still queued or in progress, or a base listing that fails (logged at Warn), keeps the base's last red set; a check not seen red for 2 h is dropped. A new head the CI enrichment has not read yet is classified from that memory at the stage pass: every check red on the head (its check digest) is remembered red on the base and the head touches no CI-owning path (an unreadable file list stays author-facing) | `050-review-pending` / `060-reviewed`, reason `base-branch-red:<check>,<check>` ("waiting for the base branch's CI to recover (…) β€” not this PR's fault"): never 070/071/072, never nudged (a 072 nudge already posted for the head is edited in place to say it was withdrawn), never stalled, still not merged (the lane's skip reason is the same token). The fix loop counts no attempt, kicks no fixer, and posts one `<!-- hive-base-branch-red -->` hold comment per head; an escalation the hive itself raised (ledger `label_applied`) over such a head has its `needs-human` removed with the hold wording. The next evaluation after the base goes green re-evaluates the head: a still-red PR is handed back as usual. Conflicts and requested changes stay author-facing (seen on [#13270](https://github.com/hivecommons/hive/pull/13270) and [#13258](https://github.com/hivecommons/hive/pull/13258), 2026-10-20) |
| CI red (required or optional) on a **fork** PR while **every** failing check's annotation says the check refuses forks (case-insensitive `fork pull request(s) cannot`, `maintainer must push this branch`, or `cannot publish` together with `fork` β€” a fork-publication / fork-secrets guard) | `085-needs-decision`, reason `fork-guard:<check>,<check>`: never 070/071/072, never nudged, never adopted (074) β€” neither the author nor the hive can turn the check green by pushing to the fork. The entry comment names the check and says a maintainer must push the branch to the repository (or run or approve it for head `<sha>` as the repository requires). A new head on the same fork PR (a merge of the base, or the author's push) keeps the guard while the lane's red check is one of the recorded `fork-guard:` checks, so the PR does not drop to 072 and nobody is nudged before the annotation for that head is read. A fork PR head that nothing has classified yet (its first red head, or a PR that came to 072 for another reason first) is classified from its own check annotations with the same rule, once per head, before it can be nudged: a head red only on fork-guarded checks goes straight to 085 and the guard is recorded for later heads (projectbluefin/utah-packages#972, where the author was nudged and sent relay rework for a fork-gated Canary five minutes before the annotation was read). The hive also does not update the branch of a fork head that waits on a fork-guarded check: the run a maintainer dispatches proves one exact head, and a merge of the base would discard it. Once the check is green the guard drops and updates resume before merge. While the PR stays parked across new heads, the entry comment is rewritten in place for each one, so it always names the current head, and it says that a run which passed on an earlier head does not cover this one even when the patch is the same, because a rebase or a merge of the base changes the code the check runs (projectbluefin/utah-packages#917). A same-repo PR with the same message, or a fork PR with any other red check, is handed back as usual ([#14273](https://github.com/hivecommons/hive/issues/14273), projectbluefin/utah-packages#862 and #869) |
| any **non-required** check on the current head is red, whatever the lane's reason (the lane gates on required checks only, so it still soaks and merges; a reader of the rollup sees βœ—). Parking modifiers (`hold`, `needs-human`, admission reasons) still win | `070-fixing` / `071-needs-author` / `072-returned-to-contributor` (by author kind), reason `optional-check-failing:<check>` (the nudge lists every red check); never 080 or 100. A red *required* check is the lane's own `required-check-failing:*` β†’ 070/071/072 ([#12538](https://github.com/hivecommons/hive/issues/12538), seen on [#12457](https://github.com/hivecommons/hive/pull/12457)) |
| `label:hold`, `label:blocked`, `held`, `sentinel`, `contributor-lane:sentinel-blocked` | **gate**, not a stage ([#17327](https://github.com/hivecommons/hive/issues/17327)): routed by the head's evidence (below) exactly as a trust cause β€” CI / 070 while checks run or fail, 071 on a reviewer question, `090-ready-for-human-merge` when the head is otherwise clean β€” with the gate reason kept on the route so the popover and the entry comment say "Held: …". The lanes refuse the merge on the label, never on the stage. `label:blocked` is a PR wearing `blocked`: its gate reason reads `blocked on owner/repo#N` when the blocked sweep has read the pull request it waits on. The lane stops at the gate before it reads mergeability, so the route takes the head's `mergeable_state` from the PR detail the enumeration fetched for the held PR this cycle (no extra request), for exactly the current head: a `dirty` head goes back to its author like any conflict (070 / 071 / 072, reason `conflicting`, the 072 nudge and fix queue included) and the gate stays a pill and a hard merge stop. `unknown`, or a state read for an older head, is not a conflict; a pass with no state at all keeps a conflict the stage already returned on the same head |
Expand Down
2 changes: 1 addition & 1 deletion src/docs/scale-envelope.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ ones to:
2. **Truncation must preserve the prioritization signal, not cut an arbitrary slice.**
This holds today and is easy to assume backwards:
- Issues are sorted by actionable rank tier, then **oldest first** within each tier before any cap is applied
(`RankActionableIssues`, `src/pkg/github/client.go:5593-5603`, descending `AgeMinutes` within each tier).
(`RankActionableIssues`, `src/pkg/github/client.go:5600-5610`, descending `AgeMinutes` within each tier).
- PRs are sorted by **review class then oldest-first within class**
(`SortPullRequestsForReview`, `src/pkg/github/review_priority.go:334`;
fixes β†’ refactors/docs β†’ tests).
Expand Down
Loading
Loading