Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/fixed-front-only-base-refresh.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
- Serialize automatic final base refreshes per repository/base across merge lanes, preserve pending preparation outside ready train seats, and allow already-current, CI-green, reviewed successors to merge without waiting for a preparing front.
1 change: 1 addition & 0 deletions src/cmd/hive/github_client_config.go
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,7 @@ func (b *boot) applyGitHubClientConfigHooks(client *github.Client) {
// pause taken in the dashboard narrows the very next enumeration and
// automerge sweep without a restart.
client.SetRepoPausedFunc(b.cfg.IsRepoPaused)
client.SetScheduledBaseRefresh(func(repo string) bool { return b.cfg.AutoMerge.RequireCurrentBaseFor(repo) })
// Per-repo custom agents (#6204). A live predicate over the shared
// config, so the hive-open-pr / hive-merge / hive-open-issue relays
// refuse an out-of-scope request without a restart.
Expand Down
2 changes: 1 addition & 1 deletion src/cmd/hive/prfollowupwire.go
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
package main

import (
Expand Down Expand Up @@ -165,7 +165,7 @@
Logger: logger,
}
opts.BaseSyncEnabled = func(repo string) bool {
return cfg != nil && cfg.Review.ContributorPRs.BaseSync
return cfg != nil && cfg.Review.ContributorPRs.BaseSync && !cfg.AutoMerge.RequireCurrentBaseFor(repo)
}
if client != nil {
opts.BranchUpdater = client
Expand Down
64 changes: 62 additions & 2 deletions src/docs/hive-merge.md
Original file line number Diff line number Diff line change
Expand Up @@ -361,6 +361,65 @@ gate, saying it will wait for a human. The rule exists because a ⚠️-titled
PR was auto-merged by the trusted-author lane while its owner was still
deciding whether to ship it.

### Final base preparation: one refresh owner, no ready-work blockade

For repositories governed by `auto_merge.require_current_base` (including
the inherited default), automatic base refreshes share **one preparation
owner per canonical repository and destination branch**, across App,
trusted-author and contributor lanes. Repository names are case-insensitive;
branch names remain case-sensitive. Unrelated destinations progress independently.

A behind head enters final preparation only after its existing CI, review
and lane admission gates pass, **without first requiring an up-to-date base**.
The owner refreshes with an expected-head SHA and then waits for CI and review
on the new head. Pending runner jobs (including queued sign-off checks) are
pending, not failures. Preparation is outside ready merge-train membership:
`base-refresh:preparing` and `base-refresh:queued-behind#N` remain at CI / Review,
never ready-to-merge. Waiting heads may continue development and review.
The adopted stale-red recovery path can also admit its existing, proven
base-fixes-failure repair; that admission never authorizes a merge.
Explicit trusted-human merge-forward repair shares the same owner and
write-ahead identity. Its conflict-repair allowance lasts only through the
existing mutation-reconciliation TTL; a failed/conflicting result returns to
the normal repair path, never a ready seat.

**Ownership serializes refreshes, not ready merges.** A successor that is
already current-base, CI-green and reviewed passes every normal merge gate
without waiting for a preparing owner. A behind successor cannot skip testing.
Train sibling refreshes and client-side follow-up/relay/supersession updates
defer to preparation on these repositories; opted-out repositories retain
their previous behavior.

Ownership and first-admission FIFO tickets live in `/data/base-preparation/`,
with per-destination file locking and atomic durable snapshots. An update
identity (PR/head/base SHA) is recorded before the request, across lanes,
concurrent sweeps and restarts. An unchanged unresolved pair waits; after the
existing mutation-reconciliation TTL it can retry only with an authoritative,
still-identical head and the API's expected-head guard. Accepted updates must
complete their normal current-head gates; no approval is transferred.
Release occurs on merge, closed/missing/retargeted/draft/conflicting/held heads,
failed checks, rejected review, or lost admission. Expected pending CI/review
keeps ownership, bounded by the existing review-pending dwell (its default when
stall detection is disabled), so a genuinely hung owner yields to successors.
Waiting tickets are removed when their head changes or stops qualifying.
Admission provenance prevents a different lane's non-admission from revoking
the owning lane's reservation or FIFO ticket. An old waiting ticket reserves
order only: its head must qualify afresh in its lane before becoming owner.

Audit logs use `automerge base preparation selected`, `refresh claimed`,
`waiting` and `released`, with repository, base, PR, head/base SHA where relevant,
and release/skip reason. `automerge update-branch` and the sweep's
`updated_branches` count actual successful refresh effects; queued/preparing
reasons do not consume the branch-update budget. Ready-only train seating and
reviewer dispatch scheduling are separate policies.

Human pushes, direct merges and external automation can still move a base.
Such a move is coalesced until the preparation owner qualifies again; it must
then refresh and re-pass current-head gates if still behind. This scheduler
cannot stop external writers and does not relax strict current-base checks.
Deploying the changed hive binary activates the policy; no agent, memory,
review, CI, hold or authorization setting needs changing.

### Merge train: ready PRs lead

Every lane groups its candidates on one base whose changed files overlap
Expand All @@ -370,8 +429,9 @@ reason `merge-train:queued-behind#<front>:<pos>/<size>:…` and, the moment
the front merges, are update-branched (`behind`) or handed to the rebase
resolver (`dirty`) so the next front is evaluated on a current base. Hive's
own repo also runs the current-base gate (`require_current_base`), which
holds a behind head until the update lands; the two cooperate — the train
refreshes, the gate waits for the refresh.
holds a behind head until the update lands. On current-base-gated repositories
the separate preparation owner above refreshes the next admitted head, rather
than refreshing every waiting sibling when a front merges.

The train's order is **readiness first**. Each pass the lane sorts the
members by what it last saw of their heads: a head at the merge gate —
Expand Down
15 changes: 15 additions & 0 deletions src/pkg/dashboard/stage_tracker_base_preparation_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
package dashboard

import "testing"

func TestBasePreparationCaptionNeverAdvertisesReady(t *testing.T) {
for reason, want := range map[string]string{
"base-refresh:queued-behind#42": "queued for base refresh behind PR #42",
"base-refresh:queued-behind#0": "waiting for admission to final base preparation",
"base-refresh:preparing": "preparing the current base; current-head CI and review must complete",
} {
if got := humanizeStageReason(reason); got != want {
t.Errorf("%s caption=%q, want %q", reason, got, want)
}
}
}
9 changes: 9 additions & 0 deletions src/pkg/dashboard/stage_tracker_caption.go
Original file line number Diff line number Diff line change
Expand Up @@ -314,6 +314,15 @@ func humanizeStageReason(reason string) string {
return clause + " — approved by @" + approver
}
lower := strings.ToLower(r)
if front, ok := strings.CutPrefix(r, "base-refresh:queued-behind#"); ok {
if front == "0" {
return "waiting for admission to final base preparation"
}
return "queued for base refresh behind PR #" + front
}
if r == "base-refresh:preparing" {
return "preparing the current base; current-head CI and review must complete"
}
switch lower {
case "actionable", "live agent claim", "live issue claim", "open pr covers it":
return ""
Expand Down
19 changes: 18 additions & 1 deletion src/pkg/github/automerge/automerge_adoption_recovery.go
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,9 @@ type adoptionRecovery struct {
// Attempted means an update-branch or re-run was issued: the escalation
// waits for CI on the result.
Attempted bool
// Deferred means preparation ownership withheld the request, so no
// recovery attempt or lifetime budget was consumed.
Deferred bool
// Reason is the structured outcome: update-branch, rerun-failed-jobs or
// skipped:<why>.
Reason string
Expand Down Expand Up @@ -213,14 +216,25 @@ func (c *Engine) recoverStaleRedAdoption(ctx context.Context, store *stagelabels
c.info("adopted PR recovery", logArgs("reason", out.Reason, "base_sha", baseSHA, "failing_check", failing)...)
return out
}
store.MarkAdoptionRecovery(key, obs.HeadSHA, baseSHA)
behind := cmp.GetBehindBy()
if behind > 0 {
if c.requireCurrentBaseFor(obs.Owner + "/" + obs.Repo) {
// Existing recovery evidence already proved that the new base
// clears this failure. This admits repair, never a merge: the
// normal current-head CI and review gates still run afterward.
if reason, err := c.admitBasePreparation(ctx, obs.Owner, obs.Repo, baseRef, obs.Number, obs.HeadSHA, obs.Lane, false); reason != "" || err != nil {
return adoptionRecoverySkipped(nonEmpty(reason, "base-refresh-state-unavailable"))
}
}
out, done := c.recoverByUpdateBranch(ctx, store, key, obs, logArgs("base_sha", baseSHA, "behind_by", behind, "failing_check", failing))
if done {
if !out.Deferred {
store.MarkAdoptionRecovery(key, obs.HeadSHA, baseSHA)
}
return out
}
}
store.MarkAdoptionRecovery(key, obs.HeadSHA, baseSHA)
return c.recoverByRerun(ctx, obs, logArgs("base_sha", baseSHA, "behind_by", behind, "failing_check", failing))
}

Expand All @@ -230,6 +244,9 @@ func (c *Engine) recoverStaleRedAdoption(ctx context.Context, store *stagelabels
func (c *Engine) recoverByUpdateBranch(ctx context.Context, store *stagelabels.Store, key string, obs stageObservation, logArgs []any) (adoptionRecovery, bool) {
fork := obs.HeadRepo != "" && !strings.EqualFold(obs.HeadRepo, obs.Owner+"/"+obs.Repo)
if err := c.executeUpdateBranchEffect(ctx, obs.Owner, obs.Repo, obs.Number, obs.HeadSHA, adoptionRecoveryClaimReason); err != nil {
if wait := preparationWaitReason(err); wait != "" {
return adoptionRecovery{Deferred: true, Reason: adoptionRecoverySkipPrefix + wait}, true
}
var out adoptionRecovery
switch classifyUpdateBranchError(err, fork) {
case updateBranchUpToDate:
Expand Down
43 changes: 43 additions & 0 deletions src/pkg/github/automerge/automerge_adoption_recovery_test.go
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package automerge

import (
"context"
"net/http"
"testing"
"time"
Expand Down Expand Up @@ -150,6 +151,48 @@ func TestAdoptedStaleRedRecovery(t *testing.T) {
}
}

func TestAdoptedRecoveryPreparationWaitDoesNotConsumeAttempt(t *testing.T) {
for _, duplicate := range []bool{false, true} {
t.Run(map[bool]string{false: "queued", true: "duplicate"}[duplicate], func(t *testing.T) {
h, key, done := recoveryHarness(t)
defer done()
h.e.requireCurrentBase = func(string) bool { return true }
h.e.basePreparation = NewBasePreparationStore("")
h.adopt.compareBehind = 2
h.adopt.baseChecks = map[string]string{"coverage": "success"}
front, head, wantReason := 8, "head8", preparationQueued(8)
if duplicate {
front, head, wantReason = 7, "aaa", basePreparationReady
}
h.adopt.superseders[front] = map[string]any{
"number": front, "state": "open",
"head": map[string]string{"sha": head},
"base": map[string]string{"ref": "v5", "sha": "base1"},
}
if reason, err := h.e.admitBasePreparation(context.Background(), "acme", "widget", "v5", front, head, stageLaneContributor, false); reason != "" || err != nil {
t.Fatalf("admit owner: reason=%q error=%v", reason, err)
}
if duplicate {
if err := h.e.claimBasePreparationUpdate(context.Background(), "acme", "widget", front, head); err != nil {
t.Fatal(err)
}
}
rec, _ := h.e.stageStoreOrShared().Get(key)
out := h.e.recoverStaleRedAdoption(context.Background(), h.e.stageStoreOrShared(), key, stageObservation{
Owner: "acme", Repo: "widget", Display: "acme/widget", Number: 7,
BaseRef: "v5", HeadSHA: "aaa", Lane: stageLaneContributor, Reason: recoveryReason,
}, rec)
if out.Attempted || out.Reason != adoptionRecoverySkipPrefix+wantReason || out.Deferred != duplicate {
t.Fatalf("recovery while waiting: %+v", out)
}
rec, _ = h.e.stageStoreOrShared().Get(key)
if rec.AdoptRecoveryCount != 0 || rec.AdoptRecoveryAttempt != "" || len(h.adopt.branchUpdates) != 0 {
t.Fatalf("waiting recovery consumed attempt: record=%+v updates=%v", rec, h.adopt.branchUpdates)
}
})
}
}

// The same head stays red after the re-run: one attempt per (head, base),
// then the escalation runs; a new hive head starts over.
func TestAdoptedStaleRedRecoveryEscalatesAfterOneAttempt(t *testing.T) {
Expand Down
Loading
Loading