Skip to content

πŸ› fix: claim release re-derives covered-by-pr from the claim ledger; close-on-merge on the webhook; ledger backfill - #14971

Merged
hivecommons-hive[bot] merged 2 commits into
v5from
fix/claim-release-rederives-bucket
Oct 28, 2026
Merged

hivecommons-hive[bot] merged 2 commits into
v5from
fix/claim-release-rederives-bucket

Conversation

@clubanderson

Copy link
Copy Markdown
Member

Fixes #14970

What

Three changes so an issue an open PR covers (bucket=claimed_by_pr) stays at / returns to 040-covered-by-pr without waiting for the reconcile pass:

  1. Claim-release path re-derives from the ledger β€” enrichCoveringPR (pkg/github/stage_facts_sink.go) now falls back to the duplicate-PR claim ledger via Client.SetPRClaimLookup when the store has no 040 memory. Only claims FilterClaimedIssues would suppress count (IssueClaim.CoversOpenIssue: open PR, hive-closing or external-authored), and the existing fresh PullRequests.Get liveness check still gates it. cmd/hive wires the lookup through getClaimLedger with the same red+stale release valve the guard uses (prClaimCoverLookup).
  2. Close-on-merge on the webhook β€” prClaimsHookFor takes an onMerged callback; a MERGED pull_request delivery whose claims the ledger recorded runs CloseOnMergeLedgerSweep for just that PR (CloseOnMergeOptions.OnlyPRRepo/OnlyPRNumber), detached, 2-min bound, same identity/trusted-author policy (closeOnMergeOptionsFor shared with the scheduled sweep). Closing no longer depends on the sweep cadence or on the issue's stage record.
  3. Ledger↔store backfill β€” Client.BackfillCoveredByLedger walks the ledger's open covering claims; a store record below 040 gets a CoveredBy fact after a fresh PR read confirms it is open. Runs on every eval cycle (cached cycles included β€” reconcileLedgerCoverRecords, not gated on SkipOptional), so the first cycle after the roll heals the 9 stuck issues. No API call when ledger and store agree.

Tests

  • pkg/github/stage_ledger_backfill_test.go: release under hive/external ledger claim β†’ 040 (one PR read); weak/merged claims ignored (no read, 010); closed PR not remembered; CoversOpenIssue table; backfill heals below-040, no-op when converged, respects red+stale and merged, disabled lane no-op; wantsPR.
  • close_on_merge_forge_test.go: OnlyPR filter sweeps one PR, leaves others unsettled.
  • cmd/hive/webhook_pr_claims_test.go: onMerged fires only for merged deliveries with recorded claims.
  • No time.Sleep. go build ./..., go vet, go test ./pkg/github/ ./pkg/stageengine/ ./cmd/hive/ green.

Evidence

See #14970: #14935 09:37 020 β†’ 09:42 010 after lease revoked under open PR #14941; #14646 under PR #14920 merged 12:54Z, close-on-merge sweep last ran 12:41Z while cached cycles continued.

@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign clubanderson for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubestellar-prow kubestellar-prow Bot added dco-signoff: yes Indicates the PR's author has signed the DCO. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Oct 28, 2026
@github-actions

github-actions Bot commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Open PR overlap check

Base branch: v5

Conflicts are checked with every PR merged onto the current tip of v5, so a PR that only needs a rebase is not reported as conflicting with its siblings.

PR Title Relation Files
#14821 πŸ› fix: preserve supersession decisions across stage reconciliation same files src/cmd/hive/main.go
#14524 ✨ feat(hub,agent): Hub-coordinated live CLI session pool for multi-tenant runner capacity same files src/cmd/hive/main.go
#14965 πŸ› fix(dashboard): name the blocking PR on tracker rows and keep webhook unblocks from being overwritten same files src/cmd/hive/eval_cycle_source.go
src/cmd/hive/main.go
src/docs/scale-envelope.md
src/pkg/github/client.go
#14921 ✨ feature: inline the linked Linear issue into review kicks same files src/cmd/hive/main.go

@hivecommons-hive

Copy link
Copy Markdown
Contributor

@clubanderson is auto-promoted for contributor-lane merging on this PR: the last 5 merged PRs by this author in this hive's repositories have no revert or sentinel-alert evidence. CI, reviewer verdict, diff fences, soak, holds and branch protection still apply.

@hivecommons-hive hivecommons-hive Bot added author/human A person pushed this PR themselves dco All commits are DCO signed-off role/trusted Author is trusted for unattended merge (trusted-author role or registry/claim tier in require_tiers) stage/071-needs-author Human-authored PR: CI red, changes requested or conflicting; diagnosis posted, the author must act labels Oct 28, 2026
@hivecommons-hive

Copy link
Copy Markdown
Contributor

@clubanderson this PR is held at stage/071-needs-author and the hive will not merge it yet.

Failing checks:

  • relative links and anchors resolve

Next: the hive does not push to your branch. Push a new head (or reply here); CI and the review swarm re-run automatically and the PR returns to the normal path.

@hivecommons-hive

Copy link
Copy Markdown
Contributor

@clubanderson this PR is held at stage/071-needs-author and the hive will not merge it yet.

Why:

  • the branch conflicts with the base and needs a rebase

Next: the hive does not push to your branch. Push a new head (or reply here); CI and the review swarm re-run automatically and the PR returns to the normal path.

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed (style) β€” two advisory findings, neither blocking. Read at head 36ed214.

  1. applyDuplicatePRGuard lost its doc comment β€” src/cmd/hive/main.go:10116-10124. reconcileLedgerCoverRecords was inserted between the existing // applyDuplicatePRGuard filters issues … block and func applyDuplicatePRGuard, so godoc now attaches that paragraph to reconcileLedgerCoverRecords and applyDuplicatePRGuard (line 10139) is undocumented. Fix: move the new function (and its comment) above the // applyDuplicatePRGuard block, or insert a blank line between the two comments and re-add the guard's comment directly above its func.
-// applyDuplicatePRGuard filters issues already claimed by an open hive-authored
-// PR out of the actionable set. Failures are logged, never fatal: the guard is
-// a safety net, and a broken net must not take the hive down with it.
 // reconcileLedgerCoverRecords runs github.BackfillCoveredByLedger with the
 ...
 }
 
+// applyDuplicatePRGuard filters issues already claimed by an open hive-authored
+// PR out of the actionable set. Failures are logged, never fatal: the guard is
+// a safety net, and a broken net must not take the hive down with it.
 func applyDuplicatePRGuard(
  1. bareRepo duplicates an existing same-package helper β€” src/pkg/github/close_on_merge.go:98-103. prclaims.go:915 already has splitRepoName(repo) (owner, name string) used by canonicalConfigRepo for exactly this owner-stripping; the new helper is a second spelling of the same idiom.
	_, want := splitRepoName(o.OnlyPRRepo)
	_, got := splitRepoName(repo)
	return o.OnlyPRRepo == "" || strings.EqualFold(want, got)

(and drop bareRepo).

β€” hive: agent=adjudicator-4 backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed β€” no findings from this perspective.

β€” hive: agent=adjudicator-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docs currency (read at head 36ed214)

src/docs/operator-reference.md:181 still describes close-on-merge as sweep-only. The issues.close_on_merge_interval row says a fresh merge "closes its issue on the next pass"; this PR adds an immediate per-PR sweep on a MERGED pull_request delivery (src/cmd/hive/webhook_pr_claims.go:47-49, 64-80; CloseOnMergeOptions.OnlyPRRepo/OnlyPRNumber, src/pkg/github/close_on_merge.go:56-62). The same row says "Both the ledger sweep and the backfill pause while the GitHub API budget mode is not normal" β€” the webhook path does not consult the budget scheduler (closeOnMergeForDelivery calls CloseOnMergeLedgerSweep directly, no closeOnMergeSched.mode), so the sentence is now inaccurate as written for that trigger. Suggested wording (adjust the budget clause if you intend to gate the webhook sweep too):

| `issues.close_on_merge_interval` | `10m`. | How often the close-on-merge **ledger sweep** walks every merged-PR claim in the duplicate-PR claim ledger. Passes in between (every governor eval cycle and manual rescan) only look at claims whose PR merged since the last full walk, so a fresh merge still closes its issue on the next pass without re-reading the whole ledger ([#11778](https://github.com/hivecommons/hive/issues/11778)). In webhook mode a merged `pull_request` delivery whose claims the ledger recorded also runs the sweep for just that PR at once (bounded to 2 min, same identity/trusted-author policy), so a merge right after a pass does not wait for the next one ([#14970](https://github.com/hivecommons/hive/issues/14970)). The scheduled ledger sweep and the backfill pause while the GitHub API budget mode is not `normal`; Hive logs one line on each pause and resume. The per-delivery sweep is not budget-gated. Env override: `HIVE_CLOSE_ON_MERGE_INTERVAL` (Go duration). |

Verified: src/docs/scale-envelope.md:37 cite update to client.go:4764-4774 matches RankActionableIssues at head.

Out of scope (pre-existing, for backlog): src/docs/webhooks.md writer table (lines 20-24) never documented the #14814 pull_request→claim-ledger write; the new merge trigger and the ledger backfill (reconcileLedgerCoverRecords, src/cmd/hive/main.go:7206) are likewise only in changelog.d.

β€” hive: agent=ci-maintainer backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Hive filed review findings that do not block this PR as backlog items:

Cap: 10 backlog item(s) per PR per day.

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correctness β€” one in-scope finding (read at head 36ed214).

Webhook close-on-merge can settle pr_not_merged from a stale PR detail cache and then block the scheduled sweep for 24h

closeOnMergeForDelivery (src/cmd/hive/webhook_pr_claims.go:71-77) spawns the sweep goroutine from inside the prClaimsHook call, which webhook_cache.go:506 runs before invalidateWebhookPR drops the PR's detail-cache entry at webhook_cache.go:519-521 (sharedPRDetailCache.invalidate, :718). The sweep reads the PR via getPullRequestForCloseOnMerge (src/pkg/github/close_on_merge.go:459), which serves cachedPRDetailAnyAllowUnknown first β€” any TTL-fresh pre-merge read (eval-cycle enrichment, review, dedupe guard all storePRDetail) still says state=open, merged=false.

If the goroutine wins that race, closeIssueForMergedPRClaimWithIssue returns CloseOnMergeNoop/"pr_not_merged" (close_on_merge.go:525-527). That reason is not in closeOnMergeTransientReasons and Noop is in Settles() (close_on_merge.go:168-176), so CloseOnMergeLedgerSweep calls ledger.MarkCloseOnMergeSettled (:254) and the scheduled sweep skips the claim via closeOnMergeClaimSettled for RecheckAfter (default 24h, config/issues.go). Net effect: an issue that today waits for the next reconcile pass can instead sit open for a day β€” a regression on exactly the #14646 case the PR targets. The scheduled sweep has the same cache read but runs minutes after merge, so it rarely hit this; the webhook path runs within milliseconds of the event.

Fix: the delivery that triggered the sweep postdates whatever the cache holds, so bypass/evict it for the OnlyPR target:

--- a/src/pkg/github/close_on_merge.go
+++ b/src/pkg/github/close_on_merge.go
@@ func (c *Client) CloseOnMergeLedgerSweep(ctx context.Context, ledger *ClaimLedger, opts CloseOnMergeOptions) CloseOnMergeSweep {
 	now := opts.now()
 	recheck := opts.recheckAfter()
+	if opts.OnlyPRNumber > 0 {
+		// A webhook-triggered sweep postdates any cached detail for its PR;
+		// webhook_cache.go invalidates the entry only after the hook returns,
+		// and a cached pre-merge read would settle "pr_not_merged" for
+		// RecheckAfter.
+		if o, n := c.splitRepo(opts.OnlyPRRepo); o != "" && n != "" {
+			sharedPRDetailCache.invalidate(o+"/"+n, opts.OnlyPRNumber)
+		}
+	}
 	claims := ledger.Claims()

Test to add to TestCloseOnMergeLedgerSweepOnlyPRFilter: storePRDetail an open/unmerged PR 8 (with body + user so closeOnMergeCachedPRUsable is true) before the sweep, then assert pull_get == 1 and Action == CloseOnMergeClosed. Without the fix it fails with pr_not_merged.

Advisory (low, non-blocking): the webhook sweep and runCloseOnMergeSweep can now run closeIssueForMergedPRClaim concurrently for the same claim (MarkCloseOnMergeSettled is only stamped afterwards, :254), so a merge landing during a scheduled sweep may post a duplicate close comment. Not a correctness blocker.

β€” hive: agent=adjudicator backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Intent alignment vs #14970 β€” the three remedies the issue names are all present (ledger fallback in enrichCoveringPR, OnlyPR webhook sweep, per-cycle backfill outside the SkipOptional gate at src/cmd/hive/main.go:7206). One ordering defect undercuts remedy 2.

Webhook close-on-merge can read the pre-merge PR from cache and no-op (pr_not_merged)

src/pkg/github/webhook_cache.go:506-507 runs the claims hook β€” which now spawns closeOnMergeForDelivery as a detached goroutine (src/cmd/hive/webhook_pr_claims.go:71) β€” before the handler invalidates the delivery's PR detail at webhook_cache.go:517-520. The sweep resolves the PR via getPullRequestForCloseOnMerge (src/pkg/github/close_on_merge.go:459), which serves any TTL-fresh cached entry with a non-empty body/user. For a PR that was open on the last snapshot, the cached entry says merged=false, so close_on_merge.go:525 returns CloseOnMergeNoop/pr_not_merged, and the issue waits for the scheduled sweep β€” the exact latency #14970 (#14646 at 12:54Z) describes. The window is a goroutine start versus a few synchronous lines, so it is intermittent rather than deterministic, and TestPRClaimsHookRunsCloseOnMergeForMergedDelivery can't see it (it only checks the callback fired).

Smallest fix: invalidate before the hook fires, so the webhook sweep's first PR read is live.

--- a/src/pkg/github/webhook_cache.go
+++ b/src/pkg/github/webhook_cache.go
@@
+	res.HeadSHA = webhookHeadSHA(event, p)
+	numbers := webhookInvalidationTargets(event, p, aliases)
+	for _, n := range numbers {
+		c.invalidateWebhookPR(aliases, n)
+	}
+	sharedWebhookTracker.recordInvalidations(len(numbers))
+	res.Invalidated = numbers
 	if d, ok := c.webhookPRClaimDelivery(event, configured, p, sharedWebhookTracker.now()); ok {
 		// The PR's claims take effect now, not on the next reconcile pass
 		// (hivecommons/hive#14814); the live open-PR index for the repo is
 		// stale the same instant, so the next guard call lists again.
 		res.ClaimedIssues = d.ClaimedIssues()
 		c.InvalidateOpenPRLinks(configured)
 		if hook := prClaimsHook(); hook != nil {
 			hook(d)
 		}
 		c.coverIssuesForWebhookPR(configured, d)
 	}
 	if merged, ok := webhookMergedPR(event, p); ok {
 		c.RecordReviewEvidenceRevert(merged)
 	}
-	res.HeadSHA = webhookHeadSHA(event, p)
-	numbers := webhookInvalidationTargets(event, p, aliases)
-	for _, n := range numbers {
-		c.invalidateWebhookPR(aliases, n)
-	}
-	sharedWebhookTracker.recordInvalidations(len(numbers))
-	res.Invalidated = numbers
 	return res, nil

Advisory (non-blocking): the webhook sweep skips the closeOnMergeScheduler budget-mode plan the scheduled path consults (src/cmd/hive/close_on_merge_sweep.go:131-136 vs webhook_pr_claims.go:71-77), so it will issue its reads/closes even when the scheduler has paused sweeps for budget. One PR per delivery is small; noting it so the choice is deliberate.

β€” hive: agent=adjudicator-3 backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive

Copy link
Copy Markdown
Contributor

@clubanderson this PR is held at stage/071-needs-author and the hive will not merge it yet.

Why:

  • the branch conflicts with the base and needs a rebase

Next: the hive does not push to your branch. Push a new head (or reply here); CI and the review swarm re-run automatically and the PR returns to the normal path.

…erge on the webhook; ledger backfill

A claim release on an issue whose covering PR the duplicate-PR claim ledger
already held re-derived 010-actionable because enrichCoveringPR only trusted
the store's memory of a 040 record. Nine issues on the hosted spoke sat at
010 with bucket=claimed_by_pr for hours while webhook-driven cached cycles
skipped the reconcile sweeps.

- enrichCoveringPR falls back to the ledger (SetPRClaimLookup) for claims
  FilterClaimedIssues would suppress (IssueClaim.CoversOpenIssue), after
  the existing liveness read; cmd/hive wires it with the red+stale valve.
- A MERGED pull_request delivery runs the close-on-merge ledger sweep for
  that PR at once (CloseOnMergeOptions.OnlyPR*), independent of the
  reconcile pass cadence.
- BackfillCoveredByLedger reconciles store records below 040 under an open
  ledger claim on every eval cycle, cached ones included.

Signed-off-by: Andrew Anderson <andy@clubanderson.com>
… insertion

Signed-off-by: Andrew Anderson <andy@clubanderson.com>
@clubanderson
clubanderson force-pushed the fix/claim-release-rederives-bucket branch from 0c3e67b to d651677 Compare October 28, 2026 15:13
@kubestellar-prow kubestellar-prow Bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 28, 2026
@hivecommons-hive hivecommons-hive Bot added stage/050-review-pending Review swarm requested; awaiting a verdict for the current head and removed stage/071-needs-author Human-authored PR: CI red, changes requested or conflicting; diagnosis posted, the author must act labels Oct 28, 2026
@hivecommons-hive
hivecommons-hive Bot merged commit 294d0c7 into v5 Oct 28, 2026
78 of 81 checks passed
@hivecommons-hive

Copy link
Copy Markdown
Contributor
Evidence bundle `#14971d651677` sha256:3ae464ecfb921314f0fc77a1d5660035a464e3add37ca8dbea65bb0f1c04bac4 β€” download from the dashboard or `hivectl review evidence #14971`.

@kubestellar-prow
kubestellar-prow Bot deleted the fix/claim-release-rederives-bucket branch October 28, 2026 15:27
@hivecommons-hive

Copy link
Copy Markdown
Contributor

merged by Hive trusted-author auto-merge: author @clubanderson holds role owner and GitHub permission admin on repo; CI green at d651677.

@github-actions

Copy link
Copy Markdown
Contributor

Thank you for your contribution! Your PR has been merged.

We'd love to hear how your experience was: share feedback

@hivecommons-hive hivecommons-hive Bot removed role/trusted Author is trusted for unattended merge (trusted-author role or registry/claim tier in require_tiers) stage/050-review-pending Review swarm requested; awaiting a verdict for the current head labels Oct 28, 2026

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks correct to me; one low-severity open question on the webhook sweep's budget behaviour.

Verified at d651677, reading the tree rather than the diff alone:

  • IssueClaim.CoversOpenIssue (pkg/github/prclaims.go:326-331) mirrors the branches in FilterClaimedIssues (prclaims.go:1941-2030): merged β†’ skip, open external β†’ suppress, open hive-closing β†’ suppress, hive Reference β†’ annotate only. So the ledger fallback in enrichCoveringPR and the backfill cover exactly the set the guard suppresses β€” no new class of issue gets booked at 040.
  • ledgerCoveringPR is only reached when rememberedCoveringPR misses, and the existing fresh PullRequests.Get liveness check below it still gates the fact, so a closed/merged claim the ledger has not yet released cannot re-cover an issue.
  • prClaimCoverLookup (cmd/hive/webhook_pr_claims.go) applies claimingPRRedStale against b.lastActionable, matching the valve the guard uses; ClaimLedger.Lookup is nil-safe (prclaims.go:1457-1473), so the lazy ledger is fine.
  • wantsPR + the extra continue in CloseOnMergeLedgerSweep leave the default path (OnlyPRNumber == 0) unchanged.
  • BackfillCoveredByLedger costs one PR read per record that disagrees with the ledger; converged records cost nothing, so running it on cached cycles is bounded by the disagreement set and shrinks after the first heal.
  • build-and-test is green on this head; the only non-green checks are the cancelled Sign-off survives the squash/overlayfs-exec-guard and pending tide.

Open question (low): runCloseOnMergeSweepWith (cmd/hive/close_on_merge_sweep.go:124-130) skips the scheduled sweep when plan.Skip reports a constrained API budget. closeOnMergeForDelivery runs CloseOnMergeLedgerSweep for the one PR without consulting the scheduler's budget mode, and spawns one goroutine per MERGED delivery with no concurrency cap. For a single PR that is a handful of reads and at most a few closes, so I don't think it's a defect β€” but if the budget gate was meant to be a hard ceiling, you may want closeOnMergeSched.mode(client) checked before the go func().

β€” hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88

thatguywob added a commit to thatguywob/hive that referenced this pull request Oct 28, 2026
Merge current v5 so the documented merge trigger and ledger-to-stage backfill ship with their implementation from hivecommons#14971. Remove the now-stale pending language.

The reported ARM64 build was canceled by a newer run of the closed adoption PR hivecommons#14980; no compiler or dependency failure was reported. Push this updated head to the original PR branch for fresh CI.

Signed-off-by: thatguywob <thatguywob@users.noreply.github.com>
hivecommons-hive Bot pushed a commit that referenced this pull request Oct 28, 2026
Document pull_request claim writes, their snapshot and stage effects, and the merge trigger and ledger backfill proposed in #14971.

Hive-Run: #14972
Hive-Plan: docs-currency-webhook-claim-ledger
Hive-Spec: #14972#webhook-ledger-documentation

Signed-off-by: thatguywob <thatguywob@users.noreply.github.com>
Co-authored-by: thatguywob <thatguywob@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author/human A person pushed this PR themselves dco All commits are DCO signed-off dco-signoff: yes Indicates the PR's author has signed the DCO. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Claim release regresses covered issues to stage/010 (bucket=claimed_by_pr); close-on-merge and stage store never heal between reconcile passes

1 participant