Repository navigation
π fix: claim release re-derives covered-by-pr from the claim ledger; close-on-merge on the webhook; ledger backfill - #14971
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Open PR overlap checkBase branch: Conflicts are checked with every PR merged onto the current tip of
|
|
@clubanderson is auto-promoted for contributor-lane merging on this PR: the last 5 merged PRs by this author in this hive's repositories have no revert or sentinel-alert evidence. CI, reviewer verdict, diff fences, soak, holds and branch protection still apply. |
|
@clubanderson this PR is held at Failing checks:
Next: the hive does not push to your branch. Push a new head (or reply here); CI and the review swarm re-run automatically and the PR returns to the normal path. |
|
@clubanderson this PR is held at Why:
Next: the hive does not push to your branch. Push a new head (or reply here); CI and the review swarm re-run automatically and the PR returns to the normal path. |
There was a problem hiding this comment.
Reviewed (style) β two advisory findings, neither blocking. Read at head 36ed214.
applyDuplicatePRGuardlost its doc comment βsrc/cmd/hive/main.go:10116-10124.reconcileLedgerCoverRecordswas inserted between the existing// applyDuplicatePRGuard filters issues β¦block andfunc applyDuplicatePRGuard, so godoc now attaches that paragraph toreconcileLedgerCoverRecordsandapplyDuplicatePRGuard(line 10139) is undocumented. Fix: move the new function (and its comment) above the// applyDuplicatePRGuardblock, or insert a blank line between the two comments and re-add the guard's comment directly above itsfunc.
-// applyDuplicatePRGuard filters issues already claimed by an open hive-authored
-// PR out of the actionable set. Failures are logged, never fatal: the guard is
-// a safety net, and a broken net must not take the hive down with it.
// reconcileLedgerCoverRecords runs github.BackfillCoveredByLedger with the
...
}
+// applyDuplicatePRGuard filters issues already claimed by an open hive-authored
+// PR out of the actionable set. Failures are logged, never fatal: the guard is
+// a safety net, and a broken net must not take the hive down with it.
func applyDuplicatePRGuard(bareRepoduplicates an existing same-package helper βsrc/pkg/github/close_on_merge.go:98-103.prclaims.go:915already hassplitRepoName(repo) (owner, name string)used bycanonicalConfigRepofor exactly this owner-stripping; the new helper is a second spelling of the same idiom.
_, want := splitRepoName(o.OnlyPRRepo)
_, got := splitRepoName(repo)
return o.OnlyPRRepo == "" || strings.EqualFold(want, got)
(and drop bareRepo).
β hive: agent=adjudicator-4 backend=copilot model=claude-fable-5.1 copilot=1.0.88
There was a problem hiding this comment.
Docs currency (read at head 36ed214)
src/docs/operator-reference.md:181 still describes close-on-merge as sweep-only. The issues.close_on_merge_interval row says a fresh merge "closes its issue on the next pass"; this PR adds an immediate per-PR sweep on a MERGED pull_request delivery (src/cmd/hive/webhook_pr_claims.go:47-49, 64-80; CloseOnMergeOptions.OnlyPRRepo/OnlyPRNumber, src/pkg/github/close_on_merge.go:56-62). The same row says "Both the ledger sweep and the backfill pause while the GitHub API budget mode is not normal" β the webhook path does not consult the budget scheduler (closeOnMergeForDelivery calls CloseOnMergeLedgerSweep directly, no closeOnMergeSched.mode), so the sentence is now inaccurate as written for that trigger. Suggested wording (adjust the budget clause if you intend to gate the webhook sweep too):
| `issues.close_on_merge_interval` | `10m`. | How often the close-on-merge **ledger sweep** walks every merged-PR claim in the duplicate-PR claim ledger. Passes in between (every governor eval cycle and manual rescan) only look at claims whose PR merged since the last full walk, so a fresh merge still closes its issue on the next pass without re-reading the whole ledger ([#11778](https://github.com/hivecommons/hive/issues/11778)). In webhook mode a merged `pull_request` delivery whose claims the ledger recorded also runs the sweep for just that PR at once (bounded to 2 min, same identity/trusted-author policy), so a merge right after a pass does not wait for the next one ([#14970](https://github.com/hivecommons/hive/issues/14970)). The scheduled ledger sweep and the backfill pause while the GitHub API budget mode is not `normal`; Hive logs one line on each pause and resume. The per-delivery sweep is not budget-gated. Env override: `HIVE_CLOSE_ON_MERGE_INTERVAL` (Go duration). |
Verified: src/docs/scale-envelope.md:37 cite update to client.go:4764-4774 matches RankActionableIssues at head.
Out of scope (pre-existing, for backlog): src/docs/webhooks.md writer table (lines 20-24) never documented the #14814 pull_requestβclaim-ledger write; the new merge trigger and the ledger backfill (reconcileLedgerCoverRecords, src/cmd/hive/main.go:7206) are likewise only in changelog.d.
β hive: agent=ci-maintainer backend=copilot model=claude-fable-5.1 copilot=1.0.88
|
Hive filed review findings that do not block this PR as backlog items:
Cap: 10 backlog item(s) per PR per day. |
There was a problem hiding this comment.
Correctness β one in-scope finding (read at head 36ed214).
Webhook close-on-merge can settle pr_not_merged from a stale PR detail cache and then block the scheduled sweep for 24h
closeOnMergeForDelivery (src/cmd/hive/webhook_pr_claims.go:71-77) spawns the sweep goroutine from inside the prClaimsHook call, which webhook_cache.go:506 runs before invalidateWebhookPR drops the PR's detail-cache entry at webhook_cache.go:519-521 (sharedPRDetailCache.invalidate, :718). The sweep reads the PR via getPullRequestForCloseOnMerge (src/pkg/github/close_on_merge.go:459), which serves cachedPRDetailAnyAllowUnknown first β any TTL-fresh pre-merge read (eval-cycle enrichment, review, dedupe guard all storePRDetail) still says state=open, merged=false.
If the goroutine wins that race, closeIssueForMergedPRClaimWithIssue returns CloseOnMergeNoop/"pr_not_merged" (close_on_merge.go:525-527). That reason is not in closeOnMergeTransientReasons and Noop is in Settles() (close_on_merge.go:168-176), so CloseOnMergeLedgerSweep calls ledger.MarkCloseOnMergeSettled (:254) and the scheduled sweep skips the claim via closeOnMergeClaimSettled for RecheckAfter (default 24h, config/issues.go). Net effect: an issue that today waits for the next reconcile pass can instead sit open for a day β a regression on exactly the #14646 case the PR targets. The scheduled sweep has the same cache read but runs minutes after merge, so it rarely hit this; the webhook path runs within milliseconds of the event.
Fix: the delivery that triggered the sweep postdates whatever the cache holds, so bypass/evict it for the OnlyPR target:
--- a/src/pkg/github/close_on_merge.go
+++ b/src/pkg/github/close_on_merge.go
@@ func (c *Client) CloseOnMergeLedgerSweep(ctx context.Context, ledger *ClaimLedger, opts CloseOnMergeOptions) CloseOnMergeSweep {
now := opts.now()
recheck := opts.recheckAfter()
+ if opts.OnlyPRNumber > 0 {
+ // A webhook-triggered sweep postdates any cached detail for its PR;
+ // webhook_cache.go invalidates the entry only after the hook returns,
+ // and a cached pre-merge read would settle "pr_not_merged" for
+ // RecheckAfter.
+ if o, n := c.splitRepo(opts.OnlyPRRepo); o != "" && n != "" {
+ sharedPRDetailCache.invalidate(o+"/"+n, opts.OnlyPRNumber)
+ }
+ }
claims := ledger.Claims()Test to add to TestCloseOnMergeLedgerSweepOnlyPRFilter: storePRDetail an open/unmerged PR 8 (with body + user so closeOnMergeCachedPRUsable is true) before the sweep, then assert pull_get == 1 and Action == CloseOnMergeClosed. Without the fix it fails with pr_not_merged.
Advisory (low, non-blocking): the webhook sweep and runCloseOnMergeSweep can now run closeIssueForMergedPRClaim concurrently for the same claim (MarkCloseOnMergeSettled is only stamped afterwards, :254), so a merge landing during a scheduled sweep may post a duplicate close comment. Not a correctness blocker.
β hive: agent=adjudicator backend=copilot model=claude-fable-5.1 copilot=1.0.88
There was a problem hiding this comment.
Intent alignment vs #14970 β the three remedies the issue names are all present (ledger fallback in enrichCoveringPR, OnlyPR webhook sweep, per-cycle backfill outside the SkipOptional gate at src/cmd/hive/main.go:7206). One ordering defect undercuts remedy 2.
Webhook close-on-merge can read the pre-merge PR from cache and no-op (pr_not_merged)
src/pkg/github/webhook_cache.go:506-507 runs the claims hook β which now spawns closeOnMergeForDelivery as a detached goroutine (src/cmd/hive/webhook_pr_claims.go:71) β before the handler invalidates the delivery's PR detail at webhook_cache.go:517-520. The sweep resolves the PR via getPullRequestForCloseOnMerge (src/pkg/github/close_on_merge.go:459), which serves any TTL-fresh cached entry with a non-empty body/user. For a PR that was open on the last snapshot, the cached entry says merged=false, so close_on_merge.go:525 returns CloseOnMergeNoop/pr_not_merged, and the issue waits for the scheduled sweep β the exact latency #14970 (#14646 at 12:54Z) describes. The window is a goroutine start versus a few synchronous lines, so it is intermittent rather than deterministic, and TestPRClaimsHookRunsCloseOnMergeForMergedDelivery can't see it (it only checks the callback fired).
Smallest fix: invalidate before the hook fires, so the webhook sweep's first PR read is live.
--- a/src/pkg/github/webhook_cache.go
+++ b/src/pkg/github/webhook_cache.go
@@
+ res.HeadSHA = webhookHeadSHA(event, p)
+ numbers := webhookInvalidationTargets(event, p, aliases)
+ for _, n := range numbers {
+ c.invalidateWebhookPR(aliases, n)
+ }
+ sharedWebhookTracker.recordInvalidations(len(numbers))
+ res.Invalidated = numbers
if d, ok := c.webhookPRClaimDelivery(event, configured, p, sharedWebhookTracker.now()); ok {
// The PR's claims take effect now, not on the next reconcile pass
// (hivecommons/hive#14814); the live open-PR index for the repo is
// stale the same instant, so the next guard call lists again.
res.ClaimedIssues = d.ClaimedIssues()
c.InvalidateOpenPRLinks(configured)
if hook := prClaimsHook(); hook != nil {
hook(d)
}
c.coverIssuesForWebhookPR(configured, d)
}
if merged, ok := webhookMergedPR(event, p); ok {
c.RecordReviewEvidenceRevert(merged)
}
- res.HeadSHA = webhookHeadSHA(event, p)
- numbers := webhookInvalidationTargets(event, p, aliases)
- for _, n := range numbers {
- c.invalidateWebhookPR(aliases, n)
- }
- sharedWebhookTracker.recordInvalidations(len(numbers))
- res.Invalidated = numbers
return res, nilAdvisory (non-blocking): the webhook sweep skips the closeOnMergeScheduler budget-mode plan the scheduled path consults (src/cmd/hive/close_on_merge_sweep.go:131-136 vs webhook_pr_claims.go:71-77), so it will issue its reads/closes even when the scheduler has paused sweeps for budget. One PR per delivery is small; noting it so the choice is deliberate.
β hive: agent=adjudicator-3 backend=copilot model=claude-fable-5.1 copilot=1.0.88
36ed214 to
0c3e67b
Compare
|
@clubanderson this PR is held at Why:
Next: the hive does not push to your branch. Push a new head (or reply here); CI and the review swarm re-run automatically and the PR returns to the normal path. |
β¦erge on the webhook; ledger backfill A claim release on an issue whose covering PR the duplicate-PR claim ledger already held re-derived 010-actionable because enrichCoveringPR only trusted the store's memory of a 040 record. Nine issues on the hosted spoke sat at 010 with bucket=claimed_by_pr for hours while webhook-driven cached cycles skipped the reconcile sweeps. - enrichCoveringPR falls back to the ledger (SetPRClaimLookup) for claims FilterClaimedIssues would suppress (IssueClaim.CoversOpenIssue), after the existing liveness read; cmd/hive wires it with the red+stale valve. - A MERGED pull_request delivery runs the close-on-merge ledger sweep for that PR at once (CloseOnMergeOptions.OnlyPR*), independent of the reconcile pass cadence. - BackfillCoveredByLedger reconciles store records below 040 under an open ledger claim on every eval cycle, cached ones included. Signed-off-by: Andrew Anderson <andy@clubanderson.com>
β¦ insertion Signed-off-by: Andrew Anderson <andy@clubanderson.com>
0c3e67b to
d651677
Compare
|
merged by Hive trusted-author auto-merge: author @clubanderson holds role owner and GitHub permission admin on repo; CI green at d651677. |
|
Thank you for your contribution! Your PR has been merged. We'd love to hear how your experience was: share feedback |
There was a problem hiding this comment.
Looks correct to me; one low-severity open question on the webhook sweep's budget behaviour.
Verified at d651677, reading the tree rather than the diff alone:
IssueClaim.CoversOpenIssue(pkg/github/prclaims.go:326-331) mirrors the branches inFilterClaimedIssues(prclaims.go:1941-2030): merged β skip, open external β suppress, open hive-closing β suppress, hiveReferenceβ annotate only. So the ledger fallback inenrichCoveringPRand the backfill cover exactly the set the guard suppresses β no new class of issue gets booked at 040.ledgerCoveringPRis only reached whenrememberedCoveringPRmisses, and the existing freshPullRequests.Getliveness check below it still gates the fact, so a closed/merged claim the ledger has not yet released cannot re-cover an issue.prClaimCoverLookup(cmd/hive/webhook_pr_claims.go) appliesclaimingPRRedStaleagainstb.lastActionable, matching the valve the guard uses;ClaimLedger.Lookupis nil-safe (prclaims.go:1457-1473), so the lazy ledger is fine.wantsPR+ the extracontinueinCloseOnMergeLedgerSweepleave the default path (OnlyPRNumber == 0) unchanged.BackfillCoveredByLedgercosts one PR read per record that disagrees with the ledger; converged records cost nothing, so running it on cached cycles is bounded by the disagreement set and shrinks after the first heal.build-and-testis green on this head; the only non-green checks are the cancelledSign-off survives the squash/overlayfs-exec-guardand pendingtide.
Open question (low): runCloseOnMergeSweepWith (cmd/hive/close_on_merge_sweep.go:124-130) skips the scheduled sweep when plan.Skip reports a constrained API budget. closeOnMergeForDelivery runs CloseOnMergeLedgerSweep for the one PR without consulting the scheduler's budget mode, and spawns one goroutine per MERGED delivery with no concurrency cap. For a single PR that is a handful of reads and at most a few closes, so I don't think it's a defect β but if the budget gate was meant to be a hard ceiling, you may want closeOnMergeSched.mode(client) checked before the go func().
β hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88
Merge current v5 so the documented merge trigger and ledger-to-stage backfill ship with their implementation from hivecommons#14971. Remove the now-stale pending language. The reported ARM64 build was canceled by a newer run of the closed adoption PR hivecommons#14980; no compiler or dependency failure was reported. Push this updated head to the original PR branch for fresh CI. Signed-off-by: thatguywob <thatguywob@users.noreply.github.com>
Document pull_request claim writes, their snapshot and stage effects, and the merge trigger and ledger backfill proposed in #14971. Hive-Run: #14972 Hive-Plan: docs-currency-webhook-claim-ledger Hive-Spec: #14972#webhook-ledger-documentation Signed-off-by: thatguywob <thatguywob@users.noreply.github.com> Co-authored-by: thatguywob <thatguywob@users.noreply.github.com>
Fixes #14970
What
Three changes so an issue an open PR covers (
bucket=claimed_by_pr) stays at / returns to040-covered-by-prwithout waiting for the reconcile pass:enrichCoveringPR(pkg/github/stage_facts_sink.go) now falls back to the duplicate-PR claim ledger viaClient.SetPRClaimLookupwhen the store has no 040 memory. Only claimsFilterClaimedIssueswould suppress count (IssueClaim.CoversOpenIssue: open PR, hive-closing or external-authored), and the existing freshPullRequests.Getliveness check still gates it.cmd/hivewires the lookup throughgetClaimLedgerwith the same red+stale release valve the guard uses (prClaimCoverLookup).prClaimsHookFortakes anonMergedcallback; a MERGEDpull_requestdelivery whose claims the ledger recorded runsCloseOnMergeLedgerSweepfor just that PR (CloseOnMergeOptions.OnlyPRRepo/OnlyPRNumber), detached, 2-min bound, same identity/trusted-author policy (closeOnMergeOptionsForshared with the scheduled sweep). Closing no longer depends on the sweep cadence or on the issue's stage record.Client.BackfillCoveredByLedgerwalks the ledger's open covering claims; a store record below 040 gets aCoveredByfact after a fresh PR read confirms it is open. Runs on every eval cycle (cached cycles included βreconcileLedgerCoverRecords, not gated onSkipOptional), so the first cycle after the roll heals the 9 stuck issues. No API call when ledger and store agree.Tests
pkg/github/stage_ledger_backfill_test.go: release under hive/external ledger claim β 040 (one PR read); weak/merged claims ignored (no read, 010); closed PR not remembered;CoversOpenIssuetable; backfill heals below-040, no-op when converged, respects red+stale and merged, disabled lane no-op;wantsPR.close_on_merge_forge_test.go:OnlyPRfilter sweeps one PR, leaves others unsettled.cmd/hive/webhook_pr_claims_test.go:onMergedfires only for merged deliveries with recorded claims.time.Sleep.go build ./...,go vet,go test ./pkg/github/ ./pkg/stageengine/ ./cmd/hive/green.Evidence
See #14970: #14935
09:37 020 β 09:42 010afterlease revokedunder open PR #14941; #14646 under PR #14920 merged 12:54Z, close-on-merge sweep last ran 12:41Z while cached cycles continued.