Repository navigation
π pr-models: keep attributed PR history past the 14-day closed-PR scan - #19371
hivecommons-hive[bot] merged 6 commits into
Conversation
β¦t every PR The Governor "PRs by model" card and Operations "Most effective models" table took closed attributed PRs only from the live per-repo scan, which stops at PRs updated in the last 14 days and keeps its result in memory. A PR closed and left alone for longer dropped out of the 30d/All windows, a restart lost everything, and models lost different shares depending on when they were used (projectbluefin: 52% of 707 PRs since 10-20 lost, from 55% of claude-opus-5.5 to 90% of claude-fable-5.1), so the ranking compared unequal samples. - pkg/github/attributed_pr_history.go: a JSON store at /data/attributed-pr-history.json (tmp+rename, like the PR rework cache) of every attributed PR the enumeration sees, open or closed: parsed trailer fields, state, times, labels, supersedes and rework stats, never a body. Written once per enumeration pass; closed records older than 400 days or beyond 20000 are pruned and coverage moves forward to match. - A one-time backfill walks closed PRs by creation, newest first, back 30 days (HIVE_ATTRIBUTED_PR_BACKFILL_LOOKBACK) and measures rework for stored closed PRs lacking it. At most 2 pages and 5 rework measurements per pass across repos, only in the normal API budget mode, cursor saved in the store, marked done per repo; a done repo costs no calls. - The live closed scan is unchanged (14-day lookback, 5-page cap); it now also records its floor so the store tracks whether scans have been continuous. Coverage start = the first floor of the current unbroken run of scans, or the backfill horizon once the backfill finished within that run, or full history when a list ran out. - Both endpoints merge the store's closed PRs under the live records and report coverage from it: new closed_prs_since, closed_pr_lookback_days as the days covered, closed_prs_partial only when the window starts before the coverage start (All: unless full). Scoring and min_merged_prs are unchanged. - Store and backfill errors only warn; enumeration never fails on them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BQYnPGFJx81c8JXR3o1sg1 Signed-off-by: Danathar <6772335+Danathar@users.noreply.github.com>
- Rename the backfill test fake (collided with backfillTestServer in review_backlog_split_backfill_test.go). - Rework tries count only errors about the PR itself. Rate limit, pacing shed and a cancelled or expired pass end the pass; 5xx/network move on to the next repo; neither costs a try. - A listing error that is not about the budget keeps that repo's cursor for the next pass and lets the pass go on to the next repo. - upsert skips closed PRs created before the retention cutoff or pruned_through, so they are not re-inserted and pruned (and the file rewritten) every pass. - Single-flight the backfill step with a TryLock: several callers run EnumerateActionable and would otherwise list the same pages twice. - Repos that never scanned are left out of the coverage start instead of making every window partial; with no scanned repo nothing is covered. - Null entries in the file are dropped on load (they panicked in the walks inside enumeration). - Open records carry last_seen_at (rewritten at most daily) and are pruned when not seen within the retention; records of repos no longer configured are outside the record cap and age out by retention. - enrichAttributedPRRework returns its cache so the history does not read the rework cache file a second time per repo. - A resumed backfill walk re-reads the page it read last, catching up to a page of PRs shifted behind the cursor by reopens; the remaining case is documented. - docs/admin-mcp.md: coverage now describes the stored history. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BQYnPGFJx81c8JXR3o1sg1 Signed-off-by: Danathar <6772335+Danathar@users.noreply.github.com>
- A rework fetch lost to a GitHub hiccup (5xx, network, timeout) now counts against its own cap of 10 and the next PR gets its turn; before, the fixed pending order put the same failing PR first every pass and the repo's backfill could never finish. Stop-class errors still end the pass without counting. - A repo whose live enumeration succeeded in this process but has no closed-scan state is uncovered (its closed PRs are missing while its open ones are live); only repos that never enumerated are left out. - A repo whose last closed scan is older than the live lookback is uncovered: PRs closed since are neither stored nor live. - Failed backfill listings (per-repo errors or hiccups) are counted per walk and reset by a good page; after 5 the walk stops, the repo stays not done, and coverage keeps reporting that it does not reach the horizon. - Note the rework cache's last-writer-wins overlap in a comment. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BQYnPGFJx81c8JXR3o1sg1 Signed-off-by: Danathar <6772335+Danathar@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BQYnPGFJx81c8JXR3o1sg1 Signed-off-by: Danathar <6772335+Danathar@users.noreply.github.com>
β¦ibuted-history Signed-off-by: Danathar <6772335+Danathar@users.noreply.github.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Open PR overlap checkBase branch: Conflicts are checked with every PR merged onto the current tip of
|
|
@Danathar this PR is back in your court ( Failing checks:
Reply here or push a new head to this branch (same sign-off); CI and the review swarm re-run automatically and the PR returns to the normal path. Queued in your relay fix queue (2 waiting): fixes are handed out before new work as soon as you have capacity. |
β¦scan's The test expected model "auto", but the trailer's model=auto normalizes (NormalizeAttributionModel) to "unknown" for live and stored PRs alike. Compare against what the live scan parsed instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BQYnPGFJx81c8JXR3o1sg1 Signed-off-by: Danathar <6772335+Danathar@users.noreply.github.com>
|
Hive status β updated in place; edits do not notify. Full history is in the Activity Monitor. Evidence bundle `#19371e76d133` sha256:30e51162bb297517db799e5c0bc0d93091cdefbffcbdc3ecd4b9e05eff018b40 β download from the dashboard or `hivectl review evidence #19371`.merged by Hive trusted-author auto-merge: author |
Summary
Governor "PRs by model" (
/api/governor/pr-models) and Operations "Most effective models" (/api/contribute/effective-models) offer 30d and "all" windows. But closed attributed PRs came only from a live per-repo scan that stops at PRs updated in the last 14 days, and the results lived only in memory. A PR closed and left alone for more than 14 days dropped out, and a restart lost everything.That skews the ranking, not just the history, because models lose different shares depending on when they were used. On projectbluefin (707 trailer-carrying PRs since 2026-10-20), the cutoff dropped 55% of claude-opus-5.5's PRs, 87% of gpt-6.1-sol's and 90% of claude-fable-5.1's, or 52% overall. The coverage block admitted the gap (
closed_prs_partial: true), but the ranking was still computed on unequal samples.This keeps a persistent history of attributed PRs and fills it back 30 days once.
What changes
/data/attributed-pr-history.jsonholds one record per attributed PR: repo, number, title, URL, labels, agent, backend, model, effort, state, created/closed/merged times, supersedes and the rework stats the cards read. It never holds PR bodies. A record is written when the hive first sees a PR and updated when it closes. It survives restarts, is written at most once per enumeration pass (only when something changed), and is pruned at 400 days or 20,000 closed records. It follows the PR rework cache in the same package: one versioned JSON file, written tmp+rename under a mutex.min_merged_prs.HIVE_ATTRIBUTED_PR_BACKFILL_LOOKBACK). It runs as one bounded step per enumeration pass:closed_prs_since(new) is where the history really starts.closed_prs_partialis true only when the requested window starts before it.Kept as before
warnAttributedClosedPRScanpath), and store or backfill errors only warn.Known limits
Tests
Nothing was run locally (AGENTS.md); CI is the first build.
π€ Generated with Claude Code
https://claude.ai/code/session_01BQYnPGFJx81c8JXR3o1sg1