Repository navigation
🐛 fix(github): historical PR accounting waits while the GitHub API budget is low - #17075
Merged
hivecommons-hive[bot] merged 1 commit intoNov 7, 2026
Merged
Conversation
The terminal-PR enrichment worker (merged-by and review reads for the throughput and review counts) ran a batch every ten seconds regardless of the GitHub API budget. After a restart the 72-hour settle replay queues every recent merged or closed PR at about two REST calls each; on the projectbluefin hive on 2026-11-07 that was 411 calls in five minutes, about 280 of them after the budget had entered conserve. The worker now checks APIBudgetMode before each batch and makes no GitHub call outside normal mode. The queue is kept and drained once the budget recovers. The pause and the resume are each logged once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BQYnPGFJx81c8JXR3o1sg1 Signed-off-by: Danathar <Danathar@users.noreply.github.com>
Contributor
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Contributor
Contributor
hivecommons-hive Bot
added a commit
that referenced
this pull request
Nov 7, 2026
… sweeps name their REST calls (#17101) Several sweeps built their list query's since from time.Now() on every pass (un-park comment reads, closed-issue stage sweep and review backlog split, shared-CI rerun, same-component sibling check). The URL changed on every call, so the ETag cache, keyed on the full URL, never matched and every pass was charged. The since sent to GitHub is now rounded down to the UTC hour (cacheableSince); callers keep filtering on their exact cutoff, so the wider window misses nothing and answers nothing twice. Untagged REST calls from the sweep workers, the governor cycle, the forgestate loop, the metrics collector, the request watchers, the mention poller and dashboard requests now carry WithRESTCaller names, so /api/gh-rate-limits no longer lumps ~4,400 calls an hour under "hive". Refs #17075 Claude-Session: https://claude.ai/code/session_01BQYnPGFJx81c8JXR3o1sg1 Signed-off-by: Danathar <Danathar@users.noreply.github.com> Co-authored-by: Danathar <Danathar@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: hivecommons-hive[bot] <280983584+hivecommons-hive[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The historical PR accounting worker reads the merger and the reviews of each recently merged or closed PR, for the throughput and review counts. With this change it waits while the GitHub API budget is in
conserveorcritical. It keeps its queue and catches up once the budget is back tonormal.Before this, it ran a batch every ten seconds whatever the budget said. After a restart it replays every PR merged or closed in the last 72 hours, at about two REST calls each. On the projectbluefin hive today, that replay spent about 400 calls in five minutes. Most of them came after the budget had already dropped below the conserve threshold.
What went wrong
The projectbluefin hive has 20 repos and one App installation, with a limit of 7000 calls an hour. It was upgraded to 7b9eeb8 at 18:21Z on 2026-11-07. Its log and its own REST accounting (
/api/statusghRateLimits.top_consumers) show this:historical PR enrichment batchlines. Each covers 8 PRs, so about 230 PRs in all.hive:pr_terminal_reviews, with none answered from the ETag cache. It lists 195 charged calls forhive:pr_terminal_merged_by, with one answered from the cache. That makes 411 calls. Both numbers stayed the same for the rest of the hour, so this is a one-off cost of the restart.github api budget mode changedshows the budget going toconserveat 18:26:19Z (remaining 793) and tocriticalat 18:28:46Z (remaining 214). The worker kept going until 18:29:19Z. About 20 of its 30 batches ran after the budget was already in conserve, which is about 280 calls.403 API rate limit of 7000 still exceeded, and all 20 repos failed to enumerate.The eval loop already skips its optional enrichment steps in conserve and critical (
APIBudgetEnrichmentSteps). This worker was added later, in #16289. It runs on its own ticker and never checked the budget. It feeds only accounting: nothing plans, claims or dispatches from it, so making it wait is safe.This PR does not fix the hive's steady-state use, which was also over budget before the restart. That is a separate and larger problem.
What changes
enrichTerminalPRBatchasksAPIBudgetMode()first. Outsidenormalit returns without making any GitHub call. Pending tasks stay queued and keep coalescing by PR, as before.historical PR enrichment paused for GitHub API budgetonce when it pauses, with the mode, the remaining count and the queue size. It logshistorical PR enrichment resumedonce when it starts again. It does not log every ten seconds.Files
src/pkg/github/pr_terminal_enrichment.go: the budget check and the log lines.src/pkg/github/pr_terminal_enrichment_test.go: the new test.src/docs/operator-reference/sweeps-and-operations.md: one paragraph in "GitHub API budget governor".changelog.d/fixed-historical-pr-enrichment-api-budget.mdTests
TestTerminalEnrichmentPausesOutsideNormalAPIBudgetqueues two historical PRs. It runs a batch at remaining 675 (conserve) and again at 214 (critical), and checks that no request reaches the test server and that both tasks are still queued. Then it runs a batch at 6000 (normal) and checks that both tasks drain.remaining=675: historical enrichment made 2 GitHub requests outside normal budget mode.TestTerminalEnrichment*tests pass, and so does the full./pkg/github/package.go vet ./pkg/github/is clean.🤖 Generated with Claude Code
https://claude.ai/code/session_01BQYnPGFJx81c8JXR3o1sg1