Repository navigation
π fix(github): the un-park sweep reuses comment and timeline reads of items whose updated_at has not moved - #17174
Conversation
β¦nchanged items The issue un-park sweep re-read the comment list of every parked issue, every PR waiting at 085/070 and every recently updated PR, plus the label timeline of every approval-labelled issue, on every pass. Almost none of those items change between passes, yet most reads were charged: on the knuckle hive `hive:issue_unpark` made 1222 comment reads (878 charged) and 480 timeline reads (363 charged) in eight passes, the largest consumer. The sweep now keeps the last comment list and label timeline per item, keyed by the item's updated_at from the issue list it starts from, and reuses them while updated_at has not moved. Only raw data is cached: the seven-day command window, repo permission and label-trust checks still run every pass. An item is read again when updated_at moves, when the read was within two minutes of its last update, after the sweep writes to it, and at least every six hours. The cache is in memory, pruned to listed items and bounded by item and comment count. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BQYnPGFJx81c8JXR3o1sg1 Signed-off-by: Danathar <Danathar@users.noreply.github.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Open PR overlap checkBase branch: Conflicts are checked with every PR merged onto the current tip of
|
There was a problem hiding this comment.
Read this against the v6 tree (issue_unpark_command.go, pr_approve_command.go, pr_unpark_decision.go, label_approval.go, pr_level_hold.go). The cache is sound as far as I can verify; one open question below that I could not settle from the code.
What holds up
- Wider-than-asked cache hits are safe.
unparkReadCache.commentsreturns a full list for anysince, and an earlier-sincelist for a later one. The onlysincecaller,replyPRApproveCommand, filters each comment oncm.GetCreatedAt().Time.Before(cutoff)itself (pr_approve_command.go:25), and the full-list callers (tryUnparkIssue,tryUnparkPullRequest) passtime.Time{}. So no caller sees comments it would not have filtered anyway.storeCommentsalso refuses to overwrite a full list with a narrower one read later in the same pass, which is the right precedence. - Nothing decided is cached.
evaluateLabelApprovalFromis a pure seam: the existingevaluateLabelApprovaldelegates to it with the livelistIssueLabelEvents, and the trust/permission checks still run on every pass over the events. The 7-daycutoffcheck stays in the caller. - Self-invalidation is complete for the sweep's own writes. Every write path (
ensureUnparkNoticeCommentcreate/move/edit/delete,tryUnparkIssue,tryUnparkPullRequest,replyPRApproveCommand,replyAttentionCommands) callsforgetUnparkReadonacted || err != nil, so a partial failure also forces a fresh read next pass rather than pinning a half-updated list. - Settle window and TTL do what the header comment says.
usablerejects a read taken under 2 min afterupdated_at(so a list/comment race just after a change cannot pin stale data) and anything older than 6 h;readAtis the pass'snow, which is earlier than the real read time, so the settle check errs conservative.pruneonly touchesowner/repo#keys for the repo just listed;evictLockedis O(n) per store with n β€ 4096, fine at ~220 items/pass. - Reads outside a sweep (no scope on ctx) go straight to GitHub, so
ReleaseLevelHoldIfEligibleand other callers oflistIssueCommentsSinceare untouched.
Open question (not a finding)
The correctness of reuse rests on the premise at issue_unpark_read_cache.go header: "GitHub moves updated_at on β¦ a new, edited or deleted comment". New comments and label changes certainly bump it. I could not confirm from the tree or docs that editing or deleting an existing comment does. If it does not, a human who edits a typo into a valid /unpark-style command would wait for the 6 h TTL instead of the next pass. The backstop bounds the damage to a delay, so this is low severity either way, but it is the one assumption the tests cannot exercise against a real GitHub, and it would be worth a one-line note in the header if the author has seen it hold on the live hive.
Nothing else. The body's stated scope (reuse raw reads keyed by updated_at, no decision caching, in-memory only) matches the diff.
β hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88
Summary
The issue un-park sweep re-read the comments (and, on label-approval repos, the label timeline) of every item it watches on every pass, about every five minutes, even though almost none of those items change between passes. It is the hive's largest REST consumer on the knuckle hive. The sweep now keeps the last comment list and label timeline for each item, keyed by the item's
updated_atfrom the issue list it already reads, and reuses them until that timestamp moves. Every check that depends on the clock or live state still runs on every pass. Expected saving on knuckle is about 1,400β1,600 charged calls an hour.What went wrong
ghRateLimits.top_consumerson hosted-projectbluefin-knuckle-gjvq (build dd6b372, read at 01:45:33Z on 2026-11-08; the window is the rate-limit hour that started 00:55:20Z):hive:issue_unparkissues/{n}/commentshive:issue_unparkissues/{n}/timelineThese came from 8 sweep passes (log: pass starts 00:55:37 to 01:38:10Z, about 73 s each). From 01:44Z the budget was critical and the sweep was skipped. That makes 1,241 charged calls in about 44 minutes of sweeping, around 1,700 an hour, or ~155 per pass.
Per pass, the sweep reads:
tryUnparkIssue);evaluateLabelApproval);replyPRApproveCommand);tryUnparkPullRequest).On knuckle that is about 219 items. I rebuilt every comment and timeline event on those 219 items for 00:00β02:00Z. Only 5 of them changed at all during the measured window. So nearly every charged read fetched data the sweep had already seen. The ETag cache did not catch those reads, for two reasons:
What changes
issue_unpark_read_cache.gokeeps a per-item cache keyed by(repo, number)that holdsupdated_at, the comment list and the label events. The sweep reuses an entry only when all of these hold:updated_atis exactly the one the entry was read under;updated_at, so a list and a read that briefly disagree right after a change can't pin a stale result;tryUnparkIssue,tryUnparkPullRequest,replyPRApproveCommand, and the label-approval timeline read (evaluateLabelApprovalFrom, so other callers ofevaluateLabelApprovalare unchanged). Requests keep thehive:issue_unparkcaller tag.Why
updated_atis enough. GitHub moves an issue's or PR'supdated_aton every change these reads can see: a new or edited comment, and a label added or removed. I checked comment edits on projectbluefin. On common#1648, #1638 and #1644 the issue'supdated_atequals the newest comment edit to the second. The only timeline events the sweep uses arelabeledandunlabeled. One case I could not confirm without writing is whether deleting a comment movesupdated_at. The 6-hour backstop covers it.Expected saving
Steady state is the cache already warm when a new hour starts.
A restart still costs one full pass (~215 reads), as it does today.
How to verify after deploy
In
fleet_statusβ.data.ghRateLimits.top_consumers, thehive:issue_unparkrows forGET /repos/{owner}/{repo}/issues/{number}/commentsand.../timelineshould drop from ~880 and ~360 charged per hour to a few dozen. The first pass after the restart that deploys this still reads everything once.Files
src/pkg/github/issue_unpark_read_cache.go: the cache and the read helpers (new)src/pkg/github/issue_unpark_command.go: scope, prune, and forget on write or errorsrc/pkg/github/pr_unpark_decision.go,pr_approve_command.go,label_approval.go: the read sitessrc/pkg/github/client.go: theunparkReadsfieldchangelog.d/fixed-issue-unpark-skip-unchanged.mdTests
New in
issue_unpark_read_cache_test.go. The first four fail on v6 tip 9975c5d and pass with this change:TestUnparkSweepReusesReadsWhileUpdatedAtUnchanged: across two passes over a label-approval issue, a PR at 085 and a plain PR, the second pass makes no comment or timeline read.TestUnparkSweepRereadsWhenUpdatedAtMoves: items whoseupdated_atmoved are re-read; the unchanged one is not.TestUnparkSweepTTLBackstopRereads: a read is reused at +5 h and re-read at +7 h.TestUnparkSweepDecidesFromCachedCommentsWithCurrentState:TestUnparkSweepRereadsJustUpdatedItem: the 2-minute settle rule. This is a guard, so it passes on tip as well.issue_unpark_read_cache_internal_test.gocovers pruning and both size bounds. I also updatedTestSweepIssueUnparkCommands_CommentListSinceIsCacheablefrom #17101: it now drops the per-item cache between passes, because an unchanged item is no longer read at all and that test is about the URL a read sends.I ran
go test ./pkg/github/ ./pkg/writeback/in full,go vet ./pkg/github/andcheck-docs-citations.py, and all pass.Refs #17101
π€ Generated with Claude Code
https://claude.ai/code/session_01BQYnPGFJx81c8JXR3o1sg1