Repository navigation
π fix(dashboard): enrich soft-linked stage-tracker issues with title and state - #14630
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Open PR overlap checkBase branch: Conflicts are checked with every PR merged onto the current tip of
|
|
hive/awaiting-ci: CI pending β sweep will merge when green. π Hive Agent: β hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88 |
There was a problem hiding this comment.
Conclusion: the enrichment does what #14606 asks and the test covers the dedupe/PR/404 cases, but it moves live GitHub reads into the /api/stage-tracker request path and the "cached for the tracker TTL" premise does not hold β I'd want the lookups memoised across rebuilds before this lands.
What I read: src/pkg/dashboard/stage_tracker.go at c21cbbb (call site 1434, stageTrackerEnrichSoftLinks 1606β1645), invalidateStageTrackerCache 1465, server.go:2638β2641, src/pkg/github/advisory_work.go:53.
-
Lookups repeat on every status publish, not every TTL (medium) β
server.go:2641callsinvalidateStageTrackerCache()on each status publish and then nudges the SPA to refetch, sostageTrackerPayloadrebuilds on the next GET regardless ofStageTrackerCacheTTL. Thegotmap instageTrackerEnrichSoftLinks(stage_tracker.go:1615) is local to one call, so every rebuild re-issues oneissues GETper distinct soft-linked number β titles/states that effectively never change are fetched once per eval cycle for the life of the PR. The PR body's "results are cached for the tracker TTL" is true only when no publish intervenes, which is not the normal cadence. A per-Servermemo keyed byowner/repo#number(with its own TTL or invalidated by the webhook path) would make the cost one GET per issue instead of one per cycle. -
Serial network calls under
stageTrackerCacheMu, in the HTTP handler (medium) βstageTrackerPayloadholds the mutex for the whole build (stage_tracker.go:1425β1426), and the enrichment now does N sequentialIssues.Getcalls inside it. Every concurrent/api/stage-trackerGET blocks behind those round-trips, and the first request after each publish pays the full latency. Previously the build was pure CPU over the status payload. If the memo in (1) lands, this mostly collapses to the cold case, but it is worth stating since the handler was designed as "never reaches GitHub". -
Stale comment (low, docs-currency) β
stage_tracker.go:1402β1405still saysstageTrackerRESTCallertags requests "listed with Charged 0: the handler never reaches GitHub"; after this change the same caller tag also labels real charged GETs. Update the comment so the rate-limit panel reads right.
Minor / not blocking: context.Background() at stage_tracker.go:1633 is bounded only by the client's HTTP timeout; passing a derived context with a short deadline (or the request ctx) would keep one slow GitHub call from stalling the whole tracker build.
I have not approved or requested changes as a review event; this is a comment for the maintainer and the scanner lane to act on.
β hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88
There was a problem hiding this comment.
Read the diff against the v5 tree (stage_tracker.go, server.go, etagcache.go, slowstart.go, app.go) and built ./pkg/github ./pkg/dashboard at c21cbbb β compiles clean. The enrichment logic itself is correct: soft-only filter at stage_tracker.go:1625, per-rebuild dedupe via got, failed lookups and PR numbers left untouched, and the test covers each of those paths. One thing a maintainer should weigh before this lands:
The issue GETs run serially, with no deadline, while stageTrackerCacheMu is held (medium).
stageTrackerPayloadtakess.stageTrackerCacheMu.Lock()withdefer Unlock()(stage_tracker.go:1426-1427) and then callsstageTrackerEnrichSoftLinks(:1434), which issues oneIssues.Getper distinct soft-linked number usingcontext.Background()(:1633). The only bound is the App HTTP client's 30s timeout (github/app.go:720), per request.UpdateStatusIfFreshcallsinvalidateStageTrackerCache()on every status publish (server.go:2641), which also takesstageTrackerCacheMu(stage_tracker.go:1466). So while a rebuild is fetching N issues, the status publisher β and every concurrentGET /api/stage-trackerβ blocks behind it. Under the slow-start caution window (slowstart.go:40-46, ~2β3s per request, globally serialized) a tracker with a dozen soft-linked issues holds that mutex for 30s+; a hung GitHub connection holds it for 30s Γ N.- Before this change nothing network-bound ran under that lock, so this is a new stall path rather than a widening of an existing one.
Suggestions, either of which would settle it: wrap the lookups in a context.WithTimeout (the repo already does this for dashboard-initiated GitHub calls, e.g. api_acmm_eval.go:280), and/or cap the number of lookups per rebuild. Moving enrichment outside the mutex would be cleaner still but is a bigger change.
Small body/code gap, for accuracy: the body says results are "cached for the tracker TTL", but server.go:2641 drops the cache on every status publish, so the GETs re-run each eval cycle, not once per TTL. The ETag transport (etagcache.go:130-134) makes the repeats conditional, so quota is fine β this only matters for how often the lock-hold above recurs.
No security, style or docs concerns; the changelog.d/fixed-14606.md fragment is correctly named.
β hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88
|
hive/awaiting-ci: pushed review fixes (memoised, time-bounded soft-link lookups) plus the #14667 soft-link binding fix on this branch. CI pending β sweep will merge when green. π Hive Agent: β hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88 |
|
hive/awaiting-ci: resynced api-reference citations (both red checks shared that drift); CI pending β sweep will merge when green. π Hive Agent: β hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88 |
|
hive/awaiting-ci: merged origin/v5 to clear the stage_tracker_test.go conflict (both sides added independent tests; kept both). CI pending β sweep will merge when green. π Hive Agent: β hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88 |
|
βΈοΈ CI is red on the base branch (relative links and anchors resolve); holding until it recovers β this is not a failure in this PR, so no fix attempt is counted and no one needs to act here. |
β¦and state Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com> Co-authored-by: clubanderson <407614+clubanderson@users.noreply.github.com>
β¦binding uncovered siblings or closed PRs Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com> Co-authored-by: clubanderson <407614+clubanderson@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
da7f1a1 to
9aed941
Compare
|
Rebased onto v5 to clear the conflict in src/pkg/dashboard/stage_tracker.go, stage_tracker_test.go, server.go, src/docs/api-reference.md. hive/awaiting-ci: CI pending β sweep will merge when green. π Hive Agent: β hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88 |
There was a problem hiding this comment.
Read the diff against the v5 tree (stage_tracker.go, advisory_work.go, pr_reference_links.go, callers in stageTrackerPayloadSource) and built pkg/dashboard + pkg/github locally. The code looks correct to me; two things a maintainer should know before merging.
The red check is not this PR's fault. relative links and anchors resolve fails on DRIFT src/docs/scale-envelope.md:37 β client.go:4676-4686 lacks anchor AgeMinutes. This branch's merge-base is c6d82d0; v5 re-anchored that citation in #14712 (d62ef97de). Running python3 src/scripts/check-docs-citations.py src/docs on the PR head reports 1 drift, on v5 0. A merge of v5 clears it; nothing in this diff touches client.go or scale-envelope.md.
Stale prose about the mutex (comment + body), behaviour is fine. stage_tracker.go:1858 says the GETs run "as it holds stageTrackerCacheMu", and the body says the lookups "stay inside the mutex: moving them out would need a rebuild-generation guard". After #14681 that is no longer how the build runs: stageTrackerPayloadSource releases stageTrackerCacheMu before encodeStageTracker (stage_tracker.go:1646β1650) and already has the generation guard (built.stale = s.stageTrackerGen != gen, :1654). So the β€3 s cold-cache GETs block only the one building request, not the publisher or other /api/stage-tracker readers β better than the body claims. Worth rewording the :1858 comment so the next reader does not go looking for a lock that isn't held.
Minor, no action needed:
stageTrackerLinkMemo.put(:1897) only evicts expired entries at the 1024 cap, so the map can exceed it if nothing has aged out. Practically bounded by distinct soft-link numbers in the tracker; noting it only because the constant reads as a hard cap.- The body's first paragraph says PR-numbered links are "left untouched"; the code drops them (
:1954), which the## Updatesection andfixed-14667.mddescribe correctly.
Checked and fine: issueRefPattern (prclaims.go:638) has the two groups prMentionUncovered's index math assumes; deps.GHClient is *ghpkg.Client (deps.go:47) so the new IssueTitleState resolves; TestStageTrackerEnrichSoftLinks exercises the GET path through httptest including the is-PR drop.
β hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88
|
Thank you for your contribution! Your PR has been merged. We'd love to hear how your experience was: share feedback |
Stage-tracker PR rows whose only linked issue is a soft
refs/title/mentionlink carried just a number, so the UI fell through to "(title unavailable)" even for real closed issues. After #14579 the URL is right but the title/state are still missing.The tracker now does one issues GET per distinct same-repo number for such links and fills in title and state. It runs only on a tracker rebuild, so results are cached for the tracker TTL. Failed lookups, and numbers that are PRs, leave the link untouched.
Closes #14606
β hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88 requested_by=@clubanderson
Update: review fixes and #14667
Why the review changes. Both reviews pointed out that "cached for the tracker TTL" did not hold:
UpdateStatusIfFreshdrops the tracker cache on every status publish, so the issues GETs re-ran every eval cycle. They also ran serially withcontext.Background()whilestageTrackerCacheMuwas held, so a slow GitHub could stall the status publisher and every/api/stage-trackerGET. Changes:Serverkeyed byowner/repo#number. Results are kept for 15 min, and failed lookups for 2 min, so persistent 404s don't use up the budget on every rebuild.context.WithTimeoutof 3 s (same pattern asapi_acmm_eval.go). Hitting the budget is not memoised; the next rebuild retries. The lookups stay inside the mutex: moving them out would need a rebuild-generation guard against the publish-time invalidation. With the memo and the bound, the lock hold is now cold-cache only and at most 3 s.stageTrackerRESTCallercomment no longer claims the handler never reaches GitHub.#14667.
remain(s),remaining,left open/for/to,not covered,out of scope,not in this PR). It also no longer binds when it sits inside a parenthetical qualifying the reference before it. SoPart of #14550 (tracker; #14559 backend-specific routes remain)no longer draws [scanner] repeat-failure: backend-specific routes (copilot, omp, openhands, muse, agy, aider) and real-run evidenceΒ #14559 as covered by the PR (github.PRReferencedIssues).Closes #14667