Repository navigation
✨ Governor PRs by model: what each model's PRs cost the hive to review - #18780
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Open PR overlap check✅ Resolved: no conflicting open PRs or same-file overlaps remain against |
|
@Danathar this PR is back in your court ( Review asks:
Reply here or push a new head to this branch (same sign-off); CI and the review swarm re-run automatically and the PR returns to the normal path. Queued in your relay fix queue (13 waiting): fixes are handed out before new work as soon as you have capacity. |
Each /api/governor/pr-models bucket gains hive_cost (usd, launches, prs_with_cost, usd_per_merged_pr), summed from the PR refs' share_usd in the cached launch-cost snapshot, plus a top-level hive_cost_coverage with the snapshot window, unattributed / backend-unsupported spend and the estimate disclaimer. usd is null, not 0, when nothing was attributed; hive_cost is omitted while the snapshot is not ready. The join uses effectivePRRepo/effectivePRKey and also matches refs recorded with a short repo name. Only the cached snapshot is read. The Governor card shows the figure in each row's tooltip and a coverage note under the header. Refs hivecommons#18717 Signed-off-by: Danathar <6772335+Danathar@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WcWm7qFmFJp26r9LPdmi3t
The test lifts renderGovernorPRModelsTile out of index.html by name, so it needs governorPRModelHiveCostTip, governorPRModelsHiveCostNote and a fmtUSD stub alongside it. Adds an assertion that the hive review cost shows in the row tooltip, "—" for null, and the coverage note. Refs hivecommons#18717 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WcWm7qFmFJp26r9LPdmi3t Signed-off-by: Danathar <6772335+Danathar@users.noreply.github.com>
27b84b9 to
28e6b42
Compare
The docker (contributor/hive) jobs failed installing cline@3.0.74 because its transitive dep ai@7.0.176 was not yet resolvable on the npm registry (publish race; v5 push build failed the same way at the same time). The version now resolves; no code change needed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Danathar <6772335+Danathar@users.noreply.github.com>
| tk := effectivePRKey(repoTail(repo), pr.Number) | ||
| if strings.Contains(repo, "/") { | ||
| add(idx.full[effectivePRKey(repo, pr.Number)]) | ||
| add(idx.short[tk]) |
There was a problem hiding this comment.
Advisory: this adds owner-less refs (idx.short[tk]) to a full-repo PR without the singleOwner check that the short-PR branch applies. If two owners have a same-named repo with the same PR number among the attributed PRs, an owner-less repo#N ref is counted for both, so share_usd is double-counted. Rare. If you want to close it, count the ref for a full-repo PR only when no other full-repo PR in the set shares that repo#N.
|
Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor. Review findings from Docs-currency review at be6bcbc: the API reference, dashboard.md, token-tracking.md and changelog fragment match the implementation (null vs 0, omitted when not ready, coverage fields). One advisory, no blocker:
merged by Hive trusted-author auto-merge: author |
Keep both sides: the bucket gains HiveCost beside v5's Repos/Weeks; the row tooltip keeps the hive review cost beside v5's slice detail; both new tooltip tests stay; the docs carry the per-model PR list and the hive_cost fields together. Signed-off-by: Danathar <6772335+Danathar@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BQYnPGFJx81c8JXR3o1sg1
There was a problem hiding this comment.
Heads-up for the merger: this PR and #18782 each add the same package-level function, so the second one to merge breaks the build.
governorPRModelsCutoff(window string, now time.Time) time.Time is declared in package dashboard by both this PR (api_governor_pr_models.go:506) and #18782 (api_governor_pr_models_close_reasons.go:85), both against v5. They are different files, so git will not report a textual conflict, but whichever merges second fails to compile with governorPRModelsCutoff redeclared in this block. Verified by reading both diffs; the bodies are identical in behaviour (7d/30d/all cutoff).
Suggested fix: land one of the two first and have the other drop its copy and reuse it (this PR's copy is api_governor_pr_models.go:506). Everything else I read looked consistent with the surrounding code. These are not duplicates of each other: #18780 adds hive cost per model, #18782 splits closed-unmerged reasons.
— hive: agent=reviewer-2 backend=claude model=claude-sonnet-5-5 claude=2.1.326
|
|
||
| // governorPRModelsCutoff is the earliest PR created_at a window counts; zero | ||
| // for "all". | ||
| func governorPRModelsCutoff(window string, now time.Time) time.Time { |
There was a problem hiding this comment.
governorPRModelsCutoff(window string, now time.Time) time.Time is declared in package dashboard by both this PR (api_governor_pr_models.go:506) and #18782 (api_governor_pr_models_close_reasons.go:85), both against v5. They are different files, so git will not report a textual conflict, but whichever merges second fails to compile with governorPRModelsCutoff redeclared in this block. Rebase on whichever lands first and drop one copy.
Picks up the governor-pr-models-ui test fix so server-tests runs against a base whose test file compiles. Signed-off-by: Danathar <6772335+Danathar@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BQYnPGFJx81c8JXR3o1sg1
There was a problem hiding this comment.
Delta re-review: no style blockers; one wrapping nit. Prior governorPRModelsCutoff finding (api_governor_pr_models.go:506) untouched by delta, not confirmed fixed (#18782 still open).
Findings not on a changed line
src/pkg/dashboard/contribute_wall_canonical.go:28: Nit: rewrap this comment line like the others.// sums; abandoned, scenarios and failure_kinds (#18721) are summed too, so a // merged row says exactly what one row over all its spellings would. Model
— hive: agent=adjudicator-4 backend=claude model=claude-sonnet-5-5 claude=2.1.326
v5 now has the close-reasons split (hivecommons#18782). Keep both sides: - the bucket carries HiveCost and ClosedUnmergedReasons; - the row tooltip keeps v5's close reasons and adds the hive review cost before the per-repo/per-week detail; - governorPRModelsCutoff was defined on both sides with the same body; keep v5's copy in api_governor_pr_models_close_reasons.go; - the close-reasons test harness lifts governorPRModelHiveCostTip and fmtUSD, which the merged row renderer now calls. Signed-off-by: Danathar <6772335+Danathar@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BQYnPGFJx81c8JXR3o1sg1
Summary. The PRs by model card now shows what each model's PRs cost the hive to review and repair. Contributors pay for the model that writes a PR, but the hive pays its own reviewers, adjudicator and fixers to handle it, and that spend was invisible per model. Each model row's tooltip now shows an estimated hive cost, read from the launch-cost data the hive already keeps. Operators can see which models are expensive to review, not just which ones merge.
What this changes
GET /api/governor/pr-models: each bucket gainshive_cost:usd: sum ofshare_usdover the model's PRs. It isnull, not0, when none had attributed usage.launches: total launches that named those PRs.prs_with_cost: how many of the model's PRs had a launch record.usd_per_merged_pr:usd / merged, ornull.hive_cost_coverageholdsready,window_hours,unattributed_usd,backend_unsupported_usdand the standard estimate disclaimer.hive_costis left off every bucket,readyisfalse, and the endpoint still answers.Why
/api/costsplits spend by the hive's own agents, and/api/launch-costsplits it by PR. Nothing joined the two to the model that wrote the PR. On the projectbluefin hive the reviewer agent is about 70% of estimated spend, so the review cost of a model that needs many rounds matters.How it works
The handler reads the cached launch-cost snapshot only (no inline fallback, no token scanning, no GitHub calls). It indexes
by_refentries withkind: "pr", then goes back over the same PRs the card counted (attributed, inside the window) and adds each matched ref'sshare_usdandlaunchesto that PR's model. It usesshare_usd, notlaunch_usd, so multi-ref launches are not double-counted.The join uses the same repo normalisation as #10687 (
effectivePRRepo,effectivePRKey). A ref recorded with a short repo name (repo#5) also matchesorg/repo#5. A short PR repo with no URL matches a full ref only when exactly one owner has thatrepo#N.Example: for a model with two merged PRs whose refs have
share_usd1.50 and 0.50, the result is"hive_cost": {"usd": 2, "launches": 3, "prs_with_cost": 2, "usd_per_merged_pr": 1}.The launch-cost window is 14 days, and the PR window can be 7d, 30d or all. That is why the coverage block and the tooltip state the launch-cost window. On backends without a usage timeline, most models will show "—". That is expected and the gap is shown, not filled.
Files
src/pkg/dashboard/api_governor_pr_models_hive_cost.go(new):applyGovernorPRModelHiveCost,launchCostSnapshotForPRModels,newPRRefCostIndex,prRefCostIndex.lookupsrc/pkg/dashboard/api_governor_pr_models.go: newhive_cost/hive_cost_coveragefields, the handler call, andgovernorPRModelsCutoff(the window cutoff moved into a helper so both passes share it)src/pkg/dashboard/static-src/js/governor-view.js+ regeneratedstatic/index.html:governorPRModelHiveCostTip,governorPRModelsHiveCostNotesrc/docs/api-reference/agents-and-controls.md,src/docs/dashboard.md,src/docs/token-tracking.md: field docschangelog.d/added-18717-pr-models-hive-cost.mdTests
api_governor_pr_models_hive_cost_test.go:TestGovernorPRModelsHiveCostJoincovers sums,prs_with_cost, launches and per-merged figures.nullfor a ref with no usage and for a model with no refs.all.TestGovernorPRModelsHiveCostShortPRRepocovers a short PR repo, including the ambiguous-owner case.TestGovernorPRModelsHiveCostNotReadychecks a not-ready snapshot, both directly and through the handler, with no collector and with an uncollected collector.dashboard/test/governor-pr-models-hive-cost-ui.test.jschecks that the tooltip shows the cost and the disclaimer, that "—" shows fornull, that nothing renders when not ready, and the coverage note.gofmt -lwas run locally, as AGENTS.md requires.Refs #18717
Part of #18723
🤖 Generated with Claude Code
https://claude.ai/code/session_01WcWm7qFmFJp26r9LPdmi3t