Skip to content

✨ Governor PRs by model: what each model's PRs cost the hive to review - #18780

Merged
hivecommons-hive[bot] merged 6 commits into
hivecommons:v5from
Danathar:feat/pr-models-review-cost
Nov 18, 2026
Merged

hivecommons-hive[bot] merged 6 commits into
hivecommons:v5from
Danathar:feat/pr-models-review-cost

Conversation

@Danathar

Copy link
Copy Markdown
Contributor

Summary. The PRs by model card now shows what each model's PRs cost the hive to review and repair. Contributors pay for the model that writes a PR, but the hive pays its own reviewers, adjudicator and fixers to handle it, and that spend was invisible per model. Each model row's tooltip now shows an estimated hive cost, read from the launch-cost data the hive already keeps. Operators can see which models are expensive to review, not just which ones merge.

What this changes

  • GET /api/governor/pr-models: each bucket gains hive_cost:
    • usd: sum of share_usd over the model's PRs. It is null, not 0, when none had attributed usage.
    • launches: total launches that named those PRs.
    • prs_with_cost: how many of the model's PRs had a launch record.
    • usd_per_merged_pr: usd / merged, or null.
  • A new top-level hive_cost_coverage holds ready, window_hours, unattributed_usd, backend_unsupported_usd and the standard estimate disclaimer.
  • When the launch-cost snapshot is not ready, or there is no collector, hive_cost is left off every bucket, ready is false, and the endpoint still answers.
  • Dashboard: each row's tooltip gets a "Hive review cost" line. A note under the header gives the launch-cost window and the spend that could not be tied to any PR.
  • Existing fields are unchanged.

Why

/api/cost splits spend by the hive's own agents, and /api/launch-cost splits it by PR. Nothing joined the two to the model that wrote the PR. On the projectbluefin hive the reviewer agent is about 70% of estimated spend, so the review cost of a model that needs many rounds matters.

How it works

The handler reads the cached launch-cost snapshot only (no inline fallback, no token scanning, no GitHub calls). It indexes by_ref entries with kind: "pr", then goes back over the same PRs the card counted (attributed, inside the window) and adds each matched ref's share_usd and launches to that PR's model. It uses share_usd, not launch_usd, so multi-ref launches are not double-counted.

The join uses the same repo normalisation as #10687 (effectivePRRepo, effectivePRKey). A ref recorded with a short repo name (repo#5) also matches org/repo#5. A short PR repo with no URL matches a full ref only when exactly one owner has that repo#N.

Example: for a model with two merged PRs whose refs have share_usd 1.50 and 0.50, the result is "hive_cost": {"usd": 2, "launches": 3, "prs_with_cost": 2, "usd_per_merged_pr": 1}.

The launch-cost window is 14 days, and the PR window can be 7d, 30d or all. That is why the coverage block and the tooltip state the launch-cost window. On backends without a usage timeline, most models will show "—". That is expected and the gap is shown, not filled.

Files

  • src/pkg/dashboard/api_governor_pr_models_hive_cost.go (new): applyGovernorPRModelHiveCost, launchCostSnapshotForPRModels, newPRRefCostIndex, prRefCostIndex.lookup
  • src/pkg/dashboard/api_governor_pr_models.go: new hive_cost / hive_cost_coverage fields, the handler call, and governorPRModelsCutoff (the window cutoff moved into a helper so both passes share it)
  • src/pkg/dashboard/static-src/js/governor-view.js + regenerated static/index.html: governorPRModelHiveCostTip, governorPRModelsHiveCostNote
  • src/docs/api-reference/agents-and-controls.md, src/docs/dashboard.md, src/docs/token-tracking.md: field docs
  • changelog.d/added-18717-pr-models-hive-cost.md

Tests

  • api_governor_pr_models_hive_cost_test.go:
    • TestGovernorPRModelsHiveCostJoin covers sums, prs_with_cost, launches and per-merged figures.
      • It checks null for a ref with no usage and for a model with no refs.
      • It checks that a ref for a PR outside the 30d window is excluded, but counted under all.
      • It checks that an issue-kind ref is ignored, and that short/full repo names join.
    • TestGovernorPRModelsHiveCostShortPRRepo covers a short PR repo, including the ambiguous-owner case.
    • TestGovernorPRModelsHiveCostNotReady checks a not-ready snapshot, both directly and through the handler, with no collector and with an uncollected collector.
  • dashboard/test/governor-pr-models-hive-cost-ui.test.js checks that the tooltip shows the cost and the disclaimer, that "—" shows for null, that nothing renders when not ready, and the coverage note.
  • CI: this PR's own v2 Tests and v2 CI checks. Only gofmt -l was run locally, as AGENTS.md requires.

Refs #18717
Part of #18723

🤖 Generated with Claude Code

https://claude.ai/code/session_01WcWm7qFmFJp26r9LPdmi3t

@kubestellar-prow kubestellar-prow Bot added the dco-signoff: yes Indicates the PR's author has signed the DCO. label Nov 17, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign danathar for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@github-actions

github-actions Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Open PR overlap check

✅ Resolved: no conflicting open PRs or same-file overlaps remain against v5.

@hivecommons-hive

hivecommons-hive Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

@Danathar this PR is back in your court (stage/072-returned-to-contributor). The hive will not merge it until the head changes.

Review asks:

  • the branch conflicts with the base and needs a rebase

Reply here or push a new head to this branch (same sign-off); CI and the review swarm re-run automatically and the PR returns to the normal path.

Queued in your relay fix queue (13 waiting): fixes are handed out before new work as soon as you have capacity.

Danathar and others added 2 commits November 17, 2026 20:49
Each /api/governor/pr-models bucket gains hive_cost (usd, launches,
prs_with_cost, usd_per_merged_pr), summed from the PR refs' share_usd in
the cached launch-cost snapshot, plus a top-level hive_cost_coverage with
the snapshot window, unattributed / backend-unsupported spend and the
estimate disclaimer. usd is null, not 0, when nothing was attributed;
hive_cost is omitted while the snapshot is not ready. The join uses
effectivePRRepo/effectivePRKey and also matches refs recorded with a
short repo name. Only the cached snapshot is read.

The Governor card shows the figure in each row's tooltip and a coverage
note under the header.

Refs hivecommons#18717

Signed-off-by: Danathar <6772335+Danathar@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WcWm7qFmFJp26r9LPdmi3t
The test lifts renderGovernorPRModelsTile out of index.html by name, so it
needs governorPRModelHiveCostTip, governorPRModelsHiveCostNote and a fmtUSD
stub alongside it. Adds an assertion that the hive review cost shows in the
row tooltip, "—" for null, and the coverage note.

Refs hivecommons#18717

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WcWm7qFmFJp26r9LPdmi3t
Signed-off-by: Danathar <6772335+Danathar@users.noreply.github.com>
@Danathar
Danathar force-pushed the feat/pr-models-review-cost branch from 27b84b9 to 28e6b42 Compare November 17, 2026 20:52
@github-actions github-actions Bot added the conflicts-with-open-pr Open PR has a merge conflict with another open PR label Nov 17, 2026
The docker (contributor/hive) jobs failed installing cline@3.0.74
because its transitive dep ai@7.0.176 was not yet resolvable on the
npm registry (publish race; v5 push build failed the same way at the
same time). The version now resolves; no code change needed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Danathar <6772335+Danathar@users.noreply.github.com>

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correctness review of #18780 at be6bcbc. The join, window cutoff and null handling check out. One rare, advisory double-count edge case is in the inline comment.

— hive: agent=adjudicator backend=claude model=claude-sonnet-5-5 claude=2.1.326

tk := effectivePRKey(repoTail(repo), pr.Number)
if strings.Contains(repo, "/") {
add(idx.full[effectivePRKey(repo, pr.Number)])
add(idx.short[tk])

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Advisory: this adds owner-less refs (idx.short[tk]) to a full-repo PR without the singleOwner check that the short-PR branch applies. If two owners have a same-named repo with the same PR number among the attributed PRs, an owner-less repo#N ref is counted for both, so share_usd is double-counted. Rare. If you want to close it, count the ref for a full-repo PR only when no other full-repo PR in the set shares that repo#N.

@hivecommons-hive

hivecommons-hive Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor.

Review findings from adjudicator-5 (folded here because this head already carries its hive review):

Docs-currency review at be6bcbc: the API reference, dashboard.md, token-tracking.md and changelog fragment match the implementation (null vs 0, omitted when not ready, coverage fields). One advisory, no blocker:

  • src/docs/operator-reference/sweeps-and-operations.md:917 still describes the PRs by model panel and says the endpoint is served "from the same cached PR snapshot". It now also reads the launch-cost snapshot and shows Hive review cost. Suggested append:
... from the same cached PR snapshot as the model outcome counts. Each row's tooltip also shows an estimated Hive review cost joined from the cached launch-cost snapshot (see token-tracking.md#launch-attribution).
Evidence bundle `#18780babecbe` sha256:15cb173867dafae4d3f8204e80d8a00adaf89ae9392d683ed70baff17727a0d3 — download from the dashboard or `hivectl review evidence #18780`.

merged by Hive trusted-author auto-merge: author Danathar holds role owner and GitHub permission maintain on repo; CI green at babecbe.

@github-actions github-actions Bot removed the conflicts-with-open-pr Open PR has a merge conflict with another open PR label Nov 17, 2026
Keep both sides: the bucket gains HiveCost beside v5's Repos/Weeks;
the row tooltip keeps the hive review cost beside v5's slice detail;
both new tooltip tests stay; the docs carry the per-model PR list and
the hive_cost fields together.

Signed-off-by: Danathar <6772335+Danathar@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BQYnPGFJx81c8JXR3o1sg1

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Heads-up for the merger: this PR and #18782 each add the same package-level function, so the second one to merge breaks the build.

governorPRModelsCutoff(window string, now time.Time) time.Time is declared in package dashboard by both this PR (api_governor_pr_models.go:506) and #18782 (api_governor_pr_models_close_reasons.go:85), both against v5. They are different files, so git will not report a textual conflict, but whichever merges second fails to compile with governorPRModelsCutoff redeclared in this block. Verified by reading both diffs; the bodies are identical in behaviour (7d/30d/all cutoff).

Suggested fix: land one of the two first and have the other drop its copy and reuse it (this PR's copy is api_governor_pr_models.go:506). Everything else I read looked consistent with the surrounding code. These are not duplicates of each other: #18780 adds hive cost per model, #18782 splits closed-unmerged reasons.

— hive: agent=reviewer-2 backend=claude model=claude-sonnet-5-5 claude=2.1.326


// governorPRModelsCutoff is the earliest PR created_at a window counts; zero
// for "all".
func governorPRModelsCutoff(window string, now time.Time) time.Time {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

governorPRModelsCutoff(window string, now time.Time) time.Time is declared in package dashboard by both this PR (api_governor_pr_models.go:506) and #18782 (api_governor_pr_models_close_reasons.go:85), both against v5. They are different files, so git will not report a textual conflict, but whichever merges second fails to compile with governorPRModelsCutoff redeclared in this block. Rebase on whichever lands first and drop one copy.

Picks up the governor-pr-models-ui test fix so server-tests runs
against a base whose test file compiles.

Signed-off-by: Danathar <6772335+Danathar@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BQYnPGFJx81c8JXR3o1sg1

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Delta re-review: no style blockers; one wrapping nit. Prior governorPRModelsCutoff finding (api_governor_pr_models.go:506) untouched by delta, not confirmed fixed (#18782 still open).

Findings not on a changed line

  • src/pkg/dashboard/contribute_wall_canonical.go:28: Nit: rewrap this comment line like the others.
    // sums; abandoned, scenarios and failure_kinds (#18721) are summed too, so a
    // merged row says exactly what one row over all its spellings would. Model
    

— hive: agent=adjudicator-4 backend=claude model=claude-sonnet-5-5 claude=2.1.326

@github-actions github-actions Bot removed the conflicts-with-open-pr Open PR has a merge conflict with another open PR label Nov 18, 2026
v5 now has the close-reasons split (hivecommons#18782). Keep both sides:
- the bucket carries HiveCost and ClosedUnmergedReasons;
- the row tooltip keeps v5's close reasons and adds the hive review cost
  before the per-repo/per-week detail;
- governorPRModelsCutoff was defined on both sides with the same body;
  keep v5's copy in api_governor_pr_models_close_reasons.go;
- the close-reasons test harness lifts governorPRModelHiveCostTip and
  fmtUSD, which the merged row renderer now calls.

Signed-off-by: Danathar <6772335+Danathar@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BQYnPGFJx81c8JXR3o1sg1
@hivecommons-hive
hivecommons-hive Bot merged commit 92923a9 into hivecommons:v5 Nov 18, 2026
74 of 75 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dco-signoff: yes Indicates the PR's author has signed the DCO. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant