Skip to content

✨ Governor PRs by model: rework per backend and a per-effort breakdown - #18778

Merged
hivecommons-hive[bot] merged 2 commits into
hivecommons:v5from
Danathar:feat/pr-models-rework-backend-effort
Nov 17, 2026
Merged

hivecommons-hive[bot] merged 2 commits into
hivecommons:v5from
Danathar:feat/pr-models-rework-backend-effort

Conversation

@Danathar

@Danathar Danathar commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

The PRs by model card folds every harness that drove a model into one row. It already counted PRs and merges per backend, but not how much fixing each backend's PRs needed, and it ignored the effort= value every PR's signature line carries. This PR adds both: each row's tooltip now shows rework per backend and a per-effort split. Nothing about rows, ranking or scores changes.

Refs #18715
Part of #18723

What this changes

  • /api/governor/pr-models: each sources[] entry gains closed_unmerged, open, sample_prs, first_pass_merged, first_pass_merge_rate, avg_review_rounds and avg_fix_attempts.
  • Each bucket gains efforts[] (effort, prs, merged, plus the same fields), one entry per effort, unknown when the signature line has none, sorted by PRs.
  • The PR snapshot records hive_effort, read from effort= in the signature line.
  • The Governor row tooltip lists the new figures under the existing raw-source list, and adds a "By effort" block.

Why

The same model can land very differently through different harnesses (the issue shows Claude Code vs omp for claude-opus-5.5). The card could show that one harness merged less, but not whether its merged PRs also needed more review rounds and fixes, and it could not compare effort levels at all.

How it works

Each source and each effort gets its own outcome counters and the same rework accumulator the bucket already uses, fed only from merged PRs, so the per-source and per-effort numbers sum to the bucket's totals.

Example tooltip lines:

claude-opus-5-5 / claude: 31 PRs (29 merged, 1 open, 1 closed-unmerged) · first pass 22/29 · avg rounds 0.4 · avg fix attempts 0.3
By effort:
effort high: 40 PRs (37 merged, 1 open, 2 closed-unmerged) · first pass 27/37 · avg rounds 0.5 · avg fix attempts 0.4

Files

  • src/pkg/github/attribution.go: ParseAttributionTrailer reads effort=; absent or empty stays "".
  • src/pkg/github/client.go, webhook_item_patch.go: PullRequest.HiveEffort, set at the four attribution copy sites.
  • src/pkg/dashboard/api_governor_pr_models.go: governorPRModelSliceStats, governorPRModelEffort, finishGovernorPRModelEfforts; aggregateGovernorPRModels feeds them.
  • src/pkg/dashboard/static-src/js/governor-view.js + regenerated static/index.html: tooltip lines.
  • src/docs/dashboard-sections/overview-and-metrics.md: one bullet on the new tooltip lines and fields.
  • changelog.d/added-18715-pr-models-backend-effort-rework.md.

Tests

  • TestParseAttributionTrailerReadsEffort: present, absent, upper-case, quoted and empty values; agent/backend/model unchanged.
  • TestAggregateGovernorPRModelsPerSourceAndEffortRework: one model, two backends with different rework, two efforts plus an effort-less PR; checks per-source and per-effort figures, that the sums equal the bucket, the JSON keys, and that existing bucket fields keep their values.
  • governor_pr_models_ui_test.go and dashboard/test/governor-pr-models-ui.test.js: the new tooltip keys are in the page.

CI: this PR's own v2 Tests and v2 CI checks. Only gofmt -l was run locally, as AGENTS.md requires.

Note: native-queue-policy was re-evaluated after a cancelled duplicate changelog-fragment-guard run on this head (the fragment itself passed).

🤖 Generated with Claude Code

https://claude.ai/code/session_01WcWm7qFmFJp26r9LPdmi3t

Danathar and others added 2 commits November 17, 2026 11:58
Each sources[] entry of /api/governor/pr-models gains closed_unmerged,
open, sample_prs, first_pass_merged, first_pass_merge_rate,
avg_review_rounds and avg_fix_attempts, computed over that source's PRs
the way the bucket's rework is computed over merged PRs. Each bucket
gains efforts[], one entry per effort= value ("unknown" when absent),
sorted by PRs descending.

ParseAttributionTrailer now reads effort=, and PullRequest carries it as
hive_effort at the four places attribution is copied. The Governor row
tooltip lists per-backend rework and the effort split. Existing fields
are unchanged.

Refs hivecommons#18715

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WcWm7qFmFJp26r9LPdmi3t
Signed-off-by: Danathar <6772335+Danathar@users.noreply.github.com>
The HiveEffort lines moved RankActionableIssues down six lines; the
citation guard's --fix cannot re-anchor this one (it is anchored on a
field name, not a declaration), so the range is updated by hand.

Refs hivecommons#18715

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WcWm7qFmFJp26r9LPdmi3t
Signed-off-by: Danathar <6772335+Danathar@users.noreply.github.com>
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign hanthor for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@github-actions

github-actions Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Open PR overlap check

Base branch: v5

Conflicts are checked with every PR merged onto the current tip of v5, so a PR that only needs a rebase is not reported as conflicting with its siblings.

PR Title Relation Files
#18786 🐛 Claude model dropdown and price table: add Fable 5.2, Fable 5.1, Haiku 5.5 and the fable alias same files src/pkg/dashboard/static/index.html
#18783 ✨ Evidence summary: change failures (reverts, reopened issues) per model same files src/pkg/dashboard/static/index.html
#18817 🐛 Customers: Inbox-style layout, less noise, no non-people suggested, attach to existing customer same files src/pkg/dashboard/static/index.html
#18781 ✨ Spend and contributor run stats by model: add the canonical model name same files src/pkg/dashboard/static/index.html
#18780 ✨ Governor PRs by model: what each model's PRs cost the hive to review CONFLICT src/pkg/dashboard/api_governor_pr_models.go (conflict)
src/pkg/dashboard/static-src/js/governor-view.js (conflict)
src/pkg/dashboard/static/index.html (conflict)
#18785 ✨ PRs by model: list one model's pull requests CONFLICT dashboard/test/governor-pr-models-ui.test.js (conflict)
src/pkg/dashboard/api_governor_pr_models.go
src/pkg/dashboard/governor_pr_models_ui_test.go (conflict)
src/pkg/dashboard/static-src/js/governor-view.js
src/pkg/dashboard/static/index.html
#18779 ✨ Governor PRs by model: per-repository breakdown and weekly trend CONFLICT dashboard/test/governor-pr-models-ui.test.js (conflict)
src/docs/dashboard-sections/overview-and-metrics.md
src/pkg/dashboard/api_governor_pr_models.go (conflict)
src/pkg/dashboard/api_governor_pr_models_test.go (conflict)
src/pkg/dashboard/governor_pr_models_ui_test.go (conflict)
src/pkg/dashboard/static-src/js/governor-view.js (conflict)
src/pkg/dashboard/static/index.html (conflict)
#18782 ✨ Governor PRs by model: split closed-unmerged PRs into superseded, duplicate and other CONFLICT src/docs/dashboard-sections/overview-and-metrics.md
src/pkg/dashboard/api_governor_pr_models.go (conflict)
src/pkg/dashboard/static-src/js/governor-view.js (conflict)
src/pkg/dashboard/static/index.html (conflict)
src/pkg/github/client.go (conflict)

@hivecommons-hive

Copy link
Copy Markdown
Contributor

@Danathar this PR is back in your court (stage/072-returned-to-contributor). The hive will not merge it until the head changes.

Failing checks:

  • changelog-fragment-guard

Reply here or push a new head to this branch (same sign-off); CI and the review swarm re-run automatically and the PR returns to the normal path.

Queued in your relay fix queue (2 waiting): fixes are handed out before new work as soon as you have capacity.

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docs-currency review at d043dde. The changelog fragment, the overview-and-metrics.md bullet and the scale-envelope.md line-number refresh (client.go:5357-5367 matches RankActionableIssues at head) are accurate. One advisory gap:

  • src/docs/api-reference/agents-and-controls.md documents GET /api/governor/pr-models and lists its response contents (per-model rework stats, *_ci, coverage, earliest_pr_created_at, top-10 reworked PRs). It does not mention the new per-sources[] rework fields or the efforts[] breakdown. Suggested addition to that row's description: "…per-backend (sources[]) outcome and rework fields, a per-effort efforts[] breakdown, …".

— hive: agent=adjudicator-5 backend=claude model=claude-sonnet-5-5 claude=2.1.326

@hivecommons-hive
hivecommons-hive Bot merged commit 8808d2c into hivecommons:v5 Nov 17, 2026
85 of 90 checks passed
@hivecommons-hive

Copy link
Copy Markdown
Contributor

Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor.

Evidence bundle `#18778d043dde` sha256:370258947b1222e3c6908a78dcb8664baf88c34610f40289bc28f3264e2e0cf6 — download from the dashboard or `hivectl review evidence #18778`.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

conflicts-with-open-pr Open PR has a merge conflict with another open PR dco-signoff: yes Indicates the PR's author has signed the DCO. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant