Repository navigation
Description
Target: v5.
Target branch:
v5. Open the PR with--base v5. The code paths are the same onv6(line numbers can differ), and the regular v5 β v6 top-up carries it there, so do not open a second PR forv6. Citations arepath:lineonv5@8881a15, plus the function name, so the pointers still work after the lines drift.In the PR body write
Refs #<this issue>. Do not use a closing keyword: a PR based onv5cannot close an issue that way, so a maintainer closes this one after the merge.
Executive summary
The Governor's PRs by model card ranks the AI models that wrote the hive's pull requests. In three ways it looks more certain than its data is:
- A model with no runs in the window shows
0%failure, which reads as "never fails" when nothing was measured. - The card does not say how much history it covers. "All" really means "what the hive still remembers": open PRs plus the closed PRs a bounded scan still holds.
- Rates on six PRs look as solid as rates on ninety.
The Operations page's Most effective models panel already fixed the first two for itself (#10689, #10688). This issue brings the same honesty to the Governor card and its endpoint. It also adds a confidence range to each rate.
The ranking does not change.
Problem
-
Unmeasured shown as 0%. When
effectiveness.runsis 0, the card renders PR runs, Fail and No ship as0%. The score tooltip still states the full formula,Γ (1 β fail) Γ (1 β no ship), although those factors were skipped. (The score is a Wilson lower bound of first-pass merges Γ· merged PRs, times the two run factors. A Wilson lower bound is a conservative success-rate estimate that penalises small samples. By design from π dashboard: PRs-by-model rank uses a confidence-weighted score and normalised model idsΒ #15619, a row with no runs keeps the bound as-is.) -
No coverage. Two things limit what the card can see:
- Closed PRs come from a rolling scan: 14 days by default, set by
HIVE_ATTRIBUTED_CLOSED_PR_LOOKBACK, and at most five 100-PR pages per repo. - Runs come from the live task-run log plus one rotated file.
π bug: Most effective models 30d/All silently use bounded recent historyΒ #10688 added a
coverageblock to/api/contribute/effective-modelsand a coverage note to the Operations panel./api/governor/pr-modelshas neither. - Closed PRs come from a rolling scan: 14 days by default, set by
-
No uncertainty on rates.
first_pass_merge_rate,verified_pr_run_rate,failure_rateandnothing_to_ship_rateare point values. The Wilson bound exists only insidescore.
Evidence
1. Zero runs shown as 0%. In window=30d, claude-fable-5.1 ranks #1: score 0.436, 6 merged, 5/6 first pass, runs: 0. The card shows it with 0% fail and 0% no-ship, above claude-opus-5.5 (score 0.408, 94 merged, 61/94 first pass, 186 runs, 0.5% fail, 25.3% no-ship). With window=all, where 26 runs fall inside the window, the same six Fable PRs score 0.235 and rank #2.
2. "All" is not all history.
| window | total attributed PRs |
|---|---|
7d |
125 |
30d |
308 |
all |
309 |
Only one more PR in all than in 30d. Yet GET /api/cost on the same hive reports merged_prs: 513 for PRs authored by hivecommons-hive[bot] since 2026-09-15. That count uses a different population: bot-authored only, counted by a separate cached search. It still shows that the PR history exists and the card cannot see it.
3. Small samples. claude-fable-5.1 shows first pass 83% on 6 merged PRs. gpt-6.1-terra shows 60% on 73. Nothing on the card says the first figure could plausibly sit anywhere from roughly 44% to 97% (Wilson 95%).
Root cause
src/pkg/dashboard/static-src/js/governor-view.js:157-159@8881a15(renderGovernorPRModelsTile) callspct(...)on the three run rates with no check oneff.runs. The tooltip string at:166always states the full formula.src/pkg/dashboard/api_governor_pr_models.go:24@8881a15:governorPRModelsResponsehas no coverage field.handleGovernorPRModels(:111) readsreadEffectiveModelRunsbut discards itsrotatedresult (:117).src/pkg/dashboard/contribute_effective_models.go:190@8881a15effectiveModelsCoveragecomputes exactly the needed block. It is called only fromhandleContributeEffectiveModels(:105).src/pkg/dashboard/contribute_effective_models.go:345@8881a15wilsonLowerBoundexists. There is no upper bound, and no interval on the response rows.
Wanted behavior
- Unmeasured run columns on the Governor card show "β" when
runsis 0. Use the same wording as the Operations panel: "no runs recorded, not a measured 0%". The score tooltip says the score is the first-pass bound only when there were no runs. - Coverage.
/api/governor/pr-modelsreturns acoverageobject. It has the same fields as the one on/api/contribute/effective-models(closed_pr_lookback_days,closed_prs_partial,runs_since,runs_partial), plusearliest_pr_created_at, the oldestcreated_atamong the PRs counted. The card shows a short coverage note when either*_partialis true. - Intervals. Each bucket's
reworkgainsfirst_pass_merge_rate_ci: [low, high]. Eacheffectivenessgainsverified_pr_run_rate_ci,failure_rate_ciandnothing_to_ship_rate_ci. All are Wilson 95% intervals with the same z aswilsonLowerBound. They are omitted, ornull, when the denominator is 0. The card shows them in the cell tooltip. Do not widen the table.
Acceptance criteria
- With
runs == 0, the card shows "β" in the three run columns. A row withruns > 0and a measured 0% still shows0%. -
coverageis present on every/api/governor/pr-modelsresponse. It matches what/api/contribute/effective-modelsreturns for the same window. -
earliest_pr_created_atequals the minimumcreated_atof the counted PRs. It is omitted when there are none. - The interval fields exist where the denominator is greater than 0, and satisfy
0 β€ low β€ rate β€ high β€ 1. For 5/6 they are β [0.44, 0.97]. -
score,rank,eligibleand every existing field are unchanged for the same input.
Test plan (CI decides)
src/pkg/dashboard/effective_models_rank_score_test.go:- next to
TestWilsonLowerBoundPenalisesSmallSamples, addTestWilsonIntervalBracketsRate, covering 0/0, 0/n, n/n and 5/6; - extend
TestGovernorPRModelsEffectivenessCarriesScoreAndUnattributedto assert the score is unchanged.
- next to
src/pkg/dashboard/api_governor_pr_models_test.go:- extend
TestHandleGovernorPRModelsIncludesEffectiveMetricsto checkcoverageand the interval keys; - model a coverage test on
TestHandleContributeEffectiveModelsReportsCoverage(incontribute_effective_models_test.go).
- extend
dashboard/test/governor-pr-models-ui.test.js:runs: 0renders "β" in the three columns, andruns: 5, failure_rate: 0renders0%;- the coverage note appears when
closed_prs_partialis true.
src/pkg/dashboard/governor_pr_models_ui_test.go: add the new strings to the embedded-HTML check.- Model the UI test on
TestContributeOperationsEffectiveModelsUnmeasuredRunRates, which is how π bug: Most effective models displays 0% run rates when no runs were recordedΒ #10689 tested the Operations panel.
Suggested implementation
- Add
wilsonInterval(successes, n) (low, high float64, ok bool)next towilsonLowerBound, and makewilsonLowerBoundcall it. - Add the
*_cifields withomitemptypointer or slice types, so they are absent when undefined. - In
handleGovernorPRModels, keeprotatedand calleffectiveModelsCoverage(governorPRModelsWindowDuration(window), ghpkg.AttributedClosedPRLookback(), runs, rotated, time.Now()). Computeearliest_pr_created_atinsideaggregateGovernorPRModels. governor-view.js: a helper that returns "β" for unmeasured run cells, the CI in tooltips, and the coverage note. Match the rest of the Governor tab for muted text and notes.- Document the new fields where the PRs-by-model columns are described in
src/docs/.
Rejected options
- Make zero-run rows ineligible, or change the score. That is the scoring design an owner chose in π dashboard: PRs-by-model rank uses a confidence-weighted score and normalised model idsΒ #15619. Not here.
- Raise the closed-PR lookback or keep PR history forever. Changing what "All" retains is a design decision. Operators can already raise
HIVE_ATTRIBUTED_CLOSED_PR_LOOKBACK. This issue only discloses coverage. - A
run_factors_appliedflag. It would duplicateruns > 0, which the response already carries.
Out of scope / do not change
- The score formula,
min_merged_prs(5), the ranking and the scan's lookback. - The Operations page panel, which already does this.
Dependencies
None. Other issues in this set also edit api_governor_pr_models.go and governor-view.js. Land them one at a time and rebase.
Rules that apply to every issue in this set
- Backward compatible. Existing response fields keep their names, types and meaning. Add new fields; never rename or repurpose one. Old dashboards and the admin MCP read these endpoints.
- No new GitHub API calls without a cache. Everything here is computed from data the hive already holds: the scheduler's PR snapshot, the contributor task-run log and the token collector.
- Tests.
AGENTS.mdforbids runninggo test,go buildorgo vetlocally. Write the tests, rungofmt -llocally, and let CI decide. Node UI tests underdashboard/test/run in CI too. - Changelog. Add a
changelog.d/fragment, as other dashboard PRs do. - Numbers. The evidence was pulled on 2026-11-17 between 9:10 and 9:20 AM EST, from the projectbluefin hive running a v6 edge build,
3b50113. It came through the same endpoints the dashboard reads:GET /api/governor/pr-models,GET /api/contribute/run-stats/modelsandGET /api/cost. The code paths quoted are identical on v5.
Activity
hivecommons-hive commentedon Nov 17, 2026
π Claim by agent
scannerreleased (holding agent relaunched).Claim history
scanner(agent on hive hosted-available-oke-11-placeholder-r05x)scanner(agent; holding agent relaunched)hivecommons-hive commentedon Nov 17, 2026
Hive status β updated in place; edits do not notify. Full history is in the Activity Monitor.
PR #18739 covers this issue: #18739
The PR body says this is partial/non-closing work (
Refs/Part ofor leave-open wording), so Hive will not add a closing keyword. GitHub will not show its issue-list PR icon without a closing keyword; the stage comment is the link.fix: Governor PRs-by-model honesty (no 0% for unmeasured runs, coveraβ¦
clubanderson commentedon Nov 17, 2026
Fixed by #18739 (merged to v5): run columns show a dash when a model has no runs, the card drops the skipped run factors from the score tooltip,
/api/governor/pr-modelsreturnscoverageandearliest_pr_created_at, and rates carry Wilson 95% intervals. Closing by hand because a v5-based PR cannot close the issue itself.github-actions commentedon Nov 17, 2026
@Danathar this issue was closed by @clubanderson as completed.
If this was closed by a merged PR, the fix only takes effect once it ships in a tagged release and your deployment updates to it β check your running version against the fix before concluding the fix failed.
If you are still seeing the problem, comment
/reopenand it will be reopened for you β you do not need write access. Please add what you are still observing (running version, logs, steps) so it is not closed the same way again.π± chore: top up v6 from v5 (2026-11-18) (#18929)