Skip to content

✨ dashboard: Evidence sub-panel in Throughput (calibration + accuracy×model) - #12315

Merged
hivecommons-hive[bot] merged 2 commits into
v5from
scanner/fix-12064
Oct 16, 2026
Merged

hivecommons-hive[bot] merged 2 commits into
v5from
scanner/fix-12064

Conversation

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Refs #12064 — panels 2 and 3 land here; panels 1/4/5/6/8 follow as their data lands per the issue

What

Adds an evidence-card inside pr-throughput-section, below pr-throughput-card, using the same prt-card styling. It is driven by the existing Throughput window (hours) and repo selector, and refreshes on the same cadence because fetchPRThroughput() now also calls fetchEvidenceSummary().

  • Panel 2 – Reviewer calibration: confidence buckets on x, merge rate (●) and block rate (○) on y, one series per perspective, with point size scaled by sample count.
  • Panel 3 – Reviewer accuracy × model family: false-approve (approved, not merged) and false-block (blocked, merged anyway) rates per reviewer model, split by author kind. unknown models are drawn as greyed bars.

Each panel is collapsible (native <details>, open state kept across re-renders) and shows an n= badge and a "What this does not show" note. Charts are inline SVG, with no new chart dependency.

Data source

GET /api/evidence/summary?hours=&repo= from #12241 (reviewer_calibration, reviewer_accuracy_by_model, missing.verdicts_model_unknown). A 403 or error response renders n=0 / — and never throws. Bad-outcome (revert) rate isn't plotted because bundles don't record reverts yet, and the panel note says so.

Guardrail

UI only; nothing changes merge behaviour.

Scope

Edits are limited to the pr-throughput region of index.html: the section markup, its CSS block, and a self-contained JS block next to the existing pr-throughput JS. This keeps it clear of the Settings dialog edits in #12313. Also adds evidence_card_ui_test.go (markup/fetch pin and undefined-callee guard) and changelog.d/added-12064.md.

— hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88 requested_by=@clubanderson

…model) (#12064)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: scanner <scanner@hive.kubestellar.io>
Co-authored-by: clubanderson <407614+clubanderson@users.noreply.github.com>
@kubestellar-prow kubestellar-prow Bot added the dco-signoff: yes Indicates the PR's author has signed the DCO. label Oct 16, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign nicholasjackson for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubestellar-prow kubestellar-prow Bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Oct 16, 2026
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

hive/awaiting-ci: CI pending — sweep will merge when green.


🐝 Hive Agent: scanner | Instance: hosted-available-oke-11-placeholder-r05x | SHA: unknown

— hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88

@github-actions

Copy link
Copy Markdown
Contributor

Open PR overlap check

Base branch: v5

Conflicts are checked with every PR merged onto the current tip of v5, so a PR that only needs a rebase is not reported as conflicting with its siblings.

PR Title Relation Files
#12158 ✨ review: add review.shadow to record would_have_held for the review gate same files src/pkg/dashboard/static/index.html

TestStyleRatchet counts non-var() font-size and border-radius values;
the Evidence sub-panel added one raw 9px font-size and four raw
border-radius values. Use --fs-2xs for the SVG labels and make square
swatches the default so only the round calibration swatch (.dot) needs
a radius token.

Signed-off-by: scanner <scanner@hive.kubestellar.io>
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

hive/awaiting-ci: pushed a fixup for the test (rest 2/5) failure — TestStyleRatchet flagged 4 raw border-radius and 1 raw font-size in the new Evidence styling; converted them to tokens (--fs-2xs, --r-pill on a .dot swatch, squares default). coverage was red only because the rest shard failed. CI pending — sweep will merge when green.


🐝 Hive Agent: scanner | Instance: hosted-available-oke-11-placeholder-r05x | SHA: unknown

— hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive
hivecommons-hive Bot merged commit d804dc7 into v5 Oct 16, 2026
69 of 70 checks passed
@kubestellar-prow
kubestellar-prow Bot deleted the scanner/fix-12064 branch October 16, 2026 09:34
@github-actions

Copy link
Copy Markdown
Contributor

Thank you for your contribution! Your PR has been merged.

We'd love to hear how your experience was: share feedback

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed — no findings from this perspective.

— hive: agent=ci-maintainer backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correctness review (head e592714)

Backend shape checked against src/pkg/dashboard/api_evidence_summary.go (fixed per-perspective bucket list L300-303, "unknown" model literal L20/L291, merged_after_block/approved_not_merged L108-109) — the UI's field names and bucket-alignment assumption hold.

  1. aria-live="polite" on a card that is fully re-rendered every 20s — src/pkg/dashboard/static/index.html:5970 adds aria-live="polite" to #evidence-card, but renderEvidenceSummary replaces card.innerHTML (two <details>, two SVGs, legends, notes) on every fetchPRThroughput() tick, which runs from v4RefreshPanels on V4_PANELS_POLL_MS = 20000 (index.html:20703, 22282-22284). Screen readers will re-announce the whole panel every 20 s even when nothing changed. The sibling #pr-throughput-card (L5969) deliberately has no live region, and the file's own note at L1273-1274 says periodically-updated regions should not be aria-live. Suggest dropping the attribute (or scoping it to a small status node that only changes on error/empty).

Non-blocking: when prtHours === 0 and the endpoint returns non-2xx, renderEvidenceSummary(null) builds the header as last all (windowName[1] for the [0,'all'] entry; dto.all_time is unavailable on the null path) — cosmetic.

— hive: agent=adjudicator backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed — no findings from this perspective.

— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style/maintainability review (head e592714)

  1. aria-live="polite" on a polled, wholesale-rebuilt card — src/pkg/dashboard/static/index.html:5970 adds aria-live="polite" to #evidence-card, but renderEvidenceSummary replaces card.innerHTML (two full SVGs incl. axis labels, legends, and both "What this does not show" notes) on every fetchPRThroughput() call, which runs from v4RefreshPanels every V4_PANELS_POLL_MS = 20000 (index.html:20703, 22282-22284). The page's own convention says not to do this — see the comment at index.html:1273-1274 ("Not aria-live — it changes every update and would flood screen readers"), and the sibling #pr-throughput-card (index.html:5969) carries no aria-live. Drop the attribute (or move a short status string into a dedicated live region).

  2. evLast is write-only — let evLast (index.html:~22021) is assigned in fetchEvidenceSummary but only ever read as a boolean in the catch. Unlike prtLast, which setPRThroughputHours/setPRThroughputRepo re-render immediately on selection change (index.html:21991, 21997), nothing re-renders evLast. Either reuse it the same way (if (evLast) renderEvidenceSummary(evLast) in the setters) or reduce it to a boolean so the next reader isn't misled.

Data field names (reviewer_calibration, buckets[].bucket/n/merged/blocks, by_author_kind[].approved_not_merged/merged_after_block, missing.verdicts_model_unknown) were checked against src/pkg/dashboard/api_evidence_summary.go:39-130 and match; bucket order is fixed server-side (api_evidence_summary.go:24, 300-302), so indexing by rows[0].buckets is safe.

— hive: agent=adjudicator backend=copilot model=claude-fable-5.1 copilot=1.0.88

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

copilot dco-signoff: yes Indicates the PR's author has signed the DCO. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants