Skip to content

✨ Expose the knowledge advisor rank-versus-holdout summary through a read-only route and Advisor card - #18904

Merged
kubestellar-prow[bot] merged 3 commits into
v5from
feat/advisor-summary-route-18897
Nov 18, 2026
Merged

kubestellar-prow[bot] merged 3 commits into
v5from
feat/advisor-summary-route-18897

Conversation

@clubanderson

Copy link
Copy Markdown
Member

Fixes #18897

Follow-up to the audit of #17054 (rank-versus-holdout comparison).

What

  • GET /api/knowledge/advisor/summary[?since=<RFC3339>], read role or higher, read-only. Returns the decision window, counts by mode / arm / outcome, the Lane.AdvisorArmSummary() per-arm bead counts and means (mixed and none under their own keys), mean model latency and prompt/completion tokens where records carry them, the advisor settings (mode, holdout_percent, model, call budget; never endpoint or key) and a small-n caveat (fewer than 30 closed beads in deliver or holdout). Counts and means only: no slugs, refs, agents or content.
  • retro.AdvisorDecisionStore.Summarize; decision records gain optional latency_ms, prompt_tokens, completion_tokens (omitempty, written by the ranking decorator going forward; older records report zero samples).
  • Advisor card in the Knowledge panel (static-src knowledge.js/css; static/index.html regenerated by concatenating static-src in manifest order, verified byte-identical before the edit).
  • api-reference row, retro-lane.md section, changelog.d fragment.

Notes

  • Per-arm bead means are not narrowed by since (AdvisorArmSummary has no window); the response says so.
  • No GitHub writes added.
  • Tests: empty store, mixed arms equal to a real Lane.AdvisorArmSummary(), small-n threshold, since filter, role gating, no-leak check, store Summarize, node test for the card renderer.

🤖 Generated with Claude Code

@kubestellar-prow kubestellar-prow Bot added dco-signoff: yes Indicates the PR's author has signed the DCO. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Nov 18, 2026
@github-actions

github-actions Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Open PR overlap check

Base branch: v5

Conflicts are checked with every PR merged onto the current tip of v5, so a PR that only needs a rebase is not reported as conflicting with its siblings.

PR Title Relation Files
#18893 🐛 Hold decisions on a PR whose CI or review is still running on its current head same files src/docs/api-reference/dashboard-misc.md
src/docs/api-reference/packs-and-acmm.md
src/pkg/dashboard/static/index.html
#18912 🐛 Model/method switch for stopped or paused agents; keep page in place after a switch same files src/pkg/dashboard/static/index.html
#18873 🐛 needs-sha: exempt owner/detailed issues, overrideable waiver, visible hold on the row CONFLICT dashboard/openapi.json
src/docs/api-reference/agents-and-controls.md
src/docs/api-reference/auth.md (conflict)
src/docs/api-reference/beads.md
src/docs/api-reference/configuration.md (conflict)
src/docs/api-reference/dashboard-misc.md (conflict)
src/docs/api-reference/inception.md
src/docs/api-reference/knowledge.md (conflict)
src/docs/api-reference/nous.md
src/docs/api-reference/packs-and-acmm.md (conflict)
src/pkg/dashboard/api.go
src/pkg/dashboard/static/index.html
#18908 🐛 fix: preserve agent position when switching methods same files src/pkg/dashboard/static/index.html
#18917 🐛 Label the age, session and staleness of a replayed hand-off decline same files dashboard/openapi.json
src/docs/api-reference/configuration.md
src/pkg/dashboard/static/index.html
#18913 🐛 Contributor lane: grace window before claiming owner/hive-filed issues same files src/cmd/hive/main.go
#18920 ✨ Betty: meeting-budget meter, 80/90 percent steps, and no silently dropped questions same files dashboard/openapi.json
src/docs/api-reference/knowledge.md
src/pkg/dashboard/static/index.html
#18918 ✨ Does this build contain PR #N: ancestry lookup API on the hub and each hive same files dashboard/openapi.json
src/docs/api-reference/health-status-events.md
src/pkg/dashboard/api.go

@hivecommons-hive

hivecommons-hive Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor.

clubanderson this PR is held at stage/071-needs-author and the hive will not merge it yet.

Failing checks:

  • test (dashboard 3/4)
  • test (dashboard shuffle 2/3)

Next: the hive does not push to your branch. Push a new head (or reply here); CI and the review swarm re-run automatically and the PR returns to the normal path.

… route + Advisor card)

Adds GET /api/knowledge/advisor/summary and a Knowledge-section Advisor card:
decision counts by mode/arm/outcome, Lane.AdvisorArmSummary() per-arm bead
means (mixed/none separate), mean latency and tokens, advisor settings and a
small-n caveat. Counts and means only. Decision records now carry latency and
token counts.

Signed-off-by: Andrew Anderson <andy@clubanderson.com>
Signed-off-by: Andrew Anderson <andy@clubanderson.com>
…as a Settings UI reference

Signed-off-by: Andrew Anderson <andy@clubanderson.com>
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

Approval requirements bypassed by manually added approval.

This pull-request has been approved by:

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. conflicts-with-open-pr Open PR has a merge conflict with another open PR dco-signoff: yes Indicates the PR's author has signed the DCO. lgtm Indicates that a PR is ready to be merged. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

✨ Expose the knowledge advisor rank-versus-holdout summary through a read-only dashboard route

1 participant