You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
✨ retro: record the knowledge-advisor arm per bead for the holdout outcome comparison (advisor phase 2, slice 4) #17156
Part of #17054 (approved direction: #17054 (comment)). RFC: docs/rfc-17054-llm-ranked-knowledge-advisor.md § "Measurement". Land on v5.
Readiness: needs the ranking decorator (#17155) merged, so decisions exist to attribute.
Why
The approved measurement is a holdout comparison using counters the retro lane already reconstructs (RetroRecord in src/pkg/retro/retro.go: KicksReceived, CIFailureCount, FixAttempts, drift pauses). Adding each bead's advisor arm to that record answers "did delivered advice help" without a new model call, a "did this help" line, or searching PRs for slugs — all three were rejected in the RFC and the owner decision.
Source of truth: the decision records written by the ranking decorator (prompt-history entries with trigger knowledge-advisor, plus the decision log line). Prefer reading the prompt-history store the retro lane already reads over parsing logs; if a kick-id to bead mapping is needed, derive it from the kick's issue/PR ref the same way RetroRecord.IssueRef/PRRef are filled today.
src/pkg/retro/analysis.go: include the arm fields in the compact record handed to the analysis model as counters only (no reasons, no pane text).
Expose a per-arm summary (counts and the mean of FixAttempts, CIFailureCount, KicksReceived per arm) on the retro lane's existing report surface so an operator can compare arms without running a model.
Acceptance criteria
A bead whose kicks were all in the holdout arm yields AdvisorArm: holdout; mixed kicks yield mixed; no decisions yield none.
The analysis prompt payload contains the new counters and no advisor reason text (test asserts absence).
The per-arm summary is produced with zero model calls (retro.analysis_model empty in the test).
Existing retro tests unchanged in outcome.
Tests expected
src/pkg/retro/retro_test.go additions with a fixture of decision records. CI is the verdict.
Housekeeping
Changelog fragment: changelog.d/added-retro-advisor-arm.md, single - bullet.
Docs: retro section of src/docs/agent-knowledge-and-memory.md (or the retro doc if separate): what the arm means and how to read the comparison.
🔒 Claimed by agent scanner (agent on hive hosted-available-oke-11-placeholder-r05x). The claim lasts 2h0m0s without activity and renews while the work continues. Reply /unclaim to release, or /claim to take it over if you outrank the holder.
Claim history
2026-11-08 06:36 UTC — claimed by scanner (agent on hive hosted-available-oke-11-placeholder-r05x)
The PR body says this is partial/non-closing work (Refs/Part of or leave-open wording), so Hive will not add a closing keyword. GitHub will not show its issue-list PR icon without a closing keyword; the stage comment is the link.
@clubanderson this issue was closed by @hivecommons-hive[bot] as completed.
If this was closed by a merged PR, the fix only takes effect once it ships in a tagged release and your deployment updates to it — check your running version against the fix before concluding the fix failed.
If you are still seeing the problem, comment /reopen and it will be reopened for you — you do not need write access. Please add what you are still observing (running version, logs, steps) so it is not closed the same way again.
Activity
hivecommons-hive commentedon Nov 8, 2026
Deferred this cycle: readiness requires the ranking decorator (#17155) merged so decisions exist to attribute. Not starting until then.
🐝 Hive Agent:
scanner| Instance:hosted-available-oke-11-placeholder-r05x| SHA:unknown— hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88
hivecommons-hive commentedon Nov 8, 2026
🔒 Claimed by agent
scanner(agent on hive hosted-available-oke-11-placeholder-r05x). The claim lasts 2h0m0s without activity and renews while the work continues. Reply/unclaimto release, or/claimto take it over if you outrank the holder.Claim history
scanner(agent on hive hosted-available-oke-11-placeholder-r05x)hivecommons-hive commentedon Nov 8, 2026
PR #17207 covers this issue: #17207
The PR body says this is partial/non-closing work (
Refs/Part ofor leave-open wording), so Hive will not add a closing keyword. GitHub will not show its issue-list PR icon without a closing keyword; the stage comment is the link.hivecommons-hive commentedon Nov 8, 2026
All children delivered: #17211 → PR #17218.
Every tracked child of this issue is closed, so the hive is closing it as completed. Reopen it if something it asked for is still missing.
github-actions commentedon Nov 8, 2026
@clubanderson this issue was closed by @hivecommons-hive[bot] as completed.
If this was closed by a merged PR, the fix only takes effect once it ships in a tagged release and your deployment updates to it — check your running version against the fix before concluding the fix failed.
If you are still seeing the problem, comment
/reopenand it will be reopened for you — you do not need write access. Please add what you are still observing (running version, logs, steps) so it is not closed the same way again.