Repository navigation
Description
Problem
A maintainer reviewing a hive-authored assessment PR asked that rules stated in review ("X is the source of truth", "we should consider this in future") land in the hive's knowledge base. This request comes from @nate-double-u (cncf/techdocs#424). #19086 and #19095 added deterministic capture into the review-rules knowledge channel, but once deployed three gaps keep the concrete example from ever being captured:
- Capture is gated on the feedback-handled state.
captureRulesonly seesd.Items, which come fromUnansweredFeedback(src/pkg/github/pr_feedback.go:236). That drops every comment named in ahive:feedback-handledmarker (line 261), every inline comment older than the hive's last reply in its thread or in a resolved thread (lines 279, 303), and every id the sweep store already recorded (src/pkg/reviewfeedback/reviewfeedback.go:215). A rule a reviewer stated before the capture code was deployed, or in a thread the agent already answered, is never classified. The inline comment carrying the maintainers.yaml rule on Add draft documentation analysis for Hive Commons (pilot) cncf/techdocs#424 is already in such a marker. - Issue-level PR comments only count when they address the hive. A conversation comment is fed to capture only when it mentions the App bot or the configured AI author, or starts with
/hive(pr_feedback.go:187,:269). A maintainer stating a rule in a plain PR comment, or addressing the operator, is never classified. - Quoted text and questions are classified as the reviewer's rule.
Classifymatches the whole body (src/pkg/reviewrules/classify.go:97). A comment that quotes the hive's own reply (a>block) and then asks "is there a way to add to the knowledge-base?" would be recorded as a rule attributed to the reviewer, with the hive's words as its text.
Smaller: future-looking phrasings such as "future assessments should ..." or "moving forward" do not match (classify.go:70 requires "for future ..."), a rule sentence that only says "we should consider this in future" is recorded without the text it refers to, and which phrases count and which associations are trusted are env-only knobs with no Settings home.
Fix (one PR)
- Rule capture reads its own candidate set: every human comment on the hive's open PR (inline, review body, conversation) whatever its handled, resolved or addressed state. The rule index tracks which comments it has evaluated, separately from the sweep's handled ids, so the first pass after deploy is a one-time backfill over the comments still visible on hive-authored PRs in the attribution window, and later passes classify only new comments. Routing kicks is unchanged.
- The classifier ignores quoted (
>) lines, fenced code and questions, and falls back to the whole comment when the rule sentence is too short to stand alone. - Broaden
going-forwardto "future assessments/analyses/reviews/work", "for the future", "moving forward". - Preferences become
review_feedback.rules.*keys in hive.yaml (enabled, trusted associations, extra phrases, disabled signals), with env vars still honoured, and an owner-only Settings card (Features, Review) to edit them in place without a restart. - Tests for each case, docs, changelog fragment.
Beads or the review-rules channel?
The operator asked whether the beads ledger (bd remember, bd create) would be enough. Compared:
beads (bd remember / advisory bead) |
review-rules channel (#19086) |
|
|---|---|---|
| Reaches the authoring agent on its next run | No direct path. Open beads appear as work or advisory findings; a fact reaches kicks only after it is closed, synthesized into bead-synth-wiki, and ranked by keyword match |
Yes. Pinned entries are primed into every matching kick ahead of ranked knowledge, keyword-less kicks included (#19095) |
| Scope per agent, workflow or repo | Per-agent store only | agent, workflow, repo, global scopes |
| Provenance | Title and free-form metadata; bd remember authors as system |
PR, comment URL, reviewer, association, time, source |
| Dedupe | Title upsert only | Per comment id plus near-duplicate text merge |
| Retraction | Close the bead, but advisory beads are re-marked stale or re-filed by the advisory loop | hivectl knowledge state rule-r-N deprecated, reconciled by the sweep |
| Owner visibility | Bead ledger view | Dashboard knowledge list, plus the "Recorded as rule R-n" reply on the PR |
| New code needed | Priming, scoping, provenance and retraction would all have to be built | None for storage; this issue only fixes what gets classified |
Choice: keep the review-rules channel and do not add a beads path. Beads would duplicate the store without reaching the agent any better. This issue stays scoped to the capture bugs and the settings.
Acceptance
- Once deployed, the maintainers.yaml inline comment on Add draft documentation analysis for Hive Commons (pilot) cncf/techdocs#424 is captured as a rule on the next sweep pass, even though it is already in a handled marker.
- A trusted reviewer's plain PR comment stating a rule is captured; one that only quotes the hive and asks a question is not.
- The four preferences are editable by the owner in Settings and take effect without a restart.
Activity
hivecommons-hive commentedon Nov 18, 2026
Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor.
🐛 Read reviewer-stated rules from every PR comment once, and give the…
🐛 Reviewer-stated rules: ignore quotes and questions, match more futu…
clubanderson commentedon Nov 18, 2026
The capture half of this issue (rule candidates from every human comment, an evaluated-id index separate from the handled ids, one-time backfill) landed in #19146 via the duplicate #19136. #19143 is rebased on it and carries the rest: the classifier fixes for quotes, code, questions and future-looking phrasings, and the review_feedback.rules.* preferences with their Settings card.
hivecommons-hive commentedon Nov 19, 2026
All children delivered: #19136 → PR #19146, #19137 → PR #19144, #19138.
Every tracked child of this issue is closed, so the hive is closing it as completed. Reopen it if something it asked for is still missing.
🐛 Reviewer-stated rules: ignore quotes and questions, match more futu…