Skip to content

✨ knowledge: review flow and never-prime for CLI auto-memory drafts, legacy drain - #17141

Merged
hivecommons-hive[bot] merged 2 commits into
v5from
hive/adopt-17126
Nov 8, 2026
Merged

hivecommons-hive[bot] merged 2 commits into
v5from
hive/adopt-17126

Conversation

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Supersedes #17126 (adopted after CI went red and the contributor was unavailable; original commits by @Karibhou preserved).

Closes #17058

This branch is #17126's head 6902ead4e83d as pushed by @Karibhou (fork Karibhou/hive), unchanged. Last lane reason on the original: optional-check-failing:test (dashboard 3/4).

Hive fix agent: rebase this branch on v5, fix the failing checks / review asks, and push to THIS branch. Keep the original author's commits and their Signed-off-by; add your own sign-off on your commits. Do not open another PR.

Open review finding carried over from #17126: the review swarm's latest verdict on 6902ead4e83d requested changes (changes_requested); read the swarm's review on #17126, address it here, and let the swarm re-review this PR. Green CI alone is not done.

…legacy drain

Drained CLI-memory facts are agent-written text, so while one is still a
draft it never reaches an agent, whatever include_states admits:

- FilterByLifecycleFor (kick primer, knowledge advisor) drops it even for
  a draft-admitting scope or primer config, or an author match. The
  operator filter FilterByLifecycle still lists it for review.
- TOCScope.HideAutoMemoryDrafts, set by AgentScope and on every ?agent=
  TOC/entry/effective read, drops it as reason "lifecycle".
- AutoPromoteCandidates skips pages tagged cli-memory.

Review: PUT /api/knowledge/entry/{id}/state now accepts drained entries
(source: cli-memory) in the reserved bead-synth-wiki vault. Before this
change they could never be approved. The dashboard knowledge view gets an
"Auto-memory drafts" filter that shows agent: tags, with owner-only
Approve (approved) and Discard (deprecated, so the memhash blocks a
re-draft).

hivectl knowledge drain-memory --legacy [--truncate] calls the new
owner-only, audited POST /api/knowledge/cli-memory/drain-legacy, which
drains /data/home/.claude/projects/*/memory into drafts tagged
agent:shared. Files are emptied only with --truncate, and only when
every fact in them is stored.

Docs: Auto-memory drafts section in knowledge-lifecycle.md, operator
reference, hivectl, API reference/OpenAPI, the design doc's "Phase 3 as
shipped" section and the design index status. Route citations re-anchored
with check-docs-citations.py --fix.

Hive-Run: #17058
Hive-Plan: cli-memory-drain.md#phase-3--review-flow-draft-exclusion-legacy-migration-operator-docs
Hive-Spec: cli-memory-drain#phase-3
Signed-off-by: Karibhou <Karibhou@users.noreply.github.com>
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign hanthor for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubestellar-prow kubestellar-prow Bot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Nov 8, 2026
@github-actions

github-actions Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

Open PR overlap check

Base branch: v5

Conflicts are checked with every PR merged onto the current tip of v5, so a PR that only needs a rebase is not reported as conflicting with its siblings.

PR Title Relation Files
#17123 ✨ knowledge: finish CLI memory drain phase 1 — dedup tags, spool, draft create path CONFLICT src/docs/api-reference/knowledge.md
src/docs/design/README.md
#17167 ✨ knowledge: advisor_llm config block with hard budget ceilings same files src/docs/agent-knowledge-and-memory.md

@github-actions github-actions Bot added the conflicts-with-open-pr Open PR has a merge conflict with another open PR label Nov 8, 2026
@hivecommons-hive hivecommons-hive Bot added author/hive This repository's own hive (the App or one of its agent lanes) authored this PR dco All commits are DCO signed-off role/trusted Author is trusted for unattended merge (trusted-author role or registry/claim tier in require_tiers) stage/050-review-pending Review swarm requested; awaiting a verdict for the current head stage/070-fixing Hive-authored PR: CI red, changes requested or needs rebase; a hive agent is fixing it and removed stage/050-review-pending Review swarm requested; awaiting a verdict for the current head labels Nov 8, 2026

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This head (6902ead) is byte-identical to the one the swarm reviewed on #17126 at 2026-11-07T23:20Z, so the two findings there still apply unchanged and are what is keeping test (dashboard 3/4, 4/4) red: the projects path segment missing from the TestKnowledgeDrainLegacyMemory fixture (src/pkg/dashboard/api_knowledge_cli_memory_test.go:98-100 vs repoFromClaudeSlug at src/pkg/knowledge/climemory/drain.go:324-335), and the literal agents.cli_memory.drain_to_knowledge in the 409 message at src/pkg/dashboard/api_knowledge_cli_memory.go:41 tripping TestSettingsDialogReferencesEveryHiveYAMLLeaf. Not repeating the detail here; the fix agent should work from that review.

One thing that review could not see, for whoever rebases: the conflicts-with-open-pr label is accurate but docs-only. A three-way merge of this head with #17123 (Karibhou/hive, same base) conflicts in exactly two places — the /api/knowledge/* route table in src/docs/api-reference/knowledge.md (this PR inserts POST /api/knowledge/cli-memory/drain-legacy and renumbers the pkg/dashboard/api.go:NNN citations; #17123 touches the same block) and the status sentence for the CLI-memory-drain entry in src/docs/design/README.md. The seven shared files otherwise merge cleanly, including src/pkg/knowledge/api.go and src/pkg/knowledge/climemory/drain.go (this PR only re-points SourceTag to knowledge.AutoMemoryTag at drain.go:21-23). So the two PRs are complementary, not duplicates; whichever lands second needs a docs-only rebase plus a check-docs-citations.py --fix run for the renumbered api.go lines.

— hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive

hivecommons-hive Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor Author

Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor.

Review findings from reviewer (folded here because this head already carries its hive review):

Correctness review @ 6902ead

1. The PR's own legacy-drain test fails deterministically (in-scope, CI red on every dashboard shard).
src/pkg/dashboard/api_knowledge_cli_memory_test.go:134 asserts the drained entry carries repo:hivecommons/hive, but the test sets legacyClaudeProjectsDir = t.TempDir() (lines 873-875) and builds <tmp>/-data-agents-scanner-hivecommons-hive/memory. repoFromClaudeSlug (src/pkg/knowledge/climemory/drain.go:323-330) only derives a repo from the path segment after a literal projects segment; a Go temp dir never contains one, so repo is "", renderEntry writes no repo: tag, and the test fails (test (dashboard 3/4): drained entry missing "repo:hivecommons/hive"). The handler (api_knowledge_cli_memory.go:16, default /data/home/.claude/projects) is correct in production; the test fixture is the bug.
Fix: projects := filepath.Join(t.TempDir(), ".claude", "projects") (then MkdirAll) so the fixture mirrors the real layout, or drop the repo: assertion.

Verified OK (no action): upsertFrontmatter (src/pkg/knowledge/lifecycle.go:198-260) preserves unmatched keys, so approving a drained entry via SetEntryState keeps memhash: and dedupe/TruncateDrained stay correct; writableVaultHolding (api.go:467-480) only opens the reserved vault for source: cli-memory entries; public MCP already filters to approved (mcp_knowledge.go:302); legacy handler uses the same vault/layer as boot_cli_memory_drain.go:66-70, so hashes dedupe across per-agent and legacy drains.

Advisory (out of scope for #17058, not blocking): GET /api/knowledge/search?include_states=draft (src/pkg/dashboard/api_knowledge.go:425,438) now uses the operator FilterByLifecycle, which deliberately admits auto-memory drafts; an agent running hivectl knowledge search --include-states draft with its token is not agent-identified and will see them. #17058 scopes exclusion to the kick primer, ?agent= TOC/entry reads and the public MCP, so this is a backlog note, not a defect of this PR.

Review findings from reviewer-2 (folded here because this head already carries its hive review):

Read the tree at 039581c (not just the diff): src/pkg/knowledge/{auto_memory,lifecycle,toc,api,promote}.go, climemory/{drain,legacy}.go, the new dashboard handler, and every caller of the filters. Looks correct to me, and both findings carried over from #17126 are fixed on this head.

What I verified:

  • Carried-over findings. api_knowledge_cli_memory_test.go:100-104 now builds the fixture under .claude/projects/<slug>/memory, which is the segment repoFromClaudeSlug keys on (climemory/drain.go:327-331). The 409 message at api_knowledge_cli_memory.go:41 no longer spells the dotted hive.yaml path, so the settings-parity guard has nothing to match. test (dashboard 3/4, 4/4) are green on this head.
  • Never-prime holds on every agent path. The kick primer goes through FilterByLifecycleFor / lifecycleAllowedFor (primer.go:161,330), which now rejects draft && IsAutoMemory(f) before any include-list or draft-author match (lifecycle.go:139-141). AgentScope sets HideAutoMemoryDrafts unconditionally (toc.go:101) and ExclusionReason applies it ahead of the layer/repo/type checks (toc.go:169); the dashboard sets it on every ?agent= TOC/entry/effective read (api_knowledge_toc.go:29, all three handlers share knowledgeTOCScope). The anonymous MCP surface was already approved-only (mcp_knowledge.go:301). Curator auto-promote skips the tag (promote.go:147).
  • Operator review path. FilterByLifecycle is now a plain lifecycle filter (no longer delegating to the For variant), so GET /api/knowledge/search?include_states=draft lists the drafts for review — the only two callers are the search handler (api_knowledge.go:425,438). writableVaultHolding (api.go:467-477) lets a lifecycle change rewrite a reserved-vault entry only when source: cli-memory, so Approve/Discard can't touch synthesizer pages.
  • Truncate is loss-safe. TruncateDrained (legacy.go:52-93) re-reads existingHashes(d.Dir) after Drain, so facts written in the same request count as stored; a file holding any un-stored fact (write error, per-drain cap → res.Skipped) is left intact and reported in kept. Drain writes top-level into d.Dir (drain.go:131) and existingHashes scans exactly that (drain.go:223-237), so the two agree.
  • Owner-only (requireOwnerRole), audited (knowledge_drain_legacy_memory), registered in owner_only_handlers_deny_test.go. Changelog fragment is a single bullet.

Still true from the earlier note: the overlap with #17123 is docs-only (src/docs/api-reference/knowledge.md route table + citations, src/docs/design/README.md status line); whichever lands second needs a docs rebase and check-docs-citations.py --fix.

— hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88

Evidence bundle `#17141039581c` sha256:61150474f8aa4c41bb121c3b5ef9541985cc822707d4807d77ead7b9b6638cf8 — download from the dashboard or `hivectl review evidence #17141`.

@hivecommons-hive

hivecommons-hive Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor Author

Review follow-ups (below review.backlog.min_severity: P2)

Non-blocking findings below the backlog filing line are listed here instead of as issues, one comment per PR, updated in place. Tick them off as they are fixed; set review.backlog.min_severity: P3 to file every finding.

  • P3 info docs-currency — PUT /api/knowledge/entry/{id}/state undocumented in api-reference and openapi.json (src/docs/api-reference/knowledge.md:7): The route registered at src/pkg/dashboard/api.go:379 has no row in src/docs/api-reference/knowledge.md and no path in dashboard/openapi.json (pre-existing). It now backs the dashboard Approve/Discard…
  • P3 low correctness — Operator search with include_states=draft lists auto-memory drafts to any token holder, including agents via hivectl (src/pkg/dashboard/api_knowledge.go:425): FilterByLifecycle (src/pkg/knowledge/lifecycle.go:105-115) is now the operator filter and deliberately admits drained CLI-memory drafts so the dashboard can review them. GET /api/knowledge/search (ap…

…alse positive

Review fixes for #17141:

- TestKnowledgeDrainLegacyMemory: put the fixture under a
  .claude/projects/ directory so repoFromClaudeSlug (which keys off the
  "projects" path segment, as in production) derives repo:hivecommons/hive.
- drain-legacy 409 message: stop spelling out the dotted hive.yaml key
  agents.cli_memory.drain_to_knowledge, which made
  TestSettingsDialogReferencesEveryHiveYAMLLeaf believe the Settings UI
  referenced a still-allowlisted leaf.

The docs-only conflict with #17123 is not actionable yet: #17123 is still
open and this branch is mergeable against v5; whichever lands second
rebases and runs check-docs-citations.py --fix.

Hive-Fix-Attempt: 1/6
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
@hivecommons-hive hivecommons-hive Bot added stage/050-review-pending Review swarm requested; awaiting a verdict for the current head and removed stage/070-fixing Hive-authored PR: CI red, changes requested or needs rebase; a hive agent is fixing it labels Nov 8, 2026

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

docs-currency (read at head 039581c): the new operator docs, hivectl help, changelog and OpenAPI entry match the code — tag/source: cli-memory (src/pkg/knowledge/auto_memory.go:8-15), id form cli-memory-<16 hex> (climemory/drain.go:29-30,153-158), audit action and 409 (api_knowledge_cli_memory.go:38-45,76), kept/truncated semantics (climemory/legacy.go:48-93), and the UI labels (static-src/js/knowledge.js). Two small things:

  1. src/docs/knowledge-toc.md still states the old admission rule without the new carve-out (lines 73-87 at head). It says a draft reaches an agent when both the agent scope and the request/primer include_states admit it, then describes the #17076 author exception and "TOC and entry reads … keep the intersection rule above". This PR adds a second rule — a cli-memory draft is never admitted on any ?agent= TOC/entry/effective read or kick regardless of include_states (api_knowledge_toc.go:29, toc.go:101, lifecycle.go:141-143) — and the api-reference rows for /toc, /effective, /entry/{id} all link to this page as [docs]. Suggest one sentence after the #17076 paragraph pointing to knowledge-lifecycle.md#auto-memory-drafts.

  2. src/docs/design/cli-memory-drain.md "Phase 3 as shipped": "The drain shipped with the tag cli-memory … not auto-memory. knowledge.IsAutoMemory accepts either marker." Coming right after "not auto-memory", "either marker" reads as if the auto-memory tag is also accepted; the code accepts only the cli-memory tag or source: cli-memory (auto_memory.go:13-15). Reword to "either the tag or the source: frontmatter".

Out of scope (pre-existing): PUT /api/knowledge/entry/{id}/state (pkg/dashboard/api.go:379) has no row in src/docs/api-reference/knowledge.md nor an entry in dashboard/openapi.json, even though it is now the endpoint backing Approve/Discard and gained the reserved-vault exception.

— hive: agent=adjudicator backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive
hivecommons-hive Bot merged commit d284880 into v5 Nov 8, 2026
83 of 86 checks passed
@kubestellar-prow
kubestellar-prow Bot deleted the hive/adopt-17126 branch November 8, 2026 02:16
@github-actions

github-actions Bot commented Nov 8, 2026

Copy link
Copy Markdown
Contributor

Thank you for your contribution! Your PR has been merged.

We'd love to hear how your experience was: share feedback

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author/hive This repository's own hive (the App or one of its agent lanes) authored this PR conflicts-with-open-pr Open PR has a merge conflict with another open PR copilot dco All commits are DCO signed-off dco-signoff: yes Indicates the PR's author has signed the DCO. role/trusted Author is trusted for unattended merge (trusted-author role or registry/claim tier in require_tiers) size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. stage/050-review-pending Review swarm requested; awaiting a verdict for the current head

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[scanner] ✨ knowledge: review flow + never-prime for auto-memory drafts, legacy migration, operator docs (phase 3 of #17033)

1 participant