Skip to content

✨ agents: drain CLI memory into draft knowledge entries before clearing - #17086

Merged
clubanderson merged 3 commits into
v5from
memory-to-beads
Nov 7, 2026
Merged

clubanderson merged 3 commits into
v5from
memory-to-beads

Conversation

@clubanderson

@clubanderson clubanderson commented Nov 7, 2026 •

Copy link
Copy Markdown
Member

Fixes #17033

Proposed by @gregoryhunt at the 2026-11-05 community meeting: he migrated ~60% of his CLI memory into beads by hand and asked whether Hive can hook memory writes so they land in beads/knowledge instead of the CLI's opaque memory store. This PR does that at kick time.

What

New agents.cli_memory.drain_to_knowledge: bool (default false, per-agent override). When on, before every non-resume kick — and before clear_on_kick wipes the store — the manager drains the agent's own CLI memory into draft knowledge entries:

CLI What is read
Claude Code ~/.claude/projects/<slug>/memory/MEMORY.md + topic files (index links skipped)
Codex every *.md under $CODEX_HOME/memories*/
Gemini ## Gemini Added Memories section of ~/.gemini/GEMINI.md

Each bullet / paragraph becomes one entry in the bead-synth-wiki vault the Primer already reads (registerKnowledgeStores), written as cli-memory-<hash16>.md with:

  • state: draft — never promoted automatically; the approved-only Primer default keeps drafts out of kicks until an operator approves them via PUT /api/knowledge/entry/{id}/state. Human review is the point.
  • source: cli-memory, agent: <name>, cli: <claude|codex|gemini>, repo: <org/repo> when the Claude project slug ends in one of the agent's repos, source_path, drained.
  • memhash: <sha256 of normalised text> — dedupe across repeated drains, in any lifecycle state and under any filename, so approving an entry does not cause it to be re-drafted.

One activity-log line per drain: cli-memory: drained N facts from <agent> into bead-synth-wiki (dupes K).

Bounds: 200 facts per drain, 512 KB per file, symlinks / hidden files / shared homes skipped. The bead-synth vault is now connected and registered when the drain is enabled even with no bead stores configured.

How

Relation to #17089

Builds on agents.cli_memory: {auto_memory, clear_on_kick} from #17089, which is already merged into v5; this PR contains only the drain commits.

Not in scope: scope: frontmatter from #17060 (not merged — plain frontmatter used).

@kubestellar-prow kubestellar-prow Bot added the dco-signoff: yes Indicates the PR's author has signed the DCO. label Nov 7, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign kellyaa for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubestellar-prow kubestellar-prow Bot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Nov 7, 2026
@github-actions

github-actions Bot commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

Open PR overlap check

Base branch: v5

Conflicts are checked with every PR merged onto the current tip of v5, so a PR that only needs a rebase is not reported as conflicting with its siblings.

PR Title Relation Files
#17097 🐛 agent(bob): preflight probes the per-agent ~/.bob that bob actually writes to same files src/crux/agent/manager.go

@hivecommons-hive

hivecommons-hive Bot commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor.

@clubanderson this PR is held at stage/071-needs-author and the hive will not merge it yet.

Failing checks:

  • relative links and anchors resolve

Next: the hive does not push to your branch. Push a new head (or reply here); CI and the review swarm re-run automatically and the PR returns to the normal path.

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Two separate things on this head (2117c26):

1. Shared CI incident (not this diff): relative links and anchors resolve is red on default branch v5 itself (Docs Link Check failure at 484f649 / 9f04cc7: CHANGELOG.md: broken link 'url'). Tracked in #17098, fix is #17096 — ignore that check here until it closes.

2. PR-local — test (rest 3/4) and coverage are genuinely red. pkg/knowledge/climemory TestDrainRejectsMissingTarget panics with a nil-pointer dereference (job 128517932168):

climemory.(*Drainer).Drain(0x0, …)  drain.go:83
climemory.TestDrainRejectsMissingTarget  drain_edge_test.go:13

drain.go:82-83 reads d.Dir before the nil guard:

func (d *Drainer) Drain(req Request) Result {
	res := Result{Store: d.Dir}          // <- dereferences d
	if d == nil || d.Dir == "" {

The test is right to pin the nil-receiver contract — it just found the bug. Suggested fix is to move the guard first:

func (d *Drainer) Drain(req Request) Result {
	if d == nil || d.Dir == "" {
		return Result{Errors: []error{errors.New("cli-memory drain: no target directory")}}
	}
	res := Result{Store: d.Dir}

(With d == nil, Store has nothing meaningful to carry anyway.) The coverage job red is downstream of the package failing, so it should clear with the same fix. This is a human-owned branch, so leaving it as a comment rather than pushing.


🐝 Hive Agent: quality | Instance: hosted-available-oke-11-placeholder-r05x | SHA: unknown

— hive: agent=quality backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive

Copy link
Copy Markdown
Contributor

The hive's GitHub App lacks actions: write, so it cannot rerun shared ci for head 2117c26dd9a6. A repository maintainer with Actions write can re-run failed jobs at https://github.com/hivecommons/hive/actions/runs/42410650591 . Alternatively, the App owner can request Actions read and write for this recovery installation and its owner must accept the update. This is a CI recovery permission block, not a source-code failure or merge approval. Hive retries after its bounded permission cooldown; no alternate credential is selected. Hive will resume automatically once the step is done; this label and note clear themselves.

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Needs a decision, not a merge:

Clear the cause and the hive re-evaluates on its next pass; the stage moves to stage/090-ready-for-human-merge, stage/095-pre-merge-soak or the merge queue on its own. Approval does not bypass CI, hold labels or other merge gates.

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read at head 2117c26. One real defect, and it is the reason test (rest 3/4) (and therefore coverage) is red; the other red check is inherited from v5.

Nil-receiver guard dereferences before it checks — medium
src/pkg/knowledge/climemory/drain.go:83-86

res := Result{Store: d.Dir}
if d == nil || d.Dir == "" {

d.Dir is read on line 83, so a nil *Drainer panics before the d == nil test on line 84 ever runs. TestDrainRejectsMissingTarget (drain_edge_test.go:11-13) calls exactly that — var nilDrainer *Drainer; nilDrainer.Drain(Request{}) — and CI shows the SIGSEGV at drain.go:83 (run 42410650591). The production path is not affected today: boot_cli_memory_drain.go always builds a non-nil Drainer and the manager calls through the CLIMemoryDrainer func hook, so this is a wrong guard plus a red suite rather than a runtime crash. Fix is to move the nil check first:

if d == nil || d.Dir == "" {
	return Result{Errors: []error{errors.New("cli-memory drain: no target directory")}}
}
res := Result{Store: d.Dir}

(If you want Store populated on the Dir == "" branch, split the two conditions.)

relative links and anchors resolve is not yours — it reports CHANGELOG.md: broken link 'url', a file this PR does not touch; the same workflow failed on v5 at 30468b7 and passed again at 2d96de7. A rebase/merge of current v5 clears it.

The rest of what I traced matches the body: the drainer only writes under d.Dir with hash-derived cli-memory-<hash16>.md names, inputs are bounded (maxMemoryFileBytes, maxFactsPerDrain, drain.go:38,42), and the Crux side only classifies store paths and calls the installed hook, so the pkg/* import boundary holds.

— hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88

Adds agents.cli_memory.drain_to_knowledge (default false, per-agent
override). Before every non-resume kick, and before clear_on_kick wipes
the store, the manager lists the agent's own CLI memory stores (Claude
~/.claude/projects/*/memory, Codex $CODEX_HOME/memories*, Gemini
~/.gemini/GEMINI.md) and the hive converts each bullet or paragraph into a
state: draft entry in the bead-synth vault the Primer already reads, with
source: cli-memory, agent:, cli:, repo: (from the Claude project slug when
it matches one of the agent's repos) and a SHA-256 memhash that makes
repeated drains idempotent across states and file names. Drafts are never
promoted automatically; the approved-only Primer default keeps them out of
kicks until a human approves them.

The parser and vault writer live in pkg/knowledge/climemory (root module);
the Crux manager only collects sources and calls the drainer installed via
crux.CLIMemoryDrainHost, so the Crux module imports no knowledge package.
One log line per drain: cli-memory: drained N facts from <agent> into
<store> (dupes K).

Proposed by @gregoryhunt at the 2026-11-05 community meeting.

Fixes #17033

Signed-off-by: Andrew Anderson <andy@clubanderson.com>
…matter hash scan and parser structure

Signed-off-by: Andrew Anderson <andy@clubanderson.com>
…ng; boot drain installer tests

Drain built Result{Store: d.Dir} before the nil-receiver guard, so the
nil-drainer edge test panicked in CI (test rest 3/4). Return the error
result first.

Add cmd/hive tests for cliMemoryDrainEnabled and installCLIMemoryDrainer:
vault connected without bead stores, drainer installed on a hosted runtime
and writes a draft entry with repo/agent tags, dedupes on second drain, and
the stand-down paths (disabled, sidecar-like runtime, no knowledge API,
unmkdirable vault path).

Signed-off-by: Andrew Anderson <andy@clubanderson.com>
@clubanderson
clubanderson merged commit 6fe79ae into v5 Nov 7, 2026
84 of 86 checks passed
@kubestellar-prow
kubestellar-prow Bot deleted the memory-to-beads branch November 7, 2026 22:20
@github-actions

github-actions Bot commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

Thank you for your contribution! Your PR has been merged.

We'd love to hear how your experience was: share feedback

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dco-signoff: yes Indicates the PR's author has signed the DCO. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

✨ knowledge: drain CLI auto-memory into beads/knowledge at session end instead of hidden MEMORY.md

1 participant