feat(history): semantic_search over messages + summaries via a shared embed-service

[antra-tess]

What

An optional fifth HistoryModule tool, semantic_search, for "I remember roughly what it was about, not the words" — the case search (substring/regex) cannot cover. Embeddings and the vector index live in a shared remote embed-service (one per fleet; index keyed by a per-store namespace), so remote residents need no local model or index files.

new HistoryModule({ semantic: { url, token, namespace } })

How it stays in sync

SemanticIndexer pushes incrementally and idempotently:

• messages — walk the native time index from the service's cursor watermark minus a 10-minute overlap; the service dedups by id + text hash, so re-sends cost nothing.
• summaries — watermarked by createdMs (a fresh L3 spans months of old timestamps; creation time is the only monotonic signal).
• background tick every 60 s (unref'd, starts at bind()), plus a bounded catch-up before each search — a backlog is reported as index.behind, never blocked on.
• service unreachable → exponential backoff; semantic_search returns a clean tool error; the other four tools are untouched.

What gets embedded per message: text blocks + the string args of think / journal / skip_reply / private_note (the agent's own diary — exactly what "what did I think about X" should find). Tool results, other tool args, thinking blocks and media are skipped.

Tool surface

query, limit (≤50), from/to, channelId (label or raw id, same resolution as the other tools; implies messages), kinds messages|summaries|both, level (implies summaries), minScore. Hits carry msg:<id> / sum:<id>, kind/level, timestamp, channel, participant/author, score, snippet.

Tests

• test/history-module-semantic.test.ts (5): fake embed-service over node:http — tool gating, text extraction, incremental sync with cursors, filter mapping, failure + backoff, validation.
• Existing history suites: 84/84. tsc --noEmit clean.
• End-to-end against the real service + a real 5.5k-message store snapshot: sync 43 ms, hits ranked correctly with time/kind filters.

Companion

connectome-host PR adds the recipe passthrough (modules.history: { semantic: { url, token, … } }).

🤖 Generated with Claude Code

[slimepriestess]

Reviewed head 11f452e together with connectome-host#144 as one feature. The shape is good: optional tool, remote index, failure posture that never touches the other four tools, and the sync is idempotent by construction. One mechanism defect blocks; the rest is small or a design question for you.

Blocking

The sync walk can freeze at the watermark (livelock), silently

Every tick re-walks the overlap window (default 10 min) behind the service's max_cursor so slightly out-of-order messages are still picked up, and the service dedups the re-sends. But the re-sends also spend the tick's item budget. Whenever the overlap window holds more indexable messages than the budget, each tick pushes the same already-indexed items, hits the budget, reports more: true, and never reaches the first new message. The thresholds are low for a busy resident: > 256 messages in any 10-minute window freezes the pre-search catch-up (every semantic_search re-sends the same 256 and reports index.behind), and > 1024 freezes the background tick, so the index stops at that point permanently while nothing logs. A backfill of an old store sticks the same way at the first dense 10 minutes in its history.

Repro (test/history-semantic-sync-converges.test.ts, red on 11f452e): 700 messages one second apart, ticks of 300 → [[512,true],[512,true],[512,true], …] forever, index frozen at 511.

Fix on my fork, ready to cherry-pick: slimepriestess/agent-framework@6b6c43c (compare). Only items with cursor > msgWm spend budget, so the overlap re-walk is a no-op for the budget exactly as it is for the service. The page-level budget check is untouched (a tick may still overshoot by up to one page, as today). With it: 700/700 after three ticks, then more: false.

Note the stub CM in the test follows the real MessageStore.queryByTime contract (inclusive bounds, oldest first, limit = first N, native /timestamp index) — I checked context-manager 0.10.0's implementation for that, since the walk depends on it.

Should fix (in the same lift)

• stop() does not cancel the 5 s first-tick timer. A module stopped before it fires still calls stats + upsert afterwards (second case in the same test file, red on head: ["stats","upsert"] after stop()). The lift tracks the handle and clears it. Relevant for the host's session switch, which stops the old module and starts a new one.

Design question — needs your ruling, not a code change from me

think's own description promises: "stays in your own context and is NOT sent to channels or other surfaces." With includePrivateTools defaulting to true, think / journal / skip_reply prose leaves the host for a fleet-shared service, in a namespace whose only separation from every other resident's is a string, behind one EMBED_TOKEN that every recipe holds. Two things follow: (1) any resident with the token can search another's namespace, private prose included; (2) the resident is not told. I'd either default includePrivateTools to false and make it an explicit per-recipe opt-in, or amend the think description when semantic is on, or both. Whatever you pick, it's the kind of thing worth a line in the recipe docs where the operator turns it on. Not blocking on code; I'd rather it be decided than assumed.

Non-blocking

• channelId + kinds: 'summaries' → kinds = ['summary'] with a channel filter the service applies to items that have no channel → empty result with no error. The description says summaries are excluded when channelId is set; an explicit kinds: 'summaries' alongside it should probably throw instead.
• A semantic_search that lands while a background tick is running gets that tick's promise, so the "bounded" pre-search catch-up can wait on a 1024-item tick. Fine as behaviour, worth one word in the tool description's "bounded".
• Known limit, as designed: a message stamped earlier than watermark − overlap that arrives later is never indexed. CM stamps at arrival (addMessage takes no timestamp) so that is clock jumps and imports only.
• Deletions/redactions in the chronicle are not propagated to the index (append-only sync). Same category as above; a redact on a store leaves the text searchable remotely.

Receipts

• npx tsc --noEmit clean on head and on the lift.
• test/history-module-semantic.test.ts 5/5; history-module-overview 28/28; history-module 45/49 under tsx (the four regex-worker cases need the built worker and fail identically on 3a423a1), all green built.
• Full built suite: head 11f452e 963 / 959 pass / 4 skipped / 0 fail; lift 6b6c43c 965 / 961 pass / 4 skipped / 0 fail (its two new cases included).

Companion host#144 reviewed in parallel; its merge waits on the AF release carrying this.

[slimepriestess]

Lift extended, one more commit on the same branch so it's still a single cherry-pick range:

slimepriestess/agent-framework@7d62acf on top of 6b6c43c (compare).

It takes Sol's #3 from connectome-host#144: /undo, /checkout, /restore, /branchto and /newtopic move the head within one store, an undone message keeps its id (new ones get fresh ids), and the remote index never forgets, so semantic_search kept returning the undone turn verbatim. Every hit is now checked against the current branch before it goes back to the model: msg:<id> through getMessage, sum:<id> through getSummary, both of which answer null off-branch because MessageStore.lookupIndex rebuilds on branch switch. At most limit local lookups. Dropped hits are counted as index.droppedOffBranch, and the tool description says so.

• Test in history-module-semantic.test.ts: index m1..m4 + s1, undo m2, search again → no msg:m2, droppedOffBranch: 1; same for a summary. Red on 6b6c43c, green on 7d62acf.
• Real Chronicle store: branchAt(id1) + switchBranch → queryMessages lists [1, 3], getMessage(2) is null. Sol's repro, reproduced.
• history-module, history-module-semantic, history-semantic-sync-converges: 57/57 on the branch.

Not in the lift: over-fetching k to refill a page after drops. A search that loses hits to the filter returns fewer than limit, with the count, which seemed the honest shape; say if you'd rather it re-query.

🤖 Generated with Claude Code

https://claude.ai/code/session_01CoQK2cP55YhezE6ajSx58h