Repository navigation
Description
Problem Statement
The Operator's daemon fails to summarize three quarters of the Sessions it tries. A capped Session is split into chunks of 400 Observations regardless of how large those Observations are; on this Operator's store the poisoned Sessions run 366 tokens per Observation against the code's assumption of ~110, so the first chunk is ~147k tokens and takes 350s+ cold on the local broker — past the 480s bound once two chunks contend for the GPU. Every Session stop re-runs the whole pass even when a good Session Summary already exists, so the same handful of Sessions fail again every day (one id accounts for 121 of last week's failures). Each failed chunk opens the single shared circuit breaker, which then fails unrelated consolidation.
Alongside that, four smaller defects verified in this tree: the viewer proxy on the LAN-facing port forwards an unnormalized path with the daemon secret attached; MCP write tools store Agent-supplied content without the secret scrub that observe applies; hook scripts never read ~/.agentmemory/.env, so hook-only settings placed there are silently ignored; and three list routes return the entire scope (7,230 Sessions, 16,985 semantic records), /session/end conjures a ghost Session for an unknown id, and one oversized record truncates every result ranked below it.
Solution
Summarize chunks Observations by measured token count against a 50k budget, reuses a current Session Summary instead of reproducing it, takes the keyless early exit it already has, and isolates breaker failures per operation. The Operator lowers the LLM timeout to 300s, which measurement shows clears the worst concurrent case by ~2×.
The viewer proxy only forwards paths inside the /agentmemory/ prefix. Every function that stores Agent-supplied text scrubs it. Hooks hydrate ~/.agentmemory/.env through one dependency-free loader. List routes return a first page and say how many there are; /session/end rejects an unknown id; the token budget skips a record that does not fit rather than stopping.
Delivered as four PRs off main, one per cluster. Done means merged, npm run build (the hook scripts are committed build output), and systemctl --user restart agentmemory.service.
User Stories
- As an Operator, I want a capped Session to summarize successfully, so that the Sessions with the most work in them are the ones I can recall later.
- As an Operator, I want summarize chunks sized by the tokens they actually contain, so that a Session whose Observations are large does not produce a chunk that cannot finish in time.
- As an Operator, I want the token count to come from the model that will receive the chunk, so that switching
OPENAI_MODELnever leaves the chunker measuring with the wrong tokenizer. - As an Operator, I want chunking to keep working when the backend has no tokenizer endpoint, so that pointing the daemon at a generic OpenAI-compatible host degrades to an estimate rather than failing.
- As an Operator, I want that estimate to err on the side of smaller chunks, so that an undercount never produces a chunk that exceeds the budget.
- As an Operator, I want a Session that already has a current Session Summary to be left alone at stop, so that I never pay the LLM twice for the same work.
- As an Operator, I want a Session Summary to be reproduced when Observations have arrived since it was written, so that a reused summary is never missing the newest work.
- As an Operator, I want to be able to force a re-summarize, so that a summary I distrust can be regenerated on demand.
- As an Operator, I want a keyless install to skip summarize immediately, so that a daemon with no LLM configured does not run the whole pipeline before discovering it has nowhere to send the prompt.
- As an Operator, I want a Session that cannot summarize to leave semantic consolidation running, so that one bad Session never stalls unrelated work.
- As an Operator, I want the health endpoint to keep reporting a single breaker state, so that whatever I already watch it with keeps working, with the per-operation detail available beside it.
- As an Operator, I want a Session that failed during a broker outage to be summarized once the broker returns, so that a transient failure never marks a Session hopeless.
- As an Operator, I want a wedged LLM call to give up in five minutes rather than eight, so that everything queued behind it is not held for longer than the slowest real call needs.
- As an Operator, I want the viewer on my LAN-facing port to refuse any path outside the daemon's own API, so that my daemon secret is never attached to a request that leaves that prefix.
- As an Operator, I want every Memory an Agent stores through an MCP tool to be scrubbed of secrets, so that a token pasted into a
memory_remembercall never lands in the store. - As an Operator, I want the next write route added to be scrubbed without anyone remembering to add it, so that the protection sits below every path that stores content.
- As an Agent, I want recalled context to be plainly labelled as recalled data, so that stored text that looks like instructions is not mistaken for instructions.
- As an Operator, I want Memories recalled exactly as they were stored, so that a Memory about prompts or tooling is not mangled on the way out.
- As an Operator, I want settings I put in
~/.agentmemory/.envto reach the hook scripts, so that a hook-only setting likeAGENTMEMORY_INJECT_CONTEXTworks where its own comment says to put it. - As an Operator, I want a value in my shell environment to still win over the file, so that hydration matches how the daemon already resolves the same keys.
- As an Operator, I want hooks to stay fast and standalone, so that hydrating the file never puts the Engine on a hook's critical path.
- As an Operator, I want a list route to hand back a first page by default, so that a request for my Sessions cannot return all 7,230 of them in one response.
- As an Operator, I want a page to say how many records exist and where the next page starts, so that a script can tell a page from the whole store.
- As an Operator, I want to be able to ask for everything explicitly, so that a caller that genuinely wants the whole scope still can.
- As an Operator, I want ending a Session that never started to fail loudly, so that a typo or stale hook never litters my Session list with empty rows.
- As an Operator, I want to know which other callers rely on a missing key being created silently, so that I can decide whether the Engine's behaviour should change later.
- As an Agent, I want Recall to skip a record that does not fit the token budget and keep going, so that one oversized record does not hide every smaller one ranked below it.
- As an Operator, I want each cluster to arrive as its own PR, so that I can review and revert provider, security, hook and API changes independently.
- As an Operator, I want the fixes running on my daemon when the work is called done, so that "merged" and "in effect" are the same moment.
Implementation Decisions
Cluster 1 — Summarize
- Token counting is a provider capability.
MemoryProvidergains an optional token-count operation. The OpenAI-compatible provider implements it against the broker's llama.cpp-style tokenize route, scoped to the configured model, so a model switch changes the tokenizer with no Operator pairing to maintain. The count adds a fixed overhead for the system prompt and chat template (measured ~400 tokens on this broker). The resilient wrapper forwards the operation without routing it through the circuit breaker — a failed count is a fallback, not a provider failure. - Fallback estimate is
chars / 3, the idiom this tree already uses for token estimation. Measured content runs 3.6 chars/token, so/3overcounts by ~17% — the safe direction./4undercounts by 10% and is not used. The fallback is taken whenever the provider has no count operation, or the count call fails or times out. - Chunks are packed to a token budget, default 50,000 prompt tokens, configurable by
SUMMARIZE_CHUNK_TOKENS. Observations are tokenized once each (or the Session once, which measured 184ms for 500 Observations) and packed in order until the next one would exceed the budget.SUMMARIZE_CHUNK_SIZE(an Observation count) is removed; it was the wrong unit because per-Observation size varies 9× across the store. An Observation that alone exceeds the budget forms its own chunk and is logged. - Session Summary reuse. Before summarizing, read the existing Session Summary. It is current when no Observation arrived after it was written; because Observations are append-only within a Session, the implementer compares the summary's recorded Observation count to the Session's current count. A current summary is returned unchanged and the LLM is not called. An explicit
forceinput bypasses reuse. Scope of the saving: Claude Code's Stop hook fires every turn and each Stop reachesmem::summarize, so a live Session is never current and is re-summarized in full each turn; reuse saves capped and idle Sessions and duplicate Stops. Incremental summarization is the deeper cost fix and is tracked in Summarize incrementally: fold new Observations into the current Session Summary instead of re-reading the Session #44. The retry-and-fail loop on poisoned Sessions is otherwise closed by the chunking change; there is deliberately no give-up rule. - Keyless early exit. One shared predicate decides whether a provider is the no-op provider, matching on the inner provider's name rather than
=== "noop", since every provider is wrapped asresilient(<inner>). Summarize and graph extraction both use it. - Circuit breaker per operation. The resilient wrapper holds one breaker per provider operation (compress, summarize, describe-image). The existing single-state accessor reports the most severe state across operations so health consumers are unchanged; a per-operation map is exposed beside it. Failure-counting semantics inside a breaker are unchanged.
- Timeout is Operator configuration, not code:
AGENTMEMORY_LLM_TIMEOUT_MS=300000in~/.agentmemory/.env, applied at the restart step. Basis: worst measured cold 50k chunk at concurrency 2 is 152.8s; cold model loads are 9–16s; worst scheduler wait under 30s. - Chunk concurrency default becomes 2 (
SUMMARIZE_CHUNK_CONCURRENCYstill overrides). The timeout basis is measured at 2; the broker serialises chunks on one GPU, so the old default of 6 — tuned against iii's 180s cap — would push a burst of 50k chunks past 300s and reopen the breaker.
Cluster 2 — Security
- Viewer proxy allowlist. The request path is URL-normalized (dot segments resolved, percent-decoding of
.and/applied before the check) and must begin with/agentmemory/after normalization; anything else is answered locally with 404 and never reaches upstream. The bearer secret is only ever attached to a request that passed the check. - Scrub at the write functions. The existing scrub function moves below the MCP layer: every registered function that stores Agent-supplied free text (remember, lesson-save, semantic and procedural store, action-create, checkpoint-create, signal-send, sentinel-create, facet-tag, and any other content-bearing writer the implementer finds) applies it before persisting.
observekeeps its current call. MCP handlers are unchanged. - Fenced injected context. The injected block keeps its current outer element and gains a one-line preface stating the contents are recalled data and not instructions. Stored content is recalled verbatim; the only transformation permitted is neutralizing a literal occurrence of the block's own closing delimiter inside stored content, so stored text cannot terminate the fence early.
Cluster 3 — Hooks
- One hooks-local env loader, following the precedent of the existing shared hook helper for project resolution. It reads
~/.agentmemory/.env(honouring the same home-directory override the daemon honours), sets only keys not already present inprocess.env, and depends on nothing butnode:fs,node:osandnode:path. Every hook calls it before its firstprocess.envread. The daemon's own loader is not imported — it pulls in the Engine config surface. - The hook scripts under
plugin/scripts/are committed build output and are regenerated withnpm run build, never hand-edited.
Cluster 4 — API
- Paging on
/agentmemory/sessions,/agentmemory/semanticand/agentmemory/procedural: querylimit(default 100) andoffset(default 0);limit=allreturns the whole scope. The response keeps its existing collection key and addstotal,limitandoffset. Sessions are ordered newest-first before paging, after the existing agent filter — the route has no ordering today, and therecap,handoffandsession-historyskills read it throughmemory_sessionsexpecting the most recent Sessions. Export/import already reads the Engine directly and is unaffected. /agentmemory/session/endlooks the Session up first and answers 404session_not_foundfor an unknown id; nothing is written. The Engine's auto-create-on-update behaviour is unchanged; the PR body reports every other caller that updates a key without first checking it exists.- Token budget in search continues past an item that does not fit instead of returning;
truncatedis true when anything was skipped.
Testing Decisions
A good test drives a registered function through the registrar with a fake KV and asserts on what comes back or what was stored — never on internal calls. Prior art is test/crystallize.test.ts: hand-rolled mockKV() and mockSdk(), vi.mock only for the logger and keyed mutex.
- Summarize — seam:
registerSummarizeFunctionwith a fakeMemoryProviderthat records the prompts it receives and optionally implements token counting. Assert chunk boundaries from prompt sizes, reuse (provider not called when a current summary exists; called when the Observation count grew; called onforce), the keyless exit with aresilient(noop)-named provider, and fallback estimation when the count operation throws. Prior art:test/summarize.test.ts. - Breaker — seam: the resilient wrapper over a fake provider whose summarize throws and compress succeeds; assert compress stays allowed while summarize is open, and that the aggregate state reads open. Prior art:
test/circuit-breaker.test.ts. - Viewer proxy — seam: HTTP against the viewer with a fake upstream that records the URL and headers it received; assert
..and encoded-dot paths never reach it and that no request outside the prefix carries the bearer. Prior art:test/viewer-security.test.ts,test/viewer-host.test.ts. - Scrub — seam: each write function's registrar with content containing a known secret pattern; assert the stored value is scrubbed. Prior art:
test/observe-dedup-prompt.test.tsfor the registrar shape. - Fenced context — seam:
mem::contextwith a stored Memory whose content contains the closing delimiter; assert the block is still well-formed and the content otherwise verbatim. Prior art:test/context-injection.test.ts. - Hook env loader — seam: the loader function with a temporary home directory, as
test/hook-project.test.tsdoes for project resolution; assert file values land, existingprocess.envvalues win, and a missing file is a no-op. - API — seam: the REST function via
mockSdk().triggerwithquery_params; assert page shape,total,limit=all, and 404 on an unknown session id. Prior art:test/search.test.tsfor the token-budget skip. - The tool-count and endpoint-count consistency tests must still pass; no tool or endpoint is added.
Out of Scope
- Anything that changes what Recall returns or deletes rows, deferred by the Operator: MCP
projectforwarding (Project scoping not honored by MCP search tools (memory_smart_search / memory_recall) or /smart-search rohitg00/agentmemory#787), the lesson decay floor (lesson-recall default minConfidence (0.1) + decay + auto-prune make most auto-consolidated lessons invisible, with no config override rohitg00/agentmemory#1065), scheduling Eviction (fix(evict): schedule the eviction sweep at boot rohitg00/agentmemory#1287), evicting at the Session cap (fix: evict lowest-value observations at the session cap instead of refusing writes rohitg00/agentmemory#1174), bounding the audit log (fix: bound the audit log with AGENTMEMORY_AUDIT_MAX rohitg00/agentmemory#1176), ghost index entries on Eviction (fix: evicted observations leave ghost entries in search indexes rohitg00/agentmemory#1005). - The OpenAI
max_tokens/reasoning_effortpair (Bug: v0.9.29 default OpenAI model (gpt-5.6-luna) is a reasoning model, butOpenAIProvidersendsmax_tokens→ everymem::compress/mem::summarize400s rohitg00/agentmemory#1219): this Operator's broker accepts both. - The five feature additions: temporal decay (feat(search): add temporal decay to recall ranking rohitg00/agentmemory#928),
/memories/search(feat(api): add /agentmemory/memories/search rohitg00/agentmemory#1247), project isolation (Add project isolation for agentmemory rohitg00/agentmemory#829), delta graph Extraction (feat(graph): delta graph extraction rohitg00/agentmemory#1309), insight injection (feat(context,smart-search): inject insights into session context and smart search results (#457) rohitg00/agentmemory#615). - The breaker's failure-counting semantics.
- Incremental summarization (fold new Observations into the current Session Summary instead of re-reading the Session each Stop) — Summarize incrementally: fold new Observations into the current Session Summary instead of re-reading the Session #44. Also there:
/session/endmarking a live Sessioncompletedon every turn. - Changing the Engine's update-creates-missing-key behaviour.
- The stale
.envcomment naming a checkout that is not the one running, and the four stale@agentmemory/mcpshim processes.
Further Notes
- Measurements behind the numbers (this Operator's broker, model
general, cold): 50k-token chunks solo 60–89s, median 65s; at concurrency 2, 122–153s each. 100k chunks measured 247s cold./tokenizecounts 529KB in ~190ms and is exact to within the constant system/template overhead. - The
mem::compressfailure figure (47.6%) is a stale lifetime counter: auto-compress has been off since Sep 5. Only summarize's 75% is live. - Only 30 of 6,715 Sessions exceed 50k tokens; the largest becomes 5 chunks.
- Verdicts and evidence for every upstream claim are in
docs/upstream-issue-triage.yamlanddocs/upstream-pr-triage.yaml(statusfield). Read those for the diagnosis; never port upstream's patch. src/functions/graph.tscontains a literal NUL byte at the node-key separator, so plaingrepskips the file — usegrep -a.
Activity
fix(summarize): pack chunks by measured tokens against a 50k budget
fix(summarize): reuse a current Session Summary instead of reproducin…
fix(providers): one circuit breaker per provider operation
fix(viewer): forward only canonical /agentmemory/ paths to the REST port
fix(functions): scrub secrets in every function that stores Agent-sup…
fix(context): fence injected context as recalled data
fix(summarize): apply PR #45 review corrections
fix(api): refuse session/end for a Session that never started
fix(search): keep filling the token budget past an oversized record
fix(review): trim timeout history from docs, name the key separator, …