Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -196,7 +196,9 @@ queue behind it so each Session stores them in arrival order (one already writin
write fails may still store ahead of it); past the cap one is dropped (500), never
written ahead of the queue. A failure after the row is stored is never queued, since replaying
it would duplicate the row. Session-cap eviction runs only after the row is stored, and its
errors are logged, not returned. A daemon restart before a write succeeds loses the queue.
errors are logged, not returned. It counts the Session's rows with `StateKV.count()` before
listing them, and lists and sorts victims only when the count exceeds the cap. A daemon restart
before a write succeeds loses the queue.
`GET /agentmemory/health` carries `capture: { lastArrivalAt, lastStoredAt, oldestPendingAt }` (ISO
or `null` since boot; `src/telemetry/capture.ts`): arrival is stamped by `api::observe` once the
body validates, before any work, and stored by `mem::observe` right after the row write, so the
Expand Down
19 changes: 19 additions & 0 deletions docs/benchmarks/2026-11-21-session-cap-count.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# 2026-11-21 — Session-cap count before listing

## Hypothesis

Counting a Session's Observations before listing them avoids loading and decoding every row when the Session is below its cap (#479). Once the count exceeds the cap, the existing victim selection still runs. A focused test verifies that the under-cap path does not list the scope.

## Method

Compare `main` at `929ccc12` with the combined local patches for #479, #439, #436 and #438 (tree `0e1cbcd0`). Each arm starts a throwaway daemon from the same synthetic SQLite seed with four Sessions of 1,800 Observations each, below the 2,000 cap. Four concurrent streams send 50 `observe` requests each while `/livez` is polled every 50 ms. Five paired rounds alternate arm order; one-minute load was 1.65–2.51. No live daemon or model provider was used.

## Results

| Metric (median of five rounds) | Main | Candidate |
| --- | ---: | ---: |
| `observe` p50 | 42 ms | 3 ms |
| `observe` p95 | 92 ms | 11 ms |
| `/livez` max | 62 ms | 28 ms |

Every round favored the candidate for `observe` p50 and p95. This reproduces the under-cap improvement seen with the count-only prototype at `fdc3d459` (p50 255 → 24 ms at load 23–28). The current run tested the combined tree, not the isolated count change; #439 also changes hashing on the capture path. At-cap latency was not measured. The combined gate passed 3,183 tests (1 skipped), typecheck and both Recall gates. The change remains local.
1 change: 1 addition & 0 deletions docs/experiments.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ A hypothesis comes from one of these, named in the Source column:

| Date | Hypothesis | Source | Eval | Result | Verdict | Retry if | Evidence |
|---|---|---|---|---|---|---|---|
| 2026-11-21 | Counting before listing reduces Observation write latency below the Session cap (#479) | incident | Five paired capture rounds; `main` vs four-patch tree | `observe` p50 42 → 3 ms, p95 92 → 11 ms; median round `/livez` max 62 → 28 ms; count-only effect on this base not isolated | inconclusive | attribution on current `main` is needed: compare the count-only patch against the same base and seed | [2026-11-21 Session-cap count](./benchmarks/2026-11-21-session-cap-count.md) |
| 2026-10-01 | Baseline: Recall loses content on the Injection paths, not on search | — | coding-agent-life-v2, all questions: search 20, pre-tool-use 15, session-start 3 | search recall 1.000 but 0/4 no-answer clean; pre-tool-use recall 0.667, leaking across projects on shared filenames; every path beats `random` | finding | — | [2026-10-01 coding-life](./benchmarks/2026-10-01-coding-agent-life-v2.md) |
| 2026-10-03 | Filtering the prompt-submit Injection by a BM25 floor (≥ 5 and ≥ 0.5× best) raises precision over the unfiltered top K | — | coding-agent-life-v2 prompt-submit, n=15 | precision 0.303 → 0.822, no-answer clean 0.333 → 0.833, recall 0.963 → 0.907; mean Injection 184 chars | kept | — | [2026-10-03 coding-life](./benchmarks/2026-10-03-coding-agent-life-v2.md), #106 |
| 2026-10-05 | A smart-search relevance floor (BM25 ≥ 3.5 and ≥ 0.5× best, or cosine ≥ 0.3 with a BM25 match) cuts noise without losing recall | — | coding-agent-life-v2 search, n=20 | bm25 precision 0.195 → 0.628, no-answer clean 0 → 0.250, recall 0.969 unchanged; embedding adapter precision 0.190 → 0.601, recall 1.000 | kept | — | [2026-10-05 coding-life](./benchmarks/2026-10-05-coding-agent-life-v2.md) |
Expand Down
6 changes: 6 additions & 0 deletions src/engine/inproc/state.ts
Original file line number Diff line number Diff line change
Expand Up @@ -337,6 +337,11 @@ export class SqliteState {
return rows.map((r) => JSON.parse(r.value));
}

count(scope: string): number {
const row = this.db.prepare("SELECT COUNT(*) AS n FROM kv WHERE scope = ?").get(scope) as { n: number };
return row.n;
}

bytes(scope: string): number {
const row = this.db
.prepare("SELECT COALESCE(SUM(LENGTH(CAST(value AS BLOB))), 0) AS n FROM kv WHERE scope = ?")
Expand Down Expand Up @@ -621,6 +626,7 @@ export function stateFunctions(
"state::delete": async (p) => cooperate(store.delete(p.scope, p.key)),
"state::list": async (p) => cooperate(store.list(p.scope)),
"state::list-page": async (p) => cooperate(store.listPage(p.scope, p.after, p.limit)),
"state::count": async (p) => cooperate(store.count(p.scope)),
"state::bytes": async (p) => cooperate(store.bytes(p.scope)),
"state::list-newest": async (p) => cooperate(store.listNewest(p.scope, p.opts)),
"state::list-scopes": async (p) => cooperate(store.listScopes(p.prefix)),
Expand Down
1 change: 1 addition & 0 deletions src/functions/observe.ts
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,7 @@ async function evictOverCap(
pendingRefReleases: string[],
): Promise<number> {
const scope = KV.observations(sessionId);
if ((await kv.count(scope)) <= cap) return 0;
const existing = await kv.list<{
id: string;
timestamp?: string;
Expand Down
7 changes: 7 additions & 0 deletions src/state/kv.ts
Original file line number Diff line number Diff line change
Expand Up @@ -208,6 +208,13 @@ export class StateKV {
})
}

async count(scope: string): Promise<number> {
return this.sdk.trigger<{ scope: string }, number>({
function_id: 'state::count',
payload: { scope },
})
}

async bytes(scope: string): Promise<number> {
return this.sdk.trigger<{ scope: string }, number>({
function_id: 'state::bytes',
Expand Down
13 changes: 13 additions & 0 deletions test/inproc-state.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,19 @@ describe("inproc state store", () => {
expect(next).toEqual([{ key: "c", value: "c" }, { key: "d", value: "d" }]);
});

it("counts exactly the current rows in one scope", async () => {
const fns = stateFunctions(store);
expect(await fns["state::count"]({ scope: "s" })).toBe(0);
await fns["state::set"]({ scope: "s", key: "a", value: 1 });
await fns["state::set"]({ scope: "s", key: "a", value: 2 });
await fns["state::set"]({ scope: "s", key: "b", value: 3 });
await fns["state::set"]({ scope: "s:other", key: "a", value: 4 });
expect(await fns["state::count"]({ scope: "s" })).toBe(2);
await fns["state::delete"]({ scope: "s", key: "a" });
expect(await fns["state::count"]({ scope: "s" })).toBe(1);
expect(await fns["state::count"]({ scope: "s:other" })).toBe(1);
});

it("reports the stored byte size of a scope", async () => {
const fns = stateFunctions(store);
expect(await fns["state::bytes"]({ scope: "s" })).toBe(0);
Expand Down
30 changes: 22 additions & 8 deletions test/observe-session-cap.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import { getSearchIndex } from "../src/functions/search.js";
import { KV } from "../src/state/schema.js";

function mockKV() {
const lists: string[] = [];
const store = new Map<string, Map<string, unknown>>();
return {
get: async <T>(scope: string, key: string): Promise<T | null> =>
Expand All @@ -34,8 +35,12 @@ function mockKV() {
}
return row;
},
list: async <T>(scope: string): Promise<T[]> =>
Array.from(store.get(scope)?.values() ?? []) as T[],
list: async <T>(scope: string): Promise<T[]> => {
lists.push(scope);
return Array.from(store.get(scope)?.values() ?? []) as T[];
},
count: async (scope: string): Promise<number> => store.get(scope)?.size ?? 0,
lists,
};
}

Expand Down Expand Up @@ -221,7 +226,7 @@ describe("mem::observe at MAX_OBS_PER_SESSION (PR#1174)", () => {
expect((await kv.list<{ id: string }>(KV.observations(SESSION))).map((o) => o.id)).toEqual(["a", "b"]);
});

it("stores the Observation and counts it when eviction fails", async () => {
it.each(["count", "list"] as const)("stores the Observation and counts it when eviction %s fails", async (failure) => {
const kv = mockKV();
const sdk = mockSdk();
await kv.set(KV.observations(SESSION), "a", stored("a", 2, "2026-01-01T00:00:00Z"));
Expand All @@ -233,11 +238,15 @@ describe("mem::observe at MAX_OBS_PER_SESSION (PR#1174)", () => {
status: "active",
observationCount: 1,
});
const list = kv.list;
kv.list = async <T>(scope: string): Promise<T[]> => {
if (scope === KV.observations(SESSION)) throw new Error("boom");
return list<T>(scope);
};
if (failure === "count") {
kv.count = async () => { throw new Error("boom"); };
} else {
const list = kv.list;
kv.list = async <T>(scope: string): Promise<T[]> => {
if (scope === KV.observations(SESSION)) throw new Error("boom");
return list<T>(scope);
};
}
registerObserveFunction(sdk as never, kv as never, undefined, 1);

const result = (await sdk.trigger({
Expand Down Expand Up @@ -299,6 +308,11 @@ describe("mem::observe at MAX_OBS_PER_SESSION (PR#1174)", () => {
expect(await kv.get(KV.imageRefs, "/nonexistent/a.png")).toBeNull();
});

it.each([2, 3])("does not list the Session's rows when its count is at most the cap %i", async (cap) => {
const { kv } = await setup([stored("a", 1, "2026-01-01T00:00:00Z")], cap);
expect(kv.lists.filter((s) => s === KV.observations(SESSION))).toHaveLength(1);
});

it("evicts nothing under the cap", async () => {
const { ids } = await setup([stored("a", 1, "2026-01-01T00:00:00Z")], 3);
expect(ids).toHaveLength(2);
Expand Down