Skip to content

/agentmemory/sessions deadlocks permanently: api::sessions fires one kv.get per session in a single Promise.all (leaks invocations, dashboard shows 0 sessions / 0% token savings) #1100

Closed

Description

@Codejune

Summary

GET /agentmemory/sessions hangs permanently (no response after 180s+, curl exits with HTTP 000) once the session count grows into the hundreds. Every attempt additionally leaks pending invocations on the worker, so a dashboard left open (which polls this endpoint) drives the server into degraded state. Because the viewer dashboard and agentmemory status both build their stats from this endpoint, the visible symptoms are:

Environment

  • agentmemory v0.9.28 (npm global install), macOS 15 (arm64), Node v25.8.1
  • 615 sessions / 9,525 observations / 114 summaries in the store
  • Store sizes are small: mem:sessions 180 KB, mem:summaries 398 KB — data volume is not the problem

Root cause

The api::sessions handler fires one kv.get(KV.summaries, id) per session inside a single Promise.all:

const sessions = await kv.list(KV.sessions);
// ...
const summaries = await Promise.all(
  filtered.map((s) => kv.get(KV.summaries, s.id).catch(() => null))
);

With 615 sessions that is a burst of 615 concurrent kv calls from one worker function. At that size the engine/worker channel never services them — the promises neither resolve nor reject (there is no timeout on kv.get, and .catch only handles rejections), so the handler waits forever and each request permanently parks its in-flight kv invocations on the worker.

Evidence / bisection

Test Result
GET /agentmemory/sessions (615-way fan-out) hangs, HTTP 000 after 180s
GET /agentmemory/sessions?agentId=zz-nonexistent (filter empties the list before the fan-out → 0 gets) HTTP 200 in 0.17s
GET /agentmemory/replay/sessions (single kv.list(KV.sessions), no fan-out) HTTP 200 in 0.05s
memory_diagnose categories sessions,summaries (sequential kv.list calls, reads every summary value) passes, "All 114 session summaries are consistent"
mem::context (same fan-out pattern but sliced to 10 sessions) works fine
worker active_invocations after 3 hung /sessions attempts +~600 per attempt, never drains

So: listing works, every summary value is readable, small bounded fan-outs work — only the unbounded concurrent kv.get burst deadlocks.

Fix

Replace the per-session fan-out with a single kv.list + in-memory join (same pattern mem::diagnose already uses; summaries carry sessionId):

 const filtered = filterAgentId ? sessions.filter((s) => s.agentId === filterAgentId) : sessions;
-const summaries = await Promise.all(filtered.map((s) => kv.get(KV.summaries, s.id).catch(() => null)));
+const summariesById = new Map((await kv.list(KV.summaries).catch(() => [])).map((sum) => [sum.sessionId, sum]));
 return {
   status_code: 200,
-  body: { sessions: filtered.map((s, i) => summaries[i] ? { ...s, summary: summaries[i] } : s) }
+  body: { sessions: filtered.map((s) => summariesById.has(s.id) ? { ...s, summary: summariesById.get(s.id) } : s) }
 };

Verified locally by patching the bundled dist and restarting the worker:

  • /agentmemory/sessions: 0.02–0.08s, HTTP 200, 615 sessions with 114 summaries attached
  • agentmemory status: Sessions: 615, Observations: 9,526, Token savings: ~760,180 tokens saved (100% reduction)
  • active_invocations: back to single digits and stable while polling the dashboard

More generally it would be worth (a) bounding concurrency anywhere a worker function fans out kv calls over an unbounded collection, and (b) adding a timeout to kv operations so a stalled call fails instead of hanging the HTTP request and leaking the invocation.

Happy to open a PR if useful.

Activity

  1. killianMei commented on Aug 6, 2026

    @killianMei

    Reproduced on the current npm latest (@agentmemory/agentmemory v0.9.28, iii v0.11.2, macOS arm64).

    This occurs with only 61 sessions and 46 summaries:

    • direct :3111/agentmemory/sessions: no response after 15s
    • viewer proxy :3113/agentmemory/sessions: 504 at 10.00s
    • /sessions?agentId=does-not-exist: 200 in 4–5ms
    • /replay/sessions: 200 in 24–32ms
    • worker active_invocations: 227 after repeated dashboard polling

    The installed dist/index.mjs still contains the unbounded Promise.all(filtered.map(... kv.get(KV.summaries, s.id))) implementation.

    I see #1136 merged the bounded-fan-out fix on Aug 2, but npm view @agentmemory/agentmemory version still returns 0.9.28 and the published package does not contain it. Could you publish a patch release, or keep this issue open until the fix is available from npm?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions