Skip to content

No store migration path: engine 0.22.1 cannot serve a 0.11.2 store; 15s readiness gate aborts; memory_export capped at 15 MiB #1584

Closed

Description

@leopold-dauvergne

What happened

A long-running daemon (agentmemory 0.9.29 + iii-engine 0.11.2, file_based kv store) degraded into a wedge: REST worker routes stopped answering while the gateway kept accepting connections. Upgrading to 0.9.30 + the pinned engine 0.22.1 gives a healthy daemon — that cannot read the old store, so there is no way to migrate the 985 memories / 27k-node graph off it.

Windows 11, native engine binaries (no Docker).

1. The old stack wedges; symptoms are gateway-level

Store C:\Users\Leo\data\state_store.db (348 MB, 331 shards), stack 0.9.29 + iii 0.11.2, ~60 min uptime:

probe result
agentmemory status (CLI, direct kv read) v0.9.29 · degraded · Sessions 349 · Observations 8719 · Memories 985 · Graph 27256 nodes / 54396 edges · Heap 917 MB
GET /agentmemory/livez 200 in 5.9 s
GET /agentmemory/health times out (>8 s, no response)
GET /agentmemory/sessions times out (>8 s, no response)
netstat ~10 sockets ESTABLISHED on 3111, never closing

Consumers time out: an extension hook that awaited smart-search was killed by the harness at 30 s, every prompt. Health went degraded → critical and heap sat at 770–917 MB within an hour of a fresh boot.

2. Engine 0.22.1 does not surface the old store

agentmemory --data-dir C:\Users\Leo\data (0.9.30, pinned engine 0.22.1 installed to ~/.agentmemory/bin/iii.exe):

  • engine boots, REST binds ~20 s
  • worker log: Loaded persisted vector index (355 vectors) → vector index is readable
  • then: agentmemory worker did not become ready within 15s. → process.exit(1) (hardcoded 15 s, no flag/env to raise it)
  • afterwards status reports Sessions 0 · Observations 0 · Memories 0 · Graph 0 nodes, 0 edges · Provider ✗ noop (no key) — memories/graph/bm25 keys are invisible, only the vector index loaded
  • in one run the engine grew to 1.3 GB RSS without ever serving REST

3. No migration surface

  • agentmemory --help (0.9.30): no export, backup, migrate, or restore command.
  • MCP surface (54 tools, enumerated via tools/list on agentmemory mcp): no import/restore/bulk-write/seed tool — memory_save inserts one record at a time.
  • memory_export takes no arguments and refuses on a real store:
    {"bytes":320615835,"error":"Response is 305.8 MiB, over the ~15 MiB limit","limitBytes":15728640,"oversized":true,"success":false}
    It also proxies over REST, so on a wedged daemon it is unreachable anyway.

Net: 305.8 MiB of history is readable by the old (wedging) stack only, and there is no supported path to move it.

4. Embeddings regressed across the upgrade

Same ~/.agentmemory/.env: 0.9.29 reports Provider ✓ llm · Embeddings ✓ embeddings; 0.9.30 on a fresh store reports Provider ✓ llm · Embeddings bm25-only. Silent recall-quality downgrade on upgrade; status shows it but nothing warns during upgrade.

Asks

  1. A documented store migration (0.11.x file_based kv → 0.22.x), or a migrate --from <dir> command that reads the old shards and replays them.
  2. Make the worker readiness gate non-fatal and configurable (flag/env, default higher than 15 s). Aborting at 15 s on a 348 MB store leaves an engine running with no worker — indistinguishable from the wedge.
  3. Paginate/filter memory_export (limit, offset, include: {memories, observations, graph}) so a 300 MB store is exportable under the shim's response cap.
  4. Detect and warn on embedding-provider loss in status/upgrade output.

Workaround applied

Fresh store + pinned 0.22.1: healthy, 34 MB heap, GET /agentmemory/health 5 ms, save→recall round trip verified. History is kept as a cold archive at the old path. Separately, the consumer side was hardened: every extension request now has a bounded timeout (8 s default, 2 s health, 6 s smart-search), so a wedged daemon costs ~10 s per prompt instead of a 30 s harness timeout.

Activity

  1. yoropico commented on Oct 21, 2026

    @yoropico

    Confirming the same two blockers on a real 0.9.29 (iii 0.11.2) store copied into 0.9.30 (iii 0.22.1), with measurements:

    A documented migration (or an export/import that pages past the frame limit — /export returns 413 on this store) would be needed for stores of this size.

  2. rohitg00 commented on Nov 9, 2026

    @rohitg00
    Owner

    Fixed in 0.9.33. It pins iii 0.25.1, which serves current stores. The first start installs the matching engine (#1634, #1694). Export and whole-scope reads page past the old 15 MiB limit (#1636). A 21 MB store that returned 500 on 0.9.32 now loads.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions