Skip to content

MAX_OBS_PER_SESSION shares its name with a hardcoded 5000 import limit: raising the cap past 5000 makes the store unrestorable #1555

Closed

Description

@yufeng-liang

Current-main boundary

Read on main 36eb328 (package.json version still 0.9.30). Two different constants share the name MAX_OBS_PER_SESSION:

  1. Runtime write cap — a user-settable env var, default 500, loaded in src/index.ts (maxObservationsPerSession: safeParseInt(env["MAX_OBS_PER_SESSION"], 500)) and enforced in src/functions/observe.ts (if (existing.length >= maxObservationsPerSession) return { success: false, error: \Session observation limit reached (${maxObservationsPerSession})` }`).
  2. Import validation constant — a hardcoded const MAX_OBS_PER_SESSION = 5e3; inside import validation, checked with if (obs.length > MAX_OBS_PER_SESSION) return { success: false, error: \Too many observations per session (max ${MAX_OBS_PER_SESSION})` }`.

Nothing connects the two. The runtime cap is documented as freely raisable, the import cap is invisible and fixed, and no startup validation or boot log warns when the first is set above the second.

Why this is a trap rather than a theoretical nit

The per-session cap is one-way: nothing ever releases slots for a live session.

  • mem::consolidate reads observations and writes memories — it never deletes observations.
  • mem::session-sweep only marks timed-out sessions abandoned — it never deletes observations.
  • The only code that deletes observations is mem::evict, and its cap is per project, not per session (DEFAULTS.maxObservationsPerProject = 1e4). A project can sit far below 10k total while one hot session is permanently wedged at the cap. Eviction is also not scheduled by default (see Feature Request: Add eviction timer + env toggle for mem::evict (parity with mem::auto-forget) #480).

So a session that reaches the cap stays there forever, and the only fix a user has is to raise MAX_OBS_PER_SESSION. Doing that past 5000 converts "this session stops recording" into "this store can no longer be restored": every mem::export → mem::import round-trip (backup restore, machine migration) for that store fails wholesale at the first validation gate, because the check is per-session and returns success: false for the entire payload, not just the offending session.

Reproduction and results (this machine)

  1. MAX_OBS_PER_SESSION=2000, local standalone daemon.
  2. One long session (e80c5b08-0801-4ead-9b5d-6b656ac2de23, project modules-web) reached 2000 observations. mem::observe then refused every subsequent observation with Session observation limit reached (2000); capture treated it as permanent and dead-lettered on the first attempt. /agentmemory/status showed 1143 dead letters, all from that one session.
  3. Raised the cap to 5000 (the highest value that stays import-compatible) and restarted. The session resumed and is now past 2374 and climbing — it will cross 5000 in time, at which point the whole store becomes un-importable and there is no reclaim path back.

Reproducing the import side directly only needs an export whose observations[sessionId].length > 5000; the validation gate is the first thing mem::import runs, so it fails before writing anything.

Required behavior

  • The import constant is renamed (e.g. MAX_IMPORT_OBS_PER_SESSION) so it stops colliding with the env var name, or the two are derived from one source.
  • At config load, if the runtime cap is greater than the import cap, either clamp it with a clear boot log line or emit a loud warning naming both values and the consequence. Silence is the bug.
  • When import rejects on the per-session count, the error names the offending sessionId, the observed count, and — if the runtime cap is configurable above it — points at MAX_OBS_PER_SESSION.
  • Independent of the above, a cap that can only ever be raised is not sustainable; fix: evict lowest-value observations at the session cap instead of refusing writes #1174 (evict at the cap instead of refusing) is the behavioural fix and is still open since 2026-08-12.

Acceptance checks

  • A config test asserts that MAX_OBS_PER_SESSION=5001 produces a warning (or a clamped value), and that 5000 does not.
  • An import test with one session holding 5001 observations asserts the failure message contains that session's id.
  • An import test with one session holding exactly 5000 observations asserts success (the current check is >), so the boundary stays where it is.

Source and existing work

  • Runtime config: src/index.ts — maxObservationsPerSession: safeParseInt(env["MAX_OBS_PER_SESSION"], 500)
  • Runtime enforcement: src/functions/observe.ts — if (existing.length >= maxObservationsPerSession)
  • Import constant + gate: src/functions/export-import.ts — const MAX_OBS_PER_SESSION = 5e3; and if (obs.length > MAX_OBS_PER_SESSION)
  • No reclaim: mem::consolidate and mem::session-sweep never delete observations; mem::evict caps at maxObservationsPerProject = 1e4

Related: #1484 (open — .env.example describes MAX_OBS_PER_SESSION as "cap before consolidation kicks in", which is wrong in the same area, but that issue is scoped to the comment), #1485 (closed — status advice for cap-rejected dead letters), #1174 (open — evict at the cap instead of refusing), #480 (closed — eviction timer / env toggle). None of them cover the import-side coupling; searching the tracker for Too many observations per session returns nothing.

Environment

agentmemory 0.9.30 (npm dist-tags.latest, published 2026-10-06 — i.e. the published artifact predates the readiness fix in #1531), Node v26.4.0, Windows 11 Pro Insider Preview (NT 10.0.26340), local standalone daemon under a watchdog task, file_based KV.

Activity

  1. akushonkamen commented on Oct 14, 2026

    @akushonkamen
    Contributor

    Confirmed — I can reproduce the coupling from source at HEAD 36eb328:

    • src/config.ts: runtime cap maxObservationsPerSession: safeParseInt(env["MAX_OBS_PER_SESSION"], 500) — user-settable, no upper bound, and the status tooling actively advises raising it.
    • src/functions/export-import.ts: a different constant that happens to share the name MAX_OBS_PER_SESSION = 5000, used as the first import validation gate: if (obs.length > MAX_OBS_PER_SESSION) rejects the whole payload with an error that doesn't name the offending session.
    • Nothing cross-validates the two at boot, so MAX_OBS_PER_SESSION=6000 starts silently and only surfaces when an export→import restore fails wholesale.

    Plan: rename the import-side constant to MAX_IMPORT_OBS_PER_SESSION, add a boot-time warning when the runtime cap exceeds the import cap, and include the offending sessionId + count in the import rejection error. Working on a fix now.

    Disclosure: I'm using AI coding assistance for this contribution (all changes reviewed and tested by me).

  2. akushonkamen commented on Oct 14, 2026

    @akushonkamen
    Contributor

    Opened #1556 with the fix described above: the import-side constant is renamed to MAX_IMPORT_OBS_PER_SESSION (moved to src/config.ts), loadConfig() warns once to stderr when MAX_OBS_PER_SESSION exceeds the ceiling or is set to 0, and the import rejection error now names the offending sessionId and count. The 5000 boundary (>) is unchanged.

    Verified: the two new test files fail on 36eb328 (3 failed / 3 passed) and pass with the fix (2 files / 6 tests); full suite vitest run test/ --exclude test/integration.test.ts → 3352 passed + 4 skipped, 0 failed; npm run build and npx tsc --noEmit clean (same 26 pre-existing errors as before the change).

  3. added a commit that references this issue on Oct 20, 2026
    a79fe1d
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions