Session performance contract: constant-time opens, shallow reconciliation, and isolated history work

[ashwin-pc]

Summary

Consolidates #64 and #66 (both closed as superseded) and builds on #52 / f9e6373.

pi-web degrades as it runs: session history only grows (append-only JSONL), the server's known-cwd set only grows, and every session-list refresh re-parses the entire corpus on the main event loop. The longer the server runs, the more each refresh costs — so a healthy host slowly turns into multi-second stalls for session switching, page loads, and even trivial endpoints.

Design principle: steady-state cost must be independent of accumulated history and uptime. Eliminate unbounded work at the source instead of building infrastructure to contain it.

Measured facts (2026-07-30 instance)

• SessionManager.list(cwd) = 480–540 ms for one cwd (122 files, 71 MB); ~3.1 s for the full corpus (558 files, 465 MB). CPU-bound JSON parsing (~150 MB/s single-thread) on the main event loop.
• Trivial localhost endpoints stall up to ~9.6 s while a scan runs; the same endpoints take 1–25 ms when the loop is clear.
• A basic turn triggers ~2 full scans per visible client (message_end ×2 + agent_end inside the 250 ms debounce) — src/realtime/realtime.ts:86-95.
• allMessagesText (the largest allocation) is built for every session on every list and then discarded — pi-web never uses it; drawer search is client-side over the fetched list (src/sessions/sessionDrawer.ts:1813). pi-web has no server-side full-text search.
• Session filenames encode <created-ISO>_<id>.jsonl; files are strictly append-only; header, first user message, and initial name sit in the first ~1 KB (p90: 756 B / 463 B measured across 122 files).
• Uptime accumulation: knownCwds grows monotonically (server + client localStorage), so the scanned corpus widens over the server's lifetime. Same failure family as pi-web server becomes unresponsive over time: event-loop saturation from per-token broadcast fan-out + session-file fd leak #24.

Operations contract

Operation	Allowed cost
Open / switch	At most one transcript parse (the selected session). Never lists, never searches.
Reconcile / list	O(directory entries) + bounded head reads. Never parses transcript bodies.
Realtime maintenance	Patch/upsert/remove one row from authoritative events.
Deep work (search, exact enrichment)	Does not exist server-side today. If ever added: off the interactive event loop, cancellable, bounded queue.

Pi JSONL stays the only durable source of truth. No persistent metadata database or sidecar index.

Phase 1 — deletions and targeted fixes (small, immediate)

• Remove agent_end / compaction_end from shouldRefreshSessionsForPiEvent — neither changes any list field (runtime/stats/unread already have dedicated events).
• Rename: patch the one row from session_info_changed (the event carries the name); drop the explicit post-rename full refresh (src/status/statusBar.ts:80).
• Delete: remove the one row from session_deleted instead of rescanning.
• Targeted ID resolution: use the deterministic filename-suffix scan (resolveSessionLocation) before findSessionInfoById's sequential per-cwd lists and listAll() fallback (delete/open paths) — this alone removes the multi-second worst-case stall.
• Prevent a hung session-list request from freezing future refreshes #64 lifecycle: AbortController timeout on list requests; the single-flight slot is always released on success/failure/timeout; at most one queued trailing refresh.

Net effect: mostly code deletion; scans per turn drop; worst stalls gone. No semantics change.

Phase 2 — shallow listing (the load-bearing fix)

Replace the full-transcript parse in pi-web's list path with a shallow projection:

• readdir + stat per known cwd; order by mtime (ties by filename timestamp). mtime is used for ordering only, never as a cache invalidation.
• id + created from the filename; cwd/name/firstMessage from a bounded head read (first N entries / ≤32 KB — covers p90 at ~1 KB); optional small tail read to catch late renames; live sessions are served exactly from in-memory state.
• messageCount leaves the list contract: exact for live sessions (free, in memory), lazily enriched or omitted for cold sessions until opened. This is the only field whose exactness fundamentally requires reading the whole file.
• Response shape otherwise unchanged; client-side drawer search (name/firstMessage/cwd) keeps working.

Result: refresh cost is O(file count) — roughly 1–2 MB of bounded reads worst case (warm: milliseconds) — independent of transcript bytes and uptime. allMessagesText is never built. No cache, no worker, no new process.

Phase 3 — budget-driven polish (only if budgets still fail)

• Authoritative one-row metadata deltas on message persistence (patch instead of refetch; absolute values, replay-safe).
• Directory watcher for external pi CLI writers + rare recovery snapshot (bootstrap, sync_required, watcher overflow).
• Pagination cursor when directory entry counts reach the thousands.

Contingencies (explicitly not scheduled)

• Worker thread / child process offload: only if a genuinely unbounded on-loop operation remains after Phase 2 or is introduced later (e.g., a future server-side full-text search feature). After Phase 2, the only transcript parse left on the loop is cold-open of the one selected session — bounded, on explicit user action (~200 ms worst observed at 27 MB). Building worker infrastructure now would contain a cost that Phase 2 deletes.
• Append-only prefix-fold memoization (exact incremental messageCount/modified): sound because files are append-only, but deferred until a measured budget failure justifies the state.
• Persistent sidecar index: only as an upstream pi design where all writers maintain it; never pi-web-only.

Non-goals / rejected

• One worker thread per active or pinned session. Agent sessions are I/O-bound (model/network/tools); a thread per session duplicates heaps while still sharing the process failure boundary. Isolation of misbehaving extensions/tools belongs to the supervised runner-process architecture (Add runtime-bound sessions and runtime UI #43), which idle-reaps runners for executing sessions only.
• A separate persistent session metadata store in pi-web.
• Simplicity guardrail: no new long-lived subsystem (worker pool, watcher, index) may land without a failing budget from this issue justifying it. The end state must have fewer session-list code paths than today, not more.

Budgets / acceptance criteria

Validate against a fixture of ≥5,000 sessions / multi-GB corpus, and a long-uptime soak.

• Warm open: no filesystem reads, p95 server handling < 25 ms.
• Cold open: reads/parses only the selected transcript; latency independent of unrelated history size.
• Steady-state list refresh: p95 < 100 ms warm; cost independent of total transcript bytes (proved by truncate-vs-full corpus comparison).
• Basic turn: no full-transcript parsing; at most one shallow refresh (Phase 3 target: zero, via row deltas).
• Rename/delete: one-row updates; zero list snapshots.
• Trivial endpoints and WS events: p95 < 50 ms locally during any list/refresh activity.
• Uptime soak: after simulated long uptime (growing corpus + growing visited-cwd set), all budgets above still hold — steady-state cost is uptime-independent.
• Hung/timed-out list request always releases single-flight state; the next refresh succeeds (Prevent a hung session-list request from freezing future refreshes #64).
• Response growth is bounded (pagination contract exists before entry counts reach thousands).

Regression tests

• Corpus-independence: list latency unchanged when transcript bodies are inflated 10×.
• Turn fanout: no full parse and ≤1 shallow refresh per turn; zero on rename/delete.
• Shallow-lister parity: name/firstMessage/created/modified/cwd match SessionManager.list on real fixtures; the messageCount and deep-rename deltas are documented and asserted.
• Hung-first-request retry (Prevent a hung session-list request from freezing future refreshes #64 scenario).
• Soak test with growing cwd set and corpus.

Related

• Decouple session open from discovery so tab switching stays fast as history grows #52 — open/discovery decoupling (implemented: f9e6373, 1ffdf17); this issue protects that critical path from event-loop starvation.
• Paginate session listing and avoid full-transcript parsing in /api/sessions #66, Prevent a hung session-list request from freezing future refreshes #64 — superseded by this issue.
• pi-web server becomes unresponsive over time: event-loop saturation from per-token broadcast fan-out + session-file fd leak #24 — prior uptime-accumulation incident (live-session/fd leaks); same failure family, fixed via leases.
• Tab switching is slow: conversation fully re-rendered 2-3x per switch (~5s main-thread block on long sessions) #10 — client-side transcript render cost (separate).
• Add runtime-bound sessions and runtime UI #43 — runner-process architecture; owns active-session isolation.