Repository navigation
fix(events): snapshot cadence follow-ups (history doubling, open counts, deferred checks) - #1
Conversation
A snapshot is built from the previous one plus the events since (combineEventsWithSnapshot). That stream also replays the previous snapshot's compactions and format retries as synthetic events, just before it, for the consumers that count them; the fold then seeded its history from the snapshot on top of them, so each snapshot doubled the history and every later one doubled it again. With the mid-turn cadence snapshotting often, it exploded: seen live, 7 compactions stored as 2.3 million entries in a 433 MB snapshot, rewritten at every turn end (8 s per write, a prompt took that long to show) until the disk was full. The fold keeps one entry per compaction or retry (same windows or attempt, same timestamp) when it seeds from a snapshot. This also shrinks a snapshot already inflated that way on the next snapshot. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A successful compaction starts its summary attempt's message again with the summary as content, then closes it once. The tail counted starts minus dones, so every compaction left one message open for good: the snapshot cadence then found the turn busy at every check and never took a mid-turn snapshot for the rest of the turn (main turn or sub-agent). The tail now counts the ids started and not done in it (and tool calls without a result), instead of subtracting counts: a message started twice counts once, and a done or result whose start is before the tail can no longer hide another open message or pending call. The counts read only their own event types through the session/type/seq index; the check measured 30 ms on a 48 MB tail (the default budget), 64 ms on 100 MB, 269 ms on 400 MB, dominated by summing the tail's payload bytes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The cadence measures the tail at most every 30 s, on whatever event comes in, and only snapshots at a quiet point (no message streaming, no tool round in flight). A deferred check still restarted the 30 s period, and a sub-agent is nearly always streaming or running a tool: the check kept landing on a busy moment and the snapshot was put off to the end of the turn. Seen live on 7 and 12 minute sub-agent runs, the tail over its budget and no mid-turn snapshot (together with the open-message miscount fixed in the next commit). A check that finds the tail over budget while the turn is busy now marks the cadence deferred, and the next quiet point (a message.done or a tool.result) checks again without waiting for the period, at most every 5 s: each check scans the tail (~0.1 s at 46 MB), and parallel tools or sub-agents can keep the turn busy over many quiet points in a row. The budget is evaluated before the in-flight test, so only an over-budget tail defers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
One defensive nit on the new set-based counts: the |
`x NOT IN (subquery)` is never true when the subquery returns a NULL: a single message.done or tool.result without an id in the tail would read openMessages and pendingToolCalls as 0, and the cadence would snapshot mid-message. The subqueries now skip NULL ids. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Good catch, fixed in 72d614e: the |
|
Reviewed Full suite on the resulting tip, on my side: 443 files / 5,994 unit tests and 49 files / 357 e2e passing, plus One nit, same class as the one I reported and purely theoretical: the co-l#4 is tracked in #2. Thanks again — still the most useful review this PR got. |
Follow-ups to co-l#401, as discussed in the thread: fixes 1–3, one commit each, each with its own test (written first, failing before the fix). co-l#4 comes later, once the sub-agent compaction fix lands.
takeSnapshot()builds fromcombineEventsWithSnapshot(), whose stubs replay the previous snapshot'scontextWindows/formatRetriesjust before it; the fold reseeds from the snapshot on top of them, so each snapshot doubles the history (1 → 2 → 4 → 8 over fourflush()es ata662baed; seen live: 7 compactions as 2.3M entries, a 433 MB snapshot)session_id, event_type, seq): 30 ms on a 48 MB tail, 64 ms on 100 MB, 269 ms on 400 MB, mostly the byte sumlastCheckAt, so a sub-agent that is nearly always busy usually missed the next 30 s check toomessage.done/tool.result) checks again, at most every 5 sVerified: lint, typecheck, format, duplicate and the full suite (unit and e2e) pass on each commit, through the pre-commit hook.
AI-Enhanced Development
Cache Impact
🤖 Generated with Claude Code