fix(compaction): mid-loop trigger uses force=true; guard slice underflow - #74
Merged
Conversation
Two small fixes uncovered by deploying PR #73 and watching wrangler tail on a Gemma-driven session that ran 6 tool-call iterations: 1. The mid-loop compaction trigger in onChatMessage()'s own-loop was the only one of the three triggers NOT passing { force: true } to maybeCompactContext(). Because maybeCompactContext early-returns when lastInputTokens === 0 (no persisted assistant message yet) *unless* force is set, the mid-loop call was a silent no-op on the very first user turn — exactly the scenario the own-loop budget check was designed for. The log line still fired so the failure was invisible. 2. maybeCompactContext built messagesToCompact via realMessages.slice(0, compactCount) and then indexed messagesToCompact[compactCount - 1], which throws TypeError when compactCount > realMessages.length (e.g. force=true on a turn with only 1 persisted message). Now guarded with an early return + log line and the toId lookup uses messagesToCompact.length - 1. Symptom in production: a Gemma session ran 6 tool-call iterations, hit the 58% mid-loop trigger, logged 'mid-loop compaction triggered', but compactionCount stayed at 0 and totalTokenInput climbed to 197k. Verified the silent-noop path via wrangler tail. Note: this fix alone does not fully unblock weak orchestrator models on the first turn — by design, compaction needs >=2 real persisted messages to have something to summarise. The deeper issue (single assistant turn ballooning before any tool result is persisted) is captured in memory/learnings/dodo-orchestrator-model-matters.md. Tests: 21 existing compaction tests still pass; typecheck clean.
jonnyparris
marked this pull request as ready for review
May 25, 2026 08:21
This was referenced May 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two small follow-ups uncovered by deploying #73 and watching
wrangler tailon a real failed session.1. Mid-loop trigger now passes
force: trueThe mid-loop compaction trigger in
onChatMessage()'s own-loop was the only one of the three triggers (loop-entry, mid-loop, pre-step) not passing{ force: true }tomaybeCompactContext(). BecausemaybeCompactContextearly-returns whenlastInputTokens === 0(no persisted assistant message yet) unlessforceis set, the mid-loop call was a silent no-op on the very first user turn — exactly the scenario the own-loop budget check was designed for. The'mid-loop compaction triggered'log line still fired so the failure was invisible.2. Slice underflow guard in
maybeCompactContextmaybeCompactContextbuiltmessagesToCompactviarealMessages.slice(0, compactCount)and then indexedmessagesToCompact[compactCount - 1], which throwsTypeErrorwhencompactCount > realMessages.length(e.g.force: trueon a turn with only 1 persisted message). Now guarded with an early return + log line, and thetoIdlookup usesmessagesToCompact.length - 1.Why
Reproduction (Gemma 4 26B on prompt 'Check whether the Dodo auto-nudge feature is working as intended'):
The log line lied.
maybeCompactContext()returned without compacting becauselastInputTokens === 0(first turn, no persisted assistant yet) andforcewasn't passed.Caveat
This does not fully unblock weak orchestrator models on the first turn. By design, compaction needs >=2 real persisted messages to have something meaningful to summarise. The deeper issue (single assistant turn ballooning to 200k tokens before any tool result is persisted, because Think persists only after streamText() completes the whole generator) is captured in
memory/learnings/dodo-orchestrator-model-matters.mdas a future investigation.Even with this fix, expect Gemma 4 26B as orchestrator to keep failing — the underlying issue is the model not the safety nets. PR is still worth landing because the silent-noop is misleading and the slice-underflow is a real crash risk under
force: true.Tests
compaction-policy-unit,compaction-pipeline,compaction-e2e)Linked context
beep-boop-🤖