Skip to content

fix(compaction): mid-loop trigger uses force=true; guard slice underflow - #74

Merged
jonnyparris merged 1 commit into
mainfrom
fix/midloop-compaction-force
May 25, 2026
Merged

fix(compaction): mid-loop trigger uses force=true; guard slice underflow#74
jonnyparris merged 1 commit into
mainfrom
fix/midloop-compaction-force

Conversation

@jonnyparris

Copy link
Copy Markdown
Owner

What

Two small follow-ups uncovered by deploying #73 and watching wrangler tail on a real failed session.

1. Mid-loop trigger now passes force: true

The mid-loop compaction trigger in onChatMessage()'s own-loop was the only one of the three triggers (loop-entry, mid-loop, pre-step) not passing { force: true } to maybeCompactContext(). Because maybeCompactContext early-returns when lastInputTokens === 0 (no persisted assistant message yet) unless force is set, the mid-loop call was a silent no-op on the very first user turn — exactly the scenario the own-loop budget check was designed for. The 'mid-loop compaction triggered' log line still fired so the failure was invisible.

2. Slice underflow guard in maybeCompactContext

maybeCompactContext built messagesToCompact via realMessages.slice(0, compactCount) and then indexed messagesToCompact[compactCount - 1], which throws TypeError when compactCount > realMessages.length (e.g. force: true on a turn with only 1 persisted message). Now guarded with an early return + log line, and the toId lookup uses messagesToCompact.length - 1.

Why

Reproduction (Gemma 4 26B on prompt 'Check whether the Dodo auto-nudge feature is working as intended'):

own-loop: step complete  step=0  budgetUsage=10%
own-loop: step complete  step=1  budgetUsage=19%
own-loop: step complete  step=2  budgetUsage=29%
own-loop: step complete  step=3  budgetUsage=38%
own-loop: step complete  step=4  budgetUsage=48%
own-loop: step complete  step=5  budgetUsage=58%
own-loop: mid-loop compaction triggered  step=6  cumulative=118659
…end of stream, compactionCount=0, totalTokenInput=197535

The log line lied. maybeCompactContext() returned without compacting because lastInputTokens === 0 (first turn, no persisted assistant yet) and force wasn't passed.

Caveat

This does not fully unblock weak orchestrator models on the first turn. By design, compaction needs >=2 real persisted messages to have something meaningful to summarise. The deeper issue (single assistant turn ballooning to 200k tokens before any tool result is persisted, because Think persists only after streamText() completes the whole generator) is captured in memory/learnings/dodo-orchestrator-model-matters.md as a future investigation.

Even with this fix, expect Gemma 4 26B as orchestrator to keep failing — the underlying issue is the model not the safety nets. PR is still worth landing because the silent-noop is misleading and the slice-underflow is a real crash risk under force: true.

Tests

  • All 21 existing compaction tests pass (compaction-policy-unit, compaction-pipeline, compaction-e2e)
  • Typecheck clean

Linked context

beep-boop-🤖

Two small fixes uncovered by deploying PR #73 and watching wrangler tail
on a Gemma-driven session that ran 6 tool-call iterations:

1. The mid-loop compaction trigger in onChatMessage()'s own-loop was
   the only one of the three triggers NOT passing { force: true } to
   maybeCompactContext(). Because maybeCompactContext early-returns
   when lastInputTokens === 0 (no persisted assistant message yet)
   *unless* force is set, the mid-loop call was a silent no-op on the
   very first user turn — exactly the scenario the own-loop budget
   check was designed for. The log line still fired so the failure
   was invisible.

2. maybeCompactContext built messagesToCompact via
   realMessages.slice(0, compactCount) and then indexed
   messagesToCompact[compactCount - 1], which throws TypeError when
   compactCount > realMessages.length (e.g. force=true on a turn with
   only 1 persisted message). Now guarded with an early return + log
   line and the toId lookup uses messagesToCompact.length - 1.

Symptom in production: a Gemma session ran 6 tool-call iterations,
hit the 58% mid-loop trigger, logged 'mid-loop compaction triggered',
but compactionCount stayed at 0 and totalTokenInput climbed to 197k.
Verified the silent-noop path via wrangler tail.

Note: this fix alone does not fully unblock weak orchestrator models
on the first turn — by design, compaction needs >=2 real persisted
messages to have something to summarise. The deeper issue (single
assistant turn ballooning before any tool result is persisted) is
captured in memory/learnings/dodo-orchestrator-model-matters.md.

Tests: 21 existing compaction tests still pass; typecheck clean.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant