fix(own-loop): prune fires on cumulative tokens; compaction logs truthfully - #76
Merged
Conversation
…hfully Two related fixes that close the gap between PR #75's prune helper and its actual deployment behaviour, found by deploying #75 and watching `wrangler tail` on a Gemma session. ## 1. Prune trigger now considers cumulative tokens, not just projected PR #75's prune block was gated on `projectedInputTokens / tokenBudget >= MID_LOOP_COMPACTION_THRESHOLD`, where `projectedInputTokens` measures the size of the next outbound message array. For typical Gemma sessions each per-step call is ~19k tokens (system prompt + tool defs dominate), so the per-iteration projection stays well below the 50% threshold even as the cumulative total climbs through 58 → 67 → 77 → 87%. The prune never fired. Now triggers on EITHER signal: - projected: any single iteration is already huge - cumulative: the running total of inputs sent this turn has climbed past threshold The cumulative signal is the one that actually matches the Gemma failure mode. Log line gains a `trigger: cumulative | projected` field. ## 2. Compaction logs distinguish summarised vs noop The own-loop's compaction trigger sites all logged "compaction triggered" unconditionally — even when `maybeCompactContext()` no-op'd because Think's persisted history had nothing to summarise. That made the bug invisible during debugging (we hit this twice — first time was PR #74's diagnosis). All three trigger sites (loop-entry, mid-loop, pre-step) now: 1. Read `getCompactionCount()` before the attempt 2. Call `maybeCompactContext` 3. Read again, log `outcome: "summarised" | "noop"` The log line name also changed from `*compaction triggered` to `*compaction attempted` for honesty. New private helper `getCompactionCount()` wraps the `sessions.getCompactions(thinkSessionId).length` lookup so the call sites read clean. ## Tests All 28 compaction + prune tests pass; typecheck clean. No new tests: both changes are observability + trigger-condition fixes, not new algorithms. The behavioural change (prune firing on cumulative) is covered by the existing prune unit tests via the `estimate` callback.
This was referenced May 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two related fixes found by deploying #75 and watching
wrangler tailon a Gemma session.1. Prune trigger considers cumulative tokens (the actual failure mode)
PR #75 gated the prune on
projectedInputTokens >= threshold— the size of the next outbound message array. For typical Gemma sessions each per-step call is ~19k tokens (system prompt + tool defs dominate), so the per-iteration projection stays small even as the cumulative climb burns through the budget across many iterations.Empirical reproduction (8 successful steps observed, no prune log fired):
Now triggers on EITHER signal. Log gains
trigger: cumulative | projected.2. Compaction logs distinguish 'summarised' vs 'noop'
The own-loop's three compaction trigger sites (loop-entry, mid-loop, pre-step) all logged
compaction triggeredunconditionally — even whenmaybeCompactContext()no-op'd because Think's persisted history had nothing to summarise. That made the bug invisible. We hit this twice already (see PR #74's caveat).Now each site:
getCompactionCount()before the attemptmaybeCompactContextoutcome: "summarised" | "noop"Log name also changed from
*compaction triggered→*compaction attemptedfor honesty.New private helper
getCompactionCount()keeps the call sites clean.Tests
All 28 existing compaction + prune tests pass. Typecheck clean. No new tests — both changes are observability + trigger-condition fixes, not new algorithms.
Linked
memory/learnings/dodo-orchestrator-model-matters.mdbeep-boop-🤖