Skip to content

fix(own-loop): prune fires on cumulative tokens; compaction logs truthfully - #76

Merged
jonnyparris merged 1 commit into
mainfrom
fix/prune-on-cumulative
May 25, 2026
Merged

fix(own-loop): prune fires on cumulative tokens; compaction logs truthfully#76
jonnyparris merged 1 commit into
mainfrom
fix/prune-on-cumulative

Conversation

@jonnyparris

Copy link
Copy Markdown
Owner

What

Two related fixes found by deploying #75 and watching wrangler tail on a Gemma session.

1. Prune trigger considers cumulative tokens (the actual failure mode)

PR #75 gated the prune on projectedInputTokens >= threshold — the size of the next outbound message array. For typical Gemma sessions each per-step call is ~19k tokens (system prompt + tool defs dominate), so the per-iteration projection stays small even as the cumulative climb burns through the budget across many iterations.

Empirical reproduction (8 successful steps observed, no prune log fired):

step 0  cumulative=19569   budget=10%   tools=[explore]
step 1  cumulative=39208   budget=19%
step 2  cumulative=58897   budget=29%
step 3  cumulative=78665   budget=38%
step 4  cumulative=98512   budget=48%
step 5  cumulative=118492  budget=58%   ← mid-loop trigger fires
step 6  cumulative=138061  budget=67%
step 7  cumulative=157700  budget=77%   ← wrap-up warning fires
step 8: ...model bails

Now triggers on EITHER signal. Log gains trigger: cumulative | projected.

2. Compaction logs distinguish 'summarised' vs 'noop'

The own-loop's three compaction trigger sites (loop-entry, mid-loop, pre-step) all logged compaction triggered unconditionally — even when maybeCompactContext() no-op'd because Think's persisted history had nothing to summarise. That made the bug invisible. We hit this twice already (see PR #74's caveat).

Now each site:

  1. Reads getCompactionCount() before the attempt
  2. Calls maybeCompactContext
  3. Reads again, logs outcome: "summarised" | "noop"

Log name also changed from *compaction triggered*compaction attempted for honesty.

New private helper getCompactionCount() keeps the call sites clean.

Tests

All 28 existing compaction + prune tests pass. Typecheck clean. No new tests — both changes are observability + trigger-condition fixes, not new algorithms.

Linked

beep-boop-🤖

…hfully

Two related fixes that close the gap between PR #75's prune helper and
its actual deployment behaviour, found by deploying #75 and watching
`wrangler tail` on a Gemma session.

## 1. Prune trigger now considers cumulative tokens, not just projected

PR #75's prune block was gated on
`projectedInputTokens / tokenBudget >= MID_LOOP_COMPACTION_THRESHOLD`,
where `projectedInputTokens` measures the size of the next outbound
message array. For typical Gemma sessions each per-step call is ~19k
tokens (system prompt + tool defs dominate), so the per-iteration
projection stays well below the 50% threshold even as the cumulative
total climbs through 58 → 67 → 77 → 87%. The prune never fired.

Now triggers on EITHER signal:
- projected: any single iteration is already huge
- cumulative: the running total of inputs sent this turn has climbed past threshold

The cumulative signal is the one that actually matches the Gemma failure
mode. Log line gains a `trigger: cumulative | projected` field.

## 2. Compaction logs distinguish summarised vs noop

The own-loop's compaction trigger sites all logged "compaction triggered"
unconditionally — even when `maybeCompactContext()` no-op'd because Think's
persisted history had nothing to summarise. That made the bug invisible
during debugging (we hit this twice — first time was PR #74's diagnosis).

All three trigger sites (loop-entry, mid-loop, pre-step) now:
1. Read `getCompactionCount()` before the attempt
2. Call `maybeCompactContext`
3. Read again, log `outcome: "summarised" | "noop"`

The log line name also changed from `*compaction triggered` to
`*compaction attempted` for honesty.

New private helper `getCompactionCount()` wraps the
`sessions.getCompactions(thinkSessionId).length` lookup so the call sites
read clean.

## Tests

All 28 compaction + prune tests pass; typecheck clean. No new tests:
both changes are observability + trigger-condition fixes, not new
algorithms. The behavioural change (prune firing on cumulative) is
covered by the existing prune unit tests via the `estimate` callback.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant