fix(own-loop): budget thresholds use projected context, not cumulative cost

[jonnyparris]

What

Make the wrap-up / hard-stop / warn thresholds fire on the size of the message array we're about to send to streamText, not on the running sum of input tokens we've spent this turn.

Why

The two metrics coincide for providers with prompt caching (Anthropic) but diverge sharply for providers without it (Workers AI: Gemma, Kimi). For a typical Gemma session each per-step inputTokens is ~20k regardless of how much real history exists — system prompt + tool definitions dominate the per-call cost. By step 8 the cumulative sum is ~158k (77% of the 205k budget) and the wrap-up injection fires telling the model to stop. But the actual message array for the next call is also ~20k — the model has ~180k of context window free.

Today's reproduction (commit 41324ab):

step 0  cumulative=19569   budget=10%
step 1  cumulative=39208   budget=19%
step 2  cumulative=58895   budget=29%
step 3  cumulative=78795   budget=38%
step 4  cumulative=98815   budget=48%
step 5  cumulative=118955  budget=58%  ← mid-loop compaction attempted (noop)
step 6  cumulative=138524  budget=68%
step 7  cumulative=158162  budget=77%  ← budget warning injection fires
step 8  cumulative=177894  budget=87%  ← wrap-up injection fires
        Gemma reads 'context budget nearly exhausted' and stops.

At step 8 the actual next-call message array was ~20k. Model had ~180k of room.

How

// Before
const budgetUsage = cumulativeInputTokens / tokenBudget;

// After
const projectedNextCallTokens = estimateMessagesTokens(messages);
const budgetUsage = projectedNextCallTokens / tokenBudget;

cumulativeInputTokens is kept for telemetry — still logged on every step-complete and on the threshold injections — but no longer gates them.

Cost runaway backstop

Without cumulative-based gating, a model in a non-doom-loop loop could burn unbounded tokens. New absolute backstop: cumulativeInputTokens >= tokenBudget * 5 (e.g. 1M tokens for a 200k model) exits with exitReason: budget-limit and a 'cost runaway' message. Generous for real multi-step work, tight enough to catch billing bombs.

What this unblocks

Gemma sessions that previously stopped at step 8 with 'context exhausted' should now keep iterating until either:

• the model decides it's done
• per-call projected context actually fills up
• the cost runaway backstop fires

Strong models with prompt caching won't notice (cumulative ≈ projected for them). Weak models with no caching get the full benefit.

Tests

All 28 existing compaction + prune tests pass. Typecheck clean.

The budget threshold logic isn't directly unit-tested (it lives inside onChatMessage's generator). Behavioural validation via post-deploy Gemma re-run on the failing prompts.

Linked

• PR fix(compaction): force/truncated bypass the minMessages guard #73, fix(compaction): mid-loop trigger uses force=true; guard slice underflow #74, feat(own-loop): in-memory tool-result prune harness safety net #75, fix(own-loop): prune fires on cumulative tokens; compaction logs truthfully #76 (compaction + prune harness chain)
• memory/learnings/dodo-orchestrator-model-matters.md

beep-boop-🤖