feat(own-loop): in-memory tool-result prune harness safety net - #75
Merged
Conversation
The harness fix for the failure mode that breaks Gemma + Kimi K2.6 as orchestrators on first-turn explorations. ## Problem Think persists the assistant message only AFTER streamText() completes the whole generator. On a long-running first user turn, sessions.getHistory() returns just [user] — realMessages.length === 1. The autocompaction summariser has nothing to summarise. Meanwhile the own-loop's closure variable `messages` accumulates tool-result entries across iterations. Weak orchestrators (Gemma 4 26B, Kimi K2.6) routinely make many tool calls in a single turn whose results compound the local array to 200k+ tokens. The pre-step hard-stop then kicks in before the model can finish, ending the session with "context budget exhausted". ## Fix New pure helper `pruneOversizedToolResults` in src/own-loop-prune.ts: - Walks the own-loop's in-memory messages array - Replaces the largest tool-result payloads with short placeholders - Preserves the tool-call envelope (toolName, toolCallId) so the model's pending tool-call still resolves - Prunes oldest-first, preserves the most recent N (default 2) - Stops as soon as projected tokens are back under target Wired into the pre-step budget check in onChatMessage's own-loop, running AFTER the Think-level compaction attempt and BEFORE the hard-stop. The order is intentional: compaction handles previous turns (cheap when it works), prune handles the current turn (the case compaction can't fix). No LLM calls, no storage writes. The placeholder tells the model the data was pruned and to re-run the tool if it still needs it. ## Why this is the right fix PR #73 made autocompaction reachable on the first turn. PR #74 made it actually call into the summariser with force=true. This PR closes the remaining gap: compaction's input source (Think's persisted history) is structurally empty on turn 1; the own-loop's in-memory history is where the bytes actually live. ## Tests - 7 new unit tests for pruneOversizedToolResults covering: under-budget no-op, oldest-largest first, preservation of recent N, envelope preservation, min-payload-chars floor, early stop, no tool messages - All 21 existing compaction tests still pass - Typecheck clean
This was referenced May 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A no-LLM, no-storage harness safety net that keeps the own-loop's in-memory
messagesarray under the context budget when Think-level compaction can't help.Why
Reproduced this morning by deploying #73 and watching
wrangler tail:messagesarrayrealMessages.length === 1PR #73 + PR #74 fixed compaction's reachability and silent-no-op bugs. This PR closes the remaining gap by attacking the actual bytes — the own-loop's in-memory tool-result payloads.
How
New module
src/own-loop-prune.tsexportingpruneOversizedToolResults(messages, opts):valueof each largetool-resultpart with a short placeholder string. Keeps the envelope (toolName,toolCallId) intact so the model's prior tool-call still resolves.targetTokens,preserveRecentToolMessages,minPrunablePayloadChars.Wired into
coding-agent.tsonChatMessage own-loop, between the pre-step Think compaction attempt and the hard-stop:Tests
test/own-loop-prune-unit.test.ts— 7 cases:All 21 existing compaction tests still pass; typecheck clean.
Risk
minPrunablePayloadChars: 2000keeps small results intact.Linked
memory/learnings/dodo-orchestrator-model-matters.mdmemory/workload/ideas-backlog.md→ 'Cap single-turn input-token growth' — this is that workbeep-boop-🤖