feat(own-loop): explicit stop guidance + same-tool repetition detector - #78
Merged
Merged
Conversation
Two related changes that together make weak orchestrators decide when to stop calling tools and write their conclusion. Found by deploying PR #77 and watching a Gemma session run 18 consecutive codemode calls without ever producing a text answer. ## 1. System prompt stop condition New rule #7 in the operating rules section, with four concrete cases: - 3 consecutive tool calls producing no new useful info → write and stop - Question answered → don't run more tools to 'verify' - Tool returning errors → write what failed and stop - Final reply must always be plain text, never a tool call Existing rules renumbered 8/9/10. ## 2. Same-tool repetition detector Layer between the existing doom-loop detector (identical tool+args) and the no-text loop (Anthropic-only). Catches the pattern: > Same tool name N times in a row, with DIFFERENT args, no text answer. Two-stage response, both provider-agnostic: 1. **Nudge** at SAME_TOOL_NUDGE_THRESHOLD (6): inject a [STOP-CHECK] system message asking the model whether it's done. The nudge fires ONCE per turn — if the model continues legitimately, it's not pestered repeatedly. 2. **Hard-break** at SAME_TOOL_HARD_BREAK_THRESHOLD (10): exit the loop with exitReason=doom-loop and a wrap-up text-delta telling the model to write its conclusion from what it has. The detector lives in src/loop-detection.ts as a pure function (detectSameToolRepetition) so it's testable without booting the DO. Reads from the same recentToolCalls buffer the doom-loop check uses, with retention bumped to cover the new harder threshold. ## Tests - 10 unit tests for detectSameToolRepetition: edge cases (empty, short buffer, degenerate threshold), happy paths (same tool with different args), negative cases (streak broken by other tool), and args-handling (JSON with colons doesn't confuse the split). - All 28 existing compaction + prune tests still pass. - Typecheck clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two complementary changes that make weak orchestrators decide when to stop calling tools and write their conclusion.
1. System prompt stop condition
New rule #7 in the operating rules. Four concrete cases the model should treat as 'time to stop':
2. Same-tool repetition detector
A new layer between the existing doom-loop detector (identical tool+args) and the Anthropic-only no-text loop:
Two-stage:
SAME_TOOL_NUDGE_THRESHOLD = 6— inject[STOP-CHECK]system message asking 'do you have enough information?' Fires once per turn.SAME_TOOL_HARD_BREAK_THRESHOLD = 10— exit with a wrap-up text-delta.Provider-agnostic. Pure function
detectSameToolRepetitionin newsrc/loop-detection.ts.Why
Reproduced today on Gemma 4 26B post-PR-77 deploy:
18 consecutive codemode calls with different args, zero text output, never wrapping up. Doom-loop didn't fire because args differed. No-text loop didn't fire because it's gated on
anthropic/models. Budget thresholds didn't fire because projected next-call size stayed reasonable (PR #77 working as intended). The session would have run forever — only the new cost runaway backstop at 5× budget would eventually catch it, after ~1M tokens of waste.Tests
detectSameToolRepetitioncovering: empty buffer, short buffer, degenerate threshold, happy path (same tool different args), negative case (streak broken), tail-window isolation (earlier mixed entries don't affect outcome), args containing colons.Linked
memory/learnings/dodo-orchestrator-model-matters.mdbeep-boop-🤖