Repository navigation
Conversation
The reasoning steps of WithForceReasoning (before the tool pick and before parameter generation) send no output cap of their own, so a model that never closes the reasoning argument runs to the client's global cap (16384 by default), and the length retry then doubles it. Observed with gemma4-26b-a4b behind LocalAI: one forced reasoning call streamed 12,969 tool-call deltas for 478-501 s until the cap, and the turn never answered. The reasoning steps now carry max_tokens from WithReasoningMaxTokens (default DefaultReasoningMaxTokens = 2048; 0 uses the default, a negative value keeps the previous uncapped behaviour). The cap goes on the request, where the bundled clients take it over their own default (see SetMaxTokens). A finish_reason=length on such a capped step is final: no length retry, the call returns an error wrapping errOutputCapReached; a complete tool call that is cut at the cap is still used. The parameter-reasoning step already falls back on any error; the tool-pick step now continues without the reasoning on this error only, other errors stay fatal. A reasoning that finishes normally is used as before. The option is forwarded to plan sub-executions. Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> Assisted-by: Claude Opus 5.5 <noreply@anthropic.com>
walcz-de
added a commit
to walcz-de/LocalAI
that referenced
this pull request
Oct 2, 2026
cogito 0400c22: forced reasoning steps get their own output cap (WithReasoningMaxTokens, default 2048) — mudler/cogito#84. A reasoning call ran 478-501 s to the 16384-token cap and the agent never answered. Assisted-by: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
With
WithForceReasoning, the forcedreasoningcalls before the tool pick and before parameter generation carry no output cap of their own. A model that never closes the reasoning argument therefore runs to the client's global cap (16384 by default), and the length retry then doubles it. With gemma4-26b-a4b behind LocalAI, one such call streamed ~13k tool-call deltas for ~8 minutes and the turn never answered.This PR adds
WithReasoningMaxTokensand sends it asmax_tokenson both reasoning calls, so the request-level cap wins over the client default (seeSetMaxTokens):DefaultReasoningMaxTokens = 20480uses the defaultA
finish_reason=lengthon a capped step is final, with no length retry:All other decisions are unchanged, and the option is forwarded to plan sub-executions.
Tests:
make testpasses (88/88).🤖 Generated with Claude Code
https://claude.ai/code/session_01Y8ngj32vPUcLs3QKgTYCxy