fix(providers): echo reasoning content back for thinking-mode models - #489
Merged
Conversation
DeepSeek (and other thinking-mode OpenAI-compatible providers) reject a follow-up request whose conversation includes an assistant message that previously carried reasoning_content but is sent back without it: "The reasoning_content in the thinking mode must be passed back to the API." RustyClaw captured the reasoning only for the live thinking display and dropped it from the turn, so tool-loop rounds and history replay both failed with a 400 on the next request. - ModelResponse gains a reasoning field; the streaming path accumulates ReasoningChunk content and the batch path captures ContentPart::ReasoningContent. - The canonical assistant_tools envelope carries the reasoning, and decode_assistant emits a genai ReasoningContent part, which the genai OpenAI adapter serializes back as the sibling reasoning_content field (the Anthropic adapter echoes it as a thinking block, which its multi-turn API also requires). - assistant_content() picks plain text vs the envelope so bare text turns are unchanged while turns with reasoning/tool calls are echoed. - ThreadMessage persists the reasoning (serde-defaulted, skipped when absent), the gateway stores it on final assistant turns (dispatch, cron) and thread_history_to_chat_messages replays it on the next request, so follow-up turns and new sessions keep working.
add_assistant_with_tool_calls hardcoded reasoning: None, so a tool-call turn persisted to history lost its thinking text and the next user message (rebuilt from history) sent it back reasoning-less — the same 400 thinking-mode providers reject. The method now takes the reasoning and dispatch passes model_resp.reasoning alongside the tool calls.
The OpenAI adapter reports reasoning in ChatResponse::reasoning_content (a dedicated field), not as a content part, so the batch conversion never saw it — scheduled turns, Google calls and compaction/summary calls all take that path. Read reasoning_content into ModelResponse (keeping the part-based arm as a fallback) and cover it with a test.
The tool loop's per-round reasoning was concatenated (R1R2R3…) and attached to the single persisted assistant message, which is not what the API returned for any one turn; assign the last round's reasoning instead.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the chat error seen with DeepSeek (and other thinking-mode OpenAI-compatible providers):
Root cause
RustyClaw captured the model's reasoning only for the live thinking display and dropped it from the turn. DeepSeek requires the
reasoning_contentof earlier assistant messages to be echoed back verbatim on every follow-up request, so both the tool loop and next-turn history replay hit a 400.Changes
ModelResponsecarries the reasoning text: the streaming path accumulatesReasoningChunkevents, the batch path capturesContentPart::ReasoningContent.assistant_toolsenvelope stores the reasoning;decode_assistantre-emits it as a genaiReasoningContentpart, which the genai OpenAI adapter serializes back as the siblingreasoning_contentfield (the Anthropic adapter echoes it as a thinking block, which its multi-turn API also requires).assistant_content()helper: bare text turns stay plain text; turns carrying reasoning/tool calls use the envelope.ThreadMessagepersists the reasoning (serde-defaulted, skipped when absent — existingthreads.jsonfiles load unchanged). Final assistant turns are stored with their reasoning (main chat dispatch + cron runs), andthread_history_to_chat_messagesreplays it, so follow-up turns and fresh sessions work.No wire/protocol changes:
ChatMessageand all frames are untouched.