Skip to content

fix(providers): echo reasoning content back for thinking-mode models - #489

Merged
rexlunae merged 7 commits into
mainfrom
fix/reasoning-echo
Aug 18, 2026
Merged

fix(providers): echo reasoning content back for thinking-mode models#489
rexlunae merged 7 commits into
mainfrom
fix/reasoning-echo

Conversation

@rexlunae

@rexlunae rexlunae commented Aug 18, 2026

Copy link
Copy Markdown
Owner

Fixes the chat error seen with DeepSeek (and other thinking-mode OpenAI-compatible providers):

Web stream error for model 'deepseek-v4-flash (adapter: OpenAI)'.
Status: 400 Bad Request
Body: {"error":{"message":"The `reasoning_content` in the thinking mode must be passed back to the API."}}

Root cause

RustyClaw captured the model's reasoning only for the live thinking display and dropped it from the turn. DeepSeek requires the reasoning_content of earlier assistant messages to be echoed back verbatim on every follow-up request, so both the tool loop and next-turn history replay hit a 400.

Changes

  • ModelResponse carries the reasoning text: the streaming path accumulates ReasoningChunk events, the batch path captures ContentPart::ReasoningContent.
  • The canonical assistant_tools envelope stores the reasoning; decode_assistant re-emits it as a genai ReasoningContent part, which the genai OpenAI adapter serializes back as the sibling reasoning_content field (the Anthropic adapter echoes it as a thinking block, which its multi-turn API also requires).
  • New assistant_content() helper: bare text turns stay plain text; turns carrying reasoning/tool calls use the envelope.
  • ThreadMessage persists the reasoning (serde-defaulted, skipped when absent — existing threads.json files load unchanged). Final assistant turns are stored with their reasoning (main chat dispatch + cron runs), and thread_history_to_chat_messages replays it, so follow-up turns and fresh sessions work.

No wire/protocol changes: ChatMessage and all frames are untouched.


Open in Devin Review

DeepSeek (and other thinking-mode OpenAI-compatible providers) reject a
follow-up request whose conversation includes an assistant message that
previously carried reasoning_content but is sent back without it:
"The reasoning_content in the thinking mode must be passed back to the
API." RustyClaw captured the reasoning only for the live thinking
display and dropped it from the turn, so tool-loop rounds and history
replay both failed with a 400 on the next request.

- ModelResponse gains a reasoning field; the streaming path accumulates
  ReasoningChunk content and the batch path captures
  ContentPart::ReasoningContent.
- The canonical assistant_tools envelope carries the reasoning, and
  decode_assistant emits a genai ReasoningContent part, which the genai
  OpenAI adapter serializes back as the sibling reasoning_content field
  (the Anthropic adapter echoes it as a thinking block, which its
  multi-turn API also requires).
- assistant_content() picks plain text vs the envelope so bare text
  turns are unchanged while turns with reasoning/tool calls are echoed.
- ThreadMessage persists the reasoning (serde-defaulted, skipped when
  absent), the gateway stores it on final assistant turns (dispatch,
  cron) and thread_history_to_chat_messages replays it on the next
  request, so follow-up turns and new sessions keep working.
devin-ai-integration[bot]

This comment was marked as resolved.

add_assistant_with_tool_calls hardcoded reasoning: None, so a tool-call
turn persisted to history lost its thinking text and the next user
message (rebuilt from history) sent it back reasoning-less — the same
400 thinking-mode providers reject. The method now takes the reasoning
and dispatch passes model_resp.reasoning alongside the tool calls.
devin-ai-integration[bot]

This comment was marked as resolved.

The OpenAI adapter reports reasoning in ChatResponse::reasoning_content (a
dedicated field), not as a content part, so the batch conversion never
saw it — scheduled turns, Google calls and compaction/summary calls all
take that path. Read reasoning_content into ModelResponse (keeping the
part-based arm as a fallback) and cover it with a test.
devin-ai-integration[bot]

This comment was marked as resolved.

The tool loop's per-round reasoning was concatenated (R1R2R3…) and
attached to the single persisted assistant message, which is not what the
API returned for any one turn; assign the last round's reasoning instead.
@rexlunae
rexlunae merged commit c3d746d into main Aug 18, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant