feat(translator): preserve multi-turn reasoning context for Gemini/Antigravity in OpenAI-compatible endpoint

[pinewhite]

Problem & Background

When downstream clients (autonomous agents like OpenCode, Claude Code, Cline, or Chat UIs like Cherry Studio/NextChat) communicate with CLIProxyAPI (CPA) via the standard OpenAI-compatible endpoint (/v1/chat/completions), multi-turn Chain-of-Thought (CoT) continuity is currently lost for reasoning models—specifically Google Gemini (via Antigravity / Cloud Code).

In multi-turn agent workflows (e.g. planning -> tool call -> tool result -> reasoning continuation):

1. The model outputs internal thinking, which CPA streams to the client as delta.reasoning_content.
2. In subsequent turns, standard OpenAI-compatible clients send back the conversation history containing:

   {
     "role": "assistant",
     "content": "Final answer...",
     "reasoning_content": "Intermediate planning, calculations, or hypothesis..."
   }

3. During upstream protocol translation (e.g., converting OpenAI messages into Google Gemini's generateContent format), the reasoning_content field is stripped/discarded. As a result, the model completely loses awareness of its prior reasoning steps, causing tool-call loops or reasoning amnesia in complex multi-turn workflows.

---

Empirical Evidence & Live A/B Verification

We conducted live A/B testing on a running CPA instance (v7.2.157) using gemini-3.8-flash-high (Antigravity provider):

Test 1: Native Gemini Endpoint (/v1beta/models/...:generateContent)

• Setup:
  • Turn 1: Instructed Gemini to calculate an intermediate variable (1024 * 82 + 952 = 84920) inside its thought process, outputting only a confirmation message in the final content.
  • Turn 2: Fed back the native response structure (including the thought: true part and the 2172-byte thoughtSignature).
  • Asked the model to recall the calculated base offset from Turn 1.
• Observed Result:
  • promptTokenCount rose from 65 to 270 (thought tokens were fully accounted for in the context).
  • Turn 2 Output: 84920 (the model accurately retrieved the intermediate value that existed solely within the prior thought).
  • Conclusion: Native Gemini explicitly supports and requires thought inheritance when a valid thoughtSignature is preserved.

Test 2: OpenAI Endpoint (/v1/chat/completions)

• Setup: The exact same two-turn task was sent via /v1/chat/completions, with the assistant's previous thought passed in reasoning_content.
• Observed Result:
  • prompt_tokens was only 86 (the thought content was entirely excluded from the prompt).
  • Turn 2 Output: 未提及 (Not mentioned); the model claimed no calculation had occurred.
  • Conclusion: Because OpenAI messages lack Google's proprietary cryptographic thoughtSignature, CPA strips reasoning_content during translation to prevent upstream HTTP 400 (Invalid/Missing thoughtSignature) errors, resulting in reasoning context loss.

---

Root Cause Analysis

1. Protocol Impedance Mismatch:
   • Google Gemini 2.5 / 3.x requires a server-signed thoughtSignature attached to thought parts to ensure thinking authenticity and prevent prompt injection.
   • The standard OpenAI Chat Completions API has no native field for binary cryptographic signatures.
2. Current Proxy Behavior:
   • To avoid upstream 400 rejection on unverified thought parts, CPA safely drops reasoning_content when translating OpenAI messages to Google contents.parts.
   • While this prevents crashes, it breaks reasoning continuity for all Agent frameworks interacting via the OpenAI endpoint.

---

Proposed Solutions & Mitigations

We would love to discuss potential approaches to bridge this gap:

Option 1: Configurable "Reasoning Replay / Text Injection" (Lightweight & Highly Effective)

• Introduce a configuration flag (e.g., reasoning_replay: true or inject_reasoning_to_content: true) in provider/model configs or global proxy settings.
• When translating OpenAI messages to upstream Gemini format:
  • If an assistant message contains reasoning_content, automatically prepend it to the message's content wrapped in a standard block, e.g.:

    ```thinking
    <reasoning_content>
    ```
    <actual_content>

  • Why this works: In live testing (and aligned with how Oh My Pi / OMP handles Antigravity natively), models treat this as historical context in their attention window without triggering signature validation errors, restoring multi-turn reasoning continuity without maintaining server-side state.

Option 2: Thought Signature Carrier / Ephemeral Cache (Stateful)

• When streaming Gemini responses to clients, optionally encode/cache the thoughtSignature mapped to (session_id, turn_index) or embed a lightweight carrier token in delta metadata, rehydrating the signature when the client sends back the turn.

Option 3: Documentation & Best Practice Recommendation

• If protocol purity is preferred, explicitly document in CPA documentation that users requiring deep multi-turn Agent reasoning on Gemini models should route through /v1beta/models/...:generateContent rather than /v1/chat/completions.

---

Environment Information

• CPA Version: v7.2.157
• Models Tested: gemini-3.8-flash-high (Antigravity provider)
• Endpoints Compared: /v1/chat/completions vs /v1beta/models/...:generateContent