Skip to content

document, container_upload, file_id and input_audio parts are dropped, and an empty message goes out in their place #5960

Description

@Android-Login-Analysis

Is it a request payload issue?

[x] Yes, this is a request payload issue. I am using a client/cURL to send a request payload, but I received an unexpected result.
[ ] No, it's another issue.

Describe the bug

Seven attachment shapes have no conversion branch on the routes below. The part is dropped, and because it was the only part in its message, the message goes out empty.

# route caller sent forwarded
1 Claude to Gemini document with base64 PDF "parts": []
2 Claude to Gemini container_upload with file_id "parts": []
3 Claude to OpenAI document with base64 PDF "messages": []
4 Claude to OpenAI container_upload with file_id "messages": []
5 OpenAI to Claude file with file_id "messages": []
6 OpenAI to Gemini file with file_id "parts": []
7 OpenAI to Claude input_audio "messages": []

Cases 1, 2 and 6 return 2xx with an empty user turn, so the model answers as if nothing was attached. Cases 3, 4, 5 and 7 send an empty messages array, which the upstream usually rejects with an error about messages that never mentions the attachment, so it reads as a caller mistake.

Case 7 is the one with the clearest precedent. input_audio is already handled on the OpenAI to Gemini path at internal/translator/gemini/openai/chat-completions/gemini_openai_request.go:178, so the shape is understood in this repository. Only the Claude target is missing it.

Root cause

Verified at c93978c4ea2e908255a2a06c37599fda3651554a.

Cases 1 and 2. internal/translator/gemini/claude/gemini_claude_request.go. The content switch has cases at lines 119, 128, 138, 161 and 191 for text, thinking, tool_use, tool_result and image. There is no document case, no container_upload case and no default.

Cases 3 and 4. internal/translator/openai/claude/openai_claude_request.go. The main switch at lines 189 to 236 covers thinking, redacted_thinking, text, image, tool_use and tool_result. convertClaudeContentPart at line 481 covers only text and image. Neither handles document or container_upload.

Case 5. internal/translator/claude/openai/chat-completions/claude_openai_request.go:390

case "file":
    fileData := part.Get("file.file_data").String()
    if strings.HasPrefix(fileData, "data:") {
        ...
    }

Only file.file_data is read, and only when it is a data URL. A file_id reference leaves claudePart empty, and lines 406 to 408 return an empty string, which the caller drops.

Case 6. internal/translator/gemini/openai/chat-completions/gemini_openai_request.go:170

case "file":
    fileData := item.Get("file.file_data").String()

No file_id branch.

Case 7. Same file as case 5. The switch at line 381 has text, image_url and file. There is no input_audio case.

CLI Type

openai-compatibility and claude.

Model Name

Configured aliases for Claude, OpenAI and Gemini targets. The defect is in request translation and is not model specific.

LLM Client

cURL.

Request Information

Built from this repository's Dockerfile at an unmodified commit. The upstream is a recording endpoint that stores the exact bytes it receives and returns a fixed valid response, so the forwarded requests are quoted verbatim. Captured at bd34ceca0420, and all five switches above are unchanged at c93978c4ea.

Case 1. Claude document to Gemini:

curl -X POST http://localhost:8317/v1/messages \
  -H 'anthropic-version: 2023-06-01' -H 'content-type: application/json' -H "x-api-key: $KEY" \
  -d '{"model":"<gemini-alias>","max_tokens":64,"messages":[{"role":"user","content":[{"type":"document","source":{"type":"base64","media_type":"application/pdf","data":"JVBERi0xLjQKMSAwIG9iajw8L1R5cGUvQ2F0YWxvZy9QYWdlcyAyIDAgUj4+ZW5kb2JqCjIgMCBvYmo8PC9UeXBlL1BhZ2VzL0tpZHNbMyAwIFJdL0NvdW50IDE+PmVuZG9iagozIDAgb2JqPDwvVHlwZS9QYWdlL1BhcmVudCAyIDAgUi9NZWRpYUJveFswIDAgOCA4XT4+ZW5kb2JqCnRyYWlsZXI8PC9Sb290IDEgMCBSPj4KJSVFT0YK"}}]}]}'

Forwarded:

{"contents": [{"role": "user", "parts": []}], "model": "...", "safetySettings": [...]}

Case 2. Same endpoint with [{"type":"container_upload","file_id":"file-example"}]. Forwarded contents is again [{"role":"user","parts":[]}].

Case 3. Same document block sent to an OpenAI alias. Forwarded request is {"max_tokens":64,"messages":[],"model":"...","stream":...}.

Case 4. Same as case 3 with the container_upload block. Forwarded messages is [].

Case 5. OpenAI file with file_id to a Claude alias:

curl -X POST http://localhost:8317/v1/chat/completions \
  -H 'content-type: application/json' -H "authorization: Bearer $KEY" \
  -d '{"model":"<claude-alias>","messages":[{"role":"user","content":[{"type":"file","file":{"file_id":"file-example"}}]}]}'

Forwarded messages is [].

Case 6. Same request to a Gemini alias. Forwarded contents is [{"role":"user","parts":[]}].

Case 7. OpenAI input_audio to a Claude alias:

curl -X POST http://localhost:8317/v1/chat/completions \
  -H 'content-type: application/json' -H "authorization: Bearer $KEY" \
  -d '{"model":"<claude-alias>","messages":[{"role":"user","content":[{"type":"input_audio","input_audio":{"format":"wav","data":"UklGRiQAAABXQVZFZm10IBAAAAABAAEAQB8AAIA+AAACABAAZGF0YQAAAAA="}}]}]}'

Forwarded messages is [].

Expected behavior

Two separate asks.

First, and independent of any conversion work: a part that cannot be converted should not take its whole message with it. Sending "messages": [] or an empty parts array turns a known translation gap into an upstream error that names the wrong thing, or into a confident answer about an attachment the model never saw. Dropping the request with a message naming the part type would be clearer than either.

Second, the conversions that are straightforward:

  • Case 7. input_audio to Claude, using the same shape already built for Gemini at gemini_openai_request.go:178.
  • Cases 5 and 6. file.file_id alongside the existing file.file_data branch.
  • Cases 1 to 4. document carries base64 bytes and a media type, which both targets accept. Gemini takes inlineData, and the OpenAI Chat Completions file part takes file_data, which this repository already builds in the other direction at claude_openai_request.go:398.

Where a shape genuinely has no target equivalent, failing closed per #5155 is the consistent answer.

OS Type

  • OS: Linux, x86_64
  • Gateway built from this repository's Dockerfile at an unmodified commit

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions