Chat completions can return an unusable assistant turn: empty content, or tool calls the client never declared

[serrade]

Summary

When an OpenAI-compatible client sends its own tools array, some models cause the local API to return a chat completion with no content and no usable tool call. Clients treat that as a provider failure rather than a turn.

Reproduced with the Cline VS Code extension (4.0.12), which surfaces:

Invalid API Response: The provider returned an empty or unparsable response. This is a provider-side issue where the model failed to generate valid output or returned tool calls that Cline cannot process.

Cline then writes Failure: I did not provide a response. into the task history, which poisons every following turn of that task. composer-2.5 usually works; claude-opus-5 and claude-sonnet-5 fail routinely.

Cause

Two independent things combine.

1. Unmappable tool calls are dropped. The bridge captures the model's first tool call and returns it, and OpenAICompatibility.resolveToolCall tries to express it as one of the client's declared tools. When no mapping is found it returns nil and the call is discarded. Both response paths then emit nothing usable:

• chatCompletionResponse sends content: "" with finish_reason: "stop".
• chatCompletionStreamToolCall returns empty Data(), so no chunk is emitted, while the caller has already counted the call. The stream closes with finish_reason: "tool_calls" and zero tool calls in it.

2. The harness out-competes the client tool channel. The SDK agent runs with the client's real workspace as local.cwd, so its built-in read/write/shell succeed against the project directly. The model has no reason to call back through the client, so it does the work internally and returns narration, which the client cannot act on. Composer 2.5 tends to follow the prompt's routing instructions; Opus and Sonnet reach for the built-ins instead.

Reproduction

Any /v1/chat/completions request that declares tools and asks for file work will do. This one fails against claude-opus-5 while succeeding against composer-2.5:

curl -s http://127.0.0.1:8787/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer <key>' \
  -d '{
    "model": "claude-opus-5",
    "stream": true,
    "messages": [
      {"role": "system", "content": "You are Cline, working in /path/to/project."},
      {"role": "user", "content": "Read package.json and report the name field."}
    ],
    "tools": [
      {"type": "function", "function": {"name": "read_file",
        "parameters": {"type": "object", "properties": {"path": {"type": "string"}}, "required": ["path"]}}},
      {"type": "function", "function": {"name": "execute_command",
        "parameters": {"type": "object", "properties": {"command": {"type": "string"}, "requires_approval": {"type": "boolean"}}, "required": ["command", "requires_approval"]}}}
    ]
  }'

Failing shape: a stream whose only chunks are the opener and a finish chunk, with finish_reason claiming tool_calls but no tool_calls delta anywhere. Expected shape: a read_file tool call, or failing that, assistant text.

Environment

• API for Cursor built from v0.1.10 (987bc50), macOS 26.0, Apple silicon
• Client: Cline 4.0.12 in VSCodium, native tool calling enabled (its default), reached through a LiteLLM gateway and also directly
• Models: claude-opus-5, claude-sonnet-5 fail; composer-2.5 mostly succeeds

Suggested direction

Two things seem worth separating:

1. Make an empty assistant turn impossible: report unmappable calls, retry once with the client's actual tool names, and fall back to the model's narration or an explanation rather than empty content.
2. Stop the harness from doing the work itself when the caller owns tool execution, so client tools are the only route to the workspace.

Happy to open pull requests for both if that direction sounds right.

[serrade]

Follow-up: there is a second path to the same user-visible failure, and it is the one Cline actually hits. Retitling the issue to cover both.

I measured both against main by calling the response builders directly:

Path 1 — the client declares tools and the model calls something unmappable (what the description above covers):

content       = "" (empty)
tool_calls    = []
finish_reason = stop
stream chunk  = empty Data(), nothing emitted, while the finish chunk still says tool_calls

Path 2 — the client declares no tools at all:

content       = null
tool_calls    = ["shell"]      <- the SDK's own tool name, forwarded verbatim
finish_reason = tool_calls

In path 2, resolveToolCall takes the tools.isEmpty branch and passes the SDK tool call straight through:

guard let tool = resolveToolSpec(...) else {
    guard tools.isEmpty else { return nil }
    return ResolvedToolCall(name: normalizedToolCall.name, arguments: normalizedToolCall.arguments)
}

Any OpenAI-compatible client will reject a tool_calls entry naming a tool it never declared, which is the "returned tool calls that Cline cannot process" half of Cline's error message. Path 1 produces the "empty or unparsable response" half.

Why this matters for Cline specifically. I had assumed Cline sends native tool schemas, since nativeToolCallEnabled defaults on. It does not, for this transport. Native tool calling in Cline 4.0.12 is implemented per provider, and the two providers that can reach a local OpenAI-compatible endpoint both omit tools from the request:

liteLlmModelId, liteLlmModelInfo             -> no tools key
openAiApiKey, openAiBaseUrl, openAiModelId   -> no tools key
groqApiKey / togetherApiKey / xaiApiKey ...  -> tools sent

So Cline always uses its XML in-prompt tool protocol against this API, and always lands in path 2. There is no client-side setting that changes it.

That has a second consequence worth noting: with no declared tools, nothing signals that the caller owns tool execution, so the SDK agent keeps the caller's real workspace as its working directory and does the file and shell work itself. The client then receives narration describing changes it never made, or a tool call it cannot execute.

Scope of the open PRs. #33 fixes path 1. #34 fixes the harness competing with the client tool channel for callers that do declare tools. Neither addresses path 2, and I would rather not widen them mid-review. I am putting together a separate change for prompt-protocol clients: detect a caller that declared no tools, never hand it a tool call it did not declare, translate the SDK's tool calls into the protocol the caller's own prompt defines, and isolate the workspace on that signal instead of on the tools array. Happy to fold it into #34 instead if you would prefer one change.

[serrade]

Correcting my previous comment. I claimed Cline cannot send tools over this transport under any
setting. That is wrong, and the error was method: I read it out of the extension's minified bundle.
Checking cline/cline at the version I was running (v4.0.12, f267bf4) shows the opposite.

Cline's OpenAI Compatible provider does send native tools. OpenAiHandler.createMessage takes a
tools argument and spreads it into the request:

// apps/vscode/src/core/api/providers/openai.ts
async *createMessage(systemPrompt: string, messages: ClineStorageMessage[], tools?: ChatCompletionTool[]): ApiStream {
  const stream = await client.chat.completions.create({
    model: modelId, messages: openAiMessages, temperature, max_tokens: maxTokens,
    reasoning_effort: reasoningEffort, stream: true, stream_options: { include_usage: true },
    ...getOpenAIToolParams(tools),
  })

getOpenAIToolParams yields tools + tool_choice: "auto" + parallel_tool_calls: false whenever
the array is non-empty. Two more corrections to that comment: nativeToolCallEnabled defaults to
true (shared/storage/state-keys.ts), so nothing needs switching on; and tools are built only when
the model ID selects a prompt variant labelled use_native_tools: 1, matched on the ID string the
user types. claude-opus-5, claude-sonnet-5, grok-4.5 and gpt-5.6 all match; composer-2.5
does not and falls through to the XML variant.

What I got right by accident is that I was seeing tools=0, because I was using the LiteLLM
provider. That one really cannot: LiteLlmHandler.createMessage(systemPrompt, messages) takes two
parameters, ignores the third, and never parses delta.tool_calls. "litellm" is also absent from
isNextGenModelProvider. So the limitation is one provider, not the client.

Consequences for this issue. Path 2 is still a real bug, but the population is narrower than I
described: genuinely tool-less callers, which includes plain curl and Cline's LiteLLM provider, not
Cline as a whole. I am withdrawing the prompt-protocol translation change I said I was preparing —
it would have solved a problem this client does not have. If path 2 is worth fixing, it looks much
smaller than what I proposed: never forward a tool call the caller did not declare.

On the plus side, #33 and #34 turn out to be exactly what the native path needs. Driving the real
extension (Cline 4.0.12, OpenAI Compatible, model id claude-opus-5) rather than a reconstruction of
it, a task now runs start to finish: Cline sends 38 native tools, claude-opus-5 calls its
read_file, replace_in_file and execute_command by name, the diffs render and get approved in
Cline's own UI, and the task closes on attempt_completion. Every turn logged executor=client with
the harness in an empty scratch directory, so the model had no route to the workspace except back
through the client — which is the behaviour #34 is meant to enforce.

The same session also produced a clean demonstration of path 2, by accident. One turn went out as
composer-2.5, which does not match any of Cline's native-tool model families, so Cline sent no
tools at all:

model=claude-opus-5  executor=client   cwd=/tmp/.../workspaces/98e0205…   (x10)
model=composer-2.5   executor=harness  cwd=/Users/alex/…/cline_test      (x1)

With no declared tools nothing signals that the caller owns execution, so the harness kept the real
project directory. That is path 2 in the wild: not a client that cannot be fixed, but any caller that
genuinely declares nothing.

Sorry for the noise on this one. Testing the corrected path did shake out further bugs of its own; I
will file those separately rather than pile more onto this thread.