Skip to content

fix: never return an empty assistant turn for tool-declaring clients - #33

Open
serrade wants to merge 1 commit into
standardagents:mainfrom
serrade:fix/no-empty-assistant-turns
Open

fix: never return an empty assistant turn for tool-declaring clients#33
serrade wants to merge 1 commit into
standardagents:mainfrom
serrade:fix/no-empty-assistant-turns

Conversation

@serrade

@serrade serrade commented Jul 30, 2026

Copy link
Copy Markdown

Fixes part of #30.

Problem

When a client declares its own tools, a chat completion can come back with no content and no usable tool call. Clients treat that as a provider failure rather than a turn, so the user sees an error instead of a response.

The bridge captures the model's first tool call and returns it. resolveToolCall then tries to express that call as one of the client's declared tools, and when nothing matches it returns nil:

guard let tool = resolveToolSpec(normalizedToolCall.name, arguments: ..., tools: tools, context: context) else {
    guard tools.isEmpty else { return nil }   // dropped here
    return ResolvedToolCall(name: normalizedToolCall.name, arguments: normalizedToolCall.arguments)
}

The call is discarded, and both response paths then emit nothing usable:

  • chatCompletionResponse sends content: "" with finish_reason: "stop".
  • chatCompletionStreamToolCall returns an empty Data(), so no chunk goes out, while the caller has already counted the call. The stream closes with finish_reason: "tool_calls" and no tool calls in it.

Fix

Three layers, so an empty turn is structurally impossible:

  • unresolvableToolCallNames and needsToolCallRepair surface what previously failed silently.
  • repairedRequest builds one retry that names the client's actual tools. The hint is written into both prompt and incrementalPrompt, because the bridge prefers the latter whenever it already has a warm agent for the session.
  • nonEmptyAssistantText supplies a body: the model's narration when there is any, an explanation of the unavailable tool otherwise. Streaming counts only chunks it actually emitted, so finish_reason can no longer claim tool calls that were never sent, and a final guard covers the case where a stream would otherwise carry nothing.

repairedOutput in LocalAPIServer performs the single retry for both the streaming and non-streaming chat paths, and only replaces the original output when the retry is genuinely better.

Mappable tool calls stream exactly as before.

Verification

  • swift build clean, npm test and npm run typecheck pass, Package macOS smoke green on my fork.
  • New tests cover the helpers, the non-streaming response, and both streaming shapes: testUnresolvableToolCallNamesReportsOnlyUnmappableCalls, testNeedsToolCallRepairOnlyWhenEveryCallIsUnmappable, testRepairedRequestNamesTheClientToolsInBothPrompts, testRepairedRequestIsSkippedWithoutClientTools, testNonEmptyAssistantTextPrefersModelNarration, testNonEmptyAssistantTextExplainsAnUnavailableTool, testChatCompletionResponseNeverReturnsEmptyContent, testChatCompletionsStreamingNeverEndsWithoutContentOrToolCall, testChatCompletionsStreamingStillForwardsMappableToolCalls.
  • Before and after, measured by calling the response builders on main directly: pre-fix, an unmappable call returns empty content with finish_reason: "stop" on the non-streaming path, and an empty Data() chunk on the streaming path while the finish chunk still reports finish_reason: "tool_calls". Post-fix the same inputs produce assistant text and finish_reason: "stop", which I also confirmed against a running server.

Scope

This addresses the empty-turn half of #30. The other half, the harness doing the work itself instead of routing through the client, is a separate change I can follow up with.

When the model called a tool that cannot be expressed as one of the
client's declared tools, resolveToolCall returned nil and the call was
dropped. chatCompletionResponse then sent empty content with
finish_reason stop, and the streaming path emitted no chunk at all while
still counting the call, so the stream closed with finish_reason
tool_calls and nothing in it. OpenAI-compatible clients treat both as a
provider failure and surface an error instead of the turn.

Three layers now make an empty turn impossible:

- unresolvableToolCallNames and needsToolCallRepair expose what used to
  fail silently.
- repairedRequest drives one retry that names the client's actual tools.
  The hint goes into both prompt and incrementalPrompt because the bridge
  prefers the latter for warm agents.
- nonEmptyAssistantText supplies the model's narration when present and
  an explanation otherwise. Streaming counts only chunks it actually
  emitted, so finish_reason can no longer claim tool calls that were
  never sent.

Mappable tool calls keep streaming exactly as before.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant