fix: never return an empty assistant turn for tool-declaring clients - #33
Open
serrade wants to merge 1 commit into
Open
fix: never return an empty assistant turn for tool-declaring clients#33serrade wants to merge 1 commit into
serrade wants to merge 1 commit into
Conversation
When the model called a tool that cannot be expressed as one of the client's declared tools, resolveToolCall returned nil and the call was dropped. chatCompletionResponse then sent empty content with finish_reason stop, and the streaming path emitted no chunk at all while still counting the call, so the stream closed with finish_reason tool_calls and nothing in it. OpenAI-compatible clients treat both as a provider failure and surface an error instead of the turn. Three layers now make an empty turn impossible: - unresolvableToolCallNames and needsToolCallRepair expose what used to fail silently. - repairedRequest drives one retry that names the client's actual tools. The hint goes into both prompt and incrementalPrompt because the bridge prefers the latter for warm agents. - nonEmptyAssistantText supplies the model's narration when present and an explanation otherwise. Streaming counts only chunks it actually emitted, so finish_reason can no longer claim tool calls that were never sent. Mappable tool calls keep streaming exactly as before.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes part of #30.
Problem
When a client declares its own
tools, a chat completion can come back with no content and no usable tool call. Clients treat that as a provider failure rather than a turn, so the user sees an error instead of a response.The bridge captures the model's first tool call and returns it.
resolveToolCallthen tries to express that call as one of the client's declared tools, and when nothing matches it returnsnil:The call is discarded, and both response paths then emit nothing usable:
chatCompletionResponsesendscontent: ""withfinish_reason: "stop".chatCompletionStreamToolCallreturns an emptyData(), so no chunk goes out, while the caller has already counted the call. The stream closes withfinish_reason: "tool_calls"and no tool calls in it.Fix
Three layers, so an empty turn is structurally impossible:
unresolvableToolCallNamesandneedsToolCallRepairsurface what previously failed silently.repairedRequestbuilds one retry that names the client's actual tools. The hint is written into bothpromptandincrementalPrompt, because the bridge prefers the latter whenever it already has a warm agent for the session.nonEmptyAssistantTextsupplies a body: the model's narration when there is any, an explanation of the unavailable tool otherwise. Streaming counts only chunks it actually emitted, sofinish_reasoncan no longer claim tool calls that were never sent, and a final guard covers the case where a stream would otherwise carry nothing.repairedOutputinLocalAPIServerperforms the single retry for both the streaming and non-streaming chat paths, and only replaces the original output when the retry is genuinely better.Mappable tool calls stream exactly as before.
Verification
swift buildclean,npm testandnpm run typecheckpass, Package macOS smoke green on my fork.testUnresolvableToolCallNamesReportsOnlyUnmappableCalls,testNeedsToolCallRepairOnlyWhenEveryCallIsUnmappable,testRepairedRequestNamesTheClientToolsInBothPrompts,testRepairedRequestIsSkippedWithoutClientTools,testNonEmptyAssistantTextPrefersModelNarration,testNonEmptyAssistantTextExplainsAnUnavailableTool,testChatCompletionResponseNeverReturnsEmptyContent,testChatCompletionsStreamingNeverEndsWithoutContentOrToolCall,testChatCompletionsStreamingStillForwardsMappableToolCalls.maindirectly: pre-fix, an unmappable call returns empty content withfinish_reason: "stop"on the non-streaming path, and an emptyData()chunk on the streaming path while the finish chunk still reportsfinish_reason: "tool_calls". Post-fix the same inputs produce assistant text andfinish_reason: "stop", which I also confirmed against a running server.Scope
This addresses the empty-turn half of #30. The other half, the harness doing the work itself instead of routing through the client, is a separate change I can follow up with.