Skip to content

fix(serve): a non-object tool_choice.function answered 500 "engine failed" - #1598

Merged
JustVugg merged 1 commit into
JustVugg:devfrom
kevin9327:fix/serve-tool-choice-non-object-function
Sep 18, 2026
Merged

JustVugg merged 1 commit into
JustVugg:devfrom
kevin9327:fix/serve-tool-choice-non-object-function

Conversation

@kevin9327

Copy link
Copy Markdown
Contributor

What happens

POST /v1/chat/completions
{"model": "...", "messages": [...],
 "tools": [{"type": "function", "function": {"name": "search"}}],
 "tool_choice": {"type": "function", "function": "search"}}
HTTP/1.1 500 Internal Server Error
{"error":{"message":"The colibri engine failed to process the request.",
          "type":"server_error","param":null,"code":"engine_error"}}

[api] 127.0.0.1 - request failed: 'str' object has no attribute 'get'

Writing the name where the object goes is an ordinary client slip, and the
server already has an answer for it -- generation_options() raises
400 "tool_choice function object must include a name." two lines further
down. It never runs, because the read above it is

name = (choice.get("function") or {}).get("name") or choice.get("name")

which raises AttributeError as soon as function is not an object. That
reaches do_POST's catch-all, and the request comes back as a 5xx that OpenAI
SDKs retry -- against an engine that was never asked anything.

Reach

The same expression is copied into the five renderers that read the forced
tool, and on /v1/chat/completions the prompt is rendered before
generation_options runs, so those crash first. /v1/completions has no
renderer in front and reached the validator's own copy.

arch before after
glm, glm53, kimi, deepseek_v4, deepseek_v41 500 (renderer) 400
qwen38 500 (generation_options) 400
olmoe, inkling, qwen36 400 ("tools unsupported") 400
/v1/completions, any arch 500 (generation_options) 400

/v1/messages is unaffected: anthropic_tools() builds its own tool_choice
and already validates it.

The change

Read the member, then check it -- the same shape as the json_schema fix
(#1587) -- through one small helper, so the six call sites that share the
expression share the guard and the existing 400 is what runs. The legacy
{"type": "function", "name": "search"} spelling still forces the tool, and a
well-formed function object renders byte-for-byte as before; both are asserted.

Verification

$ python -m unittest discover -s tests -p "test_openai_server.py" -t tests -k tool_choice
# before: FAILED (failures=19, errors=4)
#   AttributeError: 'str' object has no attribute 'get'   (generation_options)
#   AssertionError: 500 != 400                            (9 arches x 3 shapes, and /v1/completions)
# after:  Ran 6 tests ... OK

Full module and neighbours, on Windows (Python 3.11):

tests/test_openai_server.py        173 OK (baseline) -> 176 OK
tests/test_anthropic_messages.py    28 OK
tests/test_cli_output.py            34 OK
tests/test_family_registry.py       58 OK
tests/test_datapoint.py             20 OK
tests/test_chat_images.py            3 OK

The HTTPTest cases drive a real APIServer over a socket, so what is asserted
is the status the client sees.

🤖 Generated with Claude Code

…iled"

generation_options() already refuses a `tool_choice` function object that
carries no name, with a 400. It never got the chance: the line above reads
`(choice.get("function") or {}).get("name")`, so writing the name where the
object goes -- {"type": "function", "function": "search"} instead of
{"function": {"name": "search"}} -- raises AttributeError, and do_POST's
catch-all answers HTTP 500 "The colibri engine failed to process the request."
OpenAI SDKs retry a 5xx, against an engine that was never asked anything.

The five renderers that read the forced tool (GLM-5.2, GLM-5.3, Kimi, DeepSeek
V4 and V4.1) carry the same expression, and on /v1/chat/completions the prompt
is rendered before generation_options runs, so they crash first; /v1/completions
reaches the validator directly and crashed there. Both endpoints, every arch.

Read the member, then check it -- the same shape as the json_schema fix
(JustVugg#1587) -- through one helper, so the existing 400 covers all six call sites.
The legacy {"type": "function", "name": ...} spelling and a well-formed
function object are unaffected.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@JustVugg
JustVugg merged commit 43d7e86 into JustVugg:dev Sep 18, 2026
28 checks passed
JustVugg pushed a commit that referenced this pull request Sep 23, 2026
{"type": "function", "function": "search"} on tools[] is the same
client slip #1598 covered on tool_choice. generation_options already
has 400 "Tool function must be an object". The GLM and DeepSeek
declaration blocks did fn.items() first, so do_POST answered 500
"engine failed" for a request the engine never saw.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants