Skip to content

api: Qwen3.6 tool calling (#1723 on current dev, keeps #1767) - #1794

Merged
JustVugg merged 1 commit into
devfrom
feat/qwen36-tool-calling
Sep 28, 2026
Merged

JustVugg merged 1 commit into
devfrom
feat/qwen36-tool-calling

Conversation

@JustVugg

Copy link
Copy Markdown
Owner

Qwen3.6 tool calling, the work of @GenericRikka in #1723, carried onto the current dev. The commit keeps its author.

#1723 started from a base older than #1767 and replaced the whole of render_chat_qwen. Taking its side of the conflict would have removed preserve_thinking, the last_query rule and the reading of past turns that #1767 added, and every qwen36 chat request would then fail with a TypeError (7 positional arguments passed, 6 accepted). This branch keeps them.

What changed in the rebase

  • dev's render_chat_qwen is kept (the qwen36: render the template's preserve_thinking so standard clients get KV prefix reuse #1767 logic and the continuation branch). The PR's tool block, system folding, grouped <tool_response> turns and <tool_call> rendering are added on top.
  • The parser is chosen by chat_flavor() in ("qwen36", "qwen38"), so the CHAT_FLAVOR of qwen36: run dense checkpoints of the family (Qwen3.8-27B) #1777 still holds: qwen36 renders with the qwen36 template, qwen36 with the qwen38 flavor and qwen38 render with the qwen38 one, and all three parse <tool_call>.
  • reasoning_content: null counts as absent; a non-string is a 400 (the PR's own test).
  • The conformance test now crosses tools with preserve_thinking and thinking on/off. It adds an agent loop (parallel calls, non-string arguments, two tool results, a new query) and an empty system turn, which the template emits and the PR head dropped.
  • Docs: dev said Qwen3.8 refuses tools, which its code no longer does.
  • The dispatch in _parse_arch_tool_calls returns the two span maps as None for qwen. That is safe with api: logprobs, top_logprobs and echo on the OpenAI endpoints, aligned by raw-stream span #1353: the numeric logprobs channel is glm only, so no qwen reply ever has spans to align.

Checked

  • test_openai_server, test_openai_tools_fallback_e2e, both chat template tests, test_anthropic_messages, test_qwen_image_gateway, test_family_registry: 519 passed.
  • Against the official chat_template.jinja of Qwen3.6 (40 cases plus continuation) and of Qwen3.8: byte-identical.
  • End to end through the real gateway with a stub engine that answers in the model's native format, thinking on and off:
    • the prompt of turns 1, 2 and 3 equals the official template's;
    • two parallel calls come back with typed arguments and finish_reason: tool_calls;
    • streaming carries no markup in content and sends tool_calls deltas;
    • /v1/messages answers with two tool_use blocks;
    • tool_choice: none renders no tool block;
    • with thinking off, turn N+1 starts with turn N's prompt plus the reply, so the KV prefix is reused (qwen36: render the template's preserve_thinking so standard clients get KV prefix reuse #1767).

On the #1723 head, turn 3 after a tool loop matched neither the template nor the prefix.

Behaviour to know

  • A system message in the middle of the conversation is now a 400 on qwen36. The official template raises on it too, and qwen38 already refused it.
  • A forced or required tool_choice is accepted but not enforced, the same as qwen38.
  • COLI_TOOL_FALLBACK no longer applies to qwen36, which now has native tool calls.

Supersedes #1723.

Implements the checkpoint's native tool declaration/call/response
protocol and translates it to OpenAI tool_calls.

Rebased onto dev after #1767 and #1777: render_chat_qwen keeps the
template's preserve_thinking, its last_query_index rule and its reading
of a past turn (reasoning_content, or the raw reply split at </think>),
so KV prefix reuse still engages on follow-up turns; the tool-call
parser dispatch goes through chat_flavor(), so a qwen36 engine running
a Qwen3.8 template keeps the qwen38 renderer. Native tool calling
replaces the COLI_TOOL_FALLBACK path for qwen36 (OLMoE keeps it).
The conformance test renders every case with and without tools and
preserve_thinking against chat_template.jinja (sha256 e84f32a2).
@JustVugg
JustVugg merged commit af15084 into dev Sep 28, 2026
30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants