Repository navigation
Eval bug: QWEN3.8:27b + lemonade server's rocm b10472 + cline (vscode) = tools partially working (MCP failures), while vulkan works flawlessly #27612
Description
Name and Version
lemonade server 11.7.0 + rocm b10472
Operating systems
Windows
GGML backends
HIP
Hardware
AMD AI MAX 395+ 64 GB (gfx1151)
Models
Qwen3.6:27b and 35b-3ba, Qwen3.8:27b
Problem description & steps to reproduce
vscode + cline 4.1.x + lemonade server + rocm
Current status:
- Tool calling works partially with Qwen3.8:27b:
native Cline tools (read_files, search_codebase, editor, etc.) are correctly
recognized and invoked. - MCP tools still fail: when MCP servers are connected in Cline and their tools
are injected into the context, the model does not correctly invoke them on the
ROCm backend. The same setup works correctly on the Vulkan backend.
model is loaded on both backends with
--flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --no-mmap --temp 0.1 --repeat-penalty 1.0 --top-k 20 --top-p 0.95 --min-p 0 --chat-template-kwargs '{"preserve_thinking":true,"reasoning_effort":"low"}'
Hypothesis: the issue may be context-size related — MCP tool schemas significantly
increase the prompt token count, and there may be a ROCm-specific behavior difference
under high context load compared to Vulkan.
First Bad Commit
No response
Relevant log output
I don't have any, let me know if lemonade server logs are enough, I can provide them.
Activity
MetalKnight commentedon Aug 23, 2026
relates to
lemonade-sdk/lemonade#2977
[-]Eval bug: QWEN3.8:27b + rocm b10472 = tools partially working, while vulkan works flawlessly[/-][+]Eval bug: QWEN3.8:27b + lemonade server's rocm b10472 + cline (vscode) = tools partially working (MCP failures), while vulkan works flawlessly[/+]nabe2030 commentedon Aug 25, 2026
This may be the same underlying defect discussed in
#27579 (comment) and its follow-up
(#27579 (comment)) — a
process_ubatchinput-write race on HIP, amplified by the APU host-buffer path from #24233, which we found
causes tool-call corruption specifically at longer context / larger tool arrays even on a single
slot. Your MCP-tools-fail-but-native-tools-work pattern and context-size hypothesis line up with
what we saw.
If you're able to test against a build with the sync fix from #27572 applied (or the
integrated=falseworkaround as a quicker stopgap), that would help confirm or rule out theconnection.
MetalKnight commentedon Aug 25, 2026
@nabe2030 thanks for the reply!
I would gladly test the PR #27705 however I don't have the ROCM SDK to rebuild lemonade-server.exe with HIP ROCM support.
Is there any artifact I could use in the PR CI?
Otherwise I can test once https://github.com/lemonade-sdk/llama.cpp ports the fix in its fork.
or I don't if someone from lemonade's team like @superm1 could help here triggering a nightly with that fix... :)
nabe2030 commentedon Aug 26, 2026
@MetalKnight — apologies, I pointed you in a vague direction and you followed it to the
wrong PR. #27705 is a different defect (an
output_reorderindex-space mismatch onthe MTP/nextn path); it won't touch what you're seeing.
The one that matters for your symptom is #27311 (pwilkin, "Scheduler UMA ring
buffer"), or #25863 as a smaller targeted mitigation of the same thing. Both address
the write-after-read on graph input tensors that the HIP host-buffer path exposes on APUs.
On the CI artifact question: I don't think that route works. Looking at #27311's check
list, there's no Windows ROCm job — the ROCm Windows zips come from the release workflow
on master, not from PR CI. So you'd be waiting for the lemonade fork either way.
But you can probably test this today without rebuilding anything. @Superfish1000
measured the trigger precisely in #27311 on gfx1151: corruption appears whenever
n_ubatch < n_batch, and for ordinary generation wheneverprompt_tokens > n_ubatch.The only passing configuration is the one where the whole prompt lands in a single
ubatch — 7/7 in his generation tests, 8/8 on the batch-split rule.
That is a runtime flag, not a code change. If lemonade lets you pass through
--batch-sizeand--ubatch-size, try setting them equal and large enough to hold yourMCP-loaded prompt:
(pick a size above your actual prompt token count; larger ubatch costs compute-buffer
memory, so start from whatever your MCP context actually measures). If your MCP tools
start being invoked correctly at
ub == band break again when you drop-ubback to512, that both confirms the connection to this defect and gives you something usable
before any fix lands.
Your "native Cline tools work, MCP tools fail" split fits this well: MCP schemas are what
push the prompt past
n_ubatch, and the failure mode we documented in #27579 is exactlythat the head of a large tool array goes missing while the tail is read correctly — the
model keeps answering fluently and simply doesn't see the tools that were injected first.
One more thing worth knowing before you spend effort staying on ROCm: @thedelph measured
the backend switch on this chip in #27506 and found Vulkan cost 5–9% on prefill but
gained 24% on decode (49.5 → 61.5 t/s). Since Vulkan already works flawlessly for
you, there may be no throughput argument for waiting at all.
MetalKnight commentedon Aug 26, 2026
Thanks for the detailed reply,
Now I fully understand the status.
I will try to do some measurements and come back, along with some decoding speeds once the fix is merged in main and propagated to lemonade.
MetalKnight commentedon Oct 8, 2026
@IMbackK could you please clarify which PR fixed this?
thanks