feat(models): drive Gemma 4 12B's MTP head from Unsloth's sibling drafter

[tmac1973]

The 16 GB tier's Gemma 4 12B shipped with MtpSource::None and a note that Unsloth publishes a drafter separately, which MtpSource::Sibling exists for and which would fit inside the tier's headroom — "not wired up until it's been measured." This wires it up so it can be measured.

The drafter

MTP/mtp-gemma-4-12b-it-Q8_0.gguf, 465,109,248 bytes, sha256 taken from the LFS oid the same way the projector hashes are. Q8_0 over the 862 MB F16/BF16: a drafter is quality-tolerant — a rejected draft token costs throughput, not correctness — so the small one is the right trade on a tier this VRAM-tight. Stored locally as Gemma4-12B-MTP-Q8_0.gguf, normalised like the projectors because the models dir is flat and the upstream MTP/ prefix is lost on download.

The fit absorbs it. fixed_vram_bytes charges MTP_OVERHEAD_BYTES plus the drafter's own weights when the preference is on, and a 16 GB card still reaches 262144 either way — the ~2.2 GB spare above drops to ~1.2 GB. The new test asserts both that the ceiling holds and that the weights are genuinely counted, so the equality can't pass vacuously.

Two fixes the registry data alone would have left broken

find_any_model would have picked up the drafter as a model. It skipped .partial and mmproj files and nothing else, and read_dir order is arbitrary — first-run autodetect could have handed back a 465 MB draft head as "the model" and started the server on it. It now also skips filenames matching a registry Sibling, matched against the registry rather than by substring so a user's own …mtp….gguf isn't hidden.

The network drift guard didn't cover the drafter. A drafter that 404s or changes size fails a download exactly the way the weights would. Extended; passes against upstream.

Draft tuning

--spec-draft-ngl all and --spec-draft-n-max 4, both scoped to the sibling path.

--spec-draft-ngl defaults to auto, which decides how much of the drafter to keep in VRAM after the target and its KV cache have claimed theirs. A partially offloaded drafter still produces tokens, so the host round-trips are invisible and just read as "MTP bought nothing" — worth pinning rather than leaving to chance. --spec-draft-n-max 4 is what Unsloth's drafter card recommends over llama.cpp's default of 3.

Neither applies to the bundled-head Qwen 3.8 27B: there's no separate drafter to offload there, and no measurement behind moving that model off llama.cpp's defaults. Both still sit ahead of extra_args, so they stay overridable.

Upgrade path

A 12B downloaded before this lands won't have the drafter file. find_mtp_draft_for_model returns None and the server starts without the flag rather than failing — correct, but silent, and silence here reads as "MTP made no difference". That case now warns in the log naming the missing file. Re-running the download fetches only what's missing, since download_model doesn't short-circuit at the top.

Verification

• 668 Rust tests, 1613 frontend tests, clippy at -D warnings, cargo fmt --check, npm run check, npm run lint, format:check — all clean. No IPC type drift (mtp still serialises as the same boolean).
• The #[ignore]d network drift guard run explicitly, confirming the drafter URL and size against upstream.
• llama.cpp pin checked rather than assumed: LLAMA_CPP_VERSION is b9565 (2026-06-08), 16 commits past the Gemma4 MTP merge (llama : add Gemma4 MTP ggml-org/llama.cpp#23398), so the drafter's gemma4-assistant arch loads.
• End to end on a 16 GB card: the download fetched weights, projector and drafter, and llama-server came up with --model-draft … --spec-draft-ngl all --spec-draft-n-max 4 --spec-type draft-mtp.

Still open

The point of this PR is to make the benchmark possible, not to claim a speedup — no throughput numbers here yet. Unsloth's own figure (52 → 162 tok/s at 0.70 acceptance) is a B200 against a Q4_K_M target, so it says little about Vulkan against Q6. MTP_OVERHEAD_BYTES also remains the uncalibrated 512 MB placeholder its comment admits to; a with/without pair at fixed context would replace the guess with a number.

🤖 Generated with Claude Code

https://claude.ai/code/session_011UTGzKCSRDYAzxPtFZdNf7