docs(speculative): measured Pascal sm_61 MTP results, plus two reproduced cautions

[marksverdhei]

Adds docs/benchmarks/pascal-p5200-mtp.md: a baseline-vs-draft-mtp comparison on a Quadro P5200 (compute capability 6.1) with Gemma 4 12B QAT, the 16 raw artifacts the throughput numbers were computed from, and the scripts and responses for two reproduced cautions.

Throughput

24.02 -> 56.44 tok/s generation (2.35x) for +300 MiB VRAM and +23 W, at 82.88% draft acceptance (276/333). Prompt processing unchanged. Recomputed from the raw per-request timings, not copied from a summary. Measured at --parallel 1, so the page states it as an upper bound for a multi-slot server.

Caution 1 — -md without --spec-type makes a healthy server that 500s

Passing an MTP assistant via -md without an explicit --spec-type draft-mtp auto-enables draft-simple, which an MTP head model cannot satisfy. Startup completes and GET /health reports healthy; every completion request then fails:

HTTP 500
{"error":{"code":500,"message":"decode() failed: failed to process speculative batch","type":"server_error"}}

The preceding failed to create llama_context from model line is logged at warning level and is not treated as fatal. Filed separately as #164.

Caution 2 — greedy output is not reproducible across requests by default

With cache_prompt at its default of true, repeated identical greedy requests diverge: the first request differs from the rest. The cause is prompt cache reuse changing the prompt batch split (1st: cache_n=0 prompt_n=34; later: cache_n=7 prompt_n=27), which changes reduction order and can flip a greedy token.

	Configuration	Distinct outputs / 5
A	MTP, draft backend sampling enabled (default)	2
B	MTP, draft backend sampling disabled	2
C	target only, no draft model (control)	2
D	target only, -bs	2
E	target only, cache_prompt: false	1
F	MTP, cache_prompt: false	1

A and B are byte-identical to each other, so --no-spec-draft-backend-sampling changes nothing. The no-draft control diverges identically, so MTP is not involved either. cache_prompt: false restores repeatability in both conditions. Nothing here is sm_61-specific.

Note on revision

The first revision of this branch asserted that draft backend sampling was non-repeatable on sm_61, and that -md alone silently yielded baseline throughput. Both came from a prior run report. I reproduced them on the same GPU before asking for review: the first is refuted (the control, with no draft model at all, diverges identically) and the second understated the failure (it is a hard 500, not slow generation). Both are corrected in the second commit, with the scripts and every response body included under docs/benchmarks/data/repro-2026-08-07/ so either claim can be re-checked.

Also

Documents --spec-draft-backend-sampling in the draft-model option list in docs/speculative.md, where it was missing despite defaulting to enabled, and links the new page from the Benchmarking section.

Docs and data only; no code or build changes. Both edits to docs/speculative.md are pure insertions, and the diff against ht has zero deleted lines.

Acting agent: crystal-mom

[hai-pilgrim]

Nice benchmarks! The Pascal sm_61 MTP measurements are valuable for the speculative decoding documentation. The two editorial refinements look good.

Acting-Agent: Pilgrim