docs(adr): ADR-074 — MTP / speculative-decoding priority (Metal)

[oceanwaves630]

PR description authored by Claude (Anthropic agent) on behalf of @oceanwaves630.

ADR-074 — MTP / speculative-decoding priority (Metal)

Fourth sequencing ADR in the bundle set, after ADR-071 (GDN-state), ADR-072 (weight-quant), and ADR-073 (KV-quant). Docs-only.

Verdict: NEEDS-EXPERIMENT

The live MTP path's correctness is settled and needs no ranking — it is off by default, greedy-only when enabled, single-draft-token (K=1), and regression-tested to be exactly argmax-equivalent to plain greedy decode. What is genuinely gated is profitability, and it rests on a single measurement whose own author flagged it unreliable, plus an in-tree fix that has never been run.

The load-bearing facts (all pointers verified against origin/main @ 853f7b1d2):

• The only throughput number is a net -64% regression (175.7 → 63.0 tok/s free-form, Qwen3.5-0.8B Q4) — an issue-comment artifact, not a committed bench. Its own analysis attributes the loss to verify cost, not draft quality: the default verifier runs the full target model sequentially twice per round to check one draft token.
• That number's 0% acceptance is self-caveated as a possible weight-basis artifact (MTP tensors possibly not counter-rotated), so there is currently no trustworthy acceptance-rate measurement on any workload.
• The fix for the measured root cause is already in-tree and has never been run: a batch-GEMM verifier (LATTICE_MTP_BATCH=1) purpose-built to cut the sequential-forward cost, with zero tests, zero benches, and zero references anywhere else in the repo.

Structure

A Measured / source-verified reality table (M1–M16), each row tagged runtime-measured / unit-test-gate-pinned / source-read with a durable pointer, held apart from an untrusted external prior-art survey folded as data. The survey independently reached NEEDS-EXPERIMENT but from a repo-blind vantage; its central acceptance figure is an imported stale number the source issue itself caveats, and it misidentifies the bottleneck as inherent GDN verify-cost growth when the actual measured cost is the sequential-forward structure at K=1 (which has an unrun in-tree mitigation).

Decision

Gates the feature behind a defined minimal experiment:

• E1 (cheapest, zero new code): flip LATTICE_MTP_BATCH=1 and re-run the A/B under the GPU flock — decides whether MTP has any future on this architecture at the cost of two env-var flips.
• E2 (prerequisite to ranking): extend the existing counter-rotation equivalence test to the shipped Q4 checkpoint's MTP tensors, to resolve whether 0% acceptance is a weight-basis artifact.
• E3 (deferred): temperature-correct probabilistic acceptance — genuinely missing, but in a dormant, production-unreachable path; sequenced after E1/E2.
• E4 (far): tree-structured verification, gated on GDN-snapshot rollback and on E1–E3.

Includes an explicit honest closed-by-measurement branch: if the batch-GEMM verifier is net-negative everywhere even under the fast verifier, MTP is closed by measurement on this GDN-hybrid family, and the structurally-different LATTICE_SELF_SPEC mechanism (GDN-only draft, no MTP head, currently zero measured data and no design doc) becomes the leading speculative-decode lever, worth its own ADR and first measurement.

Supersedes none of ADR-006 / ADR-050 / ADR-051; sequences their open follow-ups at the priority level.

Testing

Docs-only. No executable behavior changes.

[ohdearquant]

SPEC-GATE: SIGNED (Leo). NEEDS-EXPERIMENT verdict ratified — same experiment-gated structure as ADR-073. Live correctness settled (off-default, greedy-only, K=1, argmax-equivalent regression-tested); the only throughput number is self-flagged unreliable with the in-tree never-run batch-GEMM fix, so E1 (two env-var flips + bench rerun) is correctly the single decision point. Self-spec held apart as P-separate with its own ADR requirement — right call, do not fold it into MTP in any priority doc. Rider: if E1 confirms net-negative under the fast verifier, close MTP on the GDN-hybrid family explicitly in a follow-up S-row on this ADR (measured closure, not silence), and LATTICE_SELF_SPEC becomes the lever.