test: ADR 0007 plus CLI/server parity and B=1 decode benchmark harness #2167

Part of #2166. Phase 0. No dependencies. Every later phase relies on this one for its baseline, its regression threshold, and the default decisions.

Problem / Background

Epic #2166 merges the CLI decode path (CxxGenerator, src/lib/mlxcel-core/src/generate.rs:1914-2962) and the server decode path (BatchScheduler, src/server/batch/scheduler/) into one batch-native engine. Before any code moves, three things must be on record. First, the target architecture, as an ADR. Second, the divergence we start from, measured by a harness that later phases rerun to show convergence. Third, the throughput we must not lose. No measurement of this kind exists today. src/bin/bench_decode.rs:381 and :412 time only CxxGenerator. docs/CONTINUOUS_BATCHING.md:65-79 disclaims token exactness without measuring it.

Proposed Solution

1. ADR 0007 (docs/adr/0007-unified-batch-native-engine.md, next free number after 0006). It records the target architecture: an engine layer in mlxcel-core (open, prefill, batched step, close; a single sequence is a batch of one), one KV interface with dense and paged storage behind it, one sampling and finish step, the server scheduler built on the engine, mlxcel run and chat as in-process clients of the server engine, and a thin raw generate (the llama-completion analog) on the engine. It names the engine API types. It supersedes ADR 0004's deferral of the KV, batching and scheduler abstraction, and it updates the ADR index in docs/adr/README.md. It cites llama.cpp HEAD 65840ed: llama_decode(llama_batch), common_sampler_*, tools/cli/CMakeLists.txt linking llama-server-impl, tools/cli/cli-server.h, tools/cli/cli-context.cpp, cli: new CLI experience ggml-org/llama.cpp#17824 and cli : move to HTTP-based implementation ggml-org/llama.cpp#24948.
2. Decision table in the ADR. It covers every CLI vs server default that differs today and states the value the unified engine adopts, with a one-line reason for each. Rows: repeat_last_n (CLI full history -1, server 64 at src/server/startup.rs:747); DRY sequence breakers (CLI empty at src/commands/generate.rs:1189-1260, server --dry-sequence-breaker); loop detection (off on the CLI, since src/execution/sampling.rs:175 leaves it at default and only the server control plane sets it); prefill chunk (CLI 2048 at generate.rs:282, server 512 at src/server/config.rs:1172); the samplers the CLI hard-codes off at src/commands/generate.rs:1189-1260 (frequency and presence penalty, XTC, mirostat, dynatemp). Ask the maintainer to confirm the table in a comment on this issue. Phase 5 does not start until it is confirmed.
3. Cross-path parity harness. It runs the same prompt and SamplingConfig through (a) CxxGenerator, (b) the server engine at B=1 with --decode-storage-backend dense, and (c) B=1 paged, all under MLXCEL_SDPA_DETERMINISTIC=1 (fix(cuda): add MLXCEL_SDPA_DETERMINISTIC for reproducible decode #2162). For each pair it reports the index and token ids of the first divergent token. It covers greedy, seeded sampling with penalties and DRY, and a prompt-cache hit vs miss on (b). Today these runs are expected to differ. The harness records that baseline rather than failing on it, and it takes a --expect-identical switch so later phases can turn it into a gate.
4. Server B=1 single-stream decode benchmark. It sits next to bench_decode's CxxGenerator numbers and drives the scheduler in-process, so HTTP is not in the timed region. It runs at short context (a few hundred keys) and long context (about 8K keys) on at least Qwen3-1.7B 4-bit and Llama-3.2-1B 4-bit. It reports decode tok/s and TTFT.
5. Regression threshold. The ADR records the bar that later phases must meet, for example unified single-stream decode within 3 percent of today's CxxGenerator at both context lengths. Choose the number from measured run-to-run noise on a quiet host. Do not pick it in advance.

Scope

In scope: docs/adr/0007-*.md, docs/adr/README.md, the harness and benchmark (a src/bin/ target or a test plus a script under scripts/), a Makefile target for each, and baseline results under docs/benchmark_results/.

Out of scope: any change to decode behavior. This phase only measures and decides.

Implementation Notes

• Reuse: drive the server path through the same in-process scheduler entry the existing server tests use, not through a spawned HTTP server. Reuse build_sampling_config and the bench_decode model-loading and CSV conventions.
• Prior art: two parity harnesses already exist, and both spawn mlxcel-server over HTTP, so they inform the method rather than the in-process driver. tests/speculative_parity.rs asserts greedy byte equality of speculative vs classic serving, checks the server log so a silent fallback cannot pass, and handles near-ties in check_row_parity_with_near_tie (:606). The fix(gemma4): make MTP verify byte-identical to decode on CUDA #2185 harness in docs/benchmark_results/data/gemma4-mtp-cuda-gb10-2026-10-07/harness/ has freegen_parity.py (classic vs MTP chat byte identity, failing unless the log shows the probe passing and a burst running) and mtp_rounds.py plus summarize.py (interleaved rounds with a classic-vs-classic null arm behind the perf(cuda): bucket the cuDNN SDPA plan-cache key across verify rounds #1820 hostgate from docs/benchmark_results/data/sdpa-plan-bucket-gb10-2026-09-12/harness/). Reuse the anti-fallback log check and the null-arm protocol; perf(scheduler): restore Gemma 3 decode lookahead via ring rewind #2182 measured its own floor the same way (null arm within 1 percent), which is the kind of noise figure item 5 needs.
• Edge cases: a model that does not support paged decode (supports_paged_decode_backend() == false) reports the (c) column as n/a, not as a failure. Divergence at token 0 is reported as such. With the prompt cache on, classic serving prefills a chat prompt in two forwards split at the history boundary (capture_history_boundary_snapshot, src/server/batch/scheduler/prompt_cache.rs:1088, feat: Donate a history-boundary snapshot during prefill for snapshot-only families #1143), and fix(gemma4): make MTP verify byte-identical to decode on CUDA #2185 measured that split alone changing Gemma 4 greedy output (parity held with --no-prompt-cache). Each (b) row therefore records whether the prompt cache was on, and the hit vs miss comparison states the prefill partition on each side.
• Measurement policy: verify on real checkpoints (download them if missing). Run the benchmark once, at the end, on a quiet host. Record the host, commit and MLX pin with the numbers.

Acceptance Criteria

• ADR 0007, with the architecture, the decision table and the regression threshold row (semantic rows decided with reasons; speed rows and the threshold list candidates and the deciding measurement, values TBD (measured in this PR))
• The maintainer has confirmed the decision table in a comment on this issue Replaced by the maintainer comment of 2026-10-07: no confirmation pause, the table is the authority and every row states its evidence or reason
• The parity harness runs from one documented make target (make engine-parity) and prints the first divergent token per pair. With --expect-identical it exits non-zero on divergence
• Today's parity baseline output is committed under docs/benchmark_results/ (orchestrating session, quiet host)
• The server B=1 decode benchmark runs from one documented make target (make bench-engine), on both paths, at both context lengths, next to bench_decode's CxxGenerator numbers
• The benchmark takes the prefill chunk size (both paths) and the decode storage backend (server) as parameters, measures TTFT and decode tok/s, and scripts/engine_bench_rounds.py runs the interleaved A/B rounds with a null arm (added from the 2026-10-07 comment)
• Baseline numbers for both models at both context lengths, the prefill-chunk and storage A/Bs, and the null arm are measured and committed under docs/benchmark_results/, and the TBD cells of ADR 0007 are filled in (orchestrating session, quiet host)
• The harness and benchmark call the real CxxGenerator and BatchScheduler code paths, not copies

Verification

make verify                                   # macOS gate
make verify-fmt verify-test-cuda              # Linux/CUDA gate
MLXCEL_SDPA_DETERMINISTIC=1 make <parity-target> MODEL=<Qwen3-1.7B-4bit path>
make <server-decode-bench-target> MODEL=<Llama-3.2-1B-4bit path>

A pass means the gates are green, the harness prints one row per pair with a divergence index (or identical), and the benchmark prints tok/s at both context lengths for both paths.

References verified against main at fc21784 (2026-10-07).