From b2a9b979bc822fe794d005da46c1667ddb7c738a Mon Sep 17 00:00:00 2001 From: Jeongkyu Shin Date: Mon, 21 Sep 2026 13:41:58 +0900 Subject: [PATCH] docs(benchmark): correct the #1782 record's byte-identity claim The 2026-09-11 record states that greedy text is byte-identical to classic decode on every after arm including block 16. Issue #1935 and PR #1939 established that this never held: the DFlash burst prefilled the prompt through `forward_speculative`, which drives full-attention layers one query position at a time instead of the single batched causal attention classic decode runs, so the burst entered its first round from different KV and gated-delta state at every width. `attend_per_position` predates the measurement, so the property had not held when the record was written rather than regressing afterwards. The record is corrected in place with a banner rather than edited, matching how the superseded #1820 record is handled, because #1797's issue body already cites this table and a reader arriving at the file directly needs the correction there. The banner also withdraws the attribution of block 16's divergence to the ignored chain-parity flag, which PR #1939 showed changes acceptance and burst time without changing the completion, and flags that every acceptance-dependent number needs re-checking against the fixed prefill. Refs #1935 --- .../dflash-verify-fixed-cost-gb10-2026-09-11.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/benchmark_results/dflash-verify-fixed-cost-gb10-2026-09-11.md b/docs/benchmark_results/dflash-verify-fixed-cost-gb10-2026-09-11.md index d5da47353..443ab1406 100644 --- a/docs/benchmark_results/dflash-verify-fixed-cost-gb10-2026-09-11.md +++ b/docs/benchmark_results/dflash-verify-fixed-cost-gb10-2026-09-11.md @@ -1,3 +1,5 @@ +> **Correction (2026-09-21), issued after issue #1935 and PR #1939.** The byte-identity claim in this record was never true. The DFlash burst prefilled the prompt through `forward_speculative`, which sends full-attention layers through `attend_per_position` one query position at a time, rather than through the single batched causal attention classic decode runs over a prompt, so the burst entered its first round from different KV and gated-delta state at every width. `attend_per_position` landed on 2026-05-24 (`4038da96`), before this record was taken, and `Qwen35Model` did not override the prefill hook until PR #1939 (`76c849b5`), so the property did not regress after this measurement: it had not held when this measurement was made. Do not cite the "Greedy text is byte-identical to classic on every after arm" bullet, and do not cite the attribution of block 16's divergence to the ignored chain-parity flag; the prefill was the cause, and chain parity was shown to change acceptance and burst time without changing the completion. What still holds: where the per-round time goes, the two fixes being serial rather than additive, the host-build and device-sync split, the width cliff between 4 and 6 rows and its `dispatch_multirow_width` explanation, and the separate `MLX_MAX_OPS_PER_BUFFER` finding. What needs re-checking rather than trusting: every number that depends on acceptance, since the prefill fix changes the state each round starts from. See [`dflash-verify-divergence-qwen35-gb10-2026-09-20.md`](dflash-verify-divergence-qwen35-gb10-2026-09-20.md). + # DFlash verify block: where the fixed per-round device cost goes (GB10, 2026-09-11) Issue #1782. Host: NVIDIA GB10 (sm_121, DGX Spark), Linux aarch64, MLX pin `81ba1c6a`, CUDA build (`MLX_CUDA_ARCHITECTURES=121`), release profile. Otherwise idle host (load 0.1 to 1.5 from the run's own client, no other model, build or CI job during any measured run; the self-hosted CI runner was up but idle). GPU held under the scratchpad lock for every run. Warm PTX cache (`~/.cache/mlxcel/cuda-ptx/81ba1c6a...`, rule 3 of the Volta record) and warm page cache (both checkpoints had been read by the smoke run before any measured run, rule 7). `MLX_ENABLE_TF32` was left at MLX's default; no numerical comparison in this record depends on it (byte-identity below compares token strings, not floats).