Skip to content

docs(benchmark): correct the #1782 record's byte-identity claim #1942

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
inureyes merged 1 commit into from
Sep 21, 2026
Merged

docs(benchmark): correct the #1782 record's byte-identity claim #1942

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/benchmark_results/dflash-verify-fixed-cost-gb10-2026-09-11.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
> **Correction (2026-09-21), issued after issue #1935 and PR #1939.** The byte-identity claim in this record was never true. The DFlash burst prefilled the prompt through `forward_speculative`, which sends full-attention layers through `attend_per_position` one query position at a time, rather than through the single batched causal attention classic decode runs over a prompt, so the burst entered its first round from different KV and gated-delta state at every width. `attend_per_position` landed on 2026-05-24 (`4038da96`), before this record was taken, and `Qwen35Model` did not override the prefill hook until PR #1939 (`76c849b5`), so the property did not regress after this measurement: it had not held when this measurement was made. Do not cite the "Greedy text is byte-identical to classic on every after arm" bullet, and do not cite the attribution of block 16's divergence to the ignored chain-parity flag; the prefill was the cause, and chain parity was shown to change acceptance and burst time without changing the completion. What still holds: where the per-round time goes, the two fixes being serial rather than additive, the host-build and device-sync split, the width cliff between 4 and 6 rows and its `dispatch_multirow_width` explanation, and the separate `MLX_MAX_OPS_PER_BUFFER` finding. What needs re-checking rather than trusting: every number that depends on acceptance, since the prefill fix changes the state each round starts from. See [`dflash-verify-divergence-qwen35-gb10-2026-09-20.md`](dflash-verify-divergence-qwen35-gb10-2026-09-20.md).

# DFlash verify block: where the fixed per-round device cost goes (GB10, 2026-09-11)

Issue #1782. Host: NVIDIA GB10 (sm_121, DGX Spark), Linux aarch64, MLX pin `81ba1c6a`, CUDA build (`MLX_CUDA_ARCHITECTURES=121`), release profile. Otherwise idle host (load 0.1 to 1.5 from the run's own client, no other model, build or CI job during any measured run; the self-hosted CI runner was up but idle). GPU held under the scratchpad lock for every run. Warm PTX cache (`~/.cache/mlxcel/cuda-ptx/81ba1c6a...`, rule 3 of the Volta record) and warm page cache (both checkpoints had been read by the smoke run before any measured run, rule 7). `MLX_ENABLE_TF32` was left at MLX's default; no numerical comparison in this record depends on it (byte-identity below compares token strings, not floats).
Expand Down
Loading