Repository navigation

mlx: opt-in dense GEMM for nvfp4 prefill (dequantise to bf16 above a row threshold) - #287

Draft

glennneuber wants to merge 21 commits into

mainfrom

feat/mlx-prefill-dequant-gemm

Draft

mlx: opt-in dense GEMM for nvfp4 prefill (dequantise to bf16 above a row threshold)#287
glennneuber wants to merge 21 commits into
mainfrom
feat/mlx-prefill-dequant-gemm

Commits on Sep 10, 2026
- mlx: opt-in dense GEMM for nvfp4 prefill (dequantise to bf16 above a row threshold)
  
  glennneuber
  and
  claude
  committed
  57e53b8
- task: two bench runs after the campaign — dense path 1.5-2x, FP4 2.3-4.6x at >= 2048 rows, repeatable
  
  glennneuber
  and
  claude
  committed
  7a5d8e9
- task: the MLX PTX cache must be one directory per GPU architecture
  
  glennneuber
  and
  claude
  committed
  bbd4280
- task: runner measurements — 1.2-2.1x on image prefill, but up to +6.8 GiB peak
  
  glennneuber
  and
  claude
  committed
  f8a240f
- task: the control says the flag changes output deterministically — criterion 2 fails
  
  glennneuber
  and
  claude
  committed
  5c9705e
- mlx: release the dequantised copy as soon as the matmul holds it
  
  glennneuber
  and
  claude
  committed
  0e58c23
- task: the release rework does not bound the transient — negative result
  
  glennneuber
  and
  claude
  committed
  6c75037
- task: forcing per-layer evaluation is worse — 64.7 GiB and OOM; the port does not work
  
  glennneuber
  and
  claude
  committed
  9bce168
- kernels: a Triton mixed-input GEMM ceiling probe — slower than MLX's own kernel
  
  glennneuber
  and
  claude
  committed
  efd7f8f
- task: what the Metal quantised path does differently, and what CUDA can borrow
  
  glennneuber
  and
  claude
  committed
  90b26dd
- task: is it CUDA 12.8? No for MLX (13.0, sm_120a native); plausibly for the Triton probe
  
  glennneuber
  and
  claude
  committed
  3d36fe7
- task: similar reports elsewhere, and toolchain facts checked with the host's ptxas
  
  glennneuber
  and
  claude
  committed
  e000fef
- task: a real sm120 mixed-input kernel reaches 0.65-0.84x cuBLAS — kernel route viable
  
  glennneuber
  and
  claude
  committed
  3294a48
Commits on Sep 11, 2026
- task: producer-warp sm120 kernel reaches 0.92-0.95x the dense kernel, verified
  
  glennneuber
  and
  claude
  committed
  0ad1bf4
- kernels: time the Triton prototype with the median, like the sm120 harness
  
  glennneuber
  and
  claude
  committed
  1077955
- task: correct — the new kernel avoids the memory failure, not the output change
  
  glennneuber
  and
  claude
  committed
  f157a02
- task: quiet-GPU rerun and Nsight Compute stall attribution
  
  glennneuber
  and
  claude
  committed
  fd74cf3
- task: ptxas 13.0 changes nothing for the Triton prototype; audit the quiet window
  
  glennneuber
  and
  claude
  committed
  a1e8774
- task: third sm120 iteration, the epilogue moves into the ring (+1.6-6.5 % on every row)
  
  glennneuber
  and
  claude
  committed
  7e6bc1c
- task: quiet confirmation of the third sm120 iteration; the MLX bench caveat resolved
  
  glennneuber
  and
  claude
  committed
  a2a9e9e
- task: race-check the third sm120 iteration; racecheck cannot see TMA writes
  
  glennneuber
  and
  claude
  committed
  c51417e

Commits

Commits on Sep 10, 2026

Commits on Sep 11, 2026