Repository navigation

fp8 KV cache in the in-tree prefill kernel - #368

Merged

demandal25 merged 21 commits into

amd-integrationfrom

rocm-fp8-kv-fa2

Sep 15, 2026

Merged

fp8 KV cache in the in-tree prefill kernel#368
demandal25 merged 21 commits into
amd-integrationfrom
rocm-fp8-kv-fa2

Commits on Sep 14, 2026
- feat(rocm): serve an fp8 KV cache in the in-tree prefill kernel
  
  demandal25
  and
  claude
  committed
  5188063
- fix(rocm): refuse the fp8 shapes the prefill kernel cannot serve
  
  demandal25
  and
  claude
  committed
  719560c
- test(rocm): cover fp8 KV on the prefill kernel, and the refusals
  
  demandal25
  and
  claude
  committed
  8e76161
- docs(rocm): record fp8 KV on the HIP path, and what it refuses
  
  demandal25
  and
  claude
  committed
  2835a84
- bench(rocm): give bench_fa2_prefill a KV-dtype axis
  
  demandal25
  and
  claude
  committed
  4fd9e14
- Merge branch 'amd-integration' into rocm-fp8-kv-fa2
  demandal25
  authored
  10f7243
- fix(rocm): pin the AITER output dtype to what its launchers accept
  
  demandal25
  and
  claude
  committed
  7a88247
- test(rocm): cover the single-decode k/v mismatch directly
  
  demandal25
  and
  claude
  committed
  f50c834
- fix(rocm): refuse before the URI at every public generator, not three of them
  
  demandal25
  and
  claude
  committed
  3426280
- fix(rocm): keep the fp8 KV load off an fp8 array, which gfx950 cannot construct
  
  demandal25
  and
  claude
  committed
  54cdcae
- fix(rocm): match the AITER guards to the launchers, and run them before the import
  
  demandal25
  and
  claude
  committed
  c63616a
- fix(rocm): close the customize-POD bypass of the fp8 refusal
  
  demandal25
  and
  claude
  committed
  9be6d09
- Merge branch 'amd-integration' into rocm-fp8-kv-fa2
  demandal25
  authored
  2c963d4
Commits on Sep 15, 2026
- fix(rocm): stage the fp8 KV load with bit_cast, not a punned uint32_t
  
  demandal25
  and
  claude
  committed
  81f747d
- test(rocm): cover single-decode fp8 numerics, and say which decode is exempt
  
  demandal25
  and
  claude
  committed
  1ed1569
- test(rocm): tighten the single-decode tolerance, and document the path it covers
  
  demandal25
  and
  claude
  committed
  fb9ab1b
- Merge branch 'amd-integration' into rocm-fp8-kv-fa2
  demandal25
  authored
  e8c1071
- Potential fix for pull request finding
  
  demandal25
  and
  Copilot
  authored
  8e60d07
- test(rocm): cover fp8 KV at head_dim 256
  
  demandal25
  and
  claude
  committed
  d86993c
- test(rocm): stop claiming coverage of a C++ guard this test cannot reach
  
  demandal25
  and
  claude
  committed
  cccee3b
- fix(rocm): replace the last bare assert on the fp8-query path with a sentence
  
  demandal25
  and
  claude
  committed
  4ce31b2

Commits

Commits on Sep 14, 2026

Commits on Sep 15, 2026