feat(kvcache): store the KV cache as fp8 e4m3 codes (--kv-cache-dtype… - #354

Open

ArqAlice wants to merge 4 commits into

FlashML-org:mainfrom

ArqAlice:feat/fp8-quantization

Commits on Sep 2, 2026

feat(kvcache): store the KV cache as fp8 e4m3 codes (--kv-cache-dtype fp8)
ArqAlice
committed
03fb043

Commits on Sep 4, 2026

fix(kernels): give the V tensor its own row pitch in the fp8 KV store

MT-z
and
claude
committed
811ccee
Merge pull request #1 from MT-z/fix/kv-fp8-vstore-pitch
ArqAlice
authored
5b9efc5
Merge branch 'FlashML-org:main' into feat/fp8-quantization
ArqAlice
authored
3e5bbdd