Skip to content

Faster inference by caching the KV projections - #15

Merged
rushin682 merged 2 commits into
peng-lab:mainfrom
Harry25R:perf/inference-speedup-v2
Sep 9, 2026
Merged

Faster inference by caching the KV projections#15
rushin682 merged 2 commits into
peng-lab:mainfrom
Harry25R:perf/inference-speedup-v2

test: cover the fast K/V-caching sampler on CPU

63c9e60
Select commit
Loading
Failed to load commit list.
Sign in for the full log view
package
succeeded Aug 22, 2026 in 12s