Skip to content

Add RotorQuant KV cache backend with deferred prefill on Metal - #103

Open
ttupper92618 wants to merge 22 commits into
mainfrom
kv-cache-rotorquant
Open

Add RotorQuant KV cache backend with deferred prefill on Metal#103
ttupper92618 wants to merge 22 commits into
mainfrom
kv-cache-rotorquant

Force sequential generation for Gemma 4

856b30f
Select commit
Loading
Failed to load commit list.

Select a check to view from the sidebar