kv-cache : spend the partial residency budget on the slowest link first - #68

kv-cache : spend the partial residency budget on the slowest link first#68

Piggidragon wants to merge 10 commits into

GenerelSchwerz:llama/devGenerelSchwerz/llama.cpp:llama/devfrom

Piggidragon:mgpu/kv-residency-bandwidthPiggidragon/llama.cpp:mgpu/kv-residency-bandwidthCopy head branch name to clipboard