pack: use NVMe as a cache tier so the model need not live there

[physics515]

Measured yesterday: a qwen4exp reference paging its working set off /mnt/deepmem ran at 63 s/token. That volume is a 4-device btrfs over four spinning disks shared with a household stack.

The obvious fix — copy the model to NVMe — doesn't generalise: a 207 GB pack and a 111 GB model don't both fit the fast disk, and copying is a manual step that has to be repeated. So: leave the pack where it lives and cache the ranges actually read.

mummu::diskcache::DiskCache sits in front of Pack::read_range, the single choke point every blob byte already passes through.

Keyed on the exact range, not on blocks

A general cache has to guess a block size. This one doesn't: a pack is read at (precision, offset, len) triples that are identical every token — same tensor, same slice. So a lookup is an exact hit or an exact miss, with no partial-block reassembly and no read amplification from a block size that fits nothing.

Eviction is hotness-based, not LRU

Coldest-first on decaying hotness with an LRU tiebreak. MoE routing is skewed and plain LRU discards a persistently hot expert after one cold burst — llama.cpp #25294 reports the same and evicts the same way.

The test is the MoE case directly: one entry routed repeatedly, others touched once, and the hot one must survive an insert that forces eviction even though it is no longer the most recent.

Failure behaviour is "degrade to a miss", never "fail the load"

• a cache file that vanished (something cleaning /var/tmp) or is short reads as a miss and drops its own bookkeeping
• an entry larger than the whole cache is declined, rather than evicting everything to still not fit
• writes go to a temp name and are renamed, so a crash can't leave a short file a later read trusts by size
• a pack read returns identical bytes whether they came from the blob or the cache — placement affects speed, never contents — with a test asserting exactly that against a real read_range

Opt-in, deliberately

MUMMU_DISK_CACHE_DIR (+ MUMMU_DISK_CACHE_GB, default 64). Never implicit: this writes tens of GB to whatever filesystem it's pointed at, which isn't something to start doing to someone's root volume because a model happened to load.

Scope

Read-through only — it populates on miss and never prefetches. The access order is known in advance (manifest order; workingset already exploits exactly that for RAM→VRAM), so preload is the natural next increment and is filed as its own roadmap item, along with the two other things the research flagged: O_DIRECT once the model exceeds RAM, and per-session ownership of the residency pool.

Verification

• cargo fmt --check clean; cargo clippy --workspace --all-targets clean
• workspace lib tests green at 496 (+8 new)

🤖 Generated with Claude Code