feat(runtime): NVMe cold tier (ColdPolicy::Disk) under the cold-pool planner - #149

Closed

Astrangemaninhere wants to merge 45 commits into

Neroued:masterfrom

Astrangemaninhere:disk-cold-tier

Commits on Sep 1, 2026

feat(kv): entropy-coded cold pool for INT8-tier pages (raw nibble slots)
NInfer Agent
committed
119f521
feat(kv): entropy-coded cold pool for INT8-tier pages (raw nibble slots)
NInfer Agent
committed
96403c0
feat(runtime): cold-compress pass at the decode boundary + build wiring
NInfer Agent
committed
3f8302b
refactor(kv): keep cold-pool additions self-contained (ops/types/CLI only)
NInfer Agent
committed
a496172
feat(runtime): cold-compress pass at the decode boundary + build wiring
NInfer Agent
committed
24bee58
feat(runtime): cold-compress pass at the decode boundary + build wiring
NInfer Agent
committed
a1d0c4e
feat(runtime): cold-compress pass at the decode boundary + build wiring
NInfer Agent
committed
fa343d1
feat(runtime): cold-compress pass at the decode boundary + build wiring
NInfer Agent
committed
5b6627d
feat(kv): complete cold-pool mechanism on the paged KV store
NInfer Agent
committed
e3a87d6
feat(kv): full cold-pool window mechanism on master's paged KV store
NInfer Agent
committed
21715c7
fix(kv): cold pool under host offload + multi-concurrency
NInfer Agent
committed
863a7d2
feat(rope): static YaRN factor-4 extension
NInfer Agent
committed
b240910
feat(yarn): extend context budget and attention validation to 4x native
NInfer Agent
committed
9f6c429
feat(yarn): allow explicit --kv-capacity below max_context
NInfer Agent
committed
eae9c5e
fix(yarn): relax --kv-capacity >= --max-context serve/CLI validation
NInfer Agent
committed
44ef067
fix(yarn): relax engine kv_capacity floor to explicit pool size
NInfer Agent
committed
10c9e79
fix(yarn): allow physical page pool below logical max_context
NInfer Agent
committed
2597718
feat(kv): per-layer KV storage with data-driven per-layer defaults
NInfer Agent
committed
6076942
fix(kv): declare default table for the 35B variant too (empty prior)
NInfer Agent
committed
0d28c84
fix(kv): wire --kv-layer-storage parsing into the CLI
NInfer Agent
committed
6cdfe69
feat(spec): length-based backend decision policy + DFlash2 enum/CLI
NInfer Agent
committed
f9b99f9
feat(spec): port DFlash2 draft backend + length-based switching
NInfer Agent
committed
22cccb0
fix(spec): DFlash2 smoke-test fixes
NInfer Agent
committed
a3a54fd
graphs: extend the ordinary decode ladder on demand
NInfer Agent
committed
8136ee9
merge: resolve PR4 conflict markers in program.h member block
NInfer Agent
committed
6002e7b
fix(kv): FP8 per-layer scale planes must be FP16 (PR1 interop)
NInfer Agent
committed
1fd48b4
fix(kv): copy layer_kv_dtypes into the sequence candidate (PR1 table never applied)
NInfer Agent
committed
51c00c3
fix(kv): resolve per-layer quant_group in layer views (PR1 mixed tables)
NInfer Agent
committed
c67ca8e
fix(kv): per-layer scaled/stride resolution in layer views (PR1 mixed tables)
NInfer Agent
committed
190d893
fix(kv): per-layer plane base prefix sums for mixed dtype pools (PR1)
NInfer Agent
committed
b9e1f4b
feat(kv): port GQA attention ops + NVFP4/ISO3 layer machinery (batch 1)
NInfer Agent
committed
f48bb30
feat(kv): switch main text attention to GQA ops (A1/A2/A3)
NInfer Agent
committed
767bbb7
feat(kv): NVFP4-major default layer table (NR prior: fill NVFP4, 12 sensitive layers to I8)
NInfer Agent
committed
82e400c
fix(kv): NVFP4 layers carry scale planes in layer views (scaled/stride)
NInfer Agent
committed
4527a67
feat(kv): NVFP4-major default table (10/16 layers NVFP4, 6 sensitive I8)
NInfer Agent
committed
ab4339d
fix(kv): restore NR layer table (12I8+4NVFP4) - best measured perplexity
NInfer Agent
committed
6b3ec71
feat(kv): measured-optimal 8/8 layer table (NVFP4 shallow-mid, I8 dense-upper)
NInfer Agent
committed
9228b80
feat(kv): 10L default table - L13/14 demote free (1.3344 zh / 1.1952 ctx8k, parity 8L)
NInfer Agent
committed
5acf31d
feat(kv): NVFP4-tier cold pool - entropy rANS slots + cold fixes
NInfer Agent
committed
b2075f8
feat(kv): ColdPolicy::Host - cold-slot payloads in pinned host memory
NInfer Agent
committed
e61ccf3
Revert "feat(kv): ColdPolicy::Host - cold-slot payloads in pinned host memory"
NInfer Agent
committed
03a4b70
fix(kv): cold-pool pages count as resident in restore inventory
NInfer Agent
committed
f7a9b80
fix(spec): DFlash2 graph-prep segfault + --spec auto resolution
NInfer Agent
committed
4384099
fix(spec): auto picks backend from artifact weights, MTP fallback
NInfer Agent
committed
ba288d1
feat(runtime): NVMe cold tier (ColdPolicy::Disk) under the cold-pool planner
NInfer Agent
committed
4e9f5d5