PR6: NVFP4-tier KV + entropy cold pool (10L default table, +33% context ceiling) - #6

Open

Astrangemaninhere wants to merge 48 commits into

nvfp4-cold-pool

Commits on Aug 30, 2026

graphs: extend the ordinary decode ladder on demand
NInfer Agent
committed
db77581
feat(kv): per-layer KV storage with data-driven per-layer defaults
NInfer Agent
committed
c561b53
fix(kv): declare default table for the 35B variant too (empty prior)
NInfer Agent
committed
d5438fd
fix(kv): wire --kv-layer-storage parsing into the CLI
NInfer Agent
committed
7b0d344
feat(kv): entropy-coded cold pool for INT8-tier pages (raw nibble slots)
NInfer Agent
committed
962912c
feat(kv): entropy-coded cold pool for INT8-tier pages (raw nibble slots)
NInfer Agent
committed
52102e2
feat(runtime): cold-compress pass at the decode boundary + build wiring
NInfer Agent
committed
aa68101
refactor(kv): keep cold-pool additions self-contained (ops/types/CLI only)
NInfer Agent
committed
9fd642f
feat(runtime): cold-compress pass at the decode boundary + build wiring
NInfer Agent
committed
44ad532
feat(runtime): cold-compress pass at the decode boundary + build wiring
NInfer Agent
committed
91a9cd9
feat(runtime): cold-compress pass at the decode boundary + build wiring
NInfer Agent
committed
0022161
feat(runtime): cold-compress pass at the decode boundary + build wiring
NInfer Agent
committed
562b5dc
feat(kv): complete cold-pool mechanism on the paged KV store
NInfer Agent
committed
17f56af
feat(kv): full cold-pool window mechanism on master's paged KV store
NInfer Agent
committed
d6fad56
fix(kv): cold pool under host offload + multi-concurrency
NInfer Agent
committed
c0c17ff
feat(spec): length-based backend decision policy + DFlash2 enum/CLI
NInfer Agent
committed
8bf1234
feat(rope): static YaRN factor-4 extension
NInfer Agent
committed
8e16b1a
feat(yarn): extend context budget and attention validation to 4x native
NInfer Agent
committed
2d10e70
feat(yarn): allow explicit --kv-capacity below max_context
NInfer Agent
committed
7d80637
fix(yarn): relax --kv-capacity >= --max-context serve/CLI validation
NInfer Agent
committed
d134bca
fix(yarn): relax engine kv_capacity floor to explicit pool size
NInfer Agent
committed
0500170
fix(yarn): allow physical page pool below logical max_context
NInfer Agent
committed
f2d1779

Commits on Aug 31, 2026

feat(spec): port DFlash2 draft backend + length-based switching
NInfer Agent
committed
3ac3b04
fix(spec): DFlash2 smoke-test fixes
NInfer Agent
committed
e97e571
merge: integrate PR1 per-layer KV storage into cold-pool+yarn base
NInfer Agent
committed
f8c8714
merge: integrate PR3 DFlash2 backend into cold-pool+yarn+PR1 base
NInfer Agent
committed
0895988
merge: integrate PR4 on-demand graph capture into all-PR base
NInfer Agent
committed
a070e1d
merge: resolve PR4 conflict markers in program.h member block
NInfer Agent
committed
eb5ef4d
fix(kv): FP8 per-layer scale planes must be FP16 (PR1 interop)
NInfer Agent
committed
65817e5
fix(kv): copy layer_kv_dtypes into the sequence candidate (PR1 table never applied)
NInfer Agent
committed
2f74b3d
fix(kv): resolve per-layer quant_group in layer views (PR1 mixed tables)
NInfer Agent
committed
3f1ebde
fix(kv): per-layer scaled/stride resolution in layer views (PR1 mixed tables)
NInfer Agent
committed
64ae345
fix(kv): per-layer plane base prefix sums for mixed dtype pools (PR1)
NInfer Agent
committed
fefaa81
feat(kv): port GQA attention ops + NVFP4/ISO3 layer machinery (batch 1)
NInfer Agent
committed
82f036e
feat(kv): switch main text attention to GQA ops (A1/A2/A3)
NInfer Agent
committed
8ecf1da
feat(kv): NVFP4-major default layer table (NR prior: fill NVFP4, 12 sensitive layers to I8)
NInfer Agent
committed
c04480d
fix(kv): NVFP4 layers carry scale planes in layer views (scaled/stride)
NInfer Agent
committed
e318cd7
feat(kv): NVFP4-major default table (10/16 layers NVFP4, 6 sensitive I8)
NInfer Agent
committed
ad80e69
fix(kv): restore NR layer table (12I8+4NVFP4) - best measured perplexity
NInfer Agent
committed
c10624a
feat(kv): measured-optimal 8/8 layer table (NVFP4 shallow-mid, I8 dense-upper)
NInfer Agent
committed
f1881d8
feat(kv): 10L default table - L13/14 demote free (1.3344 zh / 1.1952 ctx8k, parity 8L)
NInfer Agent
committed
821441d
feat(kv): NVFP4-tier cold pool - entropy rANS slots + cold fixes
NInfer Agent
committed
9cf0867
fix(runtime): bound shared capture target expansion

Neroued
authored and
NInfer Agent
committed
ab55203
feat(kv): ColdPolicy::Host - cold-slot payloads in pinned host memory
NInfer Agent
committed
d8854a4
Revert "feat(kv): ColdPolicy::Host - cold-slot payloads in pinned host memory"
NInfer Agent
committed
036f4b1
fix(kv): cold-pool pages count as resident in restore inventory
NInfer Agent
committed
de8c074
fix(spec): DFlash2 graph-prep segfault + --spec auto resolution
NInfer Agent
committed
9bf5135
fix(spec): auto picks backend from artifact weights, MTP fallback
NInfer Agent
committed
d4f6e0d