qwen4exp: upstream catch-up, sparse-fa, and NextN/MTP support - #340

Merged

TheTom merged 16 commits into

TheTom:feature/turboquant-kv-cachefrom

giveen:qwen4-catchup

Sep 3, 2026

Commits on Sep 2, 2026

qwen4exp: catch up to upstream through 36b101543
giveen
committed
925792c
CUDA: XOR swizzle flash attn K,V smem fp16 tiles
giveen
committed
7fd6a7c
test: make the FA V-is-view-of-K case a test case parameter
giveen
committed
f3b5039
CUDA + ggml: add sparse-fa for DSV4/GLM
giveen
committed
3ddea31
qwen4exp: add NextN/MTP draft head (--spec-type draft-mtp)
giveen
committed
209a77e

Commits on Sep 3, 2026

fix: cast F16 norm weights to F32 before CUDA broadcast-mul

giveen
and
claude
committed
7533986
fix: don't reuse the Gemma4-assistant same-position draft path for real NextN heads

giveen
and
claude
committed
3888e18
fix: qwen4exp MTP head should reuse the trunk's shared hc_head mixer

giveen
and
claude
committed
97686af
fix: export qwen4exp MTP's t_h_nextn before the inp_out_ids gather

giveen
and
claude
committed
b7f556e
fix: qwen4exp trunk double-gathered output rows, crashing MTP targets

giveen
and
claude
committed
e5a17b5
CUDA: key captured graphs by shape, not first-node address

giveen
and
claude
committed
9f8cb2d
cuda: fix turbo K/V swizzle write/read mismatch in flash-attn tile loaders

giveen
and
claude
committed
323c636
tests: cover turbo2/3/4 K/V at hsk=256 with nb<=4 and GQA in FLASH_ATTN_EXT

giveen
and
claude
committed
4ce86ae
address PR #340 review follow-ups: assert, buffer-check, comments

giveen
and
claude
committed
283e56c
fix follow-up items to match what review actually asked for

giveen
and
claude
committed
8419251
memory: skip V-cache allocation for the qwen4exp indexer KV cache

giveen
and
claude
committed
69499d4