Repository navigation
qwen4exp: avoid unused QSA pool input for dense MTP - #12
Conversation
Addresses #11 Assisted-by: Codex
Deployment and acceptance - dense MTP startup regression repairedPR #12 is merged as The original reproduction settings were restored: c262144, b1024, ub512, two unified-KV slots, q8_0, ts12,8,10,9,10, ngl99, unchanged target/draft, MTP min1/max3 onCUDA1. Normal Load job A subsequent real proxy8082 SSE request processed8192 new tokens (cache_n0) and emitted512 tokens. Its final timings and usage agree; MTP accepted371 of418 drafted tokens. PP405.740 and TG79.304tokens/s are correctness-run observations, not a performance claim. The generated two-lane benchmark fixture exposed an independent preparation bug before sending any request; that tooling follow-up is recorded under #9/PR#13 and is not mislabeled as a model failure or concurrent-behavior proof. Maintained regression evidence also satisfies the pre/post criterion: the same generated dense-MTP fixture aborts on unmodified7e0 in set_input_k_idxs, while the merged change passes the ordinary Qwen4Exp architecture/roundtrip, dense MTP and QSA MTP cases (3 tests,0 failures). The target upstream QSA optimizations remain intact. All #11 acceptance criteria are met; close this startup prerequisite. Broader performance, context>=200k, rollback/checkpoint and concurrent behavior acceptance remain in open #9. The owned GPU-task idle lease was released and read back inactive after the checks. |
Addresses #11
Summary
Dense Qwen4Exp MTP layers have a zero compression ratio and do not consume QSA pool inputs. The graph builder currently creates that input from the model-global indexer setting, leaving an unused input without a backing buffer. Gate pool construction on the selected MTP layer's compression ratio, while retaining the existing cache/index validation for QSA-enabled layers.
The architecture fixture covers ordinary Qwen4Exp, dense MTP, and QSA-enabled MTP graphs. It reproduces the dense MTP failure before the fix and passes both MTP paths after it.
Validation
test-llama-archs -a qwen4exp -v 2: 3 tests passed.git diff --checkpassed.AI disclosure
Codex implemented and reviewed this change under the owner-approved Flash Next plan.