Skip to content

qwen4exp: avoid unused QSA pool input for dense MTP - #12

Merged
novkien merged 1 commit into
masterfrom
codex/flash9-dense-mtp
Oct 3, 2026
Merged

novkien merged 1 commit into
masterfrom
codex/flash9-dense-mtp

Conversation

@novkien

@novkien novkien commented Oct 3, 2026 •

Copy link
Copy Markdown
Owner

Addresses #11

Summary

Dense Qwen4Exp MTP layers have a zero compression ratio and do not consume QSA pool inputs. The graph builder currently creates that input from the model-global indexer setting, leaving an unused input without a backing buffer. Gate pool construction on the selected MTP layer's compression ratio, while retaining the existing cache/index validation for QSA-enabled layers.

The architecture fixture covers ordinary Qwen4Exp, dense MTP, and QSA-enabled MTP graphs. It reproduces the dense MTP failure before the fix and passes both MTP paths after it.

Validation

  • test-llama-archs -a qwen4exp -v 2: 3 tests passed.
  • The patched native/RPC pair loaded the Flash Next route through proxy 8082 with two slots and pipeline parallelism enabled; three uncached 8192-token requests completed. This is startup/performance screening, not a speedup claim.
  • git diff --check passed.

AI disclosure

Codex implemented and reviewed this change under the owner-approved Flash Next plan.

@novkien
novkien merged commit 8d75def into master Oct 3, 2026
11 of 18 checks passed
@novkien

novkien commented Oct 3, 2026

Copy link
Copy Markdown
Owner Author

Deployment and acceptance - dense MTP startup regression repaired

PR #12 is merged as 8d75def06db97c8736989c795be403faf214a9d4. Both immutable native/RPC packages were built from that revision with an empty source patch, preserving their SM70/CUDA12.9 and SM120a/CUDA13.3 toolchains. The complete29-file native and22-file RPC packages were installed through normal proxy unload and locked atomic directory exchange; retained prior packages provide rollback. Installation verified old fork children had exited, and the actual new process executable and every mapped backend library match their package hashes. Both manifests report the identical merged source ID.

The original reproduction settings were restored: c262144, b1024, ub512, two unified-KV slots, q8_0, ts12,8,10,9,10, ngl99, unchanged target/draft, MTP min1/max3 onCUDA1. Normal Load job 55156ffca5504488bdcdd6db9232f1e9 succeeded in42.95seconds. Native properties report b11397-8d75def06 and two slots with n_ctx262144. Logs confirm49/49 target and50/50 draft GPU offload, pipeline copies4, no pipeline allocation fallback, and no buffer assertion.

A subsequent real proxy8082 SSE request processed8192 new tokens (cache_n0) and emitted512 tokens. Its final timings and usage agree; MTP accepted371 of418 drafted tokens. PP405.740 and TG79.304tokens/s are correctness-run observations, not a performance claim. The generated two-lane benchmark fixture exposed an independent preparation bug before sending any request; that tooling follow-up is recorded under #9/PR#13 and is not mislabeled as a model failure or concurrent-behavior proof.

Maintained regression evidence also satisfies the pre/post criterion: the same generated dense-MTP fixture aborts on unmodified7e0 in set_input_k_idxs, while the merged change passes the ordinary Qwen4Exp architecture/roundtrip, dense MTP and QSA MTP cases (3 tests,0 failures). The target upstream QSA optimizations remain intact. All #11 acceptance criteria are met; close this startup prerequisite. Broader performance, context>=200k, rollback/checkpoint and concurrent behavior acceptance remain in open #9. The owned GPU-task idle lease was released and read back inactive after the checks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants