Skip to content

[Bench][SM70] Repair #375 without disabling QPN8 - #376

Merged
yangzhuxinyzx merged 14 commits into
codex/v100-qwen38-nvfp4-decode-quality-audit-20260828-025709from
agent/v100-repair-pr375-default-on-latest-main-20260828-1205
Aug 28, 2026
Merged

yangzhuxinyzx merged 14 commits into
codex/v100-qwen38-nvfp4-decode-quality-audit-20260828-025709from
agent/v100-repair-pr375-default-on-latest-main-20260828-1205

Conversation

@yangzhuxinyzx

@yangzhuxinyzx yangzhuxinyzx commented Aug 28, 2026 •

Copy link
Copy Markdown
Contributor

Purpose

Repair #375 on the latest audited main without disabling the proven Qwen3.8 online-QPN8 performance route before matched model-level quality evidence exists.

Result

Validation

  • 27 focused benchmark/QPN8 tests passed; one CUDA extension test was deselected because this source checkout has no compiled vllm._C QPN8 operator.
  • Changed-file pre-commit passed, including Ruff, formatting, mypy, SPDX, forbidden-import, and configuration gates.
  • All commits are DCO signed.

Leonccaa and others added 13 commits August 26, 2026 11:14
Integrate the Qwen3.8 Flash Next model and PLE CPU offload work from vLLM PRs #53896 and #53899, then carry the validated 1Cat SM70 compatibility layer for FP16 activations, hybrid caches, NVFP4 emulation, and top-token sampling.

The included TP4 smoke covers checkpoint loading, PLE offload, prompt prefill, and one autoregressive decode step on four V100 32GB GPUs.

Co-authored-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>

Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>

Assisted-by: OpenAI Codex
Signed-off-by: Leonccaa <166551845+Leonccaa@users.noreply.github.com>
Match the 1Cat mRoPE, FP8 skip, short-conv metadata, and host-to-device helper signatures. Narrow optional FP8 test scales so the upstream pre-commit mypy matrix can validate the imported code.

Assisted-by: OpenAI Codex

Signed-off-by: Leonccaa <166551845+Leonccaa@users.noreply.github.com>
Widen the 1Cat hybrid state-shape protocol to the current vLLM shape union and document intentional Qwen4Exp overrides for the older 1Cat Qwen3Next and pipeline-parallel interfaces.

Assisted-by: OpenAI Codex

Signed-off-by: Leonccaa <166551845+Leonccaa@users.noreply.github.com>
Apply the pinned Ruff 0.14.0 formatter to the two PR-specific files that differ from the current 1Cat main formatting baseline.

Assisted-by: OpenAI Codex

Signed-off-by: Leonccaa <166551845+Leonccaa@users.noreply.github.com>
Use four warps for the single-split sparse QSA prefill profile on exact SM70 devices while preserving the existing launch choices everywhere else. Add a pure policy test for the original profile boundaries and the V100-specific branch.

The corresponding four-V100 TP4 gate improved 1K and 4K prefill by 5.47x and 6.16x with matching deterministic first tokens.

Assisted-by: OpenAI Codex

Signed-off-by: Leonccaa <166551845+Leonccaa@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
…atest-main-20260828-0327

[Core][SM70] Repair Qwen4Exp PLE integration from #338
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
@yangzhuxinyzx
yangzhuxinyzx marked this pull request as ready for review August 28, 2026 04:16
@yangzhuxinyzx

Copy link
Copy Markdown
Contributor Author

审计完成:锁定 head bb22d57;27 个聚焦测试通过(另 1 个仅因源码 checkout 无已编译扩展而 deselect),完整远端 pre-commit 通过。该修复保留 main 已验证的 QPN8 默认开启策略,只纳入安全的 GSM8K 参数覆盖和答案解析增强;源码树已核对为最新 main 加这两项 benchmark 改动。现在合并回 #375 分支。

@yangzhuxinyzx
yangzhuxinyzx merged commit 456adea into codex/v100-qwen38-nvfp4-decode-quality-audit-20260828-025709 Aug 28, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants