Skip to content

[Model][SM70] Exclude unsafe PP2 TP4 QPN8 gate/up - #313

Merged
yangzhuxinyzx merged 3 commits into
mainfrom
codex/v100-dsv4-pp2tp4-qpn8-accuracy-main-20260826-024109
Aug 26, 2026
Merged

yangzhuxinyzx merged 3 commits into
mainfrom
codex/v100-dsv4-pp2tp4-qpn8-accuracy-main-20260826-024109

Conversation

@yangzhuxinyzx

Copy link
Copy Markdown
Contributor

Purpose

Rebuild and repair PR #302 on the latest main at 1d64f84.

The original PR changed every admitted PP2 x TP4 QPN8 projection to its lowest-error schedule. Source audit shows that this pays a 15--36% isolated operator slowdown for errors that are already bounded around 1e-5. The actual numerical outlier is only fused shared-expert gate/up.

This repair therefore:

  • removes shared-expert gate/up from the default QPN8 operator contract so it falls back to TurboMind;
  • retains the existing measured speed-winner schedules for WQA/WKV, WQ-B, WO-B, grouped WO-A, and shared down;
  • keeps the replicated indexer excluded;
  • preserves the existing PP2 x TP4 B1 no-spec hardware/topology/tensor admission and rollback controls;
  • never reads model name, checkpoint path, model_type, or architecture identity.

This PR supersedes #302 rather than pushing into the agent-owned branch.

Numerical and performance audit

Four-pattern real-weight comparison against same-process TurboMind:

  • retained fast schedules: exact FP16 fraction at least 99.707%, relative L2 at most 1.641e-5;
  • fused shared gate/up: exact fraction 53.516%, relative L2 5.989e-4;
  • accuracy-first alternatives for the retained roles are about 15--36% slower in the isolated operator matrix.

Matrix artifact:
/data/models/v100-dsv4-0731-pp2tp4-qpn8-accuracy-matrix-20260826-r1/

The conservative all-accuracy source gate passes M=1/M=9, changing-input CUDA Graph replay, FP32/TurboMind references, grouped WO-A layout, both excluded roles, and one reused 16 MiB workspace:
/data/models/v100-dsv4-0731-pp2tp4-qpn8-accuracy-source-gate-20260826-r2-main/

Matched PP2 x TP4 B1 no-spec evidence for that slower all-accuracy variant:

  • QPN8 off median: 63.6434 tok/s, 15.71254 ms/token;
  • all-accuracy QPN8 median: 63.9120 tok/s, 15.64652 ms/token;
  • difference: about +0.42%, -0.066 ms/token;
  • pinned GSM8K64: 62/64 versus 61/64, zero invalid outputs, one directional difference at item 37.

The single GSM8K difference is recorded rather than treated as a greedy-identity failure. It also provides no evidence that selecting the lowest already-small operator error improves aggregate task quality. The accepted source repair is consequently the speed-preserving hybrid: retain every bounded speed winner and fall back only the demonstrated outlier.

Full artifacts:
/data/models/v100-dsv4-0731-pp2tp4-qpn8-accuracy-fullmodel-control-20260826-r1/
/data/models/v100-dsv4-0731-pp2tp4-qpn8-accuracy-fullmodel-candidate-20260826-r2/

Test plan and result

  • 9 focused PP2 x TP4 QPN8 route, rollback, workspace, and grouped-dispatch tests: passed under Python 3.12, Torch 2.10, and the matching source-built extension.
  • pre-commit run -a on latest main: all hooks passed.
  • git diff --check: passed.
  • no new end-to-end run: the user-requested audit is source/operator focused, and the matched artifacts above already close the conservative variant.

Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
(cherry picked from commit a253e04)
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
(cherry picked from commit bdc82e8)
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
@yangzhuxinyzx
yangzhuxinyzx marked this pull request as ready for review August 26, 2026 02:56
@yangzhuxinyzx
yangzhuxinyzx merged commit f5fc157 into main Aug 26, 2026
4 checks passed
@yangzhuxinyzx
yangzhuxinyzx deleted the codex/v100-dsv4-pp2tp4-qpn8-accuracy-main-20260826-024109 branch August 26, 2026 02:56
yangzhuxinyzx added a commit that referenced this pull request Aug 27, 2026
The audited QPN8 functionality is already present through merged PR #313. Preserve the historical commits without replaying the stale policy or tests.

Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant