Skip to content

[Bugfix][SM70] Stabilize PP2 TP4 FP8 tactics - #349

Draft
yangzhuxinyzx wants to merge 5 commits into
mainfrom
agent/private-v100-dsv4-deterministic-fp8-tactics-20260827
Draft

[Bugfix][SM70] Stabilize PP2 TP4 FP8 tactics#349
yangzhuxinyzx wants to merge 5 commits into
mainfrom
agent/private-v100-dsv4-deterministic-fp8-tactics-20260827

Conversation

@yangzhuxinyzx

Copy link
Copy Markdown
Contributor

Public migration note (2026-08-27): this Draft is the public continuation of private PR #21, mirrored at exact head c015a56 after public main resumed through #342. The original implementation notes and evidence are preserved below.

Purpose

  • Make DeepSeek V4 PP2 x TP4 FP8 launch arithmetic reproducible across
    pipeline stages and process restarts.
  • Keep dynamic scheduling search while preserving the default kernel and
    split-K accumulation tree by default.
  • Treat any output-quality regression as a hard admission failure.
  • This is private V100-specific follow-up work and does not duplicate another
    open private PR.

Test Plan

  • Compare two matched unsafe-dynamic restarts and retain exact M=1 launch
    evidence only when the trace filter reaches decode descriptors.
  • Compare fixed FP8, safe FP8, diagnostic MXFP4-QPN, and an identical candidate
    restart.
  • Require exact output hashes and row-level GSM8K/HumanEval/LongBench equality
    before admission.
  • Build the SM70 extension, audit every embedded cubin, and run focused
    warmup/cache and fresh-import tests.

Test Result

  • tests/test_envs.py: 56 passed.
  • tests/quantization/test_sm70_warmup.py: 17 passed.
  • The expanded FP8/MXFP4/default-policy set: 107 passed, 4 CUDA-skipped.
  • The scoped MXFP4 B1 safe-selector set: 84 passed, 4 CUDA-skipped; its changed
    CUDA translation unit compiles for SM70.
  • The full _C target built successfully (50/50). Extension SHA256 is
    139aaf8e2d28d497c53bbd6ea4473523ebaf0802a978294135e8e1f15745debb;
    all 38 embedded cubins are sm_70 and none targets another architecture.
  • A fresh interpreter binds this branch to that task-owned extension and sees
    the intended absent-environment defaults: FP8 safe selector on, WQA
    prescaled-M1 off, MXFP4 B1 safe selector on, and MXFP4-QPN off.
  • Changed-file pre-commit: passed.
  • The implementation currently defaults WQA prescale and MXFP4-QPN off. The
    latter remains rejected after 93/136 non-bitwise operator comparisons plus
    a model-level correct-to-wrong transition. The completed deterministic arm
    now also rejects the WQA-default-off change; it must be reverted or its
    quality root repaired before this Draft can merge.
  • The exact DSV4 MXFP4 B1 TurboMind descriptors now preserve their heuristic
    kernel/split-K tree while retaining measured swizzle. Its planned
    64-pattern, fresh-process operator/restart gate was cancelled before launch
    when testing was stopped; it is not admitted.
  • Two matched current-source unsafe-dynamic restarts measured 73.662 and
    73.592 token/s; within each restart all three performance outputs match,
    but the hashes differ across restarts, as do the chat outputs. The first
    diagnostic filter spent its trace quota on prefill, so it is explicitly not
    claimed as M=1 trace evidence. An older exact M=1 trace of the same selector
    mechanism records multiple split-K trees on three of five dense descriptors
    (5/6, 2/4, and 1/2).
  • Testing was stopped on request after the fixed-arithmetic arm completed.
    That arm records GSM8K 62/64 with zero invalid answers, HumanEval 29/32,
    LongBench 40.320, and median 64.132 token/s. Pinned-correct GSM item 37
    regresses from 2 to 0 with no compensating fix, so the hard quality gate
    fails. The safe arm was interrupted before datasets/performance; diagnostic,
    restart, and one-GPU follow-ups did not run. All owned GPU processes and
    queues were cleaned. This PR remains Draft and non-admitted.

AI assistance was used. Every changed line and the final evidence must be
reviewed by the human submitter.

yangzhuxinyzx and others added 5 commits August 27, 2026 01:45
Co-authored-by: OpenAI Codex <codex@openai.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Keep dynamic scheduling search while retaining the default FP8 kernel and split-K tree unless the unsafe diagnostic lane is explicitly requested.

Co-authored-by: OpenAI Codex <codex@openai.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Demote the non-reproducible endpoint, capture the latest PP2 x TP4 trace and strict three-arm results, and define the repeated quality gate for safe accumulation.

Co-authored-by: OpenAI Codex <codex@openai.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Disable the slower WQA prescale and non-bitwise MXFP4-QPN routes by default while preserving explicit fail-closed diagnostic opt-ins.

Co-authored-by: OpenAI Codex <codex@openai.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Keep the exact DeepSeek V4 six-route TurboMind kernel and split-K tree while allowing scheduling-only swizzle tuning.

Co-authored-by: OpenAI Codex <codex@openai.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
@yangzhuxinyzx

Copy link
Copy Markdown
Contributor Author

Latest-main audit result: keep Open Draft. The safe-selector direction is reasonable, but the current combined head is explicitly non-admitted: the fixed-arithmetic arm has a pinned correct-to-wrong GSM8K transition with no compensating fix, the safe arm was interrupted, and the new MXFP4 selector lacks its fresh-process operator/restart gate. Repair should isolate the deterministic-selector change and close quality/performance on that exact delta.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant