Skip to content

[Kernel][WIP] Screen SM70 FlashNext EP4 and DeepEP - #547

Draft
yangzhuxinyzx wants to merge 1 commit into
codex/v100-flashinfer-batch-integration-20260906-020903from
codex/v100-flashnext-sm70-ep4-20260907-030458
Draft

yangzhuxinyzx wants to merge 1 commit into
codex/v100-flashinfer-batch-integration-20260906-020903from
codex/v100-flashnext-sm70-ep4-20260907-030458

Conversation

@yangzhuxinyzx

Copy link
Copy Markdown
Contributor

Purpose

  • Prototype true TP2 x DP2 / EP4 for FlashNext NVFP4 on four V100 GPUs.
  • Add an SM70-compatible DeepEP intranode experiment based on official commit b8d90fb.
  • Preserve CUDA Graph execution and fail closed outside the audited shape/topology.
  • Record positive microbenchmark evidence and negative end-to-end evidence before any default admission.

This PR is intentionally Draft and must not be merged in its current state: true EP is slower than the existing TP4 production candidate and the surrounding sparse-QSA/HC quality signal is not yet closed.

Test Plan

  • Build the native SM70 extension and run the dedicated TP4 versus TP-local EP4 kernel benchmark.
  • Run DeepEP FP16 identity dispatch/combine in eager and CUDA Graph modes on four V100s.
  • Run fixed FlashNext NVFP4 TP2 x DP2 / EP4 at C16, 8K input, 256 output, no MTP, CUDA Graph, 256K capacity.
  • Run focused route/lifetime tests and the complete SM70 ModelOpt NVFP4 unit file.
  • Run pre-commit, including Ruff, mypy, clang-format, markdownlint and DCO sign-off.

Test Result

  • DeepEP graph dispatch + combine microbenchmark: 0.080896 ms/layer; FP16 identity max abs error 0.00390625.
  • FULL CUDA Graph captures on all four ranks and the dedicated DeepEP route is selected.
  • DeepEP true EP4: 408.609 tok/s, 39.1572 ms/step.
  • Matching AG/RS true EP4 control: 422.211 tok/s, 37.8958 ms/step.
  • Net result: -3.22% throughput and +1.2615 ms/step; rejected for default routing.
  • KV capacity: 318,423 tokens, sufficient for the 262,144-token boundary.
  • Focused tests: 8 passed. Complete SM70 ModelOpt NVFP4 file: 80 passed.
  • Pre-commit: passed.
  • Quality is not accepted: only 4/16 full completion hashes match AG/RS, and the parent experimental sparse-QSA/HC path still has an unresolved quality signal.

Detailed evidence and the stop/re-open criteria are in docs/design/sm70_v100_migration_control.md.

Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant