[Kernel][SM70] Default audited PP2 TP4 decode hot paths

[yangzhuxinyzx]

Purpose

Rebuild and repair PR #308 on the latest main at f5fc157.

This PR carries four independently audited SM70 PP2 x TP4 decode optimizations:

• use one 16-head sparse-MLA QK d-split group for the exact TP4 shape;
• fuse the existing Q normalization/RoPE programs and eight packed FP8-KV writers into one graph node;
• broadcast the physical B1 MXFP4 W13 input instead of copying it six times;
• retain top-k-6 route order and use the exact half2 reducer, removing sort and inverse-permutation work.

The validated compact-grouped, direct-top6, direct-order, broadcast-input, and fused-QNorm paths are default-on, each with an explicit environment rollback. Runtime admission remains exact hardware, dtype, tensor shape, token-count, topology, and expert-layout checks. No new model-name, checkpoint, model_type, or architecture-identity gate is added.

This PR supersedes #308 rather than modifying the agent-owned stack.

Numerical and performance evidence

Sparse MLA head grouping:

• C4 median: 32.317 to 18.412 us, about 43% lower;
• C128 median: 15.278 to 10.609 us, about 31% lower;
• SWA median: 15.135 to 10.424 us, about 31% lower;
• C128/SWA bitwise equal;
• changing-input gate finite with max absolute 1.526e-5 and max relative L2 1.807e-5.

Artifact:
/data/models/v100-dsv4-0731-pp2tp4-sparse-head16-screen-20260826-r1/

QNorm/RoPE plus KV insertion:

• 64/64 changing-input CUDA Graph patterns bitwise equal for transformed Q and complete cache storage;
• median: 9.974 to 4.076 us, 2.45x;
• projected 43-layer saving: 0.254 ms/token.

Artifact:
/data/models/v100-dsv4-0731-pp2tp4-qnorm-kv-fusion-screen-20260826-r1/

MXFP4 direct pipeline:

• generic: 108.397 us/layer;
• current direct-top6: 54.414 us/layer;
• direct-order: 46.519 us/layer;
• direct-order saving versus direct-top6 projects to 0.339 ms/token across 43 layers;
• W13, clamped SwiGLU, W2, and final output are bitwise equal initially and after route IDs change under CUDA Graph;
• top-k-6, H=4096 weighted reduction matches the existing unpermute oracle exactly.

Artifact:
/data/models/v100-dsv4-0731-pp2tp4-mxfp4-direct-order-screen-20260826-r1/results/full_pipeline.json

Audit repairs

• default the proven compact-grouped, direct-top6, and direct-order paths while retaining =0 rollbacks;
• use q.shape[0] as the fused kernel token bound instead of deriving it from slot-mapping storage;
• replace a direct torch.cuda synchronize call in the test with torch.accelerator;
• reformat the transplanted CUDA hunk against the current main and verify token equivalence.

Test plan and result

• 3 latest-main CPU route/default tests: passed under Python 3.12 and Torch 2.10 with the matching source-built extension.
• CUDA 12.8 SM70 source build from the original stack: passed; SM70 cubins only.
• focused V100 operator/graph gates listed above: passed.
• pre-commit run -a on latest main: all hooks passed.
• C++ pre/post-format normalized token stream: identical.
• git diff --check: passed.

All eight V100s are currently occupied by existing user services/tasks, so no redundant GPU rerun or end-to-end serving run was started. The existing operator artifacts are the acceptance evidence.