[Kernel][SM70] Fuse DeepSeek V4 FP16 auxiliary GEMVs

[yangzhuxinyzx]

Purpose

• Fuse the DeepSeek V4 C4 N=2048, N=512, and N=64 exact FP16 auxiliary GEMVs into one N=2624 launch on one auxiliary stream.
• Preserve the existing block-K=1024 FP32 FMA/reduction order and output dtypes.
• Keep admission default-off and restricted to SM70 M=1 exact shapes; the new flag also requires VLLM_SM70_DSV4_FP16_GEMV=1.
• Prepare one non-persistent joined buffer after checkpoint loading (20.5 MiB per C4 layer).

This is stacked on the direct-order offset repair in draft PR #308.

Test Plan

• CPU contract/default-off tests.
• Driver-free Triton SM70 compile and resource audit.
• Real V100 CUDA Graph bitwise test of the production kernel.
• Four-stream C4 overlap A/B screen with changing inputs.
• Pending before acceptance: matched PP2 x TP4 full-model endpoint, Nsight trace, GSM8K-64 and broader aggregate quality gates.

Test Result

• CPU: 2 passed.
• V100 production CUDA Graph gate: 1 passed; three outputs are bitwise equal to the existing exact kernels.
• Overlap screen: 64/64 changing inputs bitwise for all auxiliary outputs and concurrent WQA/WKV output. Warm C4 overlap moves 103.809 us to 51.639 us; cache-scrubbed moves 99.104 us to 73.856 us.
• AOT: SM70 PTX/cubin generated; 32 registers, zero stack/local/static shared memory.
• Ruff and git diff --check: passed.

Evidence:

• /data/models/v100-dsv4-0731-pp2tp4-fp13-fused-overlap-screen-20260826-r2/
• /data/models/v100-dsv4-0731-pp2tp4-fused-fp16-aux-gpu-test-20260826-r1/

The route remains opt-in pending endpoint and dataset evidence.

[yangzhuxinyzx]

Public development has ended and this PR head was preserved in the private migration archive before closure. The public repository is frozen at v1.3.0 and will be archived read-only. Thank you for the contribution.

[yangzhuxinyzx]

Public development has resumed through #342, and this PR was unarchived, unlocked, and restored as a Draft at its preserved stacked head. Later audit found this exact joined FP16 auxiliary route slower than the current packed-FP13 baseline. Active follow-up work with newer evidence is public PR #348; do not merge this archived head ahead of #348.