[Kernel][SM70] Revisit exact fused V4 auxiliary GEMVs

[yangzhuxinyzx]

Public migration note (2026-08-27): this Draft is the public continuation of private PR #20, mirrored at exact head efd6f1f after public main resumed through #342. The original implementation notes and evidence are preserved below.

This is the active follow-up to archived public #327; #327 will remain a historical Draft and must not be merged ahead of this newer evidence.

Purpose

• Fuse the three exact FP16 C4 auxiliary projections into one launch while
  preserving the existing FP32 FMA and reduction order.
• Keep the route default-off and mutually exclusive with FP13 until the strict
  endpoint quality gate passes.
• This private exact-FP16 route is distinct from the older approximate FP13
  experiment and does not duplicate another open private PR.

Test Plan

• Exercise real weights and 64 changing inputs under CUDA Graph replay.
• Compare all main and auxiliary outputs bitwise.
• Run the stabilized repeated GSM8K, HumanEval and LongBench gate before
  endpoint admission.

Test Result

• Focused host tests: 7 passed, 11 CUDA-skipped; related route tests: 24 passed.
• Changed-file pre-commit: passed.
• Current-source V100 operator screen: main 64/64 bitwise, auxiliaries 64/64
  bitwise, graph stable.
• Warm overlap: 105.894 -> 51.884 us/C4 layer; cold overlap:
  99.080 -> 74.040 us/C4 layer.
• Full-model admission is intentionally pending while the FP8 tactic quality
  baseline is stabilized; this PR remains Draft.

AI assistance was used. Every changed line and final benchmark must be reviewed
by the human submitter.

[yangzhuxinyzx]

Latest-main audit result: keep Open Draft. The fused FP16 operator is bitwise and materially faster than the unfused FP16 control, but it is mutually exclusive with the retained default-on FP13 route, which the existing worklog measures slightly faster, and the full endpoint gate is still pending. It currently improves no default path; revisit only as an exact rollback route after matched endpoint evidence.

[yangzhuxinyzx]

Re-audited against current main (7fe2339763) for the 1.5.0 release: keep this PR Open Draft.

The source conflicts are repairable (vllm/envs.py plus the migration log), and the exact fused-FP16 implementation remains bitwise versus the old three-launch FP16 control. However, it still does not improve the current default path:

• joined exact FP16: 51.639 us warm / 73.856 us cold per C4 layer;
• current default-on three-stream FP13: 50.879 us warm / 70.072 us cold;
• joined exact FP16 also retains about 20.5 MiB of extra fused weight per C4 layer.

Therefore resolving the conflicts and enabling this for 1.5.0 would add code and memory while slightly regressing the existing default performance. It remains potentially useful as an exact rollback experiment, but needs a new implementation or matched endpoint evidence that beats the current FP13 route before merge.