[Kernel] Share SM70 projection, attention and row reduction paths

[yangzhuxinyzx]

Purpose

H3 repeats projection preparation, LoRA addition and row communication across its dense workflows. This change shares FP16 projection/epilogue and D128 Attention interfaces across original floating and W8A16 weights, adds explicit FlashAttention-V100 query tiling, and exposes calibrated TP4 peer row reduction through native H3 APIs. LoRA keeps unrotated input and adds row-parallel increments before reduction; FP32 accumulation, scaling and rounding protections remain.

Current delivery concentrates on already working dense H3 workflows using retained FLASH_ATTN_V100, which is already the default H3 backend. Slower experimental replacements are excluded. Further Attention prototypes, FI optimization and new workflow expansion are paused at the user's request. The validated FI implementation remains selectable. Experimental artifacts are not installed or promoted.

The peer reduction plan owns IPC buffers, calibrates to native FP32 addition order, respects explicit memory budgets, reuses matching shapes and closes collectively. TP1/TP2 or unsupported/over-budget inputs retain ordinary reduction. Native metrics include actual peer/fallback calls and raw IPC memory. Explicit query128, prepared floating-weight columns, residual sharding and peer reduction remain opt-in; no AUTO configuration is qualified.

This is the existing shared-kernel PR, stacked on #578 (accounting) and #571 (general execution/residency). VSA and caches are separate child PRs #583/#584; this does not duplicate those changes.

Test Plan

Check shared projections/LoRA, independent FP32 Attention operators, non-H3 shapes, tails, aliases and invalid inputs. Validate peer reduction against native collectives and complete native video/audio sampling. Measure useful work over the slowest complete-denoise rank with a full request warmup and three unprofiled requests. Keep official-reference and human quality acceptance separate.

Test Result

Python 3.12.13, Torch 2.10.0+cu128, CUDA 12.8.93, V100 SXM2 32GB. All commit hooks pass. Latest scope/status changes only update documentation; existing benchmark records are not relabeled as new runs.

• Shared projection integration: 43 GPU checks plus the strengthened alias check. FA geometry: 69 GPU checks and 53 bitwise/invalid-query checks. Shared Attention: 12 GPU and 18 CPU checks.
• Retained FI build: 33 GPU numerical/storage/query-group/graph checks. Shared CPU interface/provenance suite: 46 passed, 7 GPU skips. Prototype memcheck/racecheck/synccheck report zero errors over 12 boundaries.
• Peer row plan: seven TP4 shapes and 112 numerical cases, collective/stream/ownership guards, UUID checks and isolated four-rank memcheck. Native API controls preserve complete outputs for primary W8A16 Light4, original floating Light4/no-adapter and W8A16 mixed Ref4.
• All eight official LightX2V adapter artifacts have complete TP4 native controls: six FL2V four/eight-step artifacts and two mixed image/video/audio Ref2VA artifacts. W8A16 Light4 v1.2 additionally covers first/last/both keyframes. Original floating FlashGen and FastH3 Dense also preserve final video/audio latents, every pre-encoding RGB frame and PCM bitwise. This does not establish the full task/weight/shape cross product or independent official-model acceptance.

Matched formal four-step result

TP4, single request, 1280x736 canvas, 120 requested frames aligned to 124, W8A16 Light4 v1.2, native peer reduction and the same pageable host/shared VAE policy. Each backend has one full request warmup and three unprofiled requests.

Metric	FA	FI
Median complete denoise	58.293218 s	60.969633 s
Minimum-card median useful TFLOP/s	53.235745	50.898828
Denoise CV	0.076959%	0.069457%
Median complete request	91.071940 s	89.525699 s

FA denoise is 4.389750% faster. The full request includes other stages and is slower in these FA measurements; no overall request speedup is claimed. Live allocation upper bound including IPC is 20,475,227,136 bytes/card. Both backends fail the >80 gate. Older pinned-host controls are retained separately and are not used for an isolated overall-request comparison.

Original floating H3 without LoRA also passes complete default sampling (50 sigma points, 49 updates): denoise 725.819030 to 649.973214 seconds, request 835.355602 to 700.121402 seconds, all final latent/RGB/PCM values bitwise. Candidate useful throughput is 57.353041 TFLOP/s/card. These are captured cold controls, not formal repeated performance acceptance, and the combined host/projection/residual changes are not an isolated Attention comparison.

TP4 1344x768/243-frame and 15-second/362-frame runs establish generation/memory compatibility only. VSA full numerical quality fails; cache primary-shape official quality/performance, independent official references, human audiovisual review and the full matrix remain incomplete. Keep Draft; no main/default/AUTO promotion or relaxed quality threshold.

Evidence: docs/design/minimax_h3/CURRENT_STATUS.md, CAMPAIGN_RESULTS.md, FA_DEVELOPMENT.md, SM70_EPILOGUES.md, SHARED_ATTENTION.md, EXACT_ROW_REDUCTION.md; exact commands, binary/source hashes and results under /data/minimax-h3/sm70-general-20260909/.

AI assistance: implemented and initially validated with OpenAI Codex. Human code review and independent validation remain required before promotion.