frost(sdpa): remove the legacy (pre-envelope) THD leg from the FP8/MXFP8 SM100/SM107 kernels - #622
Merged
vedaanta merged 2 commits intoAug 17, 2026
Conversation
…FP8 SM100/SM107 kernels The SM100 per-tensor FP8 and MXFP8 kernels (and the SM107 FP8 sibling) carried a THD/varlen leg from before the issue-NVIDIA#552 device-built-metadata + plan-time-envelope design existed. It was never wired, three layers deep: the engine specs declare thd=False for these cells, the adapter parked fp8 THD behind a 'thd-deferred' compile sentinel nothing unwired, and the execute paths raise NotImplementedError for THD. When fp8/mxfp8 THD lands on these arches it will follow the write_thd_meta envelope design the f16 kernels use (PRs NVIDIA#606/NVIDIA#608), not this leg. Removed: - All CFG.THD_VARLEN-gated branches in prefill_d128_fp8_sm100.py, prefill_d128_fp8_sm107.py (hunk-symmetric mirror), and prefill_d128_mxfp8_sm100.py: launch-side setup + grid, the per-batch O-descriptor store branch, the packed-LSE branch, the fake-tensor/compile ternaries (and the mxfp8 SF-tile THD kwargs no caller passes). - build_o_descs_kernel from thd_sm100.py (these three kernels were its only importers; build_thd_meta_o_descs_kernel and TENSOR_MAP_QWORDS stay — the f16 kernels use them). - The 'thd-deferred' sentinel: SM100 check_support now declines fp8/mxfp8 + THD explicitly (the spec already gates the graph path; the gate covers direct construction), and the THD compile branch is f16-only. Kept, deliberately: - _host ABI slots (o_desc_words, n_thd_units, mxfp8 SF-tile counts) — the adapter passes them positionally and the NVIDIA#606 design still uses the names; annotated as unused. - thd_tma_offsets/_thd_sf_tile_bases call sites — shared-module helpers that fold to dense identity at THD_VARLEN=0 (dense codegen byte-identical). - A trace-time guard: CFG.THD_VARLEN=1 now raises at compile instead of silently mis-executing. Validated on B200 (9.26 nightly): fp8+mxfp8 fwd sweeps 316 passed (FROST routing served 52 graphs through the edited kernels); frost fp8 file 32 passed (L0+L1); f16 dsl + THD + async/capture suites 489 passed (L0+L1) — the thd_sm100.py survivors' import path exercised end to end. sm107 file parses and is hunk-symmetric with sm100.
vedaanta
added
orig-nv-eng
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (5)
💤 Files with no reviewable changes (1)
Included review availability: Your plan includes up to 12 reviews per rolling hour; 9 remain after this review. 📝 WalkthroughWalkthroughSM100 FP8 and MXFP8 THD support now fails during support checks or tracing. The prefill kernels use dense batch-indexed execution and compilation shapes. The legacy standalone O-descriptor construction kernel was removed. ChangesFP8 THD removal
Estimated code review effort: 4 (Complex) | ~45 minutes Merge Risk: ⚪ Minimal · up to This PR removes unreachable legacy THD paths from FP8 and MXFP8 kernels while preserving supported paths and adding an explicit unsupported-case guard; no actionable merge-blocking risk remains after normal checks and review. Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
3 tasks
The legacy-THD-leg removal (previous commit) left three dead _host ABI slots annotated as "Unused ABI slot" to avoid touching adapter call sites. Delete them for real, updating the adapter in lockstep: - prefill_d128_fp8_sm100.py / prefill_d128_fp8_sm107.py (hunk-symmetric): drop o_desc_words (legacy THD per-batch O-descriptor array) and n_thd_units (legacy THD flat-grid unit count) from _host, and the matching fake_o_desc / cutlass.Int32(0) fakes from compile(). - prefill_d128_mxfp8_sm100.py: same two, plus the mxfp8-only dead slots total_q_sf_tiles / total_kv_sf_tiles (unread since the THD branches that consumed them were removed) and their Int32 fakes. - api_dsl.py (SdpaFwdDslSm100 only): _execute_fp8 / _execute_mxfp8 stop building o_desc_dummy and stop passing the o_desc_dummy / n_thd_units / total_*_sf_tiles arguments; the call argument lists now match the edited _host signatures position for position. The dense f16 path and the f16 THD path keep their own o-desc machinery (untouched), as do the SM120 class and engines.py. scratch_workspace_bytes() needed no change: the dense FP8/MXFP8 branch already returns 0 and the THD o_desc chunk belongs to the f16 THD path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Collaborator
Author
|
Second commit |
jhjpark
approved these changes
Aug 17, 2026
hxbai
pushed a commit
to hxbai/cudnn-frontend
that referenced
this pull request
Aug 18, 2026
…FP8 SM100/SM107 kernels (NVIDIA#622) * frost(sdpa): remove the legacy (pre-envelope) THD leg from the FP8/MXFP8 SM100/SM107 kernels The SM100 per-tensor FP8 and MXFP8 kernels (and the SM107 FP8 sibling) carried a THD/varlen leg from before the issue-NVIDIA#552 device-built-metadata + plan-time-envelope design existed. It was never wired, three layers deep: the engine specs declare thd=False for these cells, the adapter parked fp8 THD behind a 'thd-deferred' compile sentinel nothing unwired, and the execute paths raise NotImplementedError for THD. When fp8/mxfp8 THD lands on these arches it will follow the write_thd_meta envelope design the f16 kernels use (PRs NVIDIA#606/NVIDIA#608), not this leg. Removed: - All CFG.THD_VARLEN-gated branches in prefill_d128_fp8_sm100.py, prefill_d128_fp8_sm107.py (hunk-symmetric mirror), and prefill_d128_mxfp8_sm100.py: launch-side setup + grid, the per-batch O-descriptor store branch, the packed-LSE branch, the fake-tensor/compile ternaries (and the mxfp8 SF-tile THD kwargs no caller passes). - build_o_descs_kernel from thd_sm100.py (these three kernels were its only importers; build_thd_meta_o_descs_kernel and TENSOR_MAP_QWORDS stay — the f16 kernels use them). - The 'thd-deferred' sentinel: SM100 check_support now declines fp8/mxfp8 + THD explicitly (the spec already gates the graph path; the gate covers direct construction), and the THD compile branch is f16-only. Kept, deliberately: - _host ABI slots (o_desc_words, n_thd_units, mxfp8 SF-tile counts) — the adapter passes them positionally and the NVIDIA#606 design still uses the names; annotated as unused. - thd_tma_offsets/_thd_sf_tile_bases call sites — shared-module helpers that fold to dense identity at THD_VARLEN=0 (dense codegen byte-identical). - A trace-time guard: CFG.THD_VARLEN=1 now raises at compile instead of silently mis-executing. Validated on B200 (9.26 nightly): fp8+mxfp8 fwd sweeps 316 passed (FROST routing served 52 graphs through the edited kernels); frost fp8 file 32 passed (L0+L1); f16 dsl + THD + async/capture suites 489 passed (L0+L1) — the thd_sm100.py survivors' import path exercised end to end. sm107 file parses and is hunk-symmetric with sm100. * frost(sdpa): drop the freed THD ABI slots from the FP8/MXFP8 hosts The legacy-THD-leg removal (previous commit) left three dead _host ABI slots annotated as "Unused ABI slot" to avoid touching adapter call sites. Delete them for real, updating the adapter in lockstep: - prefill_d128_fp8_sm100.py / prefill_d128_fp8_sm107.py (hunk-symmetric): drop o_desc_words (legacy THD per-batch O-descriptor array) and n_thd_units (legacy THD flat-grid unit count) from _host, and the matching fake_o_desc / cutlass.Int32(0) fakes from compile(). - prefill_d128_mxfp8_sm100.py: same two, plus the mxfp8-only dead slots total_q_sf_tiles / total_kv_sf_tiles (unread since the THD branches that consumed them were removed) and their Int32 fakes. - api_dsl.py (SdpaFwdDslSm100 only): _execute_fp8 / _execute_mxfp8 stop building o_desc_dummy and stop passing the o_desc_dummy / n_thd_units / total_*_sf_tiles arguments; the call argument lists now match the edited _host signatures position for position. The dense f16 path and the f16 THD path keep their own o-desc machinery (untouched), as do the SM120 class and engines.py. scratch_workspace_bytes() needed no change: the dense FP8/MXFP8 branch already returns 0 and the THD o_desc chunk belongs to the f16 THD path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
vedaanta
added a commit
to vedaanta/cudnn-frontend
that referenced
this pull request
Aug 18, 2026
Per-tensor FP8 now runs head dims below the d128 tile through the same zero-padding ENVELOPE the f16/bf16 flavors use: compile() takes the actual (d_qk, d_v) so the TMA descriptors carry the real extents (OOB loads zero-fill — exact in FP8 — and O stores clip at d_v). check_support admits equal head dims, d%16==0 (16-byte TMA global-stride rule at BPE=1), d<=128; the descales are scalars so the envelope is arch-independent. MXFP8 stays exact-d128 (SF plumbing not audited for padding). This is the landing zone for the ViT d=72-in-80 contract (e.g. Qwen3-VL vision encoders) without caller-side re-padding to 128. Both fp8 kernel siblings (SM100/SM107) change in lockstep. The THD/varlen leg this PR previously carried is dropped: NVIDIA#622 removed the legacy kernel THD leg it wired; fp8 THD returns as a follow-up on the NVIDIA#606 write_thd_meta device-metadata design. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
vedaanta
added a commit
to vedaanta/cudnn-frontend
that referenced
this pull request
Aug 19, 2026
…via the write_thd_meta envelope design (issue NVIDIA#552) Port the device-built-metadata + plan-time-envelope THD design (PRs NVIDIA#606/NVIDIA#608) into the per-tensor FP8 SM100 kernel, its SM107 (Rubin) sibling (hunk-symmetric), and the block-scale MXFP8 SM100 kernel — the port NVIDIA#622 prescribed when it removed the legacy leg: - Kernels: dynamic packed token extents (cute.sym_int; plan-time-only compile keys), the shared build_thd_meta_o_descs_kernel setup launch (metadata + per -batch O TMA descriptors built device-side, no length ever reaches the host), the plan-time envelope grid with the batch == n_batch dead-unit sentinel (O-store skip; LSE/amax_o predicated on the per-sequence Q length from the device metadata), and ragged Stats in the caller's declared layout (token-major TH1 rank-2 or head-major rank-3, static-rank dispatch). - MXFP8 THD scale factors travel PACKED per-sequence-TILE-padded ([1, H, Σ_b ceil(S_b/128), SF_SMEM] tile sequences in cu_seqlens order, matching the tile base the kernel derives via _thd_sf_tile_bases). The packed tile extent is a runtime value that must come without a device read (Rule 3), so it derives from the SF buffer's byte size — THD SF buffers are exactly the packed layout (its head stride could address nothing else); the SF descriptors use B=1 + dynamic tile extents. - Adapter: factor the SM100 THD packing into _thd_pack (mirrors the SM120 class): metadata/O-desc scratch, capacity token floors, zero-capacity clamps, envelope units — used by the f16 _execute_thd and the new FP8/MXFP8 THD branches. FP8/MXFP8 serve the packed contract only (_thd_check_strides_packed; no stride keys in _thd_compile_kwargs). No Amax_S, no descale_s/scale_s — dropped on these kernels (NVIDIA#602/NVIDIA#619); the amax_o protocol (in-kernel atomicMax, device-side scale_o divide) is unchanged under THD. - Engines: the SM100 FP8/MXFP8 rows declare thd=True + cu_seq_len=True; the arch RANGE (sm 100..119) already routes cc10.7 through the SM107 sibling. - pygraph: sdpa_mxfp8 gains trailing use_padding_mask / seq_len_q / seq_len_kv / cu_seq_len_q / cu_seq_len_kv kwargs (sdpa_fp8 already had them) — the THD length carriers, and dense mxfp8 + KV padding becomes constructible for the first time (tested; stats off — padded_stats is not declared). - Tests: THD self-attention (masks x e4m3/e5m2), cross-attention + GQA, causal+sink, THD+ragged-TH1-stats, and cu_seq_len cases for both fp8 and mxfp8; dense mxfp8 KV-padding; sm107 module-level THD-leg load checks. Verified on B200 (backend 9.23.01): test/python/sdpa/frost 669 passed, 5 failed — all five are cu_seq_len graphs hitting the pre-existing native-lowering version gate (fp8-family cu_seq_len needs the unified node, cuDNN >= 9.24/9.25; develop's own f16 cu tests fail identically on this backend and are green on CI's 9.26). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
vedaanta
added a commit
to vedaanta/cudnn-frontend
that referenced
this pull request
Aug 19, 2026
…via the write_thd_meta envelope design (issue NVIDIA#552) Port the device-built-metadata + plan-time-envelope THD design (PRs NVIDIA#606/NVIDIA#608) into the per-tensor FP8 SM100 kernel, its SM107 (Rubin) sibling (hunk-symmetric), and the block-scale MXFP8 SM100 kernel — the port NVIDIA#622 prescribed when it removed the legacy leg: - Kernels: dynamic packed token extents (cute.sym_int; plan-time-only compile keys), the shared build_thd_meta_o_descs_kernel setup launch (metadata + per -batch O TMA descriptors built device-side, no length ever reaches the host), the plan-time envelope grid with the batch == n_batch dead-unit sentinel (O-store skip; LSE/amax_o predicated on the per-sequence Q length from the device metadata), and ragged Stats in the caller's declared layout (token-major TH1 rank-2 or head-major rank-3, static-rank dispatch). - MXFP8 THD scale factors travel PACKED per-sequence-TILE-padded ([1, H, Σ_b ceil(S_b/128), SF_SMEM] tile sequences in cu_seqlens order, matching the tile base the kernel derives via _thd_sf_tile_bases). The packed tile extent is a runtime value that must come without a device read (Rule 3), so it derives from the SF buffer's byte size — THD SF buffers are exactly the packed layout (its head stride could address nothing else); the SF descriptors use B=1 + dynamic tile extents. - Adapter: factor the SM100 THD packing into _thd_pack (mirrors the SM120 class): metadata/O-desc scratch, capacity token floors, zero-capacity clamps, envelope units — used by the f16 _execute_thd and the new FP8/MXFP8 THD branches. FP8/MXFP8 serve the packed contract only (_thd_check_strides_packed; no stride keys in _thd_compile_kwargs). No Amax_S, no descale_s/scale_s — dropped on these kernels (NVIDIA#602/NVIDIA#619); the amax_o protocol (in-kernel atomicMax, device-side scale_o divide) is unchanged under THD. - Engines: the SM100 FP8/MXFP8 rows declare thd=True + cu_seq_len=True; the arch RANGE (sm 100..119) already routes cc10.7 through the SM107 sibling. - pygraph: sdpa_mxfp8 gains trailing use_padding_mask / seq_len_q / seq_len_kv / cu_seq_len_q / cu_seq_len_kv kwargs (sdpa_fp8 already had them) — the THD length carriers, and dense mxfp8 + KV padding becomes constructible for the first time (tested; stats off — padded_stats is not declared). - Tests: THD self-attention (masks x e4m3/e5m2), cross-attention + GQA, causal+sink, THD+ragged-TH1-stats, and cu_seq_len cases for both fp8 and mxfp8; dense mxfp8 KV-padding; sm107 module-level THD-leg load checks. Verified on B200 (backend 9.23.01): test/python/sdpa/frost 669 passed, 5 failed — all five are cu_seq_len graphs hitting the pre-existing native-lowering version gate (fp8-family cu_seq_len needs the unified node, cuDNN >= 9.24/9.25; develop's own f16 cu tests fail identically on this backend and are green on CI's 9.26). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
vedaanta
added a commit
to vedaanta/cudnn-frontend
that referenced
this pull request
Aug 21, 2026
…via the write_thd_meta envelope design (issue NVIDIA#552) Port the device-built-metadata + plan-time-envelope THD design (PRs NVIDIA#606/NVIDIA#608) into the per-tensor FP8 SM100 kernel, its SM107 (Rubin) sibling (hunk-symmetric), and the block-scale MXFP8 SM100 kernel — the port NVIDIA#622 prescribed when it removed the legacy leg: - Kernels: dynamic packed token extents (cute.sym_int; plan-time-only compile keys), the shared build_thd_meta_o_descs_kernel setup launch (metadata + per -batch O TMA descriptors built device-side, no length ever reaches the host), the plan-time envelope grid with the batch == n_batch dead-unit sentinel (O-store skip; LSE/amax_o predicated on the per-sequence Q length from the device metadata), and ragged Stats in the caller's declared layout (token-major TH1 rank-2 or head-major rank-3, static-rank dispatch). - MXFP8 THD scale factors travel PACKED per-sequence-TILE-padded ([1, H, Σ_b ceil(S_b/128), SF_SMEM] tile sequences in cu_seqlens order, matching the tile base the kernel derives via _thd_sf_tile_bases). The packed tile extent is a runtime value that must come without a device read (Rule 3), so it derives from the SF buffer's byte size — THD SF buffers are exactly the packed layout (its head stride could address nothing else); the SF descriptors use B=1 + dynamic tile extents. - Adapter: factor the SM100 THD packing into _thd_pack (mirrors the SM120 class): metadata/O-desc scratch, capacity token floors, zero-capacity clamps, envelope units — used by the f16 _execute_thd and the new FP8/MXFP8 THD branches. FP8/MXFP8 serve the packed contract only (_thd_check_strides_packed; no stride keys in _thd_compile_kwargs). No Amax_S, no descale_s/scale_s — dropped on these kernels (NVIDIA#602/NVIDIA#619); the amax_o protocol (in-kernel atomicMax, device-side scale_o divide) is unchanged under THD. - Engines: the SM100 FP8/MXFP8 rows declare thd=True + cu_seq_len=True; the arch RANGE (sm 100..119) already routes cc10.7 through the SM107 sibling. - pygraph: sdpa_mxfp8 gains trailing use_padding_mask / seq_len_q / seq_len_kv / cu_seq_len_q / cu_seq_len_kv kwargs (sdpa_fp8 already had them) — the THD length carriers, and dense mxfp8 + KV padding becomes constructible for the first time (tested; stats off — padded_stats is not declared). - Tests: THD self-attention (masks x e4m3/e5m2), cross-attention + GQA, causal+sink, THD+ragged-TH1-stats, and cu_seq_len cases for both fp8 and mxfp8; dense mxfp8 KV-padding; sm107 module-level THD-leg load checks. Verified on B200 (backend 9.23.01): test/python/sdpa/frost 669 passed, 5 failed — all five are cu_seq_len graphs hitting the pre-existing native-lowering version gate (fp8-family cu_seq_len needs the unified node, cuDNN >= 9.24/9.25; develop's own f16 cu tests fail identically on this backend and are green on CI's 9.26). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
vedaanta
added a commit
that referenced
this pull request
Aug 21, 2026
…via the write_thd_meta envelope design (issue #552) (#648) * frost(sdpa): THD/varlen on the FP8/MXFP8 SM100/SM107 forward engines via the write_thd_meta envelope design (issue #552) Port the device-built-metadata + plan-time-envelope THD design (PRs #606/#608) into the per-tensor FP8 SM100 kernel, its SM107 (Rubin) sibling (hunk-symmetric), and the block-scale MXFP8 SM100 kernel — the port #622 prescribed when it removed the legacy leg: - Kernels: dynamic packed token extents (cute.sym_int; plan-time-only compile keys), the shared build_thd_meta_o_descs_kernel setup launch (metadata + per -batch O TMA descriptors built device-side, no length ever reaches the host), the plan-time envelope grid with the batch == n_batch dead-unit sentinel (O-store skip; LSE/amax_o predicated on the per-sequence Q length from the device metadata), and ragged Stats in the caller's declared layout (token-major TH1 rank-2 or head-major rank-3, static-rank dispatch). - MXFP8 THD scale factors travel PACKED per-sequence-TILE-padded ([1, H, Σ_b ceil(S_b/128), SF_SMEM] tile sequences in cu_seqlens order, matching the tile base the kernel derives via _thd_sf_tile_bases). The packed tile extent is a runtime value that must come without a device read (Rule 3), so it derives from the SF buffer's byte size — THD SF buffers are exactly the packed layout (its head stride could address nothing else); the SF descriptors use B=1 + dynamic tile extents. - Adapter: factor the SM100 THD packing into _thd_pack (mirrors the SM120 class): metadata/O-desc scratch, capacity token floors, zero-capacity clamps, envelope units — used by the f16 _execute_thd and the new FP8/MXFP8 THD branches. FP8/MXFP8 serve the packed contract only (_thd_check_strides_packed; no stride keys in _thd_compile_kwargs). No Amax_S, no descale_s/scale_s — dropped on these kernels (#602/#619); the amax_o protocol (in-kernel atomicMax, device-side scale_o divide) is unchanged under THD. - Engines: the SM100 FP8/MXFP8 rows declare thd=True + cu_seq_len=True; the arch RANGE (sm 100..119) already routes cc10.7 through the SM107 sibling. - pygraph: sdpa_mxfp8 gains trailing use_padding_mask / seq_len_q / seq_len_kv / cu_seq_len_q / cu_seq_len_kv kwargs (sdpa_fp8 already had them) — the THD length carriers, and dense mxfp8 + KV padding becomes constructible for the first time (tested; stats off — padded_stats is not declared). - Tests: THD self-attention (masks x e4m3/e5m2), cross-attention + GQA, causal+sink, THD+ragged-TH1-stats, and cu_seq_len cases for both fp8 and mxfp8; dense mxfp8 KV-padding; sm107 module-level THD-leg load checks. Verified on B200 (backend 9.23.01): test/python/sdpa/frost 669 passed, 5 failed — all five are cu_seq_len graphs hitting the pre-existing native-lowering version gate (fp8-family cu_seq_len needs the unified node, cuDNN >= 9.24/9.25; develop's own f16 cu tests fail identically on this backend and are green on CI's 9.26). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * frost(sdpa): PR #648 review fixes — sdpa_mxfp8 cu_seq_len docstring; E741 renames in the new mxfp8 tests - sdpa_mxfp8 docstring: document cu_seq_len_q / cu_seq_len_kv (prefix-sum semantics, mutual exclusion with seq_len_*, cuDNN 9.24+), matching the sdpa / sdpa_fp8 documentation. - test_sdpa_fwd_mxfp8_sm100.py: rename the six new call sites' O locals to o_out/o_ref (Ruff E741); pre-existing sites unchanged. Not-applicable findings, verified: the dead-unit TMA-load concern is unreachable (THD compiles always carry MASK_PADDED — _mask_flags_from forces it for thd_varlen and _validate_knobs raises otherwise — so the loader's masked-bounds branch resolves the dead unit's empty KV range from the device metadata); test_fp8_thd_leg_loads is already L0 via the file's module-level pytestmark. Validated against the LATEST 9.26 backend (9.26.0.33, headers + libs): fp8/mxfp8/sm107 suites 80 passed (including both cu_seq_len tests the local 9.23 backend gates), f16 THD suite 193 passed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * frost(sdpa): rebase follow-ups — #658 split-kv direct-call tests on the THD ABI; #661 d192 kernels join the shared FP8-family ABI; hoist _thd_lse_tokens_cap - test_sdpa_fwd_split_kv_sm100: the fp8/mxfp8 legs drive the kernel hosts positionally and predate the THD ABI (o_desc_words + n_thd_units, both dense-folded) — pass the same dummies the f16 leg already does. - prefill_d192_d128_{fp8,mxfp8}_sm100 (#661, dense-only): accept the same dense-folded THD ABI slots as their d128 siblings so the adapter's launch shape stays uniform across the SM100 FP8 family (the kernels never read them; CFG.THD_VARLEN=1 still fails at trace time — the engine rows and a check_support gate keep THD routed to d128/d128 only). - api_dsl: the THD LSE token-capacity rule (token-major and COMPACT head-major join the packed-Q floor; head-major with a declared stride carries its own extent) was triplicated across the SM100 executes — one documented helper (_thd_lse_tokens_cap) now owns the subtlety. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * frost(sdpa): fix mhas fp8/mxfp8 ragged NaNs — clamp K/V TMA past the packed total; dead-row O := 0 on zero-length KV Two bugs surfaced by the frost:rel:sdpa:sm100 CI mhas fp8 ragged sweeps (gitlab job 404201758, 16 failures): 1. NaN-poisoned capacity tails: test_mhas_v2 NaN-fills the ragged capacity tail past the packed total, and the last sequence's KV envelope tile loads step into it. The padding mask kills those columns in S (NaN-safe select), but BMM2 still computes P(0) . V(NaN) = NaN. Fix: the THD setup kernel (build_thd_meta_o_kv_descs_kernel) now also emits runtime K/V TMA descriptors with GLOBAL_DIM clamped to the device-side packed total cu_k[B] — tail loads land as TMA OOB zero-fill, zero host reads. The fp8/mxfp8 mainloops read them from two extra o_desc_words slots. 2. Zero-length KV sequences (e.g. seq_len_kv=[0, 83, 77]): an empty mainloop never writes the O TMEM, and the epilogue's `o_chunk * inv_sum(=0)` cannot zero the garbage when it happens to be NaN (uninitialized TMEM on the sequence's first tile). Port the f16 dead-row contract (O := 0, LSE := -inf) into the fp8 sm100/sm107 and mxfp8 epilogues: `row_dead = total_sum <= 0` hoisted above the sink branch, and the stored O elements (plus amax_o inputs) selected to 0 explicitly. Tests: frost fp8/mxfp8 suites get NaN-poisoned capacity tails in _dense_buf (mhas parity) and new zero-length-KV THD regression tests; mhas fp8 fwd+bwd ragged L0 sweeps now 46/46 x3 runs, frost fp8/mxfp8/split-kv/sm107 suites 166/166 on cuDNN 9.26. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Before submitting
pre-commit runand committed any formatting changes.cat-*, one or moremod-*, and oneorig-*.Affected area
FROST SM100/SM107 FP8 + SM100 MXFP8 SDPA forward kernels; the SM100 adapter's fp8-THD seams.
Summary
Dead-code removal (−349/+141): the per-tensor FP8 and MXFP8 kernels carried a THD/varlen leg from before the issue-#552 device-built-metadata + plan-time-envelope design existed. It was never reachable, three layers deep: the engine specs declare
thd=Falsefor these cells, the adapter parked fp8 THD behind a"thd-deferred"compile sentinel nothing unwired, and the execute paths raise for THD. When fp8/mxfp8 THD lands on these arches it will follow thewrite_thd_metaenvelope design the f16 kernels use (#606/#608), not this leg.Removed: every
CFG.THD_VARLENbranch in the three kernels (launch setup + grid, per-batch O-descriptor store, packed-LSE store, fake-tensor/compile ternaries, mxfp8 SF-tile THD kwargs no caller passes);build_o_descs_kernelfromthd_sm100.py(these kernels were its only importers —build_thd_meta_o_descs_kernelstays, the f16 kernels use it); the"thd-deferred"sentinel (SM100check_supportnow declines fp8/mxfp8+THD explicitly, covering direct construction).Kept, deliberately: the
_hostABI slots (adapter passes them positionally; annotated unused), the sharedthd_tma_offsets-family call sites (fold to dense identity atTHD_VARLEN=0— dense codegen byte-identical), and a trace-time guard soTHD_VARLEN=1fails loudly instead of silently mis-executing.Validation (B200, cuDNN 9.26 nightly)
sdpa/frost/test_sdpa_fwd_fp8_sm100.pyL0+L1: 32 passed.thd_sm100.pysurvivors' import path exercised end to end.Note: overlaps textually with the descale device-fold commit on #619 in the fp8
_hostsignature region — whichever merges second takes a one-hunk rebase.🤖 Generated with Claude Code
Summary by CodeRabbit