frost(sdpa): retire the bottom_right_with_swa notch — every row serves BR + SWA - #623
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (1)
Included review availability: Your plan includes up to 12 reviews per rolling hour; 7 remain after this review. 📝 WalkthroughWalkthroughThe SM100 MXFP8 engine now enables bottom-right causal attention together with sliding-window attention. The previous eligibility restriction and its SF-size mismatch explanation were removed. ChangesSM100 MXFP8 attention support
Estimated code review effort: 1 (Trivial) | ~5 minutes Merge Risk: ⚪ Minimal · up to This localized change enables bottom-right and sliding-window support for the SM100 MXFP8 path, with the supplied validation reporting 259 passing tests and no failures; no actionable merge-blocking risk remains beyond normal checks and review. Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
|
@cudnn-ci-bot run frost |
…s BR + SWA The notch guarded one kernel gap: the mxfp8 row kept bottom_right_with_swa off because the one mhas graph it admitted tripped the executor's SF-size mismatch. That was _reshape_sf reading B from the bound tensor's shape (flat F8_128x4 bindings misread), fixed in NVIDIA#606's mxfp8 commit — with it gone, all six bottom_right rows (SM100 f16/fp8/mxfp8, SM80, SM120 f16/fp8) serve the conjunction and the flag no longer differentiates anything. Remove the field, its mismatch() rule, and the five constant-True spec lines; the frost README's notch example now points at bottom_right_padded_seq_q (a live notch: on for SM80/SM120, off for the three SM100 rows, whose kernels anchor the BR diagonal at the global S_q). The two probe tests pinning BR+SWA acceptance are unchanged — they assert behavior, not the flag. Validated on B200 (9.26 nightly): graph-analyzer probe suite + full mxfp8 fwd+bwd mhas sweeps; the mxfp8 engine serves the newly admitted BR+SWA config with numerics green against the fp32 reference.
6c396f5 to
d492896
Compare
|
@cudnn-ci-bot run frost |
1 similar comment
|
@cudnn-ci-bot run frost |
|
@cudnn-ci-bot run frost |
1 similar comment
|
@cudnn-ci-bot run frost |
…s BR + SWA (NVIDIA#623) The notch guarded one kernel gap: the mxfp8 row kept bottom_right_with_swa off because the one mhas graph it admitted tripped the executor's SF-size mismatch. That was _reshape_sf reading B from the bound tensor's shape (flat F8_128x4 bindings misread), fixed in NVIDIA#606's mxfp8 commit — with it gone, all six bottom_right rows (SM100 f16/fp8/mxfp8, SM80, SM120 f16/fp8) serve the conjunction and the flag no longer differentiates anything. Remove the field, its mismatch() rule, and the five constant-True spec lines; the frost README's notch example now points at bottom_right_padded_seq_q (a live notch: on for SM80/SM120, off for the three SM100 rows, whose kernels anchor the BR diagonal at the global S_q). The two probe tests pinning BR+SWA acceptance are unchanged — they assert behavior, not the flag. Validated on B200 (9.26 nightly): graph-analyzer probe suite + full mxfp8 fwd+bwd mhas sweeps; the mxfp8 engine serves the newly admitted BR+SWA config with numerics green against the fp32 reference.
Before submitting
pre-commit runand committed any formatting changes.cat-*, one or moremod-*, and oneorig-*.Affected area
FROST engine capabilities (
python/cudnn/sdpa/fwd/engines.py) + the frost README's notch example.Summary
Retire the
bottom_right_with_swacapability notch — every row now serves the conjunction.The notch guarded exactly one kernel gap: the mxfp8 row kept it off with a note to flip it "once that plumbing is fixed" — the plumbing being the
_reshape_sfSF-size mismatch (B read from the bound tensor's shape, misread for flat F8_128x4 bindings), fixed in #606's mxfp8 commit. With that gone, all sixbottom_rightrows (SM100 f16/fp8/mxfp8, SM80, SM120 f16/fp8) serve BR + SWA and the flag differentiates nothing, so instead of flipping the last row on, this removes the field, itsmismatch()rule, and the five constant-True spec lines.The frost README's "notch" design example now points at
bottom_right_padded_seq_q— a live notch (on for SM80/SM120, off for the three SM100 rows, whose kernels anchor the BR diagonal at the global S_q). The two probe tests pinning BR+SWA acceptance (test_probe_accepts_bottom_right_with_swa,test_sm120_probe_accepts_bottom_right_with_swa) are unchanged — they assert behavior, not the flag.Validation
B200, cuDNN 9.26 nightly: graph-analyzer probe suite + full
test_sdpa_mxfp8_fwd_L0+test_sdpa_mxfp8_bwd_L0sweeps — the mxfp8 engine serves the newly admitted BR+SWA config with numerics green against the fp32 reference (routing count 2, up from 1).🤖 Generated with Claude Code