Skip to content

frost(sdpa): retire the bottom_right_with_swa notch — every row serves BR + SWA - #623

Merged
vedaanta merged 1 commit into
NVIDIA:developfrom
vedaanta:vagarwalla/frost-mxfp8-br-swa
Aug 17, 2026
Merged

frost(sdpa): retire the bottom_right_with_swa notch — every row serves BR + SWA#623
vedaanta merged 1 commit into
NVIDIA:developfrom
vedaanta:vagarwalla/frost-mxfp8-br-swa

Conversation

@vedaanta

@vedaanta vedaanta commented Aug 17, 2026

Copy link
Copy Markdown
Collaborator

Before submitting

  • I agree to license this contribution under the terms of LICENSE.txt.
  • I ran pre-commit run and committed any formatting changes.
  • I added GitHub labels: one cat-*, one or more mod-*, and one orig-*.

Affected area

FROST engine capabilities (python/cudnn/sdpa/fwd/engines.py) + the frost README's notch example.

Summary

Retire the bottom_right_with_swa capability notch — every row now serves the conjunction.

The notch guarded exactly one kernel gap: the mxfp8 row kept it off with a note to flip it "once that plumbing is fixed" — the plumbing being the _reshape_sf SF-size mismatch (B read from the bound tensor's shape, misread for flat F8_128x4 bindings), fixed in #606's mxfp8 commit. With that gone, all six bottom_right rows (SM100 f16/fp8/mxfp8, SM80, SM120 f16/fp8) serve BR + SWA and the flag differentiates nothing, so instead of flipping the last row on, this removes the field, its mismatch() rule, and the five constant-True spec lines.

The frost README's "notch" design example now points at bottom_right_padded_seq_q — a live notch (on for SM80/SM120, off for the three SM100 rows, whose kernels anchor the BR diagonal at the global S_q). The two probe tests pinning BR+SWA acceptance (test_probe_accepts_bottom_right_with_swa, test_sm120_probe_accepts_bottom_right_with_swa) are unchanged — they assert behavior, not the flag.

Validation

B200, cuDNN 9.26 nightly: graph-analyzer probe suite + full test_sdpa_mxfp8_fwd_L0 + test_sdpa_mxfp8_bwd_L0 sweeps — the mxfp8 engine serves the newly admitted BR+SWA config with numerics green against the fp32 reference (routing count 2, up from 1).

🤖 Generated with Claude Code

@vedaanta vedaanta added orig-nv-eng Reported or requested by NVIDIA engineering. mod-cutedsl CuTeDSL kernels, generated kernels, examples, or related integration work. cat-enhancements mod-frost labels Aug 17, 2026
@coderabbitai

coderabbitai Bot commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: d6f801c9-3b56-4c1c-bfc4-8ddfec253c5b

📥 Commits

Reviewing files that changed from the base of the PR and between 491805e and 6c396f5.

📒 Files selected for processing (1)
  • python/cudnn/sdpa/fwd/engines.py

Included review availability: Your plan includes up to 12 reviews per rolling hour; 7 remain after this review.


📝 Walkthrough

Walkthrough

The SM100 MXFP8 engine now enables bottom-right causal attention together with sliding-window attention. The previous eligibility restriction and its SF-size mismatch explanation were removed.

Changes

SM100 MXFP8 attention support

Layer / File(s) Summary
Enable combined attention mode
python/cudnn/sdpa/fwd/engines.py
The SM100 MXFP8 capability row now supports bottom-right causal attention with sliding-window attention. The previous exclusion was removed.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Merge Risk: ⚪ Minimal · up to 6c396

This localized change enables bottom-right and sliding-window support for the SM100 MXFP8 path, with the supplied validation reporting 259 passing tests and no failures; no actionable merge-blocking risk remains beyond normal checks and review.

Suggested reviewers: aneureka

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly and concisely describes retiring the bottom-right-with-SWA capability notch.
Description check ✅ Passed The description explains the change, rationale, affected area, and validation results, but omits several template headings and exact test commands.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@vedaanta

Copy link
Copy Markdown
Collaborator Author

@cudnn-ci-bot run frost

…s BR + SWA

The notch guarded one kernel gap: the mxfp8 row kept bottom_right_with_swa
off because the one mhas graph it admitted tripped the executor's SF-size
mismatch. That was _reshape_sf reading B from the bound tensor's shape
(flat F8_128x4 bindings misread), fixed in NVIDIA#606's mxfp8 commit — with it
gone, all six bottom_right rows (SM100 f16/fp8/mxfp8, SM80, SM120 f16/fp8)
serve the conjunction and the flag no longer differentiates anything.

Remove the field, its mismatch() rule, and the five constant-True spec
lines; the frost README's notch example now points at
bottom_right_padded_seq_q (a live notch: on for SM80/SM120, off for the
three SM100 rows, whose kernels anchor the BR diagonal at the global S_q).
The two probe tests pinning BR+SWA acceptance are unchanged — they assert
behavior, not the flag.

Validated on B200 (9.26 nightly): graph-analyzer probe suite + full mxfp8
fwd+bwd mhas sweeps; the mxfp8 engine serves the newly admitted BR+SWA
config with numerics green against the fp32 reference.
@vedaanta
vedaanta force-pushed the vagarwalla/frost-mxfp8-br-swa branch from 6c396f5 to d492896 Compare August 17, 2026 17:16
@vedaanta vedaanta changed the title frost(sdpa): serve bottom-right + sliding-window on the SM100 MXFP8 row frost(sdpa): retire the bottom_right_with_swa notch — every row serves BR + SWA Aug 17, 2026
@vedaanta

Copy link
Copy Markdown
Collaborator Author

@cudnn-ci-bot run frost

1 similar comment
@vedaanta

Copy link
Copy Markdown
Collaborator Author

@cudnn-ci-bot run frost

@vedaanta

Copy link
Copy Markdown
Collaborator Author

@cudnn-ci-bot run frost

1 similar comment
@vedaanta

Copy link
Copy Markdown
Collaborator Author

@cudnn-ci-bot run frost

@vedaanta
vedaanta merged commit b1df12b into NVIDIA:develop Aug 17, 2026
hxbai pushed a commit to hxbai/cudnn-frontend that referenced this pull request Aug 18, 2026
…s BR + SWA (NVIDIA#623)

The notch guarded one kernel gap: the mxfp8 row kept bottom_right_with_swa
off because the one mhas graph it admitted tripped the executor's SF-size
mismatch. That was _reshape_sf reading B from the bound tensor's shape
(flat F8_128x4 bindings misread), fixed in NVIDIA#606's mxfp8 commit — with it
gone, all six bottom_right rows (SM100 f16/fp8/mxfp8, SM80, SM120 f16/fp8)
serve the conjunction and the flag no longer differentiates anything.

Remove the field, its mismatch() rule, and the five constant-True spec
lines; the frost README's notch example now points at
bottom_right_padded_seq_q (a live notch: on for SM80/SM120, off for the
three SM100 rows, whose kernels anchor the BR diagonal at the global S_q).
The two probe tests pinning BR+SWA acceptance are unchanged — they assert
behavior, not the flag.

Validated on B200 (9.26 nightly): graph-analyzer probe suite + full mxfp8
fwd+bwd mhas sweeps; the mxfp8 engine serves the newly admitted BR+SWA
config with numerics green against the fp32 reference.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cat-cleanup cat-enhancements mod-cutedsl CuTeDSL kernels, generated kernels, examples, or related integration work. mod-frost orig-nv-eng Reported or requested by NVIDIA engineering.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants