Skip to content

Share one work area across a device's FFT plans of the scratch - #542

Merged
jtravs merged 12 commits into
mainfrom
feature/shared-work-area
Nov 6, 2026
Merged

jtravs merged 12 commits into
mainfrom
feature/shared-work-area

Conversation

@jtravs

@jtravs jtravs commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

The follow-up to #520, whose native code it changes. #520 has merged, and this now targets main. An AMD Instinct MI300X has run it (#541), and everything there passed.

What's here

On a CUDA or HIP build, a free-space run's transforms of the transverse scratch share one work area instead of holding one each (#529). cuFFT's guide allows that for plans launched one after another. The plans become KokkosFFT::DynPlans with automatic allocation off, and one buffer, sized by compute_required_workspace_size over the rank-2 pair and, on the volume route, the rank-3 pair, serves all four. CPU builds are unchanged.

On an NVIDIA A40, CUDA 12.8.1, 8 steps with the check off, two interleaved rounds of three runs each, against #520's head before it:

N x N x 600 Plans, GiB Total, GiB Point, s
384 2.64 to 1.32 8.04 to 6.73 4.84, 4.94 to 4.86, 4.79
750 10.06 to 5.03 26.68 to 21.66 20.24, 20.19 to 20.28, 20.10
768 21.09 to 5.28 38.46 to 22.64 21.37, 21.41 to 21.37, 21.46
900 14.48 to 7.24 37.80 to 30.56 28.74, 28.61 to 28.64, 28.50

python -m benchmarks ab --device --length 0.3, 3 rounds, the same steps and final fields with no difference: kerr_beam_384 x0.994 (0.984–1.048), kerr_beam_192 x1.030 (1.001–1.035, a window of two steps). So the memory falls by up to three quarters of the plans, and the time doesn't change. rocFFT shares too. On an AMD Instinct MI300X, ROCm 10.0.0, a run at 768 x 768 x 600 holds 5.76 GiB of plans and 23.37 GiB in all. The MI325X held 22.02 GiB of plans at 9a5d2346, and rocFFT's four plans would take 22.0 GiB of work areas of their own.

A device's split collar, a whole state at a time again (#361). #520 dropped this because it missed #361's 10 %. As the maintainer decided there, it's back for a CUDA or HIP build only, without the seam field #520 removed: in place on the state's rows, a block holds no buffer of its own, so a device takes every row at once whatever node_budget says, 14 kernels and 10 transforms a step at 768 x 768 x 600 against 119 and 220 at the node chunk. A CPU keeps the chunk. It was to stay only if an interleaved run on an AMD card gained in every round. On an AMD Instinct MI300X every round did: three rounds of fresh processes, 8 steps, check off, x0.990, x0.988 and x0.990 at 384 x 384 x 600, and x0.984, x0.983 and x0.986 at 768 x 768 x 600. The plans' memory was the same. ab --device --length 0.3 gives kerr_beam_384 x0.952 (0.912–0.999), with the fields identical. So it stays, at about 1.5 % of a point. On an NVIDIA A40, CUDA 12.8.1, python -m benchmarks ab --device --length 0.3, 3 rounds, against this branch before it: kerr_beam_384 x0.989 (0.958–0.994), kerr_beam_192 x0.971 (0.951–1.065), the routes without the collar x0.998–x1.004, and the fields identical.

The volume route's scratch in the registers' order (#365), on CUDA only. A stage of the volume route now views the transverse scratch as (y, x, omega), the order of the state and of a device's registers, so its rank-3 pair is cuFFT's frequency-fastest {N,N,T} and its input and fold stream instead of crossing layouts. Every other pass over the scratch forms its own input in (omega, y, x), as before, so the change stays inside the volume route; an in-stage absorber keeps the old order. On an NVIDIA H200, CUDA 13.1, a step (from a field-free point, (24 steps - 8) / 16), three interleaved rounds against this branch before it:

N x N x 600 Before, ms After, ms Ratio A run's memory, GiB
384 50.96–51.17 36.22–36.49 0.713 7.23 → 5.91
768 225.21–225.78 203.88–204.13 0.905 23.14 → 23.14

python -m benchmarks ab --device --length 0.3, 3 rounds: kerr_beam_384 x0.971 (0.928–1.003), fields to 3.2e-17. On the A40, where cuFFT's two orders are within 1 % at 768, the step doesn't change at 768 and falls 8 % at 384. HIP keeps the old order until rocFFT's orders are measured.

What a HIP device's square root guarantees at a hard edge (#504). This change is comments and documentation only. In a HIP build the far field's sqrt is __builtin_sqrt, so it becomes llvm.sqrt.f64. LLVM defines that intrinsic as a conforming IEEE 754 square root unless the call carries afn, and solprop's HIP build sets no fast-math option. AMD's own math documentation promises less: __dsqrt_rn is round-to-nearest-even and sqrt is 1 ulp, both measured over [0, 100]. The comment in far_field.hpp and a paragraph in accuracy.md section 7 now say so.

How it was verified

  • NVIDIA A40: pytest -m gpu over tests 251 passed, 2 skipped, and over native/tests 31 passed. A new test, test_the_plans_of_the_scratch_hold_one_work_area, holds a run's plan growth under 1.5 states.
  • NVIDIA A40, with the collar: pytest -m gpu over tests 251 passed, 2 skipped, and over native/tests 31 passed. With the scratch's order too, the same: 251 passed, 2 skipped, and 31 passed.
  • Two deliberate breaks of the order, each built and run on the A40: with the stage's time window read in the old order, 9 device tests fail across the Runge-Kutta, low-storage and multistep marches; with the low-storage fold reading the scratch in the old order, 3 do.
  • NVIDIA H200, CUDA 13.1, at c0bd23f2: pytest -m gpu tests 251 passed, 1 skipped; the split rows', edge's, collar's and transverse backends' modules 172 passed, 3 skipped.
  • HIP under ROCm 7.2.4, tests included and warnings as errors: compiles.
  • AMD Instinct MI300X, ROCm 10.0.0, gfx942, at c0bd23f2 (SEP 44's later commits haven't run on an AMD card #541). pytest -m gpu over tests: 251 passed, 1 skipped. Over native/tests: 31 passed. The split rows', edge's, collar's, transverse backends', scans', runner's and stop's modules: 270 passed, 3 skipped.
    • SEP 44's gates all hold. At 768 x 768 x 600 the check costs x1.35 (2.87 s against 2.13), and at 384, over 51 steps, x1.19. An edge read takes 32 ms, field_to_host 168 ms and energy 6.7 ms.
    • A 5-point Local scan on a kept backend finished every point: x0.79 at 768 and x0.87 at 384.
    • With the far field's contraction pragma removed, the hard-edge fuzz fails 31 of its 120 cases, and the older hard-edge test fails too.
  • HIP's double sqrt, compiled for gfx942 under ROCm 7.2.4: the ISA is a reciprocal-root estimate refined by fused multiply-adds, not the bare v_sqrt_f64.
  • CI at d046254d passed, both required checks. The head adds only documentation, the ledger, changelog lines and a comment in far_field.hpp. Its CI was running when this was written.

The scratch's order

The maintainer decided to take the fastest route, so the order stays on CUDA. At 768 x 768 x 600 a step on an NVIDIA H200 is x0.905, 204 ms against 225, and at 384 x 384 x 600 x0.713. That leaves #365's gap at 768 at 204 ms against the dedicated engine's 159.4 ms, x1.28, short of its 1.2, so #365 stays open. HIP keeps the old order, since rocFFT's two orders haven't been compared.

Closes #529 and #504. Relates to #365, #361 and #541.

🤖 Generated with Claude Code

https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ

jtravs and others added 2 commits November 5, 2026 17:12
Each plan of the transverse scratch held a work area of its own, up to a
whole state, so at 768 x 768 x 600 the four took 21 GiB, the largest row of
a run. They run one after another, and cuFFT's guide allows plans launched
sequentially to share one. KokkosFFT's DynPlan can set it, so a device's plans
of the scratch use that. A host keeps its plans. #529.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
@read-the-docs-community

read-the-docs-community Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

jtravs and others added 3 commits November 5, 2026 22:47
The maintainer asked to bring it back where it costs no capability: in place on the
state's rows, a block holds no buffer of its own, only its plans' work area, so a
device now takes every row at once, whatever the node budget, with no seam field. At
768 x 768 x 600 that is 14 kernels and 10 transforms a step where the node chunk took
119 and 220. A CPU keeps the chunk. Whether it stays depends on an interleaved run on
an AMD card, where it measured x0.978 before (#361, #541).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
SEP 44's Final and today's main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
@codecov

codecov Bot commented Nov 6, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

jtravs and others added 3 commits November 6, 2026 02:34
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
cuFFT's rank-3 transform of a frequency-fastest volume, {N,N,T}, takes
14.7 ms at 768 x 768 x 600 on an NVIDIA H200 against 17.6 ms for the
scratch's {T,N,N} (#365). A stage of the volume route now views the
transverse scratch as (y, x, omega), the order of the state and of the
low-storage registers on a device, so its input and fold stream rather
than cross layouts. Every other pass over the scratch forms its own
input in (omega, y, x), as before. An in-stage absorber, which takes its
term from the material pass's input, keeps the scratch's order. HIP is
unchanged until rocFFT is measured.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
On an NVIDIA H200, three interleaved rounds, a step takes 204 ms at
768 x 768 x 600 where it took 225 (x0.905), and 36 ms at 384 x 384 x 600
where it took 51 (x0.713), with 1.3 GiB less memory there; on the A40
the orders are within 1 % at 768 and the step doesn't change. Nine
device tests catch the window read in the old order, three the fold.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
On an AMD Instinct MI300X at #542's native code, the whole-state collar gained in
every interleaved round (x0.983 to x0.990 at 384 and 768), so it stays; rocFFT's
plans of the scratch share one work area as cuFFT's do; SEP 44's gates hold; a
kept backend's scan finishes every point faster; and the hard-edge fuzz fails 32
cases with the far field's pragma removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
jtravs and others added 2 commits November 6, 2026 06:24
HIP's double sqrt is __builtin_sqrt, so llvm.sqrt.f64, which LLVM defines
as a conforming IEEE 754 square root unless the call carries afn; the
AMDGPU backend expands it so, and solprop's HIP build sets no fast-math
option. AMD's math documentation promises less, a measured 0 or 1 ulp
over [0, 100]. The far field's comment and accuracy.md section 7 now say
so, with the MI300X's fuzz result (#504).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
@jtravs
jtravs marked this pull request as ready for review November 6, 2026 07:35
Copilot AI balanced review requested due to automatic review settings November 6, 2026 07:35

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 6, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-06T07:37:22.277128Z 931ac0d Draft marked ready
🔒 Security Review ✅ Completed 2026-11-06T07:38:33.628340Z 931ac0d Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@jtravs
jtravs changed the base branch from feature/sep-0044-native-edge to main November 6, 2026 08:00
@jtravs
jtravs merged commit 1d785cb into main Nov 6, 2026
40 of 41 checks passed
@jtravs
jtravs deleted the feature/shared-work-area branch November 6, 2026 12:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A device's FFT plans each hold a work area, four states' worth at 768 x 768

2 participants