Repository navigation
Description
What happens
On an NVIDIA H200, SEP 28 measured the low-storage march (RK4IP keeping few registers, for field-free planes on large Cartesian grids) at 51.6 ms a step at 384² by 600 and 227 ms at 768² by 600, against 30.8 and 168.7 ms for a dedicated single-transform engine: 1.67 and 1.35 times, on 2026-10-20.
A cuFFT probe found the transforms near their best: one rank-3 transform would save only 1.6 to 2.2 ms an evaluation. The rest is the gather and scatter, 32 ms a step at 768², and a fold near the card's bandwidth, so the volumes each pass reads and writes. A march that touches fewer volumes per pass is the remaining lever.
Why it matters
This is the route for the largest grids, where a GPU's memory bandwidth sets the run time, so the gap is paid at each step.
To do
- Re-measure both grids on an H200 at the current tree with a per-step timer. The tracker's
kerr_beam_384row (384 x 384 x 750, four steps, 4.16 s on an H200 at 0.1.0) is a whole run, so it is not comparable; what takes the rest of that time is worth knowing. - Count volumes read and written per pass, and try a march that touches fewer, on the option list in SEP 28.
What must not change: the march's fields within its parity bounds, and the memory ceiling that is the reason for this route (#353 measures that ceiling).
Done when
The per-step gap to a single-transform engine on an H200 at 768² by 600 is below 1.2 times, or the issue closes with the re-measured gap and the reason nothing further helps.
Checked against commit 8a9f764. Takes over the roadmap's entry "The march's register traffic on devices".
Activity
jtravs commentedon Nov 4, 2026
Re-measured with a per-step timer on the far-field branch (#497), median of steps 3 to 10 of one process, low-storage RK4IP with fixed 0.5 µm steps, fused-silica Kerr beam at 260 nm, field-free planes. The temporal collar stays on, no transverse collar. The spread was under 0.2 % within a process.
jtravs commentedon Nov 4, 2026
Decision (maintainer): try it. This stays open to try a march that touches fewer volumes per pass, as the second to-do above says. A closing note is only acceptable with the re-measured gap and the reason nothing further helps.
Where to start, from the numbers above:
backend.propagate_toaround steps 3 to 10, withboundary_check=None,fields="none"and a smallenergystatistic after each step to make sure the step had finished. That statistic isn't free (The energy statistic costs more than a far-field pass on a device, unexplained #503), but it wasn't counted in the step's time.Neither card has been re-run since #496 and #497. #496 changes what's held (
held_plane=False), which could move the numbers.jtravs commentedon Nov 4, 2026
The per-pass budget, from kernel traces. One step at 768 x 768 x 600, 4 steps averaged,
7a224ea5, 2026-11-04. This case takes the volume route (free.volume_kerrappears), which by reading is 30 volumes a step outside cuFFT's and rocFFT's own traffic.low_storage.separable_foldfree.volume_kerrlow_storage.separable_inputlow_storage.separable_normsjtravs commentedon Nov 5, 2026
Re-measured with #520. At 768 x 768 x 600, by
python -m benchmarks.device_spliton one runner's second point:Reaching x1.2 would take 202 ms on the H200, and this is how far the leads went:
expskipped, input through tiles, one reduction at the accept. The step falls from about 239 ms to 220 ms.What's left is the budget in the last comment: the eight transforms take about 144 ms, and SEP 28's probe found them near their best, and the fold's 52 ms is near the card's bandwidth for those volumes. I have no lead left that keeps the memory ceiling.
One lead remains outside the step itself. With the default check on, the temporal monitor's sums take 24 ms a step on the H200, beside
_finish's 11 to 15 ms. If both read the state, a shared pass might save some of that on a checked run. I haven't checked or tried it, and it wouldn't change this issue's number, which is an unchecked step.So I'd close this with the gap at x1.30, under "the re-measured gap and the reason nothing further helps". I'll leave that to you.
jtravs commentedon Nov 5, 2026
This corrects my last comment. I said the transforms were near their best and that I had no lead left, and both were wrong. So this stays open, as the maintainer has decided.
The dedicated single-transform engine's own figures, on an NVIDIA H200 at 768 x 768 x 600, against this route with #520:
So memory isn't what limits this route, since the engine holds less. The engine also has no temporal collar in its step, which is up to 18 ms of this route's, and that part is the price of a capability.
The transforms. The engine's plan keeps the frequency axis contiguous,
{N,N,T}. This route's scratch is(omega, y, x), so its plan is{T,N,N}. On an H200,{N,N,T}alone takes 14.7 ms a transform at 768, which is the engine's 117.6 ms for eight.{T,N,N}hasn't been timed alone on that card.An interleaved probe on an NVIDIA A40 (CUDA 12.8.1), T = 600, ms a transform:
{N,N,T}, work area{T,N,N}, work area{T,N,N}/{N,N,T}The passes. On a device the state and registers are already frequency-contiguous, and only the scratch differs, so the fold and the input cross two orders. On the same A40 at 768, a plain pass of two reads and a write takes 30.1 ms in one order, 61.9 ms crossing by lines and 117.3 ms crossing along the scratch's rows. That's a bare kernel, not this route's tiled fold, so it shows what crossing can cost and not what the fold would gain.
The 0.7 % I reported for scratch-order registers was a whole step on an A40, where the transforms are 72 % of it. It also moved the registers towards the scratch, the opposite of what the engine does.
What to try. Only the volume route's scratch, in the registers' order. The route is already a guarded fast path for an envelope Kerr beam with one polarisation, an identity crop, and no plasma, Raman, dispersion, harmonic or gaps. The state, the registers and every other route keep their layout. I haven't yet checked what else on this route reads the scratch. An H200 probe of both orders comes first, before any code changes, and if the change can't stay inside that route, it isn't worth making.
jtravs commentedon Nov 5, 2026
The maintainer has set the bar for the transform order: try it only if an H200 probe supports it, the change stays inside the volume route, and the whole step improves by at least 10 %, the bar #361 used. Otherwise this closes with the measured gap and that reason. #529's shared work area goes ahead either way, since it costs no capability.
Hold the volume route's scratch in the registers' order on CUDA
jtravs commentedon Nov 6, 2026
The transform order, measured against the bar above.
The probe. On an NVIDIA H200, CUDA 13.1, T = 600, ms a transform, median of interleaved rounds:
{N,N,T}, work area{T,N,N}, work area{T,N,N}/{N,N,T}That supported trying it, so #542 now holds the volume route's scratch as
(y, x, omega)on CUDA, the state's and the registers' order: the rank-3 pair is{N,N,T}, and the stage's input and fold stream instead of crossing layouts. It stays inside the route: every other pass over the scratch forms its own input in(omega, y, x), as before, and an in-stage absorber keeps the old order. HIP is unchanged until rocFFT's orders are measured.The step, from a field-free point with the check off, (24 steps - 8) / 16, three interleaved rounds against #542 before it:
At 768 the transforms account for it: eight are 23.4 ms apart in the probe, and the step gains 21.6 ms. Fields agree to 3.2e-17 on
python -m benchmarks ab --device. The device tests pass on the H200 and the NVIDIA A40, and on the A40 two deliberate breaks of the order, the stage's time window and the low-storage fold, fail 9 and 3 of them.Against the bar. At 768 the step is 10.5 % faster, or takes 9.5 % less time. That's on the line, and which way it's read decides it, so it's for the maintainer on #542. If it stays, the gap at 768 is 204 ms against the dedicated engine's 159.4 ms, x1.28. If it goes, this closes with 225 ms, x1.41, and that reason. This stays open until then.
jtravs commentedon Nov 6, 2026
Decided: the scratch's order stays in #542 on CUDA, as the faster route.
At 768² by 600 on an NVIDIA H200 a step now takes 204 ms against 225 before. That leaves the gap to the dedicated engine's 159.4 ms at x1.28, so this issue's "below 1.2 times" isn't met yet and it stays open. Two leads remain:
(omega, y, x), because rocFFT's two orders haven't been compared. On an AMD Instinct MI300X, ROCm 10.0.0, rocFFT's in-place rank-3 transform with the frequency innermost takes 17.4 ms at 768 and needs one state of work area at every size measured. Corrected 2026-11-06: that is rocFFT lengths{T,N,N}, which rocFFT lists innermost first, so it is the new order, this issue's{N,N,T}, not HIP's current one. A probe of HIP's current order and a step A/B on an AMD card would settle whether HIP takes the new order. That's an AMD-only question, and it doesn't touch the H200 gap.jtravs commentedon Nov 6, 2026
HIP measured, and it takes the new order too. #547 closes this issue.
On an AMD Instinct MI325X with ROCm 10.0.0, rocFFT's two orders take the same time per transform: 15.31 ms each at 768 x 768 x 600 and 4.07 against 4.10 ms at 384. Both need one state of work area. The step is still faster in the new order in every round:
Since the transforms take the same time, that gain comes from the stage's input and fold no longer crossing layouts.
ab --devicegiveskerr_beam_384x0.978, and the device tests pass on that card in the new order. The volume route now holds the scratch in the registers' order on CUDA and HIP alike.Where the target stands. On the NVIDIA H200 a step at 768 x 768 x 600 takes 204 ms, against the 159.4 ms of a dedicated single-transform engine. That is x1.28, against this issue's target of x1.2. What makes up the 204 ms:
All of those passes are bound by memory traffic, not arithmetic. One read or write of the 5.27 GiB state costs about 1.2 ms at the H200's bandwidth, so 86 ms is roughly 70 state-sized reads or writes per step.
The remaining lever is fusion, not layout. The step has to move the state fewer times: fold Kerr into the stage input, combine the fold with the next stage's input, or let the FFT library's load and store callbacks apply the propagator phase as the transform reads and writes. That is a redesign of the march. If it's taken up, it needs its own issue with a kernel trace first. Closing this one with the gap at x1.28 and that reason.
Hold the volume route's scratch in the registers' order on HIP too