Repository navigation
Hold the volume route's scratch in the registers' order on HIP too - #547
Merged
Merged
Conversation
rocFFT lists lengths innermost first, so the probe's {T, N, N} held the
frequency innermost, not HIP's current (omega, y, x).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
#365's order was CUDA-only until rocFFT's two orders were measured. On an AMD Instinct MI325X, ROCm 10.0.0, the orders take the same time a transform (15.31 ms each at 768 x 768 x 600), but the step still gains from the input and fold that no longer cross layouts: 199 against 211 ms at 768 and 52 against 57 ms at 384, three interleaved rounds, every one faster. The device tests pass on that card in the new order. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
2 tasks
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#542 held the volume route's scratch in the registers' order,
(y, x, omega), on CUDA only, and left HIP in(omega, y, x)until rocFFT's two orders had been measured. They now have been, on an AMD Instinct MI325X, and HIP is faster in the new order. This PR turns it on for HIP and corrects how the ledger labelled the earlier rocFFT probe.The change
The four
KOKKOS_ENABLE_CUDAguards on the order innative/src/general.cppnow also takeKOKKOS_ENABLE_HIP. They cover the index's field,cell(),stage_index()andplan(). The comments,docs/dev/backends.mdand #365's changelog fragment now say "on a device". There is no other code change.What was measured
All of this ran on an AMD Instinct MI325X with ROCm 10.0.0,
gfx942.rocFFT's two orders take the same time. Each transform is in place, Z2Z, with T = 600:
Both orders need one state of work area at each of these sizes.
The step still gains, every round. A step is measured from a field-free point with the check off, as (24 steps - 8 steps) / 16. The probe ran three interleaved rounds against #542's head (
931ac0dc):At 768 the eight transforms take the same time in both orders, so the step's 11.8 ms gain comes from the passes beside them. Those are the stage's input and fold, which no longer cross layouts.
A/B and tests.
python -m benchmarks ab --device --length 0.3over 3 rounds giveskerr_beam_384x0.978 (0.974–0.982), with the fields within 1.8e-17. On that card, in the new order, these pass:pytest -m gpu: 251 passed, 1 skipped.The only native difference between this branch and the build that was measured is the comment.
A correction
rocFFT lists
lengthsinnermost first. So the earlier rank-3 probe in the ledger, at lengths{T, N, N}, measured the frequency-innermost order, not HIP's old one. The ledger now names the order and adds the x-innermost work areas. The run-level memory figures stand, because they come from solprop's own plans. The comments on #529, #541 and #365 have been corrected the same way.#365
The volume route is now in the scratch's order on both device backends. #365's own target, a step within x1.2 of a dedicated single-transform engine on an NVIDIA H200, is not met: it stands at x1.28 (204 against 159.4 ms). The transforms are about 118 ms of that step. The other ~86 ms is work the step has to do: the fold, the stage's input, Kerr and the temporal collar. That work is bound by memory traffic. Closing the gap means fusing those passes so the step moves the state fewer times, which is a redesign of the march, not a layout change. I've written that up on #365 as the remaining lever, and this closes it.
Closes #365.
🤖 Generated with Claude Code
https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ