Skip to content

SEP 44's later commits haven't run on an AMD card #541

Closed

Description

@jtravs

What's missing

SEP 44 (#520) qualified on an AMD Instinct MI325X at 9a5d2346, with a HIP build for AMD_GFX942 under ROCm. Its later commits have run on an NVIDIA H200 and an NVIDIA A40, and the HIP build with its tests compiles at the head, but no AMD card has run them: a rented MI325X had no capacity for a day, and an MI300X failed twice before it started. The maintainer set SEP 44 Final without that pass.

The only C++ changed since 9a5d2346 keeps a device's split collar at the node chunk, which the MI325X had already measured (2.2 % of a step, in the ledger's "A device step beyond its transforms"), and answers the native lint and tidy lanes.

What to run on an AMD card

At #520's merge or later, with a HIP build and tests:

  1. pytest -m gpu over tests and over native/tests, which ran at 9a5d2346 only.
  2. A whole scan on a kept backend, the change that keeps a device's backend across a local scan's input-only points (#298). It has run on the NVIDIA A40 only.
  3. The hard-edge fuzz (#504), with the far field's clang fp contract(off) pragma removed as on Far-field aperture shapes, a custom absorbed_power channel, and the boundary items #497's base. It should fail there. If it passes, the fuzz can't see the bug it was written for on HIP.
  4. rocFFT's work area per plan, for #529's table, and whether rocFFT's plans of the scratch can share one work area, as cuFFT's do.
  5. python -m benchmarks.device_split 768, §7's gates, at the head, as a check against the MI325X's figures at 9a5d2346.

An AMD Instinct MI300X (gfx942) serves as well as an MI325X for 1 to 4. The figures in 5 belong to the card they ran on.

Activity

  1. jtravs commented on Nov 6, 2026

    @jtravs
    ContributorAuthor

    Done on an AMD Instinct MI300X (gfx942, ROCm 10.0.0, an Intel Xeon Platinum 8568Y+ host with 20 cores) on 2026-11-06, at c0bd23f2, #542's native code. That's #520's head plus #542's changes. A HIP build takes two of them, the shared work area and the whole-state collar, but not the CUDA-only scratch order. No MI325X had capacity. Everything passed.

    1. Device tests. pytest -m gpu over tests: 251 passed, 1 skipped. Over native/tests: 31 passed. The split rows', edge's, collar's, transverse backends', scans', runner's and stop's modules: 270 passed, 3 skipped. That includes tests/backends/test_native_edge_reading.py's byte count, which shows no state crossing in a field-free default run.

    2. A whole scan on a kept backend. A 5-point Local scan through with_input, check on, ran two rounds each way, and every point finished:

      • 768 x 768 x 600, 8 steps: 37.4 and 36.2 s a backend per point, against 29.2 and 28.9 s kept, x0.79.
      • 384 x 384 x 600, 51 steps: 27.7 and 27.0 s, against 23.7 and 23.7 s, x0.87.
    3. The hard-edge fuzz (Far-field hard edges on HIP rest on a square root whose rounding isn't guaranteed, and the fuzz isn't in the tree #504). With the far field's clang fp contract(off) removed, as on Far-field aperture shapes, a custom absorbed_power channel, and the boundary items #497's base, 31 of the fuzz's 120 cases fail: 12 of the circle's 40 seeds, 19 of the ellipse's, and none of the half-plane's. The older hard-edge test fails too. With the pragma in, they all pass under item 1. So the fuzz sees the bug on HIP.

    4. rocFFT's work area per plan (A device's FFT plans each hold a work area, four states' worth at 768 x 768 #529), asked of rocFFT directly, in place, Z2Z, in units of the state:

      • Rank 3, frequency innermost (rocFFT lengths {T, N, N}, innermost first; corrected 2026-11-06, as this is not HIP's current order): 1.00 at every N from 320 to 1200 and every T from 512 to 1024.
      • Rank 2, batched over T: 1.00 at 192 and 384, 0 at 256 and 512, and 1.10, 1.08 and 1.06 at 640, 768 and 1024.
      • Rank 1: 0.

      The plans of the scratch share one area, as cuFFT's do. At 768 x 768 x 600 a run's plans hold 5.76 GiB. Four areas of their own would hold 22.0 GiB, as the MI325X's 22.02 GiB at 9a5d2346 did.

    5. python -m benchmarks.device_split, field-free, a stated input identity, the first of a runner's points set apart:

    MI325X at 9a5d2346 MI300X at c0bd23f2 Aim
    768 x 768 x 600, 8 steps, 4 planes: check on, s 2.68 2.87 under 6
    The same, check off, s 2.00 2.13
    On against off x1.34 x1.35 under x2
    384 x 384 x 600, 51 steps, 17 planes: on against off x1.20 x1.19 under x1.25
    A step at 768, ms 208 241
    An edge read, ms 31 32 under half a step
    field_to_host, 5.27 GiB, ms 106 168 under 300
    energy, ms 6.7 6.7 under 10
    beam_radius, fwhm_r, ms 44 48
    peak_intensity, ms 74 78

    As this issue says, these figures belong to the card. They're on a different card, so this shows that the gates hold, not that one commit is faster than another.

    From the same run for #542: the whole-state collar gained in every interleaved round, so it stays (#361). #542's ledger has all of this, under "The native read on the data-centre cards" and beside #529's table.

    Closing, since every item passed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions