Skip to content

[gpu-2] Benchmark script for VlasovAmpere and VlasovMaxwell - #735

Merged
max-models merged 12 commits into
gpu-1-vlasov-ampere-e2efrom
gpu-2-vlasov-benchmark
Oct 13, 2026
Merged

max-models merged 12 commits into
gpu-1-vlasov-ampere-e2efrom
gpu-2-vlasov-benchmark

Conversation

@max-models

Copy link
Copy Markdown
Member

Solves the following issue(s):

The "first CPU vs GPU timing" item of #689, tracked in #650. This PR provides the script and a CPU baseline. The GPU numbers come from running it on the H100 (#687).

Stack: gpu-2. Merge it after gpu-1 (gpu-1-vlasov-ampere-e2e), which is its base, once all the CUDA PRs are in.

Core changes:

New script profiling/benchmarks/vlasov_benchmark.py. It runs a fixed number of time steps of a standard problem with VlasovAmpereOneSpecies or VlasovMaxwellOneSpecies, on the NumPy or the CuPy backend, and reports:

  • setup time, CUDA compile time, time per step (median and first step), diagnostics and output time;
  • time per propagator, and ns per marker and step;
  • host/device transfers in the time loop (cunumpy.profiling.count_transfers);
  • optionally all of it as JSON (--json), to compare runs.

Cases:

case models setup
landau both weak Landau damping as in the verification tests: box of length 4π (k = 0.5), 1e-3 density perturbation, control-variate weights, cubic splines in x
weibel VlasovMaxwell Weibel instability as in Kraus et al., J. Plasma Phys. 83 (2017), Sec. 6.2: box 2π/1.25, vth1 = vth3 = 0.02/√2, vth2 = √12 vth1, seed B3 = -1e-4 cos(kx)

All runs use alpha = 1, epsilon = -1, as the verification tests do. The initial magnetic energy of the Weibel seed matches the analytic value (1.2551e-8 vs 1.2566e-8). The growth rate was not checked, since that needs thousands of steps.

Sizes: smoke (8 cells × 50 ppc, 3 steps), standard (32 × 1000, 20 steps), large (128 × 20000 = 2.56M markers, 20 steps). Grid, degree, ppc, steps and dt can all be overridden.

Timing details:

  • Every timed region ends with xp.synchronize(), and times are the maximum over MPI ranks.
  • Per-propagator timing adds one synchronization per propagator and step; turn it off with --no-propagator-timing.
  • Sorting boxes are off by default, because of SortingBoxes cannot be created on the CuPy backend #726.
  • On the fake CuPy (CUNUMPY_FAKE_CUPY=1 ... --backend cupy --size smoke), the CUDA launches are emulated on the CPU. That checks the device code path without a GPU, but its times are not GPU timings, and the report says so.

CPU baseline (NumPy, --size standard: 32k markers, 20 steps, 1 rank, Apple M-series laptop):

run ms per step ns per marker and step VlasovAmpereCoupling share
VlasovAmpere, landau 67 2087 86 %
VlasovMaxwell, landau 82 2571 68 %
VlasovMaxwell, weibel 86 2678 68 %

Testing: all three cases at --size smoke and --size standard on NumPy; VlasovMaxwell weibel at --size smoke on the fake CuPy (all four propagators ran, with no host kernels and no host-to-device copies in the time loop). Not run on a GPU.

Documentation changes: the script's docstring (usage, cases, sizes, CPU reference).

🤖 Generated with Claude Code

@max-models
max-models marked this pull request as draft October 8, 2026 13:34
@max-models
max-models added this pull request to stack #729 October 12, 2026 12:10
@max-models
max-models force-pushed the gpu-2-vlasov-benchmark branch from a753483 to 71356ad Compare October 12, 2026 12:32
@max-models
max-models marked this pull request as ready for review October 12, 2026 19:05

@spossann spossann left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's use the scope-profiler instead.

max-models and others added 4 commits October 13, 2026 10:07
profiling/benchmarks/vlasov_benchmark.py runs a fixed number of time steps of a standard problem
on the NumPy or CuPy backend and reports setup time, time per step and per propagator, cost per
marker and step, and the host/device transfers of the time loop (optionally as JSON).

- Cases: weak Landau damping (both models, the verification tests' setup) and the Weibel
  instability (Vlasov-Maxwell, Kraus et al. 2017: anisotropic Maxwellian, B3 seed).
- Sizes smoke / standard / large (8 cells x 50 ppc, 32 x 1000, 128 x 20000), all overridable.
- Every timed region ends with xp.synchronize(); times are the maximum over MPI ranks.
- On the fake CuPy the CUDA launches are emulated, as a no-GPU check of the device code path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@max-models
max-models force-pushed the gpu-2-vlasov-benchmark branch from dc14651 to 0db6d8a Compare October 13, 2026 08:07
@max-models
max-models merged commit bbf500b into devel Oct 13, 2026
28 checks passed
@max-models
max-models deleted the gpu-2-vlasov-benchmark branch October 13, 2026 12:43
max-models added a commit that referenced this pull request Oct 13, 2026
Building multiple polar projector and mass-preconditioner instances
repeatedly factorizes identical matrices and creates duplicate device
copies. This change shares live 1D and poloidal factors across the polar
spaces of one Derham object.

**Stack:** based on `gpu-2-vlasov-benchmark` (#735), as the next PR in
the GPU stack.

The cache matches exact canonical sparse matrix contents and dtype,
solver kind and ILU settings, Schur pole masks/outer factors, and
backend/device. It reuses both host factorizations and their cached GPU
solves. Boundary-condition and transposed variants share only when their
matrices and configurations actually match. Updated mass weights obtain
new poloidal factors while retaining unchanged 1D factors.

Entries use weak references: live solvers keep their factors, while
obsolete factors and their keys are released when no solver uses them.
Different Derham objects have separate caches.

In the small-cylinder fixture, constructing all four solve variants for
P0–P3 reduces sparse LU calls from **88 to 9**, and dense Schur LU calls
from **24 to 8**.

Validation:

- Cache tests cover sparse storage normalization, exact value/dtype
matching, solver settings, Schur structure, transpose separation,
mass-weight updates, lifetime cleanup and isolation between Derham
objects/backends.
- Serial polar/cache/backend regression suite: **57 passed, 19
skipped**.
- Two MPI ranks, polar and cache tests: **50 passed, 1 skipped per
rank**.
- NVIDIA T1000, CuPy 14.2.0: **7 GPU cases passed**, including both
mappings and all three solvers; shared-device-factor reuse also passes
after the final ownership change.
- Reusing warmed GPU factors introduces no cunumpy host/device
transfers.
- Ruff lint/formatting, `git diff --check`, and commit hooks passed.
Assembly kernels compiled with GNU/Fortran for local verification.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants