Skip to content

Profiling examples: long-time particle equilibria for Vlasov and GuidingCenter (cylinder/torus without hole) - #762

Merged
spossann merged 5 commits into
particle-polefrom
particle-profiling
Oct 13, 2026
Merged

spossann merged 5 commits into
particle-polefrom
particle-profiling

Conversation

@spossann

Copy link
Copy Markdown
Member

Stacked on #761 (particle-pole, which is stacked on #759). It retargets once those are merged.

Summary

New profiling examples for the particle-tracing models Vlasov and GuidingCenter. They are long-time equilibrium tests with binning plots in position and velocity space, and the propagator algorithms are compared through variants.

The layout is the same as for the existing cases (#759):

  • one params file plus one submit script per model and geometry;
  • the variant is chosen with --variant;
  • rank counts 1, 2, 4, …, 256;
  • post-processing writes .png/.npy files to sim_<id>/results.

Cases

  • Vlasov/cuboid_strong_scaling
    • Setup: Maxwellian3D with constant moments (n=1, u=0, vth=1) in a periodic cube (Cuboid, side 2π) with a uniform B along z (HomogenSlab). This is an exact equilibrium.
    • Binning: η1, (η1, η2), v1, v3 and (v1, v2), compared with the initial and the exact profiles.
    • Variants: push_vxb analytic/implicit; push_eta forward Euler/Heun2/Heun3/RK4; Strang splitting; Sobol loading.
  • GuidingCenter/polar_cylinder_strong_scaling
    • Geometry: straight cylinder without hole (IGAPolarCylinder, C1 polar splines) with ScrewPinch (Lz = 2πR0, pressure offset p0 so that vth > 0 at the edge).
    • Initial condition: GyroMaxwellian2D with n(r) and vth(r) = √(p/n) from the equilibrium. It is an exact equilibrium, because the GC drifts keep r constant. Markers move through the axis.
    • Binning: η1, (η1, η2), v∥, μ and (η1, v∥). The radial profile is also compared with the exact one, n/|B0|.
  • GuidingCenter/polar_torus_strong_scaling
    • Geometry: full torus without hole (IGAPolarTorus, polar splines) with AdhocTorus.
    • Initial condition: CanonicalMaxwellian2D with n and vth as functions of ψc (an exact equilibrium). The density vanishes at the edge, where markers on orbits that leave the plasma are removed.
    • Control: a GyroMaxwellian2D with the same profiles in the flux label of the position. It is not an equilibrium in a torus.
  • GuidingCenter variants: discrete gradient 1st order (fixed point, Newton), 2nd order, explicit RK4 and Heun3. The torus adds GyroMaxwellian controls.
    • Default grid: 16×48×48, decomposed only in η2/η3, so 1–256 processes work. ppc 25, dt 0.2, T = 100.
  • profiling/examples/particle_diagnostics.py: shared post-processing.
    • Binning plots at the first and last time (2D slices also as difference maps).
    • Relative L2 deviation from t=0 and from the exact profile over time.
    • Relative drift of kinetic_energy/en_tot; scalars that start at 0, such as lost markers, are plotted as absolute values.

Notes

Checks

  • Tiny smoke runs (2 and 4 processes) of all cases complete and write the results.
  • On the cylinder with 4 processes (discrete gradient and explicit), the radial profile changes by about 1e-6 per step, no marker is lost and en_tot is constant.
  • Vlasov: kinetic_energy is constant and the v3 distribution is unchanged to 1e-16.
  • The flags of all 19 variants are accepted by the params files.
  • Two separate review agents checked the examples. Their findings are fixed:

🤖 Generated with Claude Code

spossann and others added 4 commits October 13, 2026 09:28
…ingCenter

One params file and one submit script per model and geometry (variants via --variant, ranks 1 to 256),
as for the existing profiling cases:

- Vlasov/cuboid_strong_scaling: Maxwellian3D with constant moments in a periodic cube (Cuboid, side 2 pi)
  with a uniform magnetic field (HomogenSlab), an exact equilibrium. Binned distributions in position
  (e1, e1_e2) and velocity (v1, v3, v1_v2) are compared with their initial and exact profiles.
  Variants: push_vxb analytic/implicit, push_eta forward Euler/Heun2/Heun3/RK4, Strang splitting,
  Sobol loading.
- GuidingCenter/polar_cylinder_strong_scaling: straight cylinder without hole (IGAPolarCylinder, polar
  splines) with ScrewPinch (Lz = 2 pi R0, pressure offset); GyroMaxwellian2D with n(r) and
  vth(r) = sqrt(p/n) of the equilibrium, an exact equilibrium (the drifts keep r constant).
- GuidingCenter/polar_torus_strong_scaling: full torus without hole (IGAPolarTorus, polar splines) with
  AdhocTorus; CanonicalMaxwellian2D with n and vth functions of psi_c (an exact equilibrium), and a
  GyroMaxwellian2D with the same profiles in the flux label of the position as a control.
- GuidingCenter variants: discrete gradient 1st order (fixed point, Newton), 2nd order, explicit RK4/Heun3.
- profiling/examples/particle_diagnostics.py: shared post-processing (binning plots at the first and last
  time, relative deviations over time, drift of the scalars) into sim_<id>/results.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- GuidingCenter: default grid 16x48x48 decomposed only in eta2 and eta3 (mpi_dims_mask=(False, True, True)),
  such that 1 to 256 processes work (at least degree elements per process, all polar rings on the process
  with the pole); 16x18x16 could not be decomposed onto 128 and 256 processes. ppc 25, dt 0.2, Tend 100.
- GuidingCenter torus: the density vanishes at the edge (markers on orbits that leave the plasma are
  removed), loading with B0 = B_min.
- GuidingCenter cylinder: the exact radial profile of the binning is n(r)/|B0|(r) (velocity-space Jacobian).
- particle_diagnostics: scalars that start at zero (number of lost markers) are plotted as absolute values.
- The marker loss in the first step on several processes (disc loading) is fixed on particle-pole (merged).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each submit_*.py now lives in the example folder of the params file it
launches, e.g. profiling/examples/GuidingCenter/polar_cylinder_strong_scaling/.
The sys.path insert goes one level further up (parents[3]) and params_source
is taken from the script's own directory. The ToyGyrokinetic path in the
profiling-clusters workflow is updated accordingly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@spossann
spossann requested a review from max-models October 13, 2026 14:30
@spossann
spossann merged commit f34d72e into devel Oct 13, 2026
28 checks passed
@spossann
spossann deleted the particle-profiling branch October 13, 2026 15:35
spossann added a commit that referenced this pull request Oct 13, 2026
…ines in particle propagators) (#761)

Stacked on #759 (`polar-profiling`). It retargets to `devel` once
#758/#759 are merged.

## Summary

Particle markers can now run on domains with a **pole at η1 = 0** (no
hole around the axis), with polar splines for the FEEC parts. Examples
are `IGAPolarCylinder` and `IGAPolarTorus` (HollowCylinder/HollowTorus
with `a1=0` also work in kernels). Before this PR, `Vlasov` and
`GuidingCenter` failed on such domains: several propagators did not
handle `PolarVector`, no marker boundary condition handled the axis, and
the kernels divided by det DF = 0.

This prepares the Vlasov/GuidingCenter profiling examples (next PR,
stacked on this one).

### Marker push through the axis
- **New `update_position_stage`**
(`pic/pushing/pusher_utilities_kernels.py`): the stage update of the
explicit Runge–Kutta pushers (PushEta, explicit GC bxE\* and B\*) is
done in ξ = η1·(cos 2πη2, sin 2πη2), keeping η3.
- In these coordinates the mapping and the marker velocity dξ/dt =
(∂ξ/∂η)·dη/dt are regular at the pole, and markers move through the
axis.
- Straight lines through the axis of a cylinder are exact (tested to
1e-10).
  - Without a pole the update is unchanged.
- **Axis map** in `apply_kinetic_bc_marker` (Pyccel and CUDA) and
`Particles.apply_kinetic_bc`:
- A marker with η1 < 0 is mapped to (−η1, η2 + ½), with unchanged
velocities.
- η1 is kept ≥ 1e-10, also after loading (Sobol loading starts at η =
0).
  - The boundary condition of axis 0 then applies at η1 = 1 only.
- **New field `MarkerArguments.pole`**, also in `CudaMarkerArguments`
and `pusher_args.cuh`.
- **Not supported yet:** pole domains on the CuPy backend, and
`poc`/`pol_period` ≠ 1. Both raise `NotImplementedError`.

### Polar splines in particle propagators
- **PushVxB:** the projected equilibrium `b2` (already tensor-product)
was extracted a second time, which failed with polar splines. Only the
`b2` variable is extracted now.
- **PushGuidingCenterBxEstar/Parallel:** φ, the electric field and b̃
are passed to the kernels as tensor-product coefficients (`PolarVector`
has no `_data`). The gradients use `derham.grad`, boundary operators
included, followed by E^T. Without polar splines nothing changes.

### Other fixes found along the way
- **Reflecting boundary:** only Cartesian velocities (vdim = 3) are
reflected. For Particles5D the columns 3:6 (v∥, μ, **weight**) were
overwritten.
- **`CanonicalMaxwellian2D`:** removed an unused read of
`domain.params["a1"]`, which raised a KeyError on IGAPolarTorus.
- **`AdhocTorus`, `AdhocTorusQPsi`:** the ψ derivatives were 0/0 (NaN)
on the magnetic axis. Off the axis they are unchanged (4e-16); on the
axis they take the correct limits.
- Note: AdhocTorusQPsi's interpolated ψ has ψ′(0) ≈ −6e-5 ≠ 0, so its
field is not exactly regular on the axis. That is pre-existing and not
changed here.
- **`GuidingCenter`:** `en_fB` summed over holes and ghosts instead of
the valid markers. Each removed marker added +1 to `en_tot`.

### Disc loading under MPI (pre-existing bug, not pole-related)
With `spatial="disc"` on more than one process, markers were lost in the
first push or the run hung:
- **Disc transform after the selection:** `pseudo_random` loading
selected the uniform draws on each process's domain and applied η1 → √η1
afterwards. That moved many markers to another process's domain.
- Without sorting boxes they were not sent there, so the first push
evaluated the fields outside the local domain.
- Example: GuidingCenter on 4 processes removed 15% of the markers in
the first step.
  - The transform is now applied before the domain check.
- **Sort after loading:** markers are sent to their process even without
sorting boxes (not for restart loading). Sobol loading draws on the
whole domain on each process, so it needed this too.
- **Marker array size:** the number of markers per process was estimated
with the uniform measure. With the disc measure 2η1, processes at larger
η1 overflowed the marker array: a ValueError on some processes and a
hang on the others.
- **New test:** `test_draw_disc_on_own_process` (pseudo_random and
sobol_standard) in `test_draw_parallel.py`.

### Known limitations
- **Discrete-gradient GC pushers:** they only get the axis map. Their
mid-points in η are not regular near the axis.
- In the test (ε = 1), relative energy errors near the axis reach 1e-3
to 5e-2.
- With the explicit RK4 pushers, markers that cross the axis conserve
their energy to about 1e-6.
- **Buffer size:** with few markers per process, the discrete-gradient
pushers need a larger `bufsize` on 4 ranks (load-imbalance abort in the
box sort). This is also the case on a torus with a hole, so it is not
pole-related.

## Tests
New file `src/struphy/pic/tests/test_pole.py`, which passes serially and
on 4 MPI ranks:
- the axis map in the kernel and in Python, including a marker with η1 >
1;
- reflection for vdim = 2 and 3;
- PushEta straight lines through the axis (forward Euler, Heun2, RK4)
against the exact Cartesian end points;
- Vlasov on IGAPolarCylinder with ScrewPinch and polar splines: kinetic
energy conserved to 1e-12;
- GuidingCenter on IGAPolarTorus with AdhocTorus and polar splines,
explicit and discrete-gradient pushers: energies per marker, compared by
ID, including markers that cross the axis.

Also run locally:
- `test_pushers.py`, `test_reflect.py`, `test_kernel_setup.py`,
`test_kernel_backends.py`, `test_accum_gc_gradB.py`,
`test_maxwellians.py` and `fields_background/tests` (serial);
- the default-params runs of Vlasov and GuidingCenter.

A separate review agent checked the branch before the push. Its findings
are fixed in the second commit:
- the Python axis map caught η1 > 1;
- the clamp covered only one loading branch;
- the GC gradients had dropped the boundary operators.

🤖 Generated with [Claude Code](https://claude.com/claude-code)


Follow-up (stacked): #762 adds the Vlasov/GuidingCenter profiling
examples that use this.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants