Skip to content

Particles on domains with a pole (markers through the axis, polar splines in particle propagators) - #761

Merged
spossann merged 6 commits into
polar-profilingfrom
particle-pole
Oct 13, 2026
Merged

spossann merged 6 commits into
polar-profilingfrom
particle-pole

Conversation

@spossann

@spossann spossann commented Oct 12, 2026 •

Copy link
Copy Markdown
Member

Stacked on #759 (polar-profiling). It retargets to devel once #758/#759 are merged.

Summary

Particle markers can now run on domains with a pole at η1 = 0 (no hole around the axis), with polar splines for the FEEC parts. Examples are IGAPolarCylinder and IGAPolarTorus (HollowCylinder/HollowTorus with a1=0 also work in kernels). Before this PR, Vlasov and GuidingCenter failed on such domains: several propagators did not handle PolarVector, no marker boundary condition handled the axis, and the kernels divided by det DF = 0.

This prepares the Vlasov/GuidingCenter profiling examples (next PR, stacked on this one).

Marker push through the axis

  • New update_position_stage (pic/pushing/pusher_utilities_kernels.py): the stage update of the explicit Runge–Kutta pushers (PushEta, explicit GC bxE* and B*) is done in ξ = η1·(cos 2πη2, sin 2πη2), keeping η3.
    • In these coordinates the mapping and the marker velocity dξ/dt = (∂ξ/∂η)·dη/dt are regular at the pole, and markers move through the axis.
    • Straight lines through the axis of a cylinder are exact (tested to 1e-10).
    • Without a pole the update is unchanged.
  • Axis map in apply_kinetic_bc_marker (Pyccel and CUDA) and Particles.apply_kinetic_bc:
    • A marker with η1 < 0 is mapped to (−η1, η2 + ½), with unchanged velocities.
    • η1 is kept ≥ 1e-10, also after loading (Sobol loading starts at η = 0).
    • The boundary condition of axis 0 then applies at η1 = 1 only.
  • New field MarkerArguments.pole, also in CudaMarkerArguments and pusher_args.cuh.
  • Not supported yet: pole domains on the CuPy backend, and poc/pol_period ≠ 1. Both raise NotImplementedError.

Polar splines in particle propagators

  • PushVxB: the projected equilibrium b2 (already tensor-product) was extracted a second time, which failed with polar splines. Only the b2 variable is extracted now.
  • PushGuidingCenterBxEstar/Parallel: φ, the electric field and b̃ are passed to the kernels as tensor-product coefficients (PolarVector has no _data). The gradients use derham.grad, boundary operators included, followed by E^T. Without polar splines nothing changes.

Other fixes found along the way

  • Reflecting boundary: only Cartesian velocities (vdim = 3) are reflected. For Particles5D the columns 3:6 (v∥, μ, weight) were overwritten.
  • CanonicalMaxwellian2D: removed an unused read of domain.params["a1"], which raised a KeyError on IGAPolarTorus.
  • AdhocTorus, AdhocTorusQPsi: the ψ derivatives were 0/0 (NaN) on the magnetic axis. Off the axis they are unchanged (4e-16); on the axis they take the correct limits.
    • Note: AdhocTorusQPsi's interpolated ψ has ψ′(0) ≈ −6e-5 ≠ 0, so its field is not exactly regular on the axis. That is pre-existing and not changed here.
  • GuidingCenter: en_fB summed over holes and ghosts instead of the valid markers. Each removed marker added +1 to en_tot.

Disc loading under MPI (pre-existing bug, not pole-related)

With spatial="disc" on more than one process, markers were lost in the first push or the run hung:

  • Disc transform after the selection: pseudo_random loading selected the uniform draws on each process's domain and applied η1 → √η1 afterwards. That moved many markers to another process's domain.
    • Without sorting boxes they were not sent there, so the first push evaluated the fields outside the local domain.
    • Example: GuidingCenter on 4 processes removed 15% of the markers in the first step.
    • The transform is now applied before the domain check.
  • Sort after loading: markers are sent to their process even without sorting boxes (not for restart loading). Sobol loading draws on the whole domain on each process, so it needed this too.
  • Marker array size: the number of markers per process was estimated with the uniform measure. With the disc measure 2η1, processes at larger η1 overflowed the marker array: a ValueError on some processes and a hang on the others.
  • New test: test_draw_disc_on_own_process (pseudo_random and sobol_standard) in test_draw_parallel.py.

Known limitations

  • Discrete-gradient GC pushers: they only get the axis map. Their mid-points in η are not regular near the axis.
    • In the test (ε = 1), relative energy errors near the axis reach 1e-3 to 5e-2.
    • With the explicit RK4 pushers, markers that cross the axis conserve their energy to about 1e-6.
  • Buffer size: with few markers per process, the discrete-gradient pushers need a larger bufsize on 4 ranks (load-imbalance abort in the box sort). This is also the case on a torus with a hole, so it is not pole-related.

Tests

New file src/struphy/pic/tests/test_pole.py, which passes serially and on 4 MPI ranks:

  • the axis map in the kernel and in Python, including a marker with η1 > 1;
  • reflection for vdim = 2 and 3;
  • PushEta straight lines through the axis (forward Euler, Heun2, RK4) against the exact Cartesian end points;
  • Vlasov on IGAPolarCylinder with ScrewPinch and polar splines: kinetic energy conserved to 1e-12;
  • GuidingCenter on IGAPolarTorus with AdhocTorus and polar splines, explicit and discrete-gradient pushers: energies per marker, compared by ID, including markers that cross the axis.

Also run locally:

  • test_pushers.py, test_reflect.py, test_kernel_setup.py, test_kernel_backends.py, test_accum_gc_gradB.py, test_maxwellians.py and fields_background/tests (serial);
  • the default-params runs of Vlasov and GuidingCenter.

A separate review agent checked the branch before the push. Its findings are fixed in the second commit:

  • the Python axis map caught η1 > 1;
  • the clamp covered only one loading branch;
  • the GC gradients had dropped the boundary operators.

🤖 Generated with Claude Code

Follow-up (stacked): #762 adds the Vlasov/GuidingCenter profiling examples that use this.

spossann and others added 4 commits October 12, 2026 20:35
Markers can now move through the axis of a domain with a pole (e.g. IGAPolarCylinder,
IGAPolarTorus, HollowCylinder/HollowTorus with a1=0), with polar splines for the FEEC parts.

- update_position_stage (pusher_utilities_kernels): the stage update of the explicit
  Runge-Kutta pushers (PushEta, GC bxE* and B* explicit) is done in the coordinates
  xi = eta1 (cos 2 pi eta2, sin 2 pi eta2), in which the mapping and the marker velocity
  are regular at the pole. Straight lines through the axis of a cylinder are exact.
- Axis map in apply_kinetic_bc_marker (Pyccel and CUDA) and Particles.apply_kinetic_bc:
  a marker with eta1 < 0 is mapped to (-eta1, eta2 + 1/2); eta1 is kept >= 1e-10, also at
  loading. The discrete-gradient GC pushers only get this map (mid-points in eta are not
  regular at the axis).
- New field MarkerArguments.pole (and CudaMarkerArguments, pusher_args.cuh). Pole domains
  are not supported on the CuPy backend yet, nor with poc/pol_period != 1.
- PushVxB: the projected equilibrium b2 was extracted twice (assertion with polar splines);
  only the b2 variable is extracted now.
- PushGuidingCenterBxEstar/Parallel: phi, the electric field and b_tilde are passed to the
  kernels as tensor-product coefficients (PolarVector has no _data).
- Reflecting boundary: only Cartesian velocities (vdim=3) are reflected; for Particles5D the
  columns 3:6 (v_parallel, mu, weight) were overwritten.
- CanonicalMaxwellian2D: removed the unused read of domain.params["a1"] (KeyError on
  IGAPolarTorus).
- AdhocTorus, AdhocTorusQPsi: psi derivatives are regular on the magnetic axis (were 0/0).
- GuidingCenter: the magnetic energy en_fB summed over holes and ghosts instead of the valid
  markers (each removed marker added 1 to en_tot).
- Tests in pic/tests/test_pole.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Particles.apply_kinetic_bc: the axis map took markers outside on both sides of eta1;
  a marker with eta1 > 1 was moved to the axis instead of getting the boundary condition
  at eta1=1. Only eta1 < 0 is mapped now (test added).
- The eta1 clamp at the pole is applied after all loading branches (external, restart, ...).
- GC propagators: the gradients are again taken with derham.grad (including the boundary
  operators) and then extracted to the tensor-product space, unchanged without polar
  splines. The GC pole test evaluates the electric field.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rge enough

With spatial="disc" and more than one process, markers were lost or the run hung (not related to the pole):

- pseudo_random loading kept the uniform draws on the domain of the process and applied eta1 -> sqrt(eta1)
  afterwards, which moved many markers to the domain of another process. Without sorting boxes they were
  not sent there, and the first push evaluated the fields outside the local domain (e.g. 15% of the
  GuidingCenter markers were removed in the first step on 4 processes). The transform is now applied
  before the domain check.
- After loading, markers are sent to their process also without sorting boxes (e.g. Sobol loading draws
  on the whole domain on each process).
- The number of markers per process at loading was estimated from the uniform measure; with the disc
  measure 2*eta1, processes at larger eta1 overflowed the marker array (ValueError on some processes,
  hang on the others). The estimate uses the disc measure now.
- Test test_draw_disc_on_own_process (pseudo_random and sobol_standard).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The sort after loading only runs without sorting boxes and not for restart loading (the holes of the
  restart markers array are not updated at that point).
- At least one marker per process in the disc estimate (avoids a zero-size estimate near the axis).
- Test: reduce before asserting (no deadlock if one process fails); moved above the main block.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
spossann and others added 2 commits October 13, 2026 08:34
The pole argument added to MarkerArguments/CudaMarkerArguments was missing
in marker_arguments() (kernel_test_args.py) and make_arguments()
(test_kernel_backends.py), which broke the CUDA emulation parity tests and
the fake-CuPy accumulator test on CI.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@spossann
spossann requested a review from max-models October 13, 2026 14:30
@spossann
spossann merged commit b924e04 into devel Oct 13, 2026
28 checks passed
@spossann
spossann deleted the particle-pole branch October 13, 2026 15:35
spossann added a commit that referenced this pull request Oct 13, 2026
…ingCenter (cylinder/torus without hole) (#762)

Stacked on #761 (`particle-pole`, which is stacked on #759). It
retargets once those are merged.

## Summary
New profiling examples for the particle-tracing models **Vlasov** and
**GuidingCenter**. They are long-time equilibrium tests with binning
plots in position and velocity space, and the propagator algorithms are
compared through variants.

The layout is the same as for the existing cases (#759):
- one params file plus one submit script per model and geometry;
- the variant is chosen with `--variant`;
- rank counts 1, 2, 4, …, 256;
- post-processing writes `.png`/`.npy` files to `sim_<id>/results`.

### Cases
- **`Vlasov/cuboid_strong_scaling`**
- Setup: Maxwellian3D with constant moments (n=1, u=0, vth=1) in a
periodic cube (Cuboid, side 2π) with a uniform B along z (HomogenSlab).
This is an exact equilibrium.
- Binning: η1, (η1, η2), v1, v3 and (v1, v2), compared with the initial
and the exact profiles.
- Variants: push_vxb analytic/implicit; push_eta forward
Euler/Heun2/Heun3/RK4; Strang splitting; Sobol loading.
- **`GuidingCenter/polar_cylinder_strong_scaling`**
- Geometry: straight cylinder **without hole** (IGAPolarCylinder, C1
polar splines) with ScrewPinch (Lz = 2πR0, pressure offset p0 so that
vth > 0 at the edge).
- Initial condition: GyroMaxwellian2D with n(r) and vth(r) = √(p/n) from
the equilibrium. It is an exact equilibrium, because the GC drifts keep
r constant. Markers move through the axis.
- Binning: η1, (η1, η2), v∥, μ and (η1, v∥). The radial profile is also
compared with the exact one, n/|B0|.
- **`GuidingCenter/polar_torus_strong_scaling`**
- Geometry: full torus **without hole** (IGAPolarTorus, polar splines)
with AdhocTorus.
- Initial condition: CanonicalMaxwellian2D with n and vth as functions
of ψc (an exact equilibrium). The density vanishes at the edge, where
markers on orbits that leave the plasma are removed.
- Control: a GyroMaxwellian2D with the same profiles in the flux label
of the position. It is not an equilibrium in a torus.
- **GuidingCenter variants:** discrete gradient 1st order (fixed point,
Newton), 2nd order, explicit RK4 and Heun3. The torus adds
GyroMaxwellian controls.
- Default grid: 16×48×48, decomposed only in η2/η3, so 1–256 processes
work. ppc 25, dt 0.2, T = 100.
- **`profiling/examples/particle_diagnostics.py`:** shared
post-processing.
- Binning plots at the first and last time (2D slices also as difference
maps).
  - Relative L2 deviation from t=0 and from the exact profile over time.
- Relative drift of `kinetic_energy`/`en_tot`; scalars that start at 0,
such as lost markers, are plotted as absolute values.

### Notes
- **No hole at r = 0:** a Simulation needs polar splines whenever the
domain has a pole, and polar splines need a spline mapping. So the cases
use IGAPolarCylinder/IGAPolarTorus. HollowCylinder/HollowTorus with a1=0
cannot be used.
- **Why not CircularTokamak for the cylinder:** it is a toroidal field
(B_φ ∝ 1/R, circular flux surfaces around R = R0), and its ψ is not
force-balanced: Δ*ψ ≠ 0 with constant p and g. Adding a pressure p(ψ)
cannot balance it. The force-balanced ScrewPinch is used instead.
- **Discrete-gradient GC pushers near the axis:** they are less accurate
there. They only get the axis map from #761, not the regular ξ update.
- **Marker loss on several processes:** with disc loading, the first
test runs lost markers on several processes. That is fixed in #761 (disc
loading under MPI).

## Checks
- Tiny smoke runs (2 and 4 processes) of all cases complete and write
the results.
- On the cylinder with 4 processes (discrete gradient and explicit), the
radial profile changes by about 1e-6 per step, no marker is lost and
`en_tot` is constant.
- Vlasov: `kinetic_energy` is constant and the v3 distribution is
unchanged to 1e-16.
- The flags of all 19 variants are accepted by the params files.
- Two separate review agents checked the examples. Their findings are
fixed:
  - the decomposition at 128/256 processes;
  - the marker loss under MPI (in #761);
  - the plot of scalars that start at zero;
  - the edge losses in the torus;
  - the exact radial profile;
  - the B0 used for loading.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants