Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
108 commits
Select commit Hold shift + click to select a range
3db1485
Port the Julia backend experiment to main [skip ci]
jtravs Nov 6, 2026
d46a85a
Julia: read the split collar's sums and the transverse edge [skip ci]
jtravs Nov 6, 2026
63e02bf
Julia: one source for CPU and GPU, any precision [skip ci]
jtravs Nov 6, 2026
5f64f1a
Julia: one CPU pool, measured plans, modal-product Raman [skip ci]
jtravs Nov 6, 2026
647f362
Julia: a CUDA extension and its environment [skip ci]
jtravs Nov 6, 2026
8226186
Julia: name the checkout for an installed solprop [skip ci]
jtravs Nov 6, 2026
b6b6d05
Julia: monitor reductions in the engine, zero-copy views, precompiled…
jtravs Nov 6, 2026
8ab6c4b
Julia: hold host BLAS to one thread; kernels capture values, not type…
jtravs Nov 6, 2026
5d8de46
Julia: edge energies in the engine, a GPU-safety check, tests [skip ci]
jtravs Nov 6, 2026
be38ebc
Julia: native's phi-function evaluation; edge rows bound once [skip ci]
jtravs Nov 6, 2026
cade2f1
Julia: device-synchronised section timers; profile whole timed runs […
jtravs Nov 6, 2026
6615562
Julia: plasma a sample a thread on GPUs; warm every path at construct…
jtravs Nov 6, 2026
3e2290c
Julia: fix the warm-up's out-of-range reads; warm the observers and t…
jtravs Nov 6, 2026
462908b
Julia: drop the vector exp/log experiment [skip ci]
jtravs Nov 6, 2026
aa6b947
Julia: observation populations a sample a thread on GPUs [skip ci]
jtravs Nov 6, 2026
e026295
parallel_rows: check the pass's size before reading the thread budget…
jtravs Nov 6, 2026
90840a4
Julia: pool copies across the bridge; edge synthesis on the pool; no …
jtravs Nov 6, 2026
cba0209
Julia: binary32 (SEP 33) from the same source [skip ci]
jtravs Nov 6, 2026
367559c
Julia: block exponent for the Raman kernel; docs, status and manual p…
jtravs Nov 6, 2026
348a2d1
Julia: frequency-domain synthesis where nodes match modes; real-view …
jtravs Nov 6, 2026
8a408b3
Merge remote-tracking branch 'origin/main' into feature/julia-backend…
jtravs Nov 6, 2026
e28b730
Julia: lint, types, and the registry test's backend list [skip ci]
jtravs Nov 6, 2026
3969a00
benchmarks: the run's record before the clock starts [skip ci]
jtravs Nov 6, 2026
6d451ca
Julia: Poynting statistics' transforms and observation populations in…
jtravs Nov 6, 2026
d2d2036
Julia tests: the windows' statistics against the reference's [skip ci]
jtravs Nov 6, 2026
09a60bf
Julia: its own helpers record, and a trial's tables formed in the eng…
jtravs Nov 6, 2026
a8c17f4
Julia: Poynting buffers and plan held by the engine, warmed at constr…
jtravs Nov 6, 2026
9ce6108
Julia: stage scalars read by the engine from its own tables [skip ci]
jtravs Nov 6, 2026
8b486e1
Julia: a plasma column's live samples gathered before their rates [sk…
jtravs Nov 6, 2026
db370cf
Julia: vectorised exp and log for the CPU's plasma columns [skip ci]
jtravs Nov 6, 2026
c287613
Julia: STATUS and README for the engine-side tables, statistics and p…
jtravs Nov 6, 2026
7c7ac9b
Julia: a source's forward transforms, and the alarm's, as one pool's …
jtravs Nov 6, 2026
207add1
Julia: the exponential Adams march executed by the engine [skip ci]
jtravs Nov 7, 2026
355b024
Julia: the final A/B rows, the march, and the findings' issues in the…
jtravs Nov 7, 2026
9d0180e
Julia: an AMDGPU extension and its environment [skip ci]
jtravs Nov 7, 2026
bd021ce
Julia: radial and free planes' local statistics in the engine [skip ci]
jtravs Nov 7, 2026
cfa14ce
Julia: GPU heavy reductions a thread an element, and transforms in ba…
jtravs Nov 7, 2026
6f22b54
Julia: a plasma column's constant head and tail take one fraction eac…
jtravs Nov 7, 2026
70d01bf
Julia: state-sized copies as the pool's tasks [skip ci]
jtravs Nov 7, 2026
07c8224
Julia: a heavy pass splits from a few hundred elements [skip ci]
jtravs Nov 7, 2026
4b5bcd4
Julia: products with a thin side as a loop over the pool, not BLAS [s…
jtravs Nov 7, 2026
196baa5
Julia: the runtime loads the named GPU's package, AMDGPU for rocm [sk…
jtravs Nov 7, 2026
8a6d184
Julia: the collar's half-step factor once a sample, not once an eleme…
jtravs Nov 7, 2026
d4ca182
Julia: a GPU integral's filter and trapezoid increments in one kernel…
jtravs Nov 7, 2026
231dc9a
Julia: a varying guide's power weight rebound alone, compiled at cons…
jtravs Nov 7, 2026
50158fa
Julia: GPU column and row sums, local sums and occupancy by device re…
jtravs Nov 7, 2026
daef338
Julia: a GPU's transform staging held in the engine's workspace, not …
jtravs Nov 7, 2026
48809a8
Julia: the latest CPU, A40 and MI325X rows in the ledger and STATUS […
jtravs Nov 7, 2026
6826fcd
Julia: a plasma integral's interior FIR in one pass [skip ci]
jtravs Nov 7, 2026
a8221ef
Julia: the live rates' knot and offset in the vectorised pass [skip ci]
jtravs Nov 7, 2026
710d080
Julia: --cuda-local, CUDA.jl on the system toolkit beside native [ski…
jtravs Nov 7, 2026
3dabae7
Julia: the H200 and the latest A40 rows [skip ci]
jtravs Nov 7, 2026
4d8c529
Julia: the engine's Adams march with a split collar [skip ci]
jtravs Nov 7, 2026
3fd4a09
Julia: a GPU's scratch arrays kept by the engine, not allocated a cal…
jtravs Nov 7, 2026
5f5efcf
Julia: the A40 rows at 50158fa6 and 3fd4a09f [skip ci]
jtravs Nov 7, 2026
a672202
Julia: a timed run's first window compiles (almost) nothing [skip ci]
jtravs Nov 7, 2026
d437496
Julia: the clean CPU A/B at a6722029 and the A40's binary64 [skip ci]
jtravs Nov 7, 2026
a108ae5
Julia: the A40's binary32 rerun at a6722029 [skip ci]
jtravs Nov 7, 2026
5264734
Julia: the benchmark windows' specialisations precompiled into the pa…
jtravs Nov 7, 2026
3933d4f
Julia: a handoff note for the Mac (a CPU comparison, then a Metal ext…
jtravs Nov 7, 2026
52b9a45
Julia: a Metal extension, with double-float for the operator's expone…
jtravs Nov 7, 2026
ecbcafb
Julia: line lengths in the Metal changes [skip ci]
jtravs Nov 7, 2026
3f0d33d
Julia: contraction in ordinary kernels; a flush-to-zero experiment sw…
jtravs Nov 7, 2026
e99851d
Julia: SOLPROP_JULIA_BLAS=accelerate, Apple's BLAS for the products (…
jtravs Nov 7, 2026
d872cb5
Julia: Accelerate by default on macOS; benchmarks ab --cooldown [skip…
jtravs Nov 7, 2026
6554012
Julia: the M5 Max, Metal, floating-point and A40 rows; toward a first…
jtravs Nov 7, 2026
be94de6
Julia: the Mac's drift corrected in the records; the cooldown A/B [sk…
jtravs Nov 7, 2026
19215f0
Julia: a GPU's strided product operands copied into scratch first [sk…
jtravs Nov 7, 2026
b60663f
Julia: Metal's wait costs in STATUS [skip ci]
jtravs Nov 7, 2026
4614c71
Julia: the manual page, README and STATUS brought up to date for the …
jtravs Nov 7, 2026
da5559f
Julia: launch and wait counters; a GPU's constant projection rows kep…
jtravs Nov 7, 2026
37aed94
Julia: device copies as one kernel, views included [skip ci]
jtravs Nov 7, 2026
5df088e
Julia: one value the host waits for a GPU step [skip ci]
jtravs Nov 7, 2026
a38097f
Julia: one chunk of nodes a source on a GPU [skip ci]
jtravs Nov 7, 2026
fbd75c2
Julia: a GPU trial's tables in one upload [skip ci]
jtravs Nov 7, 2026
7669f5d
Julia: the Metal tuning rows and the A40 rows at 5df088ec [skip ci]
jtravs Nov 7, 2026
0686c8b
Julia: a GPU's population statistics in one readback a chunk [skip ci]
jtravs Nov 7, 2026
9af3cd5
Julia: the population statistics' reductions parallel, not a thread a…
jtravs Nov 7, 2026
0d84bb7
Julia: the A40's Nsight profiles in the ledger [skip ci]
jtravs Nov 7, 2026
8d76f04
Julia: the A40 profile's caveat (native's sources run in CUDA graphs)…
jtravs Nov 7, 2026
aa5d654
Julia: the analytic signal from the spectrum, as native forms it [ski…
jtravs Nov 7, 2026
0f2c184
Julia: a GPU source's forward streams packed into one transform [skip…
jtravs Nov 7, 2026
9551c9a
Julia: a GPU free-space source's polarisation and currents in one tra…
jtravs Nov 7, 2026
970b5d1
Julia: lockstep batches, TSIT5 and modal RK4IP [skip ci]
jtravs Nov 7, 2026
b0155d2
Julia: several Raman components, summed into one kernel [skip ci]
jtravs Nov 7, 2026
e316b7b
Julia: tapered guides [skip ci]
jtravs Nov 7, 2026
96c3e21
Julia: the window statistics and the zero-dispersion wavelength [skip…
jtravs Nov 7, 2026
7c85fc7
Julia: resolved and radial envelopes, and the envelope's third harmon…
jtravs Nov 7, 2026
82dcffe
Julia: Raman in free space [skip ci]
jtravs Nov 7, 2026
9d81497
Julia: a GPU state copied to the host once a state [skip ci]
jtravs Nov 7, 2026
1d1abc7
Julia: the free-space operator's tables on every GPU [skip ci]
jtravs Nov 7, 2026
a03f1d8
Julia: several plasma channels [skip ci]
jtravs Nov 7, 2026
f262af8
Julia: STATUS brought up to date (Metal tuning, eleven features, A40 …
jtravs Nov 7, 2026
137c691
Julia: the fused Kerr pass back in free space without plasma or Raman…
jtravs Nov 7, 2026
a23cfd3
Julia experiment: varying Raman, Cartesian Adams, envelope plasma [sk…
jtravs Nov 8, 2026
6d697ad
Julia experiment: the A40 at 137c6915; STATUS for the new features [s…
jtravs Nov 8, 2026
51ef674
Julia experiment: on-axis plasma statistics; envelope plasma converge…
jtravs Nov 8, 2026
db522ea
Julia experiment: four-step long FFTs and transposed transverse plane…
jtravs Nov 8, 2026
8765b58
Julia experiment: inline kernel bodies on GPUs; per-kernel timer [ski…
jtravs Nov 8, 2026
67491bf
Julia experiment: tiled device transposes, kept reduction workspaces …
jtravs Nov 8, 2026
5421d04
Julia experiment: the A40 at 8765b584 and the H200 at 67491bf9 in the…
jtravs Nov 8, 2026
51a153d
Julia experiment: source evaluations as CUDA graphs, stage scalars on…
jtravs Nov 8, 2026
e3b0726
Julia experiment: large states copied directly to and from the host […
jtravs Nov 8, 2026
1e609d6
Ledger: the order-independent Adams trial graph, tried and reverted […
jtravs Nov 8, 2026
1b17fc7
Julia experiment: fused device reductions; kept staging for per-trial…
jtravs Nov 8, 2026
e8921b4
Julia experiment: AK reductions restored; MI300X and step timelines r…
jtravs Nov 9, 2026
6f686ea
Julia experiment: a saved plane's host time on the A40, in the ledger…
jtravs Nov 9, 2026
b84d3ab
Julia experiment: the H100 at 6f686ea4 (1.25 against native, binary64…
jtravs Nov 9, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -50,3 +50,8 @@ docs/introduction/lessons/searchindex
.benchmarks/
*.profraw
*.profdata
.julia-runtime/
.julia-depot/
julia/LocalPreferences.toml
julia/cuda/LocalPreferences.toml
!julia/metal/LocalPreferences.toml
36 changes: 31 additions & 5 deletions benchmarks/ab.py
Original file line number Diff line number Diff line change
Expand Up @@ -77,14 +77,34 @@ def ab_options(parser: argparse.ArgumentParser) -> None:
choices=BACKENDS,
help="the candidate arm's backend; default: --backend",
)
parser.add_argument(
"--cooldown",
type=float,
default=0.0,
help="seconds to idle before each worker, so that one worker's heat (a Julia "
"worker's compilation) does not slow the next",
)
parser.add_argument(
"--device-arm",
choices=("baseline", "candidate"),
help="run only this arm on its device (a GPU against a CPU build); the other "
"runs on the host at the row's width",
)


def arm_configuration(
configuration: Record, backend: str, device: bool | None = None
) -> Record:
"""Return one arm's configuration: its backend and that backend's precision.

def arm_configuration(configuration: Record, backend: str) -> Record:
"""Return one arm's configuration: its backend and that backend's precision."""
`device`, when given, overrides the row's device flag for this arm alone.
"""
return {
**configuration,
"backend": backend,
"precision": "single" if backend == "mlx" else "double",
# "julia32" is the julia backend in single precision (the Julia experiment).
"backend": "julia" if backend == "julia32" else backend,
"precision": "single" if backend in ("mlx", "julia32") else "double",
**({} if device is None else {"device": device}),
}


Expand Down Expand Up @@ -203,6 +223,8 @@ def ab(args: argparse.Namespace) -> int:
"""Collect and summarise a paired comparison of two prepared builds."""
if args.rounds < 1 or args.repeats < 1 or args.timeout <= 0:
raise SuiteError("rounds, repeats and timeout must be positive")
if args.device_arm is not None and args.device:
raise SuiteError("--device runs both arms on a device; --device-arm one")
started = time.perf_counter()
backends = {
"baseline": args.baseline_backend or args.backend,
Expand Down Expand Up @@ -260,12 +282,16 @@ def ab(args: argparse.Namespace) -> int:
for row in shuffled:
width = row["configuration"]["threads"]
for arm in arm_order(round_index):
if args.cooldown > 0:
time.sleep(args.cooldown)
quiet = guard(allocation)["quiet"]
events, elapsed = launch_worker(
pythons[arm],
{
"configuration": arm_configuration(
row["configuration"], backends[arm]
row["configuration"],
backends[arm],
None if args.device_arm is None else arm == args.device_arm,
),
"repeats": args.repeats,
"fields": str(
Expand Down
6 changes: 4 additions & 2 deletions benchmarks/registry.py
Original file line number Diff line number Diff line change
Expand Up @@ -139,14 +139,16 @@ def configuration( # noqa: PLR0913 - the shared command-line selection
raise SuiteError(f"{self.name} admits only {', '.join(self.steppers)}")
if not 0 < fraction <= 1 or not 0 <= warmup < 1:
raise SuiteError("length fraction must be in (0, 1], warm-up in [0, 1)")
# "julia32" is the julia backend in single precision (the Julia experiment).
single = backend in ("mlx", "julia32")
return {
"case": self.name,
"revision": self.revision,
"case_hash": self.digest,
"stepper": stepper,
"threads": width,
"backend": backend,
"precision": "single" if backend == "mlx" else "double",
"backend": "julia" if backend == "julia32" else backend,
"precision": "single" if single else "double",
"noise": noise,
"fft_size": fft_size,
"device": device,
Expand Down
6 changes: 6 additions & 0 deletions benchmarks/system.py
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,9 @@
"SOLPROP_NATIVE_RAMAN_FFT_THREADS",
"SOLPROP_NATIVE_RAMAN_TAIL_FFT_THREADS",
"SOLPROP_NATIVE_THREADED_FFT_MIN_SAMPLES",
"SOLPROP_JULIA_THREADS",
"SOLPROP_JULIA_FFT_THREADS",
"SOLPROP_JULIA_BLAS_THREADS",
"SOLPROP_BLAS_THREADS",
"OMP_NUM_THREADS",
"OPENBLAS_NUM_THREADS",
Expand Down Expand Up @@ -279,6 +282,7 @@ def thread_environment(
for key in (
"SOLPROP_THREADS",
"SOLPROP_NATIVE_THREADS",
"SOLPROP_JULIA_THREADS",
"OMP_NUM_THREADS",
"OPENBLAS_NUM_THREADS",
"MKL_NUM_THREADS",
Expand All @@ -292,6 +296,8 @@ def thread_environment(
"SOLPROP_NATIVE_FFT_THREADS",
"SOLPROP_NATIVE_RAMAN_FFT_THREADS",
"SOLPROP_NATIVE_RAMAN_TAIL_FFT_THREADS",
"SOLPROP_JULIA_FFT_THREADS",
"SOLPROP_JULIA_BLAS_THREADS",
"SOLPROP_BLAS_THREADS",
):
environment[key] = "1"
Expand Down
5 changes: 3 additions & 2 deletions benchmarks/track.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,9 @@
thread_environment,
)

BACKENDS = ("numpy_ref", "native", "mlx")
"""The backends a benchmark can name; ``mlx`` only on an Apple GPU."""
BACKENDS = ("numpy_ref", "native", "mlx", "julia", "julia32")
"""The backends a benchmark can name; ``mlx`` only on an Apple GPU, ``julia32`` the
julia backend in single precision."""

REPEAT_TOLERANCE = 1e-9
"""The relative L2 difference two repeats' final fields may have: rounding only."""
Expand Down
46 changes: 39 additions & 7 deletions benchmarks/worker.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
import importlib
import importlib.metadata
import logging
import os
import sys
import time
import traceback
Expand Down Expand Up @@ -60,8 +61,25 @@


def emit(kind: str, **values: object) -> None:
"""Flush a protocol event; the parent reads it after the worker exits."""
print(canonical({"event": kind, **values}), flush=True)
"""Flush a protocol event; the parent reads it after the worker exits.

An embedded runtime can leave the inherited pipe nonblocking (Julia's libuv
does), and a large event then stops short; it is written in blocking mode.
"""
descriptor = None
blocking = True
try:
descriptor = sys.stdout.fileno()
blocking = os.get_blocking(descriptor)
except OSError, ValueError:
pass # In-memory output capture has no operating-system descriptor.
if descriptor is not None and not blocking:
os.set_blocking(descriptor, True)
try:
print(canonical({"event": kind, **values}), flush=True)
finally:
if descriptor is not None and not blocking:
os.set_blocking(descriptor, False)


HOST_SPACES = ("OpenMP", "Threads", "Serial")
Expand All @@ -75,8 +93,18 @@ def runtime(backend: str, device: bool = False) -> Record:
other run needs a CPU build.
"""
info: Record = {}
if device and backend != "native":
raise SuiteError("only the native backend runs on a device")
if device and backend not in {"native", "julia"}:
raise SuiteError("only the native and julia backends run on a device")
if backend == "julia":
# The Julia experiment runs on its CUDA device under --device; its sources
# are identified by digest, as a native build is by its extension's.
# A GPU run takes the GPU the environment names (rocm on AMD), else CUDA.
named = os.environ.get("SOLPROP_JULIA_DEVICE")
gpu = named if named in {"cuda", "rocm", "metal"} else "cuda"
os.environ["SOLPROP_JULIA_DEVICE"] = gpu if device else "cpu"
from solprop.backends.julia import source_digest # noqa: PLC0415

info["julia_build"] = source_digest()
if backend == "native":
# solprop_native must be imported before any other OpenMP runtime.
import solprop_native # noqa: PLC0415 # pyright: ignore[reportMissingImports]
Expand All @@ -96,7 +124,7 @@ def runtime(backend: str, device: bool = False) -> Record:
import solprop # noqa: PLC0415

packages = {}
for name in ("solprop", "numpy", "scipy", "optmat", "mlx"):
for name in ("solprop", "numpy", "scipy", "optmat", "mlx", "juliacall"):
try:
packages[name] = importlib.metadata.version(name)
except importlib.metadata.PackageNotFoundError:
Expand Down Expand Up @@ -201,13 +229,17 @@ def timed_run(

engine.set_initial(problem.initial.field0, None, z=problem.initial.z)
output = MemoryOutput()
# The run's record before the clock starts: its first evaluation builds the planes'
# bases (about 0.18 s on rdw), which every arm's first repeat would otherwise pay
# inside its window.
metadata = simulation.metadata()
before = source_calls(engine)
started = time.perf_counter_ns()
run_problem(
problem,
engine,
output,
metadata=simulation.metadata(),
metadata=metadata,
stats_spec=problem.stats_spec,
progress="none",
resume_history=simulation.settings.resume_history,
Expand All @@ -232,7 +264,7 @@ def timed_run(

def _device_free(configuration: Record) -> int | None:
"""Read the accelerator's free bytes on a device run, else ``None``."""
if not configuration.get("device"):
if not configuration.get("device") or configuration["backend"] != "native":
return None
import solprop_native # noqa: PLC0415 # pyright: ignore[reportMissingImports]

Expand Down
1 change: 1 addition & 0 deletions changelog.d/julia-backend-experiment.added.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
An experimental `julia` backend (`backend="julia"`, the `julia` extra and a Julia 1.13 runtime prepared by `julia/bootstrap.py`) runs the scalar modal capillary, mode-averaged, radial, Cartesian and line routes on a CPU, or on an NVIDIA GPU with `SOLPROP_JULIA_DEVICE=cuda`, in binary64 or binary32, from one Julia source. It is an experiment, not a supported backend: `julia/STATUS.md` lists what it runs and refuses, and the [manual page](docs/guide/performance/julia.md) how to try it.
1 change: 1 addition & 0 deletions coverage-cpu.ini
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ source = solprop
branch = True
omit =
*/solprop/backends/mlx_gpu/*
*/solprop/backends/julia/*

[report]
fail_under = 100
Expand Down
Loading