Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
d33e8f2
SEP 49 phase 0: record the size and host-assembly baselines
jtravs Nov 12, 2026
a0a2f7b
SEP 49 phase 1: move the TDSE source assembly to backends/tdse_source
jtravs Nov 12, 2026
647af1d
Merge remote-tracking branch 'origin/main' into feature/sep-0049-tdse…
jtravs Nov 12, 2026
0c5a15e
SEP 49: set In progress and record phases 0 and 1 in Part II
jtravs Nov 12, 2026
0f797ab
Let a wait inside a commit raise the signal the commit holds
jtravs Nov 12, 2026
ea431aa
SEP 49 phase 2: a provider-agnostic source worker
jtravs Nov 12, 2026
8f5b64b
SEP 49 phase 2: run Pyroraptor in the worker for every TDSE use
jtravs Nov 12, 2026
63f4368
SEP 49 phase 2: keep the trial's own error when its release fails
jtravs Nov 12, 2026
7f985eb
Merge remote-tracking branch 'origin/main' into feature/sep-0049-tdse…
jtravs Nov 12, 2026
8e7771f
SEP 49: record phase 2 in Part II
jtravs Nov 12, 2026
0d31cda
SEP 49: a fresh worker after any failure; fix the docs build
jtravs Nov 12, 2026
4726d56
SEP 49: harden the source worker after review
jtravs Nov 12, 2026
42e3b22
Merge remote-tracking branch 'origin/refactor/native-structure-seam' …
jtravs Nov 12, 2026
478b380
SEP 49 phase 3: the TDSE response on native through a host source hook
jtravs Nov 12, 2026
add5cd5
Merge remote-tracking branch 'origin/refactor/native-structure-seam' …
jtravs Nov 12, 2026
0625619
SEP 49 phase 3: fix the review's defects in the native hook
jtravs Nov 12, 2026
b9e04c5
SEP 49 phase 4: ADR-078, the remaining pages, size and the manual check
jtravs Nov 12, 2026
7a5ca1f
SEP 49: accept ADR-078; pin no Pyroraptor revision
jtravs Nov 12, 2026
169a932
SEP 49: record the manual check with a real Pyroraptor
jtravs Nov 12, 2026
7d0e44d
Merge origin/main into feature/sep-0049-tdse-on-native
jtravs Nov 12, 2026
b23f804
Bound the strict-FMA check by symbol size; format the SEP's script block
jtravs Nov 12, 2026
adc76c0
SEP 49: fix the CI test failures on the merged branch
jtravs Nov 12, 2026
e6f1b5a
Merge remote-tracking branch 'origin/main' into feature/sep-0049-tdse…
jtravs Nov 13, 2026
55abe4e
SEP 49: cover the worker's own side and the binding's failure paths
jtravs Nov 13, 2026
f428114
SEP 49: Final
jtravs Nov 13, 2026
33a2394
Retrigger the documentation build (Read the Docs ended the last one f…
jtravs Nov 13, 2026
b6b22ee
Merge remote-tracking branch 'origin/main' into feature/sep-0049-tdse…
jtravs Nov 13, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ physics or numerics rather than style, ask rather than guess.
| Purpose | Ultrafast nonlinear pulse propagation (UPPE and GNLSE; field and envelope; single-mode, multimode and general modal guides; radial, Cartesian, planar and line free space) |
| Python | 3.14 — equal to `requires-python` in `pyproject.toml` |
| Platforms | solprop runs where solprop-native installs: wheels for osx-arm64, linux-64, linux-aarch64, win-64 and win-arm64; from source on Linux and macOS (GPU builds on Linux); the Queue and Slurm scan executors POSIX-only |
| Backends | `native` (required; `auto`; CPU and GPU; implements every feature except the TDSE response) and `numpy_ref` (the numerical reference). No other backend. Binary64 only (ADR-077): the single-precision code, CF4 sections, `ComparisonMultistep` and `PIController` overrides that still run on `numpy_ref` are being removed |
| Backends | `native` (required; `auto`; CPU and GPU; implements every feature, the TDSE response on CPU builds only) and `numpy_ref` (the numerical reference). No other backend. Binary64 only (ADR-077): the single-precision code, CF4 sections, `ComparisonMultistep` and `PIController` overrides that still run on `numpy_ref` are being removed |
| Type checker | `basedpyright`, pinned exactly in the dev group |
| Materials | [optmat](https://github.com/LupoLab/optmat), the curated optical-materials package |
| Planned work | [GitHub issues](https://github.com/LupoLab/solprop/issues), not the roadmap, which lists only SEPs (ADR-062); [`CONTRIBUTING.md`](CONTRIBUTING.md) says how to write and label one |
Expand Down
1 change: 1 addition & 0 deletions changelog.d/tdse-on-native.changed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
The TDSE response runs on the native backend's CPU build, and `backend="auto"` now selects native for a TDSE run as for every other; `backend="numpy_ref"` still runs it, and a native GPU build refuses it by name (SEP 49).
1 change: 1 addition & 0 deletions changelog.d/tdse-provider-unpinned.changed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
solprop no longer pins a Pyroraptor revision: a TDSE run uses whatever provider is installed (by default its latest `main`, until Pyroraptor has a release), and records the revision, build and execution space the provider reports; the provider's protocol version and its execution space against `TDSEExecution.device` are still checked (SEP 49).
1 change: 1 addition & 0 deletions changelog.d/tdse-provider-worker.changed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
The TDSE response's atomic provider now runs in a worker process of its own, never in solprop's process, for presets, bases, consistency checks and runs alike; Ctrl-C, SIGTERM and a run's stop flag now stop a TDSE run during an atomic call, the call charged and the last accepted step checkpointed (SEP 49).
10 changes: 7 additions & 3 deletions docs/dev/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,8 +117,12 @@ binding from its step record. It changes no numerical step and adds no call; the
optical file records the archive's run UUID and relative path.

Plan lifetime, allocation admission and attempted-call budgets
belong to the driver/binding; modal synthesis, P/J projection and spectral
assembly belong to the reference backend. See the [TDSE manual](../guide/media/tdse.md).
belong to the driver/binding, which runs the provider in a worker process of its own
(`driver/_source_worker.py`, provider-agnostic, with `driver/_pyroraptor.py` as the
provider's adapter): framed pipes, no pickle, cancellation while waiting; modal synthesis to the nodes, the assembly of the
macroscopic sources and their projection belong to `backends/tdse_source.py`, which
imports neither backend and which a backend calls from its source evaluation
([SEP 49](seps/archive/sep-0049-tdse-on-native.md)). See the [TDSE manual](../guide/media/tdse.md).

Geometry: mode-averaged propagation is the single-node case of multimode propagation (the
effective-area normalisation replaces the quadrature); the GNLSE is the mode-averaged
Expand Down Expand Up @@ -214,7 +218,7 @@ against `src/solprop` by `tests/test_structure_figures.py`.
| `modes/` | Mode protocols and derived quantities; capillary, antiresonant, tabulated, step-index, rectangular and planar (slab, uniform in y, per unit width) modes, with `PlanarGrid` for the planar guide resolved in y; shared gas fill, quadrature and collections; full-vector modal records, material laws and regions, bound step-index and radial graded-index providers, and the guide that pairs a provider with a mode collection, and the connection of a basis that moves along z | P |
| `linear.py`, `boundaries.py`, `nonlinear/` | Linear assembly, absorbing rates, response definitions, ionisation, Raman, normalisation and the GNLSE mapping | P |
| `translate/` | Validated `PropagationProblem` (`translate/problem.py`) and its route union (`translate/routes.py`), assembled by `assemble` (`translate/assembly.py`), with a capillary's coherent wall and taper coupling in `translate/walls.py` and the refusals and advisories assembly applies in `translate/validation.py`; parameter expansions and z tables; modal/radial/Cartesian transforms and the hybrid planar one (slab modes in x × FFT in y, `translate/planar.py`); dense tracked modal tables, the general modal route and the coupled-flow and metric records (`translate/general.py`, `translate/coupled.py`); the spatial plane measure (`translate/flux.py`, the `FluxPlan` of schema 35); the persisted frame integral and the filled input port of a problem's entrance (`translate/coupling.py`); what the general route and the coupled flow execute, checked once (`translate/admission.py`); response tables, masks and stage bundles | T |
| `backends/` | Registry and protocols, sessions, the Adams call ledger both backends keep (`ledger.py`), node chunks, scalar bodies, spatial and population statistics | B |
| `backends/` | Registry and protocols, sessions, the Adams call ledger both backends keep (`ledger.py`), node chunks, scalar bodies, spatial and population statistics, and the TDSE response's host source assembly (`tdse_source.py`) | B |
| `backends/numpy_ref/` | Reference execution, its Adams kernels and tables (`multistep.py`) and its on-axis scan (`populations.py`) | B |
| `backends/host/` | The reference's own step driver, Adams and low-storage marches, fixed-schedule group, run session and group session ([SEP 47](seps/archive/sep-0047-run-ownership.md) phase 2); the parts the Python sessions compose are in `backends/session_core.py` | B |
| `stepper/`, `log.py`, `stats.py` | Tableaux, controllers, schedules, progress reporters and statistics specifications | D |
Expand Down
10 changes: 5 additions & 5 deletions docs/dev/backends.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,8 @@ processor is at or above a build's baseline, and otherwise raises `BackendError`
to do; it stores that name before execution or worker transfer. `auto` has no fallback.
Native being installed is a packaging policy ([SEP 43](seps/archive/sep-0043-retire-numba-cpu.md) §5),
so a run does not have to prove that native could execute here, and nothing loads a runtime to
enforce it. A TDSE response's `auto` resolves before the registry, to `numpy_ref`, and
never consults native. Explicit names are preserved, so `backend="numpy_ref"` runs where
enforce it. A TDSE response's `auto` resolves to `native` like every other run (SEP 49, ADR-078).
Explicit names are preserved, so `backend="numpy_ref"` runs where
native is absent and `backend="native"` on a processor below the baseline raises the package's refusal.
`select` also accepts `"auto"`, checks `SOLPROP_REQUIRE_DEVICE` against the actual build before
anything is allocated, and logs the actual backend at INFO after
Expand Down Expand Up @@ -367,9 +367,9 @@ threading beyond what `scipy.fft` does with `workers=1`. It exists to be read an
every step of the anatomy in [architecture.md §8](architecture.md) is a named function with a
docstring stating the formula. It is slow (tens of separate array operations per stage) and
that is acceptable; it is the reference against which every other backend is validated, and
the supported executor of the one feature `native` does not run yet: the TDSE
response
([the list](../guide/performance/backends.md#features-that-run-on-the-reference-backend)).
the supported executor of any feature `native` does not run yet, of which there is none
since the TDSE response joined the native CPU build (SEP 49, ADR-078;
[the list](../guide/performance/backends.md#features-that-run-on-the-reference-backend)).
A new feature joins the list only as an accepted exception (§10).

**The fused-pass route.** `backends/numpy_ref/passes.py` holds one pure
Expand Down
65 changes: 63 additions & 2 deletions docs/dev/decisions.md
Original file line number Diff line number Diff line change
Expand Up @@ -2236,8 +2236,10 @@ Withdrawn (development-process record, not a design decision).
## ADR-077 — Two backends, binary64 only

- **Status:** accepted (maintainer), 2026-11-11. Supersedes ADR-054 and ADR-061; amends
ADR-064 (single precision under `backend="auto"`); withdraws [SEP 48](seps/archive/sep-0048-julia-backend.md). Those
records, and the SEPs implemented under them, stay as written.
ADR-064 (single precision under `backend="auto"`); withdraws
[SEP 48](seps/archive/sep-0048-julia-backend.md). Those records, and the SEPs
implemented under them, stay as written. Amended by ADR-078: native runs the TDSE
response too.
- **Context:** single precision runs on `numpy_ref` alone since `mlx` was retired
(SEP 47): no installed backend runs it quickly, and `backend="auto"` refuses it.
[SEP 36](seps/archive/sep-0036-native-reduced-precision.md) measured single and mixed
Expand Down Expand Up @@ -2274,3 +2276,62 @@ Withdrawn (development-process record, not a design decision).
- The backend registry becomes a fixed table of the two backends, with no plugin rows.
- The precision field and CF4's settings leave the hashed configuration in one release,
so resume identity changes once: a run started on an earlier release finishes on it.

## ADR-078 — The TDSE response runs on native through a host source; its provider in a worker

- **Status:** accepted (maintainer), 2026-11-12, with
[SEP 49](seps/archive/sep-0049-tdse-on-native.md) (PR #771). Amends ADR-077's "every feature
except the TDSE response, for now": native implements every feature.
- **Context:** the TDSE response ran on `numpy_ref` alone, and native refused it by name.
Its source is a complete atomic calculation at every radial node, from 93 s to over
1,300 s a call, so a native rewrite of the provider buys nothing; what native lacked
was a way to hand the provider a stage's field and take its source. The provider,
Pyroraptor, ran in solprop's process, beside the native wheel's Kokkos and OpenMP
runtimes, which rules out a GPU provider beside a CPU wheel, and a call of minutes
could not be stopped.
- **Decision:**
- *The host source hook.* The native resident Adams march takes one optional, generic
host source, `begin(cost)`, `evaluate(field, z, purpose, source)` and
`end(outcome)`, that knows nothing of the TDSE. When it is bound, each source
evaluation hands it the stage's lab-frame field, unscaled and with any noise
added, and takes the projected source in place of the engine's own response pass;
the stage finish applies the collar, the in-stage absorber and the propagator as on
every route. It is called from the thread that called `advance`, outside every
parallel region, with the interpreter lock taken for the call: the one Python call
in a native trial. A Python exception is caught at the binding layer, never inside
a Kokkos kernel. Its scope: the native CPU build, the adaptive Adams march, and one
real column of a modal route (the TDSE response's own scope). The source assembly,
`solprop.backends.tdse_source`, is shared by both backends.
- *Device builds refuse.* A native CUDA or HIP build refuses a TDSE problem by name:
a TDSE run gives the provider the GPU or every CPU thread, and solprop's optical
side is negligible beside an atomic call.
- *The provider worker.* The provider runs only in a worker process of its own (the
same interpreter, `subprocess`, length-prefixed frames, no pickle), for presets,
bases, diagnostics and runs alike, so solprop's process never imports it. A worker
is replaced after any provider failure. Its handshake refuses another protocol
version or an execution space other than `TDSEExecution.device`. No provider
revision is pinned: until Pyroraptor has a release, solprop uses whatever provider
is installed, by default the latest `main`, and records the revision, build and
execution space the worker reports as the run's build record. Ctrl-C, SIGTERM and a stop flag end it
during a call; the call stays charged.
- *The failure boundary.* An attempt reserves its atomic calls before it evaluates.
A failure before the step commits abandons the attempt (its reservation released,
nothing recorded, the state the last accepted one); a failure after the commit, in
the archive's record, leaves the committed step standing and is the run's error;
a release that fails after another error is a note on it. Both backends keep this
boundary.
- `backend="auto"` runs a TDSE problem on native; `backend="numpy_ref"` still runs it,
as the reference.
- **Why:** the response is the provider's arithmetic either way; what native adds is
every optical feature it already runs, at its speed, and one backend for production.
A generic hook keeps the engine free of the TDSE and of Python, except at that one
call. A process boundary is the simplest isolation of two native runtimes, and the
only one that lets a GPU provider sit beside a CPU wheel; a few milliseconds a call
are nothing against minutes.
- **Consequences:** per evaluation the hook adds a Python call, the worker a copy each
way, and the shared assembly 0.13–1.9 ms (SEP 49, phase 0), below 1e-4 of the
fastest recorded atomic call. Native runs without a host source are unchanged bit for
bit. The hook's two rows are listed as `host_source` in the memory ledger. CI tests
the hook and the worker with manufactured providers; compatibility with an installed
Pyroraptor is checked by hand ([SEP 49](seps/archive/sep-0049-tdse-on-native.md), section 7.4),
and the worker's handshake refuses a mismatched protocol or execution space.
14 changes: 8 additions & 6 deletions docs/dev/dependencies.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,12 +40,14 @@ is prepared; ordinary imports and non-TDSE runs never initialise its runtime. A
provider, an unsupported interpreter (the provider supports Python 3.14 only) or a
missing capability is refused with a `BackendError`.

solprop pins one provider source revision, `solprop.driver.tdse._PROVIDER_REVISION`
(currently `876b72d10b5286478125b842ea45548a5051b033`);
the binding checks the installed provider against it, and the source identity and the
loaded native build identity are recorded separately with every run. A device build
(CUDA or HIP) is selected explicitly through `TDSEExecution.device` and checked against
the build's execution space, with no fallback.
solprop pins no provider revision: until Pyroraptor has a release, it uses whatever
provider is installed, by default the latest `main`
([SEP 49](seps/archive/sep-0049-tdse-on-native.md), ADR-078). The provider runs in a worker
process, whose handshake refuses another protocol version; the revision, build and
execution space it reports are recorded with every run, and a preset's source names
that revision. A device build (CUDA or HIP) is selected explicitly through
`TDSEExecution.device` and checked against the build's execution space, with no
fallback.

## The native backend package

Expand Down
2 changes: 1 addition & 1 deletion docs/dev/figures/build.py
Original file line number Diff line number Diff line change
Expand Up @@ -724,6 +724,7 @@ class Layer(NamedTuple):
"registry",
"spatial",
"populations",
"tdse_source",
"memory",
"node_chunk",
"ledger",
Expand Down Expand Up @@ -759,7 +760,6 @@ class Layer(NamedTuple):
"populations",
"envelope_plasma",
"general_modal",
"tdse",
),
),
),
Expand Down
11 changes: 5 additions & 6 deletions docs/dev/figures/layers.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Loading