diff --git a/AGENTS.md b/AGENTS.md index fb01608c2..d23c18dea 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -24,7 +24,7 @@ physics or numerics rather than style, ask rather than guess. | Purpose | Ultrafast nonlinear pulse propagation (UPPE and GNLSE; field and envelope; single-mode, multimode and general modal guides; radial, Cartesian, planar and line free space) | | Python | 3.14 — equal to `requires-python` in `pyproject.toml` | | Platforms | solprop runs where solprop-native installs: wheels for osx-arm64, linux-64, linux-aarch64, win-64 and win-arm64; from source on Linux and macOS (GPU builds on Linux); the Queue and Slurm scan executors POSIX-only | -| Backends | `native` (required; `auto`; CPU and GPU; implements every feature except the TDSE response) and `numpy_ref` (the numerical reference). No other backend. Binary64 only (ADR-077): the single-precision code, CF4 sections, `ComparisonMultistep` and `PIController` overrides that still run on `numpy_ref` are being removed | +| Backends | `native` (required; `auto`; CPU and GPU; implements every feature, the TDSE response on CPU builds only) and `numpy_ref` (the numerical reference). No other backend. Binary64 only (ADR-077): the single-precision code, CF4 sections, `ComparisonMultistep` and `PIController` overrides that still run on `numpy_ref` are being removed | | Type checker | `basedpyright`, pinned exactly in the dev group | | Materials | [optmat](https://github.com/LupoLab/optmat), the curated optical-materials package | | Planned work | [GitHub issues](https://github.com/LupoLab/solprop/issues), not the roadmap, which lists only SEPs (ADR-062); [`CONTRIBUTING.md`](CONTRIBUTING.md) says how to write and label one | diff --git a/changelog.d/tdse-on-native.changed.md b/changelog.d/tdse-on-native.changed.md new file mode 100644 index 000000000..e4d0c8265 --- /dev/null +++ b/changelog.d/tdse-on-native.changed.md @@ -0,0 +1 @@ +The TDSE response runs on the native backend's CPU build, and `backend="auto"` now selects native for a TDSE run as for every other; `backend="numpy_ref"` still runs it, and a native GPU build refuses it by name (SEP 49). diff --git a/changelog.d/tdse-provider-unpinned.changed.md b/changelog.d/tdse-provider-unpinned.changed.md new file mode 100644 index 000000000..05750aa8b --- /dev/null +++ b/changelog.d/tdse-provider-unpinned.changed.md @@ -0,0 +1 @@ +solprop no longer pins a Pyroraptor revision: a TDSE run uses whatever provider is installed (by default its latest `main`, until Pyroraptor has a release), and records the revision, build and execution space the provider reports; the provider's protocol version and its execution space against `TDSEExecution.device` are still checked (SEP 49). diff --git a/changelog.d/tdse-provider-worker.changed.md b/changelog.d/tdse-provider-worker.changed.md new file mode 100644 index 000000000..8f8b0c5f0 --- /dev/null +++ b/changelog.d/tdse-provider-worker.changed.md @@ -0,0 +1 @@ +The TDSE response's atomic provider now runs in a worker process of its own, never in solprop's process, for presets, bases, consistency checks and runs alike; Ctrl-C, SIGTERM and a run's stop flag now stop a TDSE run during an atomic call, the call charged and the last accepted step checkpointed (SEP 49). diff --git a/docs/dev/architecture.md b/docs/dev/architecture.md index df11f3574..679dc50f4 100644 --- a/docs/dev/architecture.md +++ b/docs/dev/architecture.md @@ -117,8 +117,12 @@ binding from its step record. It changes no numerical step and adds no call; the optical file records the archive's run UUID and relative path. Plan lifetime, allocation admission and attempted-call budgets -belong to the driver/binding; modal synthesis, P/J projection and spectral -assembly belong to the reference backend. See the [TDSE manual](../guide/media/tdse.md). +belong to the driver/binding, which runs the provider in a worker process of its own +(`driver/_source_worker.py`, provider-agnostic, with `driver/_pyroraptor.py` as the +provider's adapter): framed pipes, no pickle, cancellation while waiting; modal synthesis to the nodes, the assembly of the +macroscopic sources and their projection belong to `backends/tdse_source.py`, which +imports neither backend and which a backend calls from its source evaluation +([SEP 49](seps/archive/sep-0049-tdse-on-native.md)). See the [TDSE manual](../guide/media/tdse.md). Geometry: mode-averaged propagation is the single-node case of multimode propagation (the effective-area normalisation replaces the quadrature); the GNLSE is the mode-averaged @@ -214,7 +218,7 @@ against `src/solprop` by `tests/test_structure_figures.py`. | `modes/` | Mode protocols and derived quantities; capillary, antiresonant, tabulated, step-index, rectangular and planar (slab, uniform in y, per unit width) modes, with `PlanarGrid` for the planar guide resolved in y; shared gas fill, quadrature and collections; full-vector modal records, material laws and regions, bound step-index and radial graded-index providers, and the guide that pairs a provider with a mode collection, and the connection of a basis that moves along z | P | | `linear.py`, `boundaries.py`, `nonlinear/` | Linear assembly, absorbing rates, response definitions, ionisation, Raman, normalisation and the GNLSE mapping | P | | `translate/` | Validated `PropagationProblem` (`translate/problem.py`) and its route union (`translate/routes.py`), assembled by `assemble` (`translate/assembly.py`), with a capillary's coherent wall and taper coupling in `translate/walls.py` and the refusals and advisories assembly applies in `translate/validation.py`; parameter expansions and z tables; modal/radial/Cartesian transforms and the hybrid planar one (slab modes in x × FFT in y, `translate/planar.py`); dense tracked modal tables, the general modal route and the coupled-flow and metric records (`translate/general.py`, `translate/coupled.py`); the spatial plane measure (`translate/flux.py`, the `FluxPlan` of schema 35); the persisted frame integral and the filled input port of a problem's entrance (`translate/coupling.py`); what the general route and the coupled flow execute, checked once (`translate/admission.py`); response tables, masks and stage bundles | T | -| `backends/` | Registry and protocols, sessions, the Adams call ledger both backends keep (`ledger.py`), node chunks, scalar bodies, spatial and population statistics | B | +| `backends/` | Registry and protocols, sessions, the Adams call ledger both backends keep (`ledger.py`), node chunks, scalar bodies, spatial and population statistics, and the TDSE response's host source assembly (`tdse_source.py`) | B | | `backends/numpy_ref/` | Reference execution, its Adams kernels and tables (`multistep.py`) and its on-axis scan (`populations.py`) | B | | `backends/host/` | The reference's own step driver, Adams and low-storage marches, fixed-schedule group, run session and group session ([SEP 47](seps/archive/sep-0047-run-ownership.md) phase 2); the parts the Python sessions compose are in `backends/session_core.py` | B | | `stepper/`, `log.py`, `stats.py` | Tableaux, controllers, schedules, progress reporters and statistics specifications | D | diff --git a/docs/dev/backends.md b/docs/dev/backends.md index 149a32ea0..002be250b 100644 --- a/docs/dev/backends.md +++ b/docs/dev/backends.md @@ -17,8 +17,8 @@ processor is at or above a build's baseline, and otherwise raises `BackendError` to do; it stores that name before execution or worker transfer. `auto` has no fallback. Native being installed is a packaging policy ([SEP 43](seps/archive/sep-0043-retire-numba-cpu.md) §5), so a run does not have to prove that native could execute here, and nothing loads a runtime to -enforce it. A TDSE response's `auto` resolves before the registry, to `numpy_ref`, and -never consults native. Explicit names are preserved, so `backend="numpy_ref"` runs where +enforce it. A TDSE response's `auto` resolves to `native` like every other run (SEP 49, ADR-078). +Explicit names are preserved, so `backend="numpy_ref"` runs where native is absent and `backend="native"` on a processor below the baseline raises the package's refusal. `select` also accepts `"auto"`, checks `SOLPROP_REQUIRE_DEVICE` against the actual build before anything is allocated, and logs the actual backend at INFO after @@ -367,9 +367,9 @@ threading beyond what `scipy.fft` does with `workers=1`. It exists to be read an every step of the anatomy in [architecture.md §8](architecture.md) is a named function with a docstring stating the formula. It is slow (tens of separate array operations per stage) and that is acceptable; it is the reference against which every other backend is validated, and -the supported executor of the one feature `native` does not run yet: the TDSE -response -([the list](../guide/performance/backends.md#features-that-run-on-the-reference-backend)). +the supported executor of any feature `native` does not run yet, of which there is none +since the TDSE response joined the native CPU build (SEP 49, ADR-078; +[the list](../guide/performance/backends.md#features-that-run-on-the-reference-backend)). A new feature joins the list only as an accepted exception (§10). **The fused-pass route.** `backends/numpy_ref/passes.py` holds one pure diff --git a/docs/dev/decisions.md b/docs/dev/decisions.md index 43745b8b1..dfa0361e0 100644 --- a/docs/dev/decisions.md +++ b/docs/dev/decisions.md @@ -2236,8 +2236,10 @@ Withdrawn (development-process record, not a design decision). ## ADR-077 — Two backends, binary64 only - **Status:** accepted (maintainer), 2026-11-11. Supersedes ADR-054 and ADR-061; amends - ADR-064 (single precision under `backend="auto"`); withdraws [SEP 48](seps/archive/sep-0048-julia-backend.md). Those - records, and the SEPs implemented under them, stay as written. + ADR-064 (single precision under `backend="auto"`); withdraws + [SEP 48](seps/archive/sep-0048-julia-backend.md). Those records, and the SEPs + implemented under them, stay as written. Amended by ADR-078: native runs the TDSE + response too. - **Context:** single precision runs on `numpy_ref` alone since `mlx` was retired (SEP 47): no installed backend runs it quickly, and `backend="auto"` refuses it. [SEP 36](seps/archive/sep-0036-native-reduced-precision.md) measured single and mixed @@ -2274,3 +2276,62 @@ Withdrawn (development-process record, not a design decision). - The backend registry becomes a fixed table of the two backends, with no plugin rows. - The precision field and CF4's settings leave the hashed configuration in one release, so resume identity changes once: a run started on an earlier release finishes on it. + +## ADR-078 — The TDSE response runs on native through a host source; its provider in a worker + +- **Status:** accepted (maintainer), 2026-11-12, with + [SEP 49](seps/archive/sep-0049-tdse-on-native.md) (PR #771). Amends ADR-077's "every feature + except the TDSE response, for now": native implements every feature. +- **Context:** the TDSE response ran on `numpy_ref` alone, and native refused it by name. + Its source is a complete atomic calculation at every radial node, from 93 s to over + 1,300 s a call, so a native rewrite of the provider buys nothing; what native lacked + was a way to hand the provider a stage's field and take its source. The provider, + Pyroraptor, ran in solprop's process, beside the native wheel's Kokkos and OpenMP + runtimes, which rules out a GPU provider beside a CPU wheel, and a call of minutes + could not be stopped. +- **Decision:** + - *The host source hook.* The native resident Adams march takes one optional, generic + host source, `begin(cost)`, `evaluate(field, z, purpose, source)` and + `end(outcome)`, that knows nothing of the TDSE. When it is bound, each source + evaluation hands it the stage's lab-frame field, unscaled and with any noise + added, and takes the projected source in place of the engine's own response pass; + the stage finish applies the collar, the in-stage absorber and the propagator as on + every route. It is called from the thread that called `advance`, outside every + parallel region, with the interpreter lock taken for the call: the one Python call + in a native trial. A Python exception is caught at the binding layer, never inside + a Kokkos kernel. Its scope: the native CPU build, the adaptive Adams march, and one + real column of a modal route (the TDSE response's own scope). The source assembly, + `solprop.backends.tdse_source`, is shared by both backends. + - *Device builds refuse.* A native CUDA or HIP build refuses a TDSE problem by name: + a TDSE run gives the provider the GPU or every CPU thread, and solprop's optical + side is negligible beside an atomic call. + - *The provider worker.* The provider runs only in a worker process of its own (the + same interpreter, `subprocess`, length-prefixed frames, no pickle), for presets, + bases, diagnostics and runs alike, so solprop's process never imports it. A worker + is replaced after any provider failure. Its handshake refuses another protocol + version or an execution space other than `TDSEExecution.device`. No provider + revision is pinned: until Pyroraptor has a release, solprop uses whatever provider + is installed, by default the latest `main`, and records the revision, build and + execution space the worker reports as the run's build record. Ctrl-C, SIGTERM and a stop flag end it + during a call; the call stays charged. + - *The failure boundary.* An attempt reserves its atomic calls before it evaluates. + A failure before the step commits abandons the attempt (its reservation released, + nothing recorded, the state the last accepted one); a failure after the commit, in + the archive's record, leaves the committed step standing and is the run's error; + a release that fails after another error is a note on it. Both backends keep this + boundary. + - `backend="auto"` runs a TDSE problem on native; `backend="numpy_ref"` still runs it, + as the reference. +- **Why:** the response is the provider's arithmetic either way; what native adds is + every optical feature it already runs, at its speed, and one backend for production. + A generic hook keeps the engine free of the TDSE and of Python, except at that one + call. A process boundary is the simplest isolation of two native runtimes, and the + only one that lets a GPU provider sit beside a CPU wheel; a few milliseconds a call + are nothing against minutes. +- **Consequences:** per evaluation the hook adds a Python call, the worker a copy each + way, and the shared assembly 0.13–1.9 ms (SEP 49, phase 0), below 1e-4 of the + fastest recorded atomic call. Native runs without a host source are unchanged bit for + bit. The hook's two rows are listed as `host_source` in the memory ledger. CI tests + the hook and the worker with manufactured providers; compatibility with an installed + Pyroraptor is checked by hand ([SEP 49](seps/archive/sep-0049-tdse-on-native.md), section 7.4), + and the worker's handshake refuses a mismatched protocol or execution space. diff --git a/docs/dev/dependencies.md b/docs/dev/dependencies.md index 1ea7326b4..c921ae2af 100644 --- a/docs/dev/dependencies.md +++ b/docs/dev/dependencies.md @@ -40,12 +40,14 @@ is prepared; ordinary imports and non-TDSE runs never initialise its runtime. A provider, an unsupported interpreter (the provider supports Python 3.14 only) or a missing capability is refused with a `BackendError`. -solprop pins one provider source revision, `solprop.driver.tdse._PROVIDER_REVISION` -(currently `876b72d10b5286478125b842ea45548a5051b033`); -the binding checks the installed provider against it, and the source identity and the -loaded native build identity are recorded separately with every run. A device build -(CUDA or HIP) is selected explicitly through `TDSEExecution.device` and checked against -the build's execution space, with no fallback. +solprop pins no provider revision: until Pyroraptor has a release, it uses whatever +provider is installed, by default the latest `main` +([SEP 49](seps/archive/sep-0049-tdse-on-native.md), ADR-078). The provider runs in a worker +process, whose handshake refuses another protocol version; the revision, build and +execution space it reports are recorded with every run, and a preset's source names +that revision. A device build (CUDA or HIP) is selected explicitly through +`TDSEExecution.device` and checked against the build's execution space, with no +fallback. ## The native backend package diff --git a/docs/dev/figures/build.py b/docs/dev/figures/build.py index 12fdc7119..2775c0f75 100644 --- a/docs/dev/figures/build.py +++ b/docs/dev/figures/build.py @@ -724,6 +724,7 @@ class Layer(NamedTuple): "registry", "spatial", "populations", + "tdse_source", "memory", "node_chunk", "ledger", @@ -759,7 +760,6 @@ class Layer(NamedTuple): "populations", "envelope_plasma", "general_modal", - "tdse", ), ), ), diff --git a/docs/dev/figures/layers.svg b/docs/dev/figures/layers.svg index adc03fd44..473f8892b 100644 --- a/docs/dev/figures/layers.svg +++ b/docs/dev/figures/layers.svg @@ -183,11 +183,11 @@ text { font-family: Inter, "Helvetica Neue", Helvetica, Arial, sans-serif; fill: backends/ protocol · registry · spatial -populations · memory · node_chunk -ledger · adams_core · rk_core -exponential · low_storage · session -events · export · session_core -sessions +populations · tdse_source · memory +node_chunk · ledger · adams_core +rk_core · exponential · low_storage +session · events · export +session_core · sessions backends/host/ driver · adams · low_storage @@ -197,7 +197,6 @@ text { font-family: Inter, "Helvetica Neue", Helvetica, Arial, sans-serif; fill: backend · batch · passes multistep · populations envelope_plasma · general_modal -tdse D diff --git a/docs/dev/kernel-contract.md b/docs/dev/kernel-contract.md index 8e90838d9..c9c464d05 100644 --- a/docs/dev/kernel-contract.md +++ b/docs/dev/kernel-contract.md @@ -88,11 +88,11 @@ mode-averaged effective-area route and the resolved quadrature, a wall's coupled | field | type | meaning | |---|---|---| | `transform` | frozen dataclass | §4, of kind `"mode_averaged"` or `"modal"`, with neither `geometry` nor `modal_tables` | -| `tdse` | `TDSESource` or `None` | the TDSE response, the modal route's reference-only extension (§6); `None` without one | +| `tdse` | `TDSESource` or `None` | the TDSE response, the modal route's extension (§6); `None` without one | -`TDSESource` — the TDSE response's immutable scientific identity. Only `numpy_ref` -executes it; `native` refuses it by name. Its execution resource, the atomic binding, is -the driver's and crosses explicitly (§6). +`TDSESource` — the TDSE response's immutable scientific identity. `numpy_ref` and the +native CPU build execute it; a native device build refuses it by name. Its execution +resource, the atomic binding, is the driver's and crosses explicitly (§6). | field | type | meaning | |---|---|---| @@ -278,7 +278,7 @@ invocation, including failures. Logical source-call and atomic-attempt ledgers remain separate. The base driver owns `_adams`, initially `None`, and exports and restores its logical ledger on every Adams backend. Loading a checkpoint resets numerical history but preserves charged calls; startup is charged again. Atomic -ledger restoration remains exclusive to the bound NumPy TDSE path. +ledger restoration needs a bound TDSE binding, on either backend. **The lab-frame source of the exponential formulation.** `HostDriver._adams_lab_source(lab, position, *, purpose)` is the complete source @@ -310,7 +310,8 @@ deleted its host-engine fallback, and with it its `_adams_source`, `_adams_lab_s `TDSEPurpose` label so startup, predictor, corrected-field and diagnostic observations retain their actual provenance. This is a host seam operation: the shared march neither accesses NumPy's private binding nor adds atomic -physics to a compiled backend. The native backend continues to refuse TDSE by name. +physics to a compiled backend. The native backend's march reaches the same binding +through its host source hook (§6). A corrected-field diagnostic still needs an identical cached response or a separately charged solve; a predictor label never implies accepted-field data. If the corrected-field solve fails after candidate adoption, `_adams_restore` @@ -1028,8 +1029,28 @@ stepper's decision on the latest trial. The reference backend calls it from its step record (`_record`), which every Adams attempt reaches, rejected or accepted; it changes no step, charge or reservation. None of this is part of the descriptors or the kernel problem, which compiled backends see, so it is -outside the kernel schema: the native backend refuses TDSE by name and never holds a -binding. +outside the kernel schema. + +**The native host source hook** ([SEP 49](seps/archive/sep-0049-tdse-on-native.md), +ADR-078). On the native CPU build the binding reaches the resident Adams march as a +host source: a session binding outside the kernel problem, whose three entry points the march calls +from the thread that called `advance`, outside every parallel region, holding the +interpreter lock for each. `begin(cost)` reserves an attempt's calls after the march's +budget check; `evaluate(field, z, purpose, out)` receives, read-only, the lab-frame +spectrum the engine would hand its own source pass, unscaled and with any noise added, +which the stage's entry kernel writes into the hook's field row, and writes `F`, the projected source before the +spectral collar, which the shared assembly (`solprop.backends.tdse_source`) forms as +the reference does; `end(outcome)` records an accepted or rejected attempt after its +decision, and after its commit for an accepted one, then releases the reservation, and +an attempt that leaves before a decision only releases it. The engine's own response +pass does not run; its stage finish applies the spectral collar, the in-stage absorber +and the propagator. The hook's two rows, the field and `F`, are allocated when it is +bound, handed to it in place, and listed as `host_source` in the memory ledger. An exception in an entry point is kept on the +Python side and reported to the march as a host failure: before the commit the +attempt is abandoned, after it the committed attempt is returned with the failure, +and the session raises the kept exception unchanged; an abandoned attempt's failed +release rides on the run's own error as a note. A device build refuses a TDSE +problem at preparation. The backend independently projects macroscopic P and J with the existing modal weights and crop scale. It subtracts unscaled discrete shadows, then applies @@ -1117,7 +1138,7 @@ launch's remaining planes). | `schedule` | `(n_steps,) float64` or `None` | fixed schedule to replay (lockstep batches require it); its edges must contain every save plane and hard kink, which `FixedSchedule.uniform` guarantees for a plain fixed-step run | | `save_planes` | `(n_z,) float64` | z positions the driver will ask for; steps land on them unless `interpolate_planes` is set | | `propagator_mode` | `"auto" \| "tabulate" \| "recompute"` | ADR-004 and [backends.md §3.6](backends.md). `"auto"` is the default and is the backend's own choice, resolved at construction and written into the run's `backend` metadata group: `tabulate` for `numpy_ref` and `recompute` for `native` on a modal problem; `recompute` on every backend for a separable free-space operator, whose increment `D (z₂ − z₁)` is exact. Under the low-storage march `tabulate` is one volume of `exp(h D / 2)` reused while the step holds (a varying operator's `P` and `Q` for the step), admitted on the free and planar kinds too, and `"auto"` resolves to `tabulate` for a constant operator and `recompute` for a varying one (SEP 24 §4.3), and to `recompute` on a native device build, where an extra field read costs more than the exponential | -| `integrator` | `Multistep \| LowStorageRungeKutta \| None` | The method stepping the problem instead of the tableau, or `None` for the Runge-Kutta march. `Multistep` ([SEP 13](seps/archive/sep-0013-expensive-response-steppers.md), [SEP 16](seps/archive/sep-0016-unified-multistep.md)) is the Adams march in its exponential or interaction formulation, with an optional carrier frame (`carrier_frame`, [SEP 35](seps/archive/sep-0035-carrier-frame-exponential-adams.md), added in version 37: `"off"` or `"scalar"`, the latter on `numpy_ref` and native CPU builds and refused by name elsewhere); it runs on `numpy_ref` and Kokkos on every diagonal-flow route, and beside the constant-Schur and frozen-residual coupled flows in the exponential formulation; with a `schedule` it replays the schedule's steps (`HostAdamsMarch._replay_to`); TDSE remains NumPy-only. `LowStorageRungeKutta` ([SEP 24](seps/archive/sep-0024-low-storage-interaction-picture-runge-kutta.md)) is RK4 in the midpoint interaction picture in three registers: it replays a `schedule` and nothing else, records the norm 0.0 ("not estimated"), refuses a coupled flow, the TDSE response, interpolated planes and a step past RK4's negative-real interval 2.785, and runs on `numpy_ref` and solprop-native (a resident engine whose step is one native call) | +| `integrator` | `Multistep \| LowStorageRungeKutta \| None` | The method stepping the problem instead of the tableau, or `None` for the Runge-Kutta march. `Multistep` ([SEP 13](seps/archive/sep-0013-expensive-response-steppers.md), [SEP 16](seps/archive/sep-0016-unified-multistep.md)) is the Adams march in its exponential or interaction formulation, with an optional carrier frame (`carrier_frame`, [SEP 35](seps/archive/sep-0035-carrier-frame-exponential-adams.md), added in version 37: `"off"` or `"scalar"`, the latter on `numpy_ref` and native CPU builds and refused by name elsewhere); it runs on `numpy_ref` and Kokkos on every diagonal-flow route, and beside the constant-Schur and frozen-residual coupled flows in the exponential formulation; with a `schedule` it replays the schedule's steps (`HostAdamsMarch._replay_to`); the TDSE response runs adaptive, on `numpy_ref` and the native CPU build (SEP 49). `LowStorageRungeKutta` ([SEP 24](seps/archive/sep-0024-low-storage-interaction-picture-runge-kutta.md)) is RK4 in the midpoint interaction picture in three registers: it replays a `schedule` and nothing else, records the norm 0.0 ("not estimated"), refuses a coupled flow, the TDSE response, interpolated planes and a step past RK4's negative-real interval 2.785, and runs on `numpy_ref` and solprop-native (a resident engine whose step is one native call) | | `interval_cap` | bool | whether a step is capped at the width of the operator-table interval it starts in. `assemble` turns it off on the diagonal-flow routes and on for a coupled flow or a reject-free controller, unless set ([SEP 14](seps/archive/sep-0014-stepper-efficiency.md) §4.1) | | `auto_init` | bool | whether the start-up rule takes the step after the first from the first step's own scaled norm (`PIController.start_factor`, at most 100 times `dz_init`). Set by the convenience interfaces when no `dz_init` is given; off for a coupled flow or a reject-free controller ([SEP 14](seps/archive/sep-0014-stepper-efficiency.md) §4.2) | | `interpolate_planes` | bool | steps across the save planes and observes each inside its accepted step, through DOPRI5's continuous extension or, with an `integrator`, the Adams march's own polynomial (ADR-039); the end of the run and the hard kinks are still landed on. Refused with a schedule, a Runge-Kutta tableau without an extension, the per-step collar, a coupled flow and the `modified` noise | @@ -1329,7 +1350,8 @@ library (#135): each strict file's probe, beside a control that fuses, in the in build (`tests/backends/test_native_strict_witness.py`, and the wheel stage's smoke test); the same probe in the native unit tests of an optimised build with fused multiply-add; and the wheel stage's scan of the strict files' disassembly for a fused instruction -(`native/ci/check_strict_fma.py`). The boundary checks above and the Runge–Kutta march +(`native/ci/check_strict_fma.py`, each function bounded by its symbol's size, since a +stripped extension labels only exported functions). The boundary checks above and the Runge–Kutta march (§12) are held the same way. ## 12. The Runge–Kutta march diff --git a/docs/dev/roadmap.md b/docs/dev/roadmap.md index 0e9a2ff4e..0e5065bf3 100644 --- a/docs/dev/roadmap.md +++ b/docs/dev/roadmap.md @@ -35,13 +35,6 @@ being implemented. routes, bound response and other geometries remain follow-up work; the first route is a single-mode capillary with node quadrature. -- [SEP 49 — The TDSE response on native](seps/sep-0049-tdse-on-native.md): - the TDSE response on the native CPU build through a host source hook, the one Python - call admitted inside a native trial (device builds refuse it), with the host source - assembly shared by both backends and the atomic provider in a worker process of its - own, which `numpy_ref`'s TDSE runs use too. `numpy_ref` stays a complete backend. - TDSE physics, a compiled TDSE source and device TDSE runs remain out of scope. - ## Draft enhancement proposals SEPs under discussion; their scope may change and implementation has not been accepted. diff --git a/docs/dev/seps/archive/index.md b/docs/dev/seps/archive/index.md index 69fe3ab95..7dd2e80a5 100644 --- a/docs/dev/seps/archive/index.md +++ b/docs/dev/seps/archive/index.md @@ -45,4 +45,5 @@ sep-0045-native-rk-march sep-0046-machine-calibration sep-0047-run-ownership sep-0048-julia-backend +sep-0049-tdse-on-native ``` diff --git a/docs/dev/seps/sep-0049-tdse-on-native.md b/docs/dev/seps/archive/sep-0049-tdse-on-native.md similarity index 58% rename from docs/dev/seps/sep-0049-tdse-on-native.md rename to docs/dev/seps/archive/sep-0049-tdse-on-native.md index 8fd6ef96f..cebf728dc 100644 --- a/docs/dev/seps/sep-0049-tdse-on-native.md +++ b/docs/dev/seps/archive/sep-0049-tdse-on-native.md @@ -6,17 +6,17 @@ | **Title** | The TDSE response on native | | **Author** | John Travers | | **Drafting agent** | Opus 5.5 (2026-11-12) | -| **Status** | Accepted | +| **Status** | Final | | **Review status** | Reviewed | | **Reviewers** | GPT-6 (AI); John Travers (human) | | **Type** | Change | | **Created** | 2026-11-12 | | **Resolution** | Accepted 2026-11-12 by John Travers, in [#765](https://github.com/LupoLab/solprop/pull/765) | -| **Requires** | [SEP 12](archive/sep-0012-tdse-nonlinear-response.md), [SEP 20](archive/sep-0020-hacc-helium-tdse.md) and [SEP 26](archive/sep-0026-edge-free-tdse-source.md) (the TDSE response and its source); [SEP 39](archive/sep-0039-native-adams-march.md) (the native Adams march); [SEP 47](archive/sep-0047-run-ownership.md) (backend-owned sessions); [SEP 25](archive/sep-0025-rounding-level-reproducibility.md) (agreement to the error bound); ADR-077 and the change it calls for (§3.4) | +| **Requires** | [SEP 12](sep-0012-tdse-nonlinear-response.md), [SEP 20](sep-0020-hacc-helium-tdse.md) and [SEP 26](sep-0026-edge-free-tdse-source.md) (the TDSE response and its source); [SEP 39](sep-0039-native-adams-march.md) (the native Adams march); [SEP 47](sep-0047-run-ownership.md) (backend-owned sessions); [SEP 25](sep-0025-rounding-level-reproducibility.md) (agreement to the error bound); ADR-077 and the change it calls for (§3.4) | | **Supersedes** | — | | **Superseded-by** | — | -| **Implementation** | Not started | -| **Manual pages** | — | +| **Implementation** | Opus 5.5 (AI) for John Travers, on `feature/sep-0049-tdse-on-native` ([#771](https://github.com/LupoLab/solprop/pull/771)); phases 0–4, started and complete 2026-11-12/13; the manual check of §7.4 done by hand on a CPU build and an NVIDIA A40 | +| **Manual pages** | [TDSE response](../../../guide/media/tdse.md), [Choosing a backend](../../../guide/performance/backends.md) | ## Part I — Proposal @@ -47,7 +47,7 @@ included. Slimming it to a test library, which this SEP first proposed, is defer (§10.1). Success means: a TDSE run on native agrees with the same run on `numpy_ref` within the -rounding budgets of [testing.md](../testing.md) §3 derived for its cases, with its atomic +rounding budgets of [testing.md](../../testing.md) §3 derived for its cases, with its atomic ledger, labels, reservations and archive records identical; the shared source meets independent analytic and manufactured expectations; the worker's contract holds in CI with a manufactured provider; and an explicit stop ends a run within the wait loop's @@ -115,9 +115,9 @@ native hook. Neither the hook nor the worker is specific to TDSE or Pyroraptor ( `numpy_ref` is a complete executor: its own Runge–Kutta, Adams and low-storage marches (`backends/host/`), sessions, group sessions, a lockstep batch and a registry row -([backends.md](../backends.md) §1–§2). Native runs every run through its native session +([backends.md](../../backends.md) §1–§2). Native runs every run through its native session (`solprop_native.session`), whose compiled marches advance to an event from one policy -record (SEP 47; [kernel-contract.md](../kernel-contract.md) §11–§12). Each `advance` call +record (SEP 47; [kernel-contract.md](../../kernel-contract.md) §11–§12). Each `advance` call returns after 256 attempts or 0.1 s of wall time; the session wraps it in `_interrupts.committing()`, so a SIGINT or SIGTERM arriving during the call is held and raised when the call returns. The run's stop flag is polled before each native call, and @@ -125,7 +125,7 @@ a `StopToken` also forwards a stop to the native march, which reads it between a #### 3.2 The TDSE response today -The TDSE response ([TDSE response](../../guide/media/tdse.md); SEPs 12, 20 and 26) is +The TDSE response ([TDSE response](../../../guide/media/tdse.md); SEPs 12, 20 and 26) is admitted on one route only, by `PropagationProblem._check_tdse` and `translate.validation._check_tdse_assembly_route`: @@ -189,14 +189,14 @@ is below $10^{-3}$ of a call (§6.8). The native Adams march runs the trial in its resident engine and evaluates the source natively; its rules forbid "allocation, planning, logging, Python call or unplanned fence -in a trial step" ([backends.md](../backends.md) §3.2, `native/AGENTS.md`). A TDSE source +in a trial step" ([backends.md](../../backends.md) §3.2, `native/AGENTS.md`). A TDSE source cannot be evaluated without the provider, and its evaluation is a field-dependent, minutes-long call that cannot be precomputed. With a trial lasting minutes, each `advance` call returns after one attempt anyway. #### 3.4 Decisions and changes this SEP builds on -- **ADR-077** (two backends, binary64 only; [decisions.md](../decisions.md)), and the +- **ADR-077** (two backends, binary64 only; [decisions.md](../../decisions.md)), and the change it calls for, which must be merged before implementation starts: in particular native importing nothing from `numpy_ref`. ADR-077 records that `numpy_ref` stays a complete, selectable backend (amended with the maintainer's decision in this SEP's @@ -240,7 +240,7 @@ where $P$ and $S = J - j_1$ are the macroscopic polarisation and remainder curre $n_{\text{nodes}}$ radial nodes (density and the declared scales applied once), $Q$ the modal projection with the taper scale, $g$ and $g'$ the gate and its analytic derivative (SEP 26), $p$ the stage prefactor and $c$ the independent current factor -([kernel-contract.md](../kernel-contract.md) §6). Each backend then forms what it forms +([kernel-contract.md](../../kernel-contract.md) §6). Each backend then forms what it forms for every source: the stage output $w\,F$ with the spectral collar $w$, the in-stage absorber term, and in the interaction formulation the inverse propagator $e^{-\Delta\Phi}$. That is the composition of steps 1–5 in §3.2, split after step 4. @@ -253,7 +253,7 @@ the collar, a sum with the absorber term, a phase factor) and its optical march own, so a TDSE stage and a short TDSE run on native agree with `numpy_ref`'s within the one-stage and eight-step budgets `tests/bounds.py` derives for the existing source comparisons ($16\varepsilon$ and $128\varepsilon$ peak-normalised on those cases, -[accuracy.md](../accuracy.md) §7). These are budgets derived for these cases, checked on +[accuracy.md](../../accuracy.md) §7). These are budgets derived for these cases, checked on them, not guarantees for every TDSE problem or variable-step history. Where independently rounded norms fall on either side of a decision threshold, the two runs fork; such a fork is reported and the pair is then compared to an accuracy envelope (class T), not counted @@ -425,9 +425,11 @@ provider: a run's evaluations, `basis_from_preset`, `hacc_preset`, `hacc_basis` length is a protocol error before anything is allocated. - **Handshake.** The worker's first frame carries the protocol version and the provider's identity: its build record (source revision, execution space, Kokkos configuration) and - the worker's process id. The parent refuses a protocol mismatch, a revision other than - `_PROVIDER_REVISION` or an execution space other than `TDSEExecution.device` by name, - before any evaluation. The build record in a run's metadata is this report. + the worker's process id. The parent refuses a protocol mismatch or an execution space + other than `TDSEExecution.device` by name, before any evaluation. It pins no provider + revision: until Pyroraptor has a release, solprop uses whatever provider is installed, + by default the latest `main`, and records the revision it reports (maintainer, + 2026-11-12). The build record in a run's metadata is this report. - **What crosses.** Preparation sends the descriptor's plain records (basis, model, outputs, chunks); the worker builds the models and plans and replies with its memory rows and plan records. Each atomic call sends one chunk's drive rows and receives its @@ -503,9 +505,9 @@ GUI job's stop) and the run's `StopFlag` (a stop button, a scan's supervisor). **Time.** One TDSE evaluation is a complete atomic calculation at every radial node: 93 s (production basis, 17 nodes) to 615 s (strong-field basis, 33 nodes) on compressed -fields in [SEP 12](archive/sep-0012-tdse-nonlinear-response.md)'s node selection +fields in [SEP 12](sep-0012-tdse-nonlinear-response.md)'s node selection (Part II), and 366 s (haCC(1)) and 1,307 s (haCC(5)) on one AMD MI325X -([SEP 20](archive/sep-0020-hacc-helium-tdse.md), Part II). Per evaluation the hook adds +([SEP 20](sep-0020-hacc-helium-tdse.md), Part II). Per evaluation the hook adds a GIL acquisition and a Python call (microseconds), the worker a copy each way of a few MB and a context switch (milliseconds), and the host source assembly, which `numpy_ref` already performs (of order 10–100 ms at the response lattices in use, to be measured in @@ -532,7 +534,7 @@ is removed but `numpy_ref/tdse.py`, which moves): The kernel problem does not change: the TDSE descriptor already crosses it, and the hook is a session binding outside it, as the binding is today. The hook's binding joins the -members `check_build` looks up by name, and [kernel-contract.md](../kernel-contract.md) +members `check_build` looks up by name, and [kernel-contract.md](../../kernel-contract.md) §6's "the native backend refuses TDSE by name and never holds a binding" is rewritten, with a release-note fragment. @@ -562,7 +564,8 @@ worker: 1. evaluations through the worker equal calling the manufactured provider directly, bit for bit, and the binding's call sequence, ledgers and archive are unchanged; -2. the handshake refuses a protocol, revision or execution-space mismatch by name; +2. the handshake refuses a protocol or execution-space mismatch by name, and records + another provider revision; 3. a provider exception, a worker exit, a closed pipe, a malformed or oversized frame, and a failure during start-up and preparation each end as the named failure of §6.5, with the worker reaped and both pipe ends closed; @@ -611,19 +614,20 @@ three atomic calls on the smallest preset, native against `numpy_ref` (class T), the worker, with Pyroraptor on the CPU and, where a GPU build is in use, on a GPU; and one cancellation during a call. Each check is reported in Part II with its configuration (provider revision, build, machine, threads or device). CI establishes the manufactured -contracts, not compatibility with an installed provider; the handshake's revision and -execution-space check (§6.5) refuses a mismatched provider at run time. +contracts, not compatibility with an installed provider; the handshake's protocol and +execution-space checks (§6.5) refuse a mismatched provider at run time, and the +provider's revision is recorded with every run. ### 8. Documentation -- **Manual.** [TDSE response](../../guide/media/tdse.md): runs on native's CPU build and +- **Manual.** [TDSE response](../../../guide/media/tdse.md): runs on native's CPU build and on `numpy_ref`; the provider in its worker, its threads or device, and stopping a run; - [Choosing a backend](../../guide/performance/backends.md): TDSE leaves the list of + [Choosing a backend](../../../guide/performance/backends.md): TDSE leaves the list of features that run only on the reference. -- **Design pages.** [backends.md](../backends.md) §1 (the `auto` rule) and §3.2 (the - hook's exception); [kernel-contract.md](../kernel-contract.md) §6; - [architecture.md](../architecture.md) (the shared source module, the worker); - [testing.md](../testing.md) (the oracle boundary for shared code; worker-contract +- **Design pages.** [backends.md](../../backends.md) §1 (the `auto` rule) and §3.2 (the + hook's exception); [kernel-contract.md](../../kernel-contract.md) §6; + [architecture.md](../../architecture.md) (the shared source module, the worker); + [testing.md](../../testing.md) (the oracle boundary for shared code; worker-contract tests); `AGENTS.md` and `native/AGENTS.md` (the hook's exception). - **Records.** One ADR: the host source hook (the one Python call in a native trial, its scope, its refusal on device builds), the provider worker, and the failure boundary; @@ -652,8 +656,8 @@ providers. differ on Linux, macOS and Windows. Mitigation: §7.2 in CI on every platform CI runs; the reader thread avoids platform-specific polling. - **Provider compatibility untested in CI** (Q10). Mitigation: the handshake refuses a - mismatched revision or execution space by name; manual evidence with its configuration - (§7.4). + mismatched protocol or execution space by name, and records the provider's revision; + manual evidence with its configuration (§7.4). - **Python exceptions through C++.** Mitigation: caught at the binding layer, never inside a Kokkos kernel; §7.3 items 2–3 cover every entry point and the boundary. - **A shared-code defect invisible to parity.** Mitigation: §7.1's independent @@ -722,8 +726,9 @@ Decided by the maintainer on 2026-11-12 (Part III, review follow-up): 4. **Q9. Delivery.** One implementation pull request; Pyroraptor's changes in a linked companion (§11). 5. **Q10. A real provider in CI.** None; by hand as used, reported with its configuration; - the optional import and revision check kept, and the worker's actual provider - identity reported (§6.5, §7.4). + the optional import kept, no revision pinned (the installed provider is used, by + default the latest `main`, until Pyroraptor has a release), and the worker's actual + provider identity recorded (§6.5, §7.4). 6. **Q11. The transport.** Pipes only, arrays unchanged, bounded frames, transport copies in the memory ledger, responsive to cancellation (§6.5, §6.7). @@ -811,6 +816,966 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. ## Part II — Implementation record +### Phase 0 — Baseline + +- **Scope.** Phase 0, 2026-11-12, Opus 5.5 (AI), branch + `feature/sep-0049-tdse-on-native`, based on `feature/two-backends-binary64` at + `1dc64b5f` (ADR-077's change). +- **Delivered.** This record; no code. Commit `d33e8f24`. +- **Size baseline** (`uv run poe size --all` at `1dc64b5f`, lines): + + | tree | files | total | code | + |---|---:|---:|---:| + | `src/solprop` | 269 | 225,295 | 113,232 | + | `native/src/solprop_native` (Python) | 24 | 13,859 | 8,855 | + | `tests/` (all `.py`, `wc -l`) | — | 271,204 | — | + | `native/tests/` (all `.py`, `wc -l`) | — | 13,602 | — | + | `native/src` C++ (`.cpp`, `.hpp`, `.h`, `wc -l`) | — | 34,575 | — | + + `poe size-check` is within budget. `src/solprop/backends/numpy_ref/tdse.py` is 845 + lines; `src/solprop/driver/tdse.py` 2,575. +- **The host assembly's time per evaluation.** Measured by a one-off script, not + committed: the `march_problem` problem of `tests/backends/test_tdse_march.py` + (256 µm helium capillary, 800 nm, one mode), the interaction formulation, a + manufactured binding returning analytic histories (`J = 1e-30 E + 1e-40 E^3`, + `j_1 = 1e-30 E`, and a third-order current for the cubic replacement), and + `NumpyRefBackend._nonlinear(..., purpose="predictor")` timed 60 times, the first ten + discarded, the binding's own time subtracted. That is steps 1 and 3–5 of §3.2: the + synthesis to the nodes, the assembly, the projection with every diagnostic, and the + stage output. + + | lattice | descriptor | `n_nodes` | `n_to` | `n_omega` | median | min | + |---|---|---:|---:|---:|---:|---:| + | the test window (`n_r = 4`, 60 fs, 0.4–1.6 µm) | `tdse_native`, gated | 5 | 384 | 97 | 0.26 ms | 0.25 ms | + | the test window | `optmat_cubic` | 5 | 384 | 97 | 0.13 ms | 0.12 ms | + | production-sized (`n_r = 16`, 400 fs, 0.1–4 µm) | `tdse_native`, gated | 17 | 9,216 | 2,305 | 1.9 ms | 1.9 ms | + | production-sized | `optmat_cubic` | 17 | 9,216 | 2,305 | 1.3 ms | 1.2 ms | + + Apple M5 Max, Python 3.14, the machine shared (load average about 10), so the medians + are upper estimates. §6.8 expected 10–100 ms; the measured assembly is one to two + milliseconds at a production-sized lattice, below $10^{-5}$ of the fastest recorded + atomic call (93 s). +- **Deviations.** None. The record was kept in a draft beside the SEP until the + proposal's pull request brought this file to the branch. +- **Handoff.** Phase 1 moves the assembly; the same script then times it again. + +### Phase 1 — The shared source + +- **Scope.** Phase 1, 2026-11-12, Opus 5.5 (AI), commit `a0a2f7b1`. +- **Delivered.** + - `src/solprop/backends/tdse_source.py`, moved from `backends/numpy_ref/tdse.py` with + `git mv`: `atomic_sources`, `atomic_edge_source`, `project_atomic_sources`, + `FFT_ERROR_CONSTANT` and their private projection, floor and diagnostic helpers, + unchanged, plus `node_fields` (step 1 of §3.2, moved from + `NumpyRefBackend._nonlinear_tdse`). The module imports nothing from either backend: + its four uses of `numpy_ref.passes` (`project`, `to_frequency`, and through + `node_fields` `to_time` and `synthesise`) are private one-line equivalents for the + route's one scalar mode on a field grid, and the cubic replacement's polarisation + is `_cubic_polarisation`, the compiled real Kerr term alone, in the arithmetic and + order of `passes.kerr_polarisation` and `nonlinear_sources`, and refused by name + for any other compiled stage. + - `NumpyRefBackend._nonlinear_tdse` calls `node_fields`, the binding, the two + assembly functions and the projection; the stage output stays its own. + - `tests/backends/test_tdse_source.py` (new, 10 tests) and an optional `responses` + argument of `tests/edge_source_cases.py::make_case`; the four existing modules' + imports re-pointed, their expectations unchanged. + - `docs/dev/architecture.md` (the responsibility and the module table) and the + layers figure (`docs/dev/figures/build.py`, `layers.svg`). +- **Deviations.** + - The module's entry points keep their moved names (`atomic_sources` with + `atomic_edge_source`, and `project_atomic_sources`) rather than §5.2's sketched + `assemble` and `project`, so that the move is pure and the existing tests and their + expectations run unchanged; `node_fields` is new. Each takes the caller's stage + scalars (`StageScalars`: density, Kerr density, taper scale) and the projection the + caller's prefactor, as `numpy_ref` passed them; native's hook will take both from the + same translation helpers. + - `numpy_ref` now transforms the modal field to time twice per TDSE evaluation: once + in `node_fields`, once for its own in-stage absorber term. Re-timed with phase 0's + script: 0.26 ms (test window, gated), 1.97 ms (production-sized, gated), 1.22 ms + (production-sized, cubic), no measurable change. +- **Decisions.** A problem whose cubic replacement has no compiled Kerr term (only the + accumulation stage) keeps a zero polarisation, as `passes.nonlinear_polarisation` + gave it; manufactured test problems are built that way. +- **Verification** (§7.1). `tests/backends/test_tdse_source.py`, + `test_tdse_sources.py`, `test_tdse_projection.py`, `test_tdse_edge_free.py`, + `test_tdse_march.py`, `tests/test_structure_figures.py`, + `tests/driver/test_tdse_gated_derivative_run.py` and `test_tdse_input_noise.py`: + 485 passed; ruff and basedpyright clean on the changed files; `poe size-check` within + budget. The new tests state their expectations with NumPy alone: the nodal field of + one coarse bin, `(2/N) Re(a e^{2 pi i c n / N})` times the synthesis and the taper, + within `(c_F log2 N + 4) eps` of its norm; the projection's division by the taper + (`tau = 2` is exact, so bit for bit); the edge array `rho s g' (J - j_1)` within + `gamma_3`; a source exactly zero off the active band; the complete and remainder + floors, and a gated source's truncating and edge floors, from the formulas of the + docstrings and SEP 26 §4.5, within `1e-12` relative (reassociated sums); and an + import of the module in a fresh interpreter loading neither backend. + + Each named term was then broken, one at a time, in the module, and the five modules + above run (a one-off script, not committed): + + | defect | first failing test | + |---|---| + | `S = J + j_1` | `test_tdse_sources.py::test_each_closure_has_its_own_cubic_and_remainder_definition` | + | cubic replacement added, not subtracted | the same | + | density dropped from the current | the same | + | density dropped from the shadow polarisation | `test_tdse_sources.py::test_cubic_and_remainder_calibrations_are_independent` | + | Kerr density dropped from the cubic polarisation | `test_tdse_sources.py::test_each_closure_has_its_own_cubic_and_remainder_definition` | + | temporal collar dropped from the cubic polarisation | the same | + | projection multiplied by the taper | `test_tdse_source.py::test_the_projection_divides_by_the_taper_scale` | + | taper dropped from the node synthesis | `test_tdse_source.py::test_node_fields_are_the_taper_times_the_synthesis_of_each_bin` | + | node synthesis crop off by one bin | the same | + | gate dropped | `test_tdse_sources.py::test_native_scale_preserves_complete_nonlinearity_after_linear_subtraction` | + | density dropped from the edge array | `test_tdse_source.py::test_the_edge_array_is_density_scale_and_gate_derivative_of_the_remainder` | + | `X + E/(i omega)` | `test_tdse_edge_free.py::test_t2a_a_globally_steady_current_cancels_to_the_budget_and_certificate` | + | `1/(i omega)` of the wrong sign | the same | + | `1/(i omega)` formed on every bin, DC included | `test_tdse_projection.py::test_periodic_dipole_and_its_current_give_the_same_source` | + | spectral crop off by one bin | the same | + | truncating floor halved | `test_tdse_source.py::test_the_complete_floor_is_the_documented_operand_bound` | + | cancelled operand dropped from the floor | `test_tdse_projection.py::test_cubic_roundoff_floor_counts_the_unscaled_third_order_operand` | + | third-order operand dropped from the floor | the same | + | edge transform's constant halved | `test_tdse_source.py::test_the_edge_floors_are_the_documented_section_4_5_bounds` | + | derivative-error term dropped | the same | + | division bound dropped | the same | + | ratio reported below its floor | `test_tdse_projection.py::test_periodic_dipole_and_its_current_give_the_same_source` | + + One variant is invisible and was not made visible: forming `1/(i omega)` on every bin + but DC. The current factor and the prefactor are exactly zero off the active band by + construction (`translate.tdse_source.current_numerator`, the prefactor), so the mask + is defence in depth; `test_the_source_is_exactly_zero_off_the_active_band` holds the + consequence. +- **Limitations.** Native does not call the module yet (phase 3). +- **Cost.** `tdse_source.py` 937 lines against `numpy_ref/tdse.py`'s 845 (+92: + `node_fields` and the private transforms); `numpy_ref/backend.py` +3 net; tests + +308 (new module) and +6 elsewhere. +- **Handoff.** Native's hook calls `node_fields`, the binding, `atomic_sources`, + `atomic_edge_source` and `project_atomic_sources` with its own stage scalars and + prefactor, and finishes the stage itself (§4). + +### Phase 2 — The provider worker + +- **Scope.** Phase 2, 2026-11-12, Opus 5.5 (AI), commits `0f797abb` (`_interrupts`), + `ea431aa6` (the generic worker), `8f5b64bf` (Pyroraptor in the worker) and + `63f43686` (the trial's release), then `origin/main` merged (`7f985eb1`, with + ADR-077's change, #767), and a follow-up commit on the pull request (a fresh + worker after any failure, the documentation build, small simplifications). +- **Delivered.** + - `src/solprop/driver/_source_worker.py` (new), the provider-agnostic layer of §6.5: + `Worker` starts `sys.executable -m solprop.driver._source_worker ` + with `subprocess`, in a session of its own (`CREATE_NEW_PROCESS_GROUP` on Windows), + with the parent's environment and import path; the worker moves the protocol off + descriptors 0 and 1, points standard output at standard error, ignores SIGINT and + SIGTERM, and its reader thread is the watchdog: end of file on the parent's pipe + exits the process at once (`os._exit`), during a call too. Frames are an 8-byte + length and a payload; a message is a JSON control frame and one raw frame per array, + named by dtype (byte order included) and shape; nothing is pickled; every declared + length is checked against the frame bound before anything is read. Values cross as + tagged JSON: tuples, mappings, non-finite floats, arrays, and records (a dataclass's + or a plain object's fields, received as `Record`, with values an adapter computes + beside them). The handshake carries the protocol version, the worker's process id and + the adapter's provider identity. Waiting is a reader thread and a queue polled every + `POLL_INTERVAL` (0.2 s): between polls signal handlers run, a signal a commit holds is + raised (`_interrupts.raise_held`, new) and the run's `StopFlag` is polled; any + interruption kills the worker's process group at once. `close` between calls asks + the worker to exit and kills it after 5 s; every path reaps it and closes both pipe + ends. `worker_environment` drops the OpenMP wait variables solprop-native set for + itself (its marker `SOLPROP_NATIVE_WAIT`), keeps a user's own, and adds explicit + settings. + - `src/solprop/driver/_pyroraptor.py` (new), Pyroraptor's adapter: `_load_provider` + (now only ever called in a worker), the build identity, the haCC surface, records + and objects, the SAE and haCC preparation, plan building and its checks, the preset + and basis queries, the atomic calls and the consistency diagnostic's temporary plan, + and the classification of provider exceptions. The manufactured provider of the + tests is passed to `PyroraptorAdapter(provider)`. + - `src/solprop/driver/tdse.py`: the binding, `basis_from_preset`, `hacc_preset`, + `hacc_basis` and `DiagnosticSession` talk to the adapter in a worker + (`_open_provider`); the queries use a short-lived one. The handshake's identity is + checked against `_PROVIDER_REVISION` and `TDSEExecution.device` before any request + (`_build_record`) and is the run's build record, unchanged in form. Budgets, + reservations, the archive, the validation and normalisation of every result, the + domain flags and the unqualified-model warning stay in solprop's process, on plain + records. `prepare_binding` and `DiagnosticSession` take the run's `stop`, which + `Simulation` passes. A stop, Ctrl-C or SIGTERM during a call leaves the call charged, + releases the reservation, records the evaluation in the archive as raised with + classification `interrupted`, and propagates; it is no provider failure. The memory + ledger gains `transport.solprop` and `transport.worker` (the largest chunk's drive + twice and its result once, in each process; `tdse_transport.*` in a run's memory + rows) and the estimate `estimate.transport`, admitted before the worker starts. + - `src/solprop/backends/host/adams.py`: a trial's reservation is released on every + exit (`_trial`), a failed release attached to the trial's own error as a note + (§6.3). + - Tests: `tests/driver/test_source_worker.py` (21, real workers), + `tests/driver/test_tdse_worker.py` (8, real workers), + `tests/source_adapters.py` (the manufactured adapters the worker imports), + `tests/source_channel.py` (the adapter served in this process through the same + codec, for the existing tests whose fake providers are built inside the test), two + failure-boundary tests in `tests/backends/test_tdse_march.py`, one in + `tests/driver/test_interrupts.py`; the existing binding tests re-pointed from + `tdse._load_provider` to the in-process channel, their claims unchanged. + - `docs/guide/media/tdse.md` (the worker, its threads, stopping during a call), + `docs/dev/architecture.md`, `changelog.d/tdse-provider-worker.changed.md`. +- **Deviations.** + - Three checks that need the provider's own classes or methods run in the worker, on + the raw result, and their outcome crosses with it: the result's type is named there + (`sae.grid_response`, `hacc.total_current_response`, ...), a haCC result's atom, + basis and gauge are compared there with the model's (the provider's equality), and + the shadow's `component` is evaluated there for the components the binding reads. + Everything else of validation and normalisation is the binding's, as §6.5 says. The + plan checks of preparation and the two memory admissions between model and plans + (SEP 20, C7) run in the worker too, as part of preparation; a `DriverError` crosses + by name. + - `hacc_basis` takes the caller's provider objects, which cannot cross unchanged: they + travel as records of their dataclass fields, and the worker checks their types by + qualified name. + - The archive records an interrupted evaluation as raised with classification + `interrupted`, its outcome field unchanged (`returned` or `raised`), so the archive + format does not change. A worker that exits or breaks the protocol is classified + `worker`, a new value. + - A stopped TDSE run ends with status `failed` and the message `StopRequested: ...`, + as every stopped run does (§5.1 says "interrupted or stopped"). +- **Decisions.** + - A worker is never used again after a failure of any kind, as §6.5 says: a + provider exception the worker reports, a result the binding refuses, a worker exit, + a protocol error or a cancellation each end it, and the binding's next evaluation + or diagnostic starts and prepares a fresh one, which must report the same build + (`_ProviderBinding._ensure_worker`). Decided by the maintainer on 2026-11-12, over + the implementer's first proposal to keep a worker serving after a reported provider + exception. A retry after a failed chunk therefore pays a worker start and the + model's and plans' preparation again; the budget, ledger, archive and records carry + over. `DiagnosticSession`, which makes one call, does not restart. + - The thread count is `PYRORAPTOR_THREADS`, the run's `thread_budget().threads`, in + the worker's environment: Pyroraptor reads it once at import. A device build needs + no setting. + - The worker selects its adapter as `module` or `module:factory`, so the tests' + manufactured adapters are named, not configured by environment. + - The worker ends with `os._exit` after the adapter has closed its plans and setup, + skipping interpreter finalisation. + - The worker runs as `-W ignore::RuntimeWarning:runpy`: `solprop.driver` imports + the module before `-m` runs it, which runpy warns of; `__main__` delegates to the + imported module, so the records an adapter receives are its classes. +- **Pyroraptor.** No companion change was needed: its results, monitors, outcomes and + plan records are frozen dataclasses of arrays and plain values, which cross as + records; its CPU thread count is already explicit at import (`PYRORAPTOR_THREADS`); + its atomic calls release the GIL (`nb::gil_scoped_release`), so the watchdog runs + during a call; and the worker skips interpreter finalisation. No real Pyroraptor + build was installed here, so no real call was made (§7.4 stays for phase 4). +- **Verification** (§7.2, manufactured providers in real workers unless noted): + 1. evaluations through the worker equal the same manufactured haCC provider served in + this process, bit for bit: currents, validity, monitors and outcomes, the ledger + and the whole metadata (`test_evaluations_through_the_worker_equal_the_provider_ + called_directly`); arrays of several dtypes and byte orders cross unchanged; + 2. the handshake refuses a protocol mismatch, a stale revision and another execution + space by name, the worker reaped; + 3. a provider exception (by type, message, classification and traceback), a worker + exit (its status and standard error's tail), a closed pipe, a malformed frame, an + oversized frame, a start-up failure and a preparation failure each end as their + named failure, the worker reaped and both pipe ends closed; after a provider + exception the binding's retry runs on a fresh worker + (`test_after_a_provider_exception_the_retry_gets_a_fresh_worker`); the provider's + printing on standard output never corrupts a frame; + 4. a hanging call is cancelled by the stop flag, SIGINT, SIGTERM and a SIGINT held by + `committing()`, each within the poll interval and the kill (a 10 s harness + deadline); on `numpy_ref` a binding stopped during a call keeps the call charged + with no failure recorded, and a run stopped during its thirteenth atomic call ends + with 13 calls attempted, none failed, and a terminal checkpoint at its last + accepted step (`z > 0`); + 5. a Ctrl-C to the parent's whole process group interrupts the run and spares the + worker, which the parent then ends; a parent killed with SIGKILL during a hanging + call leaves no worker (the watchdog); + 6. the worker loads neither `solprop_native` nor `numpy_ref` nor `pyroraptor`, and its + `__main__` is the worker module, started from a test, from a script that refuses + to be imported twice and from `python -c`; after a run through the worker neither + `pyroraptor` nor the manufactured provider's module is in solprop's `sys.modules`; + 7. the transport rows are in the ledger and the estimate, and an admission over + `memory_limit` is refused before any worker starts; + + and §6.3 on `numpy_ref`: an archive failure in `record_trial` after an accepted + commit leaves the new state, booked, with nothing reserved; a failed release is a + note on the trial's own error. Run on the Mac: the new modules, the existing TDSE + modules of `tests/driver/`, `tests/backends/` and `tests/output/test_tdse_archive.py`, + `tests/test_structure_figures.py`, `tests/test_seps.py`: 958 passed, 3 skipped, and + 4 failed, all native-only (`test_tdse_march.py`'s native refusals and the shared-RK + refusal): the editable solprop-native here was built before the merge of `main` and + refuses to load ("built from other sources than this checkout's native/"); it was not + rebuilt (phases 0–2 change no native code), so CI decides them. + `tests/driver/test_interrupts.py`'s native case fails for the same reason. Ruff, + basedpyright and numpydoc clean on the changed files; `poe size-check` within budget. +- **Limitations.** The isolation from a scan executor's worker process (§7.2 item 6) + is not tested; Windows paths (`CREATE_NEW_PROCESS_GROUP`, `TerminateProcess` instead + of a group kill) are written but run only in CI. The worker ignores SIGINT and SIGTERM + from the start of its `main`, after interpreter start-up; in its own session no + terminal or group signal reaches it before then. A provider that spawns processes of + its own outside the worker's process group is not ended by the kill. +- **Cost.** `_source_worker.py` 761 lines, `_pyroraptor.py` 786 (part of it moved from + `driver/tdse.py`, which went from 2,575 to 2,506); across phase 2, `src/solprop` + +2,172 −682 and tests +900 −30 (physical lines, `git diff --numstat`, before the + follow-up commit). By `tools/size_report.py`'s split, the files phases 1 and 2 + touched went from 11,071 to 12,107 code lines (+1,036), 5,613 to 5,950 docstring + lines and 447 to 493 comment lines; the driver package from 15,296 to 16,290 code + lines. §6.8 estimated +0.6k source and +0.6k test lines for the worker; the excess is + the adapter's request handlers and the parent's view of the haCC model beside the + generic layer, as the binding's validation stayed in solprop's process. +- **Review follow-up** (an independent review of the pull request, 2026-11-12). The + worker's reader queues a failure for any exception, so a reply it cannot decode (a + bad tag, nesting too deep, memory) ends the wait as a protocol error instead of + leaving it polling; the worker's watchdog and its `main` leave through `os._exit` on + every path, never through interpreter finalisation with the provider loaded; a write + to a worker that died between calls is the worker's exit, with its status and + standard error; `close` kills the worker even when a signal interrupts its grace; the + reader threads hold the queue and stream, not the worker, and a finaliser kills a + worker its owner drops; a reader still blocked in its pipe leaves the stream + unclosed rather than blocking `kill`; a stop already asked for owns a worker exit + seen after it; the worker starts with `-P` and with solprop's BLAS idle defaults + (`child_environment`); the diagnostic closes its plan only on a live worker. + `_interrupts.raise_held` acts only inside an `interruptible()` scope on the main + thread, which native's session will open around its `advance` in phase 3, so any + other commit, the terminal checkpoint's `committing(deliver=False)` among them, + keeps or drops a held signal as before. The manufactured hangs are bounded (30 s), + so a broken cancellation fails a test rather than hanging it; new tests cover the + undecodable message, the dead worker, the interrupted close, collection, the + interruptible scope and its absence, another thread, and solprop-native's own wait + settings (`_wait.py`, read by path). +- **Handoff.** Phase 3 (the native hook) calls `tdse_source` and the same binding; its + waits are inside `committing()`: the session wraps `advance` in + `_interrupts.interruptible()` too, and `Worker` raises a held signal there; §7.2 item 4 on native and §7.3 follow with it. The worker's + `request` raises `StopRequested`, `KeyboardInterrupt` or `RunTerminated` unchanged, + which the hook stores and the session re-raises (§6.3). + +### Phase 3 — The native hook + +- **Scope.** Phase 3, 2026-11-12, Opus 5.5 (AI), on the branch after the native + restructuring (#770, merged in at `42e3b22b`); one commit (below, with this report). +- **Delivered.** + - C++: `include/solprop_native/host_source.hpp` (new), the generic `HostSource` + with `begin(cost)`, `evaluate(field, z, purpose, source)` and `end(outcome)`, and + `HostSourceError`; nothing in it names TDSE or Pyroraptor (P10). The march + (`adams_march.cpp`) calls `Engine::begin_attempt` after its budget check and + `end_attempt` after each decision -- after the commit for an accepted attempt -- + through an `Attempt` guard that ends an attempt abandoned when it leaves without a + decision (a failure, a floor, a diverged replay); a host failure is + `Category::host`, naming its stage, `none` for `begin`. `ResidentEngine` forwards + both to the bound source and names each evaluation's position and purpose (the + startup at `z`, the predictor at `z + dz`). `GeneralState::bind_host_source` + allocates the hook's two rows once, the field and the source, listed as + `host_source` in the memory ledger; the stage's entry kernel + (`StageTerms::combine`) writes the lab-frame field into the field row, unscaled + and with any noise added, the workspace's stage core forms the in-stage absorber + as every route does and calls the source from the calling thread outside every + parallel region, and the stage finish (`StageTerms::finish`) takes the source with + the collar alone, skipping the engine's own response pass. Host builds only: a device build throws, and its + adapter refuses the problem first. + - The binding layer (`bindings.cpp`): `PythonHostSource` calls a Python object's + three methods under the interpreter lock the march released, handing `evaluate` + the two rows in place (the field read-only), catches `nb::python_error` there and + reports it to the march as `HostSourceError`; the state holds the object + (`State.bind_host_source`), the engine borrows it. + - Python adapter: `solprop_native/_host_source.py` (new), `TDSEHostSource`: `begin` + reserves, `evaluate` is `node_fields`, the binding's evaluation and the shared + assembly and projection with the reference's own scalars (`scalars(problem, z, + z)`, plain helpers) and prefactor (`prefactor_at` or the combined z table), `end` + records the outcome and releases in a `finally`; every exception is kept, the + first as the error and later ones as notes, and a new attempt starts with none. + `NativeBackend` takes `tdse_binding`, binds the hook for its life (closing the + state drops it, a leased one too), refuses a TDSE problem without + its binding (and the reverse) and the statistics the TDSE response does not define, + records `atomic_calls`, `tdse_execution`, `tdse_last_evaluation` and `tdse_source` + (now `tdse_source.source_record`, shared with the reference), lists the binding's + rows as `tdse_*`, and restores an atomic ledger into the binding. The session opens + `_interrupts.interruptible()` beside its commit around `advance` when a source is + bound, and reports a host failure as the kept exception, a reservation the budget + refused rewrapped with the last accepted step's position and field as the reference + does; a kept exception the march did not stop for, an abandoned attempt's failed + release, rides on the run's error as a note, as the reference's `_trial` attaches + it. + - solprop: the registry hands a TDSE binding to either backend; the driver's + `numpy_ref`-only rules for a TDSE run (`_admit_tdse_execution`, + `_check_tdse_execution`) are gone, so `backend="auto"` runs it on native; the + capability rule `tdse-numpy-ref` is removed. + - Tests: `tests/backends/test_tdse_native.py` (23); the registry, entry, defaults + and checkpoint tests that held the old refusals rewritten. + - Documentation: `docs/dev/kernel-contract.md` (§1's `TDSESource`, the source + reservations, §6's host source hook), `docs/dev/backends.md`, `AGENTS.md` and + `native/AGENTS.md` (the hook's exception to "no Python call in a trial step"), + the TDSE manual and *Choosing a backend*, `changelog.d/tdse-on-native.changed.md`. +- **Deviations.** + - The march's `end` takes an outcome, `rejected`, `accepted` or `abandoned`, not a + boolean: an attempt that leaves without a decision must still release its + reservation, which the reference does in a `finally`, and is not recorded, as the + reference records none. Three entry points remain. + - A host failure is the march's `error` reason with a new category, `host`, and its + stage, rather than a reason of its own: the session already books a failed batch + and raises its error, which is now the kept exception. + - A signal taken inside the hook's wait ends the native session as its terminal + error, the batch booked and the state eligible for its checkpoint, where the + reference's session lets the signal through to the run loop; the run loop raises + it either way. + - §7.3 item 1 reads each engine's history row zero after the first attempt, not + the stage output alone: native has no stage output outside a run. Without the + in-stage absorber the initial step is accepted on both. In the exponential + formulation the row is the startup output, equal bit for bit. In the interaction + formulation it is that output carried to the step's end, by each engine's own + complex product, so it is within 4 eps of each value: bit for bit in CI's macOS, + Linux and Windows x86-64 lanes, one ulp apart on Windows on Arm. + With the absorber, the collar rejects the initial step, the row is the startup + output, and it is within one transform's bound, peak-normalised. + - Item 4 is two runs: eight steps of 2^-27 m without the in-stage absorber (cap and + length binary-exact, so no final step is split), within the eight-step 128 eps; + and, with the absorber, whose collar holds the steps to about a + two-hundredth of the run, 0.2 um in some four hundred attempts within the + attempts times the one-step 16 eps. Each checks that the source moves its field a + millionfold beyond the budget. + - The memory row is the generic `host_source`, not §6.2's `tdse_host_source`: the + engine knows no TDSE (P10). It holds §6.2's two `n_omega` rows. + - The reference records an accepted replayed step after releasing its reservation; + native's `end` records, then releases, for every decided attempt. The hook is not + told that a step is replayed, and the two orders differ only when the record or + the release fails. + - Item 5 is met through the driver: the TDSE run tests that ask for `auto` + (`test_tdse_hacc_run.py`, `test_tdse_archive_run.py`, `test_tdse_entry.py`, the + gated-derivative and natural-output runs) now run on native, unchanged; those + that name `numpy_ref` keep it. No module was duplicated over both backends. +- **Decisions.** The Runge-Kutta and low-storage paths are not guarded against a + bound host source in C++: the translation admits a TDSE response with the adaptive + `Multistep()` only, and a zero-length Runge-Kutta trial with a source bound does not + give the reference's stage (above), so a later guard should refuse it there. +- **Verification.** Native CPU build (Apple clang, editable, the `estimate` planning), + Apple M5 Max. `tests/backends/test_tdse_native.py`, its manufactured remainder + 1e-18 A m, strong enough that `F` moves every compared field seven orders or + more beyond its budget: through the C++ gather and the binding, in both formulations, on + the gated and cubic descriptors, with and without input noise, the field the hook + receives and `F` bit for bit, the history's first row as above; the call sequence + (every reserve, evaluation purpose, record and release, in order), the atomic ledger + and the `atomic_calls` and `tdse_source` records equal to the reference's on a whole + run in both formulations, the evaluations' positions within 1e-9 relative and the + final fields within the per-attempt budget; a budget refusal, a provider failure, a + SIGTERM before the commit (raised as an exception, and a real signal the commit + holds, raised by the evaluation's `raise_held` inside the session's + `interruptible`), and an archive failure after it, each leaving the stated state + with nothing reserved and the reference's calls, the archive failure's terminal + state carrying the committed step's proposal; an abandoned attempt's failed release + as the reference's note on the floor's error; the eight-step and collar-held runs + within their budgets, with one attempt per call and the production bound (measured + 5e-5 eps against 128 eps, and 5 to 6 eps against 6,480); a device build and a + missing binding refused by name; the memory rows; closing a backend whose state a + session leases, after which the binding, held in a cycle through the kept + exception's traceback, is collected (the state's own reference, which the collector + cannot see, is dropped once the state is closed). The TDSE, worker and interrupt modules of `tests/backends` and + `tests/driver`, the compiled-multistep, resume-ledger, capability, structure and + SEP modules: 1,411 passed, 3 skipped; `native/tests/test_backend.py` and + `test_session_failure.py`: 61 passed. Non-TDSE runs, before (the #770 + build) and after, `SOLPROP_FFTW_PLAN=estimate SOLPROP_CALIBRATION=off`, one thread: + the final fields of `gnlse_soliton` (default), `rdw_one_mode` (both Adams formulations), `argon_blueshift`, `radial_filament_256` and `dense_output`, each over 0.5–2 % of its length, are identical bit for bit. `poe fmt-native`; `poe tidy-native` on the changed C++ files: + no finding. No CUDA or HIP compiler here: the device builds are left to CI's + device-compile lanes. +- **Limitations.** TDSE on a native device build is refused by name (§12, Q2). The + hook is not exercised by a real provider (§7.4, phase 4). +- **Open questions.** Q12, closed by the review below: the rows had been read at + different points (the reference's after an acceptance had carried it, native's + after a rejection), and at matched points the reviewer found them 1.3e-11 apart. The + mismatch hid the true one: native's `F` never reached its history. +- **Cost** (phase 3, `git diff --numstat` from `42e3b22b`): C++ +332 −11, native Python + +251 −12, `src/solprop` +49 −58, tests +464 −30. §6.8 estimated +0.3k C++, +0.4k + native Python and +0.5k tests for the hook. The review's fixes: C++ +51 −35, + native Python +25 −4, tests +333 −79. +- **Review** (independent, of `478b3808`; fixed in the following commit). + - Critical: the binding handed `evaluate` ownerless NumPy views, which nanobind + copies when it passes them to Python, so `out[:] = source` wrote into a copy and + native's TDSE stages were the window times zero plus the absorber. The views now + have an owner, a capsule that frees nothing; the field is read-only. + - Critical: the stage core gathered the field from the fine spectrum, which holds + it divided by `n_to` (and doubled on an analytic grid's interior), so the hook saw + the field divided by 384. The entry kernel now writes the unscaled stage input, + noise included, into the hook's field row; the stage core reads that row. + - High: the tests could not see either defect. Item 1 compared the Python adapter + with itself; the runs' remainder of 1e-27 A m moved the field by 581 eps against + a budget of 6,480 eps. The tests above replace them. + - Medium: `NativeBackend.close` unbound the hook first, which a leased state + refuses; it now only closes the state. + - Low: a kept exception the march did not stop for was neither taken nor reported, + and stayed for the next attempt; it is now a note on the run's error and cleared + at each attempt's start. A failed `end` after an accepted commit returned before + the step's proposal, count and replay cursor were kept; they now are. + - Each fix was checked by putting its defect back and running the module: the + ownerless views fail 14 tests (item 1 in every case, both run budgets, both call + sequences and the floor's note); the scaled field fails item 1 in all 8 cases + (the manufactured remainder does not depend on the field, so the runs cannot see + a scale); the unbind fails the close test; the missing note fails the note test; + the session without `interruptible` fails the held-signal test; and the lost + proposal fails the archive test. Non-TDSE runs stay bit for bit as before. +- **Handoff.** Phase 4: the manual check of §7.4 through the worker on both backends, + the ADR, the remaining pages, and the size comparison. + +### Phase 4 — Qualification and documentation + +- **Scope.** Phase 4, 2026-11-12, Opus 5.5 (AI): the review's two follow-ups to phase 3, + the ADR, the remaining pages of §8, the size report against §6.8, and the manual + check of §7.4, its procedure and its results. The SEP stays `In progress`: it + becomes `Final` after CI and an A/B. +- **Delivered.** + - Phase 3 follow-ups: + - `NativeBackend.close` now drops the host source once the state is closed + (`bind_host_source(None)`, which a closed state allows). Before, the state kept a + reference that the garbage collector cannot see. A cycle through it (a kept + exception's traceback reaching the session) was never freed, and the held-signal + test leaked eight nanobind instances. + - That test now checks, with a weak reference after `gc.collect()`, that the + binding is freed, and it fails without the fix. A session that could not close + keeps the source it still uses. + - Item 4's eight-step run actually took nine attempts, all accepted: 8e-8 m in + 10 nm steps left a last step one ulp over the cap, which the march split in two. + The cap and length are now binary-exact (2^-27 m and 2^-24 m), so the run is + eight accepted steps. The comment no longer counts the startup as an attempt. + - ADR-078 (`decisions.md`) covers: + - the host source hook, its scope and the device refusal; + - the provider worker; + - the failure boundary; + - `auto`. + + It amends ADR-077's "every feature except the TDSE response, for now", and + ADR-077's status now says so. The maintainer accepted it on 2026-11-12. + - No provider revision is pinned (maintainer, 2026-11-12): until Pyroraptor has a + release, solprop uses whatever provider is installed, by default the latest `main`. + `_PROVIDER_REVISION` and `pinned_revision` are gone, and the handshake no longer + refuses another revision. It still refuses another protocol version, and an + execution space other than `TDSEExecution.device`. The build record keeps the + revision, build and execution space the worker reports (no `source_pin`); a + preset's or a basis's `source` names the reported revision, the consistency cache + keys on it (`tdse.provider_revision`), and the archives' contexts drop + `provider_pin`. Tests now check that another revision is recorded, not refused; + the installed-provider test runs with any revision; a release note says so. + - `testing.md` gains *Shared code and out-of-process providers*: the oracle boundary + for shared code and the worker-contract tests. `backends.md` and + `kernel-contract.md` now cite ADR-078. + - The other pages of §8 were done in phases 2 and 3: + - the TDSE manual and *Choosing a backend*; + - `backends.md`, `kernel-contract.md` §6 and `architecture.md`; + - `AGENTS.md` and `native/AGENTS.md`; + - the release notes: `tdse-on-native.changed.md` for `auto`, and + `tdse-provider-worker.changed.md` for the worker and cancellation during an + atomic call. +- **Size** against §6.8. Lines are `git diff --numstat` from the tip of #770 + (`1f00b8f7`, which contains `main`), so #770's own changes are not counted. Code lines + are from `tools/size_report.py`. + + | part | `src/solprop` | native Python | C++ | tests | + |---|---|---|---|---| + | estimate, total | +0.6k | +0.4k | +0.3k | +1.3k | + | lines | +2,448 −784 | +280 −12 | +349 −11 | +2,160 −66 | + | code lines | +1,052 | +166 | — | — | + + By part: + - The shared source is a move: +34 code lines with `node_fields`, and its tests +0.3k. + - The worker and adapter are `_source_worker.py` (522 code lines) and `_pyroraptor.py` + (605), less 136 from `driver/tdse.py`: about +1.0k code lines, which is the + `src/solprop` overrun. Their tests are +1.0k. + - The hook is +166 native Python code lines and +349 C++ lines, with tests of +0.8k. + + The worker is larger than §6.8 estimated. The binding's validation and normalisation + stay in solprop's process, as §6.5 asks, while the requests the parent's view of the + haCC model makes must be served in the worker. Phase 2's report records what was + examined for removal. +- **The manual check of §7.4: procedure.** With Pyroraptor's latest `main` (no revision + is pinned), installed into solprop's environment. + + 1. Set up the environment on the CPU. On macOS this needs Homebrew LLVM, `libomp` and + FFTW; on Linux, `CXX=clang++`, as Pyroraptor's installation guide says. + + ```sh + cd # this branch + uv sync # solprop and its native CPU build + git -C switch main && git -C pull + uv pip install --python .venv/bin/python # its CPU (OpenMP) build + uv run --no-sync python -c "import pyroraptor as p; print(p.build_info())" + ``` + + From here on, use `uv run --no-sync`: a plain `uv run` or `uv sync` removes + Pyroraptor. + 2. On a GPU machine, use the same solprop environment with its native CPU build (a + native device build refuses TDSE). Install Pyroraptor's device build into it in + place of step 1's, configured as its `scripts/device_build.sh` does: + + ```sh + cd + kokkos="$PWD/build/sources/kokkos-5.2.1" + test -f "$kokkos/CMakeLists.txt" || git clone --depth 1 --branch 5.2.1 \ + https://github.com/kokkos/kokkos.git "$kokkos" + # CUDA + arch=$(scripts/device_build.sh cuda arch auto) + NVCC_WRAPPER_DEFAULT_COMPILER=g++ CMAKE_ARGS="-DPYRORAPTOR_EXECUTION_SPACE=CUDA \ + -DPYRORAPTOR_DEVICE_ARCH=$arch -DCMAKE_CXX_COMPILER=$kokkos/bin/nvcc_wrapper \ + -DFETCHCONTENT_SOURCE_DIR_KOKKOS=$kokkos" \ + uv pip install --python /.venv/bin/python --reinstall . + # or HIP + arch=$(scripts/device_build.sh hip arch auto) + CMAKE_ARGS="-DPYRORAPTOR_EXECUTION_SPACE=HIP -DPYRORAPTOR_DEVICE_ARCH=$arch \ + -DCMAKE_CXX_COMPILER=$(command -v hipcc) \ + -DFETCHCONTENT_SOURCE_DIR_KOKKOS=$kokkos" \ + uv pip install --python /.venv/bin/python --reinstall . + ``` + + 3. Save this script as `sep49_check.py` in an empty directory outside the checkout: + + ```python + """SEP 49 section 7.4: a real Pyroraptor through the worker, native against numpy_ref. + + python sep49_check.py compare DEVICE one short haCC(1) fast segment on both backends + python sep49_check.py cancel DEVICE native, Ctrl-C (SIGINT) during an atomic call + python sep49_check.py refuse DEVICE native, a device the build does not have: refused + + DEVICE is cpu, cuda or hip, matching the installed Pyroraptor build (for refuse, one + it does not match). + """ + + import json + import os + import signal + import subprocess + import sys + import threading + import time + import warnings + from pathlib import Path + + import numpy as np + + from solprop.driver import Simulation, tdse + from solprop.driver.tdse import hacc_preset + from solprop.exceptions import BackendError, ResponseBudgetError + from solprop.nonlinear.tdse import ( + TDSEConsistency, + TDSEExecution, + TDSEGasResponse, + TDSESampling, + ) + from solprop.output import open_file + + LENGTH = 2.0**-24 # one step of 60 nm: a startup and a predictor call, then a retry + BUDGET = 3 # atomic calls, at most (SEP 49 section 7.4) + RTOL = 0.1 # loose: the one step is accepted, a check of the plumbing, not accuracy + DELAY = float(os.environ.get("SEP49_DELAY", "5")) # seconds into the first call + + + def simulation(backend, device, budget, length): + preset = hacc_preset(1, "fast") + response = TDSEGasResponse( + "helium", + preset.basis, + preset.max_atomic_step, + "tdse_native", + 1.0, + preset.calibration(), + TDSESampling(4), + model="hacc", + # No weak-field consistency call: the budget is the march's alone. + consistency=TDSEConsistency(enforce=False), + ) + return Simulation.from_capillary( + 256e-6, + length, + "He", + 0.1, + wavelength=800e-9, + wavelength_limits=(400e-9, 1.6e-6), + trange=40e-15, + fwhm=6e-15, + energy=20e-6, + modes=1, + radial_rule="field_scale", + n_r=2, + noise=None, + n_save=2, + rtol=RTOL, + dz_init=LENGTH, + dz_max=LENGTH, + backend=backend, + tdse=response, + tdse_execution=TDSEExecution(device=device, max_atomic_calls=budget), + ) + + + def run(backend, device, budget=BUDGET, length=LENGTH): + path = Path(f"sep49-{backend}-{device}.h5") + path.unlink(missing_ok=True) + started = time.monotonic() + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + try: + simulation(backend, device, budget, length).run(path, progress="none") + except ResponseBudgetError as error: # the budget ends the segment + print(f"{backend}: {error}") + finally: + print(f"{backend}: {time.monotonic() - started:.1f} s", flush=True) + return path + + + def summary(path): + with open_file(path) as output: + outcome = output.outcome + plain = lambda value: json.loads(json.dumps(value, default=str)) # noqa: E731 + return { + "status": outcome.status, + "z": float(outcome.z), + "atomic_calls": plain(outcome.atomic_calls), + "logical_calls": plain(outcome.logical_calls), + "provider": plain(outcome.execution["tdse"]["build"]), + "fields": np.asarray(output.fields(convention="fft")), + } + + + def workers(): + """This process's provider workers and their CPU seconds.""" + table = subprocess.run( + ["ps", "-A", "-o", "pid=,ppid=,time=,command="], + capture_output=True, + text=True, + check=True, + ).stdout + found = {} + for line in table.splitlines(): + pid, ppid, cpu, command = line.split(None, 3) + if int(ppid) == os.getpid() and "solprop.driver._source_worker" in command: + parts = [float(part) for part in cpu.replace("-", ":").split(":")] + found[int(pid)] = sum(v * 60**i for i, v in enumerate(reversed(parts))) + return found + + + def compare(device): + results = {name: summary(run(name, device)) for name in ("numpy_ref", "native")} + reference, ours = results["numpy_ref"], results["native"] + for name, result in results.items(): + print( + name, + result["status"], + result["z"], + result["atomic_calls"], + result["logical_calls"], + ) + print("provider:", reference["provider"]) + if reference["logical_calls"]["accepted_steps"] == 0: + print("no step was accepted, so nothing to compare: lower LENGTH") + return 1 + first, last = reference["fields"][0], reference["fields"][-1] + gap = np.linalg.norm(ours["fields"][-1] - last) + difference = gap / np.linalg.norm(last) + print(f"final field, native against numpy_ref: {difference:.3e} relative") + print( + f"... and against the segment's change: {gap / np.linalg.norm(last - first):.3e}" + ) + keys = ("status", "z", "atomic_calls", "logical_calls") + same = all(ours[key] == reference[key] for key in keys) + print("same status, position and call ledgers:", same) + return 0 if same and difference <= 1e-9 else 1 + + + def cancel(device): + sent = {} + output = Path(f"sep49-native-{device}.h5") + output.unlink(missing_ok=True) + + # The binding's evaluation in this process is where an atomic call is in flight + # (a private name: this script checks solprop, it is not an API). + calling = threading.Event() + evaluate = tdse._ProviderBinding.evaluate + + def marked(self, *args, **kwargs): + calling.set() + return evaluate(self, *args, **kwargs) + + tdse._ProviderBinding.evaluate = marked + + def watch(): + calling.wait() + time.sleep(DELAY) + sent.update(worker=next(iter(workers()), None), at=time.monotonic()) + os.kill(os.getpid(), signal.SIGINT) + + threading.Thread(target=watch, daemon=True).start() + try: + # Many steps, so that a call is in flight whenever the signal comes. + run("native", device, budget=1000, length=1000 * LENGTH) + except KeyboardInterrupt: + stopped = time.monotonic() - sent["at"] + print(f"KeyboardInterrupt {stopped:.2f} s after SIGINT, during a call") + else: + print("the run finished before a call was interrupted: raise LENGTH") + return 1 + alive = sent["worker"] in workers() + print("worker", sent["worker"], "still running" if alive else "gone") + result = summary(output) + print("provider:", result["provider"]) + print( + result["status"], result["z"], result["atomic_calls"], result["logical_calls"] + ) + charged = result["atomic_calls"].get("attempted", 0) >= 1 + reserved = result["atomic_calls"].get("reserved") + return 0 if not alive and charged and reserved == 0 else 1 + + + def refuse(device): + try: + run("native", device) + except BackendError as error: + print("refused:", error) + return 0 if "device=" in str(error) else 1 + print(f"device={device!r} was not refused") + return 1 + + + if __name__ == "__main__": + mode, device = sys.argv[1], sys.argv[2] + sys.exit({"compare": compare, "cancel": cancel, "refuse": refuse}[mode](device)) + ``` + + 4. Run it from that directory. On the CPU build, `compare cpu` and `cancel cpu`; on a + GPU build, `refuse cpu` and `cancel cuda` (or `hip`), and `compare` only on a GPU + whose FP64 rate makes an atomic call short (not an A40). + + ```sh + uv run --no-sync --project python sep49_check.py compare cpu + uv run --no-sync --project python sep49_check.py cancel cpu + ``` + + `compare` runs one 60 nm step of haCC(1) fast at about 3e13 W/cm², with two radial + nodes, through the worker: first on `numpy_ref`, then on native. Each run makes at + most three atomic calls: a startup, a predictor and one retry. + - It passes (exit status 0) when both runs end with the same status, position and + call ledgers, and native's final field is within 1e-9 of `numpy_ref`'s, + relative. + - It also prints the difference against the segment's own change. + + `cancel` runs native over a longer segment and sends itself SIGINT 5 s + (`SEP49_DELAY`) into its first atomic call. It passes when: + - the run raises `KeyboardInterrupt` within about a second; + - the worker is gone and nothing is reserved; + - the file records the call as charged and the last accepted step's position. + + `refuse` asks for a device the installed build does not have (`cpu` with a CUDA + build, say) and passes when the run is refused by name. + 5. Record here, for each run: + - the provider revision and build (the `provider:` line); + - the machine; + - the threads (`SOLPROP_THREADS` or the machine's) or the device; + - both modes' output and exit status. + + Both modes were dry-run on an Apple M5 Max. The worker served the manufactured haCC + provider of `tests/hacc_cases.py` in place of Pyroraptor + (`solprop.driver.tdse._ADAPTER = "source_adapters:manufactured_tdse"`). + - `compare` took one accepted step on each backend with two atomic calls. The ledgers + were the same, and the final fields were 1.3e-13 apart, relative. + - `cancel` was run with every call after the second hanging + (`MANUFACTURED_HANG_AFTER=2`). It raised `KeyboardInterrupt` 0.11 s after the + signal: the worker was gone, three calls were charged, none were reserved, and the + run stood at its one accepted step. + + This checks the script and the plumbing, not Pyroraptor. +- **The manual check of §7.4: results** (2026-11-12, run by Opus 5.5 at the + maintainer's request, on a shared HPC cluster through Slurm). Done with this + procedure's script, and with `refuse` on the GPU. + - **Code.** solprop `7a5ca1f9a7e25c28007c8ee4768f834b6ffa4d51` (this branch). + Pyroraptor `7658b615ef99e169d30ff0b12526d2a8d8b638a8` (the latest `main`; reports + `source_revision='7658b61'`, version `0.0.1.dev26+g7658b615e`). + - **CPU builds.** + - solprop's native CPU wheel: LOCAL profile, Release, GCC 16.2, FFTW 3.3.11 with + OpenMP threads, OpenMP OpenBLAS, `-march=znver3`. + - Pyroraptor's OpenMP build: GCC 16.2, LOCAL, `x86-64-v3`, Kokkos 5.2.1. It was + installed into the same environment, so the worker ran the same interpreter. + - **CPU machine.** A node with two AMD EPYC 7763; a Slurm job with 32 cores + (`-c 32 --hint=nomultithread`). The worker reported `requested_threads=32`, the + run's whole budget. + - **`compare cpu`**: exit 0, wall 1,422 s. + - Both backends ended `complete` at z = 5.96e-8 m (2^-24 m). Each took one + accepted order-1 step: a startup call and a predictor call, 2 atomic calls, + none reserved. The call ledgers were equal. + - Native's final field was 4.1e-13 from `numpy_ref`'s, relative, and 1.4e-8 of + the segment's own change. + - Each backend's run took 707 s (`numpy_ref`) and 712 s (native). About 8.4 min of + that was the provider's model preparation in the worker; each atomic call took + about 100 s at 32 cores, below the 10 min bound set beforehand. The problem + therefore did not need reducing. + - **`cancel cpu`**: exit 0, wall 550 s. + - SIGINT came 5 s into the first atomic call (after the same preparation), and + `KeyboardInterrupt` was raised 0.13 s later. + - The worker was gone. The file records status `failed` at z = 0, with the startup + call charged (`attempted` 1) and nothing reserved. + - **GPU builds.** + - Pyroraptor's CUDA wheel: `PYRORAPTOR_EXECUTION_SPACE=CUDA`, `AMPERE86`, CUDA + 12.8.1 with `nvcc_wrapper` over system GCC 11.5, Kokkos 5.2.1. It was compiled on + the login node with 4 jobs, in 26 min. + - It was installed into a second environment beside the same solprop native CPU + wheel, whose `execution_space` is `OpenMP`. + - **GPU machine.** A node with NVIDIA A40s; a Slurm job with one A40, 16 cores and a + 20 min cap. The worker reported `execution_space='Cuda'`. + - **`refuse cpu`**: exit 0, wall 9.4 s. The run was refused by name ("TDSE requested + device='cpu', but the native build uses 'Cuda'"), and no worker remained. + - **`cancel cuda`**: exit 0, wall 681 s. + - The provider's model preparation on the A40 took about 11 min. SIGINT came 5 s + into the first atomic call, and `KeyboardInterrupt` was raised 0.47 s later. + - The worker was gone, and the device released: `nvidia-smi` listed no compute + process and 0 MiB used afterwards. + - The file records the startup call charged (`attempted` 1) and nothing reserved. + - **No GPU `compare`.** The A40's FP64 rate is about 1/64 of its FP32, so an A40 + atomic call could take hours, and SEP 49 does not change Pyroraptor's GPU + numerics. The GPU numerical comparison of native against `numpy_ref` is to be done + by hand when the provider is next used on a production GPU (Q10). + - **Logs.** The build logs, and both runs' logs with every command's output, are + kept with the maintainer's campaign records. +- **The wheel's strict-FMA check.** On the merge with `main`, CI's Linux wheel failed + `native/ci/check_strict_fma.py` with 36 fused multiply-adds "in" `march::reached`. + The disassembly of that wheel shows `reached` itself (0x5d bytes, its symbol's size) + with none. The 36 lie in 9 KB of unlabelled code that the link-time optimisation + placed after it: Kokkos kernels of the dispersive response, which call + `GeneralDispersive::respond`, `sin`, `tanh`, `pow` and `exp`. The wheel is stripped, + so `objdump` labels only exported functions and counted that code as `reached`'s. + No strict file includes a contracting header, and no strict code changed. The check + now bounds each strict function by its symbol's size from `objdump -T` and `-t`, + with a test. It passes on that wheel (333 strict functions, none fused). The + non-TDSE runs stay bit for bit as before. +- **Verification.** `tests/backends/test_tdse_native.py`: 23 passed. The held-signal test + no longer produces nanobind's leak report, and with the unbind removed it fails its + weak-reference check. The non-TDSE bitwise check is unaffected, because phase 4 + changes the native extension's comments only. +- **Handoff.** Next are CI and an A/B; then `Final`. The GPU numerical comparison stays + manual, for the provider's next production GPU use (Q10). + +### Completion — 2026-11-13 + +- **Coverage.** CI's coverage lane found the worker's own side uncovered: it runs in a + subprocess, which the parent's coverage does not see. Tests now run it in-process + (two pipes standing in for its standard streams, `os._exit` recorded): requests served + in order until `close`, adapter and codec errors reported, the provider's prints kept + out of the frames, signals ignored, a failed start reported in place of the handshake, + and the watchdog's exit on end of file. Real-worker tests cover a reply before the + handshake, a message out of turn, a close past its grace (killed), and a worker dying + mid-call (a charged `[worker]` failure with its stderr). Four defensive checks that + validation already guarantees became assertions, and a provider reply that is not a + record is now refused by name as a contract failure. `Worker._exited` waits for the + stderr reader once the worker has exited, so the last lines are never lost. +- **Performance (A/B, non-TDSE).** Baseline `main` against this branch, on one NUMA + domain of a 128-core AMD EPYC 7763 node, production threads, 3 rounds of 3 repeats, + noise ε from A/A runs on the same machine. Every case did the same work: + + | Case | Ratio | Min | Max | ε | Field difference | + |---|---|---|---|---|---| + | `argon_blueshift_envelope` | 0.964 | 0.958 | 0.989 | 0.064 | 0 | + | `rdw_one_mode` | 1.016 | 0.996 | 1.037 | 0.039 | 5.5e-15 | + | `kerr_beam_192` | 1.003 | 0.996 | 1.013 | 0.007 | 1.4e-16 | + | `radial_filament_256_saved_hdf5` | 0.973 | 0.967 | 1.021 | 0.032 | 0 | + | `he_rdw_averaged` | 0.998 | 0.992 | 0.999 | 0.020 | 5.6e-15 | + | `h2_raman_averaged` | 0.983 | 0.979 | 0.997 | 0.017 | 4.8e-15 | + | `ctc_reduced` | 0.949 | 0.911 | 1.048 | 0.052 | 3.6e-15 | + | `kerr_beam_384` | 1.002 | 1.000 | 1.019 | 0.007 | 0 | + + Geometric mean 0.986, against the limit 1 + mean(ε)/2 = 1.015: no regression. With + estimated FFT plans and calibration off, all seven cases of the bitwise check gave a + field difference of exactly 0, so non-TDSE runs are unchanged bit for bit. The + differences above come from measured FFT plans. +- **Open questions moved to issues.** The numerical comparison of native against + `numpy_ref` with the provider on a production GPU, left manual by Q10: + [#785](https://github.com/LupoLab/solprop/issues/785). The strict-FP check on object + files rather than the stripped wheel: + [#777](https://github.com/LupoLab/solprop/issues/777). +- **Status.** `Final`, set in the last commit of #771; merging it completes the SEP and + closes [#764](https://github.com/LupoLab/solprop/issues/764). + ## Part III — History - 2026-11-12: Drafted. @@ -855,6 +1820,17 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. (human)); the status stays `Draft` until accepted. - 2026-11-12: Accepted by John Travers (#765); implementation to start once ADR-077's change (#767) is merged, in one implementation pull request (§11). +- 2026-11-12: In progress: Opus 5.5 (AI) on `feature/sep-0049-tdse-on-native`, based on + #767's branch; phases 0 and 1 reported in Part II. +- 2026-11-12: Phase 2 (the provider worker) reported in Part II; implementation pull + request opened as a draft. +- 2026-11-12: Phase 3 (the native hook) reported in Part II. +- 2026-11-12: Phase 4 reported in Part II, with the manual check of §7.4 on the CPU + (compare and cancel) and on an A40 (device refusal and cancel). ADR-078 accepted by + John Travers. The + maintainer decided that solprop pins no Pyroraptor revision until Pyroraptor has a + release: the installed provider is used, by default the latest `main`, and its + reported revision recorded; §6.5, §7.2, §7.4, §9.3 and Q10 revised to match. ### AI review — 2026-11-12 @@ -952,7 +1928,7 @@ no physical constant or TDSE model was requalified by this review. [`test_a_full_adaptive_run_meets_the_reference_s_accuracy`](https://github.com/LupoLab/solprop/blob/63201dca4ab69d21b6d4ae0cd4611c4203ff47b1/tests/backends/test_backend_ladder.py) compares against a tighter reference integration and checks a declared work band. Replaying native's own schedule cannot replace that claim. The analytic P0–P2, - three-level P3 and cross-scheme checks in [testing.md](../testing.md) must remain + three-level P3 and cross-scheme checks in [testing.md](../../testing.md) must remain separately identified class-T or analytic checks, not become matched-replay tests. **Recommendation and question:** compare a third option: a small, non-production diff --git a/docs/dev/seps/index.md b/docs/dev/seps/index.md index 237f5ba26..f74ac4c0b 100644 --- a/docs/dev/seps/index.md +++ b/docs/dev/seps/index.md @@ -33,7 +33,6 @@ Proposals that are active, in draft, accepted, in progress or deferred. | [11](sep-0011-finite-density-gas-index.md) | Finite-density gas index and mixture optics | Draft | Change | | [17](sep-0017-gas-two-photon-ionisation.md) | Broadband two-photon continuum response beside strong-field ionisation | Accepted | Feature | | [32](sep-0032-raman-maxwell-bloch.md) | Raman Maxwell–Bloch dynamics | Draft | Feature | -| [49](sep-0049-tdse-on-native.md) | The TDSE response on native | Accepted | Change | SEP 13, the standalone numerical prerequisite for SEP 12, SEP 14, which carried it to the compiled backends, and SEP 16, which unified the multistep family and made it the @@ -85,6 +84,7 @@ Proposals that are final, rejected, withdrawn or superseded; the | [46](archive/sep-0046-machine-calibration.md) | Calibrating the native backend to its machine | Final | Change | | [47](archive/sep-0047-run-ownership.md) | Backend-owned execution sessions | Final | Change | | [48](archive/sep-0048-julia-backend.md) | A Julia backend: one engine for CPUs and GPUs, in both precisions | Withdrawn | Feature | +| [49](archive/sep-0049-tdse-on-native.md) | The TDSE response on native | Final | Change | ```{toctree} :hidden: @@ -101,7 +101,6 @@ sep-0010-low-rank-raman sep-0011-finite-density-gas-index sep-0017-gas-two-photon-ionisation sep-0032-raman-maxwell-bloch -sep-0049-tdse-on-native template archive/index ``` diff --git a/docs/dev/testing.md b/docs/dev/testing.md index 50c6168b0..20f2e41be 100644 --- a/docs/dev/testing.md +++ b/docs/dev/testing.md @@ -176,6 +176,22 @@ The two thread settings matter: eighteen xdist workers each initialising an eighteen-thread Kokkos pool oversubscribe the machine, and OpenMP binding pins each worker's threads to the same first cores. +### Shared code and out-of-process providers + +Parity between the backends checks what each computes independently. It cannot check +code both call: a defect in the TDSE response's source assembly +(`solprop.backends.tdse_source`, [SEP 49](seps/archive/sep-0049-tdse-on-native.md)) is the same +on both. That module's tests state their expectations independently of it, analytic or +manufactured (`tests/backends/test_tdse_source.py`, `tests/edge_source_cases.py`), and a +deliberate defect in each term is shown to fail one of them; parity then covers what each +backend does with the source. The atomic provider runs in a worker process. Its contract +is tested with real workers serving manufactured providers (`tests/source_adapters.py`, +`tests/driver/test_source_worker.py`, `tests/driver/test_tdse_worker.py`): results equal +a direct call bit for bit, the handshake's refusals, every failure, cancellation by +signal and stop flag, and clean-up. These tests have hard deadlines, so a broken +cancellation fails instead of hanging; the production code has none. An installed +Pyroraptor is not tested in CI; it is checked by hand (SEP 49, section 7.4). + ### Budgets and timing assertions The medium tier's budget is tight: time it with `poe test-medium` before adding to it. Most diff --git a/docs/guide/media/tdse.md b/docs/guide/media/tdse.md index 09cf435cd..20a49a17b 100644 --- a/docs/guide/media/tdse.md +++ b/docs/guide/media/tdse.md @@ -13,9 +13,11 @@ electrons and the atomic resonances that a rate model omits. The physics in outl :class: note The atomic engine, Pyroraptor, is a separate, optional package. It is not yet publicly -available; it is planned to be. solprop declares no dependency on it and imports it only -when a TDSE response is prepared; without it, preparing a TDSE response raises a -`BackendError` naming the missing provider. The records, their validation, identities +available; it is planned to be. solprop declares no dependency on it and never imports +it into its own process: preparing a TDSE response starts it in a worker process of its +own, the same interpreter, given the run's whole thread budget (or, for a GPU build, its +device). Without it, preparing a TDSE response raises a `BackendError` naming the missing +provider. The records, their validation, identities and descriptors work without it. The TDSE response is a research feature, not a qualified replacement for the rate models. ``` @@ -233,8 +235,9 @@ run = prop_capillary( The response is the complete electronic response of the fill, so `plasma`, `raman`, `thg`, `preionisation` and the other ionisation settings must be left unset, or the call -is refused naming them. `backend="auto"` resolves to `"numpy_ref"`, the only backend that -runs a TDSE response; the run needs the adaptive `Multistep()` integrator and a single +is refused naming them. `backend="auto"` resolves to `"native"`, as for every run: the +native CPU build runs the TDSE response, and `backend="numpy_ref"` runs it too, while a +native GPU build refuses it by name; the run needs the adaptive `Multistep()` integrator and a single scalar HE₁₁ mode. A TDSE run carries no shot noise by default: it is expensive, rarely repeated as a noise ensemble, and its checks are coherent. Input shot noise (`noise="input"`) can be asked for, with a warning; its draw is part of the @@ -250,7 +253,10 @@ responses](../waveguides/radial-quadrature.md)). With `tdse=`, every accepted st saved and only the final plane is requested by default, so saving never forces an extra atomic call; an explicit `n_save`, `save_planes` or `step_planes` is honoured. Every attempted provider call is charged against `max_atomic_calls` before it is made, failures -included. A capped or interrupted run resumes; the gate and window are part of the resume +included. Ctrl-C, SIGTERM and a run's stop flag stop it during an atomic call as well as +between steps: the worker is ended at once, the call in flight stays charged, and the run +ends at its last accepted step with a terminal checkpoint. A call has no time limit. A +capped or interrupted run resumes; the gate and window are part of the resume identity, so a run never resumes under another one. ## The raw response archive @@ -316,7 +322,7 @@ against an exact solution. - **The optical band** reaches 120 nm (SAE) or 90 nm (haCC) numerically; that is not a claim of validated ultraviolet accuracy. - **Scope.** Helium only, one scalar HE₁₁ mode on the uncoupled modal capillary route, - `numpy_ref` only (the native backend refuses it by name). Double ionisation, ionic + `numpy_ref` and the native CPU build (a native GPU build refuses it by name). Double ionisation, ionic channels beyond $n=2$, ion motion, collisions, recombination and space charge are omitted. - **Cost.** Each source evaluation is a full atomic calculation at every node: minutes @@ -328,4 +334,5 @@ against an exact solution. See also §10.6 of the primer's [ionisation chapter](../../primer/ionisation.md), [radial nodes for expensive responses](../waveguides/radial-quadrature.md), and the API reference of {mod}`solprop.nonlinear.tdse`, {mod}`solprop.driver.tdse` and -{mod}`solprop.driver.tdse_consistency`. +{mod}`solprop.driver.tdse_consistency`. The native backend's TDSE response and the provider's +worker process are [SEP 49](../../dev/seps/archive/sep-0049-tdse-on-native.md). diff --git a/docs/guide/performance/backends.md b/docs/guide/performance/backends.md index 2239e4e4b..05fecb6c7 100644 --- a/docs/guide/performance/backends.md +++ b/docs/guide/performance/backends.md @@ -24,7 +24,6 @@ repeat the first: |---|---|---|---|---| | `auto` | `native` | refuses, naming the missing features or the failed check | the load's own error, unchanged | refuses, saying how to install it | | `native` by name | `native` | as above | as above | as above | -| TDSE response, `auto` | `numpy_ref`; native isn't consulted | the same | the same | the same | | `numpy_ref` by name | runs; native isn't consulted | the same | the same | the same | `"auto"` has no fallback. These cases stay distinct, so an import failure is never reported @@ -79,16 +78,10 @@ and what to use for scans are on [Performance tuning](tuning.md). ## Features that run on the reference backend -`numpy_ref` is the numerical reference, and it is also the supported way to run these -features, which native does not run yet. Each is a limit of native, and each stays -supported on the supported platforms: - -| feature | reached by | tracked by | -|---|---|---| -| the [TDSE response](../media/tdse.md) | `auto` or by name | | - -The TDSE row's `auto` is that of the entry points; `SimulationSettings` built by hand with -`tdse_execution` [names the backend](../results/interior-launch.md). Running `numpy_ref` by +`numpy_ref` is the numerical reference, and it is also the supported way to run any +feature native does not run yet. None is listed today: the +[TDSE response](../media/tdse.md), the last, runs on the native CPU build too (a native +GPU build refuses it by name). Running `numpy_ref` by name on an unsupported platform may keep working, without a support promise. The list is not a general waiver: a new feature still lands on both `numpy_ref` and `native` ([SEP 0](../../dev/seps/sep-0000-process.md)), and joins the list only as an accepted diff --git a/native/AGENTS.md b/native/AGENTS.md index b15668949..aa858f4a1 100644 --- a/native/AGENTS.md +++ b/native/AGENTS.md @@ -22,8 +22,8 @@ contract and its conventions; where both apply, follow the stricter rule. C++ API are private. - solprop has two backends, this one and its NumPy reference, and computes in IEEE binary64 only (solprop's ADR-077): every field, stage and statistic here is double - precision, with no single or mixed mode. This backend runs every feature except the - TDSE response, on CPUs and GPUs. + precision, with no single or mixed mode. This backend runs every feature on CPUs and + GPUs, the TDSE response on CPU builds only (solprop SEP 49). - The layers are solprop -> Python backend adapter -> nanobind seam -> a Python-free C++20 core. Native code never imports Python or solprop concepts and never owns user-facing policy: it may execute an algorithm the kernel contract specifies in @@ -128,7 +128,10 @@ the gate. a device kernel's functor as its launch argument; team-level loops stay `Kokkos::`. - Allocate field-sized views, scratch and FFT plans once, when a state is constructed. A trial step performs no allocation, planning, logging, Python call or unplanned - fence. + fence. The one exception is the host source hook (solprop SEP 49, + `include/solprop_native/host_source.hpp`): on a CPU build, a bound host source is + called from the thread that called `advance`, outside every parallel region, and the + binding layer takes the interpreter lock for each call; device builds refuse it. - Keep data resident in the execution space until it is observed; `field_to_host` is the explicit synchronous boundary. Every persistent allocation appears in `memory_ledger()` with its real size in bytes. diff --git a/native/ci/check_skips.py b/native/ci/check_skips.py index 856e8ae73..aa62aa944 100644 --- a/native/ci/check_skips.py +++ b/native/ci/check_skips.py @@ -29,6 +29,7 @@ r"the Slurm \(and its Queue\) executor is POSIX only and refuses Windows", r"the Queue and Slurm executors are POSIX only", r"POSIX process signal delivery", + r"POSIX signals and groups", r"POSIX SIGKILL", r"POSIX shared locks", r"Windows has no SIGKILL", @@ -40,7 +41,7 @@ r"the real Windows query", r"NumPy's interp may fuse off x86-64", # Other backends and packages a lane doesn't install. - r"Pyroraptor at the pinned revision is not installed", + r"Pyroraptor is not installed", # A lane on native has its wheel; these are the no-native lane's. r"solprop-native is installed", # Parametrized cases that apply to one backend's route only. diff --git a/native/ci/check_strict_fma.py b/native/ci/check_strict_fma.py index 5c513c070..fa97343cb 100644 --- a/native/ci/check_strict_fma.py +++ b/native/ci/check_strict_fma.py @@ -17,6 +17,13 @@ It also fails when it finds none of the namespaces' functions, so that a change in symbol names or visibility cannot make it pass by finding nothing. +A wheel's extension is stripped: its disassembly labels only the exported functions, +and the code a link placed after one of them without a symbol of its own (another +file's local functions, under link-time optimisation) would read as that function's. +Each strict function is therefore bounded by its symbol's size from the dynamic and +static symbol tables (``objdump -T`` and ``-t``); a label whose size is unknown keeps +every line up to the next label. + Standard library only; it needs GNU ``objdump`` and reads x86-64 and AArch64 code. """ @@ -29,7 +36,7 @@ import zipfile from pathlib import Path -__all__ = ["STRICT", "fused_functions", "main"] +__all__ = ["STRICT", "fused_functions", "main", "symbol_sizes"] STRICT = re.compile( r"^[0-9a-f]+ :$") +LABEL = re.compile(r"^([0-9a-f]+) <.*>:$") +INSTRUCTION = re.compile(r"^\s*([0-9a-f]+):") +SYMBOL = re.compile(r"^([0-9a-f]+)\s.*\s\.text\S*\s+([0-9a-f]+)\s") +"""A code symbol in ``objdump -T`` or ``-t`` output: its address and its size.""" FUSED = re.compile( # x86-64: the FMA3 and AVX-512 forms (vfmadd231sd, vfnmsub132pd, vfmaddsub213pd). r"\tv?fn?m(?:add|sub)(?:sub|add)?\d{3}[ps][sdh]\b" @@ -46,22 +56,43 @@ ) +def _objdump(*arguments: str) -> str: + """Return GNU ``objdump``'s output for `arguments`.""" + return subprocess.run( + ["objdump", *arguments], check=True, capture_output=True, text=True + ).stdout + + +def symbol_sizes(library: Path) -> dict[int, int]: + """Return each code symbol's size, by address, from both symbol tables.""" + sizes: dict[int, int] = {} + for table in ("-T", "-t"): + for line in _objdump(table, str(library)).splitlines(): + if match := SYMBOL.match(line): + address, size = int(match[1], 16), int(match[2], 16) + sizes[address] = max(size, sizes.get(address, 0)) + return sizes + + def fused_functions(library: Path) -> tuple[int, dict[str, int]]: """Return the strict functions' count and, for each that fuses, its FMA count.""" - listing = subprocess.run( - ["objdump", "-d", "-C", "--no-show-raw-insn", str(library)], - check=True, - capture_output=True, - text=True, - ).stdout + sizes = symbol_sizes(library) + listing = _objdump("-d", "-C", "--no-show-raw-insn", str(library)) strict = 0 fused: dict[str, int] = {} name: str | None = None + end: int | None = None for line in listing.splitlines(): - if LABEL.match(line): + if label := LABEL.match(line): name = line if STRICT.match(line) else None strict += name is not None + start = int(label[1], 16) + end = start + sizes[start] if sizes.get(start) else None elif name is not None and FUSED.search(line): + instruction = INSTRUCTION.match(line) + if end is not None and instruction and int(instruction[1], 16) >= end: + # Past the symbol's end: unlabelled code the link placed after it. + continue fused[name] = fused.get(name, 0) + 1 return strict, fused diff --git a/native/include/solprop_native/adams_march.hpp b/native/include/solprop_native/adams_march.hpp index f7c586de5..c00d7ac0b 100644 --- a/native/include/solprop_native/adams_march.hpp +++ b/native/include/solprop_native/adams_march.hpp @@ -4,6 +4,7 @@ #pragma once +#include "solprop_native/host_source.hpp" #include "solprop_native/watch.hpp" #include @@ -235,6 +236,11 @@ class Engine { /** The last trial's carrier rotations over its `order` newest rows, for a dense observation of the step; empty without a carrier frame. */ virtual std::vector step_carrier(std::size_t /*order*/) { return {}; } + /** An attempt's start, after the budget check and before its first evaluation, and + its end (solprop SEP 49 section 6.2): a host source's reservation and its + outcome. Nothing for an engine without one. */ + virtual void begin_attempt(std::uint64_t /*cost*/) {} + virtual void end_attempt(HostOutcome /*outcome*/) {} /** The committed state's boundary reading (`monitor_reading`), for a check the march takes itself (SEP 45 section 4.8). */ virtual std::vector reading() { @@ -272,7 +278,10 @@ enum class Category : int { convergence = 1, backend = 2, value = 3, - refused = 4 + refused = 4, + /** A host source's failure (solprop SEP 49 section 6.3), its exception kept by the + host; the stage names where, `none` for its `begin`. */ + host = 5 }; /** The host's bounds for one call (SEP 39 section 4.8). */ diff --git a/native/include/solprop_native/core.hpp b/native/include/solprop_native/core.hpp index 4e7c7ba30..35b40eef4 100644 --- a/native/include/solprop_native/core.hpp +++ b/native/include/solprop_native/core.hpp @@ -4,6 +4,7 @@ #pragma once +#include "solprop_native/host_source.hpp" #include "solprop_native/launch.hpp" #include @@ -1160,6 +1161,19 @@ class GeneralState { [[nodiscard]] std::size_t axis_scans() const noexcept; [[nodiscard]] bool closed() const noexcept; void close() noexcept; + /** + * A host source (solprop SEP 49 section 6.2): from here on each stage source is + * `source`'s evaluation of the lab-frame field, the state's own response passes not + * run, and the stage's finish its collar, absorber and propagator alone. Its two + * host rows are allocated here, once, and listed in the memory ledger. Refused on a + * device build and for anything but one real column off the general and separable + * routes; null unbinds. + */ + void bind_host_source(std::shared_ptr source); + /** The bound host source, or null. */ + [[nodiscard]] HostSource* host_source() const noexcept; + /** The next host evaluation's position, m, and purpose, which the engine sets. */ + void host_evaluation(double z, HostPurpose purpose) noexcept; private: /** `function(workspace, arguments...)` inside the workspace's CPU team scope diff --git a/native/include/solprop_native/engines.hpp b/native/include/solprop_native/engines.hpp index 13f23b52e..9be46f68c 100644 --- a/native/include/solprop_native/engines.hpp +++ b/native/include/solprop_native/engines.hpp @@ -11,6 +11,7 @@ #include "solprop_native/stage_tables.hpp" #include +#include #include #include #include @@ -63,6 +64,9 @@ class ResidentEngine final : public march::Engine { double chord_phase(double z1, double z2) override; std::vector step_carrier(std::size_t order) override; std::vector reading() override; + /** A bound host source's attempt (solprop SEP 49): forwarded to it, if any. */ + void begin_attempt(std::uint64_t cost) override; + void end_attempt(HostOutcome outcome) override; private: void inject(march::Stage stage); diff --git a/native/include/solprop_native/host_source.hpp b/native/include/solprop_native/host_source.hpp new file mode 100644 index 000000000..d3212d03f --- /dev/null +++ b/native/include/solprop_native/host_source.hpp @@ -0,0 +1,61 @@ +// SPDX-FileCopyrightText: 2026 John C. Travers and contributors +// +// SPDX-License-Identifier: GPL-3.0-or-later + +#pragma once + +#include +#include +#include +#include + +/* + * A source the host evaluates for the resident Adams march (solprop SEP 49 section + * 6.2): a lab-frame field in, a spectral source out, and an attempt's start and end. + * It knows nothing of what evaluates it; solprop's TDSE response is its first user, + * through the binding layer, which wraps a Python object and takes the interpreter + * lock for each call. The engine calls it from the thread that called `advance`, + * outside every parallel region, and only on a CPU build: a device build refuses a + * problem that needs one before anything is allocated. + * + * Python-free: an exception the host raises reaches the march as a `HostSourceError`, + * the original kept by the binding layer for solprop's session to raise. + */ +namespace solprop_native { + +/** Why an evaluation is made, the march's label for it (solprop's `TDSEPurpose`). */ +enum class HostPurpose : int { startup = 0, predictor = 1 }; + +/** How an attempt ended: rejected or accepted after its decision, or abandoned by a + failure (or a floor) before one, which only releases what it reserved. */ +enum class HostOutcome : int { rejected = 0, accepted = 1, abandoned = 2 }; + +/** A host source's failure: the host's own exception is kept where it was raised. */ +class HostSourceError : public std::runtime_error { +public: + using std::runtime_error::runtime_error; +}; + +/** The three entry points of an attempt (SEP 49 section 6.2). */ +class HostSource { +public: + HostSource() = default; + HostSource(const HostSource&) = delete; + HostSource& operator=(const HostSource&) = delete; + HostSource(HostSource&&) = delete; + HostSource& operator=(HostSource&&) = delete; + virtual ~HostSource() = default; + /** Before an attempt's first evaluation, after the march's budget check: `cost` + evaluations will follow. */ + virtual void begin(std::uint64_t cost) = 0; + /** One evaluation: the lab-frame spectrum `field` (`n_omega` bins of one column) at + `z`, m, into `source`, the spectral source before the spectral collar. */ + virtual void evaluate(std::span> field, double z, + HostPurpose purpose, + std::span> source) = 0; + /** After the decision and, for an accepted attempt, after its commit; also after a + failure, as `abandoned`. Releases what `begin` reserved on every outcome. */ + virtual void end(HostOutcome outcome) = 0; +}; + +} // namespace solprop_native diff --git a/native/src/adams_march.cpp b/native/src/adams_march.cpp index 8a32a61a8..1cb5ea808 100644 --- a/native/src/adams_march.cpp +++ b/native/src/adams_march.cpp @@ -377,7 +377,9 @@ void fail(Result& result, const Stage stage, const std::exception& error) { result.reason = Reason::error; result.stage = stage; result.message = error.what(); - if (dynamic_cast(&error) != nullptr) + if (dynamic_cast(&error) != nullptr) + result.category = Category::host; + else if (dynamic_cast(&error) != nullptr) result.category = Category::convergence; else if (dynamic_cast(&error) != nullptr) result.category = Category::refused; @@ -387,6 +389,40 @@ void fail(Result& result, const Stage stage, const std::exception& error) { result.category = Category::backend; } +/** One attempt's start and end on the engine (solprop SEP 49 section 6.3): an attempt + that leaves without its decision -- a failure, a floor, a diverged replay -- ends + abandoned, which only releases what it reserved. That release's own failure is the + host's to keep beside the error that left; it never replaces it. */ +class Attempt { +public: + explicit Attempt(Engine& engine) : engine_(engine) {} + Attempt(const Attempt&) = delete; + Attempt& operator=(const Attempt&) = delete; + Attempt(Attempt&&) = delete; + Attempt& operator=(Attempt&&) = delete; + ~Attempt() { + if (!open_) + return; + try { + engine_.end_attempt(HostOutcome::abandoned); + } catch (...) { // NOLINT(bugprone-empty-catch) - kept by the host, see above + } + } + void begin(const std::uint64_t cost) { + engine_.begin_attempt(cost); + open_ = true; + } + /** End with the decision; closed first, so a failure here is not also abandoned. */ + void end(const HostOutcome outcome) { + open_ = false; + engine_.end_attempt(outcome); + } + +private: + Engine& engine_; + bool open_ = false; +}; + } // namespace Result run(const Policy& policy, State& state, Engine& engine, const Bounds& bounds, @@ -465,6 +501,13 @@ Result run(const Policy& policy, State& state, Engine& engine, const Bounds& bou result.cost = cost; return result; } + Attempt attempt(engine); + try { + attempt.begin(cost); + } catch (const std::exception& error) { + fail(result, Stage::none, error); + return result; + } if (state.history.empty()) { // A call that fails has still been charged (R4). ++state.ledger.startup; @@ -547,6 +590,12 @@ Result run(const Policy& policy, State& state, Engine& engine, const Bounds& bou return result; } record(step.end, step.dz, norm, false, trial.order); + try { + attempt.end(HostOutcome::rejected); + } catch (const std::exception& error) { + fail(result, Stage::trial, error); + return result; + } } else { // The new sample takes the lowest row the kept entries do not use. const std::size_t keep = std::min(policy.order, state.history.size()); @@ -589,6 +638,20 @@ Result run(const Policy& policy, State& state, Engine& engine, const Bounds& bou proposal(policy, step.dz, factor(policy, choice.norm, choice.control, policy.max_growth)); record(step.end, step.dz, norm, true, trial.order); + // After the commit (SEP 49 section 6.3): a failure here returns the committed + // attempt with the error, its proposal and count kept. It is the reason the + // march stops, before the floor and the watch, as the reference's record raises + // before its floor check. + try { + attempt.end(HostOutcome::accepted); + } catch (const std::exception& error) { + state.proposal = next; + ++accepted; + if (bounds.replay) + ++state.cursor; + fail(result, Stage::accept, error); + return result; + } if (next < policy.dz_min) { result.reason = Reason::floor; result.floor_proposal = next; diff --git a/native/src/bindings.cpp b/native/src/bindings.cpp index 3bf902ce3..07c52b72b 100644 --- a/native/src/bindings.cpp +++ b/native/src/bindings.cpp @@ -10,6 +10,7 @@ #include "solprop_native/dense_output.hpp" #include "solprop_native/engines.hpp" #include "solprop_native/fft_plan.hpp" +#include "solprop_native/host_source.hpp" #include "solprop_native/lagrange_basis.hpp" #include "solprop_native/low_storage.hpp" #include "solprop_native/refuse.hpp" @@ -86,12 +87,62 @@ using IndexInput = nb::ndarray, nb::c_contig, nb::device::cpu>; using RealOutput = nb::ndarray; +// A read-only NumPy view of the engine's memory (solprop SEP 49's host source field). +using HostField = + nb::ndarray, nb::c_contig, nb::device::cpu>; using ComplexStorage = std::vector>; using RealStorage = std::vector; class BoundState; class BoundSession; +// solprop SEP 49 section 6.2: a host source whose three entry points are a Python +// object's `begin(cost)`, `evaluate(field, z, purpose, out)` and `end(outcome)`. Each +// call takes the interpreter lock the march released; a Python exception is caught +// here, under that lock, and reported to the march as a `HostSourceError`, the +// exception itself kept by the Python object for solprop's session to raise. The +// object is borrowed: its state owns it, so no reference moves without the lock. +class PythonHostSource final : public native::HostSource { +public: + explicit PythonHostSource(const nb::handle owner) : owner_(owner) {} + void begin(const std::uint64_t cost) override { + call([&] { owner_.attr("begin")(cost); }); + } + void evaluate(const std::span> field, const double z, + const native::HostPurpose purpose, + const std::span> source) override { + call([&] { + // Views of the engine's two host rows, valid for the call alone: the field + // read-only, the source written in place. Each has an owner, a capsule that + // frees nothing, because nanobind copies an ownerless array it hands to + // Python, and a write into that copy would never reach the engine. + const std::array in_shape{field.size()}; + const std::array out_shape{source.size()}; + const nb::capsule borrowed_in(field.data(), [](void* /*unused*/) noexcept {}); + const nb::capsule borrowed_out(source.data(), [](void* /*unused*/) noexcept {}); + const HostField in(field.data(), 1, in_shape.data(), borrowed_in); + const ComplexOutput out(source.data(), 1, out_shape.data(), borrowed_out); + owner_.attr("evaluate")( + in, z, purpose == native::HostPurpose::startup ? "startup" : "predictor", + out); + }); + } + void end(const native::HostOutcome outcome) override { + call([&] { owner_.attr("end")(static_cast(outcome)); }); + } + +private: + template void call(const Call& body) { + const nb::gil_scoped_acquire acquire; + try { + body(); + } catch (const nb::python_error& error) { + throw native::HostSourceError(error.what()); + } + } + nb::handle owner_; +}; + template Array exact_array(const nb::handle value) { return nb::cast(value, false); } @@ -2171,6 +2222,20 @@ class BoundState { } // Closes a session that leases this state first (below, after its class). void close() noexcept; + // solprop SEP 49: bind `owner` as the state's host source, or unbind with None (on + // a closed state too, which no longer borrows it). The state keeps the object; the + // engine borrows it. + void bind_host_source(nb::object owner) { + require_unleased(); + if (owner.is_none()) { + if (general_state_) + general_state_->bind_host_source(nullptr); + host_owner_ = nb::object(); + return; + } + open_state().bind_host_source(std::make_shared(owner)); + host_owner_ = std::move(owner); + } // solprop SEP 47: the march a native execution session runs, from the policy, the // schedule and the stage tables this state was configured with, its engine over the @@ -2334,6 +2399,7 @@ class BoundState { bool march_compensated_ = false; bool march_interpolate_ = false; std::optional march_fault_; + nb::object host_owner_; }; // solprop SEP 47: a native execution session over a leased state. It keeps that state @@ -2613,6 +2679,8 @@ void BoundState::close() noexcept { general_state_->close(); general_state_.reset(); } + // A host source's owner (SEP 49) is not released here, without the interpreter + // lock: the engine that borrowed it is gone, and `bind_host_source(None)` drops it. } // solprop SEP 47 section 6.4: a statistics evaluation lowered once from the problem, @@ -3753,6 +3821,7 @@ NB_MODULE(_core, module_) { "policy"_a, "fused"_a, "compensated"_a, "interpolate"_a, "steps"_a.noconvert(), "edges"_a.noconvert()) .def("adams_march_run", &BoundState::march_run, "state"_a, "bounds"_a) + .def("bind_host_source", &BoundState::bind_host_source, "owner"_a.none()) .def( "rk_march_configure", [](BoundState& self, const nb::dict policy, const std::vector& nodes, diff --git a/native/src/engines.cpp b/native/src/engines.cpp index bdd50ba7f..788e7e5f6 100644 --- a/native/src/engines.cpp +++ b/native/src/engines.cpp @@ -40,8 +40,20 @@ ResidentEngine::ResidentEngine(GeneralState& state, const stage::Tables& tables, : state_(state), tables_(tables), anchor_(anchor), fused_(fused), compensated_(compensated), interpolate_(interpolate), fault_(fault) {} +void ResidentEngine::begin_attempt(const std::uint64_t cost) { + if (auto* host = state_.host_source()) + host->begin(cost); +} + +void ResidentEngine::end_attempt(const HostOutcome outcome) { + if (auto* host = state_.host_source()) + host->end(outcome); +} + void ResidentEngine::start(const double z) { inject(march::Stage::startup); + // A host source evaluates the startup sample at the accepted state (SEP 49). + state_.host_evaluation(z, HostPurpose::startup); const auto row = stage::row(tables_, anchor_, z, z); state_.adams_start(0, stage::trial(tables_, row)); if (compensated_) @@ -56,6 +68,8 @@ void ResidentEngine::trial(const double z, const double dz, for (std::size_t index = 0; index < stencil.n_orders; ++index) highest = std::max(highest, stencil.orders[index]); const std::size_t slots = std::min(std::max(highest, stencil.order), stencil.held); + // The trial's one source evaluation is the predictor's, at the step's end. + state_.host_evaluation(z + dz, HostPurpose::predictor); if (fused_) { // The interaction formulation: the history transported to the anchor, the // source evaluated at the step's end (solprop's `_ResidentAdams.fused_trial`). diff --git a/native/src/general/kernels.hpp b/native/src/general/kernels.hpp index 10e2d8562..28d35701b 100644 --- a/native/src/general/kernels.hpp +++ b/native/src/general/kernels.hpp @@ -1274,6 +1274,10 @@ struct StageTerms { ComplexView2 current, fine, whole, solution, error, stage_phase, stage_inverse, noise; ComplexView2 flow_input, flow_lab, flow_drive, general_scaled, general_source; ComplexView2 projected, absorber; + // A host source's rows (solprop SEP 49), `(1, n_omega)`: the lab-frame field the + // entry writes for it, and its source, read in place of the projected response, when + // `hosted`. + ComplexView2 host_spectrum, host_input; ComplexView3 stages, phi_terms, prefactor_terms; RealView tableau, phi_delta, phi_at, prefactor_weights, window; IndexView crop, gaps; @@ -1284,6 +1288,7 @@ struct StageTerms { bool add_noise = false, analytic = false, general = false, flow = false; bool carried = false, keep_lab = false, zero_response = false; bool in_stage_absorber = false; + bool hosted = false; // Every coarse bin is the fine bin of its own index (`crop` is the identity). bool identity_crop = false; @@ -1474,6 +1479,9 @@ struct StageTerms { const double analytic_scale = analytic && index > 0 && 2 * crop(index) < n_to ? 2.0 : 1.0; fine(column, crop(index)) = analytic_scale * inverse_n_to * value; + // The host source evaluates the field itself, unscaled (SEP 49 section 6.2). + if (hosted) + host_input(column, index) = value; if (keep_lab) flow_lab(column, index) = value; // The general route synthesises on the coarse grid: the pad scale and the @@ -1524,15 +1532,22 @@ struct StageTerms { // A zero weight is skipped: a z table's terms are its values at a trial's // positions, one-hot weighted (SEP 29 G1). The sum's value is unchanged; at most // the sign of an exactly zero sum could differ. - Complex prefactor{}; - for (std::size_t term = 0; term < n_prefactor; ++term) { - const double weight = prefactor_weights(stage * n_prefactor + term); - if (weight != 0.0) - prefactor += weight * prefactor_terms(term, column, index); + Complex value{}; + if (hosted) { + // The host's source carries its own factors (solprop SEP 49 section 4): the + // collar's product alone, as the reference's `stage_output` with unit factors. + value = window(index) * host_spectrum(column, index); + } else { + Complex prefactor{}; + for (std::size_t term = 0; term < n_prefactor; ++term) { + const double weight = prefactor_weights(stage * n_prefactor + term); + if (weight != 0.0) + prefactor += weight * prefactor_terms(term, column, index); + } + value = + prefactor * window(index) * + (general ? general_source(column, index) : projected(column, crop(index))); } - Complex value = - prefactor * window(index) * - (general ? general_source(column, index) : projected(column, crop(index))); if (in_stage_absorber && active(index) != 0U) value += absorber(column, crop(index)); value *= stage_inverse(column, index); @@ -1555,7 +1570,7 @@ struct StageTerms { // below: a stage that projects a source into the stage rows, off a coupled flow and // the general route, whose stores the per-bin forms keep. [[nodiscard]] bool blocks_vectorise() const noexcept { - return !flow && !general && !zero_response; + return !flow && !general && !zero_response && !hosted; } // `finish` over the `count` bins of a block from `first`, each bin the same diff --git a/native/src/general/setup.cpp b/native/src/general/setup.cpp index f53b7a601..05e425bf0 100644 --- a/native/src/general/setup.cpp +++ b/native/src/general/setup.cpp @@ -1689,6 +1689,10 @@ std::map GeneralWorkspace::memory_ledger() const { ? general_->dispersive()->memory_ledger() : DispersiveResponse::empty_ledger()); ledger.merge(base_memory_ledger()); + if (host_source_) + // solprop SEP 49: the host's field and source rows and the source's view. + ledger.emplace("host_source", + (host_input_.size() + host_spectrum_.size()) * sizeof(Complex)); return ledger; } diff --git a/native/src/general/state.cpp b/native/src/general/state.cpp index e16896fa4..2fc61bfcc 100644 --- a/native/src/general/state.cpp +++ b/native/src/general/state.cpp @@ -456,6 +456,18 @@ std::size_t GeneralState::rk_source_evaluations() const noexcept { std::size_t GeneralState::axis_scans() const noexcept { return workspace_ ? workspace_->axis_scans() : 0; } +void GeneralState::bind_host_source(std::shared_ptr source) { + with_team([&](GeneralWorkspace& workspace) { + workspace.bind_host_source(std::move(source)); + }); +} +HostSource* GeneralState::host_source() const noexcept { + return workspace_ ? workspace_->host_source() : nullptr; +} +void GeneralState::host_evaluation(const double z, const HostPurpose purpose) noexcept { + if (workspace_) + workspace_->host_evaluation(z, purpose); +} bool GeneralState::closed() const noexcept { return !workspace_; } void GeneralState::close() noexcept { workspace_.reset(); } diff --git a/native/src/general/trial.cpp b/native/src/general/trial.cpp index 9c69d1278..d2e0cfd1e 100644 --- a/native/src/general/trial.cpp +++ b/native/src/general/trial.cpp @@ -5654,6 +5654,12 @@ StageTerms GeneralWorkspace::stage_terms(const std::size_t stage) const { !problem_.raman.enabled && !absorbing() && !dispersive_; terms.in_stage_absorber = absorbing(); terms.identity_crop = identity_crop_; + // A host source is the stage's whole source (solprop SEP 49 section 6.2). + terms.hosted = static_cast(host_source_); + terms.host_spectrum = host_spectrum_; + terms.host_input = host_input_; + if (terms.hosted) + terms.zero_response = false; return terms; } @@ -5843,6 +5849,10 @@ void GeneralWorkspace::evaluate_stage_source(const std::size_t stage, const doub // low-storage march's passes. Returns whether a source was evaluated: a stage without // a response has none to finish. bool GeneralWorkspace::modal_core(const std::size_t stage) { + if (host_source_) { + host_core(); + return true; + } if (general_) { const bool general_active = general_->responds() || problem_.absorber.in_stage; if (general_active) @@ -5969,6 +5979,56 @@ bool GeneralWorkspace::modal_core(const std::size_t stage) { return true; } +void GeneralWorkspace::bind_host_source(std::shared_ptr source) { + if (!source) { + host_source_.reset(); + host_input_ = ComplexView2(); + host_spectrum_ = ComplexView2(); + return; + } +#if defined(KOKKOS_ENABLE_CUDA) || defined(KOKKOS_ENABLE_HIP) + throw BackendRefusal("a host source runs on the native CPU build only"); +#else + if (general_ || problem_.separable_geometry() || problem_.n_columns() != 1 || + problem_.dims.family != FieldFamily::field_real) + throw BackendRefusal("a host source takes one real-field column of a modal route"); + const auto n = problem_.dims.n_omega; + host_input_ = ComplexView2("solprop_native.host_source.field", 1, n); + host_spectrum_ = ComplexView2("solprop_native.host_source.spectrum", 1, n); + host_source_ = std::move(source); +#endif +} + +// solprop SEP 49: the stage core of a host source. The stage's entry kernel left the +// lab-frame field, unscaled and with any noise added, in the host source's field row +// (`StageTerms::combine`); the in-stage absorber is formed from the modal time field +// as on every route; and the host evaluates the source from the thread that called the +// march, outside every parallel region, into the row the finish reads. Both rows are +// host memory on the CPU builds that admit a host source. Host builds only. +void GeneralWorkspace::host_core() { +#if defined(KOKKOS_ENABLE_CUDA) || defined(KOKKOS_ENABLE_HIP) + throw BackendRefusal("a host source runs on the native CPU build only"); +#else + if (absorbing()) { + graph::execute(*inverse_real_plan_, fine_spectrum_, modal_real_, + KokkosFFT::Normalization::none); + form_absorber(); + graph::execute(*absorber_real_plan_, absorber_real_, absorber_spectrum_, + KokkosFFT::Normalization::none); + } + // The entry kernel's row and the absorber's work, on every instance, are done. + Kokkos::fence("solprop_native.host_source.entry"); + const auto n = problem_.dims.n_omega; + // Kokkos::complex and std::complex share one layout, two doubles. + static_assert(sizeof(Complex) == sizeof(std::complex)); + const std::span> field( + reinterpret_cast*>(host_input_.data()), n); + const std::span> source( + reinterpret_cast*>(host_spectrum_.data()), n); + host_source_->evaluate(field, host_z_, host_purpose_, source); +#endif +} + // The general modal route's stage core: the shared lab-frame input, the in-stage // absorber on the modal time field as on every route, and the general source in place // of the legacy node pipeline; the finish applies the shared normalisation and the diff --git a/native/src/general/workspace.hpp b/native/src/general/workspace.hpp index f76c8ed94..274f04668 100644 --- a/native/src/general/workspace.hpp +++ b/native/src/general/workspace.hpp @@ -184,6 +184,12 @@ class GeneralWorkspace { std::vector projected_to_host(bool coarse); std::map dispersive_path() const; std::map memory_ledger() const; + void bind_host_source(std::shared_ptr source); + [[nodiscard]] HostSource* host_source() const noexcept { return host_source_.get(); } + void host_evaluation(const double z, const HostPurpose purpose) noexcept { + host_z_ = z; + host_purpose_ = purpose; + } /** The team a compact problem's FFTW loops take (`CpuTeamScope`): its pools' widest instance, which every threaded kernel and transform of it uses. Zero for any other problem: the runtime's team. */ @@ -319,6 +325,7 @@ class GeneralWorkspace { void evaluate_stage_source(std::size_t stage, double dz, bool accumulate = true); bool modal_core(std::size_t stage); void general_core(std::size_t stage); + void host_core(); StageTerms stage_terms(std::size_t stage) const; std::vector> fine_gap_runs() const; void stage_entry(std::size_t stage, double dz); @@ -737,6 +744,12 @@ class GeneralWorkspace { ComplexView transform_work_; #endif std::unique_ptr slab_forward_, slab_inverse_; + // solprop SEP 49: a host source and its two rows, `(1, n_omega)`: the lab-frame field + // the entry kernel writes and the source the host writes and the finish reads. + std::shared_ptr host_source_; + ComplexView2 host_input_, host_spectrum_; + double host_z_ = 0.0; + HostPurpose host_purpose_ = HostPurpose::startup; ComplexView2 current_, advanced_, solution_, error_, residual_, recycled_, noise_; // solprop SEP 24 phase 3: the low-storage march's registers B and C (the state is // A) and its flows. diff --git a/native/src/solprop_native/_core.pyi b/native/src/solprop_native/_core.pyi index a57486e4a..b1e43ff97 100644 --- a/native/src/solprop_native/_core.pyi +++ b/native/src/solprop_native/_core.pyi @@ -448,6 +448,7 @@ class State: self, state: dict[str, object], bounds: dict[str, object] ) -> dict[str, Any]: ... def adams_march_fault(self, stage: str, after: int, category: str) -> None: ... + def bind_host_source(self, owner: object | None) -> None: ... def rk_march_configure( self, policy: dict[str, object], diff --git a/native/src/solprop_native/_host_source.py b/native/src/solprop_native/_host_source.py new file mode 100644 index 000000000..0b7ee0580 --- /dev/null +++ b/native/src/solprop_native/_host_source.py @@ -0,0 +1,141 @@ +# SPDX-FileCopyrightText: 2026 John C. Travers and contributors +# +# SPDX-License-Identifier: GPL-3.0-or-later + +"""The TDSE response as the native Adams march's host source (solprop SEP 49). + +The march calls three entry points through the binding layer, which takes the +interpreter lock for each: ``begin(cost)`` before an attempt, ``evaluate`` at each +source evaluation, and ``end(outcome)`` after its decision. They are the reference's +binding calls at the same points of an attempt -- ``reserve``, the evaluation with +solprop's shared source assembly (``solprop.backends.tdse_source``), ``record_trial`` +and ``release_reservation`` -- and the source each evaluation returns is ``F`` of +SEP 49 section 4, which the engine finishes as it finishes every source. An exception +raised here is kept, and the march reports only that one was; the session raises the +kept exception unchanged, with every later one attached as a note. +""" + +from typing import Any + +import numpy as np +import numpy.typing as npt +from solprop.backends import tdse_source +from solprop.translate.problem import PropagationProblem +from solprop.translate.responses import prefactor_at +from solprop.translate.stage import StageScalars, StageTerms, combine, scalars +from solprop.translate.tdse import TDSEBinding, TDSEPurpose + +__all__ = ["TDSEHostSource"] + +# The march's attempt outcomes (`HostOutcome` in host_source.hpp). +_REJECTED, _ACCEPTED, _ABANDONED = 0, 1, 2 + + +class TDSEHostSource: + """The driver's TDSE binding as the native march's host source. + + - ``binding``: the driver's binding, which the backend never owns. + - ``terms``: the stage seam's fixed arrays, from which the prefactor is combined as + the reference combines it. + """ + + def __init__( + self, binding: TDSEBinding, problem: PropagationProblem, terms: StageTerms + ) -> None: + self._binding = binding + self._problem = problem + self._terms = terms + self.error: BaseException | None = None + # The last evaluation's purpose, position and projection diagnostics, as the + # reference records them (`tdse_last_evaluation`). + self.last: dict[str, object] = { + "purpose": None, + "z": None, + "projection": None, + } + + def begin(self, cost: int) -> None: + """Reserve every atomic call of the attempt's `cost` evaluations. + + A new attempt starts with no kept exception: one the session did not take was + the previous attempt's, and was reported with it. + """ + self.error = None + self._kept(lambda: self._binding.reserve(int(cost))) + + def evaluate( + self, + field: npt.NDArray[np.complex128], + z: float, + purpose: TDSEPurpose, + out: npt.NDArray[np.complex128], + ) -> None: + """Evaluate the source of the lab-frame `field` at `z`, m, into `out`. + + Both arrays are the engine's host rows, valid for this call alone; `field` + is read-only. + """ + + def _run() -> None: + problem = self._problem + lab = np.array(field, dtype=np.complex128, copy=True).reshape(1, 1, -1) + bundle = scalars(problem, float(z), float(z)) + fields = tdse_source.node_fields(problem, lab, bundle) + result = self._binding.evaluate( + fields, z=bundle.z1, purpose=purpose, density=bundle.density + ) + polarisation, current = tdse_source.atomic_sources( + fields, result, problem, bundle + ) + source, projection = tdse_source.project_atomic_sources( + polarisation, + current, + result, + problem, + bundle, + self._prefactor(bundle), + edge=tdse_source.atomic_edge_source(result, problem, bundle), + ) + out[:] = source.reshape(-1) + self.last = {"purpose": purpose, "z": bundle.z1, "projection": projection} + + self._kept(_run) + + def end(self, outcome: int) -> None: + """Record a decided attempt's outcome, then release what it reserved. + + An abandoned attempt (a failure or a floor before its decision) is not + recorded, as the reference records none; its reservation is released all the + same. + """ + + def _run() -> None: + try: + if outcome != _ABANDONED: + self._binding.record_trial(accepted=outcome == _ACCEPTED) + finally: + self._binding.release_reservation() + + self._kept(_run) + + def take(self) -> BaseException | None: + """Return the kept exception, once.""" + error, self.error = self.error, None + return error + + def _prefactor(self, bundle: StageScalars) -> Any: + """Return the stage prefactor as the reference forms it (`HostDriver`).""" + if bundle.prefactor is None or self._terms.prefactor is None: + return prefactor_at(self._problem.responses, bundle.z1) + return combine(bundle.prefactor, self._terms.prefactor) + + def _kept(self, call: Any) -> None: + """Run an entry point, keeping its exception: the first, later ones as notes.""" + try: + call() + except BaseException as error: + if self.error is None: + self.error = error + elif error is not self.error: + self.error.add_note(f"then, on the host source's cleanup: {error!r}") + raise diff --git a/native/src/solprop_native/backend.py b/native/src/solprop_native/backend.py index 901cf74cd..9cd3ec518 100644 --- a/native/src/solprop_native/backend.py +++ b/native/src/solprop_native/backend.py @@ -6,6 +6,7 @@ from __future__ import annotations +import contextlib import dataclasses import functools import logging @@ -23,6 +24,7 @@ from solprop import transverse_monitor from solprop._helpers import Helpers from solprop._process import acquire_host_blas, thread_budget +from solprop.backends import tdse_source from solprop.backends.exponential import ( CarrierReference, carrier_reference, @@ -120,6 +122,7 @@ ModalRoute, SpatialRoute, flux_of, + tdse_descriptor, transverse_of, ) from solprop.translate.run_request import RunRequest @@ -130,6 +133,8 @@ require_device_ready, terms, ) +from solprop.translate.tdse import AtomicCallLedger, TDSEBinding +from solprop.translate.tdse_source import TDSE_UNAVAILABLE_STATISTICS from solprop.transverse_monitor import ( TransverseEdge, _EdgeReading, # pyright: ignore[reportPrivateUsage] - released together @@ -153,6 +158,7 @@ ) from ._execution import ExecutionRecord from ._helpers import NATIVE +from ._host_source import TDSEHostSource from ._load import Loaded, StageCache, collar_rests, load, propagator_mode from ._probe import NativeProbe from ._statistics import ( @@ -314,6 +320,7 @@ def __init__( fft_planning: Mapping[str, object] | None = None, far_field: FarFieldTables | None = None, options: ExecutionOptions | None = None, + tdse_binding: TDSEBinding | None = None, ) -> None: check_build() # The private gated benchmark is captured where the backend is built, as the @@ -321,6 +328,7 @@ def __init__( self._gated_plasma = _BENCHMARK_GATED.get() stepper = request.stepper _refuse_unsupported(problem, stepper) + _refuse_tdse(problem, request, tdse_binding) if self._gated_plasma: _refuse_gated_route(problem) validate_run(problem, request) @@ -441,6 +449,13 @@ def __init__( self._native_state.cpu_execution ) self._native_state.set_crossing_workers(thread_budget().threads) + # solprop SEP 49: the TDSE response is the march's host source, bound here for + # the backend's life; the driver owns the binding and closes it. + self._tdse_binding = tdse_binding + self._host_source: TDSEHostSource | None = None + if tdse_binding is not None: + self._host_source = TDSEHostSource(tdse_binding, problem, self._terms) + self._native_state.bind_host_source(self._host_source) if fft_planning is None: _keep_plans(self._native_state, wisdom) self._wisdom_digest = ( @@ -580,8 +595,13 @@ def _restore(self, start: Start, loaded: Loaded) -> tuple[str | None, float]: if start.ledgers is not None: atomic = start.ledgers.get("atomic", {}) if atomic: - raise BackendError( - f"{self.name} does not restore atomic response call ledgers" + if self._tdse_binding is None: + raise BackendError( + f"{self.name} does not restore atomic response call ledgers " + "without a TDSE binding" + ) + self._tdse_binding.restore_ledger( + AtomicCallLedger.from_record(cast("Mapping[str, object]", atomic)) ) if adams is not None: adams.restore_ledger( @@ -1407,6 +1427,12 @@ def metadata(self) -> dict[str, object]: work["node_chunk_rule"] = ( "device" if _on_device() and self.chunk_plan.automatic else "host" ) + if self._tdse_binding is not None and self._host_source is not None: + # As the reference records them (solprop SEP 49 section 5.1). + work["atomic_calls"] = dataclasses.asdict(self._tdse_binding.ledger) + work["tdse_execution"] = dict(self._tdse_binding.metadata()) + work["tdse_last_evaluation"] = dict(self._host_source.last) + work["tdse_source"] = tdse_source.source_record(self._problem) return { **work, **facts.info, @@ -1500,6 +1526,11 @@ def memory_ledger(self) -> dict[str, int]: # ones, so that an estimate made before the run counts them. field = int(ledger.get("state", 0)) ledger["rk_dense"] = 6 * field + field // 16 + if self._tdse_binding is not None: + ledger |= { + f"tdse_{name}": int(size) + for name, size in self._tdse_binding.memory_ledger().items() + } return ledger def prepare( @@ -1561,7 +1592,16 @@ def prepare_group( def close(self) -> None: """Release native plans and views; repeated calls have no effect.""" if not self._closed: + # Closing the state drops a host source with the engine that borrowed it, + # also while a session still leases the state (SEP 49). The state's own + # reference to the source is invisible to the garbage collector, which + # could then never free a cycle through it (a kept exception's traceback + # reaching the session), so it is dropped once the state is closed. A + # session that could not close still uses the state, and keeps it. self._native_state.close() + if self._host_source is not None: + with contextlib.suppress(RuntimeError): + self._native_state.bind_host_source(None) _host_blas.release() self._closed = True @@ -2871,6 +2911,28 @@ def values(self, kind: Literal["phi", "prefactor"], z: float) -> ComplexArray: return self._native.table_terms_to_host(kind, slot) +def _refuse_tdse( + problem: PropagationProblem, request: RunRequest, binding: TDSEBinding | None +) -> None: + """Refuse a TDSE problem without its binding, or a binding without one. + + And the legacy statistics the TDSE response does not define, by name, as the + reference refuses them, before anything is allocated. + """ + if (tdse_descriptor(problem.route) is None) != (binding is None): + raise BackendError("TDSE problems require an explicit host response binding") + if binding is None: + return + stats = request.observation.stats + unavailable = TDSE_UNAVAILABLE_STATISTICS.intersection( + (*stats.names, *stats.periodic) + ) + if unavailable: + raise BackendError( + f"TDSE does not define legacy statistics {sorted(unavailable)}" + ) + + def _refuse_unknown_method(stepper: StepperSettings) -> None: """Refuse a method object the native marches do not step, by name.""" integrator = stepper.integrator @@ -2945,11 +3007,11 @@ def _refuse_unsupported(problem: PropagationProblem, stepper: StepperSettings) - # Read as any object: a variant this build does not know is refused by name too. route = cast("object", problem.route) match route: - case ModalRoute(tdse=tdse) if tdse is not None: + case ModalRoute(tdse=tdse) if tdse is not None and _on_device(): + space = str(_build_facts().info["device_backend"]).upper() raise BackendError( - "solprop-native does not execute TDSEGasResponse (solprop SEP 12); " - "use backend='numpy_ref' with the independent solprop-tdse atomic " - "engine" + "the TDSE response runs on the native CPU build; this build is " + f"{space} (solprop SEP 49)" ) case ( ModalRoute() diff --git a/native/src/solprop_native/session.py b/native/src/solprop_native/session.py index 56a0341d0..ebb6fc146 100644 --- a/native/src/solprop_native/session.py +++ b/native/src/solprop_native/session.py @@ -62,13 +62,16 @@ native monitor and the draw's readings by the native transverse edge, and the noise phase at each plane, formed on the host or, for a z table's propagator phase, read from its slot in the leased state; -- no group (:mod:`solprop_native.group` runs one native session per member); and no - TDSE response, which the backend refuses by name where it is built, before it - allocates (SEP 47 section 6.3). +- no group (:mod:`solprop_native.group` runs one native session per member); +- the TDSE response on a CPU build, through the march's host source (solprop SEP 49): + each ``advance`` opens an interruptible scope beside its commit, so a signal held + while the host source waits is raised there, and a host source's failure is the + exception it kept. """ from __future__ import annotations +import contextlib import dataclasses import itertools import logging @@ -78,7 +81,7 @@ import numpy as np import numpy.typing as npt -from solprop._interrupts import committing +from solprop._interrupts import committing, interruptible from solprop._ownership import hold from solprop.backends.events import ( Eligibility, @@ -113,6 +116,7 @@ BackendError, ConvergenceError, GridError, + ResponseBudgetError, StopRequested, ) from solprop.monitor import ( @@ -158,6 +162,8 @@ ) _LOG = logging.getLogger(__name__) +# A march failure a host source raised (`march::Category::host`, solprop SEP 49). +_HOST = 5 if TYPE_CHECKING: from types import TracebackType @@ -236,7 +242,8 @@ def refusal( Decided from the problem, the request and the start alone, before anything is allocated: the statistics the session does not evaluate. Everything else native - does not run -- a TDSE response, a coupled flow beside the interaction formulation, + does not run -- a TDSE response on a device build, a coupled flow beside the + interaction formulation, stage rows the native tables do not read -- the backend refuses by name where it is built (``NativeBackend``), still before any native state (SEP 47 section 6.3). @@ -858,8 +865,10 @@ def advance( # the commit would otherwise drop beside it. failure: BaseException | None = None translated: tuple[BackendHistory, Event] | None = None + # A host source's waits inside the call take a held signal (SEP 49, 6.7). + hosted = self._host_source() is not None try: - with committing(): + with committing(), interruptible() if hosted else contextlib.nullcontext(): try: batch = self._native.advance(limit, float(deadline)) except _core.SessionLifecycleError as error: @@ -1765,6 +1774,38 @@ def _budget_error(self, event: _core.SessionEvent) -> BaseException: field, ) + def _host_source(self) -> Any: + """Return the backend's TDSE host source, or None (solprop SEP 49).""" + if self._backend is None: + return None + return self._driver(self._backend)._host_source + + def _host_error( + self, kept: BaseException, event: _core.SessionEvent + ) -> BaseException: + """Return a host source's kept exception as the reference raises it. + + A reservation the atomic budget refused names the last accepted step and + carries its field, as `NumpyRefBackend._reserve_source_calls` does; any other + exception is returned unchanged. + """ + if not isinstance(kept, ResponseBudgetError): + return kept + field = np.empty(self._shape, dtype=np.complex128) + self._native.export(self._sequence, event.generation, False, False, None, field) + z = float(event.z) + error = ResponseBudgetError( + "the TDSE atomic-call budget cannot reserve the next trial " + f"at z = {z:.6g} m ({kept.calls} charged); " + "the run stops at its last accepted step, unfinished", + z=z, + field=field, + calls=kept.calls, + ) + error.__cause__ = None + error.__suppress_context__ = True + return error + def _abandoned_step(self, event: _core.SessionEvent) -> float: """Return the length of the step a poisoned terminal event left in flight. @@ -1815,6 +1856,21 @@ def _log_finish(self) -> None: def _error(self, event: _core.SessionEvent, outcome: Outcome) -> BaseException: """Return the exception a stopped or failed session reports. + A host source's exception the march did not stop for, the abandoned attempt's + failed release, is attached to it as a note, never raised in its place (SEP 49 + section 6.3), as the reference's `_trial` attaches it. + """ + error = self._reported_error(event, outcome) + host = self._host_source() + if host is not None and (kept := host.take()) is not None and kept is not error: + error.add_note(f"releasing the source reservation failed: {kept}") + return error + + def _reported_error( + self, event: _core.SessionEvent, outcome: Outcome + ) -> BaseException: + """Return the exception the march's event reports. + A failure is formatted by the backend's own march adapter, as its native call would have raised it. """ @@ -1836,6 +1892,14 @@ def _error(self, event: _core.SessionEvent, outcome: Outcome) -> BaseException: f"the replayed schedule ends at {end:.6g} m and cannot reach " f"z = {event.target:.6g} m" ) + # A host source's failure (SEP 49 section 6.3): the exception it kept, + # unchanged; a refused reservation as the reference reports it, from the last + # accepted step. + host = self._host_source() + if int(event.category) == _HOST and host is not None: + kept = host.take() + if kept is not None: + return self._host_error(kept, event) # A statistics evaluation the native plan refused: the reference's exception. failure = BackendError(event.message) refused = translated(failure) diff --git a/native/tests/test_backend.py b/native/tests/test_backend.py index 5cfc38a37..a295bb947 100644 --- a/native/tests/test_backend.py +++ b/native/tests/test_backend.py @@ -399,14 +399,22 @@ def test_tdse_is_refused_by_name_before_integrator_or_native_setup( source_filter: str, dense_output: bool, ) -> None: - """Refuse TDSE before native setup or either supported march is selected.""" + """Refuse TDSE without its binding before native setup or a march is selected. + + The CPU build runs the TDSE response through its host source (solprop SEP 49), + which needs the driver's binding; a problem without one is refused by name first. + """ problem, run_request, grid = phase_a_case # This deliberately bypasses scientific validation to isolate the adapter's # earliest refusal. Neither a valid atomic basis nor an optional provider is - # needed to reject a route that this backend never executes. + # needed to refuse a route that comes without its binding. requested = copy.copy(problem) route = copy.copy(problem.route) - object.__setattr__(route, "tdse", SimpleNamespace(source_filter=source_filter)) + object.__setattr__( + route, + "tdse", + SimpleNamespace(descriptor=SimpleNamespace(), source_filter=source_filter), + ) object.__setattr__(requested, "route", route) requested_request = run_request.with_stepper( dataclasses.replace( @@ -421,7 +429,7 @@ def forbidden(*_arguments: object, **_keywords: object) -> NoReturn: monkeypatch.setattr(solprop_native.backend, "require_device_ready", forbidden) monkeypatch.setattr(_core, "State", forbidden) - with pytest.raises(BackendError, match=r"TDSEGasResponse.*SEP 12"): + with pytest.raises(BackendError, match="explicit host response binding"): NativeBackend(requested, requested_request, grid=grid) diff --git a/src/solprop/_interrupts.py b/src/solprop/_interrupts.py index b35faf789..ea9c77415 100644 --- a/src/solprop/_interrupts.py +++ b/src/solprop/_interrupts.py @@ -28,7 +28,9 @@ "committing", "deferred_handlers", "handled", + "interruptible", "poll", + "raise_held", ] @@ -48,6 +50,8 @@ def __init__(self) -> None: """Start outside every commit with nothing pending.""" self.depth = 0 self.pending: BaseException | None = None + # How many interruptible scopes the main thread is in (:func:`interruptible`). + self.interruptible = 0 _GUARD = _Guard() @@ -86,6 +90,42 @@ def __exit__(self, kind: type[BaseException] | None, *_: object) -> None: raise pending +@contextmanager +def interruptible() -> Iterator[None]: + """Let a long wait inside a commit take the signal the commit holds. + + A native ``advance`` runs inside :class:`committing`, and a host source it calls may + wait minutes for an atomic call (SEP 49 section 6.7). Its session marks that wait + with this scope, and the wait calls :func:`raise_held` between polls, so that a + Ctrl-C or SIGTERM stops it there, before the trial commits. Only the main thread's + scope counts, since signals reach the main thread alone; any other commit -- the + terminal checkpoint's ``committing(deliver=False)`` among them -- keeps holding. + """ + main = threading.current_thread() is threading.main_thread() + if main: + _GUARD.interruptible += 1 + try: + yield + finally: + if main: + _GUARD.interruptible -= 1 + + +def raise_held() -> None: + """Raise, now, a signal the commit holds, inside an :func:`interruptible` scope. + + Does nothing outside such a scope, on any thread but the main one, or when nothing + is held. + """ + if not _GUARD.interruptible or threading.current_thread() is not ( + threading.main_thread() + ): + return + pending, _GUARD.pending = _GUARD.pending, None + if pending is not None: + raise pending + + def _handle(signal_number: int, frame: FrameType | None) -> None: """Raise the signal's exception now, or when the current commit ends.""" del frame diff --git a/src/solprop/backends/host/adams.py b/src/solprop/backends/host/adams.py index 129ab8e6b..876fda5ce 100644 --- a/src/solprop/backends/host/adams.py +++ b/src/solprop/backends/host/adams.py @@ -17,6 +17,8 @@ """ import math +from collections.abc import Iterator +from contextlib import contextmanager from typing import Any, Literal, cast import numpy as np @@ -326,7 +328,7 @@ def march_to(self, target: float) -> None: while not reached(backend._z, target): backend._poll_stop() self._afford() - try: + with _trial(backend): if not self._history: self._start() proposal = self._collar_reentry( @@ -349,8 +351,6 @@ def march_to(self, target: float) -> None: backend._take_step_stats() if backend._natural_stop(target): return - finally: - backend._release_source_calls() if dense and backend._z > target: self._ahead = (backend._z, self._engine.hold()) backend._state = cast("Any", self._observe(target)) @@ -424,7 +424,7 @@ def replay_step(self, dz: float, end: float, slack: float) -> float: reject the step. """ self._afford() - try: + with _trial(self._backend): if not self._history: self._start() kink = adams_core.replay_kink(self._policy, end, slack) @@ -442,8 +442,6 @@ def replay_step(self, dz: float, end: float, slack: float) -> float: "controller choose them" ) raise ConvergenceError(msg) - finally: - self._backend._release_source_calls() return norm def _observe(self, position: float) -> object: @@ -803,6 +801,24 @@ def _reject(self, dz: float, norm: float, order: int) -> None: """What every march path asks of an engine, beside its own trial and acceptance.""" +@contextmanager +def _trial(backend: HostDriver) -> Iterator[None]: + """Release a trial's unused source reservation on every exit (SEP 49, 6.3). + + A failure's own error stays the one raised: a release that fails after it is + attached to it as a note, never raised in its place. + """ + try: + yield + except BaseException as error: + try: + backend._release_source_calls() + except Exception as cleanup: + error.add_note(f"releasing the source reservation failed: {cleanup}") + raise + backend._release_source_calls() + + def _check_engine(engine: object, *, exponential: bool) -> None: """Refuse an injected engine that lacks a member its path calls.""" family = "exponential" if exponential else "fused" diff --git a/src/solprop/backends/numpy_ref/backend.py b/src/solprop/backends/numpy_ref/backend.py index ac2c9135a..80413230e 100644 --- a/src/solprop/backends/numpy_ref/backend.py +++ b/src/solprop/backends/numpy_ref/backend.py @@ -31,6 +31,7 @@ from jaxtyping import Bool, Complex, Float, Inexact, Num from solprop._helpers import PLAIN +from solprop.backends import tdse_source from solprop.backends.exponential import refuse_carrier_frame, refuse_formulation from solprop.backends.host.driver import HostDriver from solprop.backends.host.low_storage import HostLowStorageMarch @@ -50,11 +51,6 @@ general_memory, general_source, ) -from solprop.backends.numpy_ref.tdse import ( - atomic_edge_source, - atomic_sources, - project_atomic_sources, -) from solprop.backends.populations import axis_refusal, population_statistics from solprop.backends.session import ExecutionOptions from solprop.backends.spatial import ( @@ -126,9 +122,7 @@ from solprop.translate.stage import StageScalars, combine from solprop.translate.tdse import AtomicCallLedger, TDSEBinding, TDSEPurpose from solprop.translate.tdse_source import ( - STEADY_CURRENT_CERTIFICATE_LIMIT, TDSE_UNAVAILABLE_STATISTICS, - steady_current_certificate, ) __all__ = ["BlockResidual", "NumpyRefBackend", "SourceTables"] @@ -934,7 +928,7 @@ def metadata(self) -> dict[str, object]: "z": self._tdse_last_z, "projection": self._tdse_last_projection, } - record["tdse_source"] = _source_record(self._problem) + record["tdse_source"] = tdse_source.source_record(self._problem) return record def restore_call_ledgers( @@ -1688,29 +1682,31 @@ def _nonlinear_tdse( inverse_propagator: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], purpose: TDSEPurpose, ) -> Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]: - """Evaluate the declared atomic binding and combine independent P/J sources.""" + """Evaluate the declared atomic binding and combine independent P/J sources. + + Steps 1 to 4 are :mod:`solprop.backends.tdse_source`'s, shared with native; + the stage output is this backend's own. + """ problem = self._problem - grids, transform = problem.grids, problem.transform + grids = problem.grids binding = self._tdse_binding assert binding is not None and tdse_of(problem.route) is not None modal_t = passes.to_time(lab[0], grids.crop_index, grids.n_to, kind="field") - fields = np.ascontiguousarray( - passes.synthesise(modal_t, transform.synthesis, bundle.taper_scale, 1)[ - :, 0, : - ] - ) + fields = tdse_source.node_fields(problem, lab[0], bundle) result = binding.evaluate( fields, z=bundle.z1, purpose=purpose, density=bundle.density ) - polarisation, current = atomic_sources(fields, result, problem, bundle) - source, projection = project_atomic_sources( + polarisation, current = tdse_source.atomic_sources( + fields, result, problem, bundle + ) + source, projection = tdse_source.project_atomic_sources( polarisation, current, result, problem, bundle, self._prefactor(bundle), - edge=atomic_edge_source(result, problem, bundle), + edge=tdse_source.atomic_edge_source(result, problem, bundle), ) self._tdse_last_purpose, self._tdse_last_z = purpose, bundle.z1 self._tdse_last_projection = projection @@ -2949,29 +2945,6 @@ def _coupled_flow_maps(problem: PropagationProblem) -> _CoupledFlow | None: return held -def _source_record(problem: PropagationProblem) -> dict[str, object]: - """Record a TDSE run's source filter and, for the gated one, its certificate. - - The steady-current certificate (SEP 26 section 4.4) is refused above its limit at - translation. It is run metadata, never part of the descriptor or any identity. - - - Returns: ``gated``, and for a gated source the certificate, its limit and the - taper breakpoints in s. - """ - descriptor = tdse_descriptor(problem.route) - assert descriptor is not None - record: dict[str, object] = {"gated": descriptor.gated} - if descriptor.gated: - record |= { - "steady_current_certificate": steady_current_certificate( - descriptor, problem.grids.omega, problem.grids.active_mask - ), - "certificate_limit": STEADY_CURRENT_CERTIFICATE_LIMIT, - "gate_breakpoints": list(descriptor.source_breakpoints()), - } - return record - - def _harmonic_carrier( problem: PropagationProblem, ) -> Complex[np.ndarray, " n_to"] | None: diff --git a/src/solprop/backends/registry.py b/src/solprop/backends/registry.py index a02f53db6..be73404d6 100644 --- a/src/solprop/backends/registry.py +++ b/src/solprop/backends/registry.py @@ -36,7 +36,6 @@ from solprop.profiles import ZProfile from solprop.spatial import FAR_FIELD_STATISTICS from solprop.translate.problem import PropagationProblem -from solprop.translate.routes import tdse_descriptor from solprop.translate.run_request import RunRequest from solprop.translate.tdse import TDSEBinding @@ -354,7 +353,8 @@ def select( profiles : collections.abc.Mapping, optional The z-profiles keyed by statistic name, for the profile statistics. tdse_binding : solprop.translate.tdse.TDSEBinding or None, optional - Driver-owned host response binding for a TDSE problem on NumPy. + Driver-owned host response binding for a TDSE problem, on either backend + (SEP 49). options : solprop.backends.session.ExecutionOptions or None, optional The run's execution options, among them an explicit node budget, from which the backend plans its node chunk (SEP 47 section 6.5); ``None`` for the @@ -389,10 +389,6 @@ def select( ensure() _check_required_device(name) - if ( - tdse_descriptor(problem.route) is not None or tdse_binding is not None - ) and name != "numpy_ref": - raise BackendError(f"{name} does not execute TDSE responses; use numpy_ref") refusal = statistics_refusal(name, request.observation.stats.names) if refusal is not None: raise BackendError(refusal) @@ -468,8 +464,8 @@ def _construct( """Import and build the named backend, whose package is installed. Imported here, not at module scope: selecting a backend must not be a reason to - import the other or initialize a native runtime. Only the reference is given a TDSE - binding; :func:`select` refuses one elsewhere. + import the other or initialize a native runtime. A TDSE binding goes to either + backend, which refuses a problem without its binding by name. """ factory: Callable[..., Backend] if name == "native": diff --git a/src/solprop/backends/numpy_ref/tdse.py b/src/solprop/backends/tdse_source.py similarity index 84% rename from src/solprop/backends/numpy_ref/tdse.py rename to src/solprop/backends/tdse_source.py index 50290d68b..234748c1c 100644 --- a/src/solprop/backends/numpy_ref/tdse.py +++ b/src/solprop/backends/tdse_source.py @@ -2,34 +2,51 @@ # # SPDX-License-Identifier: GPL-3.0-or-later -"""Assemble independently projected TDSE polarisation and current sources. - -The native closure's gated, edge-free source (SEP 26) multiplies the remainder current -``S`` by its gate ``g`` before the transform and subtracts the edge term -``DFT[g' S] / (i omega)`` on the active bins, so that the current is held after the -window instead of switched off. +"""Assemble the TDSE response's macroscopic source at one evaluation (SEP 49). + +The host source both backends call: from the lab-frame modal field to the nodal fields +the atomic binding takes (:func:`node_fields`), from the binding's unscaled microscopic +histories to the macroscopic polarisation, current and edge arrays +(:func:`atomic_sources`, :func:`atomic_edge_source`), and from those to the projected +spectral source with its diagnostics (:func:`project_atomic_sources`). Each backend then +applies the spectral collar, its in-stage absorber term and, in the interaction +formulation, its inverse propagator, as for every source. It imports nothing from +either backend. + +The gated, edge-free source (SEP 26) multiplies the remainder current ``S`` by its gate +``g`` before the transform and subtracts the edge term ``DFT[g' S] / (i omega)`` on the +active bins, so that the current is held after the window instead of switched off. + +The TDSE route is one scalar mode on a field grid (``PropagationProblem._check_tdse``), +so every transform here is the real-to-complex one and the polarisation axis has length +one. """ import math from dataclasses import dataclass import numpy as np -from jaxtyping import Bool, Complex, Float +from jaxtyping import Bool, Complex, Float, Int -from solprop.backends.numpy_ref import passes +from solprop import fourier from solprop.exceptions import BackendError from solprop.translate.linop import power_weight_at from solprop.translate.problem import PropagationProblem from solprop.translate.routes import tdse_descriptor, tdse_of from solprop.translate.stage import StageScalars from solprop.translate.tdse import TDSEResult, lattice_times, waveform_identity -from solprop.translate.tdse_source import steady_current_certificate +from solprop.translate.tdse_source import ( + STEADY_CURRENT_CERTIFICATE_LIMIT, + steady_current_certificate, +) __all__ = [ "FFT_ERROR_CONSTANT", "atomic_edge_source", "atomic_sources", + "node_fields", "project_atomic_sources", + "source_record", ] FFT_ERROR_CONSTANT = 8.0 @@ -46,6 +63,89 @@ """ +def node_fields( + problem: PropagationProblem, + lab: Complex[np.ndarray, "n_modes n_pol n_omega"], + bundle: StageScalars, +) -> Float[np.ndarray, "n_nodes n_to"]: + """Synthesise the real field at every radial node, as the atomic binding takes it. + + Step 1 of an evaluation: the lab-frame modal spectrum, its input noise already + added, is placed on the fine grid's bins with no scale (the pad scale is inside + ``transform.synthesis``), transformed to time and synthesised on the nodes with the + taper scale at the evaluation's position (``kernel-contract.md`` sections 4 and 5). + + Parameters + ---------- + problem : PropagationProblem + A TDSE problem: one scalar mode on a field grid. + lab : numpy.ndarray + Lab-frame modal spectrum in the square root of watts, shape + ``(n_modes, 1, n_omega)``. + bundle : StageScalars + Scalars at the evaluation's position; its ``taper_scale`` is read. + + Returns + ------- + numpy.ndarray + C-contiguous float64 fields in V/m, shape ``(n_nodes, n_to)``. + + Raises + ------ + BackendError + If the spectrum has more than one polarisation. + """ + grids = problem.grids + n_modes = lab.shape[0] + if lab.shape[1] != 1: + raise BackendError("the TDSE source takes one polarisation") + fine = np.zeros((n_modes, fourier.n_omega_for(grids.n_to, "field")), np.complex128) + fine[:, grids.crop_index] = lab.reshape(n_modes, -1) + modal_t = fourier.inverse(fine, kind="field") + scaled = problem.transform.synthesis * bundle.taper_scale + return np.ascontiguousarray(np.tensordot(scaled, modal_t, axes=([1], [0]))) + + +def _project( + values: Float[np.ndarray, "n_nodes 1 n_to"], + projection: Float[np.ndarray, "n_modes n_nodes"], + taper: float, +) -> Float[np.ndarray, "n_modes 1 n_to"]: + """Project nodal values onto the modes, dividing by the taper scale.""" + return np.tensordot(projection / taper, values, axes=([1], [0])) + + +def _to_frequency( + values: Float[np.ndarray, "n_modes 1 n_to"], crop_index: Int[np.ndarray, " n_omega"] +) -> Complex[np.ndarray, "n_modes 1 n_omega"]: + """Transform modal values to frequency and gather the coarse bins, unscaled. + + The crop scale is inside the prefactor and the current numerator. + """ + n_modes = values.shape[0] + spectrum = fourier.forward(values.reshape(n_modes, -1), kind="field") + return spectrum[..., crop_index].reshape(n_modes, 1, -1) + + +def _cubic_polarisation( + fields: Float[np.ndarray, "n_nodes n_to"], + problem: PropagationProblem, + bundle: StageScalars, +) -> Float[np.ndarray, "n_nodes 1 n_to"]: + """Return the compiled Kerr polarisation that replaces the atom's third order. + + ``P = eps0 chi3 rho E^3 w(t)`` in C/m², with the temporal collar ``w``: the real + cubic of a field grid, the only term the TDSE route admits beside the atom. + """ + # A validated problem (``PropagationProblem.validate``) holds this real cubic for + # ``optmat_cubic``, and no plasma, Raman or dispersive Kerr term beside it. + kerr = problem.responses.kerr_coefficients + assert kerr is not None + values = fields[:, None, :] + polarisation = kerr.coefficient * bundle.kerr_density * values**2 * values + return polarisation * problem.grids.t_window_fine + + def atomic_sources( fields: Float[np.ndarray, "n_nodes n_to"], result: TDSEResult, @@ -112,17 +212,7 @@ def atomic_sources( * result.third_order_dipole[:, None, :] ) elif descriptor.kerr_model == "optmat_cubic": - polarisation = passes.nonlinear_polarisation( - fields[:, None, :], - problem.responses, - problem.grids.t_window_fine, - kind="field", - density=bundle.density, - preionisation=0.0, - dt=problem.grids.dt_fine, - carrier=None, - scalars=bundle, - ) + polarisation = _cubic_polarisation(fields, problem, bundle) else: polarisation = np.zeros_like(current) # The existing in-stage field absorber remains separate. An explicit source @@ -206,8 +296,8 @@ def _project_spectrum( - ``values``: nodal P or J source, shape ``(n_nodes, 1, n_to)``, in C/m² or A/m². - Returns: raw FFT source, shape ``(n_modes, 1, n_omega)`` before source factors. """ - projected = passes.project(values, projection, bundle.taper_scale, 1) - return passes.to_frequency(projected, problem.grids.crop_index, kind="field") + projected = _project(values, projection, bundle.taper_scale) + return _to_frequency(projected, problem.grids.crop_index) def _roundoff_floor( @@ -245,8 +335,8 @@ def _roundoff_terms( Both feed :func:`_roundoff_floor`; the accumulation is ``operations * eps``. """ - absolute_projected = passes.project( - magnitudes, np.abs(projection), abs(bundle.taper_scale), 1 + absolute_projected = _project( + magnitudes, np.abs(projection), abs(bundle.taper_scale) ) # A Fourier coefficient is bounded by the temporal L1 sum. Keep the raw # operands in this bound: using the already cancelled remainder would hide @@ -425,10 +515,10 @@ def _edge_rounding( # noqa: PLR0917 - called positionally weights = np.abs(projection) taper = abs(bundle.taper_scale) operands = np.linalg.norm( - passes.project(edge_magnitude, weights, taper, 1), axis=-1, keepdims=True + _project(edge_magnitude, weights, taper), axis=-1, keepdims=True ) evaluation = np.linalg.norm( - passes.project(derivative_magnitude, weights, taper, 1), + _project(derivative_magnitude, weights, taper), axis=-1, keepdims=True, ) @@ -584,11 +674,8 @@ def _leading_taper( else: ratio, reason = lead_peak / peak, None scaled = abs(bundle.density * descriptor.remainder_scale) * remainder[:, None, lead] - projected = passes.project( - scaled, - np.abs(problem.transform.projection), - abs(bundle.taper_scale), - 1, + projected = _project( + scaled, np.abs(problem.transform.projection), abs(bundle.taper_scale) ) largest = np.max(projected, axis=-1, keepdims=True, initial=0.0) return { @@ -843,3 +930,35 @@ def _current_operands( factor=source.current_numerator / power_weight_at(problem, bundle.z1), inverse=_inverse_i_omega(problem) if gated else None, ) + + +def source_record(problem: PropagationProblem) -> dict[str, object]: + """Record a TDSE run's source filter and, for the gated one, its certificate. + + The steady-current certificate (SEP 26 section 4.4) is refused above its limit at + translation. It is run metadata, never part of the descriptor or any identity; + both backends record it as ``tdse_source``. + + Parameters + ---------- + problem : PropagationProblem + A TDSE problem. + + Returns + ------- + dict + ``gated``, and for a gated source the certificate, its limit and the taper + breakpoints in s. + """ + descriptor = tdse_descriptor(problem.route) + assert descriptor is not None + record: dict[str, object] = {"gated": descriptor.gated} + if descriptor.gated: + record |= { + "steady_current_certificate": steady_current_certificate( + descriptor, problem.grids.omega, problem.grids.active_mask + ), + "certificate_limit": STEADY_CURRENT_CERTIFICATE_LIMIT, + "gate_breakpoints": list(descriptor.source_breakpoints()), + } + return record diff --git a/src/solprop/capabilities.py b/src/solprop/capabilities.py index d0d47b27b..beea81f6e 100644 --- a/src/solprop/capabilities.py +++ b/src/solprop/capabilities.py @@ -597,12 +597,6 @@ class Rule: "tdse_execution is the resources of a TDSE response, which tdse declares.", _CAPILLARY, ), - Rule( - "tdse-numpy-ref", - ("tdse_execution", "backend"), - "The TDSE response runs on numpy_ref alone, named explicitly.", - _CAPILLARY, - ), Rule( "tdse-multistep", ("tdse_execution", "integrator"), @@ -683,7 +677,7 @@ def route(name: str) -> RouteCapabilities: >>> capillary.choices["shape"], capillary.choices["noise"] (('gauss', 'sech'), ('default', 'input', 'modified', 'none')) >>> [rule.id for rule in capillary.rules_for("tdse_execution")][:2] - ['tdse-execution-needs-tdse', 'tdse-numpy-ref'] + ['tdse-execution-needs-tdse', 'tdse-multistep'] """ # The recipe layer sits above this module, which the driver imports. from solprop.recipe.core import ROUTES # noqa: PLC0415 - layer above diff --git a/src/solprop/driver/_pyroraptor.py b/src/solprop/driver/_pyroraptor.py new file mode 100644 index 000000000..0f3ade89c --- /dev/null +++ b/src/solprop/driver/_pyroraptor.py @@ -0,0 +1,785 @@ +# SPDX-FileCopyrightText: 2026 John C. Travers and contributors +# +# SPDX-License-Identifier: GPL-3.0-or-later + +"""Run Pyroraptor inside the source worker: the TDSE binding's provider adapter. + +The one place solprop imports the optional provider, and only in the worker +(:mod:`solprop.driver._source_worker`, SEP 49 section 6.5). It builds the provider's +models and plans from the plain records the binding sends, makes its atomic calls and +returns their results as plain records, with the provider's errors classified. Budgets, +the archive, validation, normalisation and warnings stay in the binding +(:mod:`solprop.driver.tdse`), in solprop's own process. + +Requests: ``basis_from_preset``, ``hacc_preset`` and ``hacc_basis`` (queries); then for +a run ``prepare``, ``build_plans`` and ``evaluate``; for a consistency check +``diagnostic_plan``, ``diagnostic_evaluate`` and ``diagnostic_close``. +""" + +import importlib +import json +import sys +from collections.abc import Callable, Mapping +from contextlib import ExitStack +from dataclasses import asdict, dataclass, fields +from importlib.metadata import PackageNotFoundError, distribution +from typing import Any, Literal, Protocol, cast, runtime_checkable + +import numpy as np + +from solprop.driver._source_worker import Record +from solprop.exceptions import BackendError, DriverError, MaterialError, SolpropError +from solprop.nonlinear.tdse import HaccBasis, TDSEBasis +from solprop.translate.tdse import HACC_NORM_GROWTH_LIMIT + +__all__ = ["PyroraptorAdapter", "make_adapter"] + +# Native classes cannot occur in runtime annotations: optional-package-free test +# collection resolves every annotation through beartype. Confine their dynamic +# typing to this adapter; the seam's scientific records and arrays remain typed. +_Native = Any +# The haCC consumer surface of the provider (SEP 20 section 5.7, C1 to C5). +_HACC_SURFACE = ( + "AngularLimit", + "Argon", + "Basis", + "GridResponse", + "Helium", + "MixedGauge", + "ResponsePlan", + "TotalCurrentResponse", + "VelocityGauge", + "helium_preset", + "prepare", +) +# The shadow components the binding reads, computed here where the record's own +# ``component`` method is (section 6.5, plain-data results). +_COMPONENTS = ("total_current", "current", "charge_dipole") +_PRESET_FIELDS = ( + "identity", + "variant", + "step_margin", + "accuracy", + "box_sensitivity", + "gauge_sensitivity", + "max_peak_intensity", + "wavelength", + "window", + "device_admissible", + "source", +) + + +@runtime_checkable +class _Provider(Protocol): + """Type the optional module at its single dynamic-import boundary.""" + + AngularBasis: Callable[..., _Native] + ComplexScaling: Callable[..., _Native] + Discretization: Callable[..., _Native] + DrivenError: type[Exception] + GridResponse: type + RadialBasis: Callable[..., _Native] + ResponsePlan: Callable[..., _Native] + SpectralOptions: Callable[..., _Native] + SpectralSetup: Callable[..., _Native] + StateSelection: Callable[..., _Native] + TotalCurrentResponse: type + atomic_model: Callable[[str], _Native] + build_info: Callable[[], _Native] + # The haCC namespace is looked up by name (`_hacc_surface`): an SAE-only + # provider is still a provider, and is refused only when haCC is asked for. + solve_preset: Callable[[str, str], _Native] + unit_scale: Callable[[str], _Native] + + +def _load_provider() -> _Provider: + """Load the optional package, in the worker only.""" + if sys.version_info[:2] != (3, 14): + raise BackendError("the pinned TDSE provider requires Python 3.14") + try: + return cast("_Provider", importlib.import_module("pyroraptor")) + except (ImportError, OSError) as error: + raise BackendError( + "TDSE response requires the optional pyroraptor dependency; " + "install Pyroraptor into the environment, " + "with a matching CPU, CUDA or HIP build" + ) from error + + +def _identity(provider: _Provider) -> dict[str, object]: + """Return the provider's build and installed distribution, for the handshake.""" + record: dict[str, object] = {"native": asdict(provider.build_info())} + try: + package = distribution("pyroraptor") + except PackageNotFoundError: + record["distribution"] = None + else: + record["distribution"] = package.version + source = package.read_text("direct_url.json") + record["direct_url"] = None if source is None else json.loads(source) + return record + + +def _admit_memory(ledger: Mapping[str, int], limit: int | None) -> None: + """Apply the declared atomic byte limit. + + - ``ledger``: named persistent allocations or estimates in bytes. + - ``limit``: positive byte ceiling, or None for an unrestricted allocation. + """ + total = sum(ledger.values()) + if limit is not None and total > limit: + raise DriverError( + f"TDSE persistent data requires {total} bytes, exceeding {limit} bytes" + ) + + +def _is(value: object, kind: type) -> bool: + """Whether a provider object, or a record sent of one, is of `kind`.""" + if isinstance(value, Record): + return value.record_type == f"{kind.__module__}.{kind.__qualname__}" + return isinstance(value, kind) + + +def _hacc_surface(provider: _Provider) -> _Native: + """Return the provider's haCC namespace, refusing a build without it.""" + hacc = getattr(provider, "hacc", None) + missing = [name for name in _HACC_SURFACE if not hasattr(hacc, name)] + if missing: + raise BackendError( + "the installed TDSE provider has no haCC consumer surface " + f"(missing {', '.join(missing)}); install a Pyroraptor build with it" + ) + return hacc + + +def _hacc_fields( # noqa: PLR0917 - called positionally + hacc: _Native, + atom: _Native, + basis: _Native, + gauge: _Native, + cut_energy: float, + max_step_au: float, +) -> dict[str, object]: + """Convert provider haCC objects into the numeric fields of a ``HaccBasis``. + + - ``atom``, ``basis``, ``gauge``: provider objects, or the records of them a caller + sent. + - ``cut_energy``: spectral cut in J. + - ``max_step_au``: RK4 step ceiling in atomic units of time. + - Returns: every ``HaccBasis`` field but ``source`` and ``preset``. + """ + if _is(atom, hacc.Argon): + raise MaterialError( + "argon TDCIS is not representable as a HaccBasis: the haCC target is " + "helium only" + ) + if not _is(atom, hacc.Helium): + raise MaterialError("a HaccBasis needs a provider hacc.Helium atom") + if atom.ions not in (1, 2): + raise MaterialError( + f"Helium(ions={atom.ions}) is not representable: HaccBasis admits " + "haCC(1) (ions=1) and haCC(5) (ions=2)" + ) + if _is(gauge, hacc.MixedGauge): + gauge_radius: float | None = float(gauge.radius) + elif _is(gauge, hacc.VelocityGauge): + gauge_radius = None + else: + raise MaterialError("a HaccBasis gauge must be MixedGauge or VelocityGauge") + angular = basis.angular + limit = "particle" if angular.particle is not None else "total" + scaling = basis.scaling + return { + "ions": atom.ions, + "neutral_square": atom.neutral_square, + "radial_edges": tuple(float(edge) for edge in basis.radial.edges), + "radial_orders": tuple(basis.radial.orders), + "angular_limit": limit, + "angular_order": angular.particle if limit == "particle" else angular.total, + "scaling_radius": float(scaling.radius), + "scaling_angle": float(scaling.angle), + "infinite_order": scaling.infinite_order, + "decay": float(scaling.decay), + "gauge_radius": gauge_radius, + "cut_energy": float(cut_energy), + "max_step_au": float(max_step_au), + } + + +def _numeric_fields(basis: HaccBasis) -> dict[str, object]: + """Return a ``HaccBasis``'s fields without its provenance and preset identity.""" + record = asdict(basis) + del record["source"], record["preset"] + return record + + +def _hacc_objects( + provider: _Provider, hacc: _Native, basis: HaccBasis +) -> tuple[_Native, _Native, _Native]: + """Build the provider's atom, basis and gauge from a ``HaccBasis``.""" + angular = ( + hacc.AngularLimit(particle=basis.angular_order) + if basis.angular_limit == "particle" + else hacc.AngularLimit(total=basis.angular_order) + ) + atom = hacc.Helium(ions=basis.ions, neutral_square=basis.neutral_square) + native = hacc.Basis( + provider.RadialBasis(basis.radial_edges, basis.radial_orders), + provider.ComplexScaling( + basis.scaling_radius, + basis.scaling_angle, + basis.infinite_order, + basis.decay, + ), + angular, + ) + gauge = ( + hacc.VelocityGauge() + if basis.gauge_radius is None + else hacc.MixedGauge(basis.gauge_radius) + ) + return atom, native, gauge + + +def check_info( + info: _Native, + *, + rows: int, + samples: int, + spacing: float, + outputs: str | None, + cut_energy: float, + closure: bool, + preset: bool, + model_match: bool, +) -> None: + """Require a haCC plan's or result's ``info`` to describe exactly its model. + + - ``model_match``: whether ``info``'s atom, basis and gauge equal the model's, as + the provider compares them (in the worker). + - ``preset``: whether the model is a qualified preset, which must close. + """ + if ( + info.rows != rows + or info.samples != samples + or info.spacing != spacing + or info.shadow_order != 1 + or info.outputs != outputs + or not model_match + or info.cut_energy != cut_energy + ): + raise BackendError( + "haCC plan metadata disagree with the prepared model or contract" + ) + if bool(info.total_current_qualified) != closure: + if preset: + raise BackendError( + "provider contract violation: a haCC preset reports no " + "escaped-electron closure" + ) + raise BackendError("haCC plan closure flag disagrees with its output mode") + + +def _rebuild[T](kind: type[T], record: object) -> T: + """Return a solprop dataclass from the record of it the binding sent.""" + values = vars(record) if isinstance(record, Record) else cast("dict", record) + return kind(**{item.name: values[item.name] for item in fields(kind) if item.init}) # pyright: ignore[reportArgumentType] + + +@dataclass +class _HaccModel: + """One prepared haCC model and the output mode its plans use (SEP 20, C1). + + The model is entered into the adapter's stack first, so it closes after every plan + built on it. + """ + + hacc: _Native + basis: HaccBasis + atom: _Native + native_basis: _Native + gauge: _Native + preset: _Native + model: _Native + outputs: Literal["full", "total_current"] | None + closure: bool | None + + def plan(self, *, rows: int, spacing: float, samples: int) -> _Native: + """Construct one response plan, choosing the native output mode once. + + A custom mixed-gauge basis is first asked for the compact closed current; the + provider refuses it with ``ValueError`` when the gauge radius admits no closure, + and the full record is then used, as for the velocity gauge. For a full archive + every plan is full, and a custom mixed-gauge basis takes its closure from the + first plan's ``total_current_qualified``. + """ + arguments = { + "rows": rows, + "spacing": float(spacing), + "samples": samples, + "max_step_au": float(self.basis.max_step_au), + "field": "electric_field", + "shadow_order": 1, + "norm_growth_limit": HACC_NORM_GROWTH_LIMIT, + } + if self.outputs is None: + try: + plan = self.hacc.ResponsePlan( + self.model, outputs="total_current", **arguments + ) + except ValueError as error: + if "closure" not in str(error): + raise + self.outputs = "full" + self.closure = False + else: + self.outputs = "total_current" + self.closure = True + return plan + plan = self.hacc.ResponsePlan(self.model, outputs=self.outputs, **arguments) + if self.closure is None: + self.closure = bool(plan.info.total_current_qualified) + return plan + + def matches(self, info: _Native) -> bool: + """Whether ``info`` names this model's atom, basis and gauge.""" + return bool( + info.atom == self.atom + and info.basis == self.native_basis + and info.gauge == self.gauge + ) + + def check(self, plan: _Native, *, rows: int, samples: int, spacing: float) -> None: + """Check a plan this model built.""" + check_info( + plan.info, + rows=rows, + samples=samples, + spacing=spacing, + outputs=self.outputs, + cut_energy=self.basis.cut_energy, + closure=self.closure is True, + preset=self.preset is not None, + model_match=self.matches(plan.info), + ) + + def record(self) -> dict[str, object]: + """Return what the binding needs of the model: mode, closure, preset.""" + preset = self.preset + return { + "outputs": self.outputs, + "closure": self.closure, + "preset": None + if preset is None + else {name: getattr(preset, name) for name in _PRESET_FIELDS}, + } + + +class PyroraptorAdapter: + """Serve the TDSE binding's requests with Pyroraptor, inside the worker. + + - ``provider``: the provider module; None loads ``pyroraptor`` (the manufactured + provider of the tests is passed here). + """ + + def __init__(self, provider: _Provider | None = None) -> None: + self._provider = _load_provider() if provider is None else provider + self._stack = ExitStack() + self._plans: dict[int, _Native] = {} + self._hacc: _HaccModel | None = None + self._request: Mapping[str, Any] = {} + self._diagnostic: tuple[ExitStack, _Native] | None = None + + # -- the worker's protocol ------------------------------------------------------ + + def identity(self) -> Mapping[str, object]: + """Return the provider's build record.""" + return _identity(self._provider) + + def classify(self, error: BaseException) -> str: + """Classify a provider-side exception for the failure record (SEP 20, 5.6). + + ``native`` for a provider ``DrivenError``, ``contract`` for solprop's own + refusal, ``interface`` for a ``ValueError`` or ``TypeError``, ``unexpected`` + otherwise. + """ + if isinstance(error, self._provider.DrivenError): + return "native" + if isinstance(error, SolpropError): + return "contract" + if isinstance(error, ValueError | TypeError): + return "interface" + return "unexpected" + + def computed(self, value: object) -> Mapping[str, object]: + """Return a row outcome's ``ok`` and a shadow's components, beside fields.""" + if hasattr(value, "failure") and hasattr(value, "ok"): + return {"ok": bool(cast("Any", value).ok)} + component = getattr(value, "component", None) + orders = getattr(value, "orders", None) + if not callable(component) or not isinstance(orders, tuple): + return {} + found: dict[str, object] = {} + for quantity in _COMPONENTS: + for order in cast("tuple[int, ...]", orders): + try: + found[f"{quantity}:{order}"] = np.asarray( + component(quantity, order) + ) + except Exception as error: + found[f"{quantity}:{order}"] = { + "error": str(error), + "type": type(error).__name__, + } + return {"components": found} + + def handle(self, kind: str, value: object) -> object: + """Serve one request of the binding.""" + request = cast("Mapping[str, Any]", value) + handlers: dict[str, Callable[[Mapping[str, Any]], object]] = { + "basis_from_preset": self._basis_from_preset, + "hacc_preset": self._hacc_preset, + "hacc_basis": self._hacc_basis, + "prepare": self._prepare, + "build_plans": self._build_plans, + "evaluate": self._evaluate, + "diagnostic_plan": self._diagnostic_plan, + "diagnostic_evaluate": self._diagnostic_evaluate, + "diagnostic_close": self._diagnostic_close, + } + if kind not in handlers: + raise BackendError(f"the TDSE provider adapter has no request {kind!r}") + return handlers[kind](request) + + def close(self) -> None: + """Close any diagnostic plan, then the plans before the setup they use.""" + self._diagnostic_close({}) + self._stack.close() + + # -- queries -------------------------------------------------------------------- + + def _basis_from_preset(self, request: Mapping[str, Any]) -> object: + """Return a provider preset's SAE basis fields, provenance apart.""" + selected = self._provider.solve_preset(request["target"], request["preset"]) + basis = selected.discretization + scaling = basis.scaling + return { + "name": selected.name, + "radial_edges": basis.radial.edges, + "radial_orders": basis.radial.orders, + "lmax": basis.angular.lmax, + "cut_energy": selected.cut_energy, + "provider_source": selected.source, + "m": basis.angular.m, + "potential": basis.model.name, + "potential_version": basis.model.version, + "scaling_radius": None if scaling is None else scaling.radius, + "scaling_angle": None if scaling is None else scaling.angle, + "infinite_order": None if scaling is None else scaling.infinite_order, + "decay": None if scaling is None else scaling.decay, + "cuts": basis.cuts, + } + + def _hacc_preset(self, request: Mapping[str, Any]) -> object: + """Return a qualified haCC preset's basis fields and its records.""" + hacc = _hacc_surface(self._provider) + preset = hacc.helium_preset(request["ions"], request["variant"]) + return { + "fields": _hacc_fields( + hacc, + preset.atom, + preset.basis, + preset.gauge, + preset.cut_energy, + preset.max_step_au, + ), + "preset": {name: getattr(preset, name) for name in _PRESET_FIELDS}, + } + + def _hacc_basis(self, request: Mapping[str, Any]) -> object: + """Return the basis fields of a caller's provider objects, sent as records.""" + hacc = _hacc_surface(self._provider) + return _hacc_fields( + hacc, + request["atom"], + request["basis"], + request["gauge"], + request["cut_energy"], + request["max_step_au"], + ) + + # -- a run ---------------------------------------------------------------------- + + def _prepare(self, request: Mapping[str, Any]) -> object: + """Prepare the model and, unless deferred, its plans; admit the memory. + + - ``request``: the descriptor's plain records, the chunk sizes, the memory the + binding already holds and its limit. + """ + self._request = request + memory = dict(request["memory"]) + if request["model"] == "hacc": + model = self._prepare_hacc(_rebuild(HaccBasis, request["basis"])) + memory.update(self._model_memory(model)) + # Admit the model's measured ledger before any plan is built (C7). + _admit_memory(memory, request["memory_limit"]) + if not request["defer_plans"]: + memory.update(self._build_hacc_plans()) + else: + memory.update(self._prepare_sae(_rebuild(TDSEBasis, request["basis"]))) + _admit_memory(memory, request["memory_limit"]) + return self._prepared(memory) + + def _prepared(self, memory: Mapping[str, int]) -> dict[str, object]: + """Return the measured memory, the plan records and the haCC model's record.""" + return { + "memory": dict(memory), + "plans": { + str(rows): asdict(plan.info) for rows, plan in self._plans.items() + }, + "hacc": None if self._hacc is None else self._hacc.record(), + } + + def _prepare_sae(self, basis: TDSEBasis) -> dict[str, int]: + """Allocate shared SAE resources and one plan per distinct batch size.""" + provider, request = self._provider, self._request + model = provider.atomic_model(basis.potential) + if model.version != basis.potential_version: + raise BackendError( + "TDSE atomic potential revision differs from its descriptor" + ) + scaling = None + if basis.scaling_radius is not None: + scaling = provider.ComplexScaling( + basis.scaling_radius, + basis.scaling_angle, + basis.infinite_order, + basis.decay, + ) + discretization = provider.Discretization( + provider.RadialBasis(basis.radial_edges, basis.radial_orders), + provider.AngularBasis(basis.lmax, basis.m), + model, + scaling, + basis.cuts, + ) + operators = self._stack.enter_context(discretization.assemble()) + initial = provider.StateSelection(*request["initial"]) + setup = self._stack.enter_context( + provider.SpectralSetup( + operators, + provider.SpectralOptions( + cut_energy=basis.cut_energy, + states=(initial,), + growing=request["growing"], + growth_tolerance=request["growth_tolerance"], + window_duration=request["window_duration"], + accept_removed_component=request["accept_removed_component"], + ), + ) + ) + memory: dict[str, int] = {} + for label, ledger in ( + ("operators", operators.memory_ledger()), + ("setup", setup.memory_ledger()), + ): + for entry in ledger.entries: + memory[f"{label}.{entry.name}"] = entry.bytes + step = ( + request["max_atomic_step"] / provider.unit_scale("time").si_per_atomic_unit + ) + for rows in request["rows"]: + plan = self._stack.enter_context( + provider.ResponsePlan( + setup, + rows=rows, + spacing=request["spacing"], + samples=request["samples"], + max_step_au=step, + initial=initial, + field="electric_field", + shadow_order=request["shadow_order"], + complement=request["complement"], + outputs=request["outputs"], + ) + ) + self._plans[rows] = plan + for entry in plan.memory_ledger().entries: + if not entry.shared: + memory[f"plan.{rows}.{entry.name}"] = entry.bytes + return memory + + def _prepare_hacc(self, basis: HaccBasis) -> _HaccModel: + """Prepare the haCC model, verifying a preset against its record.""" + request = self._request + if request["complement"] == "vendor_blas": + raise BackendError( + "haCC response plans have no complement option; use the default " + "TDSEExecution(complement='deterministic'), recorded as not " + "applicable" + ) + hacc = _hacc_surface(self._provider) + atom, native_basis, gauge = _hacc_objects(self._provider, hacc, basis) + preset: _Native = None + if basis.preset is not None: + preset = self._verified_preset(hacc, basis) + prepared = preset.prepare() + else: + prepared = hacc.prepare( + atom, native_basis, gauge=gauge, cut_energy=basis.cut_energy + ) + model = self._stack.enter_context(prepared) + # The native output mode: compact with the escaped-electron closure, full + # without it (the provider refuses compact order one then, C3), and full + # throughout for a full archive. Whether the plans carry the closure is known + # beforehand for a preset (always) and for the velocity gauge (never); a + # custom mixed gauge learns it from its first plan. + outputs: Literal["full", "total_current"] | None = ( + "total_current" if basis.preset is not None else None + ) + closure: bool | None = True if basis.preset is not None else None + if basis.gauge_radius is None: + outputs, closure = "full", False + if request["archive_outputs"] == "full": + outputs = "full" + self._hacc = _HaccModel( + hacc, basis, atom, native_basis, gauge, preset, model, outputs, closure + ) + return self._hacc + + def _verified_preset(self, hacc: _Native, basis: HaccBasis) -> _Native: + """Refuse a preset the installed provider does not reproduce.""" + preset = hacc.helium_preset(basis.ions, basis.variant) + if preset.identity != basis.preset: + raise BackendError( + f"the installed provider's haCC preset is {preset.identity!r}, not " + f"the expected {basis.preset!r}; regenerate the record" + ) + found = _hacc_fields( + hacc, + preset.atom, + preset.basis, + preset.gauge, + preset.cut_energy, + preset.max_step_au, + ) + if found != _numeric_fields(basis): + raise BackendError( + f"the installed provider's preset {preset.identity!r} does not " + "reproduce the record's fields" + ) + if self._request["device"] != "cpu" and not preset.device_admissible: + raise BackendError( + f"the haCC preset {preset.identity!r} is not admitted on the " + f"{self._request['device']} device route" + ) + return preset + + @staticmethod + def _model_memory(model: _HaccModel) -> dict[str, int]: + """Return the prepared model's measured ledger.""" + return { + f"model.{entry.name}": entry.bytes + for entry in model.model.memory_ledger().entries + } + + def _build_hacc_plans(self) -> dict[str, int]: + """Build and check one haCC plan per chunk size; return their memory.""" + model, request = self._hacc, self._request + assert model is not None + spacing = float(request["spacing"]) + memory: dict[str, int] = {} + for rows in request["rows"]: + plan = self._stack.enter_context( + model.plan(rows=rows, spacing=spacing, samples=request["samples"]) + ) + model.check(plan, rows=rows, samples=request["samples"], spacing=spacing) + self._plans[rows] = plan + for entry in plan.memory_ledger().entries: + if not entry.shared: + memory[f"plan.{rows}.{entry.name}"] = entry.bytes + return memory + + def _build_plans(self, request: Mapping[str, Any]) -> object: + """Build a deferred haCC binding's plans and admit them with its memory.""" + memory = {**request["memory"], **self._build_hacc_plans()} + _admit_memory(memory, request["memory_limit"]) + return self._prepared(memory) + + def _kind(self, raw: object) -> str: + """Name a result's record type: ``model.grid_response`` or the compact one.""" + if self._hacc is not None: + namespace, prefix = self._hacc.hacc, "hacc" + else: + namespace, prefix = self._provider, "sae" + if isinstance(raw, namespace.TotalCurrentResponse): + return f"{prefix}.total_current_response" + if isinstance(raw, namespace.GridResponse): + return f"{prefix}.grid_response" + return f"other:{type(raw).__qualname__}" + + def _result(self, raw: _Native) -> dict[str, object]: + """Return one atomic call's record as returned, its type and model match.""" + info = getattr(raw, "info", None) + match = None + if self._hacc is not None and info is not None: + try: + match = self._hacc.matches(info) + except AttributeError, TypeError, ValueError: + match = False + return {"kind": self._kind(raw), "raw": raw, "model_match": match} + + def _evaluate(self, request: Mapping[str, Any]) -> object: + """Make one atomic call from the prepared state on one chunk's drive.""" + return self._result(self._plans[request["rows"]].evaluate(request["drive"])) + + # -- the haCC consistency diagnostic -------------------------------------------- + + def _diagnostic_plan(self, request: Mapping[str, Any]) -> object: + """Build a temporary one-row plan on the protocol lattice; return its memory.""" + model = self._hacc + if model is None: + raise BackendError("the consistency diagnostic needs a haCC binding") + self._diagnostic_close({}) + stack = ExitStack() + try: + plan = stack.enter_context( + model.plan( + rows=1, spacing=request["spacing"], samples=request["samples"] + ) + ) + model.check( + plan, rows=1, samples=request["samples"], spacing=request["spacing"] + ) + except BaseException: + stack.close() + raise + self._diagnostic = (stack, plan) + return { + "memory": { + f"diagnostic.{entry.name}": entry.bytes + for entry in plan.memory_ledger().entries + if not entry.shared + }, + "hacc": model.record(), + } + + def _diagnostic_evaluate(self, request: Mapping[str, Any]) -> object: + """Make the diagnostic's one atomic call.""" + if self._diagnostic is None: + raise BackendError("no consistency diagnostic plan is open") + return self._result(self._diagnostic[1].evaluate(request["drive"])) + + def _diagnostic_close(self, request: Mapping[str, Any]) -> object: + """Close the temporary plan, if one is open.""" + del request + if self._diagnostic is not None: + stack, self._diagnostic = self._diagnostic[0], None + stack.close() + return None + + +def make_adapter() -> PyroraptorAdapter: + """Return the adapter the worker serves with (its entry point).""" + return PyroraptorAdapter() diff --git a/src/solprop/driver/_source_worker.py b/src/solprop/driver/_source_worker.py new file mode 100644 index 000000000..0ee8d9553 --- /dev/null +++ b/src/solprop/driver/_source_worker.py @@ -0,0 +1,808 @@ +# SPDX-FileCopyrightText: 2026 John C. Travers and contributors +# +# SPDX-License-Identifier: GPL-3.0-or-later + +"""Run a source provider in a worker process of its own (SEP 49 section 6.5). + +Provider-agnostic: the start, framing, handshake, lifetime, signals and cancellation of +a worker in which an *adapter* module runs a provider. Nothing here knows TDSE or +Pyroraptor; ``solprop.driver._pyroraptor`` is one adapter, and the tests run a +manufactured one. + +The parent starts ``sys.executable -m solprop.driver._source_worker `` +with :mod:`subprocess` -- never ``multiprocessing``, whose spawn re-imports the parent's +``__main__`` and whose fork inherits its runtimes -- in a session of its own, so that a +terminal's Ctrl-C or a scheduler's signal to the job reaches the parent alone, which +then ends the worker. The worker moves the protocol off its standard input and output +before importing the adapter, points standard output at standard error, ignores SIGINT +and SIGTERM, and exits at once when its pipe from the parent reaches end of file, even +during a call. + +**Frames.** An 8-byte little-endian length, then the payload. A message is one control +frame, a UTF-8 JSON object ``{"kind", "body", "arrays"}``, followed by one raw frame per +entry of ``arrays``, each the array's bytes in the dtype (byte order included) and shape +its entry names. Nothing is pickled. Every frame's declared length is checked against +the channel's limit, and an array frame's against its declared shape, before anything is +allocated. + +**Values.** A body is JSON with tagged objects: ``{"$a": i}`` the i-th array, ``{"$t": +[...]}`` a tuple, ``{"$d": {...}}`` a mapping, ``{"$f": "nan"}`` a non-finite float and +``{"$r": type, "f": fields, "c": computed}`` a record -- a dataclass or a plain object's +attributes, received as a :class:`Record`; any other object travels as its type and +``repr`` only (:class:`Opaque`). +""" + +import contextlib +import dataclasses +import importlib +import json +import math +import os +import queue +import signal +import struct +import subprocess +import sys +import threading +import traceback +import weakref +from collections import deque +from collections.abc import Callable, Iterator, Mapping, Sequence +from types import SimpleNamespace +from typing import IO, Any, Protocol, cast, runtime_checkable + +import numpy as np + +from solprop._interrupts import poll, raise_held +from solprop._process import child_environment +from solprop.exceptions import BackendError +from solprop.log import StopFlag + +__all__ = [ + "CONTROL_LIMIT", + "POLL_INTERVAL", + "PROTOCOL_VERSION", + "Adapter", + "Opaque", + "Record", + "RemoteError", + "Worker", + "WorkerError", + "decode", + "encode", + "worker_environment", +] + +PROTOCOL_VERSION = 1 +"""The frame and message protocol both sides speak; a mismatch is refused.""" + +CONTROL_LIMIT = 1 << 26 +"""The default bound on one frame, 64 MiB, raised by a caller to its prepared shapes.""" + +POLL_INTERVAL = 0.2 +"""Seconds between the wait loop's checks for a signal or a stop (section 6.7).""" + +CLOSE_GRACE = 5.0 +"""Seconds an orderly close between calls waits for the worker before killing it.""" + +_HEADER = struct.Struct(" None: + super().__init__(f"{type_name.rpartition('.')[2]}: {detail}") + self.type_name = type_name + self.detail = detail + self.classification = classification + self.remote_traceback = remote_traceback + + +class Record(SimpleNamespace): + """A provider record received from the worker: its fields as attributes. + + ``record_type`` names the original type; ``computed`` holds values the adapter + computed from it (a property, a method's result) beside its fields, readable as + attributes but not among ``vars()``, so a record's fields stay what was sent. + """ + + __slots__ = ("computed", "record_type") + + def __init__( + self, + record_type: str, + fields: Mapping[str, object], + computed: Mapping[str, object] | None = None, + ) -> None: + super().__init__(**fields) + self.record_type = record_type + self.computed = dict(computed or {}) + + def __getattr__(self, name: str) -> object: + """Return a computed value of that name; called only for absent fields.""" + try: + return object.__getattribute__(self, "computed")[name] + except KeyError: + msg = f"{self.record_type} record has no field {name!r}" + raise AttributeError(msg) from None + + +@dataclasses.dataclass(frozen=True) +class Opaque: + """An object the codec does not carry: its type and ``repr`` only. + + Attributes + ---------- + type_name : str + Its ``module.qualname``. + text : str + Its ``repr``. + """ + + type_name: str + text: str + + +def _name(kind: type) -> str: + """Return a type's ``module.qualname``.""" + return f"{kind.__module__}.{kind.__qualname__}" + + +def encode( + value: object, computed: Callable[[object], Mapping[str, object]] | None = None +) -> tuple[object, list[np.ndarray]]: + """Encode a value as tagged JSON and the arrays it refers to. + + - ``computed``: for a record, the values to send beside its fields (an adapter's + choice), or None for none. + """ + arrays: list[np.ndarray] = [] + + def _walk(item: object) -> object: # noqa: PLR0911 - one return per value kind + if item is None or isinstance(item, bool | str): + return item + if isinstance(item, int): + return int(item) + if isinstance(item, float): + return item if math.isfinite(item) else {"$f": repr(item)} + if isinstance(item, np.ndarray) and not item.dtype.hasobject: + arrays.append(item) + return {"$a": len(arrays) - 1} + if isinstance(item, np.generic): + return _walk(item.item()) + if isinstance(item, tuple): + return {"$t": [_walk(entry) for entry in cast("tuple[object, ...]", item)]} + if isinstance(item, list): + return [_walk(entry) for entry in cast("list[object]", item)] + if isinstance(item, Mapping): + mapping = cast("Mapping[object, object]", item) + return {"$d": {str(key): _walk(entry) for key, entry in mapping.items()}} + fields = _fields(item) + if fields is None: + return {"$o": _name(type(item)), "repr": repr(item)} + extra = {} if computed is None else computed(item) + if isinstance(item, Record): + extra = {**item.computed, **extra} + return { + "$r": item.record_type if isinstance(item, Record) else _name(type(item)), + "f": {name: _walk(entry) for name, entry in fields.items()}, + "c": {name: _walk(entry) for name, entry in extra.items()}, + } + + return _walk(value), arrays + + +def _fields(value: object) -> dict[str, object] | None: + """Return a record's fields: a dataclass's, or a plain object's attributes.""" + if dataclasses.is_dataclass(value) and not isinstance(value, type): + return { + item.name: getattr(value, item.name) for item in dataclasses.fields(value) + } + if isinstance(value, Opaque | type) or callable(value): + return None + attributes = getattr(value, "__dict__", None) + return None if attributes is None else dict(attributes) + + +def decode(tree: object, arrays: Sequence[np.ndarray]) -> object: # noqa: PLR0911 + """Rebuild a value from tagged JSON and its arrays, inverting :func:`encode`.""" + if isinstance(tree, list): + return [decode(item, arrays) for item in cast("list[object]", tree)] + if not isinstance(tree, dict): + return tree + node = cast("dict[str, Any]", tree) + if "$a" in node: + return arrays[node["$a"]] + if "$t" in node: + return tuple(decode(item, arrays) for item in node["$t"]) + if "$d" in node: + return {key: decode(item, arrays) for key, item in node["$d"].items()} + if "$f" in node: + return float(node["$f"]) + if "$o" in node: + return Opaque(node["$o"], node["repr"]) + return Record( + node["$r"], + {key: decode(item, arrays) for key, item in node["f"].items()}, + {key: decode(item, arrays) for key, item in node["c"].items()}, + ) + + +# ------------------------------------------------------------------------- frames + + +def _read_exact(read: Callable[[int], bytes], size: int) -> bytearray | None: + """Read exactly `size` bytes; None at end of file before the first byte.""" + buffer = bytearray() + while len(buffer) < size: + chunk = read(size - len(buffer)) + if not chunk: + if buffer: + msg = "the worker pipe ended inside a frame" + raise WorkerError(msg) + return None + buffer += chunk + return buffer + + +def _read_frame(read: Callable[[int], bytes], limit: int) -> bytearray | None: + """Read one frame, refusing a declared length over `limit` before allocating.""" + header = _read_exact(read, _HEADER.size) + if header is None: + return None + (length,) = _HEADER.unpack(header) + if length > limit: + msg = f"a worker frame declares {length} bytes, over its bound of {limit}" + raise WorkerError(msg) + payload = _read_exact(read, length) if length else bytearray() + if payload is None: + msg = "the worker pipe ended inside a frame" + raise WorkerError(msg) + return payload + + +def read_message( + read: Callable[[int], bytes], limit: int +) -> tuple[str, object, list[np.ndarray]] | None: + """Read one message: its kind, its decoded body and its arrays; None at EOF.""" + control = _read_frame(read, limit) + if control is None: + return None + try: + message = json.loads(control.decode("utf-8")) + kind, body, specs = message["kind"], message["body"], message["arrays"] + if not isinstance(kind, str) or not isinstance(specs, list): + raise TypeError(type(message).__name__) + arrays: list[np.ndarray] = [] + for spec in cast("list[dict[str, Any]]", specs): + dtype = np.dtype(spec["dtype"]) + shape = tuple(int(extent) for extent in spec["shape"]) + if ( + dtype.hasobject + or dtype.kind not in "biufc" + or min(shape, default=0) < 0 + ): + raise TypeError(str(dtype)) + expected = math.prod(shape) * dtype.itemsize + if expected > limit: + msg = f"a worker array of {expected} bytes is over its bound of {limit}" + raise WorkerError(msg) + payload = _read_frame(read, expected) + if payload is None or len(payload) != expected: + msg = "a worker array frame does not match its declared shape" + raise WorkerError(msg) + arrays.append(np.frombuffer(payload, dtype=dtype).reshape(shape)) + return kind, decode(body, arrays), arrays + except WorkerError: + raise + except Exception as error: + # Any message this side cannot read (bad JSON or tags, too deep, too large) + # is the worker's protocol error, never this process's. + msg = f"a malformed worker message: {type(error).__name__}: {error}" + raise WorkerError(msg) from error + + +def write_message( + write: Callable[[bytes], object], + kind: str, + value: object, + computed: Callable[[object], Mapping[str, object]] | None = None, +) -> None: + """Write one message of `kind` carrying `value`.""" + tree, arrays = encode(value, computed) + contiguous = [np.ascontiguousarray(array) for array in arrays] + control = json.dumps( + { + "kind": kind, + "body": tree, + "arrays": [ + {"dtype": array.dtype.str, "shape": list(array.shape)} + for array in contiguous + ], + }, + allow_nan=False, + ).encode("utf-8") + parts = [_HEADER.pack(len(control)), control] + for array in contiguous: + data = array.tobytes() + parts += [_HEADER.pack(len(data)), data] + write(b"".join(parts)) + + +# ---------------------------------------------------------------------- the parent + + +def worker_environment( + environ: Mapping[str, str], explicit: Mapping[str, str] +) -> dict[str, str]: + """Return the worker's environment: the parent's, less native's own wait variables. + + The OpenMP wait variables solprop-native set for itself (ADR-071), which its marker + names, are removed so that the provider's runtime keeps its own defaults; a user's + own settings pass through. The worker imports from this process's import path. + ``explicit`` (the thread count, the device) wins. + """ + environment = child_environment(environ) + environment["PYTHONPATH"] = os.pathsep.join(entry for entry in sys.path if entry) + settings = _NATIVE_WAIT_SETTINGS.get(environment.get(_NATIVE_WAIT_MARKER, ""), {}) + if settings: + for name, value in settings.items(): + if environment.get(name, "").strip() == value: + del environment[name] + del environment[_NATIVE_WAIT_MARKER] + environment.update(explicit) + return environment + + +class Worker: + """Own one worker process, its two pipes and the threads that read them. + + Started by :meth:`start`, which waits for the handshake; ended by :meth:`close` + between calls (a shutdown request, a short grace, then a kill) or :meth:`kill` + (at once). Every path reaps the process and closes both pipe ends. A worker is + never used again after a failure of any kind or a cancellation. + + - ``adapter``: the adapter module's dotted name, imported in the worker. + - ``environment``: the worker's whole environment (:func:`worker_environment`). + - ``limit``: the bound on any frame either way, bytes. + - ``stop``: the run's flag, polled while waiting. + """ + + def __init__( + self, + adapter: str, + *, + environment: Mapping[str, str], + limit: int = CONTROL_LIMIT, + stop: StopFlag | None = None, + ) -> None: + self.adapter = adapter + self.limit = limit + self.stop = stop + self.identity: Mapping[str, object] = {} + self.pid: int | None = None + self._environment = dict(environment) + self._process: subprocess.Popen[bytes] | None = None + self._replies: queue.Queue[tuple[str, object] | BaseException | None] = ( + queue.Queue() + ) + self._tail: deque[str] = deque(maxlen=_TAIL_LINES) + self._threads: dict[IO[bytes], threading.Thread] = {} + self._usable = False + self._finalizer: weakref.finalize[Any, Any] | None = None + + # -- lifetime ------------------------------------------------------------------- + + def start(self) -> Mapping[str, object]: + """Start the worker and return its handshake's provider identity. + + A :class:`RemoteError` if the adapter could not start (a missing provider, say); + a :class:`WorkerError` if the worker exits first or speaks another protocol. + """ + creation = 0 + if sys.platform == "win32": # pragma: no cover - Windows only + creation = subprocess.CREATE_NEW_PROCESS_GROUP + self._process = subprocess.Popen( + [ + sys.executable, + # The parent's working directory is not on the worker's path. + "-P", + # The package imports this module before ``-m`` runs it, which runpy + # warns of; ``main`` runs the imported copy, so nothing is duplicated. + "-W", + "ignore::RuntimeWarning:runpy", + "-m", + "solprop.driver._source_worker", + self.adapter, + str(self.limit), + ], + stdin=subprocess.PIPE, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + env=self._environment, + start_new_session=sys.platform != "win32", + creationflags=creation, + ) + process = self._process + assert process.stdout is not None and process.stderr is not None + # The threads hold the queue, the stream and the tail, never this object, so + # a worker its owner forgets is collected, and its finaliser kills it. + readers = ( + (_reader, process.stdout, (self.limit, self._replies)), + (_drain, process.stderr, (self._tail,)), + ) + for target, stream, extra in readers: + thread = threading.Thread(target=target, args=(stream, *extra), daemon=True) + thread.start() + self._threads[stream] = thread + self._finalizer = weakref.finalize(self, _end, process, self._threads) + try: + kind, body = self._wait() + if kind != "hello": + msg = f"the source worker opened with {kind!r}, not a handshake" + raise WorkerError(msg) + hello = cast("Mapping[str, object]", body) + if hello.get("protocol") != PROTOCOL_VERSION: + msg = ( + f"the source worker speaks protocol {hello.get('protocol')!r}, " + f"not {PROTOCOL_VERSION}" + ) + raise WorkerError(msg) + except BaseException: + self.kill() + raise + self.pid = cast("int", hello.get("pid")) + self.identity = cast("Mapping[str, object]", hello.get("identity", {})) + self._usable = True + return self.identity + + def request(self, kind: str, value: object = None) -> object: + """Send one request and return the adapter's reply. + + Waits in :data:`POLL_INTERVAL` steps; between them signal handlers run, a + signal a commit holds is raised (:func:`solprop._interrupts.raise_held`) and the + run's stop flag is polled. Any interruption kills the worker before it + propagates. An exception the adapter reported is a :class:`RemoteError`; a + worker that exits, closes its pipe or sends a malformed message is a + :class:`WorkerError`. Either way the worker is ended: it is never used again + after a failure (section 6.5), and its owner starts a fresh one. + """ + if not self._usable or self._process is None: + msg = "the source worker is not running" + raise WorkerError(msg) + process = self._process + assert process.stdin is not None + stdin = process.stdin + try: + try: + write_message(stdin.write, kind, value) + stdin.flush() + except OSError as broken: + # A worker that died between calls: its status and stderr say why. + raise self._exited() from broken + reply, body = self._wait() + except BaseException: + self.kill() + raise + if reply != "reply": + self.kill() + msg = f"the source worker answered {kind!r} with {reply!r}" + raise WorkerError(msg) + return body + + def close(self) -> None: + """End the worker between calls: ask it to exit, then kill it after a grace.""" + process = self._process + if process is None: + return + try: + if self._usable and process.poll() is None: + assert process.stdin is not None + write_message(process.stdin.write, "close", None) + process.stdin.flush() + process.wait(timeout=CLOSE_GRACE) + except OSError, subprocess.TimeoutExpired, ValueError: + pass + finally: + # A signal during the grace still ends the worker before it propagates. + self.kill() + + def kill(self) -> None: + """End the worker at once, reap it and close both pipe ends.""" + self._usable = False + if self._finalizer is not None: + self._finalizer() + + @property + def alive(self) -> bool: + """Whether the worker is running and may take a request.""" + return ( + self._usable and self._process is not None and self._process.poll() is None + ) + + # -- waiting -------------------------------------------------------------------- + + def _wait(self) -> tuple[str, object]: + """Return the next message, polling signals and the stop flag meanwhile.""" + while True: + try: + item = self._replies.get(timeout=POLL_INTERVAL) + except queue.Empty: + raise_held() + poll(self.stop) + continue + if isinstance(item, BaseException) or item is None: + # A stop already asked for owns a worker exit seen after it (6.7). + raise_held() + poll(self.stop) + raise self._exited() if item is None else item + kind, body = item + if kind == "error": + report = cast("Mapping[str, str]", body) + raise RemoteError( + report["type"], + report["message"], + report["classification"], + report["traceback"], + ) + return kind, body + + def _exited(self) -> WorkerError: + """Return the failure of a worker whose pipe ended: its status, its stderr.""" + # Only a started worker is waited on, so its process and pipes exist. + process = self._process + assert process is not None and process.stderr is not None + self._usable = False + try: + status = process.wait(timeout=CLOSE_GRACE) + except subprocess.TimeoutExpired: + status = None + if status is not None: + # Its last words may still be in the pipe, which ends now that it has + # exited (unless a grandchild holds it open: the grace bounds that). + self._threads[process.stderr].join(timeout=CLOSE_GRACE) + tail = "\n".join(self._tail) + error = WorkerError( + f"the source worker ended (exit status {status}) without replying" + ) + if tail: + error.add_note(f"its last standard error:\n{tail}") + return error + + +def _end( + process: subprocess.Popen[bytes], threads: dict[IO[bytes], threading.Thread] +) -> None: + """Kill a worker's process group, reap it and close the pipes its readers left. + + A stream whose reader is still inside a read (a grandchild holding the pipe open) + is left for collection: closing it would wait on the reader's buffer lock. + """ + if process.poll() is None: + with contextlib.suppress(ProcessLookupError, PermissionError, OSError): + if sys.platform == "win32": # pragma: no cover - Windows only + process.kill() + else: + # Its own session: the group holds anything the provider started. + os.killpg(process.pid, signal.SIGKILL) + process.wait() + # Every worker is started with all three pipes. + assert process.stdin is not None + with contextlib.suppress(OSError): + process.stdin.close() + # The worker's ends are closed now, so its readers see the end of their pipes. + for stream, thread in threads.items(): + thread.join(timeout=CLOSE_GRACE) + if not thread.is_alive(): + with contextlib.suppress(OSError): + stream.close() + + +def _reader( + stream: IO[bytes], + limit: int, + replies: queue.Queue[tuple[str, object] | BaseException | None], +) -> None: + """Move the worker's messages to the reply queue until its pipe ends. + + Every way out queues something, so the wait never outlives its reader. + """ + try: + while True: + message = read_message(stream.read, limit) + if message is None: + replies.put(None) + return + kind, body, _ = message + replies.put((kind, body)) + except WorkerError as error: + replies.put(error) + except OSError, ValueError: + # The parent closed the pipe while killing the worker. + replies.put(None) + except BaseException as error: + replies.put(WorkerError(f"the worker's reply could not be read: {error!r}")) + + +def _drain(stream: IO[bytes], tail: deque[str]) -> None: + """Pass the worker's standard error on, keeping its last lines for a failure.""" + for line in iter(stream.readline, b""): + text = line.decode("utf-8", errors="replace").rstrip("\n") + tail.append(text) + with contextlib.suppress(OSError, ValueError): + sys.stderr.write(text + "\n") + + +# ----------------------------------------------------------------------- the worker + + +@runtime_checkable +class Adapter(Protocol): + """What an adapter module's ``make_adapter()`` returns, inside the worker.""" + + def identity(self) -> Mapping[str, object]: + """Return the provider's identity for the handshake.""" + ... + + def handle(self, kind: str, value: object) -> object: + """Serve one request.""" + ... + + def classify(self, error: BaseException) -> str: + """Classify an exception for the failure record.""" + ... + + def computed(self, value: object) -> Mapping[str, object]: + """Return values to send beside a record's fields.""" + ... + + def close(self) -> None: + """Release everything the adapter holds.""" + ... + + +@contextlib.contextmanager +def _private_streams() -> Iterator[tuple[int, int]]: + """Move the protocol off fds 0 and 1, pointing 1 at stderr and 0 at the null device. + + Whatever the provider or its runtime prints then goes to standard error and cannot + corrupt a frame. + """ + inbound, outbound = os.dup(0), os.dup(1) + null = os.open(os.devnull, os.O_RDONLY) + os.dup2(null, 0) + os.close(null) + os.dup2(2, 1) + sys.stdout = sys.stderr + yield inbound, outbound + + +def _watch(inbound: int, limit: int, inbox: queue.Queue[tuple[str, object]]) -> None: + """Read the parent's messages; exit the process at once when its pipe ends (P8).""" + try: + while True: + message = read_message(lambda size: os.read(inbound, size), limit) + if message is None: + break + kind, body, _ = message + inbox.put((kind, body)) + except BaseException: + pass + # The parent is gone, or wrote garbage: nothing the worker computes can be used. + os._exit(_ORPHANED) + + +def _error_report(error: BaseException, adapter: Adapter | None) -> dict[str, str]: + """Return an exception as type, message, classification and traceback text.""" + return { + "type": _name(type(error)), + "message": str(error), + "classification": "unexpected" if adapter is None else adapter.classify(error), + "traceback": "".join(traceback.format_exception(error)), + } + + +def main(argv: Sequence[str]) -> int: + """Serve one parent: handshake, requests until ``close``, then exit. + + - ``argv``: the adapter, ``module`` or ``module:factory`` (``make_adapter`` by + default), and the frame bound in bytes. + """ + adapter_name, limit = argv[0], int(argv[1]) + for number in (signal.SIGINT, signal.SIGTERM): + signal.signal(number, signal.SIG_IGN) + with _private_streams() as (inbound, outbound): + PROTOCOL_DESCRIPTORS[:] = [inbound, outbound] + + def _send(kind: str, value: object, adapter: Adapter | None = None) -> None: + write_message( + lambda data: _write_all(outbound, data), + kind, + value, + None if adapter is None else adapter.computed, + ) + + inbox: queue.Queue[tuple[str, object]] = queue.Queue() + threading.Thread( + target=_watch, args=(inbound, limit, inbox), daemon=True + ).start() + try: + module_name, _, factory = adapter_name.partition(":") + module = importlib.import_module(module_name) + adapter = cast("Adapter", getattr(module, factory or "make_adapter")()) + identity = adapter.identity() + except Exception as error: + _send("error", _error_report(error, None)) + return 1 + _send( + "hello", + {"protocol": PROTOCOL_VERSION, "pid": os.getpid(), "identity": identity}, + ) + while True: + kind, body = inbox.get() + if kind == "close": + adapter.close() + return 0 + try: + # The reply is encoded whole before a byte is written, so a reply the + # codec refuses is reported as an error, not half sent. + _send("reply", adapter.handle(kind, body), adapter) + except Exception as error: + _send("error", _error_report(error, adapter)) + + +def _write_all(descriptor: int, data: bytes) -> None: + """Write every byte to a descriptor, across partial writes.""" + view = memoryview(data) + while view: + view = view[os.write(descriptor, view) :] + + +if __name__ == "__main__": + # Run the imported module's main, not this __main__ copy's, so that the records + # an adapter receives are the classes it imports. + from solprop.driver._source_worker import main as _main + + status = 1 + try: + status = _main(sys.argv[1:]) + except BaseException: + traceback.print_exc() + finally: + # Every way out skips interpreter finalisation: the provider's runtime (Kokkos) + # is closed by the adapter or abandoned with the process, never finalised. + with contextlib.suppress(BaseException): + sys.stderr.flush() + os._exit(status) diff --git a/src/solprop/driver/settings.py b/src/solprop/driver/settings.py index 7dbca9dcb..a729bb16e 100644 --- a/src/solprop/driver/settings.py +++ b/src/solprop/driver/settings.py @@ -747,7 +747,7 @@ def _admit_tdse_execution(settings: SimulationSettings) -> None: """Refuse TDSE execution resources beside settings a TDSE run cannot step with. Each refusal names the setting at fault, in the order a caller would fix them: - the backend, then the integrator. + the execution record, then the integrator. """ if not isinstance(settings.tdse_execution, TDSEExecution): msg = ( @@ -755,14 +755,6 @@ def _admit_tdse_execution(settings: SimulationSettings) -> None: f"{type(settings.tdse_execution).__name__}" ) raise DriverError(msg, parameter="tdse_execution") - if settings.backend != "numpy_ref": - msg = ( - f"tdse_execution requires an explicit backend='numpy_ref', the one backend " - f"that runs the TDSE response; got backend={settings.backend!r}" - ) - raise DriverError( - msg, parameter=("tdse_execution", "backend"), rule="tdse-numpy-ref" - ) if not isinstance(settings.integrator, Multistep): msg = ( "tdse_execution requires integrator=Multistep(), the Adams method the TDSE " diff --git a/src/solprop/driver/simulation.py b/src/solprop/driver/simulation.py index cd2cde25c..f3a8aea23 100644 --- a/src/solprop/driver/simulation.py +++ b/src/solprop/driver/simulation.py @@ -1793,6 +1793,7 @@ def _run( # noqa: PLR0915 - run's controls, and the backend it may be handed problem, _AtomicContext(handler, state, metadata), tdse_archive, + stop, ) backend = kept or select( self.settings.backend, diff --git a/src/solprop/driver/tdse.py b/src/solprop/driver/tdse.py index b90722d39..3148bf4c9 100644 --- a/src/solprop/driver/tdse.py +++ b/src/solprop/driver/tdse.py @@ -5,8 +5,11 @@ """Own the optional TDSE runtime behind the provider-free response seam. The provider's whole-electron currents and shadow dipoles remain unscaled SI -arrays. Source assembly applies the calibration and density separately. Native -imports, persistent plans, resource admission and call accounting live here. +arrays. Source assembly applies the calibration and density separately. Resource +admission, call accounting, the archive and the validation of every returned record +live here; the provider itself runs in a worker process of its own +(``solprop.driver._source_worker``, with ``solprop.driver._pyroraptor`` as its +adapter, SEP 49), and solprop's process never imports it. Two atomic models are bound (SEP 12, SEP 20): the single-active-electron (SAE) response plan, and the helium haCC response plan, either from one of the @@ -16,31 +19,43 @@ :class:`~solprop.translate.tdse.TDSEResult` always spans the full lattice. """ +import contextlib import dataclasses -import importlib import json import math -import sys +import os import warnings -from collections.abc import Callable, Mapping +from collections.abc import Mapping from contextlib import ExitStack -from dataclasses import asdict, dataclass, is_dataclass -from importlib.metadata import PackageNotFoundError, distribution +from dataclasses import asdict, dataclass from typing import Any, Literal, Protocol, cast, runtime_checkable import numpy as np from jaxtyping import Float +from solprop import exceptions +from solprop._process import thread_budget from solprop.constants import AU_TIME, EPS0, C +from solprop.driver._pyroraptor import _PRESET_FIELDS, _admit_memory, check_info +from solprop.driver._source_worker import ( + CONTROL_LIMIT, + Record, + RemoteError, + Worker, + WorkerError, + worker_environment, +) from solprop.exceptions import ( BackendError, - DriverError, MaterialError, + RunTerminated, SolpropError, + StopRequested, TDSEArchiveError, TDSEDomainWarning, TDSEUnqualifiedModelWarning, ) +from solprop.log import StopFlag from solprop.nonlinear.tdse import ( HaccBasis, TDSEBasis, @@ -54,7 +69,6 @@ writer_memory, ) from solprop.translate.tdse import ( - HACC_NORM_GROWTH_LIMIT, AtomicCallBudget, AtomicCallLedger, AtomicWindowRecord, @@ -73,16 +87,15 @@ "estimate_binding_memory", "hacc_basis", "hacc_preset", - "pinned_revision", "prepare_binding", + "provider_revision", ] -_PROVIDER_REVISION = "876b72d10b5286478125b842ea45548a5051b033" _Preset = Literal["production", "high_accuracy", "strong_field"] _HaccVariant = Literal["standard", "fast"] -# Native classes cannot occur in runtime annotations: optional-package-free test -# collection resolves every annotation through beartype. Confine their dynamic -# typing to this adapter; the seam's scientific records and arrays remain typed. +# The adapter module the worker runs (SEP 49 section 6.5). +_ADAPTER = "solprop.driver._pyroraptor" +# Native classes never reach this process; what the worker returns are records. _Native = Any # The archive role of each evaluation purpose (SEP 20 section 5.8); a predictor or # corrector of a rejected trial reads back as "rejected". @@ -92,122 +105,147 @@ "corrected": "corrector", "diagnostic": "diagnostic", } -# The haCC consumer surface of the provider (SEP 20 section 5.7, C1 to C5). -_HACC_SURFACE = ( - "AngularLimit", - "Argon", - "Basis", - "GridResponse", - "Helium", - "MixedGauge", - "ResponsePlan", - "TotalCurrentResponse", - "VelocityGauge", - "helium_preset", - "prepare", -) # A preset's calibration admission band: from 20 eV (hc / 20 eV = 62 nm), the # presets' below-threshold bound, to 1.6 um, the band the 800 nm qualifying drive and # run_short's grid occupy (SEP 20 section 5.1). Not an independent qualification. _PRESET_WAVELENGTH_RANGE = (62e-9, 1.6e-6) +# The preset records a run's provenance carries (the provider's evidence apart). +_PRESET_RECORD = tuple(name for name in _PRESET_FIELDS if name != "source") @runtime_checkable -class _Provider(Protocol): - """Type the optional module at its single dynamic-import boundary.""" - - AngularBasis: Callable[..., _Native] - ComplexScaling: Callable[..., _Native] - Discretization: Callable[..., _Native] - DrivenError: type[Exception] - GridResponse: type - RadialBasis: Callable[..., _Native] - ResponsePlan: Callable[..., _Native] - SpectralOptions: Callable[..., _Native] - SpectralSetup: Callable[..., _Native] - StateSelection: Callable[..., _Native] - TotalCurrentResponse: type - atomic_model: Callable[[str], _Native] - build_info: Callable[[], _Native] - # The haCC namespace is looked up by name (`_hacc_surface`): an SAE-only - # provider is still a provider, and is refused only when haCC is asked for. - solve_preset: Callable[[str, _Preset], _Native] - unit_scale: Callable[[str], _Native] - - -def pinned_revision() -> str: - """Return the provider source revision this solprop is pinned to. +class _Channel(Protocol): + """The worker the binding talks to, a ``_source_worker.Worker``.""" + + identity: Mapping[str, object] + + @property + def alive(self) -> bool: + """Whether the worker may take a request.""" + ... + + def request(self, kind: str, value: object = None) -> object: + """Send one request and return its reply.""" + ... + + def close(self) -> None: + """End the worker between calls.""" + ... + + def kill(self) -> None: + """End the worker at once.""" + ... + + +def provider_revision(build: Mapping[str, object]) -> str: + """Return the provider source revision a build record names. + + solprop pins no provider revision (SEP 49 section 6.5): it uses the installed + provider and records the revision its worker reports. + + Parameters + ---------- + build : collections.abc.Mapping + A binding's or diagnostic session's build record (``_build_record``). Returns ------- str - Full Git revision; an installed build must report a prefix of it. + The provider's reported source revision, or ``""`` for an empty record. """ - return _PROVIDER_REVISION + native = build.get("native") + if not isinstance(native, Mapping): + return "" + return str(cast("Mapping[str, object]", native).get("source_revision", "")) + +def _open_provider( + *, stop: StopFlag | None = None, limit: int = CONTROL_LIMIT +) -> _Channel: + """Start a worker running the provider's adapter and wait for its handshake. -def _load_provider() -> _Provider: - """Load the optional package only when an atomic operation is requested.""" - if sys.version_info[:2] != (3, 14): - raise BackendError("the pinned TDSE provider requires Python 3.14") + The worker gets the process's whole thread budget (SEP 49 section 6.6) and the + parent's environment less native's own OpenMP wait settings. + + - ``stop``: the run's flag, polled while the worker is awaited. + - ``limit``: the bound on any frame, bytes. + """ + environment = worker_environment( + os.environ, {"PYRORAPTOR_THREADS": str(thread_budget().threads)} + ) + worker = Worker(_ADAPTER, environment=environment, limit=limit, stop=stop) try: - return cast("_Provider", importlib.import_module("pyroraptor")) - except (ImportError, OSError) as error: - raise BackendError( - "TDSE response requires the optional pyroraptor dependency; " - "install the pinned Pyroraptor revision into the environment, " - "with a matching CPU, CUDA or HIP build" - ) from error + worker.start() + except RemoteError as error: + raise _local_error(error, "TDSE provider start failed") from error + return worker + + +def _local_error(error: RemoteError, context: str) -> SolpropError: + """Return the worker's exception as this process raises it. + + solprop's own exceptions keep their class and message; any other becomes a + :class:`BackendError` naming `context`, the message and its classification. + """ + module, _, name = error.type_name.rpartition(".") + kind = getattr(exceptions, name, None) if module == "solprop.exceptions" else None + if isinstance(kind, type) and issubclass(kind, SolpropError): + try: + local = kind(error.detail) + except TypeError: + pass + else: + local.add_note( + f"raised in the TDSE provider worker:\n{error.remote_traceback}" + ) + return local + wrapped = BackendError(f"{context}: {error.detail} [{error.classification}]") + wrapped.add_note(f"the worker's traceback:\n{error.remote_traceback}") + return wrapped def _build_record( - provider: _Provider, execution: TDSEExecution | None + identity: Mapping[str, object], execution: TDSEExecution | None ) -> dict[str, object]: - """Refuse stale native extensions and execution-space mismatches before setup. + """Refuse an execution-space mismatch the worker reported; return its build. + No provider revision is refused: solprop uses the installed provider, whose + revision, build and execution space the record keeps (SEP 49 section 6.5). + + - ``identity``: the handshake's provider identity: its build and distribution. - ``execution``: required device family, or None when only resolving a preset. - Returns: serializable native build and installed distribution identities. """ - build = provider.build_info() - revision = build.source_revision - if len(revision) < 7 or not _PROVIDER_REVISION.startswith(revision): - raise BackendError( - f"TDSE native source {revision!r} does not match pinned " - f"{_PROVIDER_REVISION}" - ) - space = build.execution_space.lower() + build = cast("Mapping[str, object]", identity["native"]) + space = str(build["execution_space"]).lower() devices = {"cpu": {"openmp", "serial", "threads"}, "cuda": {"cuda"}, "hip": {"hip"}} if execution is not None and space not in devices[execution.device]: raise BackendError( f"TDSE requested device={execution.device!r}, but the native build uses " - f"{build.execution_space!r}" + f"{build['execution_space']!r}" ) record: dict[str, object] = { - "native": asdict(build), - "source_pin": _PROVIDER_REVISION, + "native": dict(build), + "distribution": identity.get("distribution"), } - try: - package = distribution("pyroraptor") - except PackageNotFoundError: - record["distribution"] = None - else: - record["distribution"] = package.version - source = package.read_text("direct_url.json") - record["direct_url"] = None if source is None else json.loads(source) + if "direct_url" in identity: + record["direct_url"] = identity["direct_url"] return record -def _hacc_surface(provider: _Provider) -> _Native: - """Return the provider's haCC namespace, refusing a build without it.""" - hacc = getattr(provider, "hacc", None) - missing = [name for name in _HACC_SURFACE if not hasattr(hacc, name)] - if missing: - raise BackendError( - "the installed TDSE provider has no haCC consumer surface " - f"(missing {', '.join(missing)}); install Pyroraptor at the pinned " - f"revision {_PROVIDER_REVISION}" - ) - return hacc +def _query(kind: str, request: Mapping[str, object]) -> tuple[Any, str]: + """Ask a short-lived worker one question (a preset, a basis) and end it. + + - Returns: the reply, and the provider revision the worker reported. + """ + worker = _open_provider() + try: + revision = provider_revision(_build_record(worker.identity, None)) + return worker.request(kind, dict(request)), revision + except RemoteError as error: + raise _local_error(error, "TDSE provider query failed") from error + finally: + worker.close() def basis_from_preset(target: str, preset: _Preset) -> TDSEBasis: @@ -218,7 +256,7 @@ def basis_from_preset(target: str, preset: _Preset) -> TDSEBasis: target : str Atomic target, currently ``helium`` only. preset : {'production', 'high_accuracy', 'strong_field'} - Name of the pinned provider's measured basis and spectral cut. + Name of the provider's measured basis and spectral cut. Returns ------- @@ -231,7 +269,7 @@ def basis_from_preset(target: str, preset: _Preset) -> TDSEBasis: MaterialError If the target or preset is unsupported. BackendError - If the optional provider is missing or its native revision is stale. + If the optional provider is missing. """ if target != "helium" or preset not in ( "production", @@ -239,127 +277,15 @@ def basis_from_preset(target: str, preset: _Preset) -> TDSEBasis: "strong_field", ): raise MaterialError("TDSE requires helium and a supported provider preset") - provider = _load_provider() - _build_record(provider, None) - selected = provider.solve_preset(target, preset) - basis = selected.discretization - scaling = basis.scaling - return TDSEBasis( - name=selected.name, - radial_edges=basis.radial.edges, - radial_orders=basis.radial.orders, - lmax=basis.angular.lmax, - cut_energy=selected.cut_energy, - source=f"pyroraptor@{_PROVIDER_REVISION}: {selected.source}", - m=basis.angular.m, - potential=basis.model.name, - potential_version=basis.model.version, - scaling_radius=None if scaling is None else scaling.radius, - scaling_angle=None if scaling is None else scaling.angle, - infinite_order=None if scaling is None else scaling.infinite_order, - decay=None if scaling is None else scaling.decay, - cuts=basis.cuts, - ) + reply, revision = _query("basis_from_preset", {"target": target, "preset": preset}) + fields = dict(reply) + provider_source = fields.pop("provider_source") + return TDSEBasis(**fields, source=f"pyroraptor@{revision}: {provider_source}") # ------------------------------------------------------------------- haCC records -def _hacc_fields( # noqa: PLR0917 - called positionally - hacc: _Native, - atom: _Native, - basis: _Native, - gauge: _Native, - cut_energy: float, - max_step_au: float, -) -> dict[str, object]: - """Convert provider haCC objects into the numeric fields of a ``HaccBasis``. - - - ``basis``: ``hacc.Basis``: radial axis, irECS and angular limit, SI. - - ``gauge``: ``hacc.MixedGauge`` or ``hacc.VelocityGauge``. - - ``cut_energy``: spectral cut in J. - - ``max_step_au``: RK4 step ceiling in atomic units of time. - - Returns: every ``HaccBasis`` field but ``source`` and ``preset``. - """ - if isinstance(atom, hacc.Argon): - raise MaterialError( - "argon TDCIS is not representable as a HaccBasis: the haCC target is " - "helium only" - ) - if not isinstance(atom, hacc.Helium): - raise MaterialError("a HaccBasis needs a provider hacc.Helium atom") - if atom.ions not in (1, 2): - raise MaterialError( - f"Helium(ions={atom.ions}) is not representable: HaccBasis admits " - "haCC(1) (ions=1) and haCC(5) (ions=2)" - ) - if isinstance(gauge, hacc.MixedGauge): - gauge_radius: float | None = float(gauge.radius) - elif isinstance(gauge, hacc.VelocityGauge): - gauge_radius = None - else: - raise MaterialError("a HaccBasis gauge must be MixedGauge or VelocityGauge") - angular = basis.angular - limit = "particle" if angular.particle is not None else "total" - scaling = basis.scaling - return { - "ions": atom.ions, - "neutral_square": atom.neutral_square, - "radial_edges": tuple(float(edge) for edge in basis.radial.edges), - "radial_orders": tuple(basis.radial.orders), - "angular_limit": limit, - "angular_order": angular.particle if limit == "particle" else angular.total, - "scaling_radius": float(scaling.radius), - "scaling_angle": float(scaling.angle), - "infinite_order": scaling.infinite_order, - "decay": float(scaling.decay), - "gauge_radius": gauge_radius, - "cut_energy": float(cut_energy), - "max_step_au": float(max_step_au), - } - - -def _numeric_fields(basis: HaccBasis) -> dict[str, object]: - """Return a ``HaccBasis``'s fields without its provenance and preset identity. - - - Returns: the model and numerical fields compared against the provider. - """ - record = asdict(basis) - del record["source"], record["preset"] - return record - - -def _hacc_objects( - provider: _Provider, hacc: _Native, basis: HaccBasis -) -> tuple[_Native, _Native, _Native]: - """Build the provider's atom, basis and gauge from a ``HaccBasis``. - - - Returns: ``hacc.Helium``, ``hacc.Basis`` and the gauge. - """ - angular = ( - hacc.AngularLimit(particle=basis.angular_order) - if basis.angular_limit == "particle" - else hacc.AngularLimit(total=basis.angular_order) - ) - atom = hacc.Helium(ions=basis.ions, neutral_square=basis.neutral_square) - native = hacc.Basis( - provider.RadialBasis(basis.radial_edges, basis.radial_orders), - provider.ComplexScaling( - basis.scaling_radius, - basis.scaling_angle, - basis.infinite_order, - basis.decay, - ), - angular, - ) - gauge = ( - hacc.VelocityGauge() - if basis.gauge_radius is None - else hacc.MixedGauge(basis.gauge_radius) - ) - return atom, native, gauge - - @dataclass(frozen=True) class HaccPreset: """Hold a provider's qualified helium haCC preset as solprop records (SEP 20). @@ -492,7 +418,7 @@ def hacc_preset(ions: Literal[1, 2], variant: _HaccVariant = "standard") -> Hacc If ``ions`` is not the integer 1 or 2, or ``variant`` is not ``'standard'`` or ``'fast'``. BackendError - If the optional provider is missing, stale or has no haCC surface, or its + If the optional provider is missing or has no haCC surface, or its preset's variant disagrees with the request. Examples @@ -506,49 +432,37 @@ def hacc_preset(ions: Literal[1, 2], variant: _HaccVariant = "standard") -> Hacc raise MaterialError("hacc_preset needs ions=1 (haCC(1)) or ions=2 (haCC(5))") if not isinstance(variant, str) or variant not in ("standard", "fast"): raise MaterialError("hacc_preset needs variant='standard' or variant='fast'") - provider = _load_provider() - _build_record(provider, None) - hacc = _hacc_surface(provider) - preset = hacc.helium_preset(ions, variant) - if preset.variant != variant: + reply, revision = _query("hacc_preset", {"ions": ions, "variant": variant}) + preset = cast("Mapping[str, Any]", reply["preset"]) + if preset["variant"] != variant: raise BackendError( - f"the provider returned the {preset.variant!r} haCC preset for the " + f"the provider returned the {preset['variant']!r} haCC preset for the " f"{variant!r} variant" ) - source = f"pyroraptor@{_PROVIDER_REVISION}: {preset.source}" + source = f"pyroraptor@{revision}: {preset['source']}" basis = HaccBasis( - **cast( - "dict[str, Any]", - _hacc_fields( - hacc, - preset.atom, - preset.basis, - preset.gauge, - preset.cut_energy, - preset.max_step_au, - ), - ), + **cast("dict[str, Any]", reply["fields"]), source=source, - preset=preset.identity, + preset=preset["identity"], ) if basis.variant != variant: raise BackendError( - f"the provider's {variant!r} preset identity {preset.identity!r} does " + f"the provider's {variant!r} preset identity {preset['identity']!r} does " "not name that variant" ) return HaccPreset( basis=basis, variant=variant, - step_margin=float(preset.step_margin), + step_margin=float(preset["step_margin"]), max_atomic_step=basis.max_step_au * AU_TIME, - accuracy=float(preset.accuracy), - box_sensitivity=float(preset.box_sensitivity), - gauge_sensitivity=float(preset.gauge_sensitivity), - max_peak_intensity=float(preset.max_peak_intensity), - wavelength=float(preset.wavelength), - window=float(preset.window), - device_admissible=bool(preset.device_admissible), - identity=preset.identity, + accuracy=float(preset["accuracy"]), + box_sensitivity=float(preset["box_sensitivity"]), + gauge_sensitivity=float(preset["gauge_sensitivity"]), + max_peak_intensity=float(preset["max_peak_intensity"]), + wavelength=float(preset["wavelength"]), + window=float(preset["window"]), + device_admissible=bool(preset["device_admissible"]), + identity=preset["identity"], source=source, ) @@ -598,8 +512,18 @@ def hacc_basis( BackendError If the optional provider is missing or has no haCC surface. """ - hacc = _hacc_surface(_load_provider()) - fields = _hacc_fields(hacc, atom, basis, gauge, cut_energy, max_step_au) + # The objects travel to the worker as records of their fields, and are checked + # there against the provider's own classes (SEP 49 section 6.5). + fields, _ = _query( + "hacc_basis", + { + "atom": atom, + "basis": basis, + "gauge": gauge, + "cut_energy": cut_energy, + "max_step_au": max_step_au, + }, + ) return HaccBasis(**cast("dict[str, Any]", fields), source=source) @@ -650,14 +574,20 @@ def _estimated_memory( descriptor's; None takes ``descriptor.outputs``. - Returns: named conservative atomic storage estimates in bytes; vendor setup workspaces and allocator overhead are not hard bounded. The haCC provider offers - no estimate before preparation (SEP 20, C7), so only its host histories are - estimated; its model and plans are admitted from their measured ledgers. + no estimate before preparation (SEP 20, C7), so only its host histories and the + transport copies are estimated; its model and plans are admitted from their + measured ledgers. """ samples = _atomic_samples(descriptor) outputs = descriptor.outputs if native_outputs is None else native_outputs histories = 48 if outputs == "full" else 20 + transport = _transport_bytes( + descriptor, max(chunk.rows for chunk in chunks), outputs + ) host = { - "estimate.host_histories": 8 * histories * chunks[-1].stop * descriptor.samples + "estimate.host_histories": 8 * histories * chunks[-1].stop * descriptor.samples, + # A call's copies on the pipe, in this process and in the worker (SEP 49). + "estimate.transport": 2 * transport, } basis = descriptor.response.basis if not isinstance(basis, TDSEBasis): @@ -679,23 +609,11 @@ def _estimated_memory( } -def _admit_memory(ledger: Mapping[str, int], limit: int | None) -> None: - """Apply the declared atomic byte limit. - - - ``ledger``: named persistent allocations or estimates in bytes. - - ``limit``: positive byte ceiling, or None for an unrestricted allocation. - """ - total = sum(ledger.values()) - if limit is not None and total > limit: - raise DriverError( - f"TDSE persistent data requires {total} bytes, exceeding {limit} bytes" - ) - - def _json_default(value: object) -> object: - """Convert a public native record or array into JSON-compatible containers.""" - if is_dataclass(value) and not isinstance(value, type): - return asdict(value) + """Convert a provider record or array into JSON-compatible containers.""" + if isinstance(value, Record): + # A provider record from the worker: its fields, as the dataclass gave them. + return dict(vars(value)) if isinstance(value, np.ndarray): return value.tolist() if isinstance(value, np.generic): @@ -768,15 +686,19 @@ def _row_chunks(execution: TDSEExecution, n_nodes: int) -> tuple[_Chunk, ...]: ) -def _classify(error: BaseException, provider: _Provider) -> str: - """Classify a provider-side exception for the failure record (SEP 20, 5.6). +def _classify(error: BaseException) -> str: + """Classify an evaluation's failure for its record (SEP 20, 5.6). - - Returns: ``native`` for a provider ``DrivenError``, ``interface`` for a - ``ValueError`` or ``TypeError`` from the provider, ``contract`` for a solprop - validation of the returned record, ``unexpected`` otherwise. + - Returns: the worker's classification of a provider exception (``native``, + ``interface``, ``contract`` or ``unexpected``); ``worker`` for a worker that + exited or broke the protocol; for this process's own validation, ``contract`` + for a solprop refusal, ``interface`` for a ``ValueError`` or ``TypeError`` and + ``unexpected`` otherwise. """ - if isinstance(error, provider.DrivenError): - return "native" + if isinstance(error, RemoteError): + return error.classification + if isinstance(error, WorkerError): + return "worker" if isinstance(error, SolpropError): return "contract" if isinstance(error, ValueError | TypeError): @@ -784,6 +706,33 @@ def _classify(error: BaseException, provider: _Provider) -> str: return "unexpected" +def _described(error: BaseException) -> tuple[str, str]: + """Return an error's type name and message, the worker's own for a remote one.""" + if isinstance(error, RemoteError): + return error.type_name.rpartition(".")[2], error.detail + return type(error).__name__, str(error) + + +def _frame_limit(descriptor: TDSEDescriptor, chunks: tuple[_Chunk, ...]) -> int: + """Bound one worker frame by the prepared shapes (SEP 49 section 6.5). + + The largest array either way is a chunk's shadow record, three float64 orders + of every row on the lattice; four orders' worth, twice over, is the bound. + """ + rows = max(chunk.rows for chunk in chunks) + return max(CONTROL_LIMIT, 2 * 4 * 8 * rows * descriptor.samples) + + +def _transport_bytes(descriptor: TDSEDescriptor, rows: int, outputs: str) -> int: + """Return one process's transport copies of the largest chunk's call. + + The drive twice (its frame is built whole before it is written) and the result + once (its arrays are read into their own buffers), in bytes. + """ + histories = 48 if outputs == "full" else 20 + return 8 * rows * _atomic_samples(descriptor) * (2 + histories) + + def _peak_intensity(fields: Float[np.ndarray, "n_nodes n_samples"]) -> list[float]: """Return each row's peak cycle-averaged intensity ``eps0 c E_max^2 / 2``. @@ -845,9 +794,13 @@ def prepare_binding( *, defer_plans: bool = False, archive: TDSEArchiveWriter | None = None, + stop: StopFlag | None = None, ) -> TDSEBinding: """Prepare persistent, stably chunked atomic plans for one response lattice. + The provider runs in a worker process the binding starts here and owns until it + is closed (SEP 49): every wait on it is cancelled by Ctrl-C, SIGTERM and `stop`. + Parameters ---------- descriptor : TDSEDescriptor @@ -867,6 +820,10 @@ def prepare_binding( written to it, input first and raw result before validation; a full archive builds the plans with the provider's full record. The caller owns and closes it. Its buffers are admitted with the binding's memory. + stop : solprop.log.StopFlag or None, optional + The run's stop flag, polled while an atomic call is awaited; setting it ends + the call at once, the worker killed, with + :class:`~solprop.exceptions.StopRequested`. Returns ------- @@ -893,174 +850,65 @@ def prepare_binding( n_nodes, defer_plans=defer_plans, archive=archive, + stop=stop, ) # ------------------------------------------------------------------ haCC model -class _HaccModel: - """Prepare one haCC model, verify it and build its response plans. - - The model is entered into the caller's stack first, so it closes after every - plan built on it (SEP 20, C1). A preset is checked against the installed - provider's identity and fields; a custom basis is admitted as unqualified. +@dataclass +class _HaccView: + """The binding's record of the haCC model the worker prepared (SEP 20, C1). - - ``archive``: ``"full"`` builds every plan with the provider's full record for a - full archive; the source still reads the same total and order-one currents. + Attributes + ---------- + basis : HaccBasis + The model's record. + outputs : {'full', 'total_current'} or None + The plans' native output mode; None for a custom mixed-gauge basis before + its first plan decides it. + closure_known : bool or None + Whether the plans carry the escaped-electron closure; None until known. + preset : mapping or None + The verified preset's records (its identity, variant, accuracy and domain), or + None for a custom basis. """ - def __init__( - self, - provider: _Provider, - basis: HaccBasis, - execution: TDSEExecution, - stack: ExitStack, - *, - archive: ArchiveOutputs = "required", - ) -> None: - if execution.complement == "vendor_blas": - raise BackendError( - "haCC response plans have no complement option; use the default " - "TDSEExecution(complement='deterministic'), recorded as not " - "applicable" - ) - self.hacc = _hacc_surface(provider) - self.basis = basis - self.atom, self.native_basis, self.gauge = _hacc_objects( - provider, self.hacc, basis - ) - self.preset: _Native = None - if basis.preset is not None: - self.preset = self._verified_preset(execution) - prepared = self.preset.prepare() - else: - prepared = self.hacc.prepare( - self.atom, - self.native_basis, - gauge=self.gauge, - cut_energy=basis.cut_energy, - ) - self.model = stack.enter_context(prepared) - self.memory = { - f"model.{entry.name}": entry.bytes - for entry in self.model.memory_ledger().entries - } - # The native output mode: compact with the escaped-electron closure, - # full without it (the provider refuses compact order one then, C3), and - # full throughout for a full archive. Whether the plans carry the closure - # is known beforehand for a preset (always) and for the velocity gauge - # (never); a custom mixed gauge learns it from its first plan. - self.outputs: Literal["full", "total_current"] | None = ( - "total_current" if basis.preset is not None else None - ) - self._closure: bool | None = True if basis.preset is not None else None - if basis.gauge_radius is None: - self.outputs = "full" - self._closure = False - if archive == "full": - self.outputs = "full" - - def _verified_preset(self, execution: TDSEExecution) -> _Native: - """Refuse a preset the installed provider does not reproduce.""" - preset = self.hacc.helium_preset(self.basis.ions, self.basis.variant) - if preset.identity != self.basis.preset: - raise BackendError( - f"the installed provider's haCC preset is {preset.identity!r}, not " - f"the expected {self.basis.preset!r}; regenerate the record" - ) - fields = _hacc_fields( - self.hacc, - preset.atom, - preset.basis, - preset.gauge, - preset.cut_energy, - preset.max_step_au, - ) - if fields != _numeric_fields(self.basis): - raise BackendError( - f"the installed provider's preset {preset.identity!r} does not " - "reproduce the record's fields" - ) - if execution.device != "cpu" and not preset.device_admissible: - raise BackendError( - f"the haCC preset {preset.identity!r} is not admitted on the " - f"{execution.device} device route" - ) - return preset - - @property - def closure(self) -> bool: - """Return whether the plans carry the escaped-electron closure.""" - return self._closure is True + basis: HaccBasis + outputs: str | None + closure_known: bool | None + preset: Mapping[str, Any] | None - def plan(self, *, rows: int, spacing: float, samples: int) -> _Native: - """Construct one response plan, choosing the native output mode once. + @classmethod + def from_reply(cls, basis: HaccBasis, reply: Mapping[str, Any]) -> "_HaccView": + """Return the view of the model a ``prepare`` reply describes.""" + return cls(basis, reply["outputs"], reply["closure"], reply["preset"]) - A custom mixed-gauge basis is first asked for the compact closed current; - the provider refuses it with ``ValueError`` when the gauge radius admits no - closure, and the full record is then used, as for the velocity gauge. For - a full archive every plan is full, and a custom mixed-gauge basis takes its - closure from the first plan's ``total_current_qualified``. + def update(self, reply: Mapping[str, Any]) -> None: + """Take the output mode and closure a plan decided, from a reply's record.""" + self.outputs, self.closure_known = reply["outputs"], reply["closure"] - - ``spacing``: response spacing in s. - """ - arguments = { - "rows": rows, - "spacing": float(spacing), - "samples": samples, - "max_step_au": float(self.basis.max_step_au), - "field": "electric_field", - "shadow_order": 1, - "norm_growth_limit": HACC_NORM_GROWTH_LIMIT, - } - if self.outputs is None: - try: - plan = self.hacc.ResponsePlan( - self.model, outputs="total_current", **arguments - ) - except ValueError as error: - if "closure" not in str(error): - raise - self.outputs = "full" - self._closure = False - else: - self.outputs = "total_current" - self._closure = True - return plan - plan = self.hacc.ResponsePlan(self.model, outputs=self.outputs, **arguments) - if self._closure is None: - self._closure = bool(plan.info.total_current_qualified) - return plan + @property + def closure(self) -> bool: + """Whether the plans carry the escaped-electron closure.""" + return self.closure_known is True def check_info( - self, info: _Native, *, rows: int, samples: int, spacing: float + self, raw: _Native, *, rows: int, samples: int, spacing: float ) -> None: - """Require a plan or result's ``info`` to describe exactly this model. - - - ``spacing``: expected spacing in s. - """ - if ( - info.rows != rows - or info.samples != samples - or info.spacing != spacing - or info.shadow_order != 1 - or info.outputs != self.outputs - or info.atom != self.atom - or info.basis != self.native_basis - or info.gauge != self.gauge - or info.cut_energy != self.basis.cut_energy - ): - raise BackendError( - "haCC plan metadata disagree with the prepared model or contract" - ) - if bool(info.total_current_qualified) != self.closure: - if self.preset is not None: - raise BackendError( - "provider contract violation: a haCC preset reports no " - "escaped-electron closure" - ) - raise BackendError("haCC plan closure flag disagrees with its output mode") + """Require a result's ``info`` to describe exactly this model.""" + check_info( + raw.info, + rows=rows, + samples=samples, + spacing=spacing, + outputs=self.outputs, + cut_energy=self.basis.cut_energy, + closure=self.closure, + preset=self.preset is not None, + model_match=_matches(raw), + ) def record(self) -> dict[str, object]: """Return the model's provenance for run metadata. @@ -1080,21 +928,60 @@ def record(self) -> dict[str, object]: "complement": "not_applicable", "preset_record": None if preset is None - else { - "identity": preset.identity, - "variant": preset.variant, - "step_margin": preset.step_margin, - "accuracy": preset.accuracy, - "box_sensitivity": preset.box_sensitivity, - "gauge_sensitivity": preset.gauge_sensitivity, - "max_peak_intensity": preset.max_peak_intensity, - "wavelength": preset.wavelength, - "window": preset.window, - "device_admissible": preset.device_admissible, - }, + else {name: preset[name] for name in _PRESET_RECORD}, } +def _returned(reply: object) -> _Native: + """Return an atomic call's record, typed by the worker's name for it. + + The record's ``record_type`` becomes ``sae.grid_response``, ``hacc.total_current_ + response`` and so on, or ``other:``; the worker's comparison of a haCC + result's atom, basis and gauge with the model rides beside its fields. + """ + answer = cast("Mapping[str, Any]", reply) + raw = answer["raw"] + if not isinstance(raw, Record): + # Nothing, or an object without fields, which the worker sends as its repr. + raise BackendError(f"the TDSE provider returned no result record: {raw!r}") + raw.computed["provider_type"] = raw.record_type.rpartition(".")[2] + raw.record_type = answer["kind"] + raw.computed["model_match"] = answer["model_match"] + return raw + + +def _kind(raw: object) -> str: + """Return a returned record's type name (:func:`_returned`).""" + return raw.record_type if isinstance(raw, Record) else "other" + + +def _matches(raw: object) -> bool: + """Whether the worker found a haCC result's model to be the prepared one.""" + return isinstance(raw, Record) and raw.computed.get("model_match") is True + + +def _fields_of(value: object) -> dict[str, Any]: + """Return a returned record's fields.""" + return dict(vars(value)) + + +def _component(shadow: _Native, quantity: str, order: int) -> np.ndarray: + """Return one shadow component, as the provider's ``component`` returned it. + + The worker evaluated the provider's method (``_pyroraptor``); a component it + refused raises here as the provider raised it. + """ + # The worker evaluated every quantity at each order the shadow reports, and the + # binding asks only for orders it has checked against those labels. + found = cast("Mapping[str, Any]", shadow.components)[f"{quantity}:{order}"] + if isinstance(found, Mapping): + refusal = cast("Mapping[str, str]", found) + if refusal["type"] == "ValueError": + raise ValueError(refusal["error"]) + raise RuntimeError(f"{refusal['type']}: {refusal['error']}") + return cast("np.ndarray", found) + + # --------------------------------------------------------------------- binding @@ -1113,6 +1000,7 @@ def __init__( *, defer_plans: bool = False, archive: TDSEArchiveWriter | None = None, + stop: StopFlag | None = None, ) -> None: self._archive = archive # A full archive widens the provider's selection, never the descriptor's, @@ -1132,7 +1020,6 @@ def __init__( self._chunks = _row_chunks(execution, n_nodes) self._budget = AtomicCallBudget(execution.max_atomic_calls, len(self._chunks)) self._stack = ExitStack() - self._plans: dict[int, _Native] = {} self._plan_records: dict[str, object] = {} self._closed = False self._cached: TDSEResult | None = None @@ -1140,7 +1027,7 @@ def __init__( self._diagnostic_cache_hits = 0 self._last_call: dict[str, object] = {} self._failures: list[object] = [] - self._hacc: _HaccModel | None = None + self._hacc: _HaccView | None = None self._window: AtomicWindowRecord | None = descriptor.atomic_window self._gate = descriptor.source_gate() self._samples = _atomic_samples(descriptor) @@ -1161,20 +1048,23 @@ def __init__( rows=self._chunks[0].rows, samples=descriptor.samples ) ) - self._provider = _load_provider() - self._build = _build_record(self._provider, execution) + # The worker is the binding's from here to close: its stack ends it. + self._stop = stop + self._worker = _open_provider( + stop=stop, limit=_frame_limit(descriptor, self._chunks) + ) + self._stack.callback(self._worker.close) try: + self._build = _build_record(self._worker.identity, execution) self._prepare() _admit_memory( self._memory, execution.memory_limit, ) except BaseException as error: + self._worker.kill() self._stack.close() - if not isinstance(error, Exception) or isinstance(error, SolpropError): - raise - kind = _classify(error, self._provider) - raise BackendError(f"TDSE preparation failed: {error} [{kind}]") from error + raise _preparation_error(error) from error @property def native_outputs(self) -> str | None: @@ -1190,152 +1080,115 @@ def native_outputs(self) -> str | None: # -- preparation --------------------------------------------------------------- def _prepare(self) -> None: - """Dispatch on the descriptor's model and admit the measured memory.""" + """Prepare the model and its plans in the worker; admit the measured memory.""" basis = self._descriptor.response.basis + reply = cast( + "Mapping[str, Any]", + self._worker.request("prepare", self._prepare_request(basis)), + ) + self._absorb(reply) if isinstance(basis, HaccBasis): - self._prepare_hacc(basis) - else: - self._prepare_sae(basis) + self._hacc = _HaccView.from_reply(basis, reply["hacc"]) + if basis.preset is not None: + self._preset_domain_at_start(self._hacc) + if not self._defer_plans: + self._warn_custom() # Include the simultaneously live provider histories, normalized chunks, - # concatenated result and exact-input diagnostic cache in host admission. + # concatenated result and exact-input diagnostic cache in host admission, and + # a call's copies on the pipe in each process (SEP 49 section 6.5). self._memory["host.result_and_cache_reserve"] = self._estimate[ "estimate.host_histories" ] - - def _prepare_sae(self, basis: TDSEBasis) -> None: - """Allocate shared SAE resources and one plan per distinct batch size.""" - provider = self._provider - model = provider.atomic_model(basis.potential) - if model.version != basis.potential_version: - raise BackendError( - "TDSE atomic potential revision differs from its descriptor" - ) - scaling = None - if basis.scaling_radius is not None: - assert basis.scaling_angle is not None and basis.infinite_order is not None - assert basis.decay is not None - scaling = provider.ComplexScaling( - basis.scaling_radius, - basis.scaling_angle, - basis.infinite_order, - basis.decay, - ) - discretization = provider.Discretization( - provider.RadialBasis(basis.radial_edges, basis.radial_orders), - provider.AngularBasis(basis.lmax, basis.m), - model, - scaling, - basis.cuts, - ) - operators = self._stack.enter_context(discretization.assemble()) - descriptor = self._descriptor - initial = provider.StateSelection( - descriptor.initial_l, descriptor.initial_index - ) - setup = self._stack.enter_context( - provider.SpectralSetup( - operators, - provider.SpectralOptions( - cut_energy=basis.cut_energy, - states=(initial,), - growing=cast("Literal['remove']", descriptor.growing), - growth_tolerance=descriptor.growth_tolerance, - window_duration=descriptor.window_duration, - accept_removed_component=descriptor.accept_removed_component, - ), - ) + transport = _transport_bytes( + self._descriptor, + max(chunk.rows for chunk in self._chunks), + self._native_outputs, ) - for label, ledger in ( - ("operators", operators.memory_ledger()), - ("setup", setup.memory_ledger()), - ): - for entry in ledger.entries: - self._memory[f"{label}.{entry.name}"] = entry.bytes - for rows in sorted({chunk.rows for chunk in self._chunks}): - plan = self._stack.enter_context( - provider.ResponsePlan( - setup, - rows=rows, - spacing=descriptor.spacing, - samples=self._samples, - max_step_au=( - descriptor.max_atomic_step - / provider.unit_scale("time").si_per_atomic_unit - ), - initial=initial, - field="electric_field", - shadow_order=descriptor.shadow_order, - complement=self._execution.complement, - outputs=self._native_outputs, - ) - ) - self._plans[rows] = plan - for entry in plan.memory_ledger().entries: - if not entry.shared: - self._memory[f"plan.{rows}.{entry.name}"] = entry.bytes - self._plan_records = { - str(rows): asdict(plan.info) for rows, plan in self._plans.items() - } + self._memory["transport.solprop"] = transport + self._memory["transport.worker"] = transport - def _prepare_hacc(self, basis: HaccBasis) -> None: - """Prepare the haCC model, admit its ledger and build the chunk plans.""" - model = _HaccModel( - self._provider, - basis, - self._execution, - self._stack, - archive="required" if self._archive is None else self._archive.outputs, - ) - self._hacc = model - self._memory.update(model.memory) - # Admit the model's measured ledger before any plan is built (C7). - _admit_memory( - self._memory, - self._execution.memory_limit, - ) - if basis.preset is not None: - self._preset_domain_at_start(model) - if not self._defer_plans: - self._build_hacc_plans() + def _prepare_request(self, basis: TDSEBasis | HaccBasis) -> dict[str, object]: + """Return the plain records the worker builds the model and plans from.""" + descriptor, execution = self._descriptor, self._execution + return { + "model": "hacc" if isinstance(basis, HaccBasis) else "sae", + "basis": asdict(basis), + "rows": sorted({chunk.rows for chunk in self._chunks}), + "samples": self._samples, + "spacing": float(descriptor.spacing), + "max_atomic_step": descriptor.max_atomic_step, + "shadow_order": descriptor.shadow_order, + "complement": execution.complement, + "outputs": self._native_outputs, + "archive_outputs": "required" + if self._archive is None + else self._archive.outputs, + "initial": (descriptor.initial_l, descriptor.initial_index), + "growing": descriptor.growing, + "growth_tolerance": descriptor.growth_tolerance, + "window_duration": descriptor.window_duration, + "accept_removed_component": descriptor.accept_removed_component, + "device": execution.device, + "memory": dict(self._memory), + "memory_limit": execution.memory_limit, + "defer_plans": self._defer_plans, + } - def _build_hacc_plans(self) -> None: - """Build one haCC plan per chunk size, then warn for a custom basis. + def _ensure_worker(self) -> None: + """Start and prepare a fresh worker when the last one failed (SEP 49, 6.5). - The custom-basis warning waits for the plans because whether the model - has the escaped-electron closure is known only once a plan exists. + A worker is never used again after a failure; the binding's records, budget + and archive are its own and carry over. The new worker must report the same + provider build. """ + if self._worker.alive: + return + self._worker = _open_provider( + stop=self._stop, limit=_frame_limit(self._descriptor, self._chunks) + ) + self._stack.callback(self._worker.close) + basis = self._descriptor.response.basis + try: + if _build_record(self._worker.identity, self._execution) != self._build: + raise BackendError( + "the TDSE provider's build changed between its workers" + ) + request = self._prepare_request(basis) + request["defer_plans"] = not self.plans_built + self._absorb( + cast("Mapping[str, Any]", self._worker.request("prepare", request)) + ) + except BaseException as error: + self._worker.kill() + raise _preparation_error(error) from error + + def _absorb(self, reply: Mapping[str, Any]) -> None: + """Take the worker's measured memory and plan records.""" + self._memory.update(cast("Mapping[str, int]", reply["memory"])) + self._plan_records = dict(cast("Mapping[str, object]", reply["plans"])) + if self._hacc is not None and reply["hacc"] is not None: + self._hacc.update(reply["hacc"]) + + def _warn_custom(self) -> None: + """Warn that a custom basis is unqualified, once its plans know the closure.""" model = self._hacc assert model is not None basis = model.basis - spacing = float(self._descriptor.spacing) - for rows in sorted({chunk.rows for chunk in self._chunks}): - plan = self._stack.enter_context( - model.plan(rows=rows, spacing=spacing, samples=self._samples) - ) - model.check_info( - plan.info, rows=rows, samples=self._samples, spacing=spacing - ) - self._plans[rows] = plan - for entry in plan.memory_ledger().entries: - if not entry.shared: - self._memory[f"plan.{rows}.{entry.name}"] = entry.bytes - self._plan_records = { - str(rows): asdict(plan.info) for rows, plan in self._plans.items() - } - if basis.preset is None: - missing = ( - "" - if model.closure - else "; its total current lacks the escaped-electron closure, so the " - "source uses the in-box first order and a total that omits the " - "momentum carried out through R0" - ) - warnings.warn( - f"{basis.label} is a custom haCC basis with no qualified accuracy " - f"and no preset domain; it is recorded as unqualified{missing}", - TDSEUnqualifiedModelWarning, - stacklevel=5, - ) + if basis.preset is not None: + return + missing = ( + "" + if model.closure + else "; its total current lacks the escaped-electron closure, so the " + "source uses the in-box first order and a total that omits the " + "momentum carried out through R0" + ) + warnings.warn( + f"{basis.label} is a custom haCC basis with no qualified accuracy " + f"and no preset domain; it is recorded as unqualified{missing}", + TDSEUnqualifiedModelWarning, + stacklevel=5, + ) @property def plans_built(self) -> bool: @@ -1344,7 +1197,7 @@ def plans_built(self) -> bool: - Returns: False only for a haCC binding prepared with ``defer_plans`` before :meth:`build_plans`. """ - return self._hacc is None or bool(self._plans) + return self._hacc is None or bool(self._plan_records) def build_plans(self) -> None: """Build and admit a deferred haCC binding's propagation plans. @@ -1356,41 +1209,48 @@ def build_plans(self) -> None: if self.plans_built: raise BackendError("the TDSE binding's plans are already built") try: - self._build_hacc_plans() + reply = self._worker.request( + "build_plans", + { + "memory": dict(self._memory), + "memory_limit": self._execution.memory_limit, + }, + ) + self._absorb(cast("Mapping[str, Any]", reply)) _admit_memory( self._memory, self._execution.memory_limit, ) except BaseException as error: + self._worker.kill() self.close() - if not isinstance(error, Exception) or isinstance(error, SolpropError): - raise - kind = _classify(error, self._provider) - raise BackendError(f"TDSE preparation failed: {error} [{kind}]") from error + raise _preparation_error(error) from error + self._warn_custom() - def _preset_domain_at_start(self, model: _HaccModel) -> None: + def _preset_domain_at_start(self, model: _HaccView) -> None: """Flag a preset run's duration or carrier beyond its qualification.""" preset = model.preset + assert preset is not None configuration = json.loads(self._descriptor.configuration_json) lattice = configuration["lattice"] duration = lattice["atomic_duration"] - if duration > preset.window: + if duration > preset["window"]: self._flag_domain( "duration", - {"atomic_duration": duration, "preset_window": preset.window}, + {"atomic_duration": duration, "preset_window": preset["window"]}, f"the atomic duration {duration:.6g} s exceeds the preset's tested " - f"window {preset.window:.6g} s", + f"window {preset['window']:.6g} s", ) carrier = lattice["reference_wavelength"] - if not math.isclose(carrier, preset.wavelength, rel_tol=1e-12): + if not math.isclose(carrier, preset["wavelength"], rel_tol=1e-12): self._flag_domain( "carrier", { "reference_wavelength": carrier, - "preset_wavelength": preset.wavelength, + "preset_wavelength": preset["wavelength"], }, f"the reference wavelength {carrier:.6g} m differs from the preset's " - f"qualifying carrier {preset.wavelength:.6g} m", + f"qualifying carrier {preset['wavelength']:.6g} m", ) def _flag_domain(self, reason: str, record: object, message: str) -> None: @@ -1555,6 +1415,7 @@ def evaluate( # noqa: PLR0912, PLR0915 - one evaluation's complete sequence ): self._diagnostic_cache_hits += 1 return self._cached + self._ensure_worker() archive = self._archive if archive is not None: archive.ensure_writable() @@ -1602,22 +1463,36 @@ def evaluate( # noqa: PLR0912, PLR0915 - one evaluation's complete sequence archived: TDSEArchiveError | None = None try: # Public evaluate always resets every row to the prepared state. - raw = self._plans[chunk.rows].evaluate(drive) + raw = _returned( + self._worker.request( + "evaluate", {"rows": chunk.rows, "drive": drive} + ) + ) # Captured as returned, before any validation (SEP 20, 5.8). archived = self._archive_result(evaluation, index, raw) self._validate_diagnostics(raw, chunk.rows, self._samples) chunk_records.append(self._native_record(raw, chunk)) results.append(self._normalize(raw, chunk, identity)) + except (KeyboardInterrupt, RunTerminated, StopRequested) as stopping: + # A user's stop during the call: the worker is already killed, the + # call stays charged, and it is no provider failure (SEP 49, 6.7). + self._budget.release_reservation() + self._archive_failure(evaluation, index, stopping, "interrupted") + raise except Exception as error: + # Whatever failed, this worker is not used again (SEP 49, 6.5): the + # next evaluation starts a fresh one. + self._worker.kill() self._budget.record_failure() - classification = _classify(error, self._provider) + classification = _classify(error) + name, detail = _described(error) failure = { "z": z, "row_start": chunk.start, "row_stop": chunk.stop, "input_identity": identity, "classification": classification, - "error": f"{type(error).__name__}: {error}", + "error": f"{name}: {detail}", "native": None if raw is None else self._failed_native_record(raw, chunk), @@ -1627,8 +1502,9 @@ def evaluate( # noqa: PLR0912, PLR0915 - one evaluation's complete sequence wrapped = BackendError( f"TDSE failure at z={z:.17g} m, " f"rows [{chunk.start}, {chunk.stop}), " - f"input={identity}: {error} [{classification}]" + f"input={identity}: {detail} [{classification}]" ) + _attach_remote(wrapped, error) if archived is not None: wrapped.add_note( f"the TDSE archive could not store this result: {archived}" @@ -1791,7 +1667,7 @@ def _call_domain( model = self._hacc if model is None or model.preset is None: return None - bound = float(model.preset.max_peak_intensity) + bound = float(model.preset["max_peak_intensity"]) before = _peak_intensity(fields) after = _peak_intensity(np.concatenate(drives, axis=0)) outside = max(before) > bound @@ -1816,7 +1692,7 @@ def _bound(self, raw: _Native) -> _Native: - Returns: the bound array, or None for a haCC model without the closure. """ - if self._hacc is None and isinstance(raw, self._provider.GridResponse): + if self._hacc is None and _kind(raw) == "sae.grid_response": return raw.plasma.neglected_current_bound return raw.neglected_current_bound @@ -1834,7 +1710,8 @@ def _failed_native_record(self, raw: _Native, chunk: _Chunk) -> object: } ) except (AttributeError, TypeError, ValueError) as error: - return {"unreadable_native_record": type(raw).__name__, "error": str(error)} + name = raw.computed["provider_type"] if isinstance(raw, Record) else "None" + return {"unreadable_native_record": name, "error": str(error)} def _validate_diagnostics(self, raw: _Native, rows: int, samples: int) -> None: """Reject malformed diagnostics before reducing or labelling row arrays.""" @@ -1859,7 +1736,7 @@ def _validate_diagnostics(self, raw: _Native, rows: int, samples: int) -> None: # The haCC monitors carry a bool non_finite flag beside float64 norms and # the int64 step index (SEP 20, C4). kinds = {"monitored_step": np.int64, "non_finite": np.bool_} - for name, vector in asdict(raw.monitors).items(): + for name, vector in _fields_of(raw.monitors).items(): if vector is None: continue dtype = np.dtype(kinds.get(name, np.float64)) @@ -1918,13 +1795,13 @@ def _hacc_currents( model = self._hacc assert model is not None model.check_info( - raw.info, + raw, rows=rows, samples=samples, spacing=float(self._descriptor.spacing), ) if model.outputs == "total_current": - if not isinstance(raw, model.hacc.TotalCurrentResponse): + if _kind(raw) != "hacc.total_current_response": raise BackendError( "TDSE compact contract returned a different result type" ) @@ -1937,14 +1814,16 @@ def _hacc_currents( "TDSE compact response has incorrect shadow array extents" ) return raw.total_current, raw.shadow_total_current[:, 0, :] - if not isinstance(raw, model.hacc.GridResponse) or raw.shadow is None: + if _kind(raw) != "hacc.grid_response" or raw.shadow is None: raise BackendError("haCC full output requires a response with shadows") if raw.shadow.orders != (1,): raise BackendError("TDSE full response has incorrect shadow-order labels") # With the closure the total's order one is returned; without it the # in-box order one is the first order of the field-impulse total, whose # added term is at least third order (SEP 20, 4.3). - first = raw.shadow.component("total_current" if model.closure else "current", 1) + first = _component( + raw.shadow, "total_current" if model.closure else "current", 1 + ) return raw.total_current, first def _sae_currents( @@ -1972,7 +1851,7 @@ def _sae_currents( "TDSE result metadata disagree with the prepared contract" ) if self._native_outputs == "full": - if not isinstance(raw, self._provider.GridResponse) or raw.shadow is None: + if _kind(raw) != "sae.grid_response" or raw.shadow is None: raise BackendError( "TDSE shadow model requires a full response with shadow products" ) @@ -1985,14 +1864,14 @@ def _sae_currents( # record; only a descriptor that needs it takes the third-order dipole. return ( raw.plasma.total_current, - raw.shadow.component("total_current", 1), - raw.shadow.component("total_current", 3) if 3 in orders else None, - raw.shadow.component("charge_dipole", 3) + _component(raw.shadow, "total_current", 1), + _component(raw.shadow, "total_current", 3) if 3 in orders else None, + _component(raw.shadow, "charge_dipole", 3) if descriptor.outputs == "full" else None, orders, ) - if not isinstance(raw, self._provider.TotalCurrentResponse): + if _kind(raw) != "sae.total_current_response": raise BackendError("TDSE compact contract returned a different result type") orders = raw.shadow_orders if orders != expected_orders or raw.shadow_total_current is None: @@ -2041,7 +1920,7 @@ def _normalize(self, raw: _Native, chunk: _Chunk, identity: str) -> TDSEResult: monitors_json=_encoded(self._native_record(raw, chunk)), outcomes_json=_encoded( [ - {"row": chunk.start + index, **asdict(value)} + {"row": chunk.start + index, **_fields_of(value)} for index, value in enumerate(raw.outcomes) ] ), @@ -2104,11 +1983,12 @@ def diagnostic_currents( model = self._hacc if model is None: raise BackendError("the consistency diagnostic needs a haCC binding") + self._ensure_worker() context = _DiagnosticContext( model, self._budget, self._failures, - self._provider, + self._worker, self._memory, self._execution.memory_limit, archive=self._archive, @@ -2168,7 +2048,10 @@ def memory_ledger(self) -> Mapping[str, int]: return dict(self._memory) def close(self) -> None: - """Close plans before setup, preserving metadata and attempted-call counts.""" + """End the worker, which closes the plans before the setup, keeping records. + + Metadata and attempted-call counts survive; so does a worker already killed. + """ if not self._closed: self._budget.release_reservation() self._stack.close() @@ -2183,14 +2066,14 @@ class _DiagnosticContext: Attributes ---------- - model : _HaccModel - Open prepared model. + model : _HaccView + The open prepared model's record. budget : AtomicCallBudget Ledger the call is charged to, as ``diagnostic``. failures : list Failure records the call appends to. - provider : _Provider - Loaded provider, for error classification. + worker : _Channel + The worker holding the model. memory : mapping Memory already held, in bytes. limit : int or None @@ -2202,10 +2085,10 @@ class _DiagnosticContext: for a standalone check) and the build and execution records. """ - model: _HaccModel + model: _HaccView budget: AtomicCallBudget failures: list[object] - provider: _Native + worker: _Channel memory: Mapping[str, int] limit: int | None archive: TDSEArchiveWriter | None = None @@ -2217,13 +2100,13 @@ def _diagnostic( field: Float[np.ndarray, "1 n_samples"], spacing: float, ) -> tuple[Float[np.ndarray, " n_samples"], Float[np.ndarray, " n_samples"]]: - """Run one charged one-row haCC diagnostic on a temporary plan. + """Run one charged one-row haCC diagnostic on a temporary plan in the worker. - ``field``: finite float64 field in V/m, shape ``(1, n_samples)``. - ``spacing``: protocol spacing in s. - Returns: owning ``j_tot`` and ``j_1`` in C m/s. """ - model, budget, provider = context.model, context.budget, context.provider + model, budget, worker = context.model, context.budget, context.worker if ( not isinstance(field, np.ndarray) or field.dtype != np.dtype(np.float64) @@ -2234,24 +2117,16 @@ def _diagnostic( ): raise BackendError("the consistency field must be finite float64 (1, samples)") samples = int(field.shape[1]) - with ExitStack() as stack: - try: - plan = stack.enter_context( - model.plan(rows=1, spacing=spacing, samples=samples) - ) - model.check_info(plan.info, rows=1, samples=samples, spacing=spacing) - extra = { - f"diagnostic.{entry.name}": entry.bytes - for entry in plan.memory_ledger().entries - if not entry.shared - } - except SolpropError: - raise - except Exception as error: - kind = _classify(error, provider) - raise BackendError( - f"TDSE diagnostic preparation failed: {error} [{kind}]" - ) from error + try: + prepared = cast( + "Mapping[str, Any]", + worker.request("diagnostic_plan", {"samples": samples, "spacing": spacing}), + ) + except RemoteError as error: + raise _local_error(error, "TDSE diagnostic preparation failed") from error + try: + model.update(prepared["hacc"]) + extra = cast("Mapping[str, int]", prepared["memory"]) # The temporary plan overlaps the propagation plans: admit their sum. _admit_memory( {**context.memory, **extra, "diagnostic.host": 8 * 20 * samples}, @@ -2259,51 +2134,73 @@ def _diagnostic( ) budget.reserve(1, calls=1) try: - drive = np.ascontiguousarray(field) - evaluation = _diagnostic_archive_begin(context, drive, spacing) - budget.begin_evaluation(purpose="diagnostic", calls=1) - budget.charge(purpose="diagnostic") - raw: _Native = None - archived: TDSEArchiveError | None = None - try: - raw = plan.evaluate(drive) - archived = _diagnostic_archive_result(context, evaluation, raw) - binding_view = _DiagnosticView(model, provider, spacing) - binding_view.validate(raw, samples) - total, first = binding_view.currents(raw, samples) - except Exception as error: - budget.record_failure() - kind = _classify(error, provider) - context.failures.append( - _failure_safe( - { - "purpose": "diagnostic", - "classification": kind, - "error": f"{type(error).__name__}: {error}", - } - ) - ) - wrapped = BackendError( - f"TDSE consistency diagnostic failed: {error} [{kind}]" - ) - if archived is not None: - wrapped.add_note( - f"the TDSE archive could not store this result: {archived}" - ) - _diagnostic_archive_failure(context, evaluation, error, kind, wrapped) - raise wrapped from error - if archived is not None: - raise archived - if context.archive is not None and evaluation is not None: - context.archive.finish_evaluation(evaluation, "returned") + total, first = _diagnostic_call(context, field, spacing) finally: budget.release_reservation() + finally: + # A failed or cancelled call has ended the worker already, which then refuses + # the close at once as not running: nothing is left to close. + with contextlib.suppress(RemoteError, WorkerError): + worker.request("diagnostic_close", {}) return ( np.array(total[0], dtype=np.float64, copy=True), np.array(first[0], dtype=np.float64, copy=True), ) +def _diagnostic_call( + context: _DiagnosticContext, + field: Float[np.ndarray, "1 n_samples"], + spacing: float, +) -> tuple[np.ndarray, np.ndarray]: + """Make, archive and validate the diagnostic's one charged atomic call.""" + model, budget = context.model, context.budget + samples = int(field.shape[1]) + drive = np.ascontiguousarray(field) + evaluation = _diagnostic_archive_begin(context, drive, spacing) + budget.begin_evaluation(purpose="diagnostic", calls=1) + budget.charge(purpose="diagnostic") + archived: TDSEArchiveError | None = None + try: + raw = _returned(context.worker.request("diagnostic_evaluate", {"drive": drive})) + archived = _diagnostic_archive_result(context, evaluation, raw) + binding_view = _DiagnosticView(model, spacing) + binding_view.validate(raw, samples) + total, first = binding_view.currents(raw, samples) + except (KeyboardInterrupt, RunTerminated, StopRequested) as stopping: + _diagnostic_archive_failure( + context, evaluation, stopping, "interrupted", stopping + ) + raise + except Exception as error: + context.worker.kill() + budget.record_failure() + kind = _classify(error) + name, detail = _described(error) + context.failures.append( + _failure_safe( + { + "purpose": "diagnostic", + "classification": kind, + "error": f"{name}: {detail}", + } + ) + ) + wrapped = BackendError(f"TDSE consistency diagnostic failed: {detail} [{kind}]") + _attach_remote(wrapped, error) + if archived is not None: + wrapped.add_note( + f"the TDSE archive could not store this result: {archived}" + ) + _diagnostic_archive_failure(context, evaluation, error, kind, wrapped) + raise wrapped from error + if archived is not None: + raise archived + if context.archive is not None and evaluation is not None: + context.archive.finish_evaluation(evaluation, "returned") + return total, first + + def _diagnostic_archive_begin( context: _DiagnosticContext, field: Float[np.ndarray, "1 n_samples"], @@ -2403,9 +2300,8 @@ class _DiagnosticView: - ``spacing``: protocol spacing in s. """ - def __init__(self, model: _HaccModel, provider: _Provider, spacing: float) -> None: + def __init__(self, model: _HaccView, spacing: float) -> None: self._model = model - self._provider = provider self._spacing = spacing def validate(self, raw: _Native, samples: int) -> None: @@ -2431,10 +2327,10 @@ def currents(self, raw: _Native, samples: int) -> tuple[np.ndarray, np.ndarray]: - Returns: ``(1, samples)`` currents in C m/s. """ model = self._model - model.check_info(raw.info, rows=1, samples=samples, spacing=self._spacing) + model.check_info(raw, rows=1, samples=samples, spacing=self._spacing) if model.outputs == "total_current": if ( - not isinstance(raw, model.hacc.TotalCurrentResponse) + _kind(raw) != "hacc.total_current_response" or raw.shadow_orders != (1,) or raw.shadow_total_current is None or raw.shadow_total_current.shape != (1, 1, samples) @@ -2442,12 +2338,14 @@ def currents(self, raw: _Native, samples: int) -> tuple[np.ndarray, np.ndarray]: raise BackendError("haCC compact diagnostic result is malformed") return raw.total_current, raw.shadow_total_current[:, 0, :] if ( - not isinstance(raw, model.hacc.GridResponse) + _kind(raw) != "hacc.grid_response" or raw.shadow is None or raw.shadow.orders != (1,) ): raise BackendError("haCC full diagnostic result is malformed") - first = raw.shadow.component("total_current" if model.closure else "current", 1) + first = _component( + raw.shadow, "total_current" if model.closure else "current", 1 + ) return raw.total_current, first @@ -2465,6 +2363,8 @@ class DiagnosticSession: Device, memory limit and the explicit call budget of the check. archive : TDSEArchiveWriter or None, optional Archive session receiving the diagnostic call, labelled ``diagnostic``. + stop : solprop.log.StopFlag or None, optional + A flag polled while the provider's worker is awaited. """ def __init__( @@ -2473,32 +2373,42 @@ def __init__( execution: TDSEExecution, *, archive: TDSEArchiveWriter | None = None, + stop: StopFlag | None = None, ) -> None: self._execution = execution self._archive = archive self._budget = AtomicCallBudget(execution.max_atomic_calls, 1) self._failures: list[object] = [] - self._stack = ExitStack() - self._provider = _load_provider() - self.build = _build_record(self._provider, execution) + self._worker = _open_provider(stop=stop) try: - self._model = _HaccModel( - self._provider, - basis, - execution, - self._stack, - archive="required" if archive is None else archive.outputs, + self.build = _build_record(self._worker.identity, execution) + reply = cast( + "Mapping[str, Any]", + self._worker.request( + "prepare", + { + "model": "hacc", + "basis": asdict(basis), + "rows": [], + "complement": execution.complement, + "archive_outputs": "required" + if archive is None + else archive.outputs, + "device": execution.device, + "memory": {}, + "memory_limit": execution.memory_limit, + "defer_plans": True, + }, + ), ) - self._memory = dict(self._model.memory) + self._model = _HaccView.from_reply(basis, reply["hacc"]) + self._memory = dict(cast("Mapping[str, int]", reply["memory"])) if archive is not None: self._memory.update(archive.memory_ledger(rows=1, samples=1)) _admit_memory(self._memory, execution.memory_limit) except BaseException as error: - self._stack.close() - if not isinstance(error, Exception) or isinstance(error, SolpropError): - raise - kind = _classify(error, self._provider) - raise BackendError(f"TDSE preparation failed: {error} [{kind}]") from error + self._worker.kill() + raise _preparation_error(error) from error @property def ledger(self) -> AtomicCallLedger: # numpydoc ignore=RT01 @@ -2529,7 +2439,7 @@ def diagnostic_currents( self._model, self._budget, self._failures, - self._provider, + self._worker, self._memory, self._execution.memory_limit, archive=self._archive, @@ -2571,5 +2481,26 @@ def metadata(self) -> Mapping[str, object]: ) def close(self) -> None: - """Close the model; repeated calls are harmless.""" - self._stack.close() + """End the worker and its model; repeated calls are harmless.""" + self._worker.close() + + +def _preparation_error(error: BaseException) -> BaseException: + """Return a preparation failure as the binding raises it. + + solprop's own refusals and interruptions as they are; a provider exception the + worker reported as solprop's class or ``TDSE preparation failed: ... [kind]``. + """ + if isinstance(error, RemoteError): + return _local_error(error, "TDSE preparation failed") + if not isinstance(error, Exception) or isinstance(error, SolpropError): + return error + return BackendError(f"TDSE preparation failed: {error} [{_classify(error)}]") + + +def _attach_remote(wrapped: BaseException, error: BaseException) -> None: + """Attach a worker's traceback or its standard error to the error raised.""" + if isinstance(error, RemoteError): + wrapped.add_note(f"the provider worker's traceback:\n{error.remote_traceback}") + for note in getattr(error, "__notes__", ()): + wrapped.add_note(note) diff --git a/src/solprop/driver/tdse_consistency.py b/src/solprop/driver/tdse_consistency.py index e12d1a63b..62c1d738a 100644 --- a/src/solprop/driver/tdse_consistency.py +++ b/src/solprop/driver/tdse_consistency.py @@ -892,7 +892,7 @@ def check_consistency( basis, response.consistency, reference_wavelength=reference_wavelength, - provider_revision=tdse_runtime.pinned_revision(), + provider_revision=tdse_runtime.provider_revision(build), computed_on={ "execution_space": native.get("execution_space") if isinstance(native, dict) @@ -936,7 +936,6 @@ def _open_check_archive( "label": basis.label, "protocol": PROTOCOL_VERSION, "execution": asdict(execution), - "provider_pin": tdse_runtime.pinned_revision(), }, ) diff --git a/src/solprop/driver/tdse_run.py b/src/solprop/driver/tdse_run.py index 84242ada3..94519b248 100644 --- a/src/solprop/driver/tdse_run.py +++ b/src/solprop/driver/tdse_run.py @@ -31,6 +31,7 @@ from solprop.driver.settings import SimulationSettings, _scheme from solprop.exceptions import DriverError, TDSEArchiveError, TDSECostWarning from solprop.grid import TimeGrid +from solprop.log import StopFlag from solprop.modes.material import JsonValue from solprop.modes.quadrature import TransverseQuadrature from solprop.nonlinear.tdse import TDSEAtomicWindow, TDSEExecution, TDSEGasResponse @@ -77,9 +78,12 @@ def _prepare_atomic( problem: PropagationProblem, context: _AtomicContext, tdse_archive: TDSEArchive | None, + stop: StopFlag | None = None, ) -> tuple[TDSEBinding, TDSEArchiveWriter | None]: """Open a TDSE run's archive and prepare its binding. + The binding's waits on its provider worker poll `stop` (SEP 49). + - Returns: the binding and the open archive session, or None without a request. A failure closes the archive before it propagates. """ @@ -108,6 +112,7 @@ def _prepare_atomic( problem.transform.n_nodes, defer_plans=descriptor.model == "hacc", archive=archive, + stop=stop, ) except BaseException as error: _close_tdse_archive(archive, error) @@ -166,7 +171,7 @@ def _consistency_gate( # noqa: PLR0917 - called positionally binding, declared, reference_wavelength=simulation.grid.reference_wavelength, - provider_revision=tdse_runtime.pinned_revision(), + provider_revision=tdse_runtime.provider_revision(build), computed_on={ "execution_space": native.get("execution_space"), "build": build, @@ -273,7 +278,6 @@ def _open_tdse_archive( # noqa: PLR0917 - called positionally "node_weights_m2": None if weights is None else weights.tolist(), "quadrature": metadata.quadrature, "execution": dataclasses.asdict(execution), - "provider_pin": tdse_runtime.pinned_revision(), # The file's own record on a resume, which the session holds and a snapshot # compares with. "optical_configuration_hash": ( @@ -367,13 +371,11 @@ def _check_tdse_execution( if settings.tdse_execution is not None: raise DriverError("tdse_execution requires a TDSEGasResponse") return - if ( - settings.tdse_execution is None - or settings.backend != "numpy_ref" - or not isinstance(settings.integrator, Multistep) + if settings.tdse_execution is None or not isinstance( + settings.integrator, Multistep ): raise DriverError( - "TDSE requires explicit NumPy, Multistep and tdse_execution settings" + "TDSE requires explicit Multistep and tdse_execution settings" ) if simulation.noise is not None and simulation.noise.model == "modified": raise DriverError("TDSE refuses noise='modified'; use 'input' or None") @@ -530,9 +532,9 @@ def _tdse_entry( tdse, atomic_window=dataclasses.replace(window, duration=trange) ) duration_source = "trange" - resolved_backend = "numpy_ref" if backend == "auto" else backend - if backend == "auto": - logger.info("backend='auto' resolves to numpy_ref for a TDSE response") + # Both backends run the TDSE response; "auto" is native, as for every run + # (SEP 49). + resolved_backend = "native" if backend == "auto" else backend final_only = save_planes is None and n_save is None resolved_steps = 1 if step_planes is None else step_planes record: dict[str, JsonValue] = { diff --git a/tests/backends/test_tdse_edge_free.py b/tests/backends/test_tdse_edge_free.py index b07f5ad50..f98595f84 100644 --- a/tests/backends/test_tdse_edge_free.py +++ b/tests/backends/test_tdse_edge_free.py @@ -35,7 +35,7 @@ from tests.hacc_cases import plain from solprop.backends.numpy_ref import NumpyRefBackend -from solprop.backends.numpy_ref.tdse import ( +from solprop.backends.tdse_source import ( FFT_ERROR_CONSTANT, atomic_edge_source, project_atomic_sources, @@ -681,8 +681,8 @@ def test_a_gate_with_no_sample_in_its_leading_taper_says_so(): def test_a_cubic_closure_records_an_ungated_source_and_no_certificate(): # The cubic replacements are never gated (the source filter's removal): the run # metadata says so and carries no certificate to qualify a gate. - from solprop.backends.numpy_ref.backend import _source_record # noqa: PLC0415 + from solprop.backends.tdse_source import source_record # noqa: PLC0415 case = make_case(kerr_model="optmat_cubic") assert not case.descriptor.gated - assert _source_record(case.problem) == {"gated": False} + assert source_record(case.problem) == {"gated": False} diff --git a/tests/backends/test_tdse_march.py b/tests/backends/test_tdse_march.py index 39c499e5f..450060fe4 100644 --- a/tests/backends/test_tdse_march.py +++ b/tests/backends/test_tdse_march.py @@ -21,7 +21,7 @@ from solprop.backends.session import Start from solprop.backends.sessions import start_session from solprop.driver import Simulation -from solprop.exceptions import BackendError, ResponseBudgetError +from solprop.exceptions import BackendError, ResponseBudgetError, TDSEArchiveError from solprop.nonlinear.tdse import ( TDSEBasis, TDSECalibration, @@ -232,27 +232,83 @@ def test_failed_chunk_is_charged_and_unused_trial_reservation_is_released( backend.close() -@pytest.mark.parametrize("backend_name", ["native"]) -def test_registry_refuses_tdse_before_constructing_a_backend( - march_problem, monkeypatch, backend_name +class _ArchiveFailsOnAcceptance(ManufacturedBinding): + """Fail to record an accepted trial, as an archive that cannot write would.""" + + def record_trial(self, *, accepted): + if accepted: + raise TDSEArchiveError("synthetic archive failure") + + +class _ReleaseFails(ManufacturedBinding): + """Fail to release a reservation, after releasing it, while ``failing`` is set.""" + + failing = True + + def release_reservation(self): + super().release_reservation() + if self.failing: + raise RuntimeError("synthetic release failure") + + +def test_an_archive_failure_after_an_accepted_commit_keeps_the_new_state( + march_problem, ): - """An unsupported explicit backend never silently redirects the atomic problem.""" + """The commit is the failure boundary (SEP 49, section 6.3).""" problem, run_request, grid = march_problem + binding = _ArchiveFailsOnAcceptance(tdse_descriptor(problem.route), 20) + backend = NumpyRefBackend(problem, run_request, grid=grid, tdse_binding=binding) + backend.set_initial(problem.initial.field0, None, z=problem.initial.z) + try: + with pytest.raises(TDSEArchiveError, match="synthetic archive failure"): + backend.propagate_to(1e-5) + # The accepted step stands and is booked; nothing stays reserved. + assert backend.observe().z > 0.0 + assert backend._adams is not None + assert backend._adams.ledger.accepted_steps == 1 + assert binding.ledger.reserved == 0 + # The startup and the predictor, two chunks each, stay charged. + assert binding.ledger.attempted == 4 + finally: + backend.close() - def forbidden(*args, **kwargs): - raise AssertionError("no backend is constructed for an unsupported TDSE route") - monkeypatch.setattr(registry, "_construct", forbidden) - with pytest.raises(BackendError, match=f"{backend_name} does not execute TDSE"): - registry.select(backend_name, problem, run_request, grid=grid) +def test_a_failed_release_is_a_note_on_the_trial_s_own_error(march_problem): + problem, run_request, grid = march_problem + binding = _ReleaseFails(tdse_descriptor(problem.route), 8, fail_at=2) + backend = NumpyRefBackend(problem, run_request, grid=grid, tdse_binding=binding) + backend.set_initial(problem.initial.field0, None, z=problem.initial.z) + try: + with pytest.raises(BackendError, match="second-chunk failure") as caught: + backend.propagate_to(1e-5) + assert "synthetic release failure" in "\n".join(caught.value.__notes__) + assert binding.ledger.reserved == 0 + assert backend.observe().z == 0.0 + finally: + binding.failing = False + backend.close() -def test_direct_native_construction_refuses_tdse_before_allocating(march_problem): - """The direct constructor enforces the same named refusal as the registry.""" - native = pytest.importorskip("solprop_native") +def test_registry_hands_a_tdse_binding_to_either_backend(march_problem, monkeypatch): + """Both backends run the TDSE response (SEP 49); the binding goes to the factory.""" problem, run_request, grid = march_problem - with pytest.raises(BackendError, match="native does not execute TDSE"): - native.NativeBackend(problem, run_request, grid=grid) + constructed = [] + + class ConstructedError(Exception): + pass + + def construct(name, *args, **kwargs): + constructed.append((name, kwargs.get("tdse_binding"))) + raise ConstructedError + + monkeypatch.setattr(registry, "_construct", construct) + monkeypatch.setattr(registry, "_native_installed", lambda: True) + monkeypatch.setattr("solprop.calibration.first_use.ensure", lambda: None) + binding = ManufacturedBinding(tdse_descriptor(problem.route), 10) + for name in ("numpy_ref", "native"): + with pytest.raises(ConstructedError): + registry.select(name, problem, run_request, grid=grid, tdse_binding=binding) + assert constructed == [("numpy_ref", binding), ("native", binding)] def test_tdse_problem_requires_a_binding_even_at_the_direct_backend(march_problem): diff --git a/tests/backends/test_tdse_native.py b/tests/backends/test_tdse_native.py new file mode 100644 index 000000000..aaadfa35c --- /dev/null +++ b/tests/backends/test_tdse_native.py @@ -0,0 +1,721 @@ +# SPDX-FileCopyrightText: 2026 John C. Travers and contributors +# +# SPDX-License-Identifier: GPL-3.0-or-later + +"""The TDSE response on native against the reference (SEP 49 section 7.3). + +Manufactured bindings replace the driver's binding, not the provider, and need no +worker: ``edge_source_cases.ManufacturedBinding`` returns a manufactured remainder +beside a large linear shadow. Native evaluates the source through its host source +hook, the reference through its own evaluation; both call the shared assembly +(``solprop.backends.tdse_source``), so ``F`` agrees bit for bit for the same field and +position, and what each backend does after it is compared within its budget. The +remainder is strong enough that ``F`` moves every compared field seven orders or more +above its budget, and each budgeted comparison checks that it does. +""" + +import dataclasses +import gc +import math +import signal +import weakref +from typing import Any, cast + +import numpy as np +import pytest + +from bounds import EPS, assert_within, fft_bound +from edge_source_cases import ManufacturedBinding +from sessions.executors import IDENTITY, Started, drive +from solprop import _interrupts +from solprop.backends import tdse_source +from solprop.backends.numpy_ref import NumpyRefBackend +from solprop.backends.numpy_ref.multistep import HostHistory +from solprop.backends.session import Start +from solprop.backends.sessions import start_session +from solprop.driver import Simulation +from solprop.exceptions import ( + BackendError, + ConvergenceError, + ResponseBudgetError, + RunTerminated, + TDSEArchiveError, +) +from solprop.nonlinear.tdse import ( + TDSEBasis, + TDSECalibration, + TDSEExecution, + TDSEGasResponse, + TDSESampling, +) +from solprop.stats import StatsSpec +from solprop.stepper.multistep import Multistep +from solprop.translate.routes import tdse_descriptor +from solprop.translate.tdse import TDSEResult, lattice_times, waveform_identity + +native = pytest.importorskip("solprop_native") + +pytestmark = pytest.mark.medium + +LENGTH = 1e-5 + + +def _law(times): + """A manufactured remainder ``S(t)``, A m per atom. + + Nine orders above the accounting fixtures' 1e-27 A m, so that ``F`` changes a run's + field far above any budget here: a source that never reached the field would fail. + """ + return 1e-18 * np.exp(-((times / 8e-15) ** 2)) * np.sin(2.4e15 * times) + + +def _silent(times): + """No remainder: the same binding, its source zero.""" + return np.zeros_like(times) + + +def _case( + formulation: Any = "interaction", + kerr_model: Any = "tdse_native", + *, + length=LENGTH, + noise=None, + **changes, +): + """A short helium capillary with a manufactured TDSE response.""" + dz_max = changes.pop("dz_max", 1e-6) + dz_init = changes.pop("dz_init", min(1e-6, dz_max)) + parent = Simulation.from_capillary( + 256e-6, + length, + "He", + 0.1, + wavelength=800e-9, + wavelength_limits=(400e-9, 1.6e-6), + trange=60e-15, + fwhm=2e-15, + energy=1e-12, + modes=1, + n_r=4, + n_save=2, + noise=noise, + seed=None if noise is None else 7, + plasma=False, + backend="numpy_ref", + integrator=Multistep(formulation=formulation), + dz_init=dz_init, + dz_max=dz_max, + ) + basis = TDSEBasis( + "manufactured", (0.0, 1e-9), (8,), 5, 1e-16, source="native fixture" + ) + calibration = TDSECalibration( + "native-fixture", + "research", + basis.identity, + (400e-9, 2e-6), + source="native fixture", + limitations=("no atomic calculation",), + ) + responses = [ + TDSEGasResponse( + "helium", + basis, + 1e-19, + kerr_model, + 1.0 if kerr_model == "tdse_native" else 2.0, + calibration, + TDSESampling(), + ) + ] + simulation = Simulation( + parent.grid, + parent.modes, + responses, + parent._field0, + settings=dataclasses.replace( + parent.settings, + tdse_execution=TDSEExecution(), + stats=StatsSpec(names=("energy",)), + **changes, + ), + noise=parent.noise, + profiles=parent.profiles, + frame=parent.frame, + ) + return simulation.translate(), simulation._run_request(), simulation.grid + + +class Unbudgeted(ManufacturedBinding): + """The manufactured histories with no budget: a lone evaluation's, any purpose. + + A cubic replacement's third-order current rides beside them. + """ + + def evaluate(self, fields, *, z, purpose, density=None): + del z, purpose, density + first = self.linear * fields + times = lattice_times(self.descriptor.samples, self.descriptor.spacing) + remainder = np.broadcast_to(_law(times), fields.shape) + cubic = self.descriptor.kerr_model != "tdse_native" + third = 2.0**-120 * fields**3 if cubic else None + return TDSEResult( + first + remainder + (0.0 if third is None else third), + first, + input_identity=waveform_identity(fields), + configuration_identity=self.descriptor.identity, + valid=np.ones(fields.shape, dtype=bool), + shadow_orders=(1, 3) if cubic else (1,), + third_order_current=third, + ) + + +class Recording(ManufacturedBinding): + """Record every binding call, with a failure or a hang injected on request.""" + + def __init__( + self, descriptor, limit, *, fail_evaluation=None, fail_record=None, signal=False + ): + super().__init__(descriptor, limit, _law) + self.signal = signal + self.raised = None + self.calls = [] + self.fail_evaluation = fail_evaluation + self.fail_record = fail_record + + def reserve(self, logical_calls): + self.calls.append(("reserve", logical_calls)) + super().reserve(logical_calls) + + def evaluate(self, fields, *, z, purpose, density=None): + self.calls.append(("evaluate", purpose, z)) + if ( + self.fail_evaluation is not None + and len(self.purposes) == self.fail_evaluation + ): + self.purposes.append(purpose) + if self.signal: + self.held() + raise self.failure() + return super().evaluate(fields, z=z, purpose=purpose, density=density) + + def failure(self) -> BaseException: + return BackendError("manufactured provider failure") + + def record_trial(self, *, accepted): + self.calls.append(("record_trial", accepted)) + commits = self.calls.count(("record_trial", True)) + if accepted and self.fail_record is not None and commits == self.fail_record: + raise TDSEArchiveError("manufactured archive failure") + + def release_reservation(self): + self.calls.append(("release",)) + super().release_reservation() + + def held(self) -> None: + """Take a real SIGTERM, then raise it as the worker's wait does between polls. + + The signal arrives inside the session's commit, which holds it; only the + session's ``interruptible`` scope lets ``raise_held`` raise it here. + """ + signal.raise_signal(signal.SIGTERM) + try: + _interrupts.raise_held() + except RunTerminated as error: + self.raised = error + raise + + +class ReleaseFails(Recording): + """Fail the first release, after releasing.""" + + def release_reservation(self): + super().release_reservation() + if self.calls.count(("release",)) == 1: + raise BackendError("manufactured release failure") + + +class Interrupted(Recording): + """Raise SIGTERM's exception from an evaluation, as the worker's wait raises a + signal it took (a held one, on native).""" + + def failure(self) -> BaseException: + return RunTerminated(signal.SIGTERM) + + +def _run(backend, problem, request, *, work_limit=256) -> Any: + """Drive a session of `backend` to its end; return the run and its record. + + A signal's exception that the session raises rather than reports, as a run loop + receives it, is returned in place of the run. + """ + session = start_session( + backend, problem, request, Start.entrance(problem), owns=False + ) + try: + run = drive( + Started(session), export=IDENTITY, work_limit=work_limit, raises=False + ) + record = dict(session.execution_record()) + except RunTerminated as signalled: + return signalled, {"z": session.status().z} + finally: + session.close() + return run, record + + +def _pair(case, binding_class=Recording, *, limit=1000, **options): + """Run the case on both backends with a fresh binding each.""" + problem, request, grid = case + descriptor = tdse_descriptor(problem.route) + runs = {} + for name, construct in ( + ("numpy_ref", NumpyRefBackend), + ("native", native.NativeBackend), + ): + bound = binding_class(descriptor, limit, **options) + backend = construct(problem, request, grid=grid, tdse_binding=bound) + try: + run, record = _run(backend, problem, request) + metadata = backend.metadata() + finally: + backend.close() + runs[name] = (run, record, bound, metadata) + return runs + + +def _unsourced(case, budget, field, *, work_limit=256): + """Assert that `field` is far from the run with no source: the comparison holds `F`. + + - ``budget``: the comparison's relative bound, which the source's effect must + exceed a millionfold. + """ + problem, request, grid = case + binding = ManufacturedBinding(tdse_descriptor(problem.route), 10**6, _silent) + backend = native.NativeBackend(problem, request, grid=grid, tdse_binding=binding) + try: + run, _ = _run(backend, problem, request, work_limit=work_limit) + finally: + backend.close() + effect = np.linalg.norm(field - run.fields[-1]) / np.linalg.norm(field) + assert effect > 1e6 * budget, f"the source moves the field by {effect:.3g} only" + + +def _reference_startup(case, monkeypatch): + """The reference's startup: the field it evaluates, its ``F`` and row zero.""" + problem, request, grid = case + seen: dict[str, Any] = {} + nonlinear = NumpyRefBackend._nonlinear_tdse + project = tdse_source.project_atomic_sources + start = HostHistory.start + + def recording_nonlinear(self, lab, bundle, inverse, purpose): + seen.setdefault("field", np.array(lab).reshape(-1)) + seen.setdefault("z", bundle.z1) + return nonlinear(self, lab, bundle, inverse, purpose) + + def recording_project(*args, **kwargs): + result = project(*args, **kwargs) + seen.setdefault("source", np.array(result[0]).reshape(-1)) + return result + + def recording_start(self, z): + row = start(self, z) + seen.setdefault("row", np.array(row).reshape(-1)) + seen.setdefault("history", self) + return row + + with monkeypatch.context() as patch: + patch.setattr(NumpyRefBackend, "_nonlinear_tdse", recording_nonlinear) + patch.setattr(tdse_source, "project_atomic_sources", recording_project) + patch.setattr(HostHistory, "start", recording_start) + binding = Unbudgeted(tdse_descriptor(problem.route), 10**6, _law) + backend = NumpyRefBackend(problem, request, grid=grid, tdse_binding=binding) + session = start_session( + backend, problem, request, Start.entrance(problem), owns=False + ) + try: + while session.status().attempts < 1: + batch = session.advance(work_limit=1) + if batch.receipt is not None: + session.acknowledge(batch.receipt) + seen["after"] = np.array(seen["history"]._store[0]).reshape(-1) + status = session.status() + seen["attempts"], seen["position"] = status.attempts, status.z + finally: + session.close() + backend.close() + return seen + + +def _native_startup(case): + """Native's startup through the C++ gather and the binding: the hook's field and + source, and the history's row zero after the first attempt.""" + problem, request, grid = case + binding = Unbudgeted(tdse_descriptor(problem.route), 10**6, _law) + backend = native.NativeBackend(problem, request, grid=grid, tdse_binding=binding) + seen: dict[str, Any] = {} + hook = backend._host_source + evaluate = hook.evaluate + + def recording(field, z, purpose, out): + evaluate(field, z, purpose, out) + if "field" not in seen: + seen.update( + field=np.array(field), + writable=field.flags.writeable, + source=np.array(out), + z=z, + purpose=purpose, + ) + + # The engine looks the entry point up on the object at each call. + hook.evaluate = recording + session = start_session( + backend, problem, request, Start.entrance(problem), owns=False + ) + try: + raw = cast("Any", session)._native + batch = raw.advance(1, math.inf) + while raw.status().attempts < 1: + raw.acknowledge(batch.sequence) + batch = raw.advance(1, math.inf) + seen["attempts"], seen["position"] = raw.status().attempts, raw.status().z + snapshot = raw.march_snapshot(batch.sequence, raw.status().generation) + assert snapshot is not None + seen["row"] = np.array(snapshot[2]["store"][0]).reshape(-1) + finally: + session.close() + backend.close() + return seen + + +@pytest.mark.parametrize("noise", [None, "input"]) +@pytest.mark.parametrize("kerr_model", ["tdse_native", "optmat_cubic"]) +@pytest.mark.parametrize("formulation", ["interaction", "exponential"]) +def test_the_startup_evaluation_is_the_reference_s( + formulation, kerr_model, noise, monkeypatch +): + """Item 1: the field the engine gathers and ``F`` bit for bit, through the C++ + gather and the binding; the history's first row after the first attempt, without + the in-stage absorber, bit for bit where it is the startup output itself and + within one complex product where the interaction formulation transported it; with + the absorber, within one transform's bound.""" + for absorber in (False, True): + changes: dict[str, Any] = {} if absorber else {"absorber": None} + case = _case(formulation, kerr_model, noise=noise, **changes) + expected = _reference_startup(case, monkeypatch) + ours = _native_startup(case) + assert ours["purpose"] == "startup" + assert ours["z"] == expected["z"] + assert not ours["writable"] + np.testing.assert_array_equal(ours["field"], expected["field"]) + np.testing.assert_array_equal(ours["source"], expected["source"]) + assert np.any(ours["source"] != 0) + if not absorber: + # The first attempt, the initial step, is accepted on both: row zero is + # the startup output, carried to the step's end in the interaction + # formulation, kept as it is in the exponential one. + assert ours["attempts"] == expected["attempts"] == 1 + assert ours["position"] == expected["position"] > 0 + if formulation == "exponential": + np.testing.assert_array_equal(expected["after"], expected["row"]) + np.testing.assert_array_equal(ours["row"], expected["after"]) + else: + # The transport is each engine's own complex product, which a + # compiler may fuse (MSVC on Arm64 does): within its rounding, a few + # eps of each value, not bits. + assert_within( + ours["row"], + expected["after"], + 4 * EPS * np.abs(expected["after"]), + elementwise=True, + what="transported startup row", + ) + else: + # The collar rejects the initial step: row zero is the startup output. + assert ours["position"] == case[0].initial.z + peak = np.max(np.abs(expected["row"])) + assert_within( + ours["row"], + expected["row"], + fft_bound(case[0].grids.n_to) * peak, + elementwise=True, + what="startup row with the in-stage absorber", + ) + + +@pytest.mark.parametrize("formulation", ["interaction", "exponential"]) +def test_the_call_sequence_ledgers_and_metadata_are_the_reference_s(formulation): + """Item 2 on a normal run, PEC: every call in the same order on both.""" + runs = _pair(_case(formulation)) + reference, ours = runs["numpy_ref"], runs["native"] + assert [call[:2] for call in ours[2].calls] == [ + call[:2] for call in reference[2].calls + ] + # The positions follow step proposals that are smooth functions of independently + # rounded defect norms (a rejection's defect cancels to a few digits), so they + # agree within an accuracy envelope, not bits; the sequence above is exact. + zs = np.array([call[2] for call in ours[2].calls if call[0] == "evaluate"]) + zr = np.array([call[2] for call in reference[2].calls if call[0] == "evaluate"]) + np.testing.assert_allclose(zs, zr, rtol=1e-9, atol=0) + assert ours[2].ledger == reference[2].ledger + for key in ("atomic_calls", "tdse_source"): + assert ours[3][key] == reference[3][key] + assert ours[3]["tdse_last_evaluation"]["purpose"] == "predictor" + assert ours[0].terminal.error is None and reference[0].terminal.error is None + # The fields the sequences came from, within the per-attempt budget (below). + budget = ours[0].status.attempts * 16 * EPS + assert_within(ours[0].fields[-1], reference[0].fields[-1], budget, what="run") + _unsourced(_case(formulation), budget, ours[0].fields[-1]) + + +def test_a_budget_refusal_stops_both_at_the_same_step_with_the_same_charges(): + """Item 2 and 3: a reservation the atomic budget refuses, before the attempt.""" + runs = _pair(_case(), limit=7) + reference, ours = runs["numpy_ref"], runs["native"] + for run, _, binding, _ in (reference, ours): + error = run.terminal.error + assert isinstance(error, ResponseBudgetError) + assert "atomic-call budget cannot reserve" in str(error) + assert binding.ledger.reserved == 0 + assert ours[2].calls == reference[2].calls + assert ours[0].terminal.error.z == reference[0].terminal.error.z + assert ours[0].terminal.error.calls == reference[0].terminal.error.calls + + +@pytest.mark.parametrize("binding_class", [Recording, Interrupted]) +def test_a_failed_evaluation_keeps_the_last_accepted_state(binding_class): + """Item 3: a provider failure, or a signal at the boundary, before commit.""" + runs = _pair(_case(), binding_class, fail_evaluation=12) + reference, ours = runs["numpy_ref"], runs["native"] + positions = [] + for run, _, binding, _ in (reference, ours): + if binding_class is Recording: + assert isinstance(run.terminal.error, BackendError) + positions.append(run.terminal.z) + elif isinstance(run, RunTerminated): + # The reference's session lets the signal through to the run loop... + pass + else: + # ...native's reports it as its terminal error, the batch booked and the + # state eligible for the checkpoint; the run loop raises it the same. + assert isinstance(run.terminal.error, RunTerminated) + assert run.terminal.eligibility.value == "eligible" + positions.append(run.terminal.z) + assert binding.ledger.reserved == 0 + # The failed evaluation's trial was not recorded; its reservation released. + assert binding.calls[-1] == ("release",) + # The last accepted step's end, to the steps' independent rounding (see above). + assert all(position > 0 for position in positions) + if len(positions) == 2: + assert positions[1] == pytest.approx(positions[0], rel=1e-9, abs=0) + assert [call[:2] for call in ours[2].calls] == [ + call[:2] for call in reference[2].calls + ] + assert ours[2].ledger == reference[2].ledger + + +def test_an_abandoned_attempt_s_failed_release_is_a_note_on_the_run_s_error(): + """Section 6.3: the step-size floor abandons the first attempt, whose release + fails; that failure rides on the floor's error as a note, on both.""" + case = _case(length=1e-2, dz_max=1e-4, dz_init=1e-4, dz_min=9e-5, absorber=None) + runs = _pair(case, ReleaseFails) + errors = [runs[name][0].terminal.error for name in ("numpy_ref", "native")] + assert isinstance(errors[0], ConvergenceError) + assert type(errors[1]) is type(errors[0]) + assert str(errors[1]) == str(errors[0]) + assert ( + errors[1].__notes__ + == errors[0].__notes__ + == ["releasing the source reservation failed: manufactured release failure"] + ) + assert runs["native"][2].calls == runs["numpy_ref"][2].calls + + +def _held_signal_run(case) -> tuple[float, weakref.ref]: + """Run `case` on native with a real SIGTERM held in an evaluation; check the stop. + + Returns the run's last position and a weak reference to its binding, which the + signal's exception reaches through its traceback (the binding keeps it). + """ + problem, request, grid = case + binding = Recording( + tdse_descriptor(problem.route), 1000, fail_evaluation=12, signal=True + ) + backend = native.NativeBackend(problem, request, grid=grid, tdse_binding=binding) + try: + with _interrupts.handled(): + if signal.getsignal(signal.SIGTERM) is not _interrupts._handle: + pytest.skip("SIGTERM has a handler of its own in this process") + run, _ = _run(backend, problem, request) + finally: + backend.close() + # Raised by the evaluation's own `raise_held`, not after the commit ended. + assert isinstance(binding.raised, RunTerminated) + assert run.terminal.error is binding.raised + assert run.terminal.eligibility.value == "eligible" + assert binding.ledger.reserved == 0 + assert binding.calls[-1] == ("release",) + return run.terminal.z, weakref.ref(binding) + + +def test_a_held_signal_stops_a_native_evaluation_before_its_commit(): + """Item 3 and section 6.7: a real SIGTERM inside the session's commit is held, and + the session's ``interruptible`` scope lets the evaluation's wait raise it. + + The closed backend leaves nothing the collector cannot see: the binding, in a + cycle through the kept exception's traceback, is freed. + """ + case = _case() + z, binding = _held_signal_run(case) + gc.collect() + assert binding() is None + # The last accepted step, as a provider failure there leaves it. + failed = _pair(case, fail_evaluation=12)["native"][0] + assert z == failed.terminal.z > 0 + + +def test_an_archive_failure_after_an_accepted_commit_keeps_the_new_state(): + """Item 3: the commit is the boundary; the archive's error is the run's, and the + terminal state is the committed step's, its next proposal included.""" + problem, request, grid = _case() + descriptor = tdse_descriptor(problem.route) + ends = {} + for name, construct in ( + ("numpy_ref", NumpyRefBackend), + ("native", native.NativeBackend), + ): + binding = Recording(descriptor, 1000, fail_record=2) + backend = construct(problem, request, grid=grid, tdse_binding=binding) + snapshots = [] + + def keep(session, batch, snapshots=snapshots): + if batch.terminal is not None: + snapshots.append(session.march_snapshot(batch.terminal.ticket)) + + session = start_session( + backend, problem, request, Start.entrance(problem), owns=False + ) + try: + run = drive(Started(session), export=IDENTITY, raises=False, between=keep) + finally: + session.close() + backend.close() + assert isinstance(run.terminal.error, TDSEArchiveError) + assert run.terminal.eligibility.value == "eligible" + assert binding.ledger.reserved == 0 + assert binding.calls[-2:] == [("record_trial", True), ("release",)] + ends[name] = (run.terminal.z, snapshots[0], binding.calls) + assert run.terminal.history is not None + step = float(run.terminal.history.dz[-1]) + (z, snapshot, calls), (reference_z, expected, reference_calls) = ( + ends["native"], + ends["numpy_ref"], + ) + # The second accepted step stands: the run stops at its end, on both. + assert z == pytest.approx(reference_z, rel=1e-12, abs=0) and z > 0 + assert snapshot.z == z + # The committed step's own proposal, not the one it was taken with. + assert snapshot.proposal == pytest.approx(expected.proposal, rel=1e-9, abs=0) + assert expected.proposal != pytest.approx(step, rel=1e-6, abs=0) + assert [call[:2] for call in calls] == [call[:2] for call in reference_calls] + + +def _agreement(case, work_limit): + """Run `case` on both backends; return the native and reference final fields and + the attempts each took, which must be the same.""" + problem, request, grid = case + descriptor = tdse_descriptor(problem.route) + fields, ledgers = {}, {} + for name, construct in ( + ("numpy_ref", NumpyRefBackend), + ("native", native.NativeBackend), + ): + binding = ManufacturedBinding(descriptor, 10_000, _law) + backend = construct(problem, request, grid=grid, tdse_binding=binding) + try: + run, _ = _run(backend, problem, request, work_limit=work_limit) + finally: + backend.close() + assert run.terminal.error is None + fields[name] = run.fields[-1] + ledgers[name] = (binding.ledger, run.status.attempts, run.status.rejected) + # The same attempts and rejections, so no decision forked on this case. + assert ledgers["native"] == ledgers["numpy_ref"] + return fields["native"], fields["numpy_ref"], ledgers["native"][1] + + +@pytest.mark.parametrize("work_limit", [1, 256]) +def test_eight_steps_agree_within_the_eight_step_budget(work_limit): + """Item 4: eight steps, one attempt per call and the production bound. + + Without the in-stage absorber's collar the steps reach their cap. Cap and length + are binary-exact (2^-27 m, 2^-24 m), so eight steps of the cap land on the end + with no split final step: each one stage of the one-step budget, 16 eps, + accumulated linearly to 128 eps. + """ + case = _case(length=2.0**-24, dz_max=2.0**-27, absorber=None) + ours, reference, attempts = _agreement(case, work_limit) + # Eight accepted steps; the startup sample is evaluated inside the first. + assert attempts == 8 + assert_within(ours, reference, 128 * EPS, what="eight steps") + _unsourced(case, 128 * EPS, ours, work_limit=work_limit) + + +def test_a_collar_held_run_agrees_within_its_per_attempt_budget(): + """Item 4 with the in-stage absorber, whose collar holds the steps to about a + two-hundredth of the run: some four hundred attempts, each one stage of 16 eps, + accumulated linearly, the absorber damping rather than amplifying.""" + case = _case(length=2e-7, dz_max=2e-7) + ours, reference, attempts = _agreement(case, 256) + assert attempts > 200 + budget = attempts * 16 * EPS + assert_within(ours, reference, budget, what="collar-held run") + _unsourced(case, budget, ours) + + +def test_a_device_build_refuses_the_tdse_response_by_name(monkeypatch): + """Item 6, by the refusal's rule: no device build runs a host source.""" + problem, request, grid = _case() + monkeypatch.setattr(native.backend, "_on_device", lambda: True) + binding = ManufacturedBinding(tdse_descriptor(problem.route), 10, _law) + with pytest.raises(BackendError, match="runs on the native CPU build"): + native.NativeBackend(problem, request, grid=grid, tdse_binding=binding) + + +def test_a_tdse_problem_needs_its_binding_and_a_binding_its_problem(): + problem, request, grid = _case() + with pytest.raises(BackendError, match="explicit host response binding"): + native.NativeBackend(problem, request, grid=grid) + + +def test_the_memory_ledger_lists_the_host_rows_and_the_binding_s(): + problem, request, grid = _case() + binding = ManufacturedBinding(tdse_descriptor(problem.route), 10, _law) + binding.memory_ledger = lambda: {"transport.solprop": 64} + backend = native.NativeBackend(problem, request, grid=grid, tdse_binding=binding) + try: + ledger = backend.memory_ledger() + finally: + backend.close() + n = problem.grids.n_omega + # The field and source rows, each one complex128 spectrum. + assert ledger["host_source"] == 2 * 16 * n + assert ledger["tdse_transport.solprop"] == 64 + + +def test_closing_the_backend_under_an_open_session_drops_the_host_source(): + """Closing while a session leases the state closes it, host source and all.""" + problem, request, grid = _case() + binding = ManufacturedBinding(tdse_descriptor(problem.route), 10, _law) + backend = native.NativeBackend(problem, request, grid=grid, tdse_binding=binding) + session = start_session( + backend, problem, request, Start.entrance(problem), owns=False + ) + try: + backend.close() + assert backend.closed + backend.close() + finally: + session.close() diff --git a/tests/backends/test_tdse_projection.py b/tests/backends/test_tdse_projection.py index d649bd33a..c933cf75d 100644 --- a/tests/backends/test_tdse_projection.py +++ b/tests/backends/test_tdse_projection.py @@ -12,7 +12,7 @@ from numpy.testing import assert_allclose, assert_array_equal from solprop.backends.numpy_ref import passes -from solprop.backends.numpy_ref.tdse import ( +from solprop.backends.tdse_source import ( atomic_edge_source, atomic_sources, project_atomic_sources, diff --git a/tests/backends/test_tdse_source.py b/tests/backends/test_tdse_source.py new file mode 100644 index 000000000..7a8657122 --- /dev/null +++ b/tests/backends/test_tdse_source.py @@ -0,0 +1,308 @@ +# SPDX-FileCopyrightText: 2026 John C. Travers and contributors +# +# SPDX-License-Identifier: GPL-3.0-or-later + +"""State the shared TDSE source's terms independently of the module (SEP 49, 7.1). + +Both backends call :mod:`solprop.backends.tdse_source`, so their agreement cannot see an +error inside it. Each test here writes its expectation from the kernel contract or the +documented bound with NumPy alone: the nodal synthesis and its taper, the projection's +division by the taper, the edge array's scaling, the zero source off the active band +and the diagnostics' rounding floors. The signs of ``S = J - j_1`` and of the cubic +replacement, the density and the gate are stated in ``test_tdse_sources.py``, and the +edge term and crop in ``test_tdse_edge_free.py`` and ``test_tdse_projection.py``. +""" + +import dataclasses +import math +import subprocess +import sys +from typing import Any + +import numpy as np +import pytest + +from edge_source_cases import EPS, EdgeCase, make_case +from solprop.backends.tdse_source import ( + FFT_ERROR_CONSTANT, + atomic_edge_source, + atomic_sources, + node_fields, + project_atomic_sources, +) +from solprop.exceptions import BackendError +from solprop.nonlinear.responses import KerrResponse +from solprop.translate.linop import power_weight_at +from solprop.translate.routes import tdse_of +from solprop.translate.tdse import TDSEResult, lattice_times, waveform_identity + + +def _result(case: EdgeCase, fields: np.ndarray, *, cubic: bool) -> TDSEResult: + """Return analytic histories whose remainder and cubic shadow are both nonzero.""" + times = lattice_times(case.grid.n_to, case.grid.dt_fine) + envelope = np.exp(-((times / 8e-15) ** 2)) + remainder = 1e-27 * envelope * np.sin(2.4e15 * times) + first = 1e-30 * fields + third = 1e-40 * fields**3 if cubic else None + total = first + remainder + (0.0 if third is None else third) + return TDSEResult( + total, + first, + input_identity=waveform_identity(fields), + configuration_identity=case.descriptor.identity, + valid=np.ones(fields.shape, dtype=bool), + shadow_orders=(1, 3) if cubic else (1,), + third_order_current=third, + ) + + +def _fields(case: EdgeCase) -> np.ndarray: + """Return a 2 fs-scale pulse of 1e9 V/m on every node.""" + times = lattice_times(case.grid.n_to, case.grid.dt_fine) + pulse = 1e9 * np.exp(-((times / 6e-15) ** 2)) * np.cos(2.35e15 * times) + rows = case.problem.transform.n_nodes + return np.ascontiguousarray(np.broadcast_to(pulse, (rows, case.grid.n_to))) + + +def _evaluate(case: EdgeCase, bundle=None) -> Any: + """Return the projected source and its diagnostics, with the inputs used.""" + bundle = case.bundle if bundle is None else bundle + fields = _fields(case) + result = _result(case, fields, cubic=not case.descriptor.gated) + polarisation, current = atomic_sources(fields, result, case.problem, bundle) + edge = atomic_edge_source(result, case.problem, bundle) + source, diagnostics = project_atomic_sources( + polarisation, current, result, case.problem, bundle, case.prefactor, edge=edge + ) + return source, diagnostics, result, (polarisation, current, edge) + + +def _cubic_case() -> EdgeCase: + """Return a cubic replacement whose compiled Kerr term gives a nonzero P.""" + return make_case(kerr_model="optmat_cubic", responses=(KerrResponse(chi3=1e-25),)) + + +@pytest.fixture(scope="module", params=["tdse_native", "optmat_cubic"]) +def case(request): + return make_case() if request.param == "tdse_native" else _cubic_case() + + +def test_node_fields_are_the_taper_times_the_synthesis_of_each_bin(): + """One coarse bin ``a`` at fine bin ``c`` is ``(2/N) Re(a exp(2 pi i c n / N))``.""" + case = make_case() + grids, transform = case.problem.grids, case.problem.transform + bundle = dataclasses.replace(case.bundle, taper_scale=1.7) + bin_index, amplitude = 17, 3.0 - 2.0j + lab = np.zeros((1, 1, grids.n_omega), dtype=np.complex128) + lab[0, 0, bin_index] = amplitude + samples = grids.n_to + phase = 2 * np.pi * grids.crop_index[bin_index] * np.arange(samples) / samples + modal = 2 / samples * np.real(amplitude * np.exp(1j * phase)) + expected = 1.7 * transform.synthesis[:, :1] * modal + fields = node_fields(case.problem, lab, bundle) + assert fields.shape == (transform.n_nodes, samples) + assert fields.flags.c_contiguous + # Normwise inverse-FFT bound (Higham, Theorem 24.2) plus the two products and the + # oracle's own cosine: (c_F log2 N + 4) eps of the norm. + budget = (FFT_ERROR_CONSTANT * math.ceil(math.log2(samples)) + 4) * EPS + error = np.linalg.norm(fields - expected) + assert budget > 0 + assert error <= budget * np.linalg.norm(expected) + + +def test_node_fields_refuse_two_polarisations(): + case = make_case() + lab = np.zeros((1, 2, case.problem.grids.n_omega), dtype=np.complex128) + with pytest.raises(BackendError, match="one polarisation"): + node_fields(case.problem, lab, case.bundle) + + +def test_the_projection_divides_by_the_taper_scale(case): + """The projection is ``W / tau`` (kernel contract section 5). + + Halving through ``tau = 2`` is exact in binary64 -- a power-of-two scaling commutes + with every rounding of the products, sums and transform -- so the identity holds + bit for bit. + """ + source, *_ = _evaluate(case) + doubled = dataclasses.replace(case.bundle, taper_scale=2.0) + halved, *_ = _evaluate(case, doubled) + assert np.any(source != 0) + np.testing.assert_array_equal(2 * halved, source) + + +def test_the_edge_array_is_density_scale_and_gate_derivative_of_the_remainder(): + """``E = rho s g'(t) (J - j_1)``, the density and scale entering once.""" + case = make_case() + bundle = dataclasses.replace(case.bundle, density=3.0) + fields = _fields(case) + result = _result(case, fields, cubic=False) + edge = atomic_edge_source(result, case.problem, bundle) + assert edge is not None + descriptor = case.descriptor + remainder = result.total_current - result.first_order_current + expected = ( + descriptor.source_gate_derivative() + * remainder + * (descriptor.remainder_scale * 3.0) + ) + # Three products, associated differently in the oracle: gamma_3 of each value. + np.testing.assert_allclose(edge[:, 0, :], expected, rtol=4 * EPS, atol=0) + assert np.any(expected != 0) + + +def test_the_source_is_exactly_zero_off_the_active_band(case): + """Both spectral factors vanish off the active band by construction, so must F.""" + source, *_ = _evaluate(case) + active = case.problem.grids.active_mask + assert np.all(np.isfinite(source)) + assert np.all(source[..., ~active] == 0) + assert np.any(source[..., active] != 0) + + +def _absolute_projection(values: np.ndarray, projection: np.ndarray, taper: float): + """Return ``sum_n |W_mn| values_n / |tau|`` per mode and sample.""" + return np.einsum("mn,nt->mt", np.abs(projection), values[:, 0, :]) / abs(taper) + + +def _floor_terms(case: EdgeCase, magnitudes: np.ndarray, factor, projection): + """Return the documented operand bound ``|factor w| sum_t |W| x / |tau|``.""" + grids = case.problem.grids + spectral = _absolute_projection(magnitudes, projection, case.bundle.taper_scale) + return np.abs(factor * grids.omega_window) * spectral.sum(axis=-1)[:, None, None] + + +def _accumulation(case: EdgeCase) -> float: + """``(8 + 2 n_nodes + 5 ceil(log2 N) [+1 for a gate]) eps`` of the floor.""" + operations = 8 + 2 * case.problem.transform.n_nodes + operations += 5 * math.ceil(math.log2(case.problem.grids.n_to)) + return (operations + (1 if case.descriptor.gated else 0)) * EPS + + +def test_the_complete_floor_is_the_documented_operand_bound(): + """The ungated (cubic) source's floors, from the absolute raw operands. + + Each floor is ``|| |c w| sum_t |W| |rho s| (|J| + |j_1| + |j_3|) / |tau| ||`` + times ``a / (1 - a)``, for the current and likewise ``|p w|`` and ``|P|`` for the + polarisation, over the fine and the embedded maps. + """ + case = _cubic_case() + _, diagnostics, result, (polarisation, _, _) = _evaluate(case) + transform, bundle = case.problem.transform, case.bundle + source = tdse_of(case.problem.route) + assert source is not None + factor = source.current_numerator / power_weight_at(case.problem, bundle.z1) + operands = ( + np.abs(result.total_current) + + np.abs(result.first_order_current) + + np.abs(result.third_order_current) + ) + scale = abs(bundle.density * case.descriptor.remainder_scale) + current = scale * operands[:, None, :] + accumulation = _accumulation(case) + ratio = accumulation / (1 - accumulation) + floors = {"current": 0.0, "polarisation": 0.0} + maps = [transform.projection] + if not np.array_equal(transform.projection, transform.projection_coarse): + # The embedded map adds its own floors only when it is a distinct rule. + maps.append(transform.projection_coarse) + for projection in maps: + floors["current"] += ratio * float( + np.linalg.norm(_floor_terms(case, current, factor, projection)) + ) + floors["polarisation"] += ratio * float( + np.linalg.norm( + _floor_terms(case, np.abs(polarisation), case.prefactor, projection) + ) + ) + # The bound is itself evaluated in floating point, with sums of a few thousand + # terms reassociated here: 1e-12 relative covers that and nothing else. + assert diagnostics["remainder_source"]["roundoff_floor"] == pytest.approx( + floors["current"], rel=1e-12, abs=0 + ) + assert diagnostics["complete_source"]["roundoff_floor"] == pytest.approx( + floors["current"] + floors["polarisation"], rel=1e-12, abs=0 + ) + assert floors["polarisation"] > 0 + + +def test_the_edge_floors_are_the_documented_section_4_5_bounds(): + """A gated source's band floors (SEP 26 section 4.5), written from the docstring. + + Per bin, with ``c_k = |factor_k w_k|``: the truncating part ``c_k sum |W| |rho s| + |g| (|J| + |j_1|) a / (1 - a)``; the edge transform ``c_k sqrt(N) (c_F eps log2 N + + (n_nodes + 6) u) ||y||_2 / omega_lo`` with ``y`` the absolute projection of + ``|rho s| |g'| (|J| + |j_1|)``, plus ``c_k sqrt(N) ||y_d||_2 / omega_lo`` for the + derivative's own error; and the division ``2 eps c_k (|X_k| + |E_k| / omega_k)``. + """ + case = make_case() + _, diagnostics, result, (_, current, edge) = _evaluate(case) + assert edge is not None + problem, bundle, descriptor = case.problem, case.bundle, case.descriptor + grids, projection = problem.grids, problem.transform.projection + source = tdse_of(problem.route) + assert source is not None + factor = source.current_numerator / power_weight_at(problem, bundle.z1) + collar = np.abs(factor * grids.omega_window) + raw = np.abs(result.total_current) + np.abs(result.first_order_current) + scale = abs(bundle.density * descriptor.remainder_scale) + gate = grids.t_window_fine if descriptor.source_gate() is None else None + assert gate is not None + accumulation = _accumulation(case) + truncating = _floor_terms( + case, scale * (raw * gate)[:, None, :], factor, projection + ) + truncating = truncating * accumulation / (1 - accumulation) + samples, nodes = grids.n_to, problem.transform.n_nodes + lowest = float(np.min(grids.omega[grids.active_mask])) + transform = FFT_ERROR_CONSTANT * EPS * math.ceil(math.log2(samples)) + ( + nodes + 6 + ) * (EPS / 2) + y = _absolute_projection( + scale * (raw * np.abs(descriptor.source_gate_derivative()))[:, None, :], + projection, + bundle.taper_scale, + ) + y_d = _absolute_projection( + scale * (raw * descriptor.source_gate_derivative_error())[:, None, :], + projection, + bundle.taper_scale, + ) + norms = transform * np.linalg.norm(y, axis=-1) + np.linalg.norm(y_d, axis=-1) + edge_bins = collar * math.sqrt(samples) * norms[:, None, None] / lowest + + def spectrum(values): + projected = np.einsum("mn,nt->mt", projection, values[:, 0, :]) + return np.fft.rfft(projected / bundle.taper_scale)[:, None, grids.crop_index] + + inverse = np.zeros(grids.n_omega) + inverse[grids.active_mask] = 1 / grids.omega[grids.active_mask] + division = ( + 2 + * EPS + * collar + * (np.abs(spectrum(current)) + np.abs(spectrum(edge)) * inverse) + ) + band = diagnostics["edge"]["bands"]["active"] + active = grids.active_mask + # As above: the bounds are evaluated in floating point and reassociated here. + assert band["truncating_floor"] == pytest.approx( + np.linalg.norm(truncating[..., active]), rel=1e-12, abs=0 + ) + assert band["edge_floor"] == pytest.approx( + np.linalg.norm(edge_bins[..., active]) + np.linalg.norm(division[..., active]), + rel=1e-12, + abs=0, + ) + assert np.linalg.norm(division[..., active]) > 0 + + +def test_the_shared_source_imports_neither_backend(): + """Native calls this module and never imports the reference (SEP 49, 6.1).""" + probe = ( + "import sys, solprop.backends.tdse_source; " + "loaded = [name for name in sys.modules if 'numpy_ref' in name " + "or name.startswith('solprop_native')]; " + "assert not loaded, loaded" + ) + subprocess.run([sys.executable, "-c", probe], check=True, timeout=120) diff --git a/tests/backends/test_tdse_sources.py b/tests/backends/test_tdse_sources.py index 24ad6206a..9dbcdb85d 100644 --- a/tests/backends/test_tdse_sources.py +++ b/tests/backends/test_tdse_sources.py @@ -16,7 +16,7 @@ from numpy.testing import assert_allclose, assert_array_equal from tests.hacc_cases import hacc_response -from solprop.backends.numpy_ref.tdse import atomic_sources +from solprop.backends.tdse_source import atomic_sources from solprop.constants import EPS0 from solprop.exceptions import BackendError from solprop.grid import FieldGrid diff --git a/tests/driver/test_initial_z_and_checkpoint.py b/tests/driver/test_initial_z_and_checkpoint.py index 7d3848c01..e971e65fa 100644 --- a/tests/driver/test_initial_z_and_checkpoint.py +++ b/tests/driver/test_initial_z_and_checkpoint.py @@ -718,8 +718,8 @@ def test_tdse_does_not_guess_how_to_resample_custom_absorbers(atomic_simulation) def test_tdse_refuses_implicit_execution_and_dense_saves(atomic_simulation): simulation = atomic_simulation - with pytest.raises(DriverError, match="explicit backend='numpy_ref'"): - dataclasses.replace(simulation.settings, backend="auto") + # backend="auto" runs a TDSE response on native (SEP 49): no refusal. + assert dataclasses.replace(simulation.settings, backend="auto").backend == "native" with pytest.raises(DriverError, match="tdse_execution"): simulation.settings = dataclasses.replace( simulation.settings, tdse_execution=None diff --git a/tests/driver/test_interface_defaults.py b/tests/driver/test_interface_defaults.py index e3e197ee9..365379fef 100644 --- a/tests/driver/test_interface_defaults.py +++ b/tests/driver/test_interface_defaults.py @@ -382,14 +382,17 @@ def test_a_copy_takes_the_route_s_default(route): def test_a_tdse_run_s_refusals_name_the_backend_and_the_integrator(): with pytest.raises(DriverError, match=r"integrator=Multistep\(\).*RungeKutta"): _capillary(tdse=_sae(), integrator=RungeKutta()) - with pytest.raises(DriverError, match="explicit backend='numpy_ref'"): + # Both backends run the TDSE response (SEP 49): native is no refusal. + assert ( SimulationSettings( save_planes=[1.0], dz_max=1.0, dz_init=1e-3, backend="native", tdse_execution=TDSEExecution(), - ) + ).backend + == "native" + ) # Not a record at all: refused where the annotations are not checked at run time. settings = SimulationSettings( save_planes=[1.0], dz_max=1.0, dz_init=1e-3, backend="numpy_ref" diff --git a/tests/driver/test_interrupts.py b/tests/driver/test_interrupts.py index cd8ff7d80..bc8dd2ce6 100644 --- a/tests/driver/test_interrupts.py +++ b/tests/driver/test_interrupts.py @@ -21,7 +21,7 @@ from driver.test_resume import N_SAVE, CrashError from driver.test_resume_history import RTOL, assert_accurate_run, soliton, tight_run from march_bounds import TIGHTER -from solprop._interrupts import committing, handled +from solprop._interrupts import committing, handled, interruptible, raise_held from solprop.exceptions import RunTerminated from solprop.log import NullReporter from solprop.output.hdf5 import open_file, resume @@ -145,6 +145,56 @@ def test_a_signal_inside_a_commit_waits_for_its_end(): assert signal.getsignal(signal.SIGTERM) is signal.SIG_DFL +@needs_default_sigterm +def test_an_interruptible_wait_inside_a_commit_takes_the_held_signal_at_once(): + """A host source's wait inside native's commit raises the signal (SEP 49, 6.7).""" + events = [] + with handled(): + with pytest.raises(RunTerminated), committing(), interruptible(): + signal.raise_signal(signal.SIGTERM) + events.append("held") + raise_held() + events.append("not reached") + events.append("nothing left to deliver") + assert events == ["held", "nothing left to deliver"] + + +@needs_default_sigterm +def test_outside_an_interruptible_scope_a_held_signal_stays_held(): + events = [] + with handled(): + # Any other commit keeps the signal for its end... + with pytest.raises(RunTerminated), committing(): + signal.raise_signal(signal.SIGTERM) + raise_held() + events.append("still held") + # ...and one that drops signals drops it. + with committing(deliver=False): + signal.raise_signal(signal.SIGTERM) + raise_held() + events.append("dropped at its end") + assert events == ["still held", "dropped at its end"] + + +@needs_default_sigterm +def test_another_thread_cannot_take_the_main_thread_s_held_signal(): + taken = [] + + def wait(): + with interruptible(): + try: + raise_held() + except RunTerminated: + taken.append("taken") + + with handled(), pytest.raises(RunTerminated), committing(), interruptible(): + signal.raise_signal(signal.SIGTERM) + other = threading.Thread(target=wait) + other.start() + other.join() + assert taken == [] + + def test_an_application_s_own_handler_is_kept(): def own(signal_number, frame): pass diff --git a/tests/driver/test_snapshots.py b/tests/driver/test_snapshots.py index 58f3f469b..99b7d511c 100644 --- a/tests/driver/test_snapshots.py +++ b/tests/driver/test_snapshots.py @@ -124,8 +124,11 @@ def watched(*args): id(backend), id(backend._adams), id(backend._tdse_binding), - id(binding._hacc.model), - tuple(id(plan) for plan in binding._plans.values()), + # The model and its plans live behind the provider's worker (SEP 49): + # the same worker, the same view of the model and the same plans. + id(binding._worker), + id(binding._hacc), + tuple(sorted(binding._plan_records)), ) ) before = (len(fake.calls), len(fake.prepared), len(fake.plan_arguments)) @@ -153,6 +156,7 @@ def watched(*args): assert controller.last_result.bytes_copied > 0 assert len(built) == 3 and all(item == built[0] for item in built) assert len(live) == 1 and all(item == identities[0] for item in identities) + assert identities[0][-1] # the plans were built before the snapshots assert len(fake.prepared) == 1 assert output.outcome is not None assert_no_extra_call(output.outcome) diff --git a/tests/driver/test_source_worker.py b/tests/driver/test_source_worker.py new file mode 100644 index 000000000..d268a3869 --- /dev/null +++ b/tests/driver/test_source_worker.py @@ -0,0 +1,879 @@ +# SPDX-FileCopyrightText: 2026 John C. Travers and contributors +# +# SPDX-License-Identifier: GPL-3.0-or-later + +"""The provider worker's contract, in real worker processes (SEP 49 section 7.2). + +Manufactured adapters (``tests/source_adapters.py``) stand in for the provider: a +provider-agnostic one whose requests misbehave on purpose, and Pyroraptor's own adapter +around the manufactured haCC provider of ``hacc_cases``. Deadlines here are the test +harness's, to detect a broken cancellation; the production code has none. +""" + +import dataclasses +import gc +import importlib.util +import io +import json +import os +import queue +import runpy +import signal +import subprocess +import sys +import threading +import time +import warnings +from pathlib import Path +from typing import Any, cast + +import numpy as np +import psutil +import pytest + +from solprop._interrupts import committing, handled, interruptible +from solprop.driver import _source_worker +from solprop.driver._source_worker import ( + CONTROL_LIMIT, + POLL_INTERVAL, + Opaque, + Record, + RemoteError, + Worker, + WorkerError, + decode, + encode, + worker_environment, +) +from solprop.exceptions import RunTerminated, StopRequested + +# How long a cancellation may take before the test calls it broken: the poll interval +# and a kill, with room for a loaded machine. +DEADLINE = 10.0 +POSIX = pytest.mark.skipif(sys.platform == "win32", reason="POSIX signals and groups") + + +def _worker(adapter="source_adapters", **options): + worker = Worker(adapter, environment=worker_environment(os.environ, {}), **options) + worker.start() + return worker + + +@pytest.fixture +def worker(): + started = _worker() + yield started + started.kill() + + +def _assert_ended(worker): + """The process is reaped and both pipe ends closed.""" + process = worker._process + assert process is not None and process.returncode is not None + assert process.stdin is not None and process.stdin.closed + assert process.stdout is not None and process.stdout.closed + assert not worker.alive + + +@dataclasses.dataclass(frozen=True) +class _Pair: + left: object + right: object + + +@pytest.mark.medium +def test_the_codec_round_trips_values_and_records_without_pickle(): + value = { + "arrays": [ + np.arange(6, dtype=">f8").reshape(2, 3), + np.array([1 + 2j], dtype=np.complex128), + np.array([True, False]), + np.zeros((0, 4), dtype=np.int64), + ], + "tuple": (1, 2.5, None, "text"), + "nonfinite": [float("nan"), float("inf"), -float("inf")], + "record": _Pair((1,), np.ones(2)), + "opaque": object(), + } + tree, arrays = encode(value) + json.dumps(tree, allow_nan=False) + back: Any = decode(json.loads(json.dumps(tree)), arrays) + for sent, received in zip(value["arrays"], back["arrays"], strict=True): + assert received.dtype == sent.dtype and received.shape == sent.shape + np.testing.assert_array_equal(received, sent) + assert back["tuple"] == value["tuple"] + assert np.isnan(back["nonfinite"][0]) and back["nonfinite"][1:] == [ + float("inf"), + -float("inf"), + ] + record = back["record"] + assert isinstance(record, Record) and record.record_type.endswith("_Pair") + assert record.left == (1,) + assert back["opaque"].type_name == "builtins.object" + + +@pytest.mark.medium +def test_the_codec_carries_numpy_scalars_and_sends_callables_as_opaque(): + tree, arrays = encode({"step": np.int32(3), "z": np.float64(0.5), "f": len}) + back: Any = decode(json.loads(json.dumps(tree)), arrays) + assert back["step"] == 3 and type(back["step"]) is int + assert back["z"] == 0.5 and type(back["z"]) is float + assert back["f"] == Opaque("builtins.builtin_function_or_method", repr(len)) + + +@pytest.mark.medium +def test_a_received_record_re_encodes_with_its_type_and_computed_values(): + """A record sent on keeps its type; the sender's computed values win.""" + tree, arrays = encode(_Pair(1, 2), lambda value: {"ok": True, "kept": 1}) + received: Any = decode(tree, arrays) + again: Any = decode(*encode(received, lambda value: {"ok": False})) + assert again.record_type == received.record_type + assert vars(again) == {"left": 1, "right": 2} + assert again.computed == {"ok": False, "kept": 1} + + +@pytest.mark.medium +def test_a_record_s_computed_values_read_as_attributes_but_are_not_fields(): + tree, arrays = encode(_Pair(1, 2), lambda value: {"ok": True}) + record: Any = decode(tree, arrays) + assert record.ok is True + assert vars(record) == {"left": 1, "right": 2} + with pytest.raises(AttributeError, match="no field 'missing'"): + record.missing # noqa: B018 - the attribute access is the test + + +@pytest.mark.medium +def test_the_worker_environment_drops_native_s_own_wait_settings_only(): + native = { + "SOLPROP_NATIVE_WAIT": "spin", + "GOMP_SPINCOUNT": "10000", + "KMP_BLOCKTIME": "0", + } + environment = worker_environment(native, {"PYRORAPTOR_THREADS": "3"}) + assert "SOLPROP_NATIVE_WAIT" not in environment + assert "GOMP_SPINCOUNT" not in environment and "KMP_BLOCKTIME" not in environment + assert environment["PYRORAPTOR_THREADS"] == "3" + user = {"OMP_WAIT_POLICY": "active", "KMP_BLOCKTIME": "200"} + assert worker_environment(user, {}).items() >= user.items() + edited = { + "SOLPROP_NATIVE_WAIT": "spin", + "GOMP_SPINCOUNT": "5", + "KMP_BLOCKTIME": "0", + } + kept = worker_environment(edited, {}) + assert kept["GOMP_SPINCOUNT"] == "5" and "KMP_BLOCKTIME" not in kept + + +@pytest.mark.medium +def test_the_native_wait_settings_are_solprop_native_s_own(monkeypatch): + """The worker removes exactly what solprop-native sets (ADR-071).""" + spec = importlib.util.find_spec("solprop_native") + if spec is None or spec.origin is None: + pytest.skip("solprop-native is not installed") + # Loaded by path: importing the package would start the native runtime. + path = Path(spec.origin).parent / "_wait.py" + loader = importlib.util.spec_from_file_location("native_wait_probe", path) + assert loader is not None and loader.loader is not None + module = importlib.util.module_from_spec(loader) + # Its dataclasses look their module up by name. + monkeypatch.setitem(sys.modules, "native_wait_probe", module) + loader.loader.exec_module(module) + assert _source_worker._NATIVE_WAIT_MARKER == module.MARKER + assert _source_worker._NATIVE_WAIT_SETTINGS == module._SETTINGS + + +@pytest.mark.medium +def test_the_worker_environment_keeps_blas_idle(): + environment = worker_environment({}, {}) + assert "OPENBLAS_THREAD_TIMEOUT" in environment + + +@pytest.mark.medium +def test_values_cross_a_real_worker_bit_for_bit(worker): + rng = np.random.default_rng(49) + sent = {"field": rng.standard_normal((3, 257)), "labels": (1, 3), "z": 0.25} + received = worker.request("echo", sent) + np.testing.assert_array_equal(received["field"], sent["field"]) + assert received["field"].dtype == np.float64 + assert received["labels"] == (1, 3) and received["z"] == 0.25 + assert worker.identity["adapter"] == "manufactured" + assert worker.pid == worker.identity["pid"] != os.getpid() + + +@pytest.mark.medium +def test_the_provider_s_printing_never_corrupts_a_frame(worker): + assert worker.request("print") == "printed" + assert worker.request("echo", [1, 2]) == [1, 2] + + +@pytest.mark.medium +def test_a_provider_exception_is_reported_by_name_and_ends_the_worker(worker): + with pytest.raises(RemoteError) as raised: + worker.request("raise") + error = raised.value + assert error.type_name == "builtins.ValueError" + assert error.detail == "manufactured provider failure" + assert error.classification == "manufactured" + assert "manufactured provider failure" in error.remote_traceback + # Never used again after a failure (section 6.5): its owner starts a fresh one. + _assert_ended(worker) + + +@pytest.mark.medium +def test_a_worker_that_exits_is_a_named_failure_with_its_status_and_stderr(worker): + with pytest.raises(WorkerError, match="exit status 7") as raised: + worker.request("exit") + assert "gives up" in "\n".join(raised.value.__notes__) + _assert_ended(worker) + with pytest.raises(WorkerError, match="not running"): + worker.request("echo", 1) + + +@pytest.mark.parametrize( + ("kind", "message"), + [ + ("close_pipe", "ended"), + ("malformed", "malformed worker message"), + ], +) +@pytest.mark.medium +def test_a_closed_pipe_or_a_malformed_frame_ends_the_worker(worker, kind, message): + with pytest.raises(WorkerError, match=message): + worker.request(kind) + _assert_ended(worker) + + +@pytest.mark.medium +def test_an_oversized_frame_is_refused_before_it_is_read(): + worker = _worker(limit=4096) + with pytest.raises(WorkerError, match="over its bound of 4096"): + worker.request("oversized", {"bytes": 10_000}) + _assert_ended(worker) + + +@pytest.mark.medium +def test_a_failure_while_starting_is_reported_by_name(): + worker = Worker( + "source_adapters:failing_adapter", + environment=worker_environment(os.environ, {}), + ) + with pytest.raises(RemoteError, match="cannot start"): + worker.start() + _assert_ended(worker) + + +@pytest.mark.medium +def test_a_protocol_mismatch_is_refused_before_any_request(monkeypatch): + monkeypatch.setattr(_source_worker, "PROTOCOL_VERSION", 99) + worker = Worker("source_adapters", environment=worker_environment(os.environ, {})) + with pytest.raises(WorkerError, match="protocol 1, not 99"): + worker.start() + _assert_ended(worker) + + +@pytest.mark.parametrize( + "body", + ['{"$d": []}', "[" * 100_000 + "]" * 100_000], + ids=["bad tag", "too deep"], +) +@pytest.mark.medium +def test_a_message_this_side_cannot_decode_is_a_protocol_error(body): + control = f'{{"kind": "reply", "body": {body}, "arrays": []}}'.encode() + stream = io.BytesIO(_source_worker._HEADER.pack(len(control)) + control) + with pytest.raises(WorkerError, match="malformed worker message"): + _source_worker.read_message(stream.read, len(control)) + + +def _frame(payload): + return _source_worker._HEADER.pack(len(payload)) + payload + + +def _control(*arrays, kind: object = "reply"): + message = {"kind": kind, "body": None, "arrays": list(arrays)} + return _frame(json.dumps(message).encode()) + + +_LIMIT = 4096 +_HEADER = _source_worker._HEADER +_FOUR = {"dtype": " 0 @@ -448,7 +459,9 @@ def unexpected_import(): @pytest.mark.parametrize( "device,space", [("cpu", "OpenMP"), ("cuda", "Cuda"), ("hip", "HIP")] ) -def test_binding_requires_matching_device_and_revision(provider, device, space): +def test_binding_requires_a_matching_device_and_records_any_revision( + provider, device, space +): provider.build = _Build(space) binding = tdse.prepare_binding(_descriptor(), TDSEExecution(device=device), 1) assert "plan.1.state" in binding.memory_ledger() @@ -457,9 +470,12 @@ def test_binding_requires_matching_device_and_revision(provider, device, space): provider.build = _Build("other") with pytest.raises(BackendError, match="native build"): tdse.prepare_binding(_descriptor(), TDSEExecution(device=device), 1) + # No revision is pinned: another provider revision is used and recorded. provider.build = _Build(space, "old0000") - with pytest.raises(BackendError, match="pinned"): - tdse.prepare_binding(_descriptor(), TDSEExecution(device=device), 1) + binding = tdse.prepare_binding(_descriptor(), TDSEExecution(device=device), 1) + build = cast("Any", binding.metadata())["build"] + assert build["native"]["source_revision"] == "old0000" + binding.close() def test_binding_input_and_resume_contract(provider): @@ -490,7 +506,7 @@ def test_binding_missing_optional_provider_is_actionable(monkeypatch, pinned_pyt def missing(name): raise ImportError("synthetic missing dependency") - monkeypatch.setattr(tdse.importlib, "import_module", missing) + monkeypatch.setattr(_pyroraptor.importlib, "import_module", missing) with pytest.raises(BackendError, match="optional pyroraptor"): tdse.basis_from_preset("helium", "production") @@ -518,7 +534,8 @@ def test_binding_module_import_does_not_load_optional_engine(): def test_binding_rejects_unsupported_interpreter_before_native_import(monkeypatch): - monkeypatch.setattr(tdse.sys, "version_info", (3, 15, 0)) + monkeypatch.setattr(_pyroraptor.sys, "version_info", (3, 15, 0)) + source_channel.in_process(monkeypatch) with pytest.raises(BackendError, match=r"Python 3\.14"): tdse.basis_from_preset("helium", "production") @@ -527,16 +544,19 @@ def test_binding_without_distribution_metadata_still_records_native_identity( monkeypatch, pinned_python ): provider = _Provider() - monkeypatch.setattr(tdse.importlib, "import_module", lambda name: provider) + monkeypatch.setattr(_pyroraptor.importlib, "import_module", lambda name: provider) def missing_distribution(name): - raise tdse.PackageNotFoundError(name) + raise _pyroraptor.PackageNotFoundError(name) - monkeypatch.setattr(tdse, "distribution", missing_distribution) + monkeypatch.setattr(_pyroraptor, "distribution", missing_distribution) binding = tdse.prepare_binding(_descriptor(), TDSEExecution(), 1) metadata = _saved_metadata(binding) assert metadata["build"]["distribution"] is None - assert metadata["build"]["source_pin"].startswith(provider.build.source_revision) + assert metadata["build"]["native"]["source_revision"] == ( + provider.build.source_revision + ) + assert "source_pin" not in metadata["build"] assert provider.assembled == 1 assert provider.calls == [] binding.close() @@ -548,7 +568,7 @@ def test_binding_records_an_installed_distribution( ): """The optional extra, when installed, adds its version and source to the record.""" provider = _Provider() - monkeypatch.setattr(tdse.importlib, "import_module", lambda name: provider) + monkeypatch.setattr(_pyroraptor.importlib, "import_module", lambda name: provider) class _Distribution: version = "1.2.3" @@ -557,7 +577,7 @@ def read_text(self, name): assert name == "direct_url.json" return direct_url - monkeypatch.setattr(tdse, "distribution", lambda name: _Distribution()) + monkeypatch.setattr(_pyroraptor, "distribution", lambda name: _Distribution()) binding = tdse.prepare_binding(_descriptor(), TDSEExecution(), 1) build = _saved_metadata(binding)["build"] assert build["distribution"] == "1.2.3" @@ -719,6 +739,87 @@ def test_malformed_native_results_preserve_diagnostics_and_charge_attempt( binding.close() +@pytest.mark.parametrize( + ("model", "failure", "message", "classification"), + [ + ("optmat_cubic", "no_record", "returned no result record: None", "contract"), + ( + "tdse_shadow", + "component_refused", + "no total_current at order 1", + "interface", + ), + ( + "tdse_shadow", + "component_broken", + "KeyError: 'no total_current at order 1'", + "unexpected", + ), + ], +) +def test_a_missing_record_or_a_refused_component_is_a_classified_failure( + provider, model, failure, message, classification +): + """A component the provider refused raises here as the provider raised it.""" + binding = tdse.prepare_binding(_descriptor(model), TDSEExecution(), 2) + provider.failure = failure + binding.reserve(1) + expected = rf"{re.escape(message)} \[{classification}\]" + with pytest.raises(BackendError, match=expected): + binding.evaluate(np.ones((2, _SAMPLES)), z=0.5, purpose="corrected") + assert binding.ledger.attempted == binding.ledger.failed_calls == 1 + failed = _saved_metadata(binding)["failures"][0] + assert failed["classification"] == classification + assert (failed["native"] is None) is (failure == "no_record") + binding.close() + + +def test_a_fresh_worker_must_report_the_same_provider_build(provider): + """A worker is replaced after a failure, never by another provider's.""" + binding = tdse.prepare_binding(_descriptor(), TDSEExecution(), 1) + provider.failure = "raise" + binding.reserve(1) + with pytest.raises(BackendError, match="synthetic native failure"): + binding.evaluate(np.ones((1, _SAMPLES)), z=0.5, purpose="corrected") + provider.failure = None + provider.build = _Build(source_revision="old0000") + binding.reserve(1) + with pytest.raises(BackendError, match="build changed between its workers"): + binding.evaluate(np.ones((1, _SAMPLES)), z=0.5, purpose="corrected") + assert binding.ledger.attempted == 1 and len(provider.calls) == 1 + binding.close() + + +def test_a_worker_s_exception_keeps_solprop_s_class_where_its_message_rebuilds_it(): + rebuilt = tdse._local_error( + RemoteError("solprop.exceptions.DriverError", "too large", "contract", "tb"), + "TDSE preparation failed", + ) + assert type(rebuilt) is DriverError and str(rebuilt) == "too large" + assert "raised in the TDSE provider worker:\ntb" in rebuilt.__notes__ + # A class with required fields beyond its message is wrapped, as a foreign one is. + wrapped = tdse._local_error( + RemoteError( + "solprop.exceptions.ResponseBudgetError", "spent", "contract", "tb" + ), + "TDSE preparation failed", + ) + assert type(wrapped) is BackendError + assert str(wrapped) == "TDSE preparation failed: spent [contract]" + assert "the worker's traceback:\ntb" in wrapped.__notes__ + + +def test_a_local_failure_while_preparing_is_wrapped_and_classified(): + """This process's own non-solprop error, as a reply it cannot read would raise.""" + error = tdse._preparation_error(ValueError("the reply has no memory record")) + assert type(error) is BackendError + assert str(error) == ( + "TDSE preparation failed: the reply has no memory record [interface]" + ) + stop = KeyboardInterrupt() + assert tdse._preparation_error(stop) is stop + + def test_second_chunk_failure_preserves_first_chunk_and_allows_fresh_retry( provider, monkeypatch ): @@ -743,5 +844,8 @@ def second_chunk_fails(plan, fields): result = binding.evaluate(np.ones((3, _SAMPLES)), z=0.5, purpose="corrected") assert_array_equal(result.total_current, np.full((3, _SAMPLES), 6.0)) assert binding.ledger.attempted == 4 - assert len(provider.created) == 2 + # The failed worker was ended with its plans; the retry's fresh worker prepared + # both chunk sizes' plans again (SEP 49 section 6.5). + assert len(provider.created) == 4 + assert provider.closed.count("plan-2") == provider.closed.count("plan-1") == 1 binding.close() diff --git a/tests/driver/test_tdse_consistency.py b/tests/driver/test_tdse_consistency.py index 7f18d8baf..ac2e6ef5c 100644 --- a/tests/driver/test_tdse_consistency.py +++ b/tests/driver/test_tdse_consistency.py @@ -21,6 +21,7 @@ import pytest from scipy.integrate import solve_ivp from tests.hacc_cases import ( + FakeBuild, alpha_law, hacc_response, helium_alpha, @@ -62,7 +63,7 @@ _LAMBDA = 800e-9 _SUPPORT = (0.0, 2 * math.pi * C / 60e-9) -_REVISION = tdse.pinned_revision() +_REVISION = FakeBuild().source_revision # the manufactured provider's def _reference(omega): diff --git a/tests/driver/test_tdse_entry.py b/tests/driver/test_tdse_entry.py index fb0ee91ff..9bade7922 100644 --- a/tests/driver/test_tdse_entry.py +++ b/tests/driver/test_tdse_entry.py @@ -87,10 +87,9 @@ def test_tdse_keyword_builds_the_one_response_run_in_one_step(caplog): simulation = _capillary(tdse=response, tdse_execution=execution) assert simulation.responses == (response,) assert simulation.settings.tdse_execution == execution - assert simulation.settings.backend == "numpy_ref" - assert "resolves to numpy_ref" in caplog.text + assert simulation.settings.backend == "native" record = plain(simulation.configuration["tdse_entry"]) - assert record["backend"] == {"requested": "auto", "resolved": "numpy_ref"} + assert record["backend"] == {"requested": "auto", "resolved": "native"} assert record["model"] == "sae" assert tdse_descriptor(simulation.translate().route) is not None # A default execution record is taken when none is given. @@ -125,10 +124,10 @@ def test_lower_level_construction_refuses_an_unresolved_window_before_any_provid ): parent = _capillary(tdse=_sae()) - def unexpected(): - pytest.fail("the provider was loaded") + def unexpected(**options): + pytest.fail("the provider's worker was started") - monkeypatch.setattr(runtime, "_load_provider", unexpected) + monkeypatch.setattr(runtime, "_open_provider", unexpected) simulation = Simulation( parent.grid, parent.modes, @@ -183,9 +182,10 @@ def test_tdse_execution_alone_and_a_foreign_record_are_refused(): ) -def test_explicit_incompatible_settings_keep_their_existing_refusals(): - with pytest.raises(DriverError, match=r"explicit backend='numpy_ref'.*native"): - _capillary(tdse=_sae(), backend="native") +def test_an_explicit_backend_is_kept_for_a_tdse_run(): + # Both backends run the TDSE response (SEP 49). + for backend in ("native", "numpy_ref"): + assert _capillary(tdse=_sae(), backend=backend).settings.backend == backend def test_statistics_default_excludes_what_a_tdse_response_cannot_supply(): diff --git a/tests/driver/test_tdse_hacc_binding.py b/tests/driver/test_tdse_hacc_binding.py index 14d4ecb38..c5f62664e 100644 --- a/tests/driver/test_tdse_hacc_binding.py +++ b/tests/driver/test_tdse_hacc_binding.py @@ -13,7 +13,7 @@ import inspect import json import warnings -from typing import Any +from typing import Any, cast import numpy as np import pytest @@ -32,8 +32,10 @@ tiny_basis, ) +import source_channel from solprop.constants import EPS0, C from solprop.driver import tdse +from solprop.driver._source_worker import RemoteError from solprop.exceptions import ( BackendError, DriverError, @@ -295,22 +297,23 @@ def test_hacc_refuses_vendor_blas_device_and_missing_surface(fake): binding = _bind(_descriptor(preset_basis(1)), TDSEExecution(device="cuda")) binding.close() fake.hacc = object() - with pytest.raises(BackendError, match=r"no haCC consumer surface.*876b72d1"): + with pytest.raises(BackendError, match=r"no haCC consumer surface"): _bind(_descriptor(), TDSEExecution(device="cuda")) -@pytest.mark.parametrize( - ("build", "device", "message"), - [ - (FakeBuild("OpenMP", "c2833fa"), "cpu", "pinned"), - (FakeBuild("OpenMP"), "cuda", "native build"), - ], -) -def test_stale_revision_and_device_mismatch_are_refused(fake, build, device, message): - fake.build = build - with pytest.raises(BackendError, match=message): - _bind(_descriptor(), TDSEExecution(device=device)) +def test_a_device_mismatch_is_refused_and_any_revision_recorded(fake): + fake.build = FakeBuild("OpenMP") + with pytest.raises(BackendError, match="native build"): + _bind(_descriptor(), TDSEExecution(device="cuda")) assert fake.prepared == [] + # No revision is pinned (SEP 49 section 6.5): another one runs, and is recorded. + fake.build = FakeBuild("OpenMP", "c2833fa") + binding = _bind(_descriptor(), TDSEExecution(device="cpu")) + build = cast("Any", binding.metadata())["build"] + assert build["native"]["source_revision"] == "c2833fa" + binding.close() + preset = tdse.hacc_preset(1) + assert preset.basis.source.startswith("pyroraptor@c2833fa: ") def test_plan_construction_driven_error_is_a_preparation_failure(fake): @@ -429,6 +432,7 @@ def test_valid_call_returns_owning_read_only_padded_arrays(fake): "wrong_type", "info", "info_atom", + "info_unreadable", "labels", "bound_none", "bound_negative", @@ -655,7 +659,8 @@ def test_hacc_preset_returns_si_records_and_a_research_calibration(fake, ions, v assert fake.variant_requests == [variant] assert preset.basis.preset == preset.identity == expected.preset assert dataclasses.replace(preset.basis, source=expected.source) == expected - assert preset.basis.source.startswith("pyroraptor@876b72d1") + # The revision the provider's worker reported, not a pin. + assert preset.basis.source.startswith(f"pyroraptor@{fake.build.source_revision}: ") assert preset.variant == preset.basis.variant == variant assert preset.step_margin == (0.62 if variant == "standard" else 0.9) assert preset.max_atomic_step == expected.max_step_au * hacc_cases.AU_TIME @@ -781,10 +786,6 @@ def test_resolvers_refuse_a_provider_without_the_hacc_surface(fake): tdse.hacc_basis(None, None, None, 1.0, 1.0, source="x") -def test_pinned_revision_is_the_documented_pin(): - assert tdse.pinned_revision() == "876b72d10b5286478125b842ea45548a5051b033" - - def test_window_record_is_reconstructed_unchanged(fake): descriptor = _descriptor( preset_basis(1), atomic_window=TDSEAtomicWindow(6e-15, 1e-15) @@ -850,7 +851,7 @@ def test_sae_binding_has_no_consistency_diagnostic(monkeypatch): from tests.driver.test_tdse_binding import _Provider # noqa: PLC0415 provider = _Provider() - monkeypatch.setattr(tdse, "_load_provider", lambda: provider) + source_channel.install(monkeypatch, provider) binding: Any = tdse.prepare_binding(sae("tdse_native"), TDSEExecution(), 1) field, spacing = _protocol() with pytest.raises(BackendError, match="needs a haCC binding"): @@ -858,6 +859,26 @@ def test_sae_binding_has_no_consistency_diagnostic(monkeypatch): binding.close() +@pytest.mark.parametrize( + ("kind", "message"), + [ + ("no_such_request", "has no request 'no_such_request'"), + ("diagnostic_plan", "needs a haCC binding"), + ("diagnostic_evaluate", "no consistency diagnostic plan is open"), + ], +) +def test_the_adapter_refuses_requests_outside_its_protocol(fake, kind, message): + """The worker's adapter refuses by name what the binding never sends.""" + worker = source_channel.InProcessWorker() + request = {"samples": 8, "spacing": 1e-17, "drive": np.zeros((1, 8))} + with pytest.raises(RemoteError, match=message) as raised: + worker.request(kind, request) + assert raised.value.type_name == "solprop.exceptions.BackendError" + assert raised.value.classification == "contract" + assert not worker.alive + assert fake.calls == [] + + def test_diagnostic_session_classifies_preparation_failures_and_reports( fake, monkeypatch ): @@ -988,7 +1009,7 @@ def test_defer_plans_must_be_a_bool_and_is_ignored_for_sae(monkeypatch): from tests.driver.test_tdse_binding import _Provider # noqa: PLC0415 provider = _Provider() - monkeypatch.setattr(tdse, "_load_provider", lambda: provider) + source_channel.install(monkeypatch, provider) binding: Any = tdse.prepare_binding( sae("tdse_native"), TDSEExecution(), 1, defer_plans=True ) diff --git a/tests/driver/test_tdse_hacc_installed.py b/tests/driver/test_tdse_hacc_installed.py index 80a61b563..9a4a40d98 100644 --- a/tests/driver/test_tdse_hacc_installed.py +++ b/tests/driver/test_tdse_hacc_installed.py @@ -4,12 +4,13 @@ """Exercise the real haCC provider surface when it is installed (SEP 20, 7.1). -Skipped unless Pyroraptor at the pinned revision is importable. The check runs in a -subprocess with one provider and one OpenMP thread, so the provider's native runtime -never shares this test process: it resolves both presets through ``hacc_preset``, -checks their identities and SI conversion against the records the fake-provider tests -use, and runs one call of the provider's own tiny consumer-example haCC(1) basis -through the binding (C3 to C5). It qualifies nothing about the atomic physics. +Skipped unless Pyroraptor is importable, at any revision: solprop pins none (SEP 49 +section 6.5). The check runs in a subprocess with one provider and one OpenMP thread, +so the provider's native runtime never shares this test process: it resolves both +presets through ``hacc_preset``, checks their identities and SI conversion against the +records the fake-provider tests use, and runs one call of the provider's own tiny +consumer-example haCC(1) basis through the binding (C3 to C5). It qualifies nothing +about the atomic physics. """ import importlib.util @@ -86,7 +87,10 @@ from tests.hacc_cases import A0 control = tiny_basis(gauge_radius=1.0 * A0) -atom, native, gauge = tdse._hacc_objects(tdse._load_provider(), hacc, control) +from solprop.driver import _pyroraptor + +provider = _pyroraptor._load_provider() +atom, native, gauge = _pyroraptor._hacc_objects(provider, hacc, control) with hacc.prepare(atom, native, gauge=gauge, cut_energy=control.cut_energy) as model: try: hacc.ResponsePlan( @@ -117,11 +121,9 @@ """ -def _pinned_provider_installed() -> bool: +def _provider_installed() -> bool: if importlib.util.find_spec("pyroraptor") is None: return False - from solprop.driver import tdse # noqa: PLC0415 - only after the spec check - probe = subprocess.run( [ sys.executable, @@ -134,8 +136,7 @@ def _pinned_provider_installed() -> bool: env=_environment(), check=False, ) - revision = probe.stdout.strip().splitlines()[-1] if probe.stdout.strip() else "" - return probe.returncode == 0 and tdse.pinned_revision().startswith(revision or "-") + return probe.returncode == 0 and bool(probe.stdout.strip()) def _environment() -> dict[str, str]: @@ -150,8 +151,8 @@ def _environment() -> dict[str, str]: @pytest.mark.skipif( - not _pinned_provider_installed(), - reason="Pyroraptor at the pinned revision is not installed", + not _provider_installed(), + reason="Pyroraptor is not installed", ) def test_installed_provider_resolves_presets_and_runs_a_tiny_custom_call(): completed = subprocess.run( diff --git a/tests/driver/test_tdse_worker.py b/tests/driver/test_tdse_worker.py new file mode 100644 index 000000000..fbaa7c064 --- /dev/null +++ b/tests/driver/test_tdse_worker.py @@ -0,0 +1,287 @@ +# SPDX-FileCopyrightText: 2026 John C. Travers and contributors +# +# SPDX-License-Identifier: GPL-3.0-or-later + +"""The TDSE binding through a real provider worker (SEP 49 section 7.2). + +The worker serves Pyroraptor's adapter around the manufactured haCC provider of +``hacc_cases`` (``tests/source_adapters.py``), which this process never imports as its +provider. The same provider served in this process (``source_channel``) is the direct +call the worker must reproduce bit for bit. +""" + +import json +import sys +import threading +import warnings +from typing import Any, cast + +import numpy as np +import pytest + +import source_channel +from driver.test_tdse_hacc_binding import _descriptor, _fields, _protocol +from driver.test_tdse_hacc_run import _simulation +from hacc_cases import FakeHaccProvider, alpha_law, preset_basis +from solprop.driver import tdse +from solprop.exceptions import ( + BackendError, + DriverError, + StopRequested, + TDSEDomainWarning, + TDSEUnqualifiedModelWarning, +) +from solprop.nonlinear.tdse import TDSEExecution +from solprop.output import open_file, resume + +DEADLINE = 10.0 +"""A test-harness bound on a cancellation; the production code has none.""" + + +@pytest.fixture +def workers(monkeypatch): + """Serve the manufactured provider in real workers; return the workers started.""" + monkeypatch.setattr(tdse, "_ADAPTER", "source_adapters:manufactured_tdse") + started = [] + original = tdse._open_provider + + def opened(**options): + worker = original(**options) + started.append(worker) + return worker + + monkeypatch.setattr(tdse, "_open_provider", opened) + return started + + +@pytest.fixture(autouse=True) +def cache(monkeypatch, tmp_path): + monkeypatch.setenv("SOLPROP_CACHE_DIR", str(tmp_path / "cache")) + + +def _bind(descriptor, execution=None, **options): + with warnings.catch_warnings(): + warnings.simplefilter("ignore", TDSEUnqualifiedModelWarning) + return tdse.prepare_binding( + descriptor, + TDSEExecution(max_batch_rows=2) if execution is None else execution, + 3, + **options, + ) + + +def _evaluate_once(descriptor): + binding = _bind(descriptor) + binding.reserve(1) + result = binding.evaluate(_fields(3), z=0.1, purpose="startup", density=2.0) + binding.record_trial(accepted=True) + binding.release_reservation() + binding.close() + return result, binding + + +def _plain(record): + return json.loads(json.dumps(record, allow_nan=False)) + + +@pytest.mark.medium +def test_evaluations_through_the_worker_equal_the_provider_called_directly(workers): + descriptor = _descriptor(preset_basis(1)) + provider_modules = ("pyroraptor", "source_adapters") + with pytest.MonkeyPatch.context() as isolated: + # Another test of this process may have loaded a provider module (a fake + # one, in-process): set them aside, so that what follows shows what this + # run imports, whatever ran before it. + for name in [n for n in sys.modules if n.split(".")[0] in provider_modules]: + isolated.delitem(sys.modules, name) + through, worker_binding = _evaluate_once(descriptor) + # The provider lived in the worker alone. + loaded = [n for n in sys.modules if n.split(".")[0] in provider_modules] + assert not loaded + with pytest.MonkeyPatch.context() as patch: + fake = source_channel.install(patch, FakeHaccProvider()) + fake.response = alpha_law(1.02, lambda5=2.0**-200) + direct, direct_binding = _evaluate_once(descriptor) + for name in ("total_current", "first_order_current", "valid"): + np.testing.assert_array_equal(getattr(through, name), getattr(direct, name)) + assert through.monitors_json == direct.monitors_json + assert through.outcomes_json == direct.outcomes_json + assert worker_binding.ledger == direct_binding.ledger + assert _plain(worker_binding.metadata()) == _plain(direct_binding.metadata()) + assert len(workers) == 1 and not workers[0].alive + + +@pytest.mark.medium +def test_the_handshake_refuses_another_execution_space_by_name(monkeypatch, workers): + monkeypatch.setattr(tdse, "_ADAPTER", "source_adapters:cuda_tdse") + with pytest.raises(BackendError, match="native build uses"): + _bind(_descriptor(preset_basis(1))) + assert len(workers) == 1 and not workers[0].alive + + +@pytest.mark.medium +def test_the_handshake_records_another_provider_revision(monkeypatch, workers): + monkeypatch.setattr(tdse, "_ADAPTER", "source_adapters:stale_tdse") + binding = _bind(_descriptor(preset_basis(1))) + try: + build = cast("Any", binding.metadata())["build"] + assert build["native"]["source_revision"] == "old0000" + finally: + binding.close() + + +@pytest.mark.medium +def test_a_failure_in_preparation_is_named_and_ends_the_worker(workers): + with pytest.raises(BackendError, match="no complement option") as raised: + _bind( + _descriptor(preset_basis(1)), + TDSEExecution(max_batch_rows=2, complement="vendor_blas"), + ) + assert "raised in the TDSE provider worker" in "\n".join(raised.value.__notes__) + assert len(workers) == 1 and not workers[0].alive + + +@pytest.mark.medium +def test_the_transport_copies_are_in_the_ledger_and_admitted_before_start(workers): + descriptor = _descriptor(preset_basis(1)) + execution = TDSEExecution(max_batch_rows=2) + estimate = tdse.estimate_binding_memory(descriptor, execution, 3) + binding = _bind(descriptor, execution) + ledger = binding.memory_ledger() + window = descriptor.atomic_window + assert window is not None + copies = 8 * 2 * window.samples * (2 + 20) + assert ledger["transport.solprop"] == ledger["transport.worker"] == copies + assert estimate["estimate.transport"] == 2 * copies + assert any(name.startswith("model.") for name in ledger) + binding.close() + limit = sum(estimate.values()) - 1 + with pytest.raises(DriverError, match="TDSE persistent data"): + _bind(descriptor, TDSEExecution(max_batch_rows=2, memory_limit=limit)) + assert len(workers) == 1 + + +@pytest.mark.medium +def test_a_stop_during_an_atomic_call_kills_the_worker_and_keeps_the_charge( + monkeypatch, workers +): + monkeypatch.setenv("MANUFACTURED_HANG_AFTER", "0") + flag = threading.Event() + binding = _bind(_descriptor(preset_basis(1)), stop=flag) + binding.reserve(1) + threading.Timer(0.3, flag.set).start() + with pytest.raises(StopRequested): + binding.evaluate(_fields(3), z=0.1, purpose="startup") + assert binding.ledger.attempted == 1 and binding.ledger.failed_calls == 0 + assert not workers[0].alive + assert binding.metadata()["failures"] == [] + binding.close() + + +@pytest.mark.medium +def test_after_a_provider_exception_the_retry_gets_a_fresh_worker( + monkeypatch, workers, tmp_path +): + """A worker is never used again after a failure (section 6.5).""" + monkeypatch.setenv("MANUFACTURED_FAIL_ONCE", str(tmp_path / "failed-once")) + binding = _bind(_descriptor(preset_basis(1))) + binding.reserve(1) + with pytest.raises(BackendError, match=r"manufactured provider failure \[native\]"): + binding.evaluate(_fields(3), z=0.1, purpose="startup") + assert not workers[0].alive + binding.reserve(1) + result = binding.evaluate(_fields(3), z=0.1, purpose="startup") + assert np.all(np.isfinite(result.total_current)) + assert len(workers) == 2 and workers[1].alive + assert binding.ledger.attempted == 3 and binding.ledger.failed_calls == 1 + binding.close() + assert not workers[1].alive + + +@pytest.mark.medium +def test_a_provider_that_cannot_start_is_named(monkeypatch, workers): + monkeypatch.setattr(tdse, "_ADAPTER", "source_adapters:failing_adapter") + expected = ( + r"TDSE provider start failed: the manufactured provider cannot start " + r"\[unexpected\]" + ) + with pytest.raises(BackendError, match=expected): + _bind(_descriptor(preset_basis(1))) + assert workers == [] + + +@pytest.mark.medium +def test_a_worker_that_dies_in_a_call_is_a_charged_worker_failure(monkeypatch, workers): + monkeypatch.setenv("MANUFACTURED_EXIT_IN_CALL", "1") + binding = _bind(_descriptor(preset_basis(1))) + binding.reserve(1) + with pytest.raises( + BackendError, match=r"exit status 9\) without replying \[worker\]" + ) as raised: + binding.evaluate(_fields(3), z=0.1, purpose="startup") + # Its last standard error rides on the error raised. + assert "exits mid-call" in "\n".join(raised.value.__notes__) + assert binding.ledger.attempted == binding.ledger.failed_calls == 1 + assert cast("Any", binding.metadata())["failures"][0]["classification"] == "worker" + assert not workers[0].alive + binding.close() + + +@pytest.mark.medium +def test_a_stop_during_the_diagnostic_call_kills_the_worker_and_keeps_the_charge( + monkeypatch, workers +): + monkeypatch.setenv("MANUFACTURED_HANG_AFTER", "0") + flag = threading.Event() + binding: Any = _bind(_descriptor(preset_basis(1)), stop=flag) + field, spacing = _protocol() + threading.Timer(0.3, flag.set).start() + with pytest.raises(StopRequested): + binding.diagnostic_currents(field, spacing) + assert binding.ledger.attempted == 1 and binding.ledger.failed_calls == 0 + assert binding.metadata()["failures"] == [] + assert len(workers) == 1 and not workers[0].alive + binding.close() + + +class _Watch: + """Pass standard error on; set a flag when the worker says its call hangs.""" + + def __init__(self, stream, flag): + self._stream, self._flag = stream, flag + + def write(self, text): + if "MANUFACTURED CALL HANGS" in text: + self._flag.set() + return self._stream.write(text) + + def flush(self): + self._stream.flush() + + +@pytest.mark.slow +def test_a_run_stopped_during_an_atomic_call_checkpoints_its_last_step( + monkeypatch, workers, tmp_path +): + """numpy_ref: the stop ends the run as stopped, the call charged (section 6.7).""" + monkeypatch.setenv("MANUFACTURED_HANG_AFTER", "12") + flag = threading.Event() + monkeypatch.setattr(sys, "stderr", _Watch(sys.stderr, flag)) + path = tmp_path / "stopped.h5" + simulation = _simulation(backend="numpy_ref") + with warnings.catch_warnings(): + warnings.simplefilter("ignore", TDSEDomainWarning) + simulation.run(path, progress="none", stop=flag) + stopped = resume(path) + assert stopped.outcome is not None + # As any stopped run: ended before its length, with a terminal checkpoint. + assert stopped.outcome.status == "failed" + assert "StopRequested" in stopped.outcome.message + calls = stopped.outcome.atomic_calls + # Twelve calls returned; the thirteenth was charged and never returned. + assert calls["attempted"] == 13 + assert calls["failed_calls"] == 0 + with open_file(path) as stored: + assert stored.terminal_checkpoint is not None + assert stored.terminal_checkpoint.plane.z > 0 + assert not workers[0].alive diff --git a/tests/edge_source_cases.py b/tests/edge_source_cases.py index 9cc029cf6..95b68a801 100644 --- a/tests/edge_source_cases.py +++ b/tests/edge_source_cases.py @@ -19,7 +19,7 @@ import numpy as np from solprop.backends.numpy_ref import passes -from solprop.backends.numpy_ref.tdse import ( +from solprop.backends.tdse_source import ( FFT_ERROR_CONSTANT, _current_operands, _current_part, @@ -135,6 +135,7 @@ def make_case( fft_size: str = "pow2", n_r: int = 4, kerr_model: str = "tdse_native", + responses: tuple = (), ) -> EdgeCase: """Assemble a capillary problem and attach a manufactured SAE descriptor. @@ -155,6 +156,9 @@ def make_case( kerr_model : str ``tdse_native`` for the gated source, or a cubic replacement, whose source is ungated. + responses : tuple + Optical responses compiled beside the atom: a Kerr term for a cubic + replacement's polarisation; none by default. Returns ------- @@ -167,7 +171,7 @@ def make_case( problem = assemble( grid, mode, - (), + responses, np.zeros((1, 1, grid.n_omega), dtype=np.complex128), controller=PIController(rtol=1e-2, atol=1e-14), save_planes=[0.001], diff --git a/tests/hacc_cases.py b/tests/hacc_cases.py index 93a3b0447..22253f89e 100644 --- a/tests/hacc_cases.py +++ b/tests/hacc_cases.py @@ -511,6 +511,11 @@ def inject(raw, failure): # noqa: PLR0912 - one injected malformation per branc raw.info = dataclasses.replace(raw.info, spacing=raw.info.spacing * 2) elif failure == "info_atom": raw.info = dataclasses.replace(raw.info, atom=FakeHelium(2, 6)) + elif failure == "info_unreadable": + # An info record the provider's comparison cannot read: no atom. + fields = {name: getattr(raw.info, name) for name in vars(raw.info)} + del fields["atom"] + raw.info = SimpleNamespace(**fields) elif failure == "wrong_type": return SimpleNamespace(**vars(raw)) elif failure == "extents": @@ -704,12 +709,11 @@ def _unused(self, *args, **kwargs): def install(monkeypatch, fake=None): - """Make ``solprop.driver.tdse`` load a fake provider; return it.""" - from solprop.driver import tdse as runtime # noqa: PLC0415 - keep tests importable + """Make ``solprop.driver.tdse`` serve a fake provider in this process; return it.""" + import source_channel # noqa: PLC0415 - keep tests importable fake = FakeHaccProvider() if fake is None else fake - monkeypatch.setattr(runtime, "_load_provider", lambda: fake) - return fake + return source_channel.install(monkeypatch, fake) def helium_alpha(omega): diff --git a/tests/source_adapters.py b/tests/source_adapters.py new file mode 100644 index 000000000..38ec31a6f --- /dev/null +++ b/tests/source_adapters.py @@ -0,0 +1,236 @@ +# SPDX-FileCopyrightText: 2026 John C. Travers and contributors +# +# SPDX-License-Identifier: GPL-3.0-or-later + +"""Manufactured adapters the source worker serves in the tests, inside the worker. + +``make_adapter`` is a provider-agnostic adapter whose requests misbehave on purpose: +it echoes, hangs, raises, exits, prints, closes its pipe, writes a malformed frame or a +message out of turn, returns what the codec refuses, starts a grandchild or makes its +own close hang. +The ``*_tdse`` factories serve the TDSE binding with Pyroraptor's adapter around the +manufactured haCC provider of ``hacc_cases`` (SEP 49 section 7.2), the module this +worker imports in place of ``pyroraptor``. +""" + +import os +import subprocess +import sys +import time +from pathlib import Path +from typing import Any, cast + +import numpy as np + +from solprop.driver import _pyroraptor, _source_worker + +HANG_SECONDS = 30.0 +"""How long a manufactured hang lasts: bounded, so that a broken cancellation fails a +test (the call returns) instead of hanging it, and far above any cancellation's time.""" + + +class _Manufactured: + """An adapter with no provider: each request names its behaviour.""" + + def __init__(self): + self._slow_close = False + + def identity(self): + return {"adapter": "manufactured", "pid": os.getpid()} + + def classify(self, error): + del error + return "manufactured" + + def computed(self, value): + del value + return {} + + def handle(self, kind, value): + if kind == "echo": + return value + if kind == "hang": + time.sleep(HANG_SECONDS) + return "woke" + if kind == "raise": + raise ValueError("manufactured provider failure") + if kind == "exit": + sys.stderr.write("the manufactured provider gives up\n") + sys.stderr.flush() + os._exit(7) + if kind == "print": + sys.stdout.write("noise on standard output\n") + sys.stdout.flush() + os.write(1, b"raw noise on descriptor 1\n") + return "printed" + if kind == "oversized": + return np.zeros(value["bytes"], dtype=np.uint8) + if kind == "malformed": + payload = b"{not json" + _source_worker._write_all( + _source_worker.PROTOCOL_DESCRIPTORS[1], + _source_worker._HEADER.pack(len(payload)) + payload, + ) + time.sleep(HANG_SECONDS) + if kind == "close_pipe": + os.close(_source_worker.PROTOCOL_DESCRIPTORS[1]) + time.sleep(HANG_SECONDS) + if kind == "modules": + spec = getattr(sys.modules["__main__"], "__spec__", None) + return { + "modules": sorted(sys.modules), + "main": None if spec is None else spec.name, + } + return self._out_of_line(kind) + + def _out_of_line(self, kind): + """Serve the environment and the requests that break the protocol.""" + if kind == "environment": + return dict(os.environ) + if kind == "hello_again": + _send_out_of_turn("hello", {"protocol": _source_worker.PROTOCOL_VERSION}) + time.sleep(HANG_SECONDS) + if kind == "unencodable": + nested = [] + nested.append(nested) + return nested + if kind == "spawn": + # In a session of its own, out of the worker's group, holding its stderr. + child = subprocess.Popen( + [sys.executable, "-c", f"import time; time.sleep({HANG_SECONDS})"], + start_new_session=True, + ) + return child.pid + if kind == "slow_close": + self._slow_close = True + return None + raise KeyError(kind) + + def close(self): + if self._slow_close: + time.sleep(HANG_SECONDS) + + +def _send_out_of_turn(kind, value): + """Write a message on the protocol pipe that the worker's loop did not send.""" + _source_worker.write_message( + lambda data: _source_worker._write_all( + _source_worker.PROTOCOL_DESCRIPTORS[1], data + ), + kind, + value, + ) + + +def make_adapter(): + return _Manufactured() + + +def failing_adapter(): + raise ValueError("the manufactured provider cannot start") + + +def early_reply_adapter(): + """Answer before the handshake, as a worker of another protocol might.""" + _send_out_of_turn("reply", "too early") + return _Manufactured() + + +def _provider(**changes): + from hacc_cases import FakeHaccProvider, alpha_law # noqa: PLC0415 - worker only + + provider = FakeHaccProvider(**changes) + # The run tests' law: a 2% linear model error and a dyadic fifth-order remainder. + provider.response = alpha_law(1.02, lambda5=2.0**-200) + marker = os.environ.get("MANUFACTURED_FAIL_ONCE") + if marker is not None: + _fail_once(provider, marker) + hang = os.environ.get("MANUFACTURED_HANG_AFTER") + if hang is not None: + _hang_after(provider, int(hang)) + if os.environ.get("MANUFACTURED_EXIT_IN_CALL") is not None: + _exit_in_call(provider) + return provider + + +def _fail_once(provider, marker): + """Make the first atomic call of all the workers sharing `marker` raise.""" + from hacc_cases import FakeDrivenError # noqa: PLC0415 - worker only + + original = provider.hacc.ResponsePlan + + def plan(model, **values): + built = original(model, **values) + evaluate = built.evaluate + + def failing(fields): + if not Path(marker).exists(): + Path(marker).touch() + raise FakeDrivenError("manufactured provider failure") + return evaluate(fields) + + built.evaluate = failing + return built + + provider.hacc.ResponsePlan = plan + + +def _hang_after(provider, calls): + """Make every atomic call after the first `calls` hang.""" + original = provider.hacc.ResponsePlan + + def plan(model, **values): + built = original(model, **values) + evaluate = built.evaluate + + def slow(fields): + if len(provider.calls) >= calls: + sys.stderr.write("MANUFACTURED CALL HANGS\n") + sys.stderr.flush() + time.sleep(HANG_SECONDS) + return evaluate(fields) + + built.evaluate = slow + return built + + provider.hacc.ResponsePlan = plan + + +def _exit_in_call(provider): + """Make every atomic call end the worker, saying why on its standard error.""" + original = provider.hacc.ResponsePlan + + def plan(model, **values): + built = original(model, **values) + + def exits(fields): + del fields + sys.stderr.write("the manufactured provider exits mid-call\n") + sys.stderr.flush() + os._exit(9) + + built.evaluate = exits + return built + + provider.hacc.ResponsePlan = plan + + +def _adapter(provider): + return _pyroraptor.PyroraptorAdapter(cast("Any", provider)) + + +def manufactured_tdse(): + return _adapter(_provider()) + + +def stale_tdse(): + """Serve the manufactured provider under another source revision.""" + from hacc_cases import FakeBuild # noqa: PLC0415 - worker only + + return _adapter(_provider(build=FakeBuild(source_revision="old0000"))) + + +def cuda_tdse(): + from hacc_cases import FakeBuild # noqa: PLC0415 - worker only + + return _adapter(_provider(build=FakeBuild(execution_space="Cuda"))) diff --git a/tests/source_channel.py b/tests/source_channel.py new file mode 100644 index 000000000..4fefaf931 --- /dev/null +++ b/tests/source_channel.py @@ -0,0 +1,92 @@ +# SPDX-FileCopyrightText: 2026 John C. Travers and contributors +# +# SPDX-License-Identifier: GPL-3.0-or-later + +"""Serve the TDSE binding's provider in this process, through the worker's codec. + +The binding talks to its provider through a worker process (SEP 49 section 6.5). Tests +whose fake providers are built and altered in the test itself serve them here instead: +the same adapter, every request and reply encoded to the protocol's bytes and decoded +again, but no process. The worker itself is tested with an importable provider in +``tests/driver/test_source_worker.py``. +""" + +import io + +from solprop.driver import _pyroraptor, _source_worker, tdse +from solprop.driver._source_worker import RemoteError, WorkerError + + +def _round_trip(value, computed=None): + """Return `value` as the other end of a pipe would read it.""" + buffer = io.BytesIO() + _source_worker.write_message(buffer.write, "reply", value, computed) + buffer.seek(0) + message = _source_worker.read_message(buffer.read, 1 << 34) + assert message is not None + return message[1] + + +def _remote(error, adapter): + report = _source_worker._error_report(error, adapter) + return RemoteError( + report["type"], report["message"], report["classification"], report["traceback"] + ) + + +class InProcessWorker: + """A worker's request protocol, its adapter served in this process.""" + + def __init__(self, make=_pyroraptor.make_adapter): + try: + self._adapter = make() + self.identity = _round_trip(self._adapter.identity()) + except Exception as error: + raise _remote(error, None) from error + self.requests = [] + self.closed = False + + def request(self, kind, value=None): + if self.closed: + raise WorkerError("the source worker is not running") + self.requests.append(kind) + sent = _round_trip(value) + try: + reply = self._adapter.handle(kind, sent) + except Exception as error: + # As a real worker: never used again after a failure. + self.close() + raise _remote(error, self._adapter) from error + return _round_trip(reply, self._adapter.computed) + + def close(self): + if not self.closed: + self.closed = True + self._adapter.close() + + kill = close + + @property + def alive(self): + return not self.closed + + +def in_process(monkeypatch): + """Make the binding serve its provider in this process; return the workers made.""" + workers = [] + + def open_provider(**options): + del options + worker = InProcessWorker() + workers.append(worker) + return worker + + monkeypatch.setattr(tdse, "_open_provider", open_provider) + return workers + + +def install(monkeypatch, provider): + """Serve `provider` as the binding's provider, in this process; return it.""" + monkeypatch.setattr(_pyroraptor, "_load_provider", lambda: provider) + in_process(monkeypatch) + return provider diff --git a/tests/test_check_skips.py b/tests/test_check_skips.py index 2c1b9aec7..2845c79f5 100644 --- a/tests/test_check_skips.py +++ b/tests/test_check_skips.py @@ -33,7 +33,7 @@ def test_an_expected_reason_and_a_pass_are_clean(tmp_path, capsys): path = junit( tmp_path, ("a", ""), - ("b", skip("Pyroraptor at the pinned revision is not installed")), + ("b", skip("Pyroraptor is not installed")), ) assert check_skips.main([str(path)]) == 0 assert "0 unexpected" in capsys.readouterr().out diff --git a/tests/test_check_strict_fma.py b/tests/test_check_strict_fma.py index 753c33227..37941efce 100644 --- a/tests/test_check_strict_fma.py +++ b/tests/test_check_strict_fma.py @@ -101,3 +101,35 @@ def test_the_native_sessions_unit_is_strict(tmp_path, monkeypatch, capsys): wheel = _wheel(tmp_path, monkeypatch, STRICT + session) assert check_strict_fma.main([wheel]) == 1 assert "solprop_native::session::apply_recipe" in capsys.readouterr().err + + +def test_code_past_a_strict_symbol_s_end_is_not_its_own(tmp_path, monkeypatch, capsys): + # A stripped wheel labels only exported functions: a link may place another file's + # unlabelled, contracting code right after a strict function (as LTO put a + # dispersive kernel after march::reached). The symbol's size bounds the function. + listing = ( + "0000000000001100 :\n" + " 1100:\tvmulsd %xmm1,%xmm0,%xmm0\n" + " 1104:\tretq\n" + " 1110:\tvfmadd231sd %xmm1,%xmm2,%xmm0\n" + ) + symbols = ( + "0000000000001100 g DF .text\t0000000000000005 Base " + "solprop_native::march::reached(double, double, int)\n" + ) + wheel = tmp_path / "solprop_native-0-cp314-abi3-linux_x86_64.whl" + with zipfile.ZipFile(wheel, "w") as archive: + archive.writestr("solprop_native/_core.abi3.so", b"") + + def objdump(command, **_): + table = "-T" in command + output = symbols if table else "" if "-t" in command else listing + return subprocess.CompletedProcess(command, 0, stdout=output, stderr="") + + monkeypatch.setattr(check_strict_fma.subprocess, "run", objdump) + assert check_strict_fma.main([str(wheel)]) == 0 + assert "1 strict functions, none fused" in capsys.readouterr().out + # Inside the symbol, the same instruction is the function's own. + listing = listing.replace(" 1110:", " 1102:") + assert check_strict_fma.main([str(wheel)]) == 1 + assert "1 fused multiply-add(s) in" in capsys.readouterr().err