From d33e8f24a5c128ce7a47fd0b8be04a7da3a0fd9b Mon Sep 17 00:00:00 2001 From: John Travers Date: Thu, 12 Nov 2026 12:50:31 +0000 Subject: [PATCH 01/20] SEP 49 phase 0: record the size and host-assembly baselines Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HAYtF5P9QqEd3HQx3WJkA3 --- docs/dev/seps/draft-sep-0049-part2.md | 55 +++++++++++++++++++++++++++ 1 file changed, 55 insertions(+) create mode 100644 docs/dev/seps/draft-sep-0049-part2.md diff --git a/docs/dev/seps/draft-sep-0049-part2.md b/docs/dev/seps/draft-sep-0049-part2.md new file mode 100644 index 000000000..9af8399f2 --- /dev/null +++ b/docs/dev/seps/draft-sep-0049-part2.md @@ -0,0 +1,55 @@ +--- +orphan: true +--- + +# SEP 49, Part II draft + +The phase reports of SEP 49's implementation, kept here until the SEP's own file reaches +this branch; they then move, unchanged, into its Part II, and this file goes. + +## Phase 0 — Baseline + +- **Scope.** Phase 0, 2026-11-12, Opus 5.5 (AI), branch + `feature/sep-0049-tdse-on-native`, based on `feature/two-backends-binary64` at + `1dc64b5f` (ADR-077's change). +- **Delivered.** This record; no code. +- **Size baseline** (`uv run poe size --all` at `1dc64b5f`, lines): + + | tree | files | total | code | + |---|---:|---:|---:| + | `src/solprop` | 269 | 225,295 | 113,232 | + | `native/src/solprop_native` (Python) | 24 | 13,859 | 8,855 | + | `tests/` (all `.py`, `wc -l`) | — | 271,204 | — | + | `native/tests/` (all `.py`, `wc -l`) | — | 13,602 | — | + | `native/src` C++ (`.cpp`, `.hpp`, `.h`, `wc -l`) | — | 34,575 | — | + + `poe size-check` is within budget. `src/solprop/backends/numpy_ref/tdse.py` is 845 + lines; `src/solprop/driver/tdse.py` 2,575. +- **The host assembly's time per evaluation.** Measured by a one-off script, not + committed: the `march_problem` problem of `tests/backends/test_tdse_march.py` + (256 µm helium capillary, 800 nm, one mode), the interaction formulation, a + manufactured binding returning analytic histories (`J = 1e-30 E + 1e-40 E^3`, + `j_1 = 1e-30 E`, and a third-order current for the cubic replacement), and + `NumpyRefBackend._nonlinear(..., purpose="predictor")` timed 60 times, the first ten + discarded, the binding's own time subtracted. That is steps 1 and 3–5 of §3.2: the + synthesis to the nodes, the assembly, the projection with every diagnostic, and the + stage output. + + | lattice | descriptor | `n_nodes` | `n_to` | `n_omega` | median | min | + |---|---|---:|---:|---:|---:|---:| + | the test window (`n_r = 4`, 60 fs, 0.4–1.6 µm) | `tdse_native`, gated | 5 | 384 | 97 | 0.26 ms | 0.25 ms | + | the test window | `optmat_cubic` | 5 | 384 | 97 | 0.13 ms | 0.12 ms | + | production-sized (`n_r = 16`, 400 fs, 0.1–4 µm) | `tdse_native`, gated | 17 | 9,216 | 2,305 | 1.9 ms | 1.9 ms | + | production-sized | `optmat_cubic` | 17 | 9,216 | 2,305 | 1.3 ms | 1.2 ms | + + Apple M5 Max, Python 3.14, the machine shared (load average about 10), so the medians + are upper estimates. §6.8 expected 10–100 ms; the measured assembly is one to two + milliseconds at a production-sized lattice, below $10^{-5}$ of the fastest recorded + atomic call (93 s). +- **Deviations.** The SEP's status (`In progress`) and Implementation field are not yet + set, because the SEP's file is not on this branch (it is on the proposal's branch, + #765); that commit follows when the file arrives. This draft is named + `draft-sep-0049-part2.md`, not `sep-0049-part2-draft.md`: `tests/test_seps.py` + collects every `sep-*.md` file in `docs/dev/seps/` as a SEP and checks its title and + header. +- **Handoff.** Phase 1 moves the assembly; the same script then times it again. From a0a2f7b19bffe9143835eb816441ba1b8135b9ca Mon Sep 17 00:00:00 2001 From: John Travers Date: Thu, 12 Nov 2026 13:09:11 +0000 Subject: [PATCH 02/20] SEP 49 phase 1: move the TDSE source assembly to backends/tdse_source The synthesis to the nodes, the assembly of P, g S and g' S and their projection move out of numpy_ref into a module that imports neither backend; numpy_ref calls it. New tests state the taper scaling, node synthesis, edge-array scaling, off-band zero and the diagnostics' floors independently of the module. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HAYtF5P9QqEd3HQx3WJkA3 --- docs/dev/architecture.md | 8 +- docs/dev/figures/build.py | 2 +- docs/dev/figures/layers.svg | 11 +- src/solprop/backends/numpy_ref/backend.py | 28 +- .../{numpy_ref/tdse.py => tdse_source.py} | 152 +++++++-- tests/backends/test_tdse_edge_free.py | 2 +- tests/backends/test_tdse_projection.py | 2 +- tests/backends/test_tdse_source.py | 308 ++++++++++++++++++ tests/backends/test_tdse_sources.py | 2 +- tests/edge_source_cases.py | 8 +- 10 files changed, 463 insertions(+), 60 deletions(-) rename src/solprop/backends/{numpy_ref/tdse.py => tdse_source.py} (86%) create mode 100644 tests/backends/test_tdse_source.py diff --git a/docs/dev/architecture.md b/docs/dev/architecture.md index bc4929c0b..4f2c171b0 100644 --- a/docs/dev/architecture.md +++ b/docs/dev/architecture.md @@ -117,8 +117,10 @@ binding from its step record. It changes no numerical step and adds no call; the optical file records the archive's run UUID and relative path. Plan lifetime, allocation admission and attempted-call budgets -belong to the driver/binding; modal synthesis, P/J projection and spectral -assembly belong to the reference backend. See the [TDSE manual](../guide/media/tdse.md). +belong to the driver/binding; modal synthesis to the nodes, the assembly of the +macroscopic sources and their projection belong to `backends/tdse_source.py`, which +imports neither backend and which a backend calls from its source evaluation +(SEP 49). See the [TDSE manual](../guide/media/tdse.md). Geometry: mode-averaged propagation is the single-node case of multimode propagation (the effective-area normalisation replaces the quadrature); the GNLSE is the mode-averaged @@ -214,7 +216,7 @@ against `src/solprop` by `tests/test_structure_figures.py`. | `modes/` | Mode protocols and derived quantities; capillary, antiresonant, tabulated, step-index, rectangular and planar (slab, uniform in y, per unit width) modes, with `PlanarGrid` for the planar guide resolved in y; shared gas fill, quadrature and collections; full-vector modal records, material laws and regions, bound step-index and radial graded-index providers, and the guide that pairs a provider with a mode collection, and the connection of a basis that moves along z | P | | `linear.py`, `boundaries.py`, `nonlinear/` | Linear assembly, absorbing rates, response definitions, ionisation, Raman, normalisation and the GNLSE mapping | P | | `translate/` | Validated `PropagationProblem` and its route union (`translate/routes.py`); parameter expansions and z tables; modal/radial/Cartesian transforms and the hybrid planar one (slab modes in x × FFT in y, `translate/planar.py`); dense tracked modal tables, the general modal route and the coupled-flow and metric records (`translate/general.py`, `translate/coupled.py`); the spatial plane measure (`translate/flux.py`, the `FluxPlan` of schema 35); the persisted frame integral and the filled input port of a problem's entrance (`translate/coupling.py`); what the general route and the coupled flow execute, checked once (`translate/admission.py`); response tables, masks and stage bundles | T | -| `backends/` | Registry and protocols, sessions, the Adams call ledger both backends keep (`ledger.py`), node chunks, scalar bodies, spatial and population statistics | B | +| `backends/` | Registry and protocols, sessions, the Adams call ledger both backends keep (`ledger.py`), node chunks, scalar bodies, spatial and population statistics, and the TDSE response's host source assembly (`tdse_source.py`) | B | | `backends/numpy_ref/` | Reference execution, its Adams kernels and tables (`multistep.py`) and its on-axis scan (`populations.py`) | B | | `backends/host/` | The reference's own step driver, Adams and low-storage marches, fixed-schedule group, run session and group session ([SEP 47](seps/archive/sep-0047-run-ownership.md) phase 2); the parts the Python sessions compose are in `backends/session_core.py` | B | | `stepper/`, `log.py`, `stats.py` | Tableaux, controllers, schedules, progress reporters and statistics specifications | D | diff --git a/docs/dev/figures/build.py b/docs/dev/figures/build.py index 385dd4c92..3c96a46c4 100644 --- a/docs/dev/figures/build.py +++ b/docs/dev/figures/build.py @@ -710,6 +710,7 @@ class Layer(NamedTuple): "registry", "spatial", "populations", + "tdse_source", "memory", "node_chunk", "ledger", @@ -745,7 +746,6 @@ class Layer(NamedTuple): "populations", "envelope_plasma", "general_modal", - "tdse", ), ), ), diff --git a/docs/dev/figures/layers.svg b/docs/dev/figures/layers.svg index 647233d39..0e30ef1d8 100644 --- a/docs/dev/figures/layers.svg +++ b/docs/dev/figures/layers.svg @@ -180,11 +180,11 @@ text { font-family: Inter, "Helvetica Neue", Helvetica, Arial, sans-serif; fill: backends/ protocol · registry · spatial -populations · memory · node_chunk -ledger · adams_core · rk_core -exponential · low_storage · session -events · export · session_core -sessions +populations · tdse_source · memory +node_chunk · ledger · adams_core +rk_core · exponential · low_storage +session · events · export +session_core · sessions backends/host/ driver · adams · low_storage @@ -194,7 +194,6 @@ text { font-family: Inter, "Helvetica Neue", Helvetica, Arial, sans-serif; fill: backend · batch · passes multistep · populations envelope_plasma · general_modal -tdse D diff --git a/src/solprop/backends/numpy_ref/backend.py b/src/solprop/backends/numpy_ref/backend.py index 6b6d1818a..3847294a8 100644 --- a/src/solprop/backends/numpy_ref/backend.py +++ b/src/solprop/backends/numpy_ref/backend.py @@ -31,6 +31,7 @@ from jaxtyping import Bool, Complex, Float, Inexact, Num from solprop._helpers import PLAIN +from solprop.backends import tdse_source from solprop.backends.exponential import refuse_carrier_frame, refuse_formulation from solprop.backends.host.driver import HostDriver from solprop.backends.host.low_storage import HostLowStorageMarch @@ -50,11 +51,6 @@ general_memory, general_source, ) -from solprop.backends.numpy_ref.tdse import ( - atomic_edge_source, - atomic_sources, - project_atomic_sources, -) from solprop.backends.populations import axis_refusal, population_statistics from solprop.backends.session import ExecutionOptions from solprop.backends.spatial import ( @@ -1688,29 +1684,31 @@ def _nonlinear_tdse( inverse_propagator: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], purpose: TDSEPurpose, ) -> Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]: - """Evaluate the declared atomic binding and combine independent P/J sources.""" + """Evaluate the declared atomic binding and combine independent P/J sources. + + Steps 1 to 4 are :mod:`solprop.backends.tdse_source`'s, shared with native; + the stage output is this backend's own. + """ problem = self._problem - grids, transform = problem.grids, problem.transform + grids = problem.grids binding = self._tdse_binding assert binding is not None and tdse_of(problem.route) is not None modal_t = passes.to_time(lab[0], grids.crop_index, grids.n_to, kind="field") - fields = np.ascontiguousarray( - passes.synthesise(modal_t, transform.synthesis, bundle.taper_scale, 1)[ - :, 0, : - ] - ) + fields = tdse_source.node_fields(problem, lab[0], bundle) result = binding.evaluate( fields, z=bundle.z1, purpose=purpose, density=bundle.density ) - polarisation, current = atomic_sources(fields, result, problem, bundle) - source, projection = project_atomic_sources( + polarisation, current = tdse_source.atomic_sources( + fields, result, problem, bundle + ) + source, projection = tdse_source.project_atomic_sources( polarisation, current, result, problem, bundle, self._prefactor(bundle), - edge=atomic_edge_source(result, problem, bundle), + edge=tdse_source.atomic_edge_source(result, problem, bundle), ) self._tdse_last_purpose, self._tdse_last_z = purpose, bundle.z1 self._tdse_last_projection = projection diff --git a/src/solprop/backends/numpy_ref/tdse.py b/src/solprop/backends/tdse_source.py similarity index 86% rename from src/solprop/backends/numpy_ref/tdse.py rename to src/solprop/backends/tdse_source.py index 50290d68b..c860b08c8 100644 --- a/src/solprop/backends/numpy_ref/tdse.py +++ b/src/solprop/backends/tdse_source.py @@ -2,24 +2,37 @@ # # SPDX-License-Identifier: GPL-3.0-or-later -"""Assemble independently projected TDSE polarisation and current sources. - -The native closure's gated, edge-free source (SEP 26) multiplies the remainder current -``S`` by its gate ``g`` before the transform and subtracts the edge term -``DFT[g' S] / (i omega)`` on the active bins, so that the current is held after the -window instead of switched off. +"""Assemble the TDSE response's macroscopic source at one evaluation (SEP 49). + +The host source both backends call: from the lab-frame modal field to the nodal fields +the atomic binding takes (:func:`node_fields`), from the binding's unscaled microscopic +histories to the macroscopic polarisation, current and edge arrays +(:func:`atomic_sources`, :func:`atomic_edge_source`), and from those to the projected +spectral source with its diagnostics (:func:`project_atomic_sources`). Each backend then +applies the spectral collar, its in-stage absorber term and, in the interaction +formulation, its inverse propagator, as for every source. It imports nothing from +either backend. + +The gated, edge-free source (SEP 26) multiplies the remainder current ``S`` by its gate +``g`` before the transform and subtracts the edge term ``DFT[g' S] / (i omega)`` on the +active bins, so that the current is held after the window instead of switched off. + +The TDSE route is one scalar mode on a field grid (``PropagationProblem._check_tdse``), +so every transform here is the real-to-complex one and the polarisation axis has length +one. """ import math from dataclasses import dataclass import numpy as np -from jaxtyping import Bool, Complex, Float +from jaxtyping import Bool, Complex, Float, Int -from solprop.backends.numpy_ref import passes +from solprop import fourier from solprop.exceptions import BackendError from solprop.translate.linop import power_weight_at from solprop.translate.problem import PropagationProblem +from solprop.translate.responses import ResponseStage from solprop.translate.routes import tdse_descriptor, tdse_of from solprop.translate.stage import StageScalars from solprop.translate.tdse import TDSEResult, lattice_times, waveform_identity @@ -29,6 +42,7 @@ "FFT_ERROR_CONSTANT", "atomic_edge_source", "atomic_sources", + "node_fields", "project_atomic_sources", ] @@ -46,6 +60,97 @@ """ +def node_fields( + problem: PropagationProblem, + lab: Complex[np.ndarray, "n_modes n_pol n_omega"], + bundle: StageScalars, +) -> Float[np.ndarray, "n_nodes n_to"]: + """Synthesise the real field at every radial node, as the atomic binding takes it. + + Step 1 of an evaluation: the lab-frame modal spectrum, its input noise already + added, is placed on the fine grid's bins with no scale (the pad scale is inside + ``transform.synthesis``), transformed to time and synthesised on the nodes with the + taper scale at the evaluation's position (``kernel-contract.md`` sections 4 and 5). + + Parameters + ---------- + problem : PropagationProblem + A TDSE problem: one scalar mode on a field grid. + lab : numpy.ndarray + Lab-frame modal spectrum in the square root of watts, shape + ``(n_modes, 1, n_omega)``. + bundle : StageScalars + Scalars at the evaluation's position; its ``taper_scale`` is read. + + Returns + ------- + numpy.ndarray + C-contiguous float64 fields in V/m, shape ``(n_nodes, n_to)``. + + Raises + ------ + BackendError + If the spectrum has more than one polarisation. + """ + grids = problem.grids + n_modes = lab.shape[0] + if lab.shape[1] != 1: + raise BackendError("the TDSE source takes one polarisation") + fine = np.zeros((n_modes, fourier.n_omega_for(grids.n_to, "field")), np.complex128) + fine[:, grids.crop_index] = lab.reshape(n_modes, -1) + modal_t = fourier.inverse(fine, kind="field") + scaled = problem.transform.synthesis * bundle.taper_scale + return np.ascontiguousarray(np.tensordot(scaled, modal_t, axes=([1], [0]))) + + +def _project( + values: Float[np.ndarray, "n_nodes 1 n_to"], + projection: Float[np.ndarray, "n_modes n_nodes"], + taper: float, +) -> Float[np.ndarray, "n_modes 1 n_to"]: + """Project nodal values onto the modes, dividing by the taper scale.""" + return np.tensordot(projection / taper, values, axes=([1], [0])) + + +def _to_frequency( + values: Float[np.ndarray, "n_modes 1 n_to"], crop_index: Int[np.ndarray, " n_omega"] +) -> Complex[np.ndarray, "n_modes 1 n_omega"]: + """Transform modal values to frequency and gather the coarse bins, unscaled. + + The crop scale is inside the prefactor and the current numerator. + """ + n_modes = values.shape[0] + spectrum = fourier.forward(values.reshape(n_modes, -1), kind="field") + return spectrum[..., crop_index].reshape(n_modes, 1, -1) + + +def _cubic_polarisation( + fields: Float[np.ndarray, "n_nodes n_to"], + problem: PropagationProblem, + bundle: StageScalars, +) -> Float[np.ndarray, "n_nodes 1 n_to"]: + """Return the compiled Kerr polarisation that replaces the atom's third order. + + ``P = eps0 chi3 rho E^3 w(t)`` in C/m², with the temporal collar ``w``: the real + cubic of a field grid, the only term the TDSE route admits beside the atom. + """ + stages = set(problem.responses.response_order) - {ResponseStage.ACCUMULATE} + kerr = problem.responses.kerr_coefficients + values = fields[:, None, :] + if not stages: + # Nothing but the accumulation compiled: an identically zero polarisation. + return np.zeros_like(values) + if ( + kerr is None + or kerr.cubic_form != "real" + or kerr.analytic_signal + or stages != {ResponseStage.KERR} + ): + raise BackendError("the cubic replacement needs the real Kerr term alone") + polarisation = kerr.coefficient * bundle.kerr_density * values**2 * values + return polarisation * problem.grids.t_window_fine + + def atomic_sources( fields: Float[np.ndarray, "n_nodes n_to"], result: TDSEResult, @@ -112,17 +217,7 @@ def atomic_sources( * result.third_order_dipole[:, None, :] ) elif descriptor.kerr_model == "optmat_cubic": - polarisation = passes.nonlinear_polarisation( - fields[:, None, :], - problem.responses, - problem.grids.t_window_fine, - kind="field", - density=bundle.density, - preionisation=0.0, - dt=problem.grids.dt_fine, - carrier=None, - scalars=bundle, - ) + polarisation = _cubic_polarisation(fields, problem, bundle) else: polarisation = np.zeros_like(current) # The existing in-stage field absorber remains separate. An explicit source @@ -206,8 +301,8 @@ def _project_spectrum( - ``values``: nodal P or J source, shape ``(n_nodes, 1, n_to)``, in C/m² or A/m². - Returns: raw FFT source, shape ``(n_modes, 1, n_omega)`` before source factors. """ - projected = passes.project(values, projection, bundle.taper_scale, 1) - return passes.to_frequency(projected, problem.grids.crop_index, kind="field") + projected = _project(values, projection, bundle.taper_scale) + return _to_frequency(projected, problem.grids.crop_index) def _roundoff_floor( @@ -245,8 +340,8 @@ def _roundoff_terms( Both feed :func:`_roundoff_floor`; the accumulation is ``operations * eps``. """ - absolute_projected = passes.project( - magnitudes, np.abs(projection), abs(bundle.taper_scale), 1 + absolute_projected = _project( + magnitudes, np.abs(projection), abs(bundle.taper_scale) ) # A Fourier coefficient is bounded by the temporal L1 sum. Keep the raw # operands in this bound: using the already cancelled remainder would hide @@ -425,10 +520,10 @@ def _edge_rounding( # noqa: PLR0917 - called positionally weights = np.abs(projection) taper = abs(bundle.taper_scale) operands = np.linalg.norm( - passes.project(edge_magnitude, weights, taper, 1), axis=-1, keepdims=True + _project(edge_magnitude, weights, taper), axis=-1, keepdims=True ) evaluation = np.linalg.norm( - passes.project(derivative_magnitude, weights, taper, 1), + _project(derivative_magnitude, weights, taper), axis=-1, keepdims=True, ) @@ -584,11 +679,8 @@ def _leading_taper( else: ratio, reason = lead_peak / peak, None scaled = abs(bundle.density * descriptor.remainder_scale) * remainder[:, None, lead] - projected = passes.project( - scaled, - np.abs(problem.transform.projection), - abs(bundle.taper_scale), - 1, + projected = _project( + scaled, np.abs(problem.transform.projection), abs(bundle.taper_scale) ) largest = np.max(projected, axis=-1, keepdims=True, initial=0.0) return { diff --git a/tests/backends/test_tdse_edge_free.py b/tests/backends/test_tdse_edge_free.py index b204cf1bb..df0772ea5 100644 --- a/tests/backends/test_tdse_edge_free.py +++ b/tests/backends/test_tdse_edge_free.py @@ -35,7 +35,7 @@ from tests.hacc_cases import plain from solprop.backends.numpy_ref import NumpyRefBackend -from solprop.backends.numpy_ref.tdse import ( +from solprop.backends.tdse_source import ( FFT_ERROR_CONSTANT, atomic_edge_source, project_atomic_sources, diff --git a/tests/backends/test_tdse_projection.py b/tests/backends/test_tdse_projection.py index 088e3c3a1..eed44dd45 100644 --- a/tests/backends/test_tdse_projection.py +++ b/tests/backends/test_tdse_projection.py @@ -12,7 +12,7 @@ from numpy.testing import assert_allclose, assert_array_equal from solprop.backends.numpy_ref import passes -from solprop.backends.numpy_ref.tdse import ( +from solprop.backends.tdse_source import ( atomic_edge_source, atomic_sources, project_atomic_sources, diff --git a/tests/backends/test_tdse_source.py b/tests/backends/test_tdse_source.py new file mode 100644 index 000000000..7a8657122 --- /dev/null +++ b/tests/backends/test_tdse_source.py @@ -0,0 +1,308 @@ +# SPDX-FileCopyrightText: 2026 John C. Travers and contributors +# +# SPDX-License-Identifier: GPL-3.0-or-later + +"""State the shared TDSE source's terms independently of the module (SEP 49, 7.1). + +Both backends call :mod:`solprop.backends.tdse_source`, so their agreement cannot see an +error inside it. Each test here writes its expectation from the kernel contract or the +documented bound with NumPy alone: the nodal synthesis and its taper, the projection's +division by the taper, the edge array's scaling, the zero source off the active band +and the diagnostics' rounding floors. The signs of ``S = J - j_1`` and of the cubic +replacement, the density and the gate are stated in ``test_tdse_sources.py``, and the +edge term and crop in ``test_tdse_edge_free.py`` and ``test_tdse_projection.py``. +""" + +import dataclasses +import math +import subprocess +import sys +from typing import Any + +import numpy as np +import pytest + +from edge_source_cases import EPS, EdgeCase, make_case +from solprop.backends.tdse_source import ( + FFT_ERROR_CONSTANT, + atomic_edge_source, + atomic_sources, + node_fields, + project_atomic_sources, +) +from solprop.exceptions import BackendError +from solprop.nonlinear.responses import KerrResponse +from solprop.translate.linop import power_weight_at +from solprop.translate.routes import tdse_of +from solprop.translate.tdse import TDSEResult, lattice_times, waveform_identity + + +def _result(case: EdgeCase, fields: np.ndarray, *, cubic: bool) -> TDSEResult: + """Return analytic histories whose remainder and cubic shadow are both nonzero.""" + times = lattice_times(case.grid.n_to, case.grid.dt_fine) + envelope = np.exp(-((times / 8e-15) ** 2)) + remainder = 1e-27 * envelope * np.sin(2.4e15 * times) + first = 1e-30 * fields + third = 1e-40 * fields**3 if cubic else None + total = first + remainder + (0.0 if third is None else third) + return TDSEResult( + total, + first, + input_identity=waveform_identity(fields), + configuration_identity=case.descriptor.identity, + valid=np.ones(fields.shape, dtype=bool), + shadow_orders=(1, 3) if cubic else (1,), + third_order_current=third, + ) + + +def _fields(case: EdgeCase) -> np.ndarray: + """Return a 2 fs-scale pulse of 1e9 V/m on every node.""" + times = lattice_times(case.grid.n_to, case.grid.dt_fine) + pulse = 1e9 * np.exp(-((times / 6e-15) ** 2)) * np.cos(2.35e15 * times) + rows = case.problem.transform.n_nodes + return np.ascontiguousarray(np.broadcast_to(pulse, (rows, case.grid.n_to))) + + +def _evaluate(case: EdgeCase, bundle=None) -> Any: + """Return the projected source and its diagnostics, with the inputs used.""" + bundle = case.bundle if bundle is None else bundle + fields = _fields(case) + result = _result(case, fields, cubic=not case.descriptor.gated) + polarisation, current = atomic_sources(fields, result, case.problem, bundle) + edge = atomic_edge_source(result, case.problem, bundle) + source, diagnostics = project_atomic_sources( + polarisation, current, result, case.problem, bundle, case.prefactor, edge=edge + ) + return source, diagnostics, result, (polarisation, current, edge) + + +def _cubic_case() -> EdgeCase: + """Return a cubic replacement whose compiled Kerr term gives a nonzero P.""" + return make_case(kerr_model="optmat_cubic", responses=(KerrResponse(chi3=1e-25),)) + + +@pytest.fixture(scope="module", params=["tdse_native", "optmat_cubic"]) +def case(request): + return make_case() if request.param == "tdse_native" else _cubic_case() + + +def test_node_fields_are_the_taper_times_the_synthesis_of_each_bin(): + """One coarse bin ``a`` at fine bin ``c`` is ``(2/N) Re(a exp(2 pi i c n / N))``.""" + case = make_case() + grids, transform = case.problem.grids, case.problem.transform + bundle = dataclasses.replace(case.bundle, taper_scale=1.7) + bin_index, amplitude = 17, 3.0 - 2.0j + lab = np.zeros((1, 1, grids.n_omega), dtype=np.complex128) + lab[0, 0, bin_index] = amplitude + samples = grids.n_to + phase = 2 * np.pi * grids.crop_index[bin_index] * np.arange(samples) / samples + modal = 2 / samples * np.real(amplitude * np.exp(1j * phase)) + expected = 1.7 * transform.synthesis[:, :1] * modal + fields = node_fields(case.problem, lab, bundle) + assert fields.shape == (transform.n_nodes, samples) + assert fields.flags.c_contiguous + # Normwise inverse-FFT bound (Higham, Theorem 24.2) plus the two products and the + # oracle's own cosine: (c_F log2 N + 4) eps of the norm. + budget = (FFT_ERROR_CONSTANT * math.ceil(math.log2(samples)) + 4) * EPS + error = np.linalg.norm(fields - expected) + assert budget > 0 + assert error <= budget * np.linalg.norm(expected) + + +def test_node_fields_refuse_two_polarisations(): + case = make_case() + lab = np.zeros((1, 2, case.problem.grids.n_omega), dtype=np.complex128) + with pytest.raises(BackendError, match="one polarisation"): + node_fields(case.problem, lab, case.bundle) + + +def test_the_projection_divides_by_the_taper_scale(case): + """The projection is ``W / tau`` (kernel contract section 5). + + Halving through ``tau = 2`` is exact in binary64 -- a power-of-two scaling commutes + with every rounding of the products, sums and transform -- so the identity holds + bit for bit. + """ + source, *_ = _evaluate(case) + doubled = dataclasses.replace(case.bundle, taper_scale=2.0) + halved, *_ = _evaluate(case, doubled) + assert np.any(source != 0) + np.testing.assert_array_equal(2 * halved, source) + + +def test_the_edge_array_is_density_scale_and_gate_derivative_of_the_remainder(): + """``E = rho s g'(t) (J - j_1)``, the density and scale entering once.""" + case = make_case() + bundle = dataclasses.replace(case.bundle, density=3.0) + fields = _fields(case) + result = _result(case, fields, cubic=False) + edge = atomic_edge_source(result, case.problem, bundle) + assert edge is not None + descriptor = case.descriptor + remainder = result.total_current - result.first_order_current + expected = ( + descriptor.source_gate_derivative() + * remainder + * (descriptor.remainder_scale * 3.0) + ) + # Three products, associated differently in the oracle: gamma_3 of each value. + np.testing.assert_allclose(edge[:, 0, :], expected, rtol=4 * EPS, atol=0) + assert np.any(expected != 0) + + +def test_the_source_is_exactly_zero_off_the_active_band(case): + """Both spectral factors vanish off the active band by construction, so must F.""" + source, *_ = _evaluate(case) + active = case.problem.grids.active_mask + assert np.all(np.isfinite(source)) + assert np.all(source[..., ~active] == 0) + assert np.any(source[..., active] != 0) + + +def _absolute_projection(values: np.ndarray, projection: np.ndarray, taper: float): + """Return ``sum_n |W_mn| values_n / |tau|`` per mode and sample.""" + return np.einsum("mn,nt->mt", np.abs(projection), values[:, 0, :]) / abs(taper) + + +def _floor_terms(case: EdgeCase, magnitudes: np.ndarray, factor, projection): + """Return the documented operand bound ``|factor w| sum_t |W| x / |tau|``.""" + grids = case.problem.grids + spectral = _absolute_projection(magnitudes, projection, case.bundle.taper_scale) + return np.abs(factor * grids.omega_window) * spectral.sum(axis=-1)[:, None, None] + + +def _accumulation(case: EdgeCase) -> float: + """``(8 + 2 n_nodes + 5 ceil(log2 N) [+1 for a gate]) eps`` of the floor.""" + operations = 8 + 2 * case.problem.transform.n_nodes + operations += 5 * math.ceil(math.log2(case.problem.grids.n_to)) + return (operations + (1 if case.descriptor.gated else 0)) * EPS + + +def test_the_complete_floor_is_the_documented_operand_bound(): + """The ungated (cubic) source's floors, from the absolute raw operands. + + Each floor is ``|| |c w| sum_t |W| |rho s| (|J| + |j_1| + |j_3|) / |tau| ||`` + times ``a / (1 - a)``, for the current and likewise ``|p w|`` and ``|P|`` for the + polarisation, over the fine and the embedded maps. + """ + case = _cubic_case() + _, diagnostics, result, (polarisation, _, _) = _evaluate(case) + transform, bundle = case.problem.transform, case.bundle + source = tdse_of(case.problem.route) + assert source is not None + factor = source.current_numerator / power_weight_at(case.problem, bundle.z1) + operands = ( + np.abs(result.total_current) + + np.abs(result.first_order_current) + + np.abs(result.third_order_current) + ) + scale = abs(bundle.density * case.descriptor.remainder_scale) + current = scale * operands[:, None, :] + accumulation = _accumulation(case) + ratio = accumulation / (1 - accumulation) + floors = {"current": 0.0, "polarisation": 0.0} + maps = [transform.projection] + if not np.array_equal(transform.projection, transform.projection_coarse): + # The embedded map adds its own floors only when it is a distinct rule. + maps.append(transform.projection_coarse) + for projection in maps: + floors["current"] += ratio * float( + np.linalg.norm(_floor_terms(case, current, factor, projection)) + ) + floors["polarisation"] += ratio * float( + np.linalg.norm( + _floor_terms(case, np.abs(polarisation), case.prefactor, projection) + ) + ) + # The bound is itself evaluated in floating point, with sums of a few thousand + # terms reassociated here: 1e-12 relative covers that and nothing else. + assert diagnostics["remainder_source"]["roundoff_floor"] == pytest.approx( + floors["current"], rel=1e-12, abs=0 + ) + assert diagnostics["complete_source"]["roundoff_floor"] == pytest.approx( + floors["current"] + floors["polarisation"], rel=1e-12, abs=0 + ) + assert floors["polarisation"] > 0 + + +def test_the_edge_floors_are_the_documented_section_4_5_bounds(): + """A gated source's band floors (SEP 26 section 4.5), written from the docstring. + + Per bin, with ``c_k = |factor_k w_k|``: the truncating part ``c_k sum |W| |rho s| + |g| (|J| + |j_1|) a / (1 - a)``; the edge transform ``c_k sqrt(N) (c_F eps log2 N + + (n_nodes + 6) u) ||y||_2 / omega_lo`` with ``y`` the absolute projection of + ``|rho s| |g'| (|J| + |j_1|)``, plus ``c_k sqrt(N) ||y_d||_2 / omega_lo`` for the + derivative's own error; and the division ``2 eps c_k (|X_k| + |E_k| / omega_k)``. + """ + case = make_case() + _, diagnostics, result, (_, current, edge) = _evaluate(case) + assert edge is not None + problem, bundle, descriptor = case.problem, case.bundle, case.descriptor + grids, projection = problem.grids, problem.transform.projection + source = tdse_of(problem.route) + assert source is not None + factor = source.current_numerator / power_weight_at(problem, bundle.z1) + collar = np.abs(factor * grids.omega_window) + raw = np.abs(result.total_current) + np.abs(result.first_order_current) + scale = abs(bundle.density * descriptor.remainder_scale) + gate = grids.t_window_fine if descriptor.source_gate() is None else None + assert gate is not None + accumulation = _accumulation(case) + truncating = _floor_terms( + case, scale * (raw * gate)[:, None, :], factor, projection + ) + truncating = truncating * accumulation / (1 - accumulation) + samples, nodes = grids.n_to, problem.transform.n_nodes + lowest = float(np.min(grids.omega[grids.active_mask])) + transform = FFT_ERROR_CONSTANT * EPS * math.ceil(math.log2(samples)) + ( + nodes + 6 + ) * (EPS / 2) + y = _absolute_projection( + scale * (raw * np.abs(descriptor.source_gate_derivative()))[:, None, :], + projection, + bundle.taper_scale, + ) + y_d = _absolute_projection( + scale * (raw * descriptor.source_gate_derivative_error())[:, None, :], + projection, + bundle.taper_scale, + ) + norms = transform * np.linalg.norm(y, axis=-1) + np.linalg.norm(y_d, axis=-1) + edge_bins = collar * math.sqrt(samples) * norms[:, None, None] / lowest + + def spectrum(values): + projected = np.einsum("mn,nt->mt", projection, values[:, 0, :]) + return np.fft.rfft(projected / bundle.taper_scale)[:, None, grids.crop_index] + + inverse = np.zeros(grids.n_omega) + inverse[grids.active_mask] = 1 / grids.omega[grids.active_mask] + division = ( + 2 + * EPS + * collar + * (np.abs(spectrum(current)) + np.abs(spectrum(edge)) * inverse) + ) + band = diagnostics["edge"]["bands"]["active"] + active = grids.active_mask + # As above: the bounds are evaluated in floating point and reassociated here. + assert band["truncating_floor"] == pytest.approx( + np.linalg.norm(truncating[..., active]), rel=1e-12, abs=0 + ) + assert band["edge_floor"] == pytest.approx( + np.linalg.norm(edge_bins[..., active]) + np.linalg.norm(division[..., active]), + rel=1e-12, + abs=0, + ) + assert np.linalg.norm(division[..., active]) > 0 + + +def test_the_shared_source_imports_neither_backend(): + """Native calls this module and never imports the reference (SEP 49, 6.1).""" + probe = ( + "import sys, solprop.backends.tdse_source; " + "loaded = [name for name in sys.modules if 'numpy_ref' in name " + "or name.startswith('solprop_native')]; " + "assert not loaded, loaded" + ) + subprocess.run([sys.executable, "-c", probe], check=True, timeout=120) diff --git a/tests/backends/test_tdse_sources.py b/tests/backends/test_tdse_sources.py index 1d12b4764..797919d9f 100644 --- a/tests/backends/test_tdse_sources.py +++ b/tests/backends/test_tdse_sources.py @@ -16,7 +16,7 @@ from numpy.testing import assert_allclose, assert_array_equal from tests.hacc_cases import hacc_response -from solprop.backends.numpy_ref.tdse import atomic_sources +from solprop.backends.tdse_source import atomic_sources from solprop.constants import EPS0 from solprop.exceptions import BackendError from solprop.grid import FieldGrid diff --git a/tests/edge_source_cases.py b/tests/edge_source_cases.py index 0459f7bbd..e8cd7c7c6 100644 --- a/tests/edge_source_cases.py +++ b/tests/edge_source_cases.py @@ -19,7 +19,7 @@ import numpy as np from solprop.backends.numpy_ref import passes -from solprop.backends.numpy_ref.tdse import ( +from solprop.backends.tdse_source import ( FFT_ERROR_CONSTANT, _current_operands, _current_part, @@ -134,6 +134,7 @@ def make_case( fft_size: str = "pow2", n_r: int = 4, kerr_model: str = "tdse_native", + responses: tuple = (), ) -> EdgeCase: """Assemble a capillary problem and attach a manufactured SAE descriptor. @@ -154,6 +155,9 @@ def make_case( kerr_model : str ``tdse_native`` for the gated source, or a cubic replacement, whose source is ungated. + responses : tuple + Optical responses compiled beside the atom: a Kerr term for a cubic + replacement's polarisation; none by default. Returns ------- @@ -166,7 +170,7 @@ def make_case( problem = assemble( grid, mode, - (), + responses, np.zeros((1, 1, grid.n_omega), dtype=np.complex128), controller=PIController(rtol=1e-2, atol=1e-14), save_planes=[0.001], From 0c5a15ee5705845e804ec7215622aceb5b721f01 Mon Sep 17 00:00:00 2001 From: John Travers Date: Thu, 12 Nov 2026 13:10:35 +0000 Subject: [PATCH 03/20] SEP 49: set In progress and record phases 0 and 1 in Part II Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HAYtF5P9QqEd3HQx3WJkA3 --- docs/dev/architecture.md | 2 +- docs/dev/seps/draft-sep-0049-part2.md | 55 --------- docs/dev/seps/index.md | 2 +- docs/dev/seps/sep-0049-tdse-on-native.md | 141 ++++++++++++++++++++++- 4 files changed, 141 insertions(+), 59 deletions(-) delete mode 100644 docs/dev/seps/draft-sep-0049-part2.md diff --git a/docs/dev/architecture.md b/docs/dev/architecture.md index 4f2c171b0..50ba57ea0 100644 --- a/docs/dev/architecture.md +++ b/docs/dev/architecture.md @@ -120,7 +120,7 @@ Plan lifetime, allocation admission and attempted-call budgets belong to the driver/binding; modal synthesis to the nodes, the assembly of the macroscopic sources and their projection belong to `backends/tdse_source.py`, which imports neither backend and which a backend calls from its source evaluation -(SEP 49). See the [TDSE manual](../guide/media/tdse.md). +([SEP 49](seps/sep-0049-tdse-on-native.md)). See the [TDSE manual](../guide/media/tdse.md). Geometry: mode-averaged propagation is the single-node case of multimode propagation (the effective-area normalisation replaces the quadrature); the GNLSE is the mode-averaged diff --git a/docs/dev/seps/draft-sep-0049-part2.md b/docs/dev/seps/draft-sep-0049-part2.md deleted file mode 100644 index 9af8399f2..000000000 --- a/docs/dev/seps/draft-sep-0049-part2.md +++ /dev/null @@ -1,55 +0,0 @@ ---- -orphan: true ---- - -# SEP 49, Part II draft - -The phase reports of SEP 49's implementation, kept here until the SEP's own file reaches -this branch; they then move, unchanged, into its Part II, and this file goes. - -## Phase 0 — Baseline - -- **Scope.** Phase 0, 2026-11-12, Opus 5.5 (AI), branch - `feature/sep-0049-tdse-on-native`, based on `feature/two-backends-binary64` at - `1dc64b5f` (ADR-077's change). -- **Delivered.** This record; no code. -- **Size baseline** (`uv run poe size --all` at `1dc64b5f`, lines): - - | tree | files | total | code | - |---|---:|---:|---:| - | `src/solprop` | 269 | 225,295 | 113,232 | - | `native/src/solprop_native` (Python) | 24 | 13,859 | 8,855 | - | `tests/` (all `.py`, `wc -l`) | — | 271,204 | — | - | `native/tests/` (all `.py`, `wc -l`) | — | 13,602 | — | - | `native/src` C++ (`.cpp`, `.hpp`, `.h`, `wc -l`) | — | 34,575 | — | - - `poe size-check` is within budget. `src/solprop/backends/numpy_ref/tdse.py` is 845 - lines; `src/solprop/driver/tdse.py` 2,575. -- **The host assembly's time per evaluation.** Measured by a one-off script, not - committed: the `march_problem` problem of `tests/backends/test_tdse_march.py` - (256 µm helium capillary, 800 nm, one mode), the interaction formulation, a - manufactured binding returning analytic histories (`J = 1e-30 E + 1e-40 E^3`, - `j_1 = 1e-30 E`, and a third-order current for the cubic replacement), and - `NumpyRefBackend._nonlinear(..., purpose="predictor")` timed 60 times, the first ten - discarded, the binding's own time subtracted. That is steps 1 and 3–5 of §3.2: the - synthesis to the nodes, the assembly, the projection with every diagnostic, and the - stage output. - - | lattice | descriptor | `n_nodes` | `n_to` | `n_omega` | median | min | - |---|---|---:|---:|---:|---:|---:| - | the test window (`n_r = 4`, 60 fs, 0.4–1.6 µm) | `tdse_native`, gated | 5 | 384 | 97 | 0.26 ms | 0.25 ms | - | the test window | `optmat_cubic` | 5 | 384 | 97 | 0.13 ms | 0.12 ms | - | production-sized (`n_r = 16`, 400 fs, 0.1–4 µm) | `tdse_native`, gated | 17 | 9,216 | 2,305 | 1.9 ms | 1.9 ms | - | production-sized | `optmat_cubic` | 17 | 9,216 | 2,305 | 1.3 ms | 1.2 ms | - - Apple M5 Max, Python 3.14, the machine shared (load average about 10), so the medians - are upper estimates. §6.8 expected 10–100 ms; the measured assembly is one to two - milliseconds at a production-sized lattice, below $10^{-5}$ of the fastest recorded - atomic call (93 s). -- **Deviations.** The SEP's status (`In progress`) and Implementation field are not yet - set, because the SEP's file is not on this branch (it is on the proposal's branch, - #765); that commit follows when the file arrives. This draft is named - `draft-sep-0049-part2.md`, not `sep-0049-part2-draft.md`: `tests/test_seps.py` - collects every `sep-*.md` file in `docs/dev/seps/` as a SEP and checks its title and - header. -- **Handoff.** Phase 1 moves the assembly; the same script then times it again. diff --git a/docs/dev/seps/index.md b/docs/dev/seps/index.md index 237f5ba26..96eda3347 100644 --- a/docs/dev/seps/index.md +++ b/docs/dev/seps/index.md @@ -33,7 +33,7 @@ Proposals that are active, in draft, accepted, in progress or deferred. | [11](sep-0011-finite-density-gas-index.md) | Finite-density gas index and mixture optics | Draft | Change | | [17](sep-0017-gas-two-photon-ionisation.md) | Broadband two-photon continuum response beside strong-field ionisation | Accepted | Feature | | [32](sep-0032-raman-maxwell-bloch.md) | Raman Maxwell–Bloch dynamics | Draft | Feature | -| [49](sep-0049-tdse-on-native.md) | The TDSE response on native | Accepted | Change | +| [49](sep-0049-tdse-on-native.md) | The TDSE response on native | In progress | Change | SEP 13, the standalone numerical prerequisite for SEP 12, SEP 14, which carried it to the compiled backends, and SEP 16, which unified the multistep family and made it the diff --git a/docs/dev/seps/sep-0049-tdse-on-native.md b/docs/dev/seps/sep-0049-tdse-on-native.md index 1f49977a4..08ba8eea9 100644 --- a/docs/dev/seps/sep-0049-tdse-on-native.md +++ b/docs/dev/seps/sep-0049-tdse-on-native.md @@ -6,7 +6,7 @@ | **Title** | The TDSE response on native | | **Author** | John Travers | | **Drafting agent** | Opus 5.5 (2026-11-12) | -| **Status** | Accepted | +| **Status** | In progress | | **Review status** | Reviewed | | **Reviewers** | GPT-6 (AI); John Travers (human) | | **Type** | Change | @@ -15,7 +15,7 @@ | **Requires** | [SEP 12](archive/sep-0012-tdse-nonlinear-response.md), [SEP 20](archive/sep-0020-hacc-helium-tdse.md) and [SEP 26](archive/sep-0026-edge-free-tdse-source.md) (the TDSE response and its source); [SEP 39](archive/sep-0039-native-adams-march.md) (the native Adams march); [SEP 47](archive/sep-0047-run-ownership.md) (backend-owned sessions); [SEP 25](archive/sep-0025-rounding-level-reproducibility.md) (agreement to the error bound); ADR-077 and the change it calls for (§3.4) | | **Supersedes** | — | | **Superseded-by** | — | -| **Implementation** | Not started | +| **Implementation** | In progress: Opus 5.5 (AI), on `feature/sep-0049-tdse-on-native`; phases 0–2 | | **Manual pages** | — | ## Part I — Proposal @@ -811,6 +811,141 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. ## Part II — Implementation record +### Phase 0 — Baseline + +- **Scope.** Phase 0, 2026-11-12, Opus 5.5 (AI), branch + `feature/sep-0049-tdse-on-native`, based on `feature/two-backends-binary64` at + `1dc64b5f` (ADR-077's change). +- **Delivered.** This record; no code. Commit `d33e8f24`. +- **Size baseline** (`uv run poe size --all` at `1dc64b5f`, lines): + + | tree | files | total | code | + |---|---:|---:|---:| + | `src/solprop` | 269 | 225,295 | 113,232 | + | `native/src/solprop_native` (Python) | 24 | 13,859 | 8,855 | + | `tests/` (all `.py`, `wc -l`) | — | 271,204 | — | + | `native/tests/` (all `.py`, `wc -l`) | — | 13,602 | — | + | `native/src` C++ (`.cpp`, `.hpp`, `.h`, `wc -l`) | — | 34,575 | — | + + `poe size-check` is within budget. `src/solprop/backends/numpy_ref/tdse.py` is 845 + lines; `src/solprop/driver/tdse.py` 2,575. +- **The host assembly's time per evaluation.** Measured by a one-off script, not + committed: the `march_problem` problem of `tests/backends/test_tdse_march.py` + (256 µm helium capillary, 800 nm, one mode), the interaction formulation, a + manufactured binding returning analytic histories (`J = 1e-30 E + 1e-40 E^3`, + `j_1 = 1e-30 E`, and a third-order current for the cubic replacement), and + `NumpyRefBackend._nonlinear(..., purpose="predictor")` timed 60 times, the first ten + discarded, the binding's own time subtracted. That is steps 1 and 3–5 of §3.2: the + synthesis to the nodes, the assembly, the projection with every diagnostic, and the + stage output. + + | lattice | descriptor | `n_nodes` | `n_to` | `n_omega` | median | min | + |---|---|---:|---:|---:|---:|---:| + | the test window (`n_r = 4`, 60 fs, 0.4–1.6 µm) | `tdse_native`, gated | 5 | 384 | 97 | 0.26 ms | 0.25 ms | + | the test window | `optmat_cubic` | 5 | 384 | 97 | 0.13 ms | 0.12 ms | + | production-sized (`n_r = 16`, 400 fs, 0.1–4 µm) | `tdse_native`, gated | 17 | 9,216 | 2,305 | 1.9 ms | 1.9 ms | + | production-sized | `optmat_cubic` | 17 | 9,216 | 2,305 | 1.3 ms | 1.2 ms | + + Apple M5 Max, Python 3.14, the machine shared (load average about 10), so the medians + are upper estimates. §6.8 expected 10–100 ms; the measured assembly is one to two + milliseconds at a production-sized lattice, below $10^{-5}$ of the fastest recorded + atomic call (93 s). +- **Deviations.** None. The record was kept in a draft beside the SEP until the + proposal's pull request brought this file to the branch. +- **Handoff.** Phase 1 moves the assembly; the same script then times it again. + +### Phase 1 — The shared source + +- **Scope.** Phase 1, 2026-11-12, Opus 5.5 (AI), commit `a0a2f7b1`. +- **Delivered.** + - `src/solprop/backends/tdse_source.py`, moved from `backends/numpy_ref/tdse.py` with + `git mv`: `atomic_sources`, `atomic_edge_source`, `project_atomic_sources`, + `FFT_ERROR_CONSTANT` and their private projection, floor and diagnostic helpers, + unchanged, plus `node_fields` (step 1 of §3.2, moved from + `NumpyRefBackend._nonlinear_tdse`). The module imports nothing from either backend: + its four uses of `numpy_ref.passes` (`project`, `to_frequency`, and through + `node_fields` `to_time` and `synthesise`) are private one-line equivalents for the + route's one scalar mode on a field grid, and the cubic replacement's polarisation + is `_cubic_polarisation`, the compiled real Kerr term alone, in the arithmetic and + order of `passes.kerr_polarisation` and `nonlinear_sources`, and refused by name + for any other compiled stage. + - `NumpyRefBackend._nonlinear_tdse` calls `node_fields`, the binding, the two + assembly functions and the projection; the stage output stays its own. + - `tests/backends/test_tdse_source.py` (new, 10 tests) and an optional `responses` + argument of `tests/edge_source_cases.py::make_case`; the four existing modules' + imports re-pointed, their expectations unchanged. + - `docs/dev/architecture.md` (the responsibility and the module table) and the + layers figure (`docs/dev/figures/build.py`, `layers.svg`). +- **Deviations.** + - The module's entry points keep their moved names (`atomic_sources` with + `atomic_edge_source`, and `project_atomic_sources`) rather than §5.2's sketched + `assemble` and `project`, so that the move is pure and the existing tests and their + expectations run unchanged; `node_fields` is new. Each takes the caller's stage + scalars (`StageScalars`: density, Kerr density, taper scale) and the projection the + caller's prefactor, as `numpy_ref` passed them; native's hook will take both from the + same translation helpers. + - `numpy_ref` now transforms the modal field to time twice per TDSE evaluation: once + in `node_fields`, once for its own in-stage absorber term. Re-timed with phase 0's + script: 0.26 ms (test window, gated), 1.97 ms (production-sized, gated), 1.22 ms + (production-sized, cubic), no measurable change. +- **Decisions.** A problem whose cubic replacement has no compiled Kerr term (only the + accumulation stage) keeps a zero polarisation, as `passes.nonlinear_polarisation` + gave it; manufactured test problems are built that way. +- **Verification** (§7.1). `tests/backends/test_tdse_source.py`, + `test_tdse_sources.py`, `test_tdse_projection.py`, `test_tdse_edge_free.py`, + `test_tdse_march.py`, `tests/test_structure_figures.py`, + `tests/driver/test_tdse_gated_derivative_run.py` and `test_tdse_input_noise.py`: + 485 passed; ruff and basedpyright clean on the changed files; `poe size-check` within + budget. The new tests state their expectations with NumPy alone: the nodal field of + one coarse bin, `(2/N) Re(a e^{2 pi i c n / N})` times the synthesis and the taper, + within `(c_F log2 N + 4) eps` of its norm; the projection's division by the taper + (`tau = 2` is exact, so bit for bit); the edge array `rho s g' (J - j_1)` within + `gamma_3`; a source exactly zero off the active band; the complete and remainder + floors, and a gated source's truncating and edge floors, from the formulas of the + docstrings and SEP 26 §4.5, within `1e-12` relative (reassociated sums); and an + import of the module in a fresh interpreter loading neither backend. + + Each named term was then broken, one at a time, in the module, and the five modules + above run (a one-off script, not committed): + + | defect | first failing test | + |---|---| + | `S = J + j_1` | `test_tdse_sources.py::test_each_closure_has_its_own_cubic_and_remainder_definition` | + | cubic replacement added, not subtracted | the same | + | density dropped from the current | the same | + | density dropped from the shadow polarisation | `test_tdse_sources.py::test_cubic_and_remainder_calibrations_are_independent` | + | Kerr density dropped from the cubic polarisation | `test_tdse_sources.py::test_each_closure_has_its_own_cubic_and_remainder_definition` | + | temporal collar dropped from the cubic polarisation | the same | + | projection multiplied by the taper | `test_tdse_source.py::test_the_projection_divides_by_the_taper_scale` | + | taper dropped from the node synthesis | `test_tdse_source.py::test_node_fields_are_the_taper_times_the_synthesis_of_each_bin` | + | node synthesis crop off by one bin | the same | + | gate dropped | `test_tdse_sources.py::test_native_scale_preserves_complete_nonlinearity_after_linear_subtraction` | + | density dropped from the edge array | `test_tdse_source.py::test_the_edge_array_is_density_scale_and_gate_derivative_of_the_remainder` | + | `X + E/(i omega)` | `test_tdse_edge_free.py::test_t2a_a_globally_steady_current_cancels_to_the_budget_and_certificate` | + | `1/(i omega)` of the wrong sign | the same | + | `1/(i omega)` formed on every bin, DC included | `test_tdse_projection.py::test_periodic_dipole_and_its_current_give_the_same_source` | + | spectral crop off by one bin | the same | + | truncating floor halved | `test_tdse_source.py::test_the_complete_floor_is_the_documented_operand_bound` | + | cancelled operand dropped from the floor | `test_tdse_projection.py::test_cubic_roundoff_floor_counts_the_unscaled_third_order_operand` | + | third-order operand dropped from the floor | the same | + | edge transform's constant halved | `test_tdse_source.py::test_the_edge_floors_are_the_documented_section_4_5_bounds` | + | derivative-error term dropped | the same | + | division bound dropped | the same | + | ratio reported below its floor | `test_tdse_projection.py::test_periodic_dipole_and_its_current_give_the_same_source` | + + One variant is invisible and was not made visible: forming `1/(i omega)` on every bin + but DC. The current factor and the prefactor are exactly zero off the active band by + construction (`translate.tdse_source.current_numerator`, the prefactor), so the mask + is defence in depth; `test_the_source_is_exactly_zero_off_the_active_band` holds the + consequence. +- **Limitations.** Native does not call the module yet (phase 3). +- **Cost.** `tdse_source.py` 937 lines against `numpy_ref/tdse.py`'s 845 (+92: + `node_fields` and the private transforms); `numpy_ref/backend.py` +3 net; tests + +308 (new module) and +6 elsewhere. +- **Handoff.** Native's hook calls `node_fields`, the binding, `atomic_sources`, + `atomic_edge_source` and `project_atomic_sources` with its own stage scalars and + prefactor, and finishes the stage itself (§4). + ## Part III — History - 2026-11-12: Drafted. @@ -855,6 +990,8 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. (human)); the status stays `Draft` until accepted. - 2026-11-12: Accepted by John Travers (#765); implementation to start once ADR-077's change (#767) is merged, in one implementation pull request (§11). +- 2026-11-12: In progress: Opus 5.5 (AI) on `feature/sep-0049-tdse-on-native`, based on + #767's branch; phases 0 and 1 reported in Part II. ### AI review — 2026-11-12 From 0f797abb317b3cdb880d523ab6a647687e3461d6 Mon Sep 17 00:00:00 2001 From: John Travers Date: Thu, 12 Nov 2026 13:33:23 +0000 Subject: [PATCH 04/20] Let a wait inside a commit raise the signal the commit holds _interrupts.raise_held, for a host source's wait inside native's advance (SEP 49 section 6.7); a no-op outside a commit. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HAYtF5P9QqEd3HQx3WJkA3 --- src/solprop/_interrupts.py | 15 +++++++++++++++ tests/driver/test_interrupts.py | 17 ++++++++++++++++- 2 files changed, 31 insertions(+), 1 deletion(-) diff --git a/src/solprop/_interrupts.py b/src/solprop/_interrupts.py index b35faf789..2d30451ee 100644 --- a/src/solprop/_interrupts.py +++ b/src/solprop/_interrupts.py @@ -29,6 +29,7 @@ "deferred_handlers", "handled", "poll", + "raise_held", ] @@ -86,6 +87,20 @@ def __exit__(self, kind: type[BaseException] | None, *_: object) -> None: raise pending +def raise_held() -> None: + """Raise, now, a signal the enclosing commit is holding. + + For a wait inside a commit that is not itself one: a native ``advance`` runs inside + :class:`committing`, and a host source it calls may wait minutes for an atomic call + (SEP 49 section 6.7). Its wait loop calls this between polls, so that a Ctrl-C or + SIGTERM stops it there, before the trial commits, rather than when the call ends. + Outside a commit nothing is ever held, and this does nothing. + """ + pending, _GUARD.pending = _GUARD.pending, None + if pending is not None: + raise pending + + def _handle(signal_number: int, frame: FrameType | None) -> None: """Raise the signal's exception now, or when the current commit ends.""" del frame diff --git a/tests/driver/test_interrupts.py b/tests/driver/test_interrupts.py index cd8ff7d80..a0841d265 100644 --- a/tests/driver/test_interrupts.py +++ b/tests/driver/test_interrupts.py @@ -21,7 +21,7 @@ from driver.test_resume import N_SAVE, CrashError from driver.test_resume_history import RTOL, assert_accurate_run, soliton, tight_run from march_bounds import TIGHTER -from solprop._interrupts import committing, handled +from solprop._interrupts import committing, handled, raise_held from solprop.exceptions import RunTerminated from solprop.log import NullReporter from solprop.output.hdf5 import open_file, resume @@ -145,6 +145,21 @@ def test_a_signal_inside_a_commit_waits_for_its_end(): assert signal.getsignal(signal.SIGTERM) is signal.SIG_DFL +@needs_default_sigterm +def test_a_wait_inside_a_commit_takes_the_held_signal_at_once(): + """A host source's wait inside native's commit raises the signal (SEP 49, 6.7).""" + events = [] + with handled(): + raise_held() # nothing is held outside a commit + with pytest.raises(RunTerminated), committing(): + signal.raise_signal(signal.SIGTERM) + events.append("held") + raise_held() + events.append("not reached") + events.append("nothing left to deliver") + assert events == ["held", "nothing left to deliver"] + + def test_an_application_s_own_handler_is_kept(): def own(signal_number, frame): pass From ea431aa651892f4157a788d4982a5555dd0a3caf Mon Sep 17 00:00:00 2001 From: John Travers Date: Thu, 12 Nov 2026 13:33:23 +0000 Subject: [PATCH 05/20] SEP 49 phase 2: a provider-agnostic source worker Start with subprocess (never multiprocessing) in a session of its own, protocol on private descriptors with stdout pointed at stderr, SIGINT and SIGTERM ignored, a watchdog that exits on parent-pipe EOF; length-prefixed frames, JSON control, arrays as raw bytes, no pickle, bounded frames; handshake with protocol version and provider identity; a reader thread and a 0.2 s wait loop that polls signals (held ones included) and the stop flag; kill at once on cancellation, a short grace only for close; reap and close on every path. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HAYtF5P9QqEd3HQx3WJkA3 --- src/solprop/driver/_source_worker.py | 765 +++++++++++++++++++++++++++ tests/driver/test_source_worker.py | 367 +++++++++++++ tests/source_adapters.py | 138 +++++ 3 files changed, 1270 insertions(+) create mode 100644 src/solprop/driver/_source_worker.py create mode 100644 tests/driver/test_source_worker.py create mode 100644 tests/source_adapters.py diff --git a/src/solprop/driver/_source_worker.py b/src/solprop/driver/_source_worker.py new file mode 100644 index 000000000..b30e7a8eb --- /dev/null +++ b/src/solprop/driver/_source_worker.py @@ -0,0 +1,765 @@ +# SPDX-FileCopyrightText: 2026 John C. Travers and contributors +# +# SPDX-License-Identifier: GPL-3.0-or-later + +"""Run a source provider in a worker process of its own (SEP 49 section 6.5). + +Provider-agnostic: the start, framing, handshake, lifetime, signals and cancellation of +a worker in which an *adapter* module runs a provider. Nothing here knows TDSE or +Pyroraptor; ``solprop.driver._pyroraptor`` is one adapter, and the tests run a +manufactured one. + +The parent starts ``sys.executable -m solprop.driver._source_worker `` +with :mod:`subprocess` -- never ``multiprocessing``, whose spawn re-imports the parent's +``__main__`` and whose fork inherits its runtimes -- in a session of its own, so that a +terminal's Ctrl-C or a scheduler's signal to the job reaches the parent alone, which +then ends the worker. The worker moves the protocol off its standard input and output +before importing the adapter, points standard output at standard error, ignores SIGINT +and SIGTERM, and exits at once when its pipe from the parent reaches end of file, even +during a call. + +**Frames.** An 8-byte little-endian length, then the payload. A message is one control +frame, a UTF-8 JSON object ``{"kind", "body", "arrays"}``, followed by one raw frame per +entry of ``arrays``, each the array's bytes in the dtype (byte order included) and shape +its entry names. Nothing is pickled. Every frame's declared length is checked against +the channel's limit, and an array frame's against its declared shape, before anything is +allocated. + +**Values.** A body is JSON with tagged objects: ``{"$a": i}`` the i-th array, ``{"$t": +[...]}`` a tuple, ``{"$d": {...}}`` a mapping, ``{"$f": "nan"}`` a non-finite float and +``{"$r": type, "f": fields, "c": computed}`` a record -- a dataclass or a plain object's +attributes, received as a :class:`Record`; any other object travels as its type and +``repr`` only (:class:`Opaque`). +""" + +import contextlib +import dataclasses +import importlib +import json +import math +import os +import queue +import signal +import struct +import subprocess +import sys +import threading +import traceback +from collections import deque +from collections.abc import Callable, Iterator, Mapping, Sequence +from types import SimpleNamespace +from typing import IO, Any, Protocol, cast, runtime_checkable + +import numpy as np + +from solprop._interrupts import poll, raise_held +from solprop.exceptions import BackendError +from solprop.log import StopFlag + +__all__ = [ + "CONTROL_LIMIT", + "POLL_INTERVAL", + "PROTOCOL_VERSION", + "Adapter", + "Opaque", + "Record", + "RemoteError", + "Worker", + "WorkerError", + "decode", + "encode", + "worker_environment", +] + +PROTOCOL_VERSION = 1 +"""The frame and message protocol both sides speak; a mismatch is refused.""" + +CONTROL_LIMIT = 1 << 26 +"""The default bound on one frame, 64 MiB, raised by a caller to its prepared shapes.""" + +POLL_INTERVAL = 0.2 +"""Seconds between the wait loop's checks for a signal or a stop (section 6.7).""" + +CLOSE_GRACE = 5.0 +"""Seconds an orderly close between calls waits for the worker before killing it.""" + +_HEADER = struct.Struct(" None: + super().__init__(f"{type_name.rpartition('.')[2]}: {detail}") + self.type_name = type_name + self.detail = detail + self.classification = classification + self.remote_traceback = remote_traceback + + +class Record(SimpleNamespace): + """A provider record received from the worker: its fields as attributes. + + ``record_type`` names the original type; ``computed`` holds values the adapter + computed from it (a property, a method's result) beside its fields, readable as + attributes but not among ``vars()``, so a record's fields stay what was sent. + """ + + __slots__ = ("computed", "record_type") + + def __init__( + self, + record_type: str, + fields: Mapping[str, object], + computed: Mapping[str, object] | None = None, + ) -> None: + super().__init__(**fields) + self.record_type = record_type + self.computed = dict(computed or {}) + + def __getattr__(self, name: str) -> object: + """Return a computed value of that name; called only for absent fields.""" + try: + return object.__getattribute__(self, "computed")[name] + except KeyError: + msg = f"{self.record_type} record has no field {name!r}" + raise AttributeError(msg) from None + + +@dataclasses.dataclass(frozen=True) +class Opaque: + """An object the codec does not carry: its type and ``repr`` only. + + Attributes + ---------- + type_name : str + Its ``module.qualname``. + text : str + Its ``repr``. + """ + + type_name: str + text: str + + +def _name(kind: type) -> str: + """Return a type's ``module.qualname``.""" + return f"{kind.__module__}.{kind.__qualname__}" + + +def encode( + value: object, computed: Callable[[object], Mapping[str, object]] | None = None +) -> tuple[object, list[np.ndarray]]: + """Encode a value as tagged JSON and the arrays it refers to. + + - ``computed``: for a record, the values to send beside its fields (an adapter's + choice), or None for none. + """ + arrays: list[np.ndarray] = [] + + def _walk(item: object) -> object: # noqa: PLR0911 - one return per value kind + if item is None or isinstance(item, bool | str): + return item + if isinstance(item, int): + return int(item) + if isinstance(item, float): + return item if math.isfinite(item) else {"$f": repr(item)} + if isinstance(item, np.ndarray) and not item.dtype.hasobject: + arrays.append(item) + return {"$a": len(arrays) - 1} + if isinstance(item, np.generic): + return _walk(item.item()) + if isinstance(item, tuple): + return {"$t": [_walk(entry) for entry in cast("tuple[object, ...]", item)]} + if isinstance(item, list): + return [_walk(entry) for entry in cast("list[object]", item)] + if isinstance(item, Mapping): + mapping = cast("Mapping[object, object]", item) + return {"$d": {str(key): _walk(entry) for key, entry in mapping.items()}} + fields = _fields(item) + if fields is None: + return {"$o": _name(type(item)), "repr": repr(item)} + extra = {} if computed is None else computed(item) + if isinstance(item, Record): + extra = {**item.computed, **extra} + return { + "$r": item.record_type if isinstance(item, Record) else _name(type(item)), + "f": {name: _walk(entry) for name, entry in fields.items()}, + "c": {name: _walk(entry) for name, entry in extra.items()}, + } + + return _walk(value), arrays + + +def _fields(value: object) -> dict[str, object] | None: + """Return a record's fields: a dataclass's, or a plain object's attributes.""" + if dataclasses.is_dataclass(value) and not isinstance(value, type): + return { + item.name: getattr(value, item.name) for item in dataclasses.fields(value) + } + if isinstance(value, Opaque | type) or callable(value): + return None + attributes = getattr(value, "__dict__", None) + return None if attributes is None else dict(attributes) + + +def decode(tree: object, arrays: Sequence[np.ndarray]) -> object: # noqa: PLR0911 + """Rebuild a value from tagged JSON and its arrays, inverting :func:`encode`.""" + if isinstance(tree, list): + return [decode(item, arrays) for item in cast("list[object]", tree)] + if not isinstance(tree, dict): + return tree + node = cast("dict[str, Any]", tree) + if "$a" in node: + return arrays[node["$a"]] + if "$t" in node: + return tuple(decode(item, arrays) for item in node["$t"]) + if "$d" in node: + return {key: decode(item, arrays) for key, item in node["$d"].items()} + if "$f" in node: + return float(node["$f"]) + if "$o" in node: + return Opaque(node["$o"], node["repr"]) + return Record( + node["$r"], + {key: decode(item, arrays) for key, item in node["f"].items()}, + {key: decode(item, arrays) for key, item in node["c"].items()}, + ) + + +# ------------------------------------------------------------------------- frames + + +def _read_exact(read: Callable[[int], bytes], size: int) -> bytearray | None: + """Read exactly `size` bytes; None at end of file before the first byte.""" + buffer = bytearray() + while len(buffer) < size: + chunk = read(size - len(buffer)) + if not chunk: + if buffer: + msg = "the worker pipe ended inside a frame" + raise WorkerError(msg) + return None + buffer += chunk + return buffer + + +def _read_frame(read: Callable[[int], bytes], limit: int) -> bytearray | None: + """Read one frame, refusing a declared length over `limit` before allocating.""" + header = _read_exact(read, _HEADER.size) + if header is None: + return None + (length,) = _HEADER.unpack(header) + if length > limit: + msg = f"a worker frame declares {length} bytes, over its bound of {limit}" + raise WorkerError(msg) + payload = _read_exact(read, length) if length else bytearray() + if payload is None: + msg = "the worker pipe ended inside a frame" + raise WorkerError(msg) + return payload + + +def read_message( + read: Callable[[int], bytes], limit: int +) -> tuple[str, object, list[np.ndarray]] | None: + """Read one message: its kind, its decoded body and its arrays; None at EOF.""" + control = _read_frame(read, limit) + if control is None: + return None + try: + message = json.loads(control.decode("utf-8")) + kind, body, specs = message["kind"], message["body"], message["arrays"] + if not isinstance(kind, str) or not isinstance(specs, list): + raise TypeError(type(message).__name__) + arrays: list[np.ndarray] = [] + for spec in cast("list[dict[str, Any]]", specs): + dtype = np.dtype(spec["dtype"]) + shape = tuple(int(extent) for extent in spec["shape"]) + if ( + dtype.hasobject + or dtype.kind not in "biufc" + or min(shape, default=0) < 0 + ): + raise TypeError(str(dtype)) + expected = math.prod(shape) * dtype.itemsize + if expected > limit: + msg = f"a worker array of {expected} bytes is over its bound of {limit}" + raise WorkerError(msg) + payload = _read_frame(read, expected) + if payload is None or len(payload) != expected: + msg = "a worker array frame does not match its declared shape" + raise WorkerError(msg) + arrays.append(np.frombuffer(payload, dtype=dtype).reshape(shape)) + return kind, decode(body, arrays), arrays + except WorkerError: + raise + except (ValueError, TypeError, KeyError, IndexError, UnicodeDecodeError) as error: + msg = f"a malformed worker message: {type(error).__name__}: {error}" + raise WorkerError(msg) from error + + +def write_message( + write: Callable[[bytes], object], + kind: str, + value: object, + computed: Callable[[object], Mapping[str, object]] | None = None, +) -> None: + """Write one message of `kind` carrying `value`.""" + tree, arrays = encode(value, computed) + contiguous = [np.ascontiguousarray(array) for array in arrays] + control = json.dumps( + { + "kind": kind, + "body": tree, + "arrays": [ + {"dtype": array.dtype.str, "shape": list(array.shape)} + for array in contiguous + ], + }, + allow_nan=False, + ).encode("utf-8") + parts = [_HEADER.pack(len(control)), control] + for array in contiguous: + data = array.tobytes() + parts += [_HEADER.pack(len(data)), data] + write(b"".join(parts)) + + +# ---------------------------------------------------------------------- the parent + + +def worker_environment( + environ: Mapping[str, str], explicit: Mapping[str, str] +) -> dict[str, str]: + """Return the worker's environment: the parent's, less native's own wait variables. + + The OpenMP wait variables solprop-native set for itself (ADR-071), which its marker + names, are removed so that the provider's runtime keeps its own defaults; a user's + own settings pass through. The worker imports from this process's import path. + ``explicit`` (the thread count, the device) wins. + """ + environment = dict(environ) + environment["PYTHONPATH"] = os.pathsep.join(entry for entry in sys.path if entry) + settings = _NATIVE_WAIT_SETTINGS.get(environment.get(_NATIVE_WAIT_MARKER, ""), {}) + if settings: + for name, value in settings.items(): + if environment.get(name, "").strip() == value: + del environment[name] + del environment[_NATIVE_WAIT_MARKER] + environment.update(explicit) + return environment + + +class Worker: + """Own one worker process, its two pipes and the threads that read them. + + Started by :meth:`start`, which waits for the handshake; ended by :meth:`close` + between calls (a shutdown request, a short grace, then a kill) or :meth:`kill` + (at once). Every path reaps the process and closes both pipe ends. A worker is + never used again after a transport failure or a cancellation; a provider exception + the adapter reports in a well-formed reply leaves it serving. + + - ``adapter``: the adapter module's dotted name, imported in the worker. + - ``environment``: the worker's whole environment (:func:`worker_environment`). + - ``limit``: the bound on any frame either way, bytes. + - ``stop``: the run's flag, polled while waiting. + """ + + def __init__( + self, + adapter: str, + *, + environment: Mapping[str, str], + limit: int = CONTROL_LIMIT, + stop: StopFlag | None = None, + ) -> None: + self.adapter = adapter + self.limit = limit + self.stop = stop + self.identity: Mapping[str, object] = {} + self.pid: int | None = None + self._environment = dict(environment) + self._process: subprocess.Popen[bytes] | None = None + self._replies: queue.Queue[tuple[str, object] | BaseException | None] = ( + queue.Queue() + ) + self._tail: deque[str] = deque(maxlen=_TAIL_LINES) + self._threads: list[threading.Thread] = [] + self._usable = False + self._cancelling = False + + # -- lifetime ------------------------------------------------------------------- + + def start(self) -> Mapping[str, object]: + """Start the worker and return its handshake's provider identity. + + A :class:`RemoteError` if the adapter could not start (a missing provider, say); + a :class:`WorkerError` if the worker exits first or speaks another protocol. + """ + creation = 0 + if sys.platform == "win32": # pragma: no cover - Windows only + creation = subprocess.CREATE_NEW_PROCESS_GROUP + self._process = subprocess.Popen( + [ + sys.executable, + # The package imports this module before ``-m`` runs it, which runpy + # warns of; ``main`` runs the imported copy, so nothing is duplicated. + "-W", + "ignore::RuntimeWarning:runpy", + "-m", + "solprop.driver._source_worker", + self.adapter, + str(self.limit), + ], + stdin=subprocess.PIPE, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + env=self._environment, + start_new_session=sys.platform != "win32", + creationflags=creation, + ) + process = self._process + assert process.stdout is not None and process.stderr is not None + for target, stream in ( + (self._read, process.stdout), + (self._drain, process.stderr), + ): + thread = threading.Thread(target=target, args=(stream,), daemon=True) + thread.start() + self._threads.append(thread) + try: + kind, body = self._wait() + if kind != "hello": + msg = f"the source worker opened with {kind!r}, not a handshake" + raise WorkerError(msg) + hello = cast("Mapping[str, object]", body) + if hello.get("protocol") != PROTOCOL_VERSION: + msg = ( + f"the source worker speaks protocol {hello.get('protocol')!r}, " + f"not {PROTOCOL_VERSION}" + ) + raise WorkerError(msg) + except BaseException: + self.kill() + raise + self.pid = cast("int", hello.get("pid")) + self.identity = cast("Mapping[str, object]", hello.get("identity", {})) + self._usable = True + return self.identity + + def request(self, kind: str, value: object = None) -> object: + """Send one request and return the adapter's reply. + + Waits in :data:`POLL_INTERVAL` steps; between them signal handlers run, a + signal a commit holds is raised (:func:`solprop._interrupts.raise_held`) and the + run's stop flag is polled. Any interruption kills the worker before it + propagates. An exception the adapter reported is a :class:`RemoteError`, and the + worker keeps serving; a worker that exits, closes its pipe or sends a malformed + message is a :class:`WorkerError`. + """ + if not self._usable or self._process is None: + msg = "the source worker is not running" + raise WorkerError(msg) + process = self._process + assert process.stdin is not None + stdin = process.stdin + try: + write_message(stdin.write, kind, value) + stdin.flush() + reply, body = self._wait() + except RemoteError: + raise + except BaseException: + self.kill() + raise + if reply != "reply": + self.kill() + msg = f"the source worker answered {kind!r} with {reply!r}" + raise WorkerError(msg) + return body + + def close(self) -> None: + """End the worker between calls: ask it to exit, then kill it after a grace.""" + process = self._process + if process is None: + return + if self._usable and process.poll() is None: + try: + assert process.stdin is not None + write_message(process.stdin.write, "close", None) + process.stdin.flush() + process.wait(timeout=CLOSE_GRACE) + except OSError, subprocess.TimeoutExpired, ValueError: + pass + self.kill() + + def kill(self) -> None: + """End the worker at once, reap it and close both pipe ends.""" + self._usable = False + process = self._process + if process is None: + return + self._cancelling = True + if process.poll() is None: + with contextlib.suppress(ProcessLookupError, PermissionError, OSError): + if sys.platform == "win32": # pragma: no cover - Windows only + process.kill() + else: + # Its own session: the group holds anything the provider started. + os.killpg(process.pid, signal.SIGKILL) + process.wait() + if process.stdin is not None: + with contextlib.suppress(OSError): + process.stdin.close() + # The worker's ends are closed now, so its readers see the end of their pipes. + for thread in self._threads: + thread.join(timeout=CLOSE_GRACE) + self._threads = [] + for stream in (process.stdout, process.stderr): + if stream is not None: + with contextlib.suppress(OSError): + stream.close() + + @property + def alive(self) -> bool: + """Whether the worker is running and may take a request.""" + return ( + self._usable and self._process is not None and self._process.poll() is None + ) + + # -- waiting -------------------------------------------------------------------- + + def _wait(self) -> tuple[str, object]: + """Return the next message, polling signals and the stop flag meanwhile.""" + while True: + try: + item = self._replies.get(timeout=POLL_INTERVAL) + except queue.Empty: + raise_held() + poll(self.stop) + continue + if isinstance(item, BaseException): + raise item + if item is None: + raise self._exited() + kind, body = item + if kind == "error": + report = cast("Mapping[str, str]", body) + raise RemoteError( + report["type"], + report["message"], + report["classification"], + report["traceback"], + ) + return kind, body + + def _exited(self) -> WorkerError: + """Return the failure of a worker whose pipe ended: its status, its stderr.""" + process = self._process + self._usable = False + status: int | None = None + if process is not None: + try: + status = process.wait(timeout=CLOSE_GRACE) + except subprocess.TimeoutExpired: + status = None + tail = "\n".join(self._tail) + error = WorkerError( + f"the source worker ended (exit status {status}) without replying" + ) + if tail: + error.add_note(f"its last standard error:\n{tail}") + return error + + def _read(self, stream: IO[bytes]) -> None: + """Move the worker's messages to the reply queue until its pipe ends.""" + try: + while True: + message = read_message(stream.read, self.limit) + if message is None: + self._replies.put(None) + return + kind, body, _ = message + self._replies.put((kind, body)) + except WorkerError as error: + self._replies.put(error) + except OSError, ValueError: + # The parent closed the pipe while killing the worker. + self._replies.put(None) + + def _drain(self, stream: IO[bytes]) -> None: + """Pass the worker's standard error on, keeping its last lines for a failure.""" + for line in iter(stream.readline, b""): + text = line.decode("utf-8", errors="replace").rstrip("\n") + self._tail.append(text) + with contextlib.suppress(OSError, ValueError): + sys.stderr.write(text + "\n") + + +# ----------------------------------------------------------------------- the worker + + +@runtime_checkable +class Adapter(Protocol): + """What an adapter module's ``make_adapter()`` returns, inside the worker.""" + + def identity(self) -> Mapping[str, object]: + """Return the provider's identity for the handshake.""" + ... + + def handle(self, kind: str, value: object) -> object: + """Serve one request.""" + ... + + def classify(self, error: BaseException) -> str: + """Classify an exception for the failure record.""" + ... + + def computed(self, value: object) -> Mapping[str, object]: + """Return values to send beside a record's fields.""" + ... + + def close(self) -> None: + """Release everything the adapter holds.""" + ... + + +@contextlib.contextmanager +def _private_streams() -> Iterator[tuple[int, int]]: + """Move the protocol off fds 0 and 1, pointing 1 at stderr and 0 at the null device. + + Whatever the provider or its runtime prints then goes to standard error and cannot + corrupt a frame. + """ + inbound, outbound = os.dup(0), os.dup(1) + null = os.open(os.devnull, os.O_RDONLY) + os.dup2(null, 0) + os.close(null) + os.dup2(2, 1) + sys.stdout = sys.stderr + yield inbound, outbound + + +def _watch(inbound: int, limit: int, inbox: queue.Queue[tuple[str, object]]) -> None: + """Read the parent's messages; exit the process at once when its pipe ends (P8).""" + try: + while True: + message = read_message(lambda size: os.read(inbound, size), limit) + if message is None: + break + kind, body, _ = message + inbox.put((kind, body)) + except WorkerError, OSError: + pass + # The parent is gone, or wrote garbage: nothing the worker computes can be used. + os._exit(_ORPHANED) + + +def _error_report(error: BaseException, adapter: Adapter | None) -> dict[str, str]: + """Return an exception as type, message, classification and traceback text.""" + return { + "type": _name(type(error)), + "message": str(error), + "classification": "unexpected" if adapter is None else adapter.classify(error), + "traceback": "".join(traceback.format_exception(error)), + } + + +def main(argv: Sequence[str]) -> int: + """Serve one parent: handshake, requests until ``close``, then exit. + + - ``argv``: the adapter, ``module`` or ``module:factory`` (``make_adapter`` by + default), and the frame bound in bytes. + """ + adapter_name, limit = argv[0], int(argv[1]) + for number in (signal.SIGINT, signal.SIGTERM): + signal.signal(number, signal.SIG_IGN) + with _private_streams() as (inbound, outbound): + PROTOCOL_DESCRIPTORS[:] = [inbound, outbound] + + def _send(kind: str, value: object, adapter: Adapter | None = None) -> None: + write_message( + lambda data: _write_all(outbound, data), + kind, + value, + None if adapter is None else adapter.computed, + ) + + inbox: queue.Queue[tuple[str, object]] = queue.Queue() + threading.Thread( + target=_watch, args=(inbound, limit, inbox), daemon=True + ).start() + try: + module_name, _, factory = adapter_name.partition(":") + module = importlib.import_module(module_name) + adapter = cast("Adapter", getattr(module, factory or "make_adapter")()) + identity = adapter.identity() + except Exception as error: + _send("error", _error_report(error, None)) + return 1 + _send( + "hello", + {"protocol": PROTOCOL_VERSION, "pid": os.getpid(), "identity": identity}, + ) + while True: + kind, body = inbox.get() + if kind == "close": + adapter.close() + return 0 + try: + reply = adapter.handle(kind, body) + except Exception as error: + _send("error", _error_report(error, adapter)) + continue + _send("reply", reply, adapter) + + +def _write_all(descriptor: int, data: bytes) -> None: + """Write every byte to a descriptor, across partial writes.""" + view = memoryview(data) + while view: + view = view[os.write(descriptor, view) :] + + +if __name__ == "__main__": + # Run the imported module's main, not this __main__ copy's, so that the records + # an adapter receives are the classes it imports. + from solprop.driver._source_worker import main as _main + + status = _main(sys.argv[1:]) + # Skip interpreter finalisation: the provider's runtime is already closed by the + # adapter, and nothing else of the worker's needs flushing. + sys.stderr.flush() + os._exit(status) diff --git a/tests/driver/test_source_worker.py b/tests/driver/test_source_worker.py new file mode 100644 index 000000000..5553469d1 --- /dev/null +++ b/tests/driver/test_source_worker.py @@ -0,0 +1,367 @@ +# SPDX-FileCopyrightText: 2026 John C. Travers and contributors +# +# SPDX-License-Identifier: GPL-3.0-or-later + +"""The provider worker's contract, in real worker processes (SEP 49 section 7.2). + +Manufactured adapters (``tests/source_adapters.py``) stand in for the provider: a +provider-agnostic one whose requests misbehave on purpose, and Pyroraptor's own adapter +around the manufactured haCC provider of ``hacc_cases``. Deadlines here are the test +harness's, to detect a broken cancellation; the production code has none. +""" + +import dataclasses +import json +import os +import signal +import subprocess +import sys +import threading +import time +from typing import Any + +import numpy as np +import pytest + +from solprop._interrupts import committing, handled +from solprop.driver import _source_worker +from solprop.driver._source_worker import ( + POLL_INTERVAL, + Record, + RemoteError, + Worker, + WorkerError, + decode, + encode, + worker_environment, +) +from solprop.exceptions import RunTerminated, StopRequested + +pytestmark = pytest.mark.medium + +# How long a cancellation may take before the test calls it broken: the poll interval +# and a kill, with room for a loaded machine. +DEADLINE = 10.0 +POSIX = pytest.mark.skipif(sys.platform == "win32", reason="POSIX signals and groups") + + +def _worker(adapter="source_adapters", **options): + worker = Worker(adapter, environment=worker_environment(os.environ, {}), **options) + worker.start() + return worker + + +@pytest.fixture +def worker(): + started = _worker() + yield started + started.kill() + + +def _assert_ended(worker): + """The process is reaped and both pipe ends closed.""" + process = worker._process + assert process is not None and process.returncode is not None + assert process.stdin is not None and process.stdin.closed + assert process.stdout is not None and process.stdout.closed + assert not worker.alive + + +@dataclasses.dataclass(frozen=True) +class _Pair: + left: object + right: object + + +def test_the_codec_round_trips_values_and_records_without_pickle(): + value = { + "arrays": [ + np.arange(6, dtype=">f8").reshape(2, 3), + np.array([1 + 2j], dtype=np.complex128), + np.array([True, False]), + np.zeros((0, 4), dtype=np.int64), + ], + "tuple": (1, 2.5, None, "text"), + "nonfinite": [float("nan"), float("inf"), -float("inf")], + "record": _Pair((1,), np.ones(2)), + "opaque": object(), + } + tree, arrays = encode(value) + json.dumps(tree, allow_nan=False) + back: Any = decode(json.loads(json.dumps(tree)), arrays) + for sent, received in zip(value["arrays"], back["arrays"], strict=True): + assert received.dtype == sent.dtype and received.shape == sent.shape + np.testing.assert_array_equal(received, sent) + assert back["tuple"] == value["tuple"] + assert np.isnan(back["nonfinite"][0]) and back["nonfinite"][1:] == [ + float("inf"), + -float("inf"), + ] + record = back["record"] + assert isinstance(record, Record) and record.record_type.endswith("_Pair") + assert record.left == (1,) + assert back["opaque"].type_name == "builtins.object" + + +def test_a_record_s_computed_values_read_as_attributes_but_are_not_fields(): + tree, arrays = encode(_Pair(1, 2), lambda value: {"ok": True}) + record: Any = decode(tree, arrays) + assert record.ok is True + assert vars(record) == {"left": 1, "right": 2} + with pytest.raises(AttributeError, match="no field 'missing'"): + record.missing # noqa: B018 - the attribute access is the test + + +def test_the_worker_environment_drops_native_s_own_wait_settings_only(): + native = { + "SOLPROP_NATIVE_WAIT": "spin", + "GOMP_SPINCOUNT": "10000", + "KMP_BLOCKTIME": "0", + } + environment = worker_environment(native, {"PYRORAPTOR_THREADS": "3"}) + assert "SOLPROP_NATIVE_WAIT" not in environment + assert "GOMP_SPINCOUNT" not in environment and "KMP_BLOCKTIME" not in environment + assert environment["PYRORAPTOR_THREADS"] == "3" + user = {"OMP_WAIT_POLICY": "active", "KMP_BLOCKTIME": "200"} + assert worker_environment(user, {}).items() >= user.items() + edited = { + "SOLPROP_NATIVE_WAIT": "spin", + "GOMP_SPINCOUNT": "5", + "KMP_BLOCKTIME": "0", + } + kept = worker_environment(edited, {}) + assert kept["GOMP_SPINCOUNT"] == "5" and "KMP_BLOCKTIME" not in kept + + +def test_values_cross_a_real_worker_bit_for_bit(worker): + rng = np.random.default_rng(49) + sent = {"field": rng.standard_normal((3, 257)), "labels": (1, 3), "z": 0.25} + received = worker.request("echo", sent) + np.testing.assert_array_equal(received["field"], sent["field"]) + assert received["field"].dtype == np.float64 + assert received["labels"] == (1, 3) and received["z"] == 0.25 + assert worker.identity["adapter"] == "manufactured" + assert worker.pid == worker.identity["pid"] != os.getpid() + + +def test_the_provider_s_printing_never_corrupts_a_frame(worker): + assert worker.request("print") == "printed" + assert worker.request("echo", [1, 2]) == [1, 2] + + +def test_a_provider_exception_is_reported_by_name_and_the_worker_serves_on(worker): + with pytest.raises(RemoteError) as raised: + worker.request("raise") + error = raised.value + assert error.type_name == "builtins.ValueError" + assert error.detail == "manufactured provider failure" + assert error.classification == "manufactured" + assert "manufactured provider failure" in error.remote_traceback + assert worker.request("echo", 3) == 3 + + +def test_a_worker_that_exits_is_a_named_failure_with_its_status_and_stderr(worker): + with pytest.raises(WorkerError, match="exit status 7") as raised: + worker.request("exit") + assert "gives up" in "\n".join(raised.value.__notes__) + _assert_ended(worker) + with pytest.raises(WorkerError, match="not running"): + worker.request("echo", 1) + + +@pytest.mark.parametrize( + ("kind", "message"), + [ + ("close_pipe", "ended"), + ("malformed", "malformed worker message"), + ], +) +def test_a_closed_pipe_or_a_malformed_frame_ends_the_worker(worker, kind, message): + with pytest.raises(WorkerError, match=message): + worker.request(kind) + _assert_ended(worker) + + +def test_an_oversized_frame_is_refused_before_it_is_read(): + worker = _worker(limit=4096) + with pytest.raises(WorkerError, match="over its bound of 4096"): + worker.request("oversized", {"bytes": 10_000}) + _assert_ended(worker) + + +def test_a_failure_while_starting_is_reported_by_name(): + worker = Worker( + "source_adapters:failing_adapter", + environment=worker_environment(os.environ, {}), + ) + with pytest.raises(RemoteError, match="cannot start"): + worker.start() + _assert_ended(worker) + + +def test_a_protocol_mismatch_is_refused_before_any_request(monkeypatch): + monkeypatch.setattr(_source_worker, "PROTOCOL_VERSION", 99) + worker = Worker("source_adapters", environment=worker_environment(os.environ, {})) + with pytest.raises(WorkerError, match="protocol 1, not 99"): + worker.start() + _assert_ended(worker) + + +def test_close_between_calls_ends_the_worker_in_order(worker): + worker.close() + _assert_ended(worker) + assert worker._process is not None and worker._process.returncode == 0 + + +def _timed(worker, kind, expected): + started = time.monotonic() + with pytest.raises(expected): + worker.request(kind) + elapsed = time.monotonic() - started + _assert_ended(worker) + return elapsed + + +def test_a_stop_flag_cancels_a_hanging_call_at_the_next_poll(worker): + flag = threading.Event() + worker.stop = flag + threading.Timer(0.3, flag.set).start() + assert _timed(worker, "hang", StopRequested) < DEADLINE + + +@POSIX +@pytest.mark.parametrize( + ("number", "expected"), + [(signal.SIGINT, KeyboardInterrupt), (signal.SIGTERM, RunTerminated)], +) +def test_a_signal_cancels_a_hanging_call(worker, number, expected): + if signal.getsignal(signal.SIGTERM) is not signal.SIG_DFL: + pytest.skip("an application handler owns SIGTERM here") + with handled(): + threading.Timer(0.3, os.kill, (os.getpid(), number)).start() + assert _timed(worker, "hang", expected) < DEADLINE + + +@POSIX +def test_a_signal_held_by_a_commit_is_taken_by_the_wait(worker): + """Native's ``advance`` waits inside ``committing``: the loop raises the signal.""" + with handled(): + threading.Timer(0.3, os.kill, (os.getpid(), signal.SIGINT)).start() + started = time.monotonic() + with pytest.raises(KeyboardInterrupt), committing(): + worker.request("hang") + assert time.monotonic() - started < DEADLINE + _assert_ended(worker) + + +def test_the_worker_imports_neither_backend_nor_the_parent_s_main(worker): + loaded = worker.request("modules") + names = loaded["modules"] + assert not [name for name in names if name.split(".")[0] == "solprop_native"] + assert not [name for name in names if "numpy_ref" in name] + assert "pyroraptor" not in names + assert loaded["main"] == "solprop.driver._source_worker" + assert POLL_INTERVAL <= 0.2 + + +_CHILD = """ +import json, os, sys +from solprop.driver._source_worker import Worker, worker_environment +worker = Worker("source_adapters", environment=worker_environment(os.environ, {})) +worker.start() +loaded = worker.request("modules") +print(json.dumps({"pid": worker.pid, "main": loaded["main"], + "native": [m for m in loaded["modules"] + if m.split(".")[0] == "solprop_native" + or "numpy_ref" in m]}), flush=True) +""" + + +@pytest.mark.slow +def test_a_worker_started_from_a_script_or_python_c_never_re_imports_it(tmp_path): + script = tmp_path / "user_script.py" + script.write_text( + "import sys\n" + "if __name__ != '__main__':\n" + " raise SystemExit('the user script was imported again')\n" + _CHILD, + encoding="utf-8", + ) + environment = {**os.environ, "PYTHONPATH": os.pathsep.join(sys.path)} + for command in ([sys.executable, str(script)], [sys.executable, "-c", _CHILD]): + done = subprocess.run( + command, + capture_output=True, + text=True, + timeout=120, + check=True, + env=environment, + cwd=tmp_path, + ) + report = json.loads(done.stdout.strip().splitlines()[-1]) + assert report["main"] == "solprop.driver._source_worker" + assert report["native"] == [] + + +_HANGING_PARENT = """ +import json, os, sys +from solprop._interrupts import handled +from solprop.driver._source_worker import Worker, WorkerError, worker_environment +worker = Worker("source_adapters", environment=worker_environment(os.environ, {})) +worker.start() +print(json.dumps({"worker": worker.pid}), flush=True) +try: + with handled(): + worker.request("hang") +except KeyboardInterrupt: + print(json.dumps({"ended": "interrupted"}), flush=True) +except WorkerError as error: + print(json.dumps({"ended": "provider failure"}), flush=True) +""" + + +def _gone(pid): + """Wait, within the harness deadline, for a process to disappear.""" + started = time.monotonic() + while time.monotonic() - started < DEADLINE: + try: + os.kill(pid, 0) + except ProcessLookupError: + return True + time.sleep(0.05) + return False + + +def _hanging_parent(): + environment = {**os.environ, "PYTHONPATH": os.pathsep.join(sys.path)} + parent = subprocess.Popen( + [sys.executable, "-c", _HANGING_PARENT], + stdout=subprocess.PIPE, + text=True, + env=environment, + start_new_session=True, + ) + assert parent.stdout is not None + worker = json.loads(parent.stdout.readline())["worker"] + time.sleep(2 * POLL_INTERVAL) + return parent, worker + + +@POSIX +@pytest.mark.slow +def test_a_group_wide_ctrl_c_interrupts_the_run_and_spares_the_worker(): + parent, worker = _hanging_parent() + os.killpg(parent.pid, signal.SIGINT) + out, _ = parent.communicate(timeout=DEADLINE) + assert json.loads(out.strip().splitlines()[-1]) == {"ended": "interrupted"} + assert _gone(worker) + + +@POSIX +@pytest.mark.slow +def test_a_parent_killed_outright_leaves_no_worker_behind(): + parent, worker = _hanging_parent() + os.kill(parent.pid, signal.SIGKILL) + parent.wait(timeout=DEADLINE) + assert parent.stdout is not None + parent.stdout.close() + assert _gone(worker) diff --git a/tests/source_adapters.py b/tests/source_adapters.py new file mode 100644 index 000000000..320978cdf --- /dev/null +++ b/tests/source_adapters.py @@ -0,0 +1,138 @@ +# SPDX-FileCopyrightText: 2026 John C. Travers and contributors +# +# SPDX-License-Identifier: GPL-3.0-or-later + +"""Manufactured adapters the source worker serves in the tests, inside the worker. + +``make_adapter`` is a provider-agnostic adapter whose requests misbehave on purpose: +it echoes, hangs, raises, exits, prints, closes its pipe or writes a malformed frame. +The ``*_tdse`` factories serve the TDSE binding with Pyroraptor's adapter around the +manufactured haCC provider of ``hacc_cases`` (SEP 49 section 7.2), the module this +worker imports in place of ``pyroraptor``. +""" + +import os +import sys +import time +from typing import Any, cast + +import numpy as np + +from solprop.driver import _pyroraptor, _source_worker + + +class _Manufactured: + """An adapter with no provider: each request names its behaviour.""" + + def identity(self): + return {"adapter": "manufactured", "pid": os.getpid()} + + def classify(self, error): + del error + return "manufactured" + + def computed(self, value): + del value + return {} + + def handle(self, kind, value): + if kind == "echo": + return value + if kind == "hang": + time.sleep(3600) + if kind == "raise": + raise ValueError("manufactured provider failure") + if kind == "exit": + sys.stderr.write("the manufactured provider gives up\n") + sys.stderr.flush() + os._exit(7) + if kind == "print": + sys.stdout.write("noise on standard output\n") + sys.stdout.flush() + os.write(1, b"raw noise on descriptor 1\n") + return "printed" + if kind == "oversized": + return np.zeros(value["bytes"], dtype=np.uint8) + if kind == "malformed": + payload = b"{not json" + _source_worker._write_all( + _source_worker.PROTOCOL_DESCRIPTORS[1], + _source_worker._HEADER.pack(len(payload)) + payload, + ) + time.sleep(3600) + if kind == "close_pipe": + os.close(_source_worker.PROTOCOL_DESCRIPTORS[1]) + time.sleep(3600) + if kind == "modules": + spec = getattr(sys.modules["__main__"], "__spec__", None) + return { + "modules": sorted(sys.modules), + "main": None if spec is None else spec.name, + } + if kind == "environment": + return dict(os.environ) + raise KeyError(kind) + + def close(self): + pass + + +def make_adapter(): + return _Manufactured() + + +def failing_adapter(): + raise ValueError("the manufactured provider cannot start") + + +def _provider(**changes): + from hacc_cases import FakeHaccProvider, alpha_law # noqa: PLC0415 - worker only + + provider = FakeHaccProvider(**changes) + # The run tests' law: a 2% linear model error and a dyadic fifth-order remainder. + provider.response = alpha_law(1.02, lambda5=2.0**-200) + hang = os.environ.get("MANUFACTURED_HANG_AFTER") + if hang is not None: + _hang_after(provider, int(hang)) + return provider + + +def _hang_after(provider, calls): + """Make every atomic call after the first `calls` hang.""" + original = provider.hacc.ResponsePlan + + def plan(model, **values): + built = original(model, **values) + evaluate = built.evaluate + + def slow(fields): + if len(provider.calls) >= calls: + sys.stderr.write("MANUFACTURED CALL HANGS\n") + sys.stderr.flush() + time.sleep(3600) + return evaluate(fields) + + built.evaluate = slow + return built + + provider.hacc.ResponsePlan = plan + + +def _adapter(provider): + return _pyroraptor.PyroraptorAdapter(cast("Any", provider)) + + +def manufactured_tdse(): + return _adapter(_provider()) + + +def stale_tdse(): + from hacc_cases import FakeBuild # noqa: PLC0415 - worker only + + return _adapter(_provider(build=FakeBuild(source_revision="old0000"))) + + +def cuda_tdse(): + from hacc_cases import FakeBuild # noqa: PLC0415 - worker only + + return _adapter(_provider(build=FakeBuild(execution_space="Cuda"))) From 8f5b64bf237fa896b85e2f36775176bed6e5ab4f Mon Sep 17 00:00:00 2001 From: John Travers Date: Thu, 12 Nov 2026 13:33:23 +0000 Subject: [PATCH 06/20] SEP 49 phase 2: run Pyroraptor in the worker for every TDSE use The binding, basis_from_preset, hacc_preset, hacc_basis and DiagnosticSession talk to Pyroraptor's adapter in a worker; solprop's process never imports the provider. Budgets, the archive, validation, normalisation and warnings stay in the binding, on plain records. Transport copies join the memory ledger and the estimate; the run's stop flag reaches the binding; a stop during a call kills the worker, keeps the charge and records the evaluation as interrupted. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HAYtF5P9QqEd3HQx3WJkA3 --- changelog.d/tdse-provider-worker.changed.md | 1 + docs/dev/architecture.md | 4 +- docs/guide/media/tdse.md | 13 +- src/solprop/driver/_pyroraptor.py | 786 ++++++++++++ src/solprop/driver/simulation.py | 5 + src/solprop/driver/tdse.py | 1255 +++++++++---------- tests/driver/test_tdse_archive_binding.py | 5 +- tests/driver/test_tdse_binding.py | 31 +- tests/driver/test_tdse_entry.py | 6 +- tests/driver/test_tdse_hacc_binding.py | 5 +- tests/driver/test_tdse_hacc_installed.py | 5 +- tests/driver/test_tdse_worker.py | 204 +++ tests/hacc_cases.py | 7 +- tests/source_channel.py | 86 ++ 14 files changed, 1704 insertions(+), 709 deletions(-) create mode 100644 changelog.d/tdse-provider-worker.changed.md create mode 100644 src/solprop/driver/_pyroraptor.py create mode 100644 tests/driver/test_tdse_worker.py create mode 100644 tests/source_channel.py diff --git a/changelog.d/tdse-provider-worker.changed.md b/changelog.d/tdse-provider-worker.changed.md new file mode 100644 index 000000000..8f8b0c5f0 --- /dev/null +++ b/changelog.d/tdse-provider-worker.changed.md @@ -0,0 +1 @@ +The TDSE response's atomic provider now runs in a worker process of its own, never in solprop's process, for presets, bases, consistency checks and runs alike; Ctrl-C, SIGTERM and a run's stop flag now stop a TDSE run during an atomic call, the call charged and the last accepted step checkpointed (SEP 49). diff --git a/docs/dev/architecture.md b/docs/dev/architecture.md index 50ba57ea0..9945a97d4 100644 --- a/docs/dev/architecture.md +++ b/docs/dev/architecture.md @@ -117,7 +117,9 @@ binding from its step record. It changes no numerical step and adds no call; the optical file records the archive's run UUID and relative path. Plan lifetime, allocation admission and attempted-call budgets -belong to the driver/binding; modal synthesis to the nodes, the assembly of the +belong to the driver/binding, which runs the provider in a worker process of its own +(`driver/_source_worker.py`, provider-agnostic, with `driver/_pyroraptor.py` as the +provider's adapter): framed pipes, no pickle, cancellation while waiting; modal synthesis to the nodes, the assembly of the macroscopic sources and their projection belong to `backends/tdse_source.py`, which imports neither backend and which a backend calls from its source evaluation ([SEP 49](seps/sep-0049-tdse-on-native.md)). See the [TDSE manual](../guide/media/tdse.md). diff --git a/docs/guide/media/tdse.md b/docs/guide/media/tdse.md index 09cf435cd..75268f55d 100644 --- a/docs/guide/media/tdse.md +++ b/docs/guide/media/tdse.md @@ -13,9 +13,11 @@ electrons and the atomic resonances that a rate model omits. The physics in outl :class: note The atomic engine, Pyroraptor, is a separate, optional package. It is not yet publicly -available; it is planned to be. solprop declares no dependency on it and imports it only -when a TDSE response is prepared; without it, preparing a TDSE response raises a -`BackendError` naming the missing provider. The records, their validation, identities +available; it is planned to be. solprop declares no dependency on it and never imports +it into its own process: preparing a TDSE response starts it in a worker process of its +own, the same interpreter, given the run's whole thread budget (or, for a GPU build, its +device). Without it, preparing a TDSE response raises a `BackendError` naming the missing +provider. The records, their validation, identities and descriptors work without it. The TDSE response is a research feature, not a qualified replacement for the rate models. ``` @@ -250,7 +252,10 @@ responses](../waveguides/radial-quadrature.md)). With `tdse=`, every accepted st saved and only the final plane is requested by default, so saving never forces an extra atomic call; an explicit `n_save`, `save_planes` or `step_planes` is honoured. Every attempted provider call is charged against `max_atomic_calls` before it is made, failures -included. A capped or interrupted run resumes; the gate and window are part of the resume +included. Ctrl-C, SIGTERM and a run's stop flag stop it during an atomic call as well as +between steps: the worker is ended at once, the call in flight stays charged, and the run +ends at its last accepted step with a terminal checkpoint. A call has no time limit. A +capped or interrupted run resumes; the gate and window are part of the resume identity, so a run never resumes under another one. ## The raw response archive diff --git a/src/solprop/driver/_pyroraptor.py b/src/solprop/driver/_pyroraptor.py new file mode 100644 index 000000000..6de646399 --- /dev/null +++ b/src/solprop/driver/_pyroraptor.py @@ -0,0 +1,786 @@ +# SPDX-FileCopyrightText: 2026 John C. Travers and contributors +# +# SPDX-License-Identifier: GPL-3.0-or-later + +"""Run Pyroraptor inside the source worker: the TDSE binding's provider adapter. + +The one place solprop imports the optional provider, and only in the worker +(:mod:`solprop.driver._source_worker`, SEP 49 section 6.5). It builds the provider's +models and plans from the plain records the binding sends, makes its atomic calls and +returns their results as plain records, with the provider's errors classified. Budgets, +the archive, validation, normalisation and warnings stay in the binding +(:mod:`solprop.driver.tdse`), in solprop's own process. + +Requests: ``basis_from_preset``, ``hacc_preset`` and ``hacc_basis`` (queries); then for +a run ``prepare``, ``build_plans`` and ``evaluate``; for a consistency check +``diagnostic_plan``, ``diagnostic_evaluate`` and ``diagnostic_close``. +""" + +import importlib +import json +import sys +from collections.abc import Callable, Mapping +from contextlib import ExitStack +from dataclasses import asdict, dataclass, fields +from importlib.metadata import PackageNotFoundError, distribution +from typing import Any, Literal, Protocol, cast, runtime_checkable + +import numpy as np + +from solprop.driver._source_worker import Record +from solprop.exceptions import BackendError, DriverError, MaterialError, SolpropError +from solprop.nonlinear.tdse import HaccBasis, TDSEBasis +from solprop.translate.tdse import HACC_NORM_GROWTH_LIMIT + +__all__ = ["PyroraptorAdapter", "make_adapter"] + +# Native classes cannot occur in runtime annotations: optional-package-free test +# collection resolves every annotation through beartype. Confine their dynamic +# typing to this adapter; the seam's scientific records and arrays remain typed. +_Native = Any +# The haCC consumer surface of the provider (SEP 20 section 5.7, C1 to C5). +_HACC_SURFACE = ( + "AngularLimit", + "Argon", + "Basis", + "GridResponse", + "Helium", + "MixedGauge", + "ResponsePlan", + "TotalCurrentResponse", + "VelocityGauge", + "helium_preset", + "prepare", +) +# The shadow components the binding reads, computed here where the record's own +# ``component`` method is (section 6.5, plain-data results). +_COMPONENTS = ("total_current", "current", "charge_dipole") +_PRESET_FIELDS = ( + "identity", + "variant", + "step_margin", + "accuracy", + "box_sensitivity", + "gauge_sensitivity", + "max_peak_intensity", + "wavelength", + "window", + "device_admissible", + "source", +) + + +@runtime_checkable +class _Provider(Protocol): + """Type the optional module at its single dynamic-import boundary.""" + + AngularBasis: Callable[..., _Native] + ComplexScaling: Callable[..., _Native] + Discretization: Callable[..., _Native] + DrivenError: type[Exception] + GridResponse: type + RadialBasis: Callable[..., _Native] + ResponsePlan: Callable[..., _Native] + SpectralOptions: Callable[..., _Native] + SpectralSetup: Callable[..., _Native] + StateSelection: Callable[..., _Native] + TotalCurrentResponse: type + atomic_model: Callable[[str], _Native] + build_info: Callable[[], _Native] + # The haCC namespace is looked up by name (`_hacc_surface`): an SAE-only + # provider is still a provider, and is refused only when haCC is asked for. + solve_preset: Callable[[str, str], _Native] + unit_scale: Callable[[str], _Native] + + +def _load_provider() -> _Provider: + """Load the optional package, in the worker only.""" + if sys.version_info[:2] != (3, 14): + raise BackendError("the pinned TDSE provider requires Python 3.14") + try: + return cast("_Provider", importlib.import_module("pyroraptor")) + except (ImportError, OSError) as error: + raise BackendError( + "TDSE response requires the optional pyroraptor dependency; " + "install the pinned Pyroraptor revision into the environment, " + "with a matching CPU, CUDA or HIP build" + ) from error + + +def _identity(provider: _Provider) -> dict[str, object]: + """Return the provider's build and installed distribution, for the handshake.""" + record: dict[str, object] = {"native": asdict(provider.build_info())} + try: + package = distribution("pyroraptor") + except PackageNotFoundError: + record["distribution"] = None + else: + record["distribution"] = package.version + source = package.read_text("direct_url.json") + record["direct_url"] = None if source is None else json.loads(source) + return record + + +def _admit_memory(ledger: Mapping[str, int], limit: int | None) -> None: + """Apply the declared atomic byte limit. + + - ``ledger``: named persistent allocations or estimates in bytes. + - ``limit``: positive byte ceiling, or None for an unrestricted allocation. + """ + total = sum(ledger.values()) + if limit is not None and total > limit: + raise DriverError( + f"TDSE persistent data requires {total} bytes, exceeding {limit} bytes" + ) + + +def _is(value: object, kind: type) -> bool: + """Whether a provider object, or a record sent of one, is of `kind`.""" + if isinstance(value, Record): + return value.record_type == f"{kind.__module__}.{kind.__qualname__}" + return isinstance(value, kind) + + +def _hacc_surface(provider: _Provider, revision: str) -> _Native: + """Return the provider's haCC namespace, refusing a build without it.""" + hacc = getattr(provider, "hacc", None) + missing = [name for name in _HACC_SURFACE if not hasattr(hacc, name)] + if missing: + raise BackendError( + "the installed TDSE provider has no haCC consumer surface " + f"(missing {', '.join(missing)}); install Pyroraptor at the pinned " + f"revision {revision}" + ) + return hacc + + +def _hacc_fields( # noqa: PLR0917 - called positionally + hacc: _Native, + atom: _Native, + basis: _Native, + gauge: _Native, + cut_energy: float, + max_step_au: float, +) -> dict[str, object]: + """Convert provider haCC objects into the numeric fields of a ``HaccBasis``. + + - ``atom``, ``basis``, ``gauge``: provider objects, or the records of them a caller + sent. + - ``cut_energy``: spectral cut in J. + - ``max_step_au``: RK4 step ceiling in atomic units of time. + - Returns: every ``HaccBasis`` field but ``source`` and ``preset``. + """ + if _is(atom, hacc.Argon): + raise MaterialError( + "argon TDCIS is not representable as a HaccBasis: the haCC target is " + "helium only" + ) + if not _is(atom, hacc.Helium): + raise MaterialError("a HaccBasis needs a provider hacc.Helium atom") + if atom.ions not in (1, 2): + raise MaterialError( + f"Helium(ions={atom.ions}) is not representable: HaccBasis admits " + "haCC(1) (ions=1) and haCC(5) (ions=2)" + ) + if _is(gauge, hacc.MixedGauge): + gauge_radius: float | None = float(gauge.radius) + elif _is(gauge, hacc.VelocityGauge): + gauge_radius = None + else: + raise MaterialError("a HaccBasis gauge must be MixedGauge or VelocityGauge") + angular = basis.angular + limit = "particle" if angular.particle is not None else "total" + scaling = basis.scaling + return { + "ions": atom.ions, + "neutral_square": atom.neutral_square, + "radial_edges": tuple(float(edge) for edge in basis.radial.edges), + "radial_orders": tuple(basis.radial.orders), + "angular_limit": limit, + "angular_order": angular.particle if limit == "particle" else angular.total, + "scaling_radius": float(scaling.radius), + "scaling_angle": float(scaling.angle), + "infinite_order": scaling.infinite_order, + "decay": float(scaling.decay), + "gauge_radius": gauge_radius, + "cut_energy": float(cut_energy), + "max_step_au": float(max_step_au), + } + + +def _numeric_fields(basis: HaccBasis) -> dict[str, object]: + """Return a ``HaccBasis``'s fields without its provenance and preset identity.""" + record = asdict(basis) + del record["source"], record["preset"] + return record + + +def _hacc_objects( + provider: _Provider, hacc: _Native, basis: HaccBasis +) -> tuple[_Native, _Native, _Native]: + """Build the provider's atom, basis and gauge from a ``HaccBasis``.""" + angular = ( + hacc.AngularLimit(particle=basis.angular_order) + if basis.angular_limit == "particle" + else hacc.AngularLimit(total=basis.angular_order) + ) + atom = hacc.Helium(ions=basis.ions, neutral_square=basis.neutral_square) + native = hacc.Basis( + provider.RadialBasis(basis.radial_edges, basis.radial_orders), + provider.ComplexScaling( + basis.scaling_radius, + basis.scaling_angle, + basis.infinite_order, + basis.decay, + ), + angular, + ) + gauge = ( + hacc.VelocityGauge() + if basis.gauge_radius is None + else hacc.MixedGauge(basis.gauge_radius) + ) + return atom, native, gauge + + +def check_info( + info: _Native, + *, + rows: int, + samples: int, + spacing: float, + outputs: str | None, + cut_energy: float, + closure: bool, + preset: bool, + model_match: bool, +) -> None: + """Require a haCC plan's or result's ``info`` to describe exactly its model. + + - ``model_match``: whether ``info``'s atom, basis and gauge equal the model's, as + the provider compares them (in the worker). + - ``preset``: whether the model is a qualified preset, which must close. + """ + if ( + info.rows != rows + or info.samples != samples + or info.spacing != spacing + or info.shadow_order != 1 + or info.outputs != outputs + or not model_match + or info.cut_energy != cut_energy + ): + raise BackendError( + "haCC plan metadata disagree with the prepared model or contract" + ) + if bool(info.total_current_qualified) != closure: + if preset: + raise BackendError( + "provider contract violation: a haCC preset reports no " + "escaped-electron closure" + ) + raise BackendError("haCC plan closure flag disagrees with its output mode") + + +def _rebuild[T](kind: type[T], record: object) -> T: + """Return a solprop dataclass from the record of it the binding sent.""" + values = vars(record) if isinstance(record, Record) else cast("dict", record) + return kind(**{item.name: values[item.name] for item in fields(kind) if item.init}) # pyright: ignore[reportArgumentType] + + +@dataclass +class _HaccModel: + """One prepared haCC model and the output mode its plans use (SEP 20, C1). + + The model is entered into the adapter's stack first, so it closes after every plan + built on it. + """ + + hacc: _Native + basis: HaccBasis + atom: _Native + native_basis: _Native + gauge: _Native + preset: _Native + model: _Native + outputs: Literal["full", "total_current"] | None + closure: bool | None + + def plan(self, *, rows: int, spacing: float, samples: int) -> _Native: + """Construct one response plan, choosing the native output mode once. + + A custom mixed-gauge basis is first asked for the compact closed current; the + provider refuses it with ``ValueError`` when the gauge radius admits no closure, + and the full record is then used, as for the velocity gauge. For a full archive + every plan is full, and a custom mixed-gauge basis takes its closure from the + first plan's ``total_current_qualified``. + """ + arguments = { + "rows": rows, + "spacing": float(spacing), + "samples": samples, + "max_step_au": float(self.basis.max_step_au), + "field": "electric_field", + "shadow_order": 1, + "norm_growth_limit": HACC_NORM_GROWTH_LIMIT, + } + if self.outputs is None: + try: + plan = self.hacc.ResponsePlan( + self.model, outputs="total_current", **arguments + ) + except ValueError as error: + if "closure" not in str(error): + raise + self.outputs = "full" + self.closure = False + else: + self.outputs = "total_current" + self.closure = True + return plan + plan = self.hacc.ResponsePlan(self.model, outputs=self.outputs, **arguments) + if self.closure is None: + self.closure = bool(plan.info.total_current_qualified) + return plan + + def matches(self, info: _Native) -> bool: + """Whether ``info`` names this model's atom, basis and gauge.""" + return bool( + info.atom == self.atom + and info.basis == self.native_basis + and info.gauge == self.gauge + ) + + def check(self, plan: _Native, *, rows: int, samples: int, spacing: float) -> None: + """Check a plan this model built.""" + check_info( + plan.info, + rows=rows, + samples=samples, + spacing=spacing, + outputs=self.outputs, + cut_energy=self.basis.cut_energy, + closure=self.closure is True, + preset=self.preset is not None, + model_match=self.matches(plan.info), + ) + + def record(self) -> dict[str, object]: + """Return what the binding needs of the model: mode, closure, preset.""" + preset = self.preset + return { + "outputs": self.outputs, + "closure": self.closure, + "preset": None + if preset is None + else {name: getattr(preset, name) for name in _PRESET_FIELDS}, + } + + +class PyroraptorAdapter: + """Serve the TDSE binding's requests with Pyroraptor, inside the worker. + + - ``provider``: the provider module; None loads ``pyroraptor`` (the manufactured + provider of the tests is passed here). + """ + + def __init__(self, provider: _Provider | None = None) -> None: + self._provider = _load_provider() if provider is None else provider + self._stack = ExitStack() + self._plans: dict[int, _Native] = {} + self._hacc: _HaccModel | None = None + self._request: Mapping[str, Any] = {} + self._diagnostic: tuple[ExitStack, _Native] | None = None + + # -- the worker's protocol ------------------------------------------------------ + + def identity(self) -> Mapping[str, object]: + """Return the provider's build record.""" + return _identity(self._provider) + + def classify(self, error: BaseException) -> str: + """Classify a provider-side exception for the failure record (SEP 20, 5.6). + + ``native`` for a provider ``DrivenError``, ``contract`` for solprop's own + refusal, ``interface`` for a ``ValueError`` or ``TypeError``, ``unexpected`` + otherwise. + """ + if isinstance(error, self._provider.DrivenError): + return "native" + if isinstance(error, SolpropError): + return "contract" + if isinstance(error, ValueError | TypeError): + return "interface" + return "unexpected" + + def computed(self, value: object) -> Mapping[str, object]: + """Return a row outcome's ``ok`` and a shadow's components, beside fields.""" + if hasattr(value, "failure") and hasattr(value, "ok"): + return {"ok": bool(cast("Any", value).ok)} + component = getattr(value, "component", None) + orders = getattr(value, "orders", None) + if not callable(component) or not isinstance(orders, tuple): + return {} + found: dict[str, object] = {} + for quantity in _COMPONENTS: + for order in cast("tuple[int, ...]", orders): + try: + found[f"{quantity}:{order}"] = np.asarray( + component(quantity, order) + ) + except Exception as error: + found[f"{quantity}:{order}"] = { + "error": str(error), + "type": type(error).__name__, + } + return {"components": found} + + def handle(self, kind: str, value: object) -> object: + """Serve one request of the binding.""" + request = cast("Mapping[str, Any]", value) + handlers: dict[str, Callable[[Mapping[str, Any]], object]] = { + "basis_from_preset": self._basis_from_preset, + "hacc_preset": self._hacc_preset, + "hacc_basis": self._hacc_basis, + "prepare": self._prepare, + "build_plans": self._build_plans, + "evaluate": self._evaluate, + "diagnostic_plan": self._diagnostic_plan, + "diagnostic_evaluate": self._diagnostic_evaluate, + "diagnostic_close": self._diagnostic_close, + } + if kind not in handlers: + raise BackendError(f"the TDSE provider adapter has no request {kind!r}") + return handlers[kind](request) + + def close(self) -> None: + """Close any diagnostic plan, then the plans before the setup they use.""" + self._diagnostic_close({}) + self._stack.close() + + # -- queries -------------------------------------------------------------------- + + def _basis_from_preset(self, request: Mapping[str, Any]) -> object: + """Return a provider preset's SAE basis fields, provenance apart.""" + selected = self._provider.solve_preset(request["target"], request["preset"]) + basis = selected.discretization + scaling = basis.scaling + return { + "name": selected.name, + "radial_edges": basis.radial.edges, + "radial_orders": basis.radial.orders, + "lmax": basis.angular.lmax, + "cut_energy": selected.cut_energy, + "provider_source": selected.source, + "m": basis.angular.m, + "potential": basis.model.name, + "potential_version": basis.model.version, + "scaling_radius": None if scaling is None else scaling.radius, + "scaling_angle": None if scaling is None else scaling.angle, + "infinite_order": None if scaling is None else scaling.infinite_order, + "decay": None if scaling is None else scaling.decay, + "cuts": basis.cuts, + } + + def _hacc_preset(self, request: Mapping[str, Any]) -> object: + """Return a qualified haCC preset's basis fields and its records.""" + hacc = _hacc_surface(self._provider, request["revision"]) + preset = hacc.helium_preset(request["ions"], request["variant"]) + return { + "fields": _hacc_fields( + hacc, + preset.atom, + preset.basis, + preset.gauge, + preset.cut_energy, + preset.max_step_au, + ), + "preset": {name: getattr(preset, name) for name in _PRESET_FIELDS}, + } + + def _hacc_basis(self, request: Mapping[str, Any]) -> object: + """Return the basis fields of a caller's provider objects, sent as records.""" + hacc = _hacc_surface(self._provider, request["revision"]) + return _hacc_fields( + hacc, + request["atom"], + request["basis"], + request["gauge"], + request["cut_energy"], + request["max_step_au"], + ) + + # -- a run ---------------------------------------------------------------------- + + def _prepare(self, request: Mapping[str, Any]) -> object: + """Prepare the model and, unless deferred, its plans; admit the memory. + + - ``request``: the descriptor's plain records, the chunk sizes, the memory the + binding already holds and its limit. + """ + self._request = request + memory = dict(request["memory"]) + if request["model"] == "hacc": + model = self._prepare_hacc(_rebuild(HaccBasis, request["basis"])) + memory.update(self._model_memory(model)) + # Admit the model's measured ledger before any plan is built (C7). + _admit_memory(memory, request["memory_limit"]) + if not request["defer_plans"]: + memory.update(self._build_hacc_plans()) + else: + memory.update(self._prepare_sae(_rebuild(TDSEBasis, request["basis"]))) + _admit_memory(memory, request["memory_limit"]) + return self._prepared(memory) + + def _prepared(self, memory: Mapping[str, int]) -> dict[str, object]: + """Return the measured memory, the plan records and the haCC model's record.""" + return { + "memory": dict(memory), + "plans": { + str(rows): asdict(plan.info) for rows, plan in self._plans.items() + }, + "hacc": None if self._hacc is None else self._hacc.record(), + } + + def _prepare_sae(self, basis: TDSEBasis) -> dict[str, int]: + """Allocate shared SAE resources and one plan per distinct batch size.""" + provider, request = self._provider, self._request + model = provider.atomic_model(basis.potential) + if model.version != basis.potential_version: + raise BackendError( + "TDSE atomic potential revision differs from its descriptor" + ) + scaling = None + if basis.scaling_radius is not None: + scaling = provider.ComplexScaling( + basis.scaling_radius, + basis.scaling_angle, + basis.infinite_order, + basis.decay, + ) + discretization = provider.Discretization( + provider.RadialBasis(basis.radial_edges, basis.radial_orders), + provider.AngularBasis(basis.lmax, basis.m), + model, + scaling, + basis.cuts, + ) + operators = self._stack.enter_context(discretization.assemble()) + initial = provider.StateSelection(*request["initial"]) + setup = self._stack.enter_context( + provider.SpectralSetup( + operators, + provider.SpectralOptions( + cut_energy=basis.cut_energy, + states=(initial,), + growing=request["growing"], + growth_tolerance=request["growth_tolerance"], + window_duration=request["window_duration"], + accept_removed_component=request["accept_removed_component"], + ), + ) + ) + memory: dict[str, int] = {} + for label, ledger in ( + ("operators", operators.memory_ledger()), + ("setup", setup.memory_ledger()), + ): + for entry in ledger.entries: + memory[f"{label}.{entry.name}"] = entry.bytes + step = ( + request["max_atomic_step"] / provider.unit_scale("time").si_per_atomic_unit + ) + for rows in request["rows"]: + plan = self._stack.enter_context( + provider.ResponsePlan( + setup, + rows=rows, + spacing=request["spacing"], + samples=request["samples"], + max_step_au=step, + initial=initial, + field="electric_field", + shadow_order=request["shadow_order"], + complement=request["complement"], + outputs=request["outputs"], + ) + ) + self._plans[rows] = plan + for entry in plan.memory_ledger().entries: + if not entry.shared: + memory[f"plan.{rows}.{entry.name}"] = entry.bytes + return memory + + def _prepare_hacc(self, basis: HaccBasis) -> _HaccModel: + """Prepare the haCC model, verifying a preset against its record.""" + request = self._request + if request["complement"] == "vendor_blas": + raise BackendError( + "haCC response plans have no complement option; use the default " + "TDSEExecution(complement='deterministic'), recorded as not " + "applicable" + ) + hacc = _hacc_surface(self._provider, request["revision"]) + atom, native_basis, gauge = _hacc_objects(self._provider, hacc, basis) + preset: _Native = None + if basis.preset is not None: + preset = self._verified_preset(hacc, basis) + prepared = preset.prepare() + else: + prepared = hacc.prepare( + atom, native_basis, gauge=gauge, cut_energy=basis.cut_energy + ) + model = self._stack.enter_context(prepared) + # The native output mode: compact with the escaped-electron closure, full + # without it (the provider refuses compact order one then, C3), and full + # throughout for a full archive. Whether the plans carry the closure is known + # beforehand for a preset (always) and for the velocity gauge (never); a + # custom mixed gauge learns it from its first plan. + outputs: Literal["full", "total_current"] | None = ( + "total_current" if basis.preset is not None else None + ) + closure: bool | None = True if basis.preset is not None else None + if basis.gauge_radius is None: + outputs, closure = "full", False + if request["archive_outputs"] == "full": + outputs = "full" + self._hacc = _HaccModel( + hacc, basis, atom, native_basis, gauge, preset, model, outputs, closure + ) + return self._hacc + + def _verified_preset(self, hacc: _Native, basis: HaccBasis) -> _Native: + """Refuse a preset the installed provider does not reproduce.""" + preset = hacc.helium_preset(basis.ions, basis.variant) + if preset.identity != basis.preset: + raise BackendError( + f"the installed provider's haCC preset is {preset.identity!r}, not " + f"the expected {basis.preset!r}; regenerate the record" + ) + found = _hacc_fields( + hacc, + preset.atom, + preset.basis, + preset.gauge, + preset.cut_energy, + preset.max_step_au, + ) + if found != _numeric_fields(basis): + raise BackendError( + f"the installed provider's preset {preset.identity!r} does not " + "reproduce the record's fields" + ) + if self._request["device"] != "cpu" and not preset.device_admissible: + raise BackendError( + f"the haCC preset {preset.identity!r} is not admitted on the " + f"{self._request['device']} device route" + ) + return preset + + @staticmethod + def _model_memory(model: _HaccModel) -> dict[str, int]: + """Return the prepared model's measured ledger.""" + return { + f"model.{entry.name}": entry.bytes + for entry in model.model.memory_ledger().entries + } + + def _build_hacc_plans(self) -> dict[str, int]: + """Build and check one haCC plan per chunk size; return their memory.""" + model, request = self._hacc, self._request + assert model is not None + spacing = float(request["spacing"]) + memory: dict[str, int] = {} + for rows in request["rows"]: + plan = self._stack.enter_context( + model.plan(rows=rows, spacing=spacing, samples=request["samples"]) + ) + model.check(plan, rows=rows, samples=request["samples"], spacing=spacing) + self._plans[rows] = plan + for entry in plan.memory_ledger().entries: + if not entry.shared: + memory[f"plan.{rows}.{entry.name}"] = entry.bytes + return memory + + def _build_plans(self, request: Mapping[str, Any]) -> object: + """Build a deferred haCC binding's plans and admit them with its memory.""" + memory = {**request["memory"], **self._build_hacc_plans()} + _admit_memory(memory, request["memory_limit"]) + return self._prepared(memory) + + def _kind(self, raw: object) -> str: + """Name a result's record type: ``model.grid_response`` or the compact one.""" + if self._hacc is not None: + namespace, prefix = self._hacc.hacc, "hacc" + else: + namespace, prefix = self._provider, "sae" + if isinstance(raw, namespace.TotalCurrentResponse): + return f"{prefix}.total_current_response" + if isinstance(raw, namespace.GridResponse): + return f"{prefix}.grid_response" + return f"other:{type(raw).__qualname__}" + + def _result(self, raw: _Native) -> dict[str, object]: + """Return one atomic call's record as returned, its type and model match.""" + info = getattr(raw, "info", None) + match = None + if self._hacc is not None and info is not None: + try: + match = self._hacc.matches(info) + except AttributeError, TypeError, ValueError: + match = False + return {"kind": self._kind(raw), "raw": raw, "model_match": match} + + def _evaluate(self, request: Mapping[str, Any]) -> object: + """Make one atomic call from the prepared state on one chunk's drive.""" + return self._result(self._plans[request["rows"]].evaluate(request["drive"])) + + # -- the haCC consistency diagnostic -------------------------------------------- + + def _diagnostic_plan(self, request: Mapping[str, Any]) -> object: + """Build a temporary one-row plan on the protocol lattice; return its memory.""" + model = self._hacc + if model is None: + raise BackendError("the consistency diagnostic needs a haCC binding") + self._diagnostic_close({}) + stack = ExitStack() + try: + plan = stack.enter_context( + model.plan( + rows=1, spacing=request["spacing"], samples=request["samples"] + ) + ) + model.check( + plan, rows=1, samples=request["samples"], spacing=request["spacing"] + ) + except BaseException: + stack.close() + raise + self._diagnostic = (stack, plan) + return { + "memory": { + f"diagnostic.{entry.name}": entry.bytes + for entry in plan.memory_ledger().entries + if not entry.shared + }, + "hacc": model.record(), + } + + def _diagnostic_evaluate(self, request: Mapping[str, Any]) -> object: + """Make the diagnostic's one atomic call.""" + if self._diagnostic is None: + raise BackendError("no consistency diagnostic plan is open") + return self._result(self._diagnostic[1].evaluate(request["drive"])) + + def _diagnostic_close(self, request: Mapping[str, Any]) -> object: + """Close the temporary plan, if one is open.""" + del request + if self._diagnostic is not None: + stack, self._diagnostic = self._diagnostic[0], None + stack.close() + return None + + +def make_adapter() -> PyroraptorAdapter: + """Return the adapter the worker serves with (its entry point).""" + return PyroraptorAdapter() diff --git a/src/solprop/driver/simulation.py b/src/solprop/driver/simulation.py index 184e6773e..752ebbd4d 100644 --- a/src/solprop/driver/simulation.py +++ b/src/solprop/driver/simulation.py @@ -3196,6 +3196,7 @@ def _run( # noqa: PLR0915 - run's controls, and the backend it may be handed problem, _AtomicContext(handler, state, metadata), tdse_archive, + stop, ) backend = kept or select( self.settings.backend, @@ -3344,9 +3345,12 @@ def _prepare_atomic( problem: PropagationProblem, context: _AtomicContext, tdse_archive: TDSEArchive | None, + stop: StopFlag | None = None, ) -> tuple[TDSEBinding, TDSEArchiveWriter | None]: """Open a TDSE run's archive and prepare its binding. + The binding's waits on its provider worker poll `stop` (SEP 49). + - Returns: the binding and the open archive session, or None without a request. A failure closes the archive before it propagates. """ @@ -3375,6 +3379,7 @@ def _prepare_atomic( problem.transform.n_nodes, defer_plans=descriptor.model == "hacc", archive=archive, + stop=stop, ) except BaseException as error: _close_tdse_archive(archive, error) diff --git a/src/solprop/driver/tdse.py b/src/solprop/driver/tdse.py index b90722d39..d2c6c3d08 100644 --- a/src/solprop/driver/tdse.py +++ b/src/solprop/driver/tdse.py @@ -5,8 +5,11 @@ """Own the optional TDSE runtime behind the provider-free response seam. The provider's whole-electron currents and shadow dipoles remain unscaled SI -arrays. Source assembly applies the calibration and density separately. Native -imports, persistent plans, resource admission and call accounting live here. +arrays. Source assembly applies the calibration and density separately. Resource +admission, call accounting, the archive and the validation of every returned record +live here; the provider itself runs in a worker process of its own +(:mod:`solprop.driver._source_worker`, with :mod:`solprop.driver._pyroraptor` as its +adapter, SEP 49), and solprop's process never imports it. Two atomic models are bound (SEP 12, SEP 20): the single-active-electron (SAE) response plan, and the helium haCC response plan, either from one of the @@ -16,31 +19,43 @@ :class:`~solprop.translate.tdse.TDSEResult` always spans the full lattice. """ +import contextlib import dataclasses -import importlib import json import math -import sys +import os import warnings -from collections.abc import Callable, Mapping +from collections.abc import Mapping from contextlib import ExitStack from dataclasses import asdict, dataclass, is_dataclass -from importlib.metadata import PackageNotFoundError, distribution from typing import Any, Literal, Protocol, cast, runtime_checkable import numpy as np from jaxtyping import Float +from solprop import exceptions +from solprop._process import thread_budget from solprop.constants import AU_TIME, EPS0, C +from solprop.driver._pyroraptor import _admit_memory, check_info +from solprop.driver._source_worker import ( + CONTROL_LIMIT, + Record, + RemoteError, + Worker, + WorkerError, + worker_environment, +) from solprop.exceptions import ( BackendError, - DriverError, MaterialError, + RunTerminated, SolpropError, + StopRequested, TDSEArchiveError, TDSEDomainWarning, TDSEUnqualifiedModelWarning, ) +from solprop.log import StopFlag from solprop.nonlinear.tdse import ( HaccBasis, TDSEBasis, @@ -54,7 +69,6 @@ writer_memory, ) from solprop.translate.tdse import ( - HACC_NORM_GROWTH_LIMIT, AtomicCallBudget, AtomicCallLedger, AtomicWindowRecord, @@ -80,9 +94,9 @@ _PROVIDER_REVISION = "876b72d10b5286478125b842ea45548a5051b033" _Preset = Literal["production", "high_accuracy", "strong_field"] _HaccVariant = Literal["standard", "fast"] -# Native classes cannot occur in runtime annotations: optional-package-free test -# collection resolves every annotation through beartype. Confine their dynamic -# typing to this adapter; the seam's scientific records and arrays remain typed. +# The adapter module the worker runs (SEP 49 section 6.5). +_ADAPTER = "solprop.driver._pyroraptor" +# Native classes never reach this process; what the worker returns are records. _Native = Any # The archive role of each evaluation purpose (SEP 20 section 5.8); a predictor or # corrector of a rejected trial reads back as "rejected". @@ -92,47 +106,42 @@ "corrected": "corrector", "diagnostic": "diagnostic", } -# The haCC consumer surface of the provider (SEP 20 section 5.7, C1 to C5). -_HACC_SURFACE = ( - "AngularLimit", - "Argon", - "Basis", - "GridResponse", - "Helium", - "MixedGauge", - "ResponsePlan", - "TotalCurrentResponse", - "VelocityGauge", - "helium_preset", - "prepare", -) # A preset's calibration admission band: from 20 eV (hc / 20 eV = 62 nm), the # presets' below-threshold bound, to 1.6 um, the band the 800 nm qualifying drive and # run_short's grid occupy (SEP 20 section 5.1). Not an independent qualification. _PRESET_WAVELENGTH_RANGE = (62e-9, 1.6e-6) +# The preset records a run's provenance carries (the provider's evidence apart). +_PRESET_RECORD = ( + "identity", + "variant", + "step_margin", + "accuracy", + "box_sensitivity", + "gauge_sensitivity", + "max_peak_intensity", + "wavelength", + "window", + "device_admissible", +) @runtime_checkable -class _Provider(Protocol): - """Type the optional module at its single dynamic-import boundary.""" - - AngularBasis: Callable[..., _Native] - ComplexScaling: Callable[..., _Native] - Discretization: Callable[..., _Native] - DrivenError: type[Exception] - GridResponse: type - RadialBasis: Callable[..., _Native] - ResponsePlan: Callable[..., _Native] - SpectralOptions: Callable[..., _Native] - SpectralSetup: Callable[..., _Native] - StateSelection: Callable[..., _Native] - TotalCurrentResponse: type - atomic_model: Callable[[str], _Native] - build_info: Callable[[], _Native] - # The haCC namespace is looked up by name (`_hacc_surface`): an SAE-only - # provider is still a provider, and is refused only when haCC is asked for. - solve_preset: Callable[[str, _Preset], _Native] - unit_scale: Callable[[str], _Native] +class _Channel(Protocol): + """The worker the binding talks to, a ``_source_worker.Worker``.""" + + identity: Mapping[str, object] + + def request(self, kind: str, value: object = None) -> object: + """Send one request and return its reply.""" + ... + + def close(self) -> None: + """End the worker between calls.""" + ... + + def kill(self) -> None: + """End the worker at once.""" + ... def pinned_revision() -> str: @@ -146,68 +155,98 @@ def pinned_revision() -> str: return _PROVIDER_REVISION -def _load_provider() -> _Provider: - """Load the optional package only when an atomic operation is requested.""" - if sys.version_info[:2] != (3, 14): - raise BackendError("the pinned TDSE provider requires Python 3.14") +def _open_provider( + *, stop: StopFlag | None = None, limit: int = CONTROL_LIMIT +) -> _Channel: + """Start a worker running the provider's adapter and wait for its handshake. + + The worker gets the process's whole thread budget (SEP 49 section 6.6) and the + parent's environment less native's own OpenMP wait settings. + + - ``stop``: the run's flag, polled while the worker is awaited. + - ``limit``: the bound on any frame, bytes. + """ + environment = worker_environment( + os.environ, {"PYRORAPTOR_THREADS": str(thread_budget().threads)} + ) + worker = Worker(_ADAPTER, environment=environment, limit=limit, stop=stop) try: - return cast("_Provider", importlib.import_module("pyroraptor")) - except (ImportError, OSError) as error: - raise BackendError( - "TDSE response requires the optional pyroraptor dependency; " - "install the pinned Pyroraptor revision into the environment, " - "with a matching CPU, CUDA or HIP build" - ) from error + worker.start() + except RemoteError as error: + raise _local_error(error, "TDSE provider start failed") from error + return worker + + +def _local_error(error: RemoteError, context: str) -> SolpropError: + """Return the worker's exception as this process raises it. + + solprop's own exceptions keep their class and message; any other becomes a + :class:`BackendError` naming `context`, the message and its classification. + """ + module, _, name = error.type_name.rpartition(".") + kind = getattr(exceptions, name, None) if module == "solprop.exceptions" else None + if isinstance(kind, type) and issubclass(kind, SolpropError): + try: + local = kind(error.detail) + except TypeError: + pass + else: + local.add_note( + f"raised in the TDSE provider worker:\n{error.remote_traceback}" + ) + return local + wrapped = BackendError(f"{context}: {error.detail} [{error.classification}]") + wrapped.add_note(f"the worker's traceback:\n{error.remote_traceback}") + return wrapped def _build_record( - provider: _Provider, execution: TDSEExecution | None + identity: Mapping[str, object], execution: TDSEExecution | None ) -> dict[str, object]: - """Refuse stale native extensions and execution-space mismatches before setup. + """Refuse a stale provider or an execution-space mismatch the worker reported. + - ``identity``: the handshake's provider identity: its build and distribution. - ``execution``: required device family, or None when only resolving a preset. - Returns: serializable native build and installed distribution identities. """ - build = provider.build_info() - revision = build.source_revision + build = cast("Mapping[str, object]", identity["native"]) + revision = str(build["source_revision"]) if len(revision) < 7 or not _PROVIDER_REVISION.startswith(revision): raise BackendError( f"TDSE native source {revision!r} does not match pinned " f"{_PROVIDER_REVISION}" ) - space = build.execution_space.lower() + space = str(build["execution_space"]).lower() devices = {"cpu": {"openmp", "serial", "threads"}, "cuda": {"cuda"}, "hip": {"hip"}} if execution is not None and space not in devices[execution.device]: raise BackendError( f"TDSE requested device={execution.device!r}, but the native build uses " - f"{build.execution_space!r}" + f"{build['execution_space']!r}" ) record: dict[str, object] = { - "native": asdict(build), + "native": dict(build), "source_pin": _PROVIDER_REVISION, + "distribution": identity.get("distribution"), } - try: - package = distribution("pyroraptor") - except PackageNotFoundError: - record["distribution"] = None - else: - record["distribution"] = package.version - source = package.read_text("direct_url.json") - record["direct_url"] = None if source is None else json.loads(source) + if "direct_url" in identity: + record["direct_url"] = identity["direct_url"] return record -def _hacc_surface(provider: _Provider) -> _Native: - """Return the provider's haCC namespace, refusing a build without it.""" - hacc = getattr(provider, "hacc", None) - missing = [name for name in _HACC_SURFACE if not hasattr(hacc, name)] - if missing: - raise BackendError( - "the installed TDSE provider has no haCC consumer surface " - f"(missing {', '.join(missing)}); install Pyroraptor at the pinned " - f"revision {_PROVIDER_REVISION}" - ) - return hacc +def _query(kind: str, request: Mapping[str, object], *, checked: bool) -> Any: + """Ask a short-lived worker one question (a preset, a basis) and end it. + + - ``checked``: refuse a stale provider first, as resolving a preset does. + """ + worker = _open_provider() + try: + if checked: + _build_record(worker.identity, None) + return worker.request(kind, {**request, "revision": _PROVIDER_REVISION}) + except RemoteError as error: + raise _local_error(error, "TDSE provider query failed") from error + finally: + worker.close() def basis_from_preset(target: str, preset: _Preset) -> TDSEBasis: @@ -239,127 +278,18 @@ def basis_from_preset(target: str, preset: _Preset) -> TDSEBasis: "strong_field", ): raise MaterialError("TDSE requires helium and a supported provider preset") - provider = _load_provider() - _build_record(provider, None) - selected = provider.solve_preset(target, preset) - basis = selected.discretization - scaling = basis.scaling + fields = dict( + _query("basis_from_preset", {"target": target, "preset": preset}, checked=True) + ) + provider_source = fields.pop("provider_source") return TDSEBasis( - name=selected.name, - radial_edges=basis.radial.edges, - radial_orders=basis.radial.orders, - lmax=basis.angular.lmax, - cut_energy=selected.cut_energy, - source=f"pyroraptor@{_PROVIDER_REVISION}: {selected.source}", - m=basis.angular.m, - potential=basis.model.name, - potential_version=basis.model.version, - scaling_radius=None if scaling is None else scaling.radius, - scaling_angle=None if scaling is None else scaling.angle, - infinite_order=None if scaling is None else scaling.infinite_order, - decay=None if scaling is None else scaling.decay, - cuts=basis.cuts, + **fields, source=f"pyroraptor@{_PROVIDER_REVISION}: {provider_source}" ) # ------------------------------------------------------------------- haCC records -def _hacc_fields( # noqa: PLR0917 - called positionally - hacc: _Native, - atom: _Native, - basis: _Native, - gauge: _Native, - cut_energy: float, - max_step_au: float, -) -> dict[str, object]: - """Convert provider haCC objects into the numeric fields of a ``HaccBasis``. - - - ``basis``: ``hacc.Basis``: radial axis, irECS and angular limit, SI. - - ``gauge``: ``hacc.MixedGauge`` or ``hacc.VelocityGauge``. - - ``cut_energy``: spectral cut in J. - - ``max_step_au``: RK4 step ceiling in atomic units of time. - - Returns: every ``HaccBasis`` field but ``source`` and ``preset``. - """ - if isinstance(atom, hacc.Argon): - raise MaterialError( - "argon TDCIS is not representable as a HaccBasis: the haCC target is " - "helium only" - ) - if not isinstance(atom, hacc.Helium): - raise MaterialError("a HaccBasis needs a provider hacc.Helium atom") - if atom.ions not in (1, 2): - raise MaterialError( - f"Helium(ions={atom.ions}) is not representable: HaccBasis admits " - "haCC(1) (ions=1) and haCC(5) (ions=2)" - ) - if isinstance(gauge, hacc.MixedGauge): - gauge_radius: float | None = float(gauge.radius) - elif isinstance(gauge, hacc.VelocityGauge): - gauge_radius = None - else: - raise MaterialError("a HaccBasis gauge must be MixedGauge or VelocityGauge") - angular = basis.angular - limit = "particle" if angular.particle is not None else "total" - scaling = basis.scaling - return { - "ions": atom.ions, - "neutral_square": atom.neutral_square, - "radial_edges": tuple(float(edge) for edge in basis.radial.edges), - "radial_orders": tuple(basis.radial.orders), - "angular_limit": limit, - "angular_order": angular.particle if limit == "particle" else angular.total, - "scaling_radius": float(scaling.radius), - "scaling_angle": float(scaling.angle), - "infinite_order": scaling.infinite_order, - "decay": float(scaling.decay), - "gauge_radius": gauge_radius, - "cut_energy": float(cut_energy), - "max_step_au": float(max_step_au), - } - - -def _numeric_fields(basis: HaccBasis) -> dict[str, object]: - """Return a ``HaccBasis``'s fields without its provenance and preset identity. - - - Returns: the model and numerical fields compared against the provider. - """ - record = asdict(basis) - del record["source"], record["preset"] - return record - - -def _hacc_objects( - provider: _Provider, hacc: _Native, basis: HaccBasis -) -> tuple[_Native, _Native, _Native]: - """Build the provider's atom, basis and gauge from a ``HaccBasis``. - - - Returns: ``hacc.Helium``, ``hacc.Basis`` and the gauge. - """ - angular = ( - hacc.AngularLimit(particle=basis.angular_order) - if basis.angular_limit == "particle" - else hacc.AngularLimit(total=basis.angular_order) - ) - atom = hacc.Helium(ions=basis.ions, neutral_square=basis.neutral_square) - native = hacc.Basis( - provider.RadialBasis(basis.radial_edges, basis.radial_orders), - provider.ComplexScaling( - basis.scaling_radius, - basis.scaling_angle, - basis.infinite_order, - basis.decay, - ), - angular, - ) - gauge = ( - hacc.VelocityGauge() - if basis.gauge_radius is None - else hacc.MixedGauge(basis.gauge_radius) - ) - return atom, native, gauge - - @dataclass(frozen=True) class HaccPreset: """Hold a provider's qualified helium haCC preset as solprop records (SEP 20). @@ -506,49 +436,37 @@ def hacc_preset(ions: Literal[1, 2], variant: _HaccVariant = "standard") -> Hacc raise MaterialError("hacc_preset needs ions=1 (haCC(1)) or ions=2 (haCC(5))") if not isinstance(variant, str) or variant not in ("standard", "fast"): raise MaterialError("hacc_preset needs variant='standard' or variant='fast'") - provider = _load_provider() - _build_record(provider, None) - hacc = _hacc_surface(provider) - preset = hacc.helium_preset(ions, variant) - if preset.variant != variant: + reply = _query("hacc_preset", {"ions": ions, "variant": variant}, checked=True) + preset = cast("Mapping[str, Any]", reply["preset"]) + if preset["variant"] != variant: raise BackendError( - f"the provider returned the {preset.variant!r} haCC preset for the " + f"the provider returned the {preset['variant']!r} haCC preset for the " f"{variant!r} variant" ) - source = f"pyroraptor@{_PROVIDER_REVISION}: {preset.source}" + source = f"pyroraptor@{_PROVIDER_REVISION}: {preset['source']}" basis = HaccBasis( - **cast( - "dict[str, Any]", - _hacc_fields( - hacc, - preset.atom, - preset.basis, - preset.gauge, - preset.cut_energy, - preset.max_step_au, - ), - ), + **cast("dict[str, Any]", reply["fields"]), source=source, - preset=preset.identity, + preset=preset["identity"], ) if basis.variant != variant: raise BackendError( - f"the provider's {variant!r} preset identity {preset.identity!r} does " + f"the provider's {variant!r} preset identity {preset['identity']!r} does " "not name that variant" ) return HaccPreset( basis=basis, variant=variant, - step_margin=float(preset.step_margin), + step_margin=float(preset["step_margin"]), max_atomic_step=basis.max_step_au * AU_TIME, - accuracy=float(preset.accuracy), - box_sensitivity=float(preset.box_sensitivity), - gauge_sensitivity=float(preset.gauge_sensitivity), - max_peak_intensity=float(preset.max_peak_intensity), - wavelength=float(preset.wavelength), - window=float(preset.window), - device_admissible=bool(preset.device_admissible), - identity=preset.identity, + accuracy=float(preset["accuracy"]), + box_sensitivity=float(preset["box_sensitivity"]), + gauge_sensitivity=float(preset["gauge_sensitivity"]), + max_peak_intensity=float(preset["max_peak_intensity"]), + wavelength=float(preset["wavelength"]), + window=float(preset["window"]), + device_admissible=bool(preset["device_admissible"]), + identity=preset["identity"], source=source, ) @@ -598,8 +516,19 @@ def hacc_basis( BackendError If the optional provider is missing or has no haCC surface. """ - hacc = _hacc_surface(_load_provider()) - fields = _hacc_fields(hacc, atom, basis, gauge, cut_energy, max_step_au) + # The objects travel to the worker as records of their fields, and are checked + # there against the provider's own classes (SEP 49 section 6.5). + fields = _query( + "hacc_basis", + { + "atom": atom, + "basis": basis, + "gauge": gauge, + "cut_energy": cut_energy, + "max_step_au": max_step_au, + }, + checked=False, + ) return HaccBasis(**cast("dict[str, Any]", fields), source=source) @@ -650,14 +579,20 @@ def _estimated_memory( descriptor's; None takes ``descriptor.outputs``. - Returns: named conservative atomic storage estimates in bytes; vendor setup workspaces and allocator overhead are not hard bounded. The haCC provider offers - no estimate before preparation (SEP 20, C7), so only its host histories are - estimated; its model and plans are admitted from their measured ledgers. + no estimate before preparation (SEP 20, C7), so only its host histories and the + transport copies are estimated; its model and plans are admitted from their + measured ledgers. """ samples = _atomic_samples(descriptor) outputs = descriptor.outputs if native_outputs is None else native_outputs histories = 48 if outputs == "full" else 20 + transport = _transport_bytes( + descriptor, max(chunk.rows for chunk in chunks), outputs + ) host = { - "estimate.host_histories": 8 * histories * chunks[-1].stop * descriptor.samples + "estimate.host_histories": 8 * histories * chunks[-1].stop * descriptor.samples, + # A call's copies on the pipe, in this process and in the worker (SEP 49). + "estimate.transport": 2 * transport, } basis = descriptor.response.basis if not isinstance(basis, TDSEBasis): @@ -679,23 +614,13 @@ def _estimated_memory( } -def _admit_memory(ledger: Mapping[str, int], limit: int | None) -> None: - """Apply the declared atomic byte limit. - - - ``ledger``: named persistent allocations or estimates in bytes. - - ``limit``: positive byte ceiling, or None for an unrestricted allocation. - """ - total = sum(ledger.values()) - if limit is not None and total > limit: - raise DriverError( - f"TDSE persistent data requires {total} bytes, exceeding {limit} bytes" - ) - - def _json_default(value: object) -> object: """Convert a public native record or array into JSON-compatible containers.""" if is_dataclass(value) and not isinstance(value, type): return asdict(value) + if isinstance(value, Record): + # A provider record from the worker: its fields, as the dataclass gave them. + return dict(vars(value)) if isinstance(value, np.ndarray): return value.tolist() if isinstance(value, np.generic): @@ -768,15 +693,19 @@ def _row_chunks(execution: TDSEExecution, n_nodes: int) -> tuple[_Chunk, ...]: ) -def _classify(error: BaseException, provider: _Provider) -> str: - """Classify a provider-side exception for the failure record (SEP 20, 5.6). +def _classify(error: BaseException) -> str: + """Classify an evaluation's failure for its record (SEP 20, 5.6). - - Returns: ``native`` for a provider ``DrivenError``, ``interface`` for a - ``ValueError`` or ``TypeError`` from the provider, ``contract`` for a solprop - validation of the returned record, ``unexpected`` otherwise. + - Returns: the worker's classification of a provider exception (``native``, + ``interface``, ``contract`` or ``unexpected``); ``worker`` for a worker that + exited or broke the protocol; for this process's own validation, ``contract`` + for a solprop refusal, ``interface`` for a ``ValueError`` or ``TypeError`` and + ``unexpected`` otherwise. """ - if isinstance(error, provider.DrivenError): - return "native" + if isinstance(error, RemoteError): + return error.classification + if isinstance(error, WorkerError): + return "worker" if isinstance(error, SolpropError): return "contract" if isinstance(error, ValueError | TypeError): @@ -784,6 +713,33 @@ def _classify(error: BaseException, provider: _Provider) -> str: return "unexpected" +def _described(error: BaseException) -> tuple[str, str]: + """Return an error's type name and message, the worker's own for a remote one.""" + if isinstance(error, RemoteError): + return error.type_name.rpartition(".")[2], error.detail + return type(error).__name__, str(error) + + +def _frame_limit(descriptor: TDSEDescriptor, chunks: tuple[_Chunk, ...]) -> int: + """Bound one worker frame by the prepared shapes (SEP 49 section 6.5). + + The largest array either way is a chunk's shadow record, three float64 orders + of every row on the lattice; four orders' worth, twice over, is the bound. + """ + rows = max(chunk.rows for chunk in chunks) + return max(CONTROL_LIMIT, 2 * 4 * 8 * rows * descriptor.samples) + + +def _transport_bytes(descriptor: TDSEDescriptor, rows: int, outputs: str) -> int: + """Return one process's transport copies of the largest chunk's call. + + The drive twice (its frame is built whole before it is written) and the result + once (its arrays are read into their own buffers), in bytes. + """ + histories = 48 if outputs == "full" else 20 + return 8 * rows * _atomic_samples(descriptor) * (2 + histories) + + def _peak_intensity(fields: Float[np.ndarray, "n_nodes n_samples"]) -> list[float]: """Return each row's peak cycle-averaged intensity ``eps0 c E_max^2 / 2``. @@ -845,9 +801,13 @@ def prepare_binding( *, defer_plans: bool = False, archive: TDSEArchiveWriter | None = None, + stop: StopFlag | None = None, ) -> TDSEBinding: """Prepare persistent, stably chunked atomic plans for one response lattice. + The provider runs in a worker process the binding starts here and owns until it + is closed (SEP 49): every wait on it is cancelled by Ctrl-C, SIGTERM and `stop`. + Parameters ---------- descriptor : TDSEDescriptor @@ -867,6 +827,9 @@ def prepare_binding( written to it, input first and raw result before validation; a full archive builds the plans with the provider's full record. The caller owns and closes it. Its buffers are admitted with the binding's memory. + stop : solprop.log.StopFlag or None, optional + The run's stop flag, polled while an atomic call is awaited; setting it ends + the call at once, the worker killed, with :class:`StopRequested`. Returns ------- @@ -893,174 +856,65 @@ def prepare_binding( n_nodes, defer_plans=defer_plans, archive=archive, + stop=stop, ) # ------------------------------------------------------------------ haCC model -class _HaccModel: - """Prepare one haCC model, verify it and build its response plans. +@dataclass +class _HaccView: + """The binding's record of the haCC model the worker prepared (SEP 20, C1). - The model is entered into the caller's stack first, so it closes after every - plan built on it (SEP 20, C1). A preset is checked against the installed - provider's identity and fields; a custom basis is admitted as unqualified. - - - ``archive``: ``"full"`` builds every plan with the provider's full record for a - full archive; the source still reads the same total and order-one currents. + Attributes + ---------- + basis : HaccBasis + The model's record. + outputs : {'full', 'total_current'} or None + The plans' native output mode; None for a custom mixed-gauge basis before + its first plan decides it. + closure_known : bool or None + Whether the plans carry the escaped-electron closure; None until known. + preset : mapping or None + The verified preset's records (its identity, variant, accuracy and domain), or + None for a custom basis. """ - def __init__( - self, - provider: _Provider, - basis: HaccBasis, - execution: TDSEExecution, - stack: ExitStack, - *, - archive: ArchiveOutputs = "required", - ) -> None: - if execution.complement == "vendor_blas": - raise BackendError( - "haCC response plans have no complement option; use the default " - "TDSEExecution(complement='deterministic'), recorded as not " - "applicable" - ) - self.hacc = _hacc_surface(provider) - self.basis = basis - self.atom, self.native_basis, self.gauge = _hacc_objects( - provider, self.hacc, basis - ) - self.preset: _Native = None - if basis.preset is not None: - self.preset = self._verified_preset(execution) - prepared = self.preset.prepare() - else: - prepared = self.hacc.prepare( - self.atom, - self.native_basis, - gauge=self.gauge, - cut_energy=basis.cut_energy, - ) - self.model = stack.enter_context(prepared) - self.memory = { - f"model.{entry.name}": entry.bytes - for entry in self.model.memory_ledger().entries - } - # The native output mode: compact with the escaped-electron closure, - # full without it (the provider refuses compact order one then, C3), and - # full throughout for a full archive. Whether the plans carry the closure - # is known beforehand for a preset (always) and for the velocity gauge - # (never); a custom mixed gauge learns it from its first plan. - self.outputs: Literal["full", "total_current"] | None = ( - "total_current" if basis.preset is not None else None - ) - self._closure: bool | None = True if basis.preset is not None else None - if basis.gauge_radius is None: - self.outputs = "full" - self._closure = False - if archive == "full": - self.outputs = "full" - - def _verified_preset(self, execution: TDSEExecution) -> _Native: - """Refuse a preset the installed provider does not reproduce.""" - preset = self.hacc.helium_preset(self.basis.ions, self.basis.variant) - if preset.identity != self.basis.preset: - raise BackendError( - f"the installed provider's haCC preset is {preset.identity!r}, not " - f"the expected {self.basis.preset!r}; regenerate the record" - ) - fields = _hacc_fields( - self.hacc, - preset.atom, - preset.basis, - preset.gauge, - preset.cut_energy, - preset.max_step_au, - ) - if fields != _numeric_fields(self.basis): - raise BackendError( - f"the installed provider's preset {preset.identity!r} does not " - "reproduce the record's fields" - ) - if execution.device != "cpu" and not preset.device_admissible: - raise BackendError( - f"the haCC preset {preset.identity!r} is not admitted on the " - f"{execution.device} device route" - ) - return preset - - @property - def closure(self) -> bool: - """Return whether the plans carry the escaped-electron closure.""" - return self._closure is True + basis: HaccBasis + outputs: str | None + closure_known: bool | None + preset: Mapping[str, Any] | None - def plan(self, *, rows: int, spacing: float, samples: int) -> _Native: - """Construct one response plan, choosing the native output mode once. + @classmethod + def from_reply(cls, basis: HaccBasis, reply: Mapping[str, Any]) -> "_HaccView": + """Return the view of the model a ``prepare`` reply describes.""" + return cls(basis, reply["outputs"], reply["closure"], reply["preset"]) - A custom mixed-gauge basis is first asked for the compact closed current; - the provider refuses it with ``ValueError`` when the gauge radius admits no - closure, and the full record is then used, as for the velocity gauge. For - a full archive every plan is full, and a custom mixed-gauge basis takes its - closure from the first plan's ``total_current_qualified``. + def update(self, reply: Mapping[str, Any]) -> None: + """Take the output mode and closure a plan decided, from a reply's record.""" + self.outputs, self.closure_known = reply["outputs"], reply["closure"] - - ``spacing``: response spacing in s. - """ - arguments = { - "rows": rows, - "spacing": float(spacing), - "samples": samples, - "max_step_au": float(self.basis.max_step_au), - "field": "electric_field", - "shadow_order": 1, - "norm_growth_limit": HACC_NORM_GROWTH_LIMIT, - } - if self.outputs is None: - try: - plan = self.hacc.ResponsePlan( - self.model, outputs="total_current", **arguments - ) - except ValueError as error: - if "closure" not in str(error): - raise - self.outputs = "full" - self._closure = False - else: - self.outputs = "total_current" - self._closure = True - return plan - plan = self.hacc.ResponsePlan(self.model, outputs=self.outputs, **arguments) - if self._closure is None: - self._closure = bool(plan.info.total_current_qualified) - return plan + @property + def closure(self) -> bool: + """Whether the plans carry the escaped-electron closure.""" + return self.closure_known is True def check_info( - self, info: _Native, *, rows: int, samples: int, spacing: float + self, raw: _Native, *, rows: int, samples: int, spacing: float ) -> None: - """Require a plan or result's ``info`` to describe exactly this model. - - - ``spacing``: expected spacing in s. - """ - if ( - info.rows != rows - or info.samples != samples - or info.spacing != spacing - or info.shadow_order != 1 - or info.outputs != self.outputs - or info.atom != self.atom - or info.basis != self.native_basis - or info.gauge != self.gauge - or info.cut_energy != self.basis.cut_energy - ): - raise BackendError( - "haCC plan metadata disagree with the prepared model or contract" - ) - if bool(info.total_current_qualified) != self.closure: - if self.preset is not None: - raise BackendError( - "provider contract violation: a haCC preset reports no " - "escaped-electron closure" - ) - raise BackendError("haCC plan closure flag disagrees with its output mode") + """Require a result's ``info`` to describe exactly this model.""" + check_info( + raw.info, + rows=rows, + samples=samples, + spacing=spacing, + outputs=self.outputs, + cut_energy=self.basis.cut_energy, + closure=self.closure, + preset=self.preset is not None, + model_match=_matches(raw), + ) def record(self) -> dict[str, object]: """Return the model's provenance for run metadata. @@ -1080,21 +934,61 @@ def record(self) -> dict[str, object]: "complement": "not_applicable", "preset_record": None if preset is None - else { - "identity": preset.identity, - "variant": preset.variant, - "step_margin": preset.step_margin, - "accuracy": preset.accuracy, - "box_sensitivity": preset.box_sensitivity, - "gauge_sensitivity": preset.gauge_sensitivity, - "max_peak_intensity": preset.max_peak_intensity, - "wavelength": preset.wavelength, - "window": preset.window, - "device_admissible": preset.device_admissible, - }, + else {name: preset[name] for name in _PRESET_RECORD}, } +def _returned(reply: object) -> _Native: + """Return an atomic call's record, typed by the worker's name for it. + + The record's ``record_type`` becomes ``sae.grid_response``, ``hacc.total_current_ + response`` and so on, or ``other:``; the worker's comparison of a haCC + result's atom, basis and gauge with the model rides beside its fields. + """ + answer = cast("Mapping[str, Any]", reply) + raw = answer["raw"] + if isinstance(raw, Record): + raw.computed["provider_type"] = raw.record_type.rpartition(".")[2] + raw.record_type = answer["kind"] + raw.computed["model_match"] = answer["model_match"] + return raw + + +def _kind(raw: object) -> str: + """Return a returned record's type name (:func:`_returned`).""" + return raw.record_type if isinstance(raw, Record) else "other" + + +def _matches(raw: object) -> bool: + """Whether the worker found a haCC result's model to be the prepared one.""" + return isinstance(raw, Record) and raw.computed.get("model_match") is True + + +def _fields_of(value: object) -> dict[str, Any]: + """Return a returned record's fields.""" + if isinstance(value, Record): + return dict(vars(value)) + return asdict(cast("Any", value)) + + +def _component(shadow: _Native, quantity: str, order: int) -> np.ndarray: + """Return one shadow component, as the provider's ``component`` returned it. + + The worker evaluated the provider's method (``_pyroraptor``); a component it + refused raises here as the provider raised it. + """ + components = cast("Mapping[str, Any]", shadow.components) + found = components.get(f"{quantity}:{order}") + if found is None: + raise ValueError(f"order must be one of {shadow.orders}") + if isinstance(found, Mapping): + refusal = cast("Mapping[str, str]", found) + if refusal["type"] == "ValueError": + raise ValueError(refusal["error"]) + raise RuntimeError(f"{refusal['type']}: {refusal['error']}") + return cast("np.ndarray", found) + + # --------------------------------------------------------------------- binding @@ -1113,6 +1007,7 @@ def __init__( *, defer_plans: bool = False, archive: TDSEArchiveWriter | None = None, + stop: StopFlag | None = None, ) -> None: self._archive = archive # A full archive widens the provider's selection, never the descriptor's, @@ -1132,7 +1027,6 @@ def __init__( self._chunks = _row_chunks(execution, n_nodes) self._budget = AtomicCallBudget(execution.max_atomic_calls, len(self._chunks)) self._stack = ExitStack() - self._plans: dict[int, _Native] = {} self._plan_records: dict[str, object] = {} self._closed = False self._cached: TDSEResult | None = None @@ -1140,7 +1034,7 @@ def __init__( self._diagnostic_cache_hits = 0 self._last_call: dict[str, object] = {} self._failures: list[object] = [] - self._hacc: _HaccModel | None = None + self._hacc: _HaccView | None = None self._window: AtomicWindowRecord | None = descriptor.atomic_window self._gate = descriptor.source_gate() self._samples = _atomic_samples(descriptor) @@ -1161,20 +1055,22 @@ def __init__( rows=self._chunks[0].rows, samples=descriptor.samples ) ) - self._provider = _load_provider() - self._build = _build_record(self._provider, execution) + # The worker is the binding's from here to close: its stack ends it. + self._worker = _open_provider( + stop=stop, limit=_frame_limit(descriptor, self._chunks) + ) + self._stack.callback(self._worker.close) try: + self._build = _build_record(self._worker.identity, execution) self._prepare() _admit_memory( self._memory, execution.memory_limit, ) except BaseException as error: + self._worker.kill() self._stack.close() - if not isinstance(error, Exception) or isinstance(error, SolpropError): - raise - kind = _classify(error, self._provider) - raise BackendError(f"TDSE preparation failed: {error} [{kind}]") from error + raise _preparation_error(error) from error @property def native_outputs(self) -> str | None: @@ -1190,152 +1086,88 @@ def native_outputs(self) -> str | None: # -- preparation --------------------------------------------------------------- def _prepare(self) -> None: - """Dispatch on the descriptor's model and admit the measured memory.""" + """Prepare the model and its plans in the worker; admit the measured memory.""" basis = self._descriptor.response.basis + reply = cast( + "Mapping[str, Any]", + self._worker.request("prepare", self._prepare_request(basis)), + ) + self._absorb(reply) if isinstance(basis, HaccBasis): - self._prepare_hacc(basis) - else: - self._prepare_sae(basis) + self._hacc = _HaccView.from_reply(basis, reply["hacc"]) + if basis.preset is not None: + self._preset_domain_at_start(self._hacc) + if not self._defer_plans: + self._warn_custom() # Include the simultaneously live provider histories, normalized chunks, - # concatenated result and exact-input diagnostic cache in host admission. + # concatenated result and exact-input diagnostic cache in host admission, and + # a call's copies on the pipe in each process (SEP 49 section 6.5). self._memory["host.result_and_cache_reserve"] = self._estimate[ "estimate.host_histories" ] - - def _prepare_sae(self, basis: TDSEBasis) -> None: - """Allocate shared SAE resources and one plan per distinct batch size.""" - provider = self._provider - model = provider.atomic_model(basis.potential) - if model.version != basis.potential_version: - raise BackendError( - "TDSE atomic potential revision differs from its descriptor" - ) - scaling = None - if basis.scaling_radius is not None: - assert basis.scaling_angle is not None and basis.infinite_order is not None - assert basis.decay is not None - scaling = provider.ComplexScaling( - basis.scaling_radius, - basis.scaling_angle, - basis.infinite_order, - basis.decay, - ) - discretization = provider.Discretization( - provider.RadialBasis(basis.radial_edges, basis.radial_orders), - provider.AngularBasis(basis.lmax, basis.m), - model, - scaling, - basis.cuts, - ) - operators = self._stack.enter_context(discretization.assemble()) - descriptor = self._descriptor - initial = provider.StateSelection( - descriptor.initial_l, descriptor.initial_index + transport = _transport_bytes( + self._descriptor, + max(chunk.rows for chunk in self._chunks), + self._native_outputs, ) - setup = self._stack.enter_context( - provider.SpectralSetup( - operators, - provider.SpectralOptions( - cut_energy=basis.cut_energy, - states=(initial,), - growing=cast("Literal['remove']", descriptor.growing), - growth_tolerance=descriptor.growth_tolerance, - window_duration=descriptor.window_duration, - accept_removed_component=descriptor.accept_removed_component, - ), - ) - ) - for label, ledger in ( - ("operators", operators.memory_ledger()), - ("setup", setup.memory_ledger()), - ): - for entry in ledger.entries: - self._memory[f"{label}.{entry.name}"] = entry.bytes - for rows in sorted({chunk.rows for chunk in self._chunks}): - plan = self._stack.enter_context( - provider.ResponsePlan( - setup, - rows=rows, - spacing=descriptor.spacing, - samples=self._samples, - max_step_au=( - descriptor.max_atomic_step - / provider.unit_scale("time").si_per_atomic_unit - ), - initial=initial, - field="electric_field", - shadow_order=descriptor.shadow_order, - complement=self._execution.complement, - outputs=self._native_outputs, - ) - ) - self._plans[rows] = plan - for entry in plan.memory_ledger().entries: - if not entry.shared: - self._memory[f"plan.{rows}.{entry.name}"] = entry.bytes - self._plan_records = { - str(rows): asdict(plan.info) for rows, plan in self._plans.items() - } + self._memory["transport.solprop"] = transport + self._memory["transport.worker"] = transport - def _prepare_hacc(self, basis: HaccBasis) -> None: - """Prepare the haCC model, admit its ledger and build the chunk plans.""" - model = _HaccModel( - self._provider, - basis, - self._execution, - self._stack, - archive="required" if self._archive is None else self._archive.outputs, - ) - self._hacc = model - self._memory.update(model.memory) - # Admit the model's measured ledger before any plan is built (C7). - _admit_memory( - self._memory, - self._execution.memory_limit, - ) - if basis.preset is not None: - self._preset_domain_at_start(model) - if not self._defer_plans: - self._build_hacc_plans() + def _prepare_request(self, basis: TDSEBasis | HaccBasis) -> dict[str, object]: + """Return the plain records the worker builds the model and plans from.""" + descriptor, execution = self._descriptor, self._execution + return { + "model": "hacc" if isinstance(basis, HaccBasis) else "sae", + "basis": asdict(basis), + "rows": sorted({chunk.rows for chunk in self._chunks}), + "samples": self._samples, + "spacing": float(descriptor.spacing), + "max_atomic_step": descriptor.max_atomic_step, + "shadow_order": descriptor.shadow_order, + "complement": execution.complement, + "outputs": self._native_outputs, + "archive_outputs": "required" + if self._archive is None + else self._archive.outputs, + "initial": (descriptor.initial_l, descriptor.initial_index), + "growing": descriptor.growing, + "growth_tolerance": descriptor.growth_tolerance, + "window_duration": descriptor.window_duration, + "accept_removed_component": descriptor.accept_removed_component, + "device": execution.device, + "memory": dict(self._memory), + "memory_limit": execution.memory_limit, + "defer_plans": self._defer_plans, + "revision": _PROVIDER_REVISION, + } - def _build_hacc_plans(self) -> None: - """Build one haCC plan per chunk size, then warn for a custom basis. + def _absorb(self, reply: Mapping[str, Any]) -> None: + """Take the worker's measured memory and plan records.""" + self._memory.update(cast("Mapping[str, int]", reply["memory"])) + self._plan_records = dict(cast("Mapping[str, object]", reply["plans"])) + if self._hacc is not None and reply["hacc"] is not None: + self._hacc.update(reply["hacc"]) - The custom-basis warning waits for the plans because whether the model - has the escaped-electron closure is known only once a plan exists. - """ + def _warn_custom(self) -> None: + """Warn that a custom basis is unqualified, once its plans know the closure.""" model = self._hacc assert model is not None basis = model.basis - spacing = float(self._descriptor.spacing) - for rows in sorted({chunk.rows for chunk in self._chunks}): - plan = self._stack.enter_context( - model.plan(rows=rows, spacing=spacing, samples=self._samples) - ) - model.check_info( - plan.info, rows=rows, samples=self._samples, spacing=spacing - ) - self._plans[rows] = plan - for entry in plan.memory_ledger().entries: - if not entry.shared: - self._memory[f"plan.{rows}.{entry.name}"] = entry.bytes - self._plan_records = { - str(rows): asdict(plan.info) for rows, plan in self._plans.items() - } - if basis.preset is None: - missing = ( - "" - if model.closure - else "; its total current lacks the escaped-electron closure, so the " - "source uses the in-box first order and a total that omits the " - "momentum carried out through R0" - ) - warnings.warn( - f"{basis.label} is a custom haCC basis with no qualified accuracy " - f"and no preset domain; it is recorded as unqualified{missing}", - TDSEUnqualifiedModelWarning, - stacklevel=5, - ) + if basis.preset is not None: + return + missing = ( + "" + if model.closure + else "; its total current lacks the escaped-electron closure, so the " + "source uses the in-box first order and a total that omits the " + "momentum carried out through R0" + ) + warnings.warn( + f"{basis.label} is a custom haCC basis with no qualified accuracy " + f"and no preset domain; it is recorded as unqualified{missing}", + TDSEUnqualifiedModelWarning, + stacklevel=5, + ) @property def plans_built(self) -> bool: @@ -1344,7 +1176,7 @@ def plans_built(self) -> bool: - Returns: False only for a haCC binding prepared with ``defer_plans`` before :meth:`build_plans`. """ - return self._hacc is None or bool(self._plans) + return self._hacc is None or bool(self._plan_records) def build_plans(self) -> None: """Build and admit a deferred haCC binding's propagation plans. @@ -1356,41 +1188,48 @@ def build_plans(self) -> None: if self.plans_built: raise BackendError("the TDSE binding's plans are already built") try: - self._build_hacc_plans() + reply = self._worker.request( + "build_plans", + { + "memory": dict(self._memory), + "memory_limit": self._execution.memory_limit, + }, + ) + self._absorb(cast("Mapping[str, Any]", reply)) _admit_memory( self._memory, self._execution.memory_limit, ) except BaseException as error: + self._worker.kill() self.close() - if not isinstance(error, Exception) or isinstance(error, SolpropError): - raise - kind = _classify(error, self._provider) - raise BackendError(f"TDSE preparation failed: {error} [{kind}]") from error + raise _preparation_error(error) from error + self._warn_custom() - def _preset_domain_at_start(self, model: _HaccModel) -> None: + def _preset_domain_at_start(self, model: _HaccView) -> None: """Flag a preset run's duration or carrier beyond its qualification.""" preset = model.preset + assert preset is not None configuration = json.loads(self._descriptor.configuration_json) lattice = configuration["lattice"] duration = lattice["atomic_duration"] - if duration > preset.window: + if duration > preset["window"]: self._flag_domain( "duration", - {"atomic_duration": duration, "preset_window": preset.window}, + {"atomic_duration": duration, "preset_window": preset["window"]}, f"the atomic duration {duration:.6g} s exceeds the preset's tested " - f"window {preset.window:.6g} s", + f"window {preset['window']:.6g} s", ) carrier = lattice["reference_wavelength"] - if not math.isclose(carrier, preset.wavelength, rel_tol=1e-12): + if not math.isclose(carrier, preset["wavelength"], rel_tol=1e-12): self._flag_domain( "carrier", { "reference_wavelength": carrier, - "preset_wavelength": preset.wavelength, + "preset_wavelength": preset["wavelength"], }, f"the reference wavelength {carrier:.6g} m differs from the preset's " - f"qualifying carrier {preset.wavelength:.6g} m", + f"qualifying carrier {preset['wavelength']:.6g} m", ) def _flag_domain(self, reason: str, record: object, message: str) -> None: @@ -1602,22 +1441,33 @@ def evaluate( # noqa: PLR0912, PLR0915 - one evaluation's complete sequence archived: TDSEArchiveError | None = None try: # Public evaluate always resets every row to the prepared state. - raw = self._plans[chunk.rows].evaluate(drive) + raw = _returned( + self._worker.request( + "evaluate", {"rows": chunk.rows, "drive": drive} + ) + ) # Captured as returned, before any validation (SEP 20, 5.8). archived = self._archive_result(evaluation, index, raw) self._validate_diagnostics(raw, chunk.rows, self._samples) chunk_records.append(self._native_record(raw, chunk)) results.append(self._normalize(raw, chunk, identity)) + except (KeyboardInterrupt, RunTerminated, StopRequested) as stopping: + # A user's stop during the call: the worker is already killed, the + # call stays charged, and it is no provider failure (SEP 49, 6.7). + self._budget.release_reservation() + self._archive_failure(evaluation, index, stopping, "interrupted") + raise except Exception as error: self._budget.record_failure() - classification = _classify(error, self._provider) + classification = _classify(error) + name, detail = _described(error) failure = { "z": z, "row_start": chunk.start, "row_stop": chunk.stop, "input_identity": identity, "classification": classification, - "error": f"{type(error).__name__}: {error}", + "error": f"{name}: {detail}", "native": None if raw is None else self._failed_native_record(raw, chunk), @@ -1627,8 +1477,9 @@ def evaluate( # noqa: PLR0912, PLR0915 - one evaluation's complete sequence wrapped = BackendError( f"TDSE failure at z={z:.17g} m, " f"rows [{chunk.start}, {chunk.stop}), " - f"input={identity}: {error} [{classification}]" + f"input={identity}: {detail} [{classification}]" ) + _attach_remote(wrapped, error) if archived is not None: wrapped.add_note( f"the TDSE archive could not store this result: {archived}" @@ -1791,7 +1642,7 @@ def _call_domain( model = self._hacc if model is None or model.preset is None: return None - bound = float(model.preset.max_peak_intensity) + bound = float(model.preset["max_peak_intensity"]) before = _peak_intensity(fields) after = _peak_intensity(np.concatenate(drives, axis=0)) outside = max(before) > bound @@ -1816,7 +1667,7 @@ def _bound(self, raw: _Native) -> _Native: - Returns: the bound array, or None for a haCC model without the closure. """ - if self._hacc is None and isinstance(raw, self._provider.GridResponse): + if self._hacc is None and _kind(raw) == "sae.grid_response": return raw.plasma.neglected_current_bound return raw.neglected_current_bound @@ -1834,7 +1685,8 @@ def _failed_native_record(self, raw: _Native, chunk: _Chunk) -> object: } ) except (AttributeError, TypeError, ValueError) as error: - return {"unreadable_native_record": type(raw).__name__, "error": str(error)} + name = raw.computed["provider_type"] if isinstance(raw, Record) else "None" + return {"unreadable_native_record": name, "error": str(error)} def _validate_diagnostics(self, raw: _Native, rows: int, samples: int) -> None: """Reject malformed diagnostics before reducing or labelling row arrays.""" @@ -1859,7 +1711,7 @@ def _validate_diagnostics(self, raw: _Native, rows: int, samples: int) -> None: # The haCC monitors carry a bool non_finite flag beside float64 norms and # the int64 step index (SEP 20, C4). kinds = {"monitored_step": np.int64, "non_finite": np.bool_} - for name, vector in asdict(raw.monitors).items(): + for name, vector in _fields_of(raw.monitors).items(): if vector is None: continue dtype = np.dtype(kinds.get(name, np.float64)) @@ -1918,13 +1770,13 @@ def _hacc_currents( model = self._hacc assert model is not None model.check_info( - raw.info, + raw, rows=rows, samples=samples, spacing=float(self._descriptor.spacing), ) if model.outputs == "total_current": - if not isinstance(raw, model.hacc.TotalCurrentResponse): + if _kind(raw) != "hacc.total_current_response": raise BackendError( "TDSE compact contract returned a different result type" ) @@ -1937,14 +1789,16 @@ def _hacc_currents( "TDSE compact response has incorrect shadow array extents" ) return raw.total_current, raw.shadow_total_current[:, 0, :] - if not isinstance(raw, model.hacc.GridResponse) or raw.shadow is None: + if _kind(raw) != "hacc.grid_response" or raw.shadow is None: raise BackendError("haCC full output requires a response with shadows") if raw.shadow.orders != (1,): raise BackendError("TDSE full response has incorrect shadow-order labels") # With the closure the total's order one is returned; without it the # in-box order one is the first order of the field-impulse total, whose # added term is at least third order (SEP 20, 4.3). - first = raw.shadow.component("total_current" if model.closure else "current", 1) + first = _component( + raw.shadow, "total_current" if model.closure else "current", 1 + ) return raw.total_current, first def _sae_currents( @@ -1972,7 +1826,7 @@ def _sae_currents( "TDSE result metadata disagree with the prepared contract" ) if self._native_outputs == "full": - if not isinstance(raw, self._provider.GridResponse) or raw.shadow is None: + if _kind(raw) != "sae.grid_response" or raw.shadow is None: raise BackendError( "TDSE shadow model requires a full response with shadow products" ) @@ -1985,14 +1839,14 @@ def _sae_currents( # record; only a descriptor that needs it takes the third-order dipole. return ( raw.plasma.total_current, - raw.shadow.component("total_current", 1), - raw.shadow.component("total_current", 3) if 3 in orders else None, - raw.shadow.component("charge_dipole", 3) + _component(raw.shadow, "total_current", 1), + _component(raw.shadow, "total_current", 3) if 3 in orders else None, + _component(raw.shadow, "charge_dipole", 3) if descriptor.outputs == "full" else None, orders, ) - if not isinstance(raw, self._provider.TotalCurrentResponse): + if _kind(raw) != "sae.total_current_response": raise BackendError("TDSE compact contract returned a different result type") orders = raw.shadow_orders if orders != expected_orders or raw.shadow_total_current is None: @@ -2041,7 +1895,7 @@ def _normalize(self, raw: _Native, chunk: _Chunk, identity: str) -> TDSEResult: monitors_json=_encoded(self._native_record(raw, chunk)), outcomes_json=_encoded( [ - {"row": chunk.start + index, **asdict(value)} + {"row": chunk.start + index, **_fields_of(value)} for index, value in enumerate(raw.outcomes) ] ), @@ -2108,7 +1962,7 @@ def diagnostic_currents( model, self._budget, self._failures, - self._provider, + self._worker, self._memory, self._execution.memory_limit, archive=self._archive, @@ -2168,7 +2022,10 @@ def memory_ledger(self) -> Mapping[str, int]: return dict(self._memory) def close(self) -> None: - """Close plans before setup, preserving metadata and attempted-call counts.""" + """End the worker, which closes the plans before the setup, keeping records. + + Metadata and attempted-call counts survive; so does a worker already killed. + """ if not self._closed: self._budget.release_reservation() self._stack.close() @@ -2183,14 +2040,14 @@ class _DiagnosticContext: Attributes ---------- - model : _HaccModel - Open prepared model. + model : _HaccView + The open prepared model's record. budget : AtomicCallBudget Ledger the call is charged to, as ``diagnostic``. failures : list Failure records the call appends to. - provider : _Provider - Loaded provider, for error classification. + worker : _Channel + The worker holding the model. memory : mapping Memory already held, in bytes. limit : int or None @@ -2202,10 +2059,10 @@ class _DiagnosticContext: for a standalone check) and the build and execution records. """ - model: _HaccModel + model: _HaccView budget: AtomicCallBudget failures: list[object] - provider: _Native + worker: _Channel memory: Mapping[str, int] limit: int | None archive: TDSEArchiveWriter | None = None @@ -2217,13 +2074,13 @@ def _diagnostic( field: Float[np.ndarray, "1 n_samples"], spacing: float, ) -> tuple[Float[np.ndarray, " n_samples"], Float[np.ndarray, " n_samples"]]: - """Run one charged one-row haCC diagnostic on a temporary plan. + """Run one charged one-row haCC diagnostic on a temporary plan in the worker. - ``field``: finite float64 field in V/m, shape ``(1, n_samples)``. - ``spacing``: protocol spacing in s. - Returns: owning ``j_tot`` and ``j_1`` in C m/s. """ - model, budget, provider = context.model, context.budget, context.provider + model, budget, worker = context.model, context.budget, context.worker if ( not isinstance(field, np.ndarray) or field.dtype != np.dtype(np.float64) @@ -2234,24 +2091,16 @@ def _diagnostic( ): raise BackendError("the consistency field must be finite float64 (1, samples)") samples = int(field.shape[1]) - with ExitStack() as stack: - try: - plan = stack.enter_context( - model.plan(rows=1, spacing=spacing, samples=samples) - ) - model.check_info(plan.info, rows=1, samples=samples, spacing=spacing) - extra = { - f"diagnostic.{entry.name}": entry.bytes - for entry in plan.memory_ledger().entries - if not entry.shared - } - except SolpropError: - raise - except Exception as error: - kind = _classify(error, provider) - raise BackendError( - f"TDSE diagnostic preparation failed: {error} [{kind}]" - ) from error + try: + prepared = cast( + "Mapping[str, Any]", + worker.request("diagnostic_plan", {"samples": samples, "spacing": spacing}), + ) + except RemoteError as error: + raise _local_error(error, "TDSE diagnostic preparation failed") from error + try: + model.update(prepared["hacc"]) + extra = cast("Mapping[str, int]", prepared["memory"]) # The temporary plan overlaps the propagation plans: admit their sum. _admit_memory( {**context.memory, **extra, "diagnostic.host": 8 * 20 * samples}, @@ -2259,51 +2108,70 @@ def _diagnostic( ) budget.reserve(1, calls=1) try: - drive = np.ascontiguousarray(field) - evaluation = _diagnostic_archive_begin(context, drive, spacing) - budget.begin_evaluation(purpose="diagnostic", calls=1) - budget.charge(purpose="diagnostic") - raw: _Native = None - archived: TDSEArchiveError | None = None - try: - raw = plan.evaluate(drive) - archived = _diagnostic_archive_result(context, evaluation, raw) - binding_view = _DiagnosticView(model, provider, spacing) - binding_view.validate(raw, samples) - total, first = binding_view.currents(raw, samples) - except Exception as error: - budget.record_failure() - kind = _classify(error, provider) - context.failures.append( - _failure_safe( - { - "purpose": "diagnostic", - "classification": kind, - "error": f"{type(error).__name__}: {error}", - } - ) - ) - wrapped = BackendError( - f"TDSE consistency diagnostic failed: {error} [{kind}]" - ) - if archived is not None: - wrapped.add_note( - f"the TDSE archive could not store this result: {archived}" - ) - _diagnostic_archive_failure(context, evaluation, error, kind, wrapped) - raise wrapped from error - if archived is not None: - raise archived - if context.archive is not None and evaluation is not None: - context.archive.finish_evaluation(evaluation, "returned") + total, first = _diagnostic_call(context, field, spacing) finally: budget.release_reservation() + finally: + with contextlib.suppress(RemoteError, WorkerError): + worker.request("diagnostic_close", {}) return ( np.array(total[0], dtype=np.float64, copy=True), np.array(first[0], dtype=np.float64, copy=True), ) +def _diagnostic_call( + context: _DiagnosticContext, + field: Float[np.ndarray, "1 n_samples"], + spacing: float, +) -> tuple[np.ndarray, np.ndarray]: + """Make, archive and validate the diagnostic's one charged atomic call.""" + model, budget = context.model, context.budget + samples = int(field.shape[1]) + drive = np.ascontiguousarray(field) + evaluation = _diagnostic_archive_begin(context, drive, spacing) + budget.begin_evaluation(purpose="diagnostic", calls=1) + budget.charge(purpose="diagnostic") + archived: TDSEArchiveError | None = None + try: + raw = _returned(context.worker.request("diagnostic_evaluate", {"drive": drive})) + archived = _diagnostic_archive_result(context, evaluation, raw) + binding_view = _DiagnosticView(model, spacing) + binding_view.validate(raw, samples) + total, first = binding_view.currents(raw, samples) + except (KeyboardInterrupt, RunTerminated, StopRequested) as stopping: + _diagnostic_archive_failure( + context, evaluation, stopping, "interrupted", stopping + ) + raise + except Exception as error: + budget.record_failure() + kind = _classify(error) + name, detail = _described(error) + context.failures.append( + _failure_safe( + { + "purpose": "diagnostic", + "classification": kind, + "error": f"{name}: {detail}", + } + ) + ) + wrapped = BackendError(f"TDSE consistency diagnostic failed: {detail} [{kind}]") + _attach_remote(wrapped, error) + if archived is not None: + wrapped.add_note( + f"the TDSE archive could not store this result: {archived}" + ) + _diagnostic_archive_failure(context, evaluation, error, kind, wrapped) + raise wrapped from error + if archived is not None: + raise archived + if context.archive is not None and evaluation is not None: + context.archive.finish_evaluation(evaluation, "returned") + return total, first + + def _diagnostic_archive_begin( context: _DiagnosticContext, field: Float[np.ndarray, "1 n_samples"], @@ -2403,9 +2271,8 @@ class _DiagnosticView: - ``spacing``: protocol spacing in s. """ - def __init__(self, model: _HaccModel, provider: _Provider, spacing: float) -> None: + def __init__(self, model: _HaccView, spacing: float) -> None: self._model = model - self._provider = provider self._spacing = spacing def validate(self, raw: _Native, samples: int) -> None: @@ -2431,10 +2298,10 @@ def currents(self, raw: _Native, samples: int) -> tuple[np.ndarray, np.ndarray]: - Returns: ``(1, samples)`` currents in C m/s. """ model = self._model - model.check_info(raw.info, rows=1, samples=samples, spacing=self._spacing) + model.check_info(raw, rows=1, samples=samples, spacing=self._spacing) if model.outputs == "total_current": if ( - not isinstance(raw, model.hacc.TotalCurrentResponse) + _kind(raw) != "hacc.total_current_response" or raw.shadow_orders != (1,) or raw.shadow_total_current is None or raw.shadow_total_current.shape != (1, 1, samples) @@ -2442,12 +2309,14 @@ def currents(self, raw: _Native, samples: int) -> tuple[np.ndarray, np.ndarray]: raise BackendError("haCC compact diagnostic result is malformed") return raw.total_current, raw.shadow_total_current[:, 0, :] if ( - not isinstance(raw, model.hacc.GridResponse) + _kind(raw) != "hacc.grid_response" or raw.shadow is None or raw.shadow.orders != (1,) ): raise BackendError("haCC full diagnostic result is malformed") - first = raw.shadow.component("total_current" if model.closure else "current", 1) + first = _component( + raw.shadow, "total_current" if model.closure else "current", 1 + ) return raw.total_current, first @@ -2465,6 +2334,8 @@ class DiagnosticSession: Device, memory limit and the explicit call budget of the check. archive : TDSEArchiveWriter or None, optional Archive session receiving the diagnostic call, labelled ``diagnostic``. + stop : solprop.log.StopFlag or None, optional + A flag polled while the provider's worker is awaited. """ def __init__( @@ -2473,32 +2344,43 @@ def __init__( execution: TDSEExecution, *, archive: TDSEArchiveWriter | None = None, + stop: StopFlag | None = None, ) -> None: self._execution = execution self._archive = archive self._budget = AtomicCallBudget(execution.max_atomic_calls, 1) self._failures: list[object] = [] - self._stack = ExitStack() - self._provider = _load_provider() - self.build = _build_record(self._provider, execution) + self._worker = _open_provider(stop=stop) try: - self._model = _HaccModel( - self._provider, - basis, - execution, - self._stack, - archive="required" if archive is None else archive.outputs, + self.build = _build_record(self._worker.identity, execution) + reply = cast( + "Mapping[str, Any]", + self._worker.request( + "prepare", + { + "model": "hacc", + "basis": asdict(basis), + "rows": [], + "complement": execution.complement, + "archive_outputs": "required" + if archive is None + else archive.outputs, + "device": execution.device, + "memory": {}, + "memory_limit": execution.memory_limit, + "defer_plans": True, + "revision": _PROVIDER_REVISION, + }, + ), ) - self._memory = dict(self._model.memory) + self._model = _HaccView.from_reply(basis, reply["hacc"]) + self._memory = dict(cast("Mapping[str, int]", reply["memory"])) if archive is not None: self._memory.update(archive.memory_ledger(rows=1, samples=1)) _admit_memory(self._memory, execution.memory_limit) except BaseException as error: - self._stack.close() - if not isinstance(error, Exception) or isinstance(error, SolpropError): - raise - kind = _classify(error, self._provider) - raise BackendError(f"TDSE preparation failed: {error} [{kind}]") from error + self._worker.kill() + raise _preparation_error(error) from error @property def ledger(self) -> AtomicCallLedger: # numpydoc ignore=RT01 @@ -2529,7 +2411,7 @@ def diagnostic_currents( self._model, self._budget, self._failures, - self._provider, + self._worker, self._memory, self._execution.memory_limit, archive=self._archive, @@ -2571,5 +2453,26 @@ def metadata(self) -> Mapping[str, object]: ) def close(self) -> None: - """Close the model; repeated calls are harmless.""" - self._stack.close() + """End the worker and its model; repeated calls are harmless.""" + self._worker.close() + + +def _preparation_error(error: BaseException) -> BaseException: + """Return a preparation failure as the binding raises it. + + solprop's own refusals and interruptions as they are; a provider exception the + worker reported as solprop's class or ``TDSE preparation failed: ... [kind]``. + """ + if isinstance(error, RemoteError): + return _local_error(error, "TDSE preparation failed") + if not isinstance(error, Exception) or isinstance(error, SolpropError): + return error + return BackendError(f"TDSE preparation failed: {error} [{_classify(error)}]") + + +def _attach_remote(wrapped: BaseException, error: BaseException) -> None: + """Attach a worker's traceback or its standard error to the error raised.""" + if isinstance(error, RemoteError): + wrapped.add_note(f"the provider worker's traceback:\n{error.remote_traceback}") + for note in getattr(error, "__notes__", ()): + wrapped.add_note(note) diff --git a/tests/driver/test_tdse_archive_binding.py b/tests/driver/test_tdse_archive_binding.py index 24eefc5da..1947dd688 100644 --- a/tests/driver/test_tdse_archive_binding.py +++ b/tests/driver/test_tdse_archive_binding.py @@ -33,6 +33,7 @@ ) from tests.tdse_archive_cases import Info, SaeMonitors, Shadow, writer +import source_channel from solprop.driver import tdse from solprop.driver.tdse_consistency import check_consistency, protocol_field from solprop.exceptions import ( @@ -164,9 +165,7 @@ def response_plan( # pyright: ignore[reportIncompatibleMethodOverride] - a rich @pytest.fixture def sae(monkeypatch): - fake = _RichProvider() - monkeypatch.setattr(tdse, "_load_provider", lambda: fake) - return fake + return source_channel.install(monkeypatch, _RichProvider()) @pytest.fixture diff --git a/tests/driver/test_tdse_binding.py b/tests/driver/test_tdse_binding.py index 5813a30b0..927fb4327 100644 --- a/tests/driver/test_tdse_binding.py +++ b/tests/driver/test_tdse_binding.py @@ -15,7 +15,8 @@ import pytest from numpy.testing import assert_array_equal -from solprop.driver import tdse +import source_channel +from solprop.driver import _pyroraptor, tdse from solprop.exceptions import ( BackendError, DriverError, @@ -304,7 +305,8 @@ def solve_preset(self, target, preset): @pytest.fixture def pinned_python(monkeypatch): """Pass the provider's interpreter gate: these tests fake the provider itself.""" - monkeypatch.setattr(tdse.sys, "version_info", (3, 14, 0)) + monkeypatch.setattr(_pyroraptor.sys, "version_info", (3, 14, 0)) + source_channel.in_process(monkeypatch) def _basis(): @@ -335,9 +337,7 @@ def _descriptor(model: TDSEKerrModel = "optmat_cubic", basis: TDSEBasis | None = @pytest.fixture def provider(monkeypatch): - fake = _Provider() - monkeypatch.setattr(tdse, "_load_provider", lambda: cast("tdse._Provider", fake)) - return fake + return source_channel.install(monkeypatch, _Provider()) @pytest.mark.parametrize("model", ["optmat_cubic", "tdse_shadow", "tdse_native"]) @@ -435,10 +435,10 @@ def test_binding_admits_memory_before_native_setup(provider): def test_binding_memory_estimate_does_not_import_native_engine(monkeypatch): - def unexpected_import(): - pytest.fail("memory estimate imported the optional provider") + def unexpected_start(**options): + pytest.fail("memory estimate started the provider's worker") - monkeypatch.setattr(tdse, "_load_provider", unexpected_import) + monkeypatch.setattr(tdse, "_open_provider", unexpected_start) assert ( sum(tdse.estimate_binding_memory(_descriptor(), TDSEExecution(), 2).values()) > 0 @@ -490,7 +490,7 @@ def test_binding_missing_optional_provider_is_actionable(monkeypatch, pinned_pyt def missing(name): raise ImportError("synthetic missing dependency") - monkeypatch.setattr(tdse.importlib, "import_module", missing) + monkeypatch.setattr(_pyroraptor.importlib, "import_module", missing) with pytest.raises(BackendError, match="optional pyroraptor"): tdse.basis_from_preset("helium", "production") @@ -518,7 +518,8 @@ def test_binding_module_import_does_not_load_optional_engine(): def test_binding_rejects_unsupported_interpreter_before_native_import(monkeypatch): - monkeypatch.setattr(tdse.sys, "version_info", (3, 15, 0)) + monkeypatch.setattr(_pyroraptor.sys, "version_info", (3, 15, 0)) + source_channel.in_process(monkeypatch) with pytest.raises(BackendError, match=r"Python 3\.14"): tdse.basis_from_preset("helium", "production") @@ -527,12 +528,12 @@ def test_binding_without_distribution_metadata_still_records_native_identity( monkeypatch, pinned_python ): provider = _Provider() - monkeypatch.setattr(tdse.importlib, "import_module", lambda name: provider) + monkeypatch.setattr(_pyroraptor.importlib, "import_module", lambda name: provider) def missing_distribution(name): - raise tdse.PackageNotFoundError(name) + raise _pyroraptor.PackageNotFoundError(name) - monkeypatch.setattr(tdse, "distribution", missing_distribution) + monkeypatch.setattr(_pyroraptor, "distribution", missing_distribution) binding = tdse.prepare_binding(_descriptor(), TDSEExecution(), 1) metadata = _saved_metadata(binding) assert metadata["build"]["distribution"] is None @@ -548,7 +549,7 @@ def test_binding_records_an_installed_distribution( ): """The optional extra, when installed, adds its version and source to the record.""" provider = _Provider() - monkeypatch.setattr(tdse.importlib, "import_module", lambda name: provider) + monkeypatch.setattr(_pyroraptor.importlib, "import_module", lambda name: provider) class _Distribution: version = "1.2.3" @@ -557,7 +558,7 @@ def read_text(self, name): assert name == "direct_url.json" return direct_url - monkeypatch.setattr(tdse, "distribution", lambda name: _Distribution()) + monkeypatch.setattr(_pyroraptor, "distribution", lambda name: _Distribution()) binding = tdse.prepare_binding(_descriptor(), TDSEExecution(), 1) build = _saved_metadata(binding)["build"] assert build["distribution"] == "1.2.3" diff --git a/tests/driver/test_tdse_entry.py b/tests/driver/test_tdse_entry.py index b81b62b09..6a441a0b0 100644 --- a/tests/driver/test_tdse_entry.py +++ b/tests/driver/test_tdse_entry.py @@ -125,10 +125,10 @@ def test_lower_level_construction_refuses_an_unresolved_window_before_any_provid ): parent = _capillary(tdse=_sae()) - def unexpected(): - pytest.fail("the provider was loaded") + def unexpected(**options): + pytest.fail("the provider's worker was started") - monkeypatch.setattr(runtime, "_load_provider", unexpected) + monkeypatch.setattr(runtime, "_open_provider", unexpected) simulation = Simulation( parent.grid, parent.modes, diff --git a/tests/driver/test_tdse_hacc_binding.py b/tests/driver/test_tdse_hacc_binding.py index 14d4ecb38..69d775d2a 100644 --- a/tests/driver/test_tdse_hacc_binding.py +++ b/tests/driver/test_tdse_hacc_binding.py @@ -32,6 +32,7 @@ tiny_basis, ) +import source_channel from solprop.constants import EPS0, C from solprop.driver import tdse from solprop.exceptions import ( @@ -850,7 +851,7 @@ def test_sae_binding_has_no_consistency_diagnostic(monkeypatch): from tests.driver.test_tdse_binding import _Provider # noqa: PLC0415 provider = _Provider() - monkeypatch.setattr(tdse, "_load_provider", lambda: provider) + source_channel.install(monkeypatch, provider) binding: Any = tdse.prepare_binding(sae("tdse_native"), TDSEExecution(), 1) field, spacing = _protocol() with pytest.raises(BackendError, match="needs a haCC binding"): @@ -988,7 +989,7 @@ def test_defer_plans_must_be_a_bool_and_is_ignored_for_sae(monkeypatch): from tests.driver.test_tdse_binding import _Provider # noqa: PLC0415 provider = _Provider() - monkeypatch.setattr(tdse, "_load_provider", lambda: provider) + source_channel.install(monkeypatch, provider) binding: Any = tdse.prepare_binding( sae("tdse_native"), TDSEExecution(), 1, defer_plans=True ) diff --git a/tests/driver/test_tdse_hacc_installed.py b/tests/driver/test_tdse_hacc_installed.py index 80a61b563..c901d2cdb 100644 --- a/tests/driver/test_tdse_hacc_installed.py +++ b/tests/driver/test_tdse_hacc_installed.py @@ -86,7 +86,10 @@ from tests.hacc_cases import A0 control = tiny_basis(gauge_radius=1.0 * A0) -atom, native, gauge = tdse._hacc_objects(tdse._load_provider(), hacc, control) +from solprop.driver import _pyroraptor + +provider = _pyroraptor._load_provider() +atom, native, gauge = _pyroraptor._hacc_objects(provider, hacc, control) with hacc.prepare(atom, native, gauge=gauge, cut_energy=control.cut_energy) as model: try: hacc.ResponsePlan( diff --git a/tests/driver/test_tdse_worker.py b/tests/driver/test_tdse_worker.py new file mode 100644 index 000000000..91d448b78 --- /dev/null +++ b/tests/driver/test_tdse_worker.py @@ -0,0 +1,204 @@ +# SPDX-FileCopyrightText: 2026 John C. Travers and contributors +# +# SPDX-License-Identifier: GPL-3.0-or-later + +"""The TDSE binding through a real provider worker (SEP 49 section 7.2). + +The worker serves Pyroraptor's adapter around the manufactured haCC provider of +``hacc_cases`` (``tests/source_adapters.py``), which this process never imports as its +provider. The same provider served in this process (``source_channel``) is the direct +call the worker must reproduce bit for bit. +""" + +import json +import sys +import threading +import warnings + +import numpy as np +import pytest + +import source_channel +from driver.test_tdse_hacc_binding import _descriptor, _fields +from driver.test_tdse_hacc_run import _simulation +from hacc_cases import FakeHaccProvider, alpha_law, preset_basis +from solprop.driver import tdse +from solprop.exceptions import ( + BackendError, + DriverError, + StopRequested, + TDSEDomainWarning, + TDSEUnqualifiedModelWarning, +) +from solprop.nonlinear.tdse import TDSEExecution +from solprop.output import open_file, resume + +pytestmark = pytest.mark.medium + +DEADLINE = 10.0 +"""A test-harness bound on a cancellation; the production code has none.""" + + +@pytest.fixture +def workers(monkeypatch): + """Serve the manufactured provider in real workers; return the workers started.""" + monkeypatch.setattr(tdse, "_ADAPTER", "source_adapters:manufactured_tdse") + started = [] + original = tdse._open_provider + + def opened(**options): + worker = original(**options) + started.append(worker) + return worker + + monkeypatch.setattr(tdse, "_open_provider", opened) + return started + + +@pytest.fixture(autouse=True) +def cache(monkeypatch, tmp_path): + monkeypatch.setenv("SOLPROP_CACHE_DIR", str(tmp_path / "cache")) + + +def _bind(descriptor, execution=None, **options): + with warnings.catch_warnings(): + warnings.simplefilter("ignore", TDSEUnqualifiedModelWarning) + return tdse.prepare_binding( + descriptor, + TDSEExecution(max_batch_rows=2) if execution is None else execution, + 3, + **options, + ) + + +def _evaluate_once(descriptor): + binding = _bind(descriptor) + binding.reserve(1) + result = binding.evaluate(_fields(3), z=0.1, purpose="startup", density=2.0) + binding.record_trial(accepted=True) + binding.release_reservation() + binding.close() + return result, binding + + +def _plain(record): + return json.loads(json.dumps(record, allow_nan=False)) + + +def test_evaluations_through_the_worker_equal_the_provider_called_directly(workers): + descriptor = _descriptor(preset_basis(1)) + through, worker_binding = _evaluate_once(descriptor) + with pytest.MonkeyPatch.context() as patch: + fake = source_channel.install(patch, FakeHaccProvider()) + fake.response = alpha_law(1.02, lambda5=2.0**-200) + direct, direct_binding = _evaluate_once(descriptor) + for name in ("total_current", "first_order_current", "valid"): + np.testing.assert_array_equal(getattr(through, name), getattr(direct, name)) + assert through.monitors_json == direct.monitors_json + assert through.outcomes_json == direct.outcomes_json + assert worker_binding.ledger == direct_binding.ledger + assert _plain(worker_binding.metadata()) == _plain(direct_binding.metadata()) + # The provider lived in the worker alone. + assert "pyroraptor" not in sys.modules and "source_adapters" not in sys.modules + assert len(workers) == 1 and not workers[0].alive + + +@pytest.mark.parametrize( + ("factory", "message"), + [("stale_tdse", "does not match pinned"), ("cuda_tdse", "native build uses")], +) +def test_the_handshake_refuses_a_stale_or_other_provider_by_name( + monkeypatch, workers, factory, message +): + monkeypatch.setattr(tdse, "_ADAPTER", f"source_adapters:{factory}") + with pytest.raises(BackendError, match=message): + _bind(_descriptor(preset_basis(1))) + assert len(workers) == 1 and not workers[0].alive + + +def test_a_failure_in_preparation_is_named_and_ends_the_worker(workers): + with pytest.raises(BackendError, match="no complement option") as raised: + _bind( + _descriptor(preset_basis(1)), + TDSEExecution(max_batch_rows=2, complement="vendor_blas"), + ) + assert "raised in the TDSE provider worker" in "\n".join(raised.value.__notes__) + assert len(workers) == 1 and not workers[0].alive + + +def test_the_transport_copies_are_in_the_ledger_and_admitted_before_start(workers): + descriptor = _descriptor(preset_basis(1)) + execution = TDSEExecution(max_batch_rows=2) + estimate = tdse.estimate_binding_memory(descriptor, execution, 3) + binding = _bind(descriptor, execution) + ledger = binding.memory_ledger() + window = descriptor.atomic_window + assert window is not None + copies = 8 * 2 * window.samples * (2 + 20) + assert ledger["transport.solprop"] == ledger["transport.worker"] == copies + assert estimate["estimate.transport"] == 2 * copies + assert any(name.startswith("model.") for name in ledger) + binding.close() + limit = sum(estimate.values()) - 1 + with pytest.raises(DriverError, match="TDSE persistent data"): + _bind(descriptor, TDSEExecution(max_batch_rows=2, memory_limit=limit)) + assert len(workers) == 1 + + +def test_a_stop_during_an_atomic_call_kills_the_worker_and_keeps_the_charge( + monkeypatch, workers +): + monkeypatch.setenv("MANUFACTURED_HANG_AFTER", "0") + flag = threading.Event() + binding = _bind(_descriptor(preset_basis(1)), stop=flag) + binding.reserve(1) + threading.Timer(0.3, flag.set).start() + with pytest.raises(StopRequested): + binding.evaluate(_fields(3), z=0.1, purpose="startup") + assert binding.ledger.attempted == 1 and binding.ledger.failed_calls == 0 + assert not workers[0].alive + assert binding.metadata()["failures"] == [] + binding.close() + + +class _Watch: + """Pass standard error on; set a flag when the worker says its call hangs.""" + + def __init__(self, stream, flag): + self._stream, self._flag = stream, flag + + def write(self, text): + if "MANUFACTURED CALL HANGS" in text: + self._flag.set() + return self._stream.write(text) + + def flush(self): + self._stream.flush() + + +@pytest.mark.slow +def test_a_run_stopped_during_an_atomic_call_checkpoints_its_last_step( + monkeypatch, workers, tmp_path +): + """numpy_ref: the stop ends the run as stopped, the call charged (section 6.7).""" + monkeypatch.setenv("MANUFACTURED_HANG_AFTER", "12") + flag = threading.Event() + monkeypatch.setattr(sys, "stderr", _Watch(sys.stderr, flag)) + path = tmp_path / "stopped.h5" + simulation = _simulation(backend="numpy_ref") + with warnings.catch_warnings(): + warnings.simplefilter("ignore", TDSEDomainWarning) + simulation.run(path, progress="none", stop=flag) + stopped = resume(path) + assert stopped.outcome is not None + # As any stopped run: ended before its length, with a terminal checkpoint. + assert stopped.outcome.status == "failed" + assert "StopRequested" in stopped.outcome.message + calls = stopped.outcome.atomic_calls + # Twelve calls returned; the thirteenth was charged and never returned. + assert calls["attempted"] == 13 + assert calls["failed_calls"] == 0 + with open_file(path) as stored: + assert stored.terminal_checkpoint is not None + assert stored.terminal_checkpoint.plane.z > 0 + assert not workers[0].alive diff --git a/tests/hacc_cases.py b/tests/hacc_cases.py index 93a3b0447..9ae1287e7 100644 --- a/tests/hacc_cases.py +++ b/tests/hacc_cases.py @@ -704,12 +704,11 @@ def _unused(self, *args, **kwargs): def install(monkeypatch, fake=None): - """Make ``solprop.driver.tdse`` load a fake provider; return it.""" - from solprop.driver import tdse as runtime # noqa: PLC0415 - keep tests importable + """Make ``solprop.driver.tdse`` serve a fake provider in this process; return it.""" + import source_channel # noqa: PLC0415 - keep tests importable fake = FakeHaccProvider() if fake is None else fake - monkeypatch.setattr(runtime, "_load_provider", lambda: fake) - return fake + return source_channel.install(monkeypatch, fake) def helium_alpha(omega): diff --git a/tests/source_channel.py b/tests/source_channel.py new file mode 100644 index 000000000..768a8a278 --- /dev/null +++ b/tests/source_channel.py @@ -0,0 +1,86 @@ +# SPDX-FileCopyrightText: 2026 John C. Travers and contributors +# +# SPDX-License-Identifier: GPL-3.0-or-later + +"""Serve the TDSE binding's provider in this process, through the worker's codec. + +The binding talks to its provider through a worker process (SEP 49 section 6.5). Tests +whose fake providers are built and altered in the test itself serve them here instead: +the same adapter, every request and reply encoded to the protocol's bytes and decoded +again, but no process. The worker itself is tested with an importable provider in +``tests/driver/test_source_worker.py``. +""" + +import io + +from solprop.driver import _pyroraptor, _source_worker, tdse +from solprop.driver._source_worker import RemoteError, WorkerError + + +def _round_trip(value, computed=None): + """Return `value` as the other end of a pipe would read it.""" + buffer = io.BytesIO() + _source_worker.write_message(buffer.write, "reply", value, computed) + buffer.seek(0) + message = _source_worker.read_message(buffer.read, 1 << 34) + assert message is not None + return message[1] + + +def _remote(error, adapter): + report = _source_worker._error_report(error, adapter) + return RemoteError( + report["type"], report["message"], report["classification"], report["traceback"] + ) + + +class InProcessWorker: + """A worker's request protocol, its adapter served in this process.""" + + def __init__(self, make=_pyroraptor.make_adapter): + try: + self._adapter = make() + self.identity = _round_trip(self._adapter.identity()) + except Exception as error: + raise _remote(error, None) from error + self.requests = [] + self.closed = False + + def request(self, kind, value=None): + if self.closed: + raise WorkerError("the source worker is not running") + self.requests.append(kind) + sent = _round_trip(value) + try: + reply = self._adapter.handle(kind, sent) + except Exception as error: + raise _remote(error, self._adapter) from error + return _round_trip(reply, self._adapter.computed) + + def close(self): + if not self.closed: + self.closed = True + self._adapter.close() + + kill = close + + +def in_process(monkeypatch): + """Make the binding serve its provider in this process; return the workers made.""" + workers = [] + + def open_provider(**options): + del options + worker = InProcessWorker() + workers.append(worker) + return worker + + monkeypatch.setattr(tdse, "_open_provider", open_provider) + return workers + + +def install(monkeypatch, provider): + """Serve `provider` as the binding's provider, in this process; return it.""" + monkeypatch.setattr(_pyroraptor, "_load_provider", lambda: provider) + in_process(monkeypatch) + return provider From 63f43686b617d27f96e1136edf1dafc72ba57b6f Mon Sep 17 00:00:00 2001 From: John Travers Date: Thu, 12 Nov 2026 13:33:23 +0000 Subject: [PATCH 07/20] SEP 49 phase 2: keep the trial's own error when its release fails The host Adams march releases a trial's reservation on every exit and attaches a failed release to the original error as a note; tests of an archive failure after an accepted commit and of the note. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HAYtF5P9QqEd3HQx3WJkA3 --- src/solprop/backends/host/adams.py | 28 +++++++++++--- tests/backends/test_tdse_march.py | 59 +++++++++++++++++++++++++++++- 2 files changed, 80 insertions(+), 7 deletions(-) diff --git a/src/solprop/backends/host/adams.py b/src/solprop/backends/host/adams.py index e162054dd..05aa079b0 100644 --- a/src/solprop/backends/host/adams.py +++ b/src/solprop/backends/host/adams.py @@ -17,6 +17,8 @@ """ import math +from collections.abc import Iterator +from contextlib import contextmanager from typing import Any, Literal, cast import numpy as np @@ -331,7 +333,7 @@ def march_to(self, target: float) -> None: while not reached(backend._z, target): backend._poll_stop() self._afford() - try: + with _trial(backend): if not self._history: self._start() proposal = self._collar_reentry( @@ -354,8 +356,6 @@ def march_to(self, target: float) -> None: backend._take_step_stats() if backend._natural_stop(target): return - finally: - backend._release_source_calls() if dense and backend._z > target: self._ahead = (backend._z, self._engine.hold()) backend._state = cast("Any", self._observe(target)) @@ -429,7 +429,7 @@ def replay_step(self, dz: float, end: float, slack: float) -> float: reject the step. """ self._afford() - try: + with _trial(self._backend): if not self._history: self._start() kink = adams_core.replay_kink(self._policy, end, slack) @@ -447,8 +447,6 @@ def replay_step(self, dz: float, end: float, slack: float) -> float: "controller choose them" ) raise ConvergenceError(msg) - finally: - self._backend._release_source_calls() return norm def _observe(self, position: float) -> object: @@ -808,6 +806,24 @@ def _reject(self, dz: float, norm: float, order: int) -> None: """What every march path asks of an engine, beside its own trial and acceptance.""" +@contextmanager +def _trial(backend: HostDriver) -> Iterator[None]: + """Release a trial's unused source reservation on every exit (SEP 49, 6.3). + + A failure's own error stays the one raised: a release that fails after it is + attached to it as a note, never raised in its place. + """ + try: + yield + except BaseException as error: + try: + backend._release_source_calls() + except Exception as cleanup: + error.add_note(f"releasing the source reservation failed: {cleanup}") + raise + backend._release_source_calls() + + def _check_engine(engine: object, *, exponential: bool) -> None: """Refuse an injected engine that lacks a member its path calls.""" family = "exponential" if exponential else "fused" diff --git a/tests/backends/test_tdse_march.py b/tests/backends/test_tdse_march.py index 39c499e5f..fcd3e7ab3 100644 --- a/tests/backends/test_tdse_march.py +++ b/tests/backends/test_tdse_march.py @@ -21,7 +21,7 @@ from solprop.backends.session import Start from solprop.backends.sessions import start_session from solprop.driver import Simulation -from solprop.exceptions import BackendError, ResponseBudgetError +from solprop.exceptions import BackendError, ResponseBudgetError, TDSEArchiveError from solprop.nonlinear.tdse import ( TDSEBasis, TDSECalibration, @@ -232,6 +232,63 @@ def test_failed_chunk_is_charged_and_unused_trial_reservation_is_released( backend.close() +class _ArchiveFailsOnAcceptance(ManufacturedBinding): + """Fail to record an accepted trial, as an archive that cannot write would.""" + + def record_trial(self, *, accepted): + if accepted: + raise TDSEArchiveError("synthetic archive failure") + + +class _ReleaseFails(ManufacturedBinding): + """Fail to release a reservation, after releasing it, while ``failing`` is set.""" + + failing = True + + def release_reservation(self): + super().release_reservation() + if self.failing: + raise RuntimeError("synthetic release failure") + + +def test_an_archive_failure_after_an_accepted_commit_keeps_the_new_state( + march_problem, +): + """The commit is the failure boundary (SEP 49, section 6.3).""" + problem, run_request, grid = march_problem + binding = _ArchiveFailsOnAcceptance(tdse_descriptor(problem.route), 20) + backend = NumpyRefBackend(problem, run_request, grid=grid, tdse_binding=binding) + backend.set_initial(problem.initial.field0, None, z=problem.initial.z) + try: + with pytest.raises(TDSEArchiveError, match="synthetic archive failure"): + backend.propagate_to(1e-5) + # The accepted step stands and is booked; nothing stays reserved. + assert backend.observe().z > 0.0 + assert backend._adams is not None + assert backend._adams.ledger.accepted_steps == 1 + assert binding.ledger.reserved == 0 + # The startup and the predictor, two chunks each, stay charged. + assert binding.ledger.attempted == 4 + finally: + backend.close() + + +def test_a_failed_release_is_a_note_on_the_trial_s_own_error(march_problem): + problem, run_request, grid = march_problem + binding = _ReleaseFails(tdse_descriptor(problem.route), 8, fail_at=2) + backend = NumpyRefBackend(problem, run_request, grid=grid, tdse_binding=binding) + backend.set_initial(problem.initial.field0, None, z=problem.initial.z) + try: + with pytest.raises(BackendError, match="second-chunk failure") as caught: + backend.propagate_to(1e-5) + assert "synthetic release failure" in "\n".join(caught.value.__notes__) + assert binding.ledger.reserved == 0 + assert backend.observe().z == 0.0 + finally: + binding.failing = False + backend.close() + + @pytest.mark.parametrize("backend_name", ["native"]) def test_registry_refuses_tdse_before_constructing_a_backend( march_problem, monkeypatch, backend_name From 8e7771f6c00897d64f1f05e67334d984014b03d1 Mon Sep 17 00:00:00 2001 From: John Travers Date: Thu, 12 Nov 2026 13:35:25 +0000 Subject: [PATCH 08/20] SEP 49: record phase 2 in Part II Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HAYtF5P9QqEd3HQx3WJkA3 --- docs/dev/seps/sep-0049-tdse-on-native.md | 162 +++++++++++++++++++++++ 1 file changed, 162 insertions(+) diff --git a/docs/dev/seps/sep-0049-tdse-on-native.md b/docs/dev/seps/sep-0049-tdse-on-native.md index 08ba8eea9..a427cac80 100644 --- a/docs/dev/seps/sep-0049-tdse-on-native.md +++ b/docs/dev/seps/sep-0049-tdse-on-native.md @@ -946,6 +946,166 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. `atomic_edge_source` and `project_atomic_sources` with its own stage scalars and prefactor, and finishes the stage itself (§4). +### Phase 2 — The provider worker + +- **Scope.** Phase 2, 2026-11-12, Opus 5.5 (AI), commits `0f797abb` (`_interrupts`), + `ea431aa6` (the generic worker), `8f5b64bf` (Pyroraptor in the worker) and + `63f43686` (the trial's release), then `origin/main` merged (`7f985eb1`, with + ADR-077's change, #767). +- **Delivered.** + - `src/solprop/driver/_source_worker.py` (new), the provider-agnostic layer of §6.5: + `Worker` starts `sys.executable -m solprop.driver._source_worker ` + with `subprocess`, in a session of its own (`CREATE_NEW_PROCESS_GROUP` on Windows), + with the parent's environment and import path; the worker moves the protocol off + descriptors 0 and 1, points standard output at standard error, ignores SIGINT and + SIGTERM, and its reader thread is the watchdog: end of file on the parent's pipe + exits the process at once (`os._exit`), during a call too. Frames are an 8-byte + length and a payload; a message is a JSON control frame and one raw frame per array, + named by dtype (byte order included) and shape; nothing is pickled; every declared + length is checked against the frame bound before anything is read. Values cross as + tagged JSON: tuples, mappings, non-finite floats, arrays, and records (a dataclass's + or a plain object's fields, received as `Record`, with values an adapter computes + beside them). The handshake carries the protocol version, the worker's process id and + the adapter's provider identity. Waiting is a reader thread and a queue polled every + `POLL_INTERVAL` (0.2 s): between polls signal handlers run, a signal a commit holds is + raised (`_interrupts.raise_held`, new) and the run's `StopFlag` is polled; any + interruption kills the worker's process group at once. `close` between calls asks + the worker to exit and kills it after 5 s; every path reaps it and closes both pipe + ends. `worker_environment` drops the OpenMP wait variables solprop-native set for + itself (its marker `SOLPROP_NATIVE_WAIT`), keeps a user's own, and adds explicit + settings. + - `src/solprop/driver/_pyroraptor.py` (new), Pyroraptor's adapter: `_load_provider` + (now only ever called in a worker), the build identity, the haCC surface, records + and objects, the SAE and haCC preparation, plan building and its checks, the preset + and basis queries, the atomic calls and the consistency diagnostic's temporary plan, + and the classification of provider exceptions. The manufactured provider of the + tests is passed to `PyroraptorAdapter(provider)`. + - `src/solprop/driver/tdse.py`: the binding, `basis_from_preset`, `hacc_preset`, + `hacc_basis` and `DiagnosticSession` talk to the adapter in a worker + (`_open_provider`); the queries use a short-lived one. The handshake's identity is + checked against `_PROVIDER_REVISION` and `TDSEExecution.device` before any request + (`_build_record`) and is the run's build record, unchanged in form. Budgets, + reservations, the archive, the validation and normalisation of every result, the + domain flags and the unqualified-model warning stay in solprop's process, on plain + records. `prepare_binding` and `DiagnosticSession` take the run's `stop`, which + `Simulation` passes. A stop, Ctrl-C or SIGTERM during a call leaves the call charged, + releases the reservation, records the evaluation in the archive as raised with + classification `interrupted`, and propagates; it is no provider failure. The memory + ledger gains `transport.solprop` and `transport.worker` (the largest chunk's drive + twice and its result once, in each process; `tdse_transport.*` in a run's memory + rows) and the estimate `estimate.transport`, admitted before the worker starts. + - `src/solprop/backends/host/adams.py`: a trial's reservation is released on every + exit (`_trial`), a failed release attached to the trial's own error as a note + (§6.3). + - Tests: `tests/driver/test_source_worker.py` (21, real workers), + `tests/driver/test_tdse_worker.py` (7, real workers), + `tests/source_adapters.py` (the manufactured adapters the worker imports), + `tests/source_channel.py` (the adapter served in this process through the same + codec, for the existing tests whose fake providers are built inside the test), two + failure-boundary tests in `tests/backends/test_tdse_march.py`, one in + `tests/driver/test_interrupts.py`; the existing binding tests re-pointed from + `tdse._load_provider` to the in-process channel, their claims unchanged. + - `docs/guide/media/tdse.md` (the worker, its threads, stopping during a call), + `docs/dev/architecture.md`, `changelog.d/tdse-provider-worker.changed.md`. +- **Deviations.** + - A provider exception the adapter reports in a well-formed reply leaves the worker + serving; only a transport failure (an exit, a closed pipe, a malformed or oversized + frame) or a cancellation discards it. §6.5 says a worker is never reused "after a + failure"; read literally that would end every run's worker at its first provider + exception, which the binding's existing contract (a failed chunk is charged and a + fresh retry may follow) and Pyroraptor's own worker pool do not. The adapter's state + after a reported exception is defined: every call starts from the prepared state. + - Three checks that need the provider's own classes or methods run in the worker, on + the raw result, and their outcome crosses with it: the result's type is named there + (`sae.grid_response`, `hacc.total_current_response`, ...), a haCC result's atom, + basis and gauge are compared there with the model's (the provider's equality), and + the shadow's `component` is evaluated there for the components the binding reads. + Everything else of validation and normalisation is the binding's, as §6.5 says. The + plan checks of preparation and the two memory admissions between model and plans + (SEP 20, C7) run in the worker too, as part of preparation; a `DriverError` crosses + by name. + - `hacc_basis` takes the caller's provider objects, which cannot cross unchanged: they + travel as records of their dataclass fields, and the worker checks their types by + qualified name. + - The archive records an interrupted evaluation as raised with classification + `interrupted`, its outcome field unchanged (`returned` or `raised`), so the archive + format does not change. A worker that exits or breaks the protocol is classified + `worker`, a new value. + - A stopped TDSE run ends with status `failed` and the message `StopRequested: ...`, + as every stopped run does (§5.1 says "interrupted or stopped"). +- **Decisions.** + - The thread count is `PYRORAPTOR_THREADS`, the run's `thread_budget().threads`, in + the worker's environment: Pyroraptor reads it once at import. A device build needs + no setting. + - The worker selects its adapter as `module` or `module:factory`, so the tests' + manufactured adapters are named, not configured by environment. + - The worker ends with `os._exit` after the adapter has closed its plans and setup, + skipping interpreter finalisation. + - The worker runs as `-W ignore::RuntimeWarning:runpy`: `solprop.driver` imports + the module before `-m` runs it, which runpy warns of; `__main__` delegates to the + imported module, so the records an adapter receives are its classes. +- **Pyroraptor.** No companion change was needed: its results, monitors, outcomes and + plan records are frozen dataclasses of arrays and plain values, which cross as + records; its CPU thread count is already explicit at import (`PYRORAPTOR_THREADS`); + its atomic calls release the GIL (`nb::gil_scoped_release`), so the watchdog runs + during a call; and the worker skips interpreter finalisation. No real Pyroraptor + build was installed here, so no real call was made (§7.4 stays for phase 4). +- **Verification** (§7.2, manufactured providers in real workers unless noted): + 1. evaluations through the worker equal the same manufactured haCC provider served in + this process, bit for bit: currents, validity, monitors and outcomes, the ledger + and the whole metadata (`test_evaluations_through_the_worker_equal_the_provider_ + called_directly`); arrays of several dtypes and byte orders cross unchanged; + 2. the handshake refuses a protocol mismatch, a stale revision and another execution + space by name, the worker reaped; + 3. a provider exception (by type, message, classification and traceback), a worker + exit (its status and standard error's tail), a closed pipe, a malformed frame, an + oversized frame, a start-up failure and a preparation failure each end as their + named failure, the worker reaped and both pipe ends closed; the provider's printing + on standard output never corrupts a frame; + 4. a hanging call is cancelled by the stop flag, SIGINT, SIGTERM and a SIGINT held by + `committing()`, each within the poll interval and the kill (a 10 s harness + deadline); on `numpy_ref` a binding stopped during a call keeps the call charged + with no failure recorded, and a run stopped during its thirteenth atomic call ends + with 13 calls attempted, none failed, and a terminal checkpoint at its last + accepted step (`z > 0`); + 5. a Ctrl-C to the parent's whole process group interrupts the run and spares the + worker, which the parent then ends; a parent killed with SIGKILL during a hanging + call leaves no worker (the watchdog); + 6. the worker loads neither `solprop_native` nor `numpy_ref` nor `pyroraptor`, and its + `__main__` is the worker module, started from a test, from a script that refuses + to be imported twice and from `python -c`; after a run through the worker neither + `pyroraptor` nor the manufactured provider's module is in solprop's `sys.modules`; + 7. the transport rows are in the ledger and the estimate, and an admission over + `memory_limit` is refused before any worker starts; + + and §6.3 on `numpy_ref`: an archive failure in `record_trial` after an accepted + commit leaves the new state, booked, with nothing reserved; a failed release is a + note on the trial's own error. Run on the Mac: the new modules, the existing TDSE + modules of `tests/driver/`, `tests/backends/` and `tests/output/test_tdse_archive.py`, + `tests/test_structure_figures.py`, `tests/test_seps.py`: 958 passed, 3 skipped, and + 4 failed, all native-only (`test_tdse_march.py`'s native refusals and the shared-RK + refusal): the editable solprop-native here was built before the merge of `main` and + refuses to load ("built from other sources than this checkout's native/"); it was not + rebuilt (phases 0–2 change no native code), so CI decides them. + `tests/driver/test_interrupts.py`'s native case fails for the same reason. Ruff, + basedpyright and numpydoc clean on the changed files; `poe size-check` within budget. +- **Limitations.** The isolation from a scan executor's worker process (§7.2 item 6) + is not tested; Windows paths (`CREATE_NEW_PROCESS_GROUP`, `TerminateProcess` instead + of a group kill) are written but run only in CI. The worker ignores SIGINT and SIGTERM + from the start of its `main`, after interpreter start-up; in its own session no + terminal or group signal reaches it before then. A provider that spawns processes of + its own outside the worker's process group is not ended by the kill. +- **Cost.** `_source_worker.py` 765 lines, `_pyroraptor.py` 786 (most of it moved from + `driver/tdse.py`, which went from 2,575 to 2,478); across phase 2, `src/solprop` + +2,172 −682 and tests +900 −30 (physical lines, `git diff --numstat`). The driver + package's code lines (`poe size`) went from 15,296 to 16,278. §6.8 estimated +0.6k + source and +0.6k test lines for the worker. +- **Handoff.** Phase 3 (the native hook) calls `tdse_source` and the same binding; its + waits are inside `committing()`, which `Worker` already handles through + `_interrupts.raise_held`; §7.2 item 4 on native and §7.3 follow with it. The worker's + `request` raises `StopRequested`, `KeyboardInterrupt` or `RunTerminated` unchanged, + which the hook stores and the session re-raises (§6.3). + ## Part III — History - 2026-11-12: Drafted. @@ -992,6 +1152,8 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. change (#767) is merged, in one implementation pull request (§11). - 2026-11-12: In progress: Opus 5.5 (AI) on `feature/sep-0049-tdse-on-native`, based on #767's branch; phases 0 and 1 reported in Part II. +- 2026-11-12: Phase 2 (the provider worker) reported in Part II; implementation pull + request opened as a draft. ### AI review — 2026-11-12 From 0d31cdad5fec001675c3184950827856d4a10b89 Mon Sep 17 00:00:00 2001 From: John Travers Date: Thu, 12 Nov 2026 13:47:35 +0000 Subject: [PATCH 09/20] SEP 49: a fresh worker after any failure; fix the docs build The maintainer's decision: a worker is never used again after a failure, so a provider exception or a refused result ends it and the binding's next call prepares a fresh one. Resolve the three cross-reference warnings Read the Docs raised; drop a dead flag, derive the preset record names, and record the decision in Part II. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HAYtF5P9QqEd3HQx3WJkA3 --- docs/dev/seps/sep-0049-tdse-on-native.md | 41 +++++++++------ src/solprop/driver/_source_worker.py | 14 ++---- src/solprop/driver/tdse.py | 64 +++++++++++++++++------- tests/driver/test_source_worker.py | 5 +- tests/driver/test_tdse_binding.py | 5 +- tests/driver/test_tdse_worker.py | 19 +++++++ tests/source_adapters.py | 26 ++++++++++ tests/source_channel.py | 6 +++ 8 files changed, 134 insertions(+), 46 deletions(-) diff --git a/docs/dev/seps/sep-0049-tdse-on-native.md b/docs/dev/seps/sep-0049-tdse-on-native.md index a427cac80..3675abee2 100644 --- a/docs/dev/seps/sep-0049-tdse-on-native.md +++ b/docs/dev/seps/sep-0049-tdse-on-native.md @@ -951,7 +951,8 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. - **Scope.** Phase 2, 2026-11-12, Opus 5.5 (AI), commits `0f797abb` (`_interrupts`), `ea431aa6` (the generic worker), `8f5b64bf` (Pyroraptor in the worker) and `63f43686` (the trial's release), then `origin/main` merged (`7f985eb1`, with - ADR-077's change, #767). + ADR-077's change, #767), and a follow-up commit on the pull request (a fresh + worker after any failure, the documentation build, small simplifications). - **Delivered.** - `src/solprop/driver/_source_worker.py` (new), the provider-agnostic layer of §6.5: `Worker` starts `sys.executable -m solprop.driver._source_worker ` @@ -998,7 +999,7 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. exit (`_trial`), a failed release attached to the trial's own error as a note (§6.3). - Tests: `tests/driver/test_source_worker.py` (21, real workers), - `tests/driver/test_tdse_worker.py` (7, real workers), + `tests/driver/test_tdse_worker.py` (8, real workers), `tests/source_adapters.py` (the manufactured adapters the worker imports), `tests/source_channel.py` (the adapter served in this process through the same codec, for the existing tests whose fake providers are built inside the test), two @@ -1008,13 +1009,6 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. - `docs/guide/media/tdse.md` (the worker, its threads, stopping during a call), `docs/dev/architecture.md`, `changelog.d/tdse-provider-worker.changed.md`. - **Deviations.** - - A provider exception the adapter reports in a well-formed reply leaves the worker - serving; only a transport failure (an exit, a closed pipe, a malformed or oversized - frame) or a cancellation discards it. §6.5 says a worker is never reused "after a - failure"; read literally that would end every run's worker at its first provider - exception, which the binding's existing contract (a failed chunk is charged and a - fresh retry may follow) and Pyroraptor's own worker pool do not. The adapter's state - after a reported exception is defined: every call starts from the prepared state. - Three checks that need the provider's own classes or methods run in the worker, on the raw result, and their outcome crosses with it: the result's type is named there (`sae.grid_response`, `hacc.total_current_response`, ...), a haCC result's atom, @@ -1034,6 +1028,15 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. - A stopped TDSE run ends with status `failed` and the message `StopRequested: ...`, as every stopped run does (§5.1 says "interrupted or stopped"). - **Decisions.** + - A worker is never used again after a failure of any kind, as §6.5 says: a + provider exception the worker reports, a result the binding refuses, a worker exit, + a protocol error or a cancellation each end it, and the binding's next evaluation + or diagnostic starts and prepares a fresh one, which must report the same build + (`_ProviderBinding._ensure_worker`). Decided by the maintainer on 2026-11-12, over + the implementer's first proposal to keep a worker serving after a reported provider + exception. A retry after a failed chunk therefore pays a worker start and the + model's and plans' preparation again; the budget, ledger, archive and records carry + over. `DiagnosticSession`, which makes one call, does not restart. - The thread count is `PYRORAPTOR_THREADS`, the run's `thread_budget().threads`, in the worker's environment: Pyroraptor reads it once at import. A device build needs no setting. @@ -1060,8 +1063,10 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. 3. a provider exception (by type, message, classification and traceback), a worker exit (its status and standard error's tail), a closed pipe, a malformed frame, an oversized frame, a start-up failure and a preparation failure each end as their - named failure, the worker reaped and both pipe ends closed; the provider's printing - on standard output never corrupts a frame; + named failure, the worker reaped and both pipe ends closed; after a provider + exception the binding's retry runs on a fresh worker + (`test_after_a_provider_exception_the_retry_gets_a_fresh_worker`); the provider's + printing on standard output never corrupts a frame; 4. a hanging call is cancelled by the stop flag, SIGINT, SIGTERM and a SIGINT held by `committing()`, each within the poll interval and the kill (a 10 s harness deadline); on `numpy_ref` a binding stopped during a call keeps the call charged @@ -1095,11 +1100,15 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. from the start of its `main`, after interpreter start-up; in its own session no terminal or group signal reaches it before then. A provider that spawns processes of its own outside the worker's process group is not ended by the kill. -- **Cost.** `_source_worker.py` 765 lines, `_pyroraptor.py` 786 (most of it moved from - `driver/tdse.py`, which went from 2,575 to 2,478); across phase 2, `src/solprop` - +2,172 −682 and tests +900 −30 (physical lines, `git diff --numstat`). The driver - package's code lines (`poe size`) went from 15,296 to 16,278. §6.8 estimated +0.6k - source and +0.6k test lines for the worker. +- **Cost.** `_source_worker.py` 761 lines, `_pyroraptor.py` 786 (part of it moved from + `driver/tdse.py`, which went from 2,575 to 2,506); across phase 2, `src/solprop` + +2,172 −682 and tests +900 −30 (physical lines, `git diff --numstat`, before the + follow-up commit). By `tools/size_report.py`'s split, the files phases 1 and 2 + touched went from 11,071 to 12,107 code lines (+1,036), 5,613 to 5,950 docstring + lines and 447 to 493 comment lines; the driver package from 15,296 to 16,290 code + lines. §6.8 estimated +0.6k source and +0.6k test lines for the worker; the excess is + the adapter's request handlers and the parent's view of the haCC model beside the + generic layer, as the binding's validation stayed in solprop's process. - **Handoff.** Phase 3 (the native hook) calls `tdse_source` and the same binding; its waits are inside `committing()`, which `Worker` already handles through `_interrupts.raise_held`; §7.2 item 4 on native and §7.3 follow with it. The worker's diff --git a/src/solprop/driver/_source_worker.py b/src/solprop/driver/_source_worker.py index b30e7a8eb..86cd49fab 100644 --- a/src/solprop/driver/_source_worker.py +++ b/src/solprop/driver/_source_worker.py @@ -388,8 +388,7 @@ class Worker: Started by :meth:`start`, which waits for the handshake; ended by :meth:`close` between calls (a shutdown request, a short grace, then a kill) or :meth:`kill` (at once). Every path reaps the process and closes both pipe ends. A worker is - never used again after a transport failure or a cancellation; a provider exception - the adapter reports in a well-formed reply leaves it serving. + never used again after a failure of any kind or a cancellation. - ``adapter``: the adapter module's dotted name, imported in the worker. - ``environment``: the worker's whole environment (:func:`worker_environment`). @@ -418,7 +417,6 @@ def __init__( self._tail: deque[str] = deque(maxlen=_TAIL_LINES) self._threads: list[threading.Thread] = [] self._usable = False - self._cancelling = False # -- lifetime ------------------------------------------------------------------- @@ -485,9 +483,10 @@ def request(self, kind: str, value: object = None) -> object: Waits in :data:`POLL_INTERVAL` steps; between them signal handlers run, a signal a commit holds is raised (:func:`solprop._interrupts.raise_held`) and the run's stop flag is polled. Any interruption kills the worker before it - propagates. An exception the adapter reported is a :class:`RemoteError`, and the - worker keeps serving; a worker that exits, closes its pipe or sends a malformed - message is a :class:`WorkerError`. + propagates. An exception the adapter reported is a :class:`RemoteError`; a + worker that exits, closes its pipe or sends a malformed message is a + :class:`WorkerError`. Either way the worker is ended: it is never used again + after a failure (section 6.5), and its owner starts a fresh one. """ if not self._usable or self._process is None: msg = "the source worker is not running" @@ -499,8 +498,6 @@ def request(self, kind: str, value: object = None) -> object: write_message(stdin.write, kind, value) stdin.flush() reply, body = self._wait() - except RemoteError: - raise except BaseException: self.kill() raise @@ -531,7 +528,6 @@ def kill(self) -> None: process = self._process if process is None: return - self._cancelling = True if process.poll() is None: with contextlib.suppress(ProcessLookupError, PermissionError, OSError): if sys.platform == "win32": # pragma: no cover - Windows only diff --git a/src/solprop/driver/tdse.py b/src/solprop/driver/tdse.py index d2c6c3d08..a6c476a03 100644 --- a/src/solprop/driver/tdse.py +++ b/src/solprop/driver/tdse.py @@ -8,7 +8,7 @@ arrays. Source assembly applies the calibration and density separately. Resource admission, call accounting, the archive and the validation of every returned record live here; the provider itself runs in a worker process of its own -(:mod:`solprop.driver._source_worker`, with :mod:`solprop.driver._pyroraptor` as its +(``solprop.driver._source_worker``, with ``solprop.driver._pyroraptor`` as its adapter, SEP 49), and solprop's process never imports it. Two atomic models are bound (SEP 12, SEP 20): the single-active-electron (SAE) @@ -36,7 +36,7 @@ from solprop import exceptions from solprop._process import thread_budget from solprop.constants import AU_TIME, EPS0, C -from solprop.driver._pyroraptor import _admit_memory, check_info +from solprop.driver._pyroraptor import _PRESET_FIELDS, _admit_memory, check_info from solprop.driver._source_worker import ( CONTROL_LIMIT, Record, @@ -111,18 +111,7 @@ # run_short's grid occupy (SEP 20 section 5.1). Not an independent qualification. _PRESET_WAVELENGTH_RANGE = (62e-9, 1.6e-6) # The preset records a run's provenance carries (the provider's evidence apart). -_PRESET_RECORD = ( - "identity", - "variant", - "step_margin", - "accuracy", - "box_sensitivity", - "gauge_sensitivity", - "max_peak_intensity", - "wavelength", - "window", - "device_admissible", -) +_PRESET_RECORD = tuple(name for name in _PRESET_FIELDS if name != "source") @runtime_checkable @@ -131,6 +120,11 @@ class _Channel(Protocol): identity: Mapping[str, object] + @property + def alive(self) -> bool: + """Whether the worker may take a request.""" + ... + def request(self, kind: str, value: object = None) -> object: """Send one request and return its reply.""" ... @@ -829,7 +823,8 @@ def prepare_binding( and closes it. Its buffers are admitted with the binding's memory. stop : solprop.log.StopFlag or None, optional The run's stop flag, polled while an atomic call is awaited; setting it ends - the call at once, the worker killed, with :class:`StopRequested`. + the call at once, the worker killed, with + :class:`~solprop.exceptions.StopRequested`. Returns ------- @@ -966,9 +961,7 @@ def _matches(raw: object) -> bool: def _fields_of(value: object) -> dict[str, Any]: """Return a returned record's fields.""" - if isinstance(value, Record): - return dict(vars(value)) - return asdict(cast("Any", value)) + return dict(vars(value)) def _component(shadow: _Native, quantity: str, order: int) -> np.ndarray: @@ -1056,6 +1049,7 @@ def __init__( ) ) # The worker is the binding's from here to close: its stack ends it. + self._stop = stop self._worker = _open_provider( stop=stop, limit=_frame_limit(descriptor, self._chunks) ) @@ -1141,6 +1135,34 @@ def _prepare_request(self, basis: TDSEBasis | HaccBasis) -> dict[str, object]: "revision": _PROVIDER_REVISION, } + def _ensure_worker(self) -> None: + """Start and prepare a fresh worker when the last one failed (SEP 49, 6.5). + + A worker is never used again after a failure; the binding's records, budget + and archive are its own and carry over. The new worker must report the same + provider build. + """ + if self._worker.alive: + return + self._worker = _open_provider( + stop=self._stop, limit=_frame_limit(self._descriptor, self._chunks) + ) + self._stack.callback(self._worker.close) + basis = self._descriptor.response.basis + try: + if _build_record(self._worker.identity, self._execution) != self._build: + raise BackendError( + "the TDSE provider's build changed between its workers" + ) + request = self._prepare_request(basis) + request["defer_plans"] = not self.plans_built + self._absorb( + cast("Mapping[str, Any]", self._worker.request("prepare", request)) + ) + except BaseException as error: + self._worker.kill() + raise _preparation_error(error) from error + def _absorb(self, reply: Mapping[str, Any]) -> None: """Take the worker's measured memory and plan records.""" self._memory.update(cast("Mapping[str, int]", reply["memory"])) @@ -1394,6 +1416,7 @@ def evaluate( # noqa: PLR0912, PLR0915 - one evaluation's complete sequence ): self._diagnostic_cache_hits += 1 return self._cached + self._ensure_worker() archive = self._archive if archive is not None: archive.ensure_writable() @@ -1458,6 +1481,9 @@ def evaluate( # noqa: PLR0912, PLR0915 - one evaluation's complete sequence self._archive_failure(evaluation, index, stopping, "interrupted") raise except Exception as error: + # Whatever failed, this worker is not used again (SEP 49, 6.5): the + # next evaluation starts a fresh one. + self._worker.kill() self._budget.record_failure() classification = _classify(error) name, detail = _described(error) @@ -1958,6 +1984,7 @@ def diagnostic_currents( model = self._hacc if model is None: raise BackendError("the consistency diagnostic needs a haCC binding") + self._ensure_worker() context = _DiagnosticContext( model, self._budget, @@ -2145,6 +2172,7 @@ def _diagnostic_call( ) raise except Exception as error: + context.worker.kill() budget.record_failure() kind = _classify(error) name, detail = _described(error) diff --git a/tests/driver/test_source_worker.py b/tests/driver/test_source_worker.py index 5553469d1..e40829a65 100644 --- a/tests/driver/test_source_worker.py +++ b/tests/driver/test_source_worker.py @@ -149,7 +149,7 @@ def test_the_provider_s_printing_never_corrupts_a_frame(worker): assert worker.request("echo", [1, 2]) == [1, 2] -def test_a_provider_exception_is_reported_by_name_and_the_worker_serves_on(worker): +def test_a_provider_exception_is_reported_by_name_and_ends_the_worker(worker): with pytest.raises(RemoteError) as raised: worker.request("raise") error = raised.value @@ -157,7 +157,8 @@ def test_a_provider_exception_is_reported_by_name_and_the_worker_serves_on(worke assert error.detail == "manufactured provider failure" assert error.classification == "manufactured" assert "manufactured provider failure" in error.remote_traceback - assert worker.request("echo", 3) == 3 + # Never used again after a failure (section 6.5): its owner starts a fresh one. + _assert_ended(worker) def test_a_worker_that_exits_is_a_named_failure_with_its_status_and_stderr(worker): diff --git a/tests/driver/test_tdse_binding.py b/tests/driver/test_tdse_binding.py index 927fb4327..29f3ab372 100644 --- a/tests/driver/test_tdse_binding.py +++ b/tests/driver/test_tdse_binding.py @@ -744,5 +744,8 @@ def second_chunk_fails(plan, fields): result = binding.evaluate(np.ones((3, _SAMPLES)), z=0.5, purpose="corrected") assert_array_equal(result.total_current, np.full((3, _SAMPLES), 6.0)) assert binding.ledger.attempted == 4 - assert len(provider.created) == 2 + # The failed worker was ended with its plans; the retry's fresh worker prepared + # both chunk sizes' plans again (SEP 49 section 6.5). + assert len(provider.created) == 4 + assert provider.closed.count("plan-2") == provider.closed.count("plan-1") == 1 binding.close() diff --git a/tests/driver/test_tdse_worker.py b/tests/driver/test_tdse_worker.py index 91d448b78..bcefb3aa3 100644 --- a/tests/driver/test_tdse_worker.py +++ b/tests/driver/test_tdse_worker.py @@ -161,6 +161,25 @@ def test_a_stop_during_an_atomic_call_kills_the_worker_and_keeps_the_charge( binding.close() +def test_after_a_provider_exception_the_retry_gets_a_fresh_worker( + monkeypatch, workers, tmp_path +): + """A worker is never used again after a failure (section 6.5).""" + monkeypatch.setenv("MANUFACTURED_FAIL_ONCE", str(tmp_path / "failed-once")) + binding = _bind(_descriptor(preset_basis(1))) + binding.reserve(1) + with pytest.raises(BackendError, match=r"manufactured provider failure \[native\]"): + binding.evaluate(_fields(3), z=0.1, purpose="startup") + assert not workers[0].alive + binding.reserve(1) + result = binding.evaluate(_fields(3), z=0.1, purpose="startup") + assert np.all(np.isfinite(result.total_current)) + assert len(workers) == 2 and workers[1].alive + assert binding.ledger.attempted == 3 and binding.ledger.failed_calls == 1 + binding.close() + assert not workers[1].alive + + class _Watch: """Pass standard error on; set a flag when the worker says its call hangs.""" diff --git a/tests/source_adapters.py b/tests/source_adapters.py index 320978cdf..4d087c887 100644 --- a/tests/source_adapters.py +++ b/tests/source_adapters.py @@ -14,6 +14,7 @@ import os import sys import time +from pathlib import Path from typing import Any, cast import numpy as np @@ -91,12 +92,37 @@ def _provider(**changes): provider = FakeHaccProvider(**changes) # The run tests' law: a 2% linear model error and a dyadic fifth-order remainder. provider.response = alpha_law(1.02, lambda5=2.0**-200) + marker = os.environ.get("MANUFACTURED_FAIL_ONCE") + if marker is not None: + _fail_once(provider, marker) hang = os.environ.get("MANUFACTURED_HANG_AFTER") if hang is not None: _hang_after(provider, int(hang)) return provider +def _fail_once(provider, marker): + """Make the first atomic call of all the workers sharing `marker` raise.""" + from hacc_cases import FakeDrivenError # noqa: PLC0415 - worker only + + original = provider.hacc.ResponsePlan + + def plan(model, **values): + built = original(model, **values) + evaluate = built.evaluate + + def failing(fields): + if not Path(marker).exists(): + Path(marker).touch() + raise FakeDrivenError("manufactured provider failure") + return evaluate(fields) + + built.evaluate = failing + return built + + provider.hacc.ResponsePlan = plan + + def _hang_after(provider, calls): """Make every atomic call after the first `calls` hang.""" original = provider.hacc.ResponsePlan diff --git a/tests/source_channel.py b/tests/source_channel.py index 768a8a278..4fefaf931 100644 --- a/tests/source_channel.py +++ b/tests/source_channel.py @@ -54,6 +54,8 @@ def request(self, kind, value=None): try: reply = self._adapter.handle(kind, sent) except Exception as error: + # As a real worker: never used again after a failure. + self.close() raise _remote(error, self._adapter) from error return _round_trip(reply, self._adapter.computed) @@ -64,6 +66,10 @@ def close(self): kill = close + @property + def alive(self): + return not self.closed + def in_process(monkeypatch): """Make the binding serve its provider in this process; return the workers made.""" From 4726d56037bf29e793a480545b65ded2c7f8518a Mon Sep 17 00:00:00 2001 From: John Travers Date: Thu, 12 Nov 2026 13:55:59 +0000 Subject: [PATCH 10/20] SEP 49: harden the source worker after review Queue a failure for any reader exception; leave the worker through os._exit on every path; map a write to a dead worker to its exit; kill in close's finally; static reader threads and a finaliser; no blocking close of a stream still read; a stop owns an exit seen after it; -P and BLAS idle defaults for the worker; raise_held only inside an interruptible() scope on the main thread. Bounded manufactured hangs, signal guards and a zombie-aware orphan check in the tests. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HAYtF5P9QqEd3HQx3WJkA3 --- docs/dev/seps/sep-0049-tdse-on-native.md | 24 ++- src/solprop/_interrupts.py | 37 ++++- src/solprop/driver/_source_worker.py | 190 ++++++++++++++--------- src/solprop/driver/tdse.py | 6 +- tests/driver/test_interrupts.py | 43 ++++- tests/driver/test_source_worker.py | 115 +++++++++++++- tests/source_adapters.py | 13 +- 7 files changed, 328 insertions(+), 100 deletions(-) diff --git a/docs/dev/seps/sep-0049-tdse-on-native.md b/docs/dev/seps/sep-0049-tdse-on-native.md index 3675abee2..79d8ddc48 100644 --- a/docs/dev/seps/sep-0049-tdse-on-native.md +++ b/docs/dev/seps/sep-0049-tdse-on-native.md @@ -1109,9 +1109,29 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. lines. §6.8 estimated +0.6k source and +0.6k test lines for the worker; the excess is the adapter's request handlers and the parent's view of the haCC model beside the generic layer, as the binding's validation stayed in solprop's process. +- **Review follow-up** (an independent review of the pull request, 2026-11-12). The + worker's reader queues a failure for any exception, so a reply it cannot decode (a + bad tag, nesting too deep, memory) ends the wait as a protocol error instead of + leaving it polling; the worker's watchdog and its `main` leave through `os._exit` on + every path, never through interpreter finalisation with the provider loaded; a write + to a worker that died between calls is the worker's exit, with its status and + standard error; `close` kills the worker even when a signal interrupts its grace; the + reader threads hold the queue and stream, not the worker, and a finaliser kills a + worker its owner drops; a reader still blocked in its pipe leaves the stream + unclosed rather than blocking `kill`; a stop already asked for owns a worker exit + seen after it; the worker starts with `-P` and with solprop's BLAS idle defaults + (`child_environment`); the diagnostic closes its plan only on a live worker. + `_interrupts.raise_held` acts only inside an `interruptible()` scope on the main + thread, which native's session will open around its `advance` in phase 3, so any + other commit, the terminal checkpoint's `committing(deliver=False)` among them, + keeps or drops a held signal as before. The manufactured hangs are bounded (30 s), + so a broken cancellation fails a test rather than hanging it; new tests cover the + undecodable message, the dead worker, the interrupted close, collection, the + interruptible scope and its absence, another thread, and solprop-native's own wait + settings (`_wait.py`, read by path). - **Handoff.** Phase 3 (the native hook) calls `tdse_source` and the same binding; its - waits are inside `committing()`, which `Worker` already handles through - `_interrupts.raise_held`; §7.2 item 4 on native and §7.3 follow with it. The worker's + waits are inside `committing()`: the session wraps `advance` in + `_interrupts.interruptible()` too, and `Worker` raises a held signal there; §7.2 item 4 on native and §7.3 follow with it. The worker's `request` raises `StopRequested`, `KeyboardInterrupt` or `RunTerminated` unchanged, which the hook stores and the session re-raises (§6.3). diff --git a/src/solprop/_interrupts.py b/src/solprop/_interrupts.py index 2d30451ee..ea9c77415 100644 --- a/src/solprop/_interrupts.py +++ b/src/solprop/_interrupts.py @@ -28,6 +28,7 @@ "committing", "deferred_handlers", "handled", + "interruptible", "poll", "raise_held", ] @@ -49,6 +50,8 @@ def __init__(self) -> None: """Start outside every commit with nothing pending.""" self.depth = 0 self.pending: BaseException | None = None + # How many interruptible scopes the main thread is in (:func:`interruptible`). + self.interruptible = 0 _GUARD = _Guard() @@ -87,15 +90,37 @@ def __exit__(self, kind: type[BaseException] | None, *_: object) -> None: raise pending +@contextmanager +def interruptible() -> Iterator[None]: + """Let a long wait inside a commit take the signal the commit holds. + + A native ``advance`` runs inside :class:`committing`, and a host source it calls may + wait minutes for an atomic call (SEP 49 section 6.7). Its session marks that wait + with this scope, and the wait calls :func:`raise_held` between polls, so that a + Ctrl-C or SIGTERM stops it there, before the trial commits. Only the main thread's + scope counts, since signals reach the main thread alone; any other commit -- the + terminal checkpoint's ``committing(deliver=False)`` among them -- keeps holding. + """ + main = threading.current_thread() is threading.main_thread() + if main: + _GUARD.interruptible += 1 + try: + yield + finally: + if main: + _GUARD.interruptible -= 1 + + def raise_held() -> None: - """Raise, now, a signal the enclosing commit is holding. + """Raise, now, a signal the commit holds, inside an :func:`interruptible` scope. - For a wait inside a commit that is not itself one: a native ``advance`` runs inside - :class:`committing`, and a host source it calls may wait minutes for an atomic call - (SEP 49 section 6.7). Its wait loop calls this between polls, so that a Ctrl-C or - SIGTERM stops it there, before the trial commits, rather than when the call ends. - Outside a commit nothing is ever held, and this does nothing. + Does nothing outside such a scope, on any thread but the main one, or when nothing + is held. """ + if not _GUARD.interruptible or threading.current_thread() is not ( + threading.main_thread() + ): + return pending, _GUARD.pending = _GUARD.pending, None if pending is not None: raise pending diff --git a/src/solprop/driver/_source_worker.py b/src/solprop/driver/_source_worker.py index 86cd49fab..ae5915004 100644 --- a/src/solprop/driver/_source_worker.py +++ b/src/solprop/driver/_source_worker.py @@ -45,6 +45,7 @@ import sys import threading import traceback +import weakref from collections import deque from collections.abc import Callable, Iterator, Mapping, Sequence from types import SimpleNamespace @@ -53,6 +54,7 @@ import numpy as np from solprop._interrupts import poll, raise_held +from solprop._process import child_environment from solprop.exceptions import BackendError from solprop.log import StopFlag @@ -325,7 +327,9 @@ def read_message( return kind, decode(body, arrays), arrays except WorkerError: raise - except (ValueError, TypeError, KeyError, IndexError, UnicodeDecodeError) as error: + except Exception as error: + # Any message this side cannot read (bad JSON or tags, too deep, too large) + # is the worker's protocol error, never this process's. msg = f"a malformed worker message: {type(error).__name__}: {error}" raise WorkerError(msg) from error @@ -370,7 +374,7 @@ def worker_environment( own settings pass through. The worker imports from this process's import path. ``explicit`` (the thread count, the device) wins. """ - environment = dict(environ) + environment = child_environment(environ) environment["PYTHONPATH"] = os.pathsep.join(entry for entry in sys.path if entry) settings = _NATIVE_WAIT_SETTINGS.get(environment.get(_NATIVE_WAIT_MARKER, ""), {}) if settings: @@ -415,8 +419,9 @@ def __init__( queue.Queue() ) self._tail: deque[str] = deque(maxlen=_TAIL_LINES) - self._threads: list[threading.Thread] = [] + self._threads: dict[IO[bytes], threading.Thread] = {} self._usable = False + self._finalizer: weakref.finalize[Any, Any] | None = None # -- lifetime ------------------------------------------------------------------- @@ -432,6 +437,8 @@ def start(self) -> Mapping[str, object]: self._process = subprocess.Popen( [ sys.executable, + # The parent's working directory is not on the worker's path. + "-P", # The package imports this module before ``-m`` runs it, which runpy # warns of; ``main`` runs the imported copy, so nothing is duplicated. "-W", @@ -450,13 +457,17 @@ def start(self) -> Mapping[str, object]: ) process = self._process assert process.stdout is not None and process.stderr is not None - for target, stream in ( - (self._read, process.stdout), - (self._drain, process.stderr), - ): - thread = threading.Thread(target=target, args=(stream,), daemon=True) + # The threads hold the queue, the stream and the tail, never this object, so + # a worker its owner forgets is collected, and its finaliser kills it. + readers = ( + (_reader, process.stdout, (self.limit, self._replies)), + (_drain, process.stderr, (self._tail,)), + ) + for target, stream, extra in readers: + thread = threading.Thread(target=target, args=(stream, *extra), daemon=True) thread.start() - self._threads.append(thread) + self._threads[stream] = thread + self._finalizer = weakref.finalize(self, _end, process, self._threads) try: kind, body = self._wait() if kind != "hello": @@ -495,8 +506,12 @@ def request(self, kind: str, value: object = None) -> object: assert process.stdin is not None stdin = process.stdin try: - write_message(stdin.write, kind, value) - stdin.flush() + try: + write_message(stdin.write, kind, value) + stdin.flush() + except OSError as broken: + # A worker that died between calls: its status and stderr say why. + raise self._exited() from broken reply, body = self._wait() except BaseException: self.kill() @@ -512,41 +527,23 @@ def close(self) -> None: process = self._process if process is None: return - if self._usable and process.poll() is None: - try: + try: + if self._usable and process.poll() is None: assert process.stdin is not None write_message(process.stdin.write, "close", None) process.stdin.flush() process.wait(timeout=CLOSE_GRACE) - except OSError, subprocess.TimeoutExpired, ValueError: - pass - self.kill() + except OSError, subprocess.TimeoutExpired, ValueError: + pass + finally: + # A signal during the grace still ends the worker before it propagates. + self.kill() def kill(self) -> None: """End the worker at once, reap it and close both pipe ends.""" self._usable = False - process = self._process - if process is None: - return - if process.poll() is None: - with contextlib.suppress(ProcessLookupError, PermissionError, OSError): - if sys.platform == "win32": # pragma: no cover - Windows only - process.kill() - else: - # Its own session: the group holds anything the provider started. - os.killpg(process.pid, signal.SIGKILL) - process.wait() - if process.stdin is not None: - with contextlib.suppress(OSError): - process.stdin.close() - # The worker's ends are closed now, so its readers see the end of their pipes. - for thread in self._threads: - thread.join(timeout=CLOSE_GRACE) - self._threads = [] - for stream in (process.stdout, process.stderr): - if stream is not None: - with contextlib.suppress(OSError): - stream.close() + if self._finalizer is not None: + self._finalizer() @property def alive(self) -> bool: @@ -566,10 +563,11 @@ def _wait(self) -> tuple[str, object]: raise_held() poll(self.stop) continue - if isinstance(item, BaseException): - raise item - if item is None: - raise self._exited() + if isinstance(item, BaseException) or item is None: + # A stop already asked for owns a worker exit seen after it (6.7). + raise_held() + poll(self.stop) + raise self._exited() if item is None else item kind, body = item if kind == "error": report = cast("Mapping[str, str]", body) @@ -599,29 +597,67 @@ def _exited(self) -> WorkerError: error.add_note(f"its last standard error:\n{tail}") return error - def _read(self, stream: IO[bytes]) -> None: - """Move the worker's messages to the reply queue until its pipe ends.""" - try: - while True: - message = read_message(stream.read, self.limit) - if message is None: - self._replies.put(None) - return - kind, body, _ = message - self._replies.put((kind, body)) - except WorkerError as error: - self._replies.put(error) - except OSError, ValueError: - # The parent closed the pipe while killing the worker. - self._replies.put(None) - - def _drain(self, stream: IO[bytes]) -> None: - """Pass the worker's standard error on, keeping its last lines for a failure.""" - for line in iter(stream.readline, b""): - text = line.decode("utf-8", errors="replace").rstrip("\n") - self._tail.append(text) - with contextlib.suppress(OSError, ValueError): - sys.stderr.write(text + "\n") + +def _end( + process: subprocess.Popen[bytes], threads: dict[IO[bytes], threading.Thread] +) -> None: + """Kill a worker's process group, reap it and close the pipes its readers left. + + A stream whose reader is still inside a read (a grandchild holding the pipe open) + is left for collection: closing it would wait on the reader's buffer lock. + """ + if process.poll() is None: + with contextlib.suppress(ProcessLookupError, PermissionError, OSError): + if sys.platform == "win32": # pragma: no cover - Windows only + process.kill() + else: + # Its own session: the group holds anything the provider started. + os.killpg(process.pid, signal.SIGKILL) + process.wait() + if process.stdin is not None: + with contextlib.suppress(OSError): + process.stdin.close() + # The worker's ends are closed now, so its readers see the end of their pipes. + for stream, thread in threads.items(): + thread.join(timeout=CLOSE_GRACE) + if not thread.is_alive(): + with contextlib.suppress(OSError): + stream.close() + + +def _reader( + stream: IO[bytes], + limit: int, + replies: queue.Queue[tuple[str, object] | BaseException | None], +) -> None: + """Move the worker's messages to the reply queue until its pipe ends. + + Every way out queues something, so the wait never outlives its reader. + """ + try: + while True: + message = read_message(stream.read, limit) + if message is None: + replies.put(None) + return + kind, body, _ = message + replies.put((kind, body)) + except WorkerError as error: + replies.put(error) + except OSError, ValueError: + # The parent closed the pipe while killing the worker. + replies.put(None) + except BaseException as error: + replies.put(WorkerError(f"the worker's reply could not be read: {error!r}")) + + +def _drain(stream: IO[bytes], tail: deque[str]) -> None: + """Pass the worker's standard error on, keeping its last lines for a failure.""" + for line in iter(stream.readline, b""): + text = line.decode("utf-8", errors="replace").rstrip("\n") + tail.append(text) + with contextlib.suppress(OSError, ValueError): + sys.stderr.write(text + "\n") # ----------------------------------------------------------------------- the worker @@ -677,7 +713,7 @@ def _watch(inbound: int, limit: int, inbox: queue.Queue[tuple[str, object]]) -> break kind, body, _ = message inbox.put((kind, body)) - except WorkerError, OSError: + except BaseException: pass # The parent is gone, or wrote garbage: nothing the worker computes can be used. os._exit(_ORPHANED) @@ -735,11 +771,11 @@ def _send(kind: str, value: object, adapter: Adapter | None = None) -> None: adapter.close() return 0 try: - reply = adapter.handle(kind, body) + # The reply is encoded whole before a byte is written, so a reply the + # codec refuses is reported as an error, not half sent. + _send("reply", adapter.handle(kind, body), adapter) except Exception as error: _send("error", _error_report(error, adapter)) - continue - _send("reply", reply, adapter) def _write_all(descriptor: int, data: bytes) -> None: @@ -754,8 +790,14 @@ def _write_all(descriptor: int, data: bytes) -> None: # an adapter receives are the classes it imports. from solprop.driver._source_worker import main as _main - status = _main(sys.argv[1:]) - # Skip interpreter finalisation: the provider's runtime is already closed by the - # adapter, and nothing else of the worker's needs flushing. - sys.stderr.flush() - os._exit(status) + status = 1 + try: + status = _main(sys.argv[1:]) + except BaseException: + traceback.print_exc() + finally: + # Every way out skips interpreter finalisation: the provider's runtime (Kokkos) + # is closed by the adapter or abandoned with the process, never finalised. + with contextlib.suppress(BaseException): + sys.stderr.flush() + os._exit(status) diff --git a/src/solprop/driver/tdse.py b/src/solprop/driver/tdse.py index a6c476a03..09aebed53 100644 --- a/src/solprop/driver/tdse.py +++ b/src/solprop/driver/tdse.py @@ -2139,8 +2139,10 @@ def _diagnostic( finally: budget.release_reservation() finally: - with contextlib.suppress(RemoteError, WorkerError): - worker.request("diagnostic_close", {}) + # A failed or cancelled call has ended the worker already; nothing to close. + if worker.alive: + with contextlib.suppress(RemoteError, WorkerError): + worker.request("diagnostic_close", {}) return ( np.array(total[0], dtype=np.float64, copy=True), np.array(first[0], dtype=np.float64, copy=True), diff --git a/tests/driver/test_interrupts.py b/tests/driver/test_interrupts.py index a0841d265..bc8dd2ce6 100644 --- a/tests/driver/test_interrupts.py +++ b/tests/driver/test_interrupts.py @@ -21,7 +21,7 @@ from driver.test_resume import N_SAVE, CrashError from driver.test_resume_history import RTOL, assert_accurate_run, soliton, tight_run from march_bounds import TIGHTER -from solprop._interrupts import committing, handled, raise_held +from solprop._interrupts import committing, handled, interruptible, raise_held from solprop.exceptions import RunTerminated from solprop.log import NullReporter from solprop.output.hdf5 import open_file, resume @@ -146,12 +146,11 @@ def test_a_signal_inside_a_commit_waits_for_its_end(): @needs_default_sigterm -def test_a_wait_inside_a_commit_takes_the_held_signal_at_once(): +def test_an_interruptible_wait_inside_a_commit_takes_the_held_signal_at_once(): """A host source's wait inside native's commit raises the signal (SEP 49, 6.7).""" events = [] with handled(): - raise_held() # nothing is held outside a commit - with pytest.raises(RunTerminated), committing(): + with pytest.raises(RunTerminated), committing(), interruptible(): signal.raise_signal(signal.SIGTERM) events.append("held") raise_held() @@ -160,6 +159,42 @@ def test_a_wait_inside_a_commit_takes_the_held_signal_at_once(): assert events == ["held", "nothing left to deliver"] +@needs_default_sigterm +def test_outside_an_interruptible_scope_a_held_signal_stays_held(): + events = [] + with handled(): + # Any other commit keeps the signal for its end... + with pytest.raises(RunTerminated), committing(): + signal.raise_signal(signal.SIGTERM) + raise_held() + events.append("still held") + # ...and one that drops signals drops it. + with committing(deliver=False): + signal.raise_signal(signal.SIGTERM) + raise_held() + events.append("dropped at its end") + assert events == ["still held", "dropped at its end"] + + +@needs_default_sigterm +def test_another_thread_cannot_take_the_main_thread_s_held_signal(): + taken = [] + + def wait(): + with interruptible(): + try: + raise_held() + except RunTerminated: + taken.append("taken") + + with handled(), pytest.raises(RunTerminated), committing(), interruptible(): + signal.raise_signal(signal.SIGTERM) + other = threading.Thread(target=wait) + other.start() + other.join() + assert taken == [] + + def test_an_application_s_own_handler_is_kept(): def own(signal_number, frame): pass diff --git a/tests/driver/test_source_worker.py b/tests/driver/test_source_worker.py index e40829a65..1fa0361ea 100644 --- a/tests/driver/test_source_worker.py +++ b/tests/driver/test_source_worker.py @@ -11,6 +11,9 @@ """ import dataclasses +import gc +import importlib.util +import io import json import os import signal @@ -18,12 +21,14 @@ import sys import threading import time +from pathlib import Path from typing import Any import numpy as np +import psutil import pytest -from solprop._interrupts import committing, handled +from solprop._interrupts import committing, handled, interruptible from solprop.driver import _source_worker from solprop.driver._source_worker import ( POLL_INTERVAL, @@ -133,6 +138,28 @@ def test_the_worker_environment_drops_native_s_own_wait_settings_only(): assert kept["GOMP_SPINCOUNT"] == "5" and "KMP_BLOCKTIME" not in kept +def test_the_native_wait_settings_are_solprop_native_s_own(monkeypatch): + """The worker removes exactly what solprop-native sets (ADR-071).""" + spec = importlib.util.find_spec("solprop_native") + if spec is None or spec.origin is None: + pytest.skip("solprop-native is not installed") + # Loaded by path: importing the package would start the native runtime. + path = Path(spec.origin).parent / "_wait.py" + loader = importlib.util.spec_from_file_location("native_wait_probe", path) + assert loader is not None and loader.loader is not None + module = importlib.util.module_from_spec(loader) + # Its dataclasses look their module up by name. + monkeypatch.setitem(sys.modules, "native_wait_probe", module) + loader.loader.exec_module(module) + assert _source_worker._NATIVE_WAIT_MARKER == module.MARKER + assert _source_worker._NATIVE_WAIT_SETTINGS == module._SETTINGS + + +def test_the_worker_environment_keeps_blas_idle(): + environment = worker_environment({}, {}) + assert "OPENBLAS_THREAD_TIMEOUT" in environment + + def test_values_cross_a_real_worker_bit_for_bit(worker): rng = np.random.default_rng(49) sent = {"field": rng.standard_normal((3, 257)), "labels": (1, 3), "z": 0.25} @@ -208,6 +235,53 @@ def test_a_protocol_mismatch_is_refused_before_any_request(monkeypatch): _assert_ended(worker) +@pytest.mark.parametrize( + "body", + ['{"$d": []}', "[" * 100_000 + "]" * 100_000], + ids=["bad tag", "too deep"], +) +def test_a_message_this_side_cannot_decode_is_a_protocol_error(body): + control = f'{{"kind": "reply", "body": {body}, "arrays": []}}'.encode() + stream = io.BytesIO(_source_worker._HEADER.pack(len(control)) + control) + with pytest.raises(WorkerError, match="malformed worker message"): + _source_worker.read_message(stream.read, len(control)) + + +@POSIX +def test_a_worker_that_died_between_calls_is_a_named_failure(worker): + assert worker._process is not None + os.killpg(worker._process.pid, signal.SIGKILL) + worker._process.wait(timeout=DEADLINE) + with pytest.raises(WorkerError, match="exit status -9"): + worker.request("echo", 1) + _assert_ended(worker) + + +def test_an_interruption_during_close_still_ends_the_worker(worker, monkeypatch): + process = worker._process + assert process is not None + wait = process.wait + + def interrupted(timeout=None): + if timeout is not None: + raise KeyboardInterrupt + return wait() + + monkeypatch.setattr(process, "wait", interrupted) + with pytest.raises(KeyboardInterrupt): + worker.close() + assert process.returncode is not None and not worker.alive + + +def test_a_worker_its_owner_forgets_is_ended_when_collected(): + worker = _worker() + process = worker._process + assert process is not None + del worker + gc.collect() + assert process.returncode is not None + + def test_close_between_calls_ends_the_worker_in_order(worker): worker.close() _assert_ended(worker) @@ -236,25 +310,46 @@ def test_a_stop_flag_cancels_a_hanging_call_at_the_next_poll(worker): [(signal.SIGINT, KeyboardInterrupt), (signal.SIGTERM, RunTerminated)], ) def test_a_signal_cancels_a_hanging_call(worker, number, expected): - if signal.getsignal(signal.SIGTERM) is not signal.SIG_DFL: - pytest.skip("an application handler owns SIGTERM here") + defaults = { + signal.SIGTERM: signal.SIG_DFL, + signal.SIGINT: signal.default_int_handler, + } + if signal.getsignal(number) is not defaults[number]: + pytest.skip("an application handler owns the signal here") with handled(): threading.Timer(0.3, os.kill, (os.getpid(), number)).start() assert _timed(worker, "hang", expected) < DEADLINE @POSIX -def test_a_signal_held_by_a_commit_is_taken_by_the_wait(worker): +def test_a_signal_held_by_a_commit_is_taken_by_an_interruptible_wait(worker): """Native's ``advance`` waits inside ``committing``: the loop raises the signal.""" + if signal.getsignal(signal.SIGINT) is not signal.default_int_handler: + pytest.skip("an application handler owns SIGINT here") with handled(): threading.Timer(0.3, os.kill, (os.getpid(), signal.SIGINT)).start() started = time.monotonic() - with pytest.raises(KeyboardInterrupt), committing(): + with pytest.raises(KeyboardInterrupt), committing(), interruptible(): worker.request("hang") assert time.monotonic() - started < DEADLINE _assert_ended(worker) +@POSIX +def test_outside_an_interruptible_scope_the_commit_keeps_the_signal(worker): + """The stop flag ends the wait; the signal stays held, dropped with the stop.""" + if signal.getsignal(signal.SIGINT) is not signal.default_int_handler: + pytest.skip("an application handler owns SIGINT here") + flag = threading.Event() + worker.stop = flag + with handled(): + threading.Timer(0.3, os.kill, (os.getpid(), signal.SIGINT)).start() + threading.Timer(1.0, flag.set).start() + with pytest.raises(StopRequested), committing(): + worker.request("hang") + _assert_ended(worker) + + def test_the_worker_imports_neither_backend_nor_the_parent_s_main(worker): loaded = worker.request("modules") names = loaded["modules"] @@ -321,12 +416,16 @@ def test_a_worker_started_from_a_script_or_python_c_never_re_imports_it(tmp_path def _gone(pid): - """Wait, within the harness deadline, for a process to disappear.""" + """Wait, within the harness deadline, for a process to end. + + A zombie has ended: in a container nothing may reap an orphan. + """ started = time.monotonic() while time.monotonic() - started < DEADLINE: try: - os.kill(pid, 0) - except ProcessLookupError: + if psutil.Process(pid).status() == psutil.STATUS_ZOMBIE: + return True + except psutil.NoSuchProcess: return True time.sleep(0.05) return False diff --git a/tests/source_adapters.py b/tests/source_adapters.py index 4d087c887..369dd4ee4 100644 --- a/tests/source_adapters.py +++ b/tests/source_adapters.py @@ -21,6 +21,10 @@ from solprop.driver import _pyroraptor, _source_worker +HANG_SECONDS = 30.0 +"""How long a manufactured hang lasts: bounded, so that a broken cancellation fails a +test (the call returns) instead of hanging it, and far above any cancellation's time.""" + class _Manufactured: """An adapter with no provider: each request names its behaviour.""" @@ -40,7 +44,8 @@ def handle(self, kind, value): if kind == "echo": return value if kind == "hang": - time.sleep(3600) + time.sleep(HANG_SECONDS) + return "woke" if kind == "raise": raise ValueError("manufactured provider failure") if kind == "exit": @@ -60,10 +65,10 @@ def handle(self, kind, value): _source_worker.PROTOCOL_DESCRIPTORS[1], _source_worker._HEADER.pack(len(payload)) + payload, ) - time.sleep(3600) + time.sleep(HANG_SECONDS) if kind == "close_pipe": os.close(_source_worker.PROTOCOL_DESCRIPTORS[1]) - time.sleep(3600) + time.sleep(HANG_SECONDS) if kind == "modules": spec = getattr(sys.modules["__main__"], "__spec__", None) return { @@ -135,7 +140,7 @@ def slow(fields): if len(provider.calls) >= calls: sys.stderr.write("MANUFACTURED CALL HANGS\n") sys.stderr.flush() - time.sleep(3600) + time.sleep(HANG_SECONDS) return evaluate(fields) built.evaluate = slow From 478b3808bb665434b36cce7fb80b5a0c2bf487f0 Mon Sep 17 00:00:00 2001 From: John Travers Date: Thu, 12 Nov 2026 14:53:29 +0000 Subject: [PATCH 11/20] SEP 49 phase 3: the TDSE response on native through a host source hook The resident Adams march calls a generic host source -- begin(cost) after its budget check, evaluate(field, z, purpose, out) at each source evaluation, and end(outcome) after the decision and the commit -- from the advancing thread, outside every parallel region; the binding layer takes the interpreter lock and keeps a Python exception for the session to raise. The engine's own response pass is skipped and its finish applies the collar, absorber and propagator. Device builds refuse a TDSE problem by name. The native adapter binds the driver's binding through the shared source assembly, records the TDSE metadata and memory rows, restores atomic ledgers and opens an interruptible scope around advance; auto now runs TDSE on native. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HAYtF5P9QqEd3HQx3WJkA3 --- AGENTS.md | 2 +- changelog.d/tdse-on-native.changed.md | 1 + docs/dev/backends.md | 10 +- docs/dev/kernel-contract.md | 37 +- docs/dev/seps/sep-0049-tdse-on-native.md | 114 ++++- docs/guide/media/tdse.md | 7 +- docs/guide/performance/backends.md | 15 +- native/AGENTS.md | 9 +- native/include/solprop_native/adams_march.hpp | 11 +- native/include/solprop_native/core.hpp | 14 + native/include/solprop_native/engines.hpp | 4 + native/include/solprop_native/host_source.hpp | 61 +++ native/src/adams_march.cpp | 59 ++- native/src/bindings.cpp | 59 +++ native/src/engines.cpp | 14 + native/src/general/kernels.hpp | 29 +- native/src/general/setup.cpp | 5 + native/src/general/state.cpp | 12 + native/src/general/trial.cpp | 61 +++ native/src/general/workspace.hpp | 14 + native/src/solprop_native/_core.pyi | 1 + native/src/solprop_native/_host_source.py | 135 ++++++ native/src/solprop_native/backend.py | 66 ++- native/src/solprop_native/session.py | 61 ++- native/tests/test_backend.py | 16 +- src/solprop/backends/numpy_ref/backend.py | 27 +- src/solprop/backends/registry.py | 12 +- src/solprop/backends/tdse_source.py | 38 +- src/solprop/capabilities.py | 8 +- src/solprop/driver/simulation.py | 22 +- tests/backends/test_tdse_march.py | 39 +- tests/backends/test_tdse_native.py | 432 ++++++++++++++++++ tests/driver/test_initial_z_and_checkpoint.py | 4 +- tests/driver/test_interface_defaults.py | 7 +- tests/driver/test_tdse_entry.py | 12 +- 35 files changed, 1270 insertions(+), 148 deletions(-) create mode 100644 changelog.d/tdse-on-native.changed.md create mode 100644 native/include/solprop_native/host_source.hpp create mode 100644 native/src/solprop_native/_host_source.py create mode 100644 tests/backends/test_tdse_native.py diff --git a/AGENTS.md b/AGENTS.md index 3a1b1f192..682a30c72 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -24,7 +24,7 @@ physics or numerics rather than style, ask rather than guess. | Purpose | Ultrafast nonlinear pulse propagation (UPPE and GNLSE; field and envelope; single-mode, multimode and general modal guides; radial, Cartesian, planar and line free space) | | Python | 3.14 — equal to `requires-python` in `pyproject.toml` | | Platforms | solprop runs where solprop-native installs: wheels for osx-arm64, linux-64, linux-aarch64, win-64 and win-arm64; from source on Linux and macOS (GPU builds on Linux); the Queue and Slurm scan executors POSIX-only | -| Backends | `native` (required; `auto`; CPU and GPU; implements every feature except the TDSE response) and `numpy_ref` (the numerical reference). No other backend. Binary64 only (ADR-077): the single-precision code, CF4 sections, `ComparisonMultistep` and `PIController` overrides that still run on `numpy_ref` are being removed | +| Backends | `native` (required; `auto`; CPU and GPU; implements every feature, the TDSE response on CPU builds only) and `numpy_ref` (the numerical reference). No other backend. Binary64 only (ADR-077): the single-precision code, CF4 sections, `ComparisonMultistep` and `PIController` overrides that still run on `numpy_ref` are being removed | | Type checker | `basedpyright`, pinned exactly in the dev group | | Materials | [optmat](https://github.com/LupoLab/optmat), the curated optical-materials package | | Planned work | [GitHub issues](https://github.com/LupoLab/solprop/issues), not the roadmap, which lists only SEPs (ADR-062); [`CONTRIBUTING.md`](CONTRIBUTING.md) says how to write and label one | diff --git a/changelog.d/tdse-on-native.changed.md b/changelog.d/tdse-on-native.changed.md new file mode 100644 index 000000000..e4d0c8265 --- /dev/null +++ b/changelog.d/tdse-on-native.changed.md @@ -0,0 +1 @@ +The TDSE response runs on the native backend's CPU build, and `backend="auto"` now selects native for a TDSE run as for every other; `backend="numpy_ref"` still runs it, and a native GPU build refuses it by name (SEP 49). diff --git a/docs/dev/backends.md b/docs/dev/backends.md index d4e92bd84..a7f1f3086 100644 --- a/docs/dev/backends.md +++ b/docs/dev/backends.md @@ -17,8 +17,8 @@ processor is at or above a build's baseline, and otherwise raises `BackendError` to do; it stores that name before execution or worker transfer. `auto` has no fallback. Native being installed is a packaging policy ([SEP 43](seps/archive/sep-0043-retire-numba-cpu.md) §5), so a run does not have to prove that native could execute here, and nothing loads a runtime to -enforce it. A TDSE response's `auto` resolves before the registry, to `numpy_ref`, and -never consults native. Explicit names are preserved, so `backend="numpy_ref"` runs where +enforce it. A TDSE response's `auto` resolves to `native` like every other run (SEP 49). +Explicit names are preserved, so `backend="numpy_ref"` runs where native is absent and `backend="native"` on a processor below the baseline raises the package's refusal. `select` also accepts `"auto"`, checks `SOLPROP_REQUIRE_DEVICE` against the actual build before anything is allocated, and logs the actual backend at INFO after @@ -367,9 +367,9 @@ threading beyond what `scipy.fft` does with `workers=1`. It exists to be read an every step of the anatomy in [architecture.md §8](architecture.md) is a named function with a docstring stating the formula. It is slow (tens of separate array operations per stage) and that is acceptable; it is the reference against which every other backend is validated, and -the supported executor of the one feature `native` does not run yet: the TDSE -response -([the list](../guide/performance/backends.md#features-that-run-on-the-reference-backend)). +the supported executor of any feature `native` does not run yet, of which there is none +since the TDSE response joined the native CPU build (SEP 49; +[the list](../guide/performance/backends.md#features-that-run-on-the-reference-backend)). A new feature joins the list only as an accepted exception (§10). **The fused-pass route.** `backends/numpy_ref/passes.py` holds one pure diff --git a/docs/dev/kernel-contract.md b/docs/dev/kernel-contract.md index 9f6248453..13d5e47d4 100644 --- a/docs/dev/kernel-contract.md +++ b/docs/dev/kernel-contract.md @@ -88,11 +88,11 @@ mode-averaged effective-area route and the resolved quadrature, a wall's coupled | field | type | meaning | |---|---|---| | `transform` | frozen dataclass | §4, of kind `"mode_averaged"` or `"modal"`, with neither `geometry` nor `modal_tables` | -| `tdse` | `TDSESource` or `None` | the TDSE response, the modal route's reference-only extension (§6); `None` without one | +| `tdse` | `TDSESource` or `None` | the TDSE response, the modal route's extension (§6); `None` without one | -`TDSESource` — the TDSE response's immutable scientific identity. Only `numpy_ref` -executes it; `native` refuses it by name. Its execution resource, the atomic binding, is -the driver's and crosses explicitly (§6). +`TDSESource` — the TDSE response's immutable scientific identity. `numpy_ref` and the +native CPU build execute it; a native device build refuses it by name. Its execution +resource, the atomic binding, is the driver's and crosses explicitly (§6). | field | type | meaning | |---|---|---| @@ -278,7 +278,7 @@ invocation, including failures. Logical source-call and atomic-attempt ledgers remain separate. The base driver owns `_adams`, initially `None`, and exports and restores its logical ledger on every Adams backend. Loading a checkpoint resets numerical history but preserves charged calls; startup is charged again. Atomic -ledger restoration remains exclusive to the bound NumPy TDSE path. +ledger restoration needs a bound TDSE binding, on either backend. **The lab-frame source of the exponential formulation.** `HostDriver._adams_lab_source(lab, position, *, purpose)` is the complete source @@ -310,7 +310,8 @@ deleted its host-engine fallback, and with it its `_adams_source`, `_adams_lab_s `TDSEPurpose` label so startup, predictor, corrected-field and diagnostic observations retain their actual provenance. This is a host seam operation: the shared march neither accesses NumPy's private binding nor adds atomic -physics to a compiled backend. The native backend continues to refuse TDSE by name. +physics to a compiled backend. The native backend's march reaches the same binding +through its host source hook (§6). A corrected-field diagnostic still needs an identical cached response or a separately charged solve; a predictor label never implies accepted-field data. If the corrected-field solve fails after candidate adoption, `_adams_restore` @@ -1028,8 +1029,26 @@ stepper's decision on the latest trial. The reference backend calls it from its step record (`_record`), which every Adams attempt reaches, rejected or accepted; it changes no step, charge or reservation. None of this is part of the descriptors or the kernel problem, which compiled backends see, so it is -outside the kernel schema: the native backend refuses TDSE by name and never holds a -binding. +outside the kernel schema. + +**The native host source hook** ([SEP 49](seps/sep-0049-tdse-on-native.md)). On the +native CPU build the binding reaches the resident Adams march as a host source: a +session binding outside the kernel problem, whose three entry points the march calls +from the thread that called `advance`, outside every parallel region, holding the +interpreter lock for each. `begin(cost)` reserves an attempt's calls after the march's +budget check; `evaluate(field, z, purpose, out)` receives the lab-frame spectrum the +engine would hand its own source pass and writes `F`, the projected source before the +spectral collar, which the shared assembly (`solprop.backends.tdse_source`) forms as +the reference does; `end(outcome)` records an accepted or rejected attempt after its +decision, and after its commit for an accepted one, then releases the reservation, and +an attempt that leaves before a decision only releases it. The engine's own response +pass does not run; its stage finish applies the spectral collar, the in-stage absorber +and the propagator. The hook's two host rows are allocated when it is bound and listed +as `host_source` in the memory ledger. An exception in an entry point is kept on the +Python side and reported to the march as a host failure: before the commit the +attempt is abandoned, after it the committed attempt is returned with the failure, +and the session raises the kept exception unchanged. A device build refuses a TDSE +problem at preparation. The backend independently projects macroscopic P and J with the existing modal weights and crop scale. It subtracts unscaled discrete shadows, then applies @@ -1117,7 +1136,7 @@ launch's remaining planes). | `schedule` | `(n_steps,) float64` or `None` | fixed schedule to replay (lockstep batches require it); its edges must contain every save plane and hard kink, which `FixedSchedule.uniform` guarantees for a plain fixed-step run | | `save_planes` | `(n_z,) float64` | z positions the driver will ask for; steps land on them unless `interpolate_planes` is set | | `propagator_mode` | `"auto" \| "tabulate" \| "recompute"` | ADR-004 and [backends.md §3.6](backends.md). `"auto"` is the default and is the backend's own choice, resolved at construction and written into the run's `backend` metadata group: `tabulate` for `numpy_ref` and `recompute` for `native` on a modal problem; `recompute` on every backend for a separable free-space operator, whose increment `D (z₂ − z₁)` is exact. Under the low-storage march `tabulate` is one volume of `exp(h D / 2)` reused while the step holds (a varying operator's `P` and `Q` for the step), admitted on the free and planar kinds too, and `"auto"` resolves to `tabulate` for a constant operator and `recompute` for a varying one (SEP 24 §4.3), and to `recompute` on a native device build, where an extra field read costs more than the exponential | -| `integrator` | `Multistep \| LowStorageRungeKutta \| None` | The method stepping the problem instead of the tableau, or `None` for the Runge-Kutta march. `Multistep` ([SEP 13](seps/archive/sep-0013-expensive-response-steppers.md), [SEP 16](seps/archive/sep-0016-unified-multistep.md)) is the Adams march in its exponential or interaction formulation, with an optional carrier frame (`carrier_frame`, [SEP 35](seps/archive/sep-0035-carrier-frame-exponential-adams.md), added in version 37: `"off"` or `"scalar"`, the latter on `numpy_ref` and native CPU builds and refused by name elsewhere); it runs on `numpy_ref` and Kokkos on every diagonal-flow route, and beside the constant-Schur and frozen-residual coupled flows in the exponential formulation; with a `schedule` it replays the schedule's steps (`HostAdamsMarch._replay_to`); TDSE remains NumPy-only. `LowStorageRungeKutta` ([SEP 24](seps/archive/sep-0024-low-storage-interaction-picture-runge-kutta.md)) is RK4 in the midpoint interaction picture in three registers: it replays a `schedule` and nothing else, records the norm 0.0 ("not estimated"), refuses a coupled flow, the TDSE response, interpolated planes and a step past RK4's negative-real interval 2.785, and runs on `numpy_ref` and solprop-native (a resident engine whose step is one native call) | +| `integrator` | `Multistep \| LowStorageRungeKutta \| None` | The method stepping the problem instead of the tableau, or `None` for the Runge-Kutta march. `Multistep` ([SEP 13](seps/archive/sep-0013-expensive-response-steppers.md), [SEP 16](seps/archive/sep-0016-unified-multistep.md)) is the Adams march in its exponential or interaction formulation, with an optional carrier frame (`carrier_frame`, [SEP 35](seps/archive/sep-0035-carrier-frame-exponential-adams.md), added in version 37: `"off"` or `"scalar"`, the latter on `numpy_ref` and native CPU builds and refused by name elsewhere); it runs on `numpy_ref` and Kokkos on every diagonal-flow route, and beside the constant-Schur and frozen-residual coupled flows in the exponential formulation; with a `schedule` it replays the schedule's steps (`HostAdamsMarch._replay_to`); the TDSE response runs adaptive, on `numpy_ref` and the native CPU build (SEP 49). `LowStorageRungeKutta` ([SEP 24](seps/archive/sep-0024-low-storage-interaction-picture-runge-kutta.md)) is RK4 in the midpoint interaction picture in three registers: it replays a `schedule` and nothing else, records the norm 0.0 ("not estimated"), refuses a coupled flow, the TDSE response, interpolated planes and a step past RK4's negative-real interval 2.785, and runs on `numpy_ref` and solprop-native (a resident engine whose step is one native call) | | `interval_cap` | bool | whether a step is capped at the width of the operator-table interval it starts in. `assemble` turns it off on the diagonal-flow routes and on for a coupled flow or a reject-free controller, unless set ([SEP 14](seps/archive/sep-0014-stepper-efficiency.md) §4.1) | | `auto_init` | bool | whether the start-up rule takes the step after the first from the first step's own scaled norm (`PIController.start_factor`, at most 100 times `dz_init`). Set by the convenience interfaces when no `dz_init` is given; off for a coupled flow or a reject-free controller ([SEP 14](seps/archive/sep-0014-stepper-efficiency.md) §4.2) | | `interpolate_planes` | bool | steps across the save planes and observes each inside its accepted step, through DOPRI5's continuous extension or, with an `integrator`, the Adams march's own polynomial (ADR-039); the end of the run and the hard kinks are still landed on. Refused with a schedule, a Runge-Kutta tableau without an extension, the per-step collar, a coupled flow and the `modified` noise | diff --git a/docs/dev/seps/sep-0049-tdse-on-native.md b/docs/dev/seps/sep-0049-tdse-on-native.md index 79d8ddc48..acb2e1c99 100644 --- a/docs/dev/seps/sep-0049-tdse-on-native.md +++ b/docs/dev/seps/sep-0049-tdse-on-native.md @@ -15,7 +15,7 @@ | **Requires** | [SEP 12](archive/sep-0012-tdse-nonlinear-response.md), [SEP 20](archive/sep-0020-hacc-helium-tdse.md) and [SEP 26](archive/sep-0026-edge-free-tdse-source.md) (the TDSE response and its source); [SEP 39](archive/sep-0039-native-adams-march.md) (the native Adams march); [SEP 47](archive/sep-0047-run-ownership.md) (backend-owned sessions); [SEP 25](archive/sep-0025-rounding-level-reproducibility.md) (agreement to the error bound); ADR-077 and the change it calls for (§3.4) | | **Supersedes** | — | | **Superseded-by** | — | -| **Implementation** | In progress: Opus 5.5 (AI), on `feature/sep-0049-tdse-on-native`; phases 0–2 | +| **Implementation** | In progress: Opus 5.5 (AI), on `feature/sep-0049-tdse-on-native`; phases 0–3 | | **Manual pages** | — | ## Part I — Proposal @@ -1135,6 +1135,117 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. `request` raises `StopRequested`, `KeyboardInterrupt` or `RunTerminated` unchanged, which the hook stores and the session re-raises (§6.3). +### Phase 3 — The native hook + +- **Scope.** Phase 3, 2026-11-12, Opus 5.5 (AI), on the branch after the native + restructuring (#770, merged in at `42e3b22b`); one commit (below, with this report). +- **Delivered.** + - C++: `include/solprop_native/host_source.hpp` (new), the generic `HostSource` + with `begin(cost)`, `evaluate(field, z, purpose, source)` and `end(outcome)`, and + `HostSourceError`; nothing in it names TDSE or Pyroraptor (P10). The march + (`adams_march.cpp`) calls `Engine::begin_attempt` after its budget check and + `end_attempt` after each decision -- after the commit for an accepted attempt -- + through an `Attempt` guard that ends an attempt abandoned when it leaves without a + decision (a failure, a floor, a diverged replay); a host failure is + `Category::host`, naming its stage, `none` for `begin`. `ResidentEngine` forwards + both to the bound source and names each evaluation's position and purpose (the + startup at `z`, the predictor at `z + dz`). `GeneralState::bind_host_source` + allocates the two host rows and the source's view once, listed as `host_source` + in the memory ledger; the workspace's stage core then gathers the lab-frame bins + from the fine spectrum, forms the in-stage absorber as every route does, calls the + source from the calling thread outside every parallel region, and the stage finish + (`StageTerms::finish`) takes the source with the collar alone, skipping the + engine's own response pass. Host builds only: a device build throws, and its + adapter refuses the problem first. + - The binding layer (`bindings.cpp`): `PythonHostSource` calls a Python object's + three methods under the interpreter lock the march released, catches + `nb::python_error` there and reports it to the march as `HostSourceError`; the + state holds the object (`State.bind_host_source`), the engine borrows it. + - Python adapter: `solprop_native/_host_source.py` (new), `TDSEHostSource`: `begin` + reserves, `evaluate` is `node_fields`, the binding's evaluation and the shared + assembly and projection with the reference's own scalars (`scalars(problem, z, + z)`, plain helpers) and prefactor (`prefactor_at` or the combined z table), `end` + records the outcome and releases in a `finally`; every exception is kept, the + first as the error and later ones as notes. `NativeBackend` takes `tdse_binding`, + binds the hook for its life and unbinds it on close, refuses a TDSE problem without + its binding (and the reverse) and the statistics the TDSE response does not define, + records `atomic_calls`, `tdse_execution`, `tdse_last_evaluation` and `tdse_source` + (now `tdse_source.source_record`, shared with the reference), lists the binding's + rows as `tdse_*`, and restores an atomic ledger into the binding. The session opens + `_interrupts.interruptible()` beside its commit around `advance` when a source is + bound, and reports a host failure as the kept exception, a reservation the budget + refused rewrapped with the last accepted step's position and field as the reference + does. + - solprop: the registry hands a TDSE binding to either backend; the driver's + `numpy_ref`-only rules for a TDSE run (`_admit_tdse_execution`, + `_check_tdse_execution`) are gone, so `backend="auto"` runs it on native; the + capability rule `tdse-numpy-ref` is removed. + - Tests: `tests/backends/test_tdse_native.py` (13); the registry, entry, defaults + and checkpoint tests that held the old refusals rewritten. + - Documentation: `docs/dev/kernel-contract.md` (§1's `TDSESource`, the source + reservations, §6's host source hook), `docs/dev/backends.md`, `AGENTS.md` and + `native/AGENTS.md` (the hook's exception to "no Python call in a trial step"), + the TDSE manual and *Choosing a backend*, `changelog.d/tdse-on-native.changed.md`. +- **Deviations.** + - The march's `end` takes an outcome, `rejected`, `accepted` or `abandoned`, not a + boolean: an attempt that leaves without a decision must still release its + reservation, which the reference does in a `finally`, and is not recorded, as the + reference records none. Three entry points remain. + - A host failure is the march's `error` reason with a new category, `host`, and its + stage, rather than a reason of its own: the session already books a failed batch + and raises its error, which is now the kept exception. + - A signal taken inside the hook's wait ends the native session as its terminal + error, the batch booked and the state eligible for its checkpoint, where the + reference's session lets the signal through to the run loop; the run loop raises + it either way. + - §7.3 item 1's stage comparison is not made one stage at a time. The reference's + `_adams_lab_source` on a freshly loaded backend is not its run's startup sample + (about 47 against the 1.3e7, mostly the in-stage absorber's term, the run's + startup row holds), and the two engines' startup rows read after one attempt are + 1 % apart while whole runs agree to 1e-13, so what those rows hold is not the same + quantity and is left as Q12. `F` is compared bit for bit, and the finish through + the runs of items 2 to 4. + - Item 4 is a short adaptive run, not eight steps: the case's collar holds its steps + to about a two-hundredth of its length, so 0.2 um is some four hundred attempts; + its budget is the attempts times the one-step 16 eps. + - Item 5 is met through the driver: the TDSE run tests that ask for `auto` + (`test_tdse_hacc_run.py`, `test_tdse_archive_run.py`, `test_tdse_entry.py`, the + gated-derivative and natural-output runs) now run on native, unchanged; those + that name `numpy_ref` keep it. No module was duplicated over both backends. +- **Decisions.** The Runge-Kutta and low-storage paths are not guarded against a + bound host source in C++: the translation admits a TDSE response with the adaptive + `Multistep()` only, and a zero-length Runge-Kutta trial with a source bound does not + give the reference's stage (above), so a later guard should refuse it there. +- **Verification.** Native CPU build (Apple clang, editable, the `estimate` planning), + Apple M5 Max. `tests/backends/test_tdse_native.py`: `F` bit for bit on the gated and + cubic descriptors; the call sequence (every reserve, evaluation purpose, record and + release, in order), the atomic ledger and the `atomic_calls` and `tdse_source` + records equal to the reference's on a whole run in both formulations, the + evaluations' positions within 1e-9 relative; a budget refusal, a provider failure + and a SIGTERM before the commit, and an archive failure after it, each leaving the + stated state with nothing reserved and the reference's calls; a 0.2 um adaptive run + within its budget, with one attempt per call and the production bound (measured + 1.3e-13 against 1.4e-12); a device build and a missing binding refused by name; the + memory rows. The TDSE, worker and interrupt modules of `tests/backends` and + `tests/driver`, the compiled-multistep, resume-ledger, capability, structure and + SEP modules: 1,411 passed, 3 skipped; `native/tests/test_backend.py` and + `test_session_failure.py`: 61 passed. Non-TDSE runs, before (the #770 + build) and after, `SOLPROP_FFTW_PLAN=estimate SOLPROP_CALIBRATION=off`, one thread: + the final fields of `gnlse_soliton` (default), `rdw_one_mode` (both Adams formulations), `argon_blueshift`, `radial_filament_256` and `dense_output`, each over 0.5–2 % of its length, are identical bit for bit. `poe fmt-native`; `poe tidy-native` on the changed C++ files: + no finding. No CUDA or HIP compiler here: the device builds are left to CI's + device-compile lanes. +- **Limitations.** TDSE on a native device build is refused by name (§12, Q2). The + hook is not exercised by a real provider (§7.4, phase 4). +- **Open questions.** Q12: what the two engines' Adams startup rows hold for a TDSE + problem, 1 % apart while their runs agree to rounding, and why the reference's + `_adams_lab_source` outside a run is not its run's sample; until settled, item 1 + compares `F` and the runs compare the finish (owner: phase 4). +- **Cost** (phase 3, `git diff --numstat` from `42e3b22b`): C++ +332 −11, native Python + +251 −12, `src/solprop` +49 −58, tests +464 −30. §6.8 estimated +0.3k C++, +0.4k + native Python and +0.5k tests for the hook. +- **Handoff.** Phase 4: the manual check of §7.4 through the worker on both backends, + the ADR, the remaining pages, Q12, and the size comparison. + ## Part III — History - 2026-11-12: Drafted. @@ -1183,6 +1294,7 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. #767's branch; phases 0 and 1 reported in Part II. - 2026-11-12: Phase 2 (the provider worker) reported in Part II; implementation pull request opened as a draft. +- 2026-11-12: Phase 3 (the native hook) reported in Part II. ### AI review — 2026-11-12 diff --git a/docs/guide/media/tdse.md b/docs/guide/media/tdse.md index 75268f55d..36f0ca8c5 100644 --- a/docs/guide/media/tdse.md +++ b/docs/guide/media/tdse.md @@ -235,8 +235,9 @@ run = prop_capillary( The response is the complete electronic response of the fill, so `plasma`, `raman`, `thg`, `preionisation` and the other ionisation settings must be left unset, or the call -is refused naming them. `backend="auto"` resolves to `"numpy_ref"`, the only backend that -runs a TDSE response; the run needs the adaptive `Multistep()` integrator and a single +is refused naming them. `backend="auto"` resolves to `"native"`, as for every run: the +native CPU build runs the TDSE response, and `backend="numpy_ref"` runs it too, while a +native GPU build refuses it by name; the run needs the adaptive `Multistep()` integrator and a single scalar HE₁₁ mode. A TDSE run carries no shot noise by default: it is expensive, rarely repeated as a noise ensemble, and its checks are coherent. Input shot noise (`noise="input"`) can be asked for, with a warning; its draw is part of the @@ -321,7 +322,7 @@ against an exact solution. - **The optical band** reaches 120 nm (SAE) or 90 nm (haCC) numerically; that is not a claim of validated ultraviolet accuracy. - **Scope.** Helium only, one scalar HE₁₁ mode on the uncoupled modal capillary route, - `numpy_ref` only (the native backend refuses it by name). Double ionisation, ionic + `numpy_ref` and the native CPU build (a native GPU build refuses it by name). Double ionisation, ionic channels beyond $n=2$, ion motion, collisions, recombination and space charge are omitted. - **Cost.** Each source evaluation is a full atomic calculation at every node: minutes diff --git a/docs/guide/performance/backends.md b/docs/guide/performance/backends.md index 2239e4e4b..05fecb6c7 100644 --- a/docs/guide/performance/backends.md +++ b/docs/guide/performance/backends.md @@ -24,7 +24,6 @@ repeat the first: |---|---|---|---|---| | `auto` | `native` | refuses, naming the missing features or the failed check | the load's own error, unchanged | refuses, saying how to install it | | `native` by name | `native` | as above | as above | as above | -| TDSE response, `auto` | `numpy_ref`; native isn't consulted | the same | the same | the same | | `numpy_ref` by name | runs; native isn't consulted | the same | the same | the same | `"auto"` has no fallback. These cases stay distinct, so an import failure is never reported @@ -79,16 +78,10 @@ and what to use for scans are on [Performance tuning](tuning.md). ## Features that run on the reference backend -`numpy_ref` is the numerical reference, and it is also the supported way to run these -features, which native does not run yet. Each is a limit of native, and each stays -supported on the supported platforms: - -| feature | reached by | tracked by | -|---|---|---| -| the [TDSE response](../media/tdse.md) | `auto` or by name | | - -The TDSE row's `auto` is that of the entry points; `SimulationSettings` built by hand with -`tdse_execution` [names the backend](../results/interior-launch.md). Running `numpy_ref` by +`numpy_ref` is the numerical reference, and it is also the supported way to run any +feature native does not run yet. None is listed today: the +[TDSE response](../media/tdse.md), the last, runs on the native CPU build too (a native +GPU build refuses it by name). Running `numpy_ref` by name on an unsupported platform may keep working, without a support promise. The list is not a general waiver: a new feature still lands on both `numpy_ref` and `native` ([SEP 0](../../dev/seps/sep-0000-process.md)), and joins the list only as an accepted diff --git a/native/AGENTS.md b/native/AGENTS.md index b15668949..aa858f4a1 100644 --- a/native/AGENTS.md +++ b/native/AGENTS.md @@ -22,8 +22,8 @@ contract and its conventions; where both apply, follow the stricter rule. C++ API are private. - solprop has two backends, this one and its NumPy reference, and computes in IEEE binary64 only (solprop's ADR-077): every field, stage and statistic here is double - precision, with no single or mixed mode. This backend runs every feature except the - TDSE response, on CPUs and GPUs. + precision, with no single or mixed mode. This backend runs every feature on CPUs and + GPUs, the TDSE response on CPU builds only (solprop SEP 49). - The layers are solprop -> Python backend adapter -> nanobind seam -> a Python-free C++20 core. Native code never imports Python or solprop concepts and never owns user-facing policy: it may execute an algorithm the kernel contract specifies in @@ -128,7 +128,10 @@ the gate. a device kernel's functor as its launch argument; team-level loops stay `Kokkos::`. - Allocate field-sized views, scratch and FFT plans once, when a state is constructed. A trial step performs no allocation, planning, logging, Python call or unplanned - fence. + fence. The one exception is the host source hook (solprop SEP 49, + `include/solprop_native/host_source.hpp`): on a CPU build, a bound host source is + called from the thread that called `advance`, outside every parallel region, and the + binding layer takes the interpreter lock for each call; device builds refuse it. - Keep data resident in the execution space until it is observed; `field_to_host` is the explicit synchronous boundary. Every persistent allocation appears in `memory_ledger()` with its real size in bytes. diff --git a/native/include/solprop_native/adams_march.hpp b/native/include/solprop_native/adams_march.hpp index f7c586de5..c00d7ac0b 100644 --- a/native/include/solprop_native/adams_march.hpp +++ b/native/include/solprop_native/adams_march.hpp @@ -4,6 +4,7 @@ #pragma once +#include "solprop_native/host_source.hpp" #include "solprop_native/watch.hpp" #include @@ -235,6 +236,11 @@ class Engine { /** The last trial's carrier rotations over its `order` newest rows, for a dense observation of the step; empty without a carrier frame. */ virtual std::vector step_carrier(std::size_t /*order*/) { return {}; } + /** An attempt's start, after the budget check and before its first evaluation, and + its end (solprop SEP 49 section 6.2): a host source's reservation and its + outcome. Nothing for an engine without one. */ + virtual void begin_attempt(std::uint64_t /*cost*/) {} + virtual void end_attempt(HostOutcome /*outcome*/) {} /** The committed state's boundary reading (`monitor_reading`), for a check the march takes itself (SEP 45 section 4.8). */ virtual std::vector reading() { @@ -272,7 +278,10 @@ enum class Category : int { convergence = 1, backend = 2, value = 3, - refused = 4 + refused = 4, + /** A host source's failure (solprop SEP 49 section 6.3), its exception kept by the + host; the stage names where, `none` for its `begin`. */ + host = 5 }; /** The host's bounds for one call (SEP 39 section 4.8). */ diff --git a/native/include/solprop_native/core.hpp b/native/include/solprop_native/core.hpp index 4e7c7ba30..35b40eef4 100644 --- a/native/include/solprop_native/core.hpp +++ b/native/include/solprop_native/core.hpp @@ -4,6 +4,7 @@ #pragma once +#include "solprop_native/host_source.hpp" #include "solprop_native/launch.hpp" #include @@ -1160,6 +1161,19 @@ class GeneralState { [[nodiscard]] std::size_t axis_scans() const noexcept; [[nodiscard]] bool closed() const noexcept; void close() noexcept; + /** + * A host source (solprop SEP 49 section 6.2): from here on each stage source is + * `source`'s evaluation of the lab-frame field, the state's own response passes not + * run, and the stage's finish its collar, absorber and propagator alone. Its two + * host rows are allocated here, once, and listed in the memory ledger. Refused on a + * device build and for anything but one real column off the general and separable + * routes; null unbinds. + */ + void bind_host_source(std::shared_ptr source); + /** The bound host source, or null. */ + [[nodiscard]] HostSource* host_source() const noexcept; + /** The next host evaluation's position, m, and purpose, which the engine sets. */ + void host_evaluation(double z, HostPurpose purpose) noexcept; private: /** `function(workspace, arguments...)` inside the workspace's CPU team scope diff --git a/native/include/solprop_native/engines.hpp b/native/include/solprop_native/engines.hpp index 13f23b52e..9be46f68c 100644 --- a/native/include/solprop_native/engines.hpp +++ b/native/include/solprop_native/engines.hpp @@ -11,6 +11,7 @@ #include "solprop_native/stage_tables.hpp" #include +#include #include #include #include @@ -63,6 +64,9 @@ class ResidentEngine final : public march::Engine { double chord_phase(double z1, double z2) override; std::vector step_carrier(std::size_t order) override; std::vector reading() override; + /** A bound host source's attempt (solprop SEP 49): forwarded to it, if any. */ + void begin_attempt(std::uint64_t cost) override; + void end_attempt(HostOutcome outcome) override; private: void inject(march::Stage stage); diff --git a/native/include/solprop_native/host_source.hpp b/native/include/solprop_native/host_source.hpp new file mode 100644 index 000000000..d3212d03f --- /dev/null +++ b/native/include/solprop_native/host_source.hpp @@ -0,0 +1,61 @@ +// SPDX-FileCopyrightText: 2026 John C. Travers and contributors +// +// SPDX-License-Identifier: GPL-3.0-or-later + +#pragma once + +#include +#include +#include +#include + +/* + * A source the host evaluates for the resident Adams march (solprop SEP 49 section + * 6.2): a lab-frame field in, a spectral source out, and an attempt's start and end. + * It knows nothing of what evaluates it; solprop's TDSE response is its first user, + * through the binding layer, which wraps a Python object and takes the interpreter + * lock for each call. The engine calls it from the thread that called `advance`, + * outside every parallel region, and only on a CPU build: a device build refuses a + * problem that needs one before anything is allocated. + * + * Python-free: an exception the host raises reaches the march as a `HostSourceError`, + * the original kept by the binding layer for solprop's session to raise. + */ +namespace solprop_native { + +/** Why an evaluation is made, the march's label for it (solprop's `TDSEPurpose`). */ +enum class HostPurpose : int { startup = 0, predictor = 1 }; + +/** How an attempt ended: rejected or accepted after its decision, or abandoned by a + failure (or a floor) before one, which only releases what it reserved. */ +enum class HostOutcome : int { rejected = 0, accepted = 1, abandoned = 2 }; + +/** A host source's failure: the host's own exception is kept where it was raised. */ +class HostSourceError : public std::runtime_error { +public: + using std::runtime_error::runtime_error; +}; + +/** The three entry points of an attempt (SEP 49 section 6.2). */ +class HostSource { +public: + HostSource() = default; + HostSource(const HostSource&) = delete; + HostSource& operator=(const HostSource&) = delete; + HostSource(HostSource&&) = delete; + HostSource& operator=(HostSource&&) = delete; + virtual ~HostSource() = default; + /** Before an attempt's first evaluation, after the march's budget check: `cost` + evaluations will follow. */ + virtual void begin(std::uint64_t cost) = 0; + /** One evaluation: the lab-frame spectrum `field` (`n_omega` bins of one column) at + `z`, m, into `source`, the spectral source before the spectral collar. */ + virtual void evaluate(std::span> field, double z, + HostPurpose purpose, + std::span> source) = 0; + /** After the decision and, for an accepted attempt, after its commit; also after a + failure, as `abandoned`. Releases what `begin` reserved on every outcome. */ + virtual void end(HostOutcome outcome) = 0; +}; + +} // namespace solprop_native diff --git a/native/src/adams_march.cpp b/native/src/adams_march.cpp index 8a32a61a8..014d8a973 100644 --- a/native/src/adams_march.cpp +++ b/native/src/adams_march.cpp @@ -377,7 +377,9 @@ void fail(Result& result, const Stage stage, const std::exception& error) { result.reason = Reason::error; result.stage = stage; result.message = error.what(); - if (dynamic_cast(&error) != nullptr) + if (dynamic_cast(&error) != nullptr) + result.category = Category::host; + else if (dynamic_cast(&error) != nullptr) result.category = Category::convergence; else if (dynamic_cast(&error) != nullptr) result.category = Category::refused; @@ -387,6 +389,40 @@ void fail(Result& result, const Stage stage, const std::exception& error) { result.category = Category::backend; } +/** One attempt's start and end on the engine (solprop SEP 49 section 6.3): an attempt + that leaves without its decision -- a failure, a floor, a diverged replay -- ends + abandoned, which only releases what it reserved. That release's own failure is the + host's to keep beside the error that left; it never replaces it. */ +class Attempt { +public: + explicit Attempt(Engine& engine) : engine_(engine) {} + Attempt(const Attempt&) = delete; + Attempt& operator=(const Attempt&) = delete; + Attempt(Attempt&&) = delete; + Attempt& operator=(Attempt&&) = delete; + ~Attempt() { + if (!open_) + return; + try { + engine_.end_attempt(HostOutcome::abandoned); + } catch (...) { // NOLINT(bugprone-empty-catch) - kept by the host, see above + } + } + void begin(const std::uint64_t cost) { + engine_.begin_attempt(cost); + open_ = true; + } + /** End with the decision; closed first, so a failure here is not also abandoned. */ + void end(const HostOutcome outcome) { + open_ = false; + engine_.end_attempt(outcome); + } + +private: + Engine& engine_; + bool open_ = false; +}; + } // namespace Result run(const Policy& policy, State& state, Engine& engine, const Bounds& bounds, @@ -465,6 +501,13 @@ Result run(const Policy& policy, State& state, Engine& engine, const Bounds& bou result.cost = cost; return result; } + Attempt attempt(engine); + try { + attempt.begin(cost); + } catch (const std::exception& error) { + fail(result, Stage::none, error); + return result; + } if (state.history.empty()) { // A call that fails has still been charged (R4). ++state.ledger.startup; @@ -547,6 +590,12 @@ Result run(const Policy& policy, State& state, Engine& engine, const Bounds& bou return result; } record(step.end, step.dz, norm, false, trial.order); + try { + attempt.end(HostOutcome::rejected); + } catch (const std::exception& error) { + fail(result, Stage::trial, error); + return result; + } } else { // The new sample takes the lowest row the kept entries do not use. const std::size_t keep = std::min(policy.order, state.history.size()); @@ -589,6 +638,14 @@ Result run(const Policy& policy, State& state, Engine& engine, const Bounds& bou proposal(policy, step.dz, factor(policy, choice.norm, choice.control, policy.max_growth)); record(step.end, step.dz, norm, true, trial.order); + // After the commit (SEP 49 section 6.3): a failure here returns the committed + // attempt with the error. + try { + attempt.end(HostOutcome::accepted); + } catch (const std::exception& error) { + fail(result, Stage::accept, error); + return result; + } if (next < policy.dz_min) { result.reason = Reason::floor; result.floor_proposal = next; diff --git a/native/src/bindings.cpp b/native/src/bindings.cpp index c90967e53..391491b9a 100644 --- a/native/src/bindings.cpp +++ b/native/src/bindings.cpp @@ -10,6 +10,7 @@ #include "solprop_native/dense_output.hpp" #include "solprop_native/engines.hpp" #include "solprop_native/fft_plan.hpp" +#include "solprop_native/host_source.hpp" #include "solprop_native/lagrange_basis.hpp" #include "solprop_native/low_storage.hpp" #include "solprop_native/refuse.hpp" @@ -92,6 +93,49 @@ using RealStorage = std::vector; class BoundState; class BoundSession; +// solprop SEP 49 section 6.2: a host source whose three entry points are a Python +// object's `begin(cost)`, `evaluate(field, z, purpose, out)` and `end(outcome)`. Each +// call takes the interpreter lock the march released; a Python exception is caught +// here, under that lock, and reported to the march as a `HostSourceError`, the +// exception itself kept by the Python object for solprop's session to raise. The +// object is borrowed: its state owns it, so no reference moves without the lock. +class PythonHostSource final : public native::HostSource { +public: + explicit PythonHostSource(const nb::handle owner) : owner_(owner) {} + void begin(const std::uint64_t cost) override { + call([&] { owner_.attr("begin")(cost); }); + } + void evaluate(const std::span> field, const double z, + const native::HostPurpose purpose, + const std::span> source) override { + call([&] { + // Views of the engine's two host rows, valid for the call alone. + const std::array in_shape{field.size()}; + const std::array out_shape{source.size()}; + const ComplexOutput in(const_cast*>(field.data()), 1, + in_shape.data(), nb::handle()); + const ComplexOutput out(source.data(), 1, out_shape.data(), nb::handle()); + owner_.attr("evaluate")( + in, z, purpose == native::HostPurpose::startup ? "startup" : "predictor", + out); + }); + } + void end(const native::HostOutcome outcome) override { + call([&] { owner_.attr("end")(static_cast(outcome)); }); + } + +private: + template void call(const Call& body) { + const nb::gil_scoped_acquire acquire; + try { + body(); + } catch (const nb::python_error& error) { + throw native::HostSourceError(error.what()); + } + } + nb::handle owner_; +}; + template Array exact_array(const nb::handle value) { return nb::cast(value, false); } @@ -2168,6 +2212,19 @@ class BoundState { } // Closes a session that leases this state first (below, after its class). void close() noexcept; + // solprop SEP 49: bind `owner` as the state's host source, or unbind with None. The + // state keeps the object; the engine borrows it. + void bind_host_source(nb::object owner) { + require_unleased(); + if (owner.is_none()) { + if (general_state_) + general_state_->bind_host_source(nullptr); + host_owner_ = nb::object(); + return; + } + open_state().bind_host_source(std::make_shared(owner)); + host_owner_ = std::move(owner); + } // solprop SEP 47: the march a native execution session runs, from the policy, the // schedule and the stage tables this state was configured with, its engine over the @@ -2331,6 +2388,7 @@ class BoundState { bool march_compensated_ = false; bool march_interpolate_ = false; std::optional march_fault_; + nb::object host_owner_; }; // solprop SEP 47: a native execution session over a leased state. It keeps that state @@ -3750,6 +3808,7 @@ NB_MODULE(_core, module_) { "policy"_a, "fused"_a, "compensated"_a, "interpolate"_a, "steps"_a.noconvert(), "edges"_a.noconvert()) .def("adams_march_run", &BoundState::march_run, "state"_a, "bounds"_a) + .def("bind_host_source", &BoundState::bind_host_source, "owner"_a.none()) .def( "rk_march_configure", [](BoundState& self, const nb::dict policy, const std::vector& nodes, diff --git a/native/src/engines.cpp b/native/src/engines.cpp index bdd50ba7f..788e7e5f6 100644 --- a/native/src/engines.cpp +++ b/native/src/engines.cpp @@ -40,8 +40,20 @@ ResidentEngine::ResidentEngine(GeneralState& state, const stage::Tables& tables, : state_(state), tables_(tables), anchor_(anchor), fused_(fused), compensated_(compensated), interpolate_(interpolate), fault_(fault) {} +void ResidentEngine::begin_attempt(const std::uint64_t cost) { + if (auto* host = state_.host_source()) + host->begin(cost); +} + +void ResidentEngine::end_attempt(const HostOutcome outcome) { + if (auto* host = state_.host_source()) + host->end(outcome); +} + void ResidentEngine::start(const double z) { inject(march::Stage::startup); + // A host source evaluates the startup sample at the accepted state (SEP 49). + state_.host_evaluation(z, HostPurpose::startup); const auto row = stage::row(tables_, anchor_, z, z); state_.adams_start(0, stage::trial(tables_, row)); if (compensated_) @@ -56,6 +68,8 @@ void ResidentEngine::trial(const double z, const double dz, for (std::size_t index = 0; index < stencil.n_orders; ++index) highest = std::max(highest, stencil.orders[index]); const std::size_t slots = std::min(std::max(highest, stencil.order), stencil.held); + // The trial's one source evaluation is the predictor's, at the step's end. + state_.host_evaluation(z + dz, HostPurpose::predictor); if (fused_) { // The interaction formulation: the history transported to the anchor, the // source evaluated at the step's end (solprop's `_ResidentAdams.fused_trial`). diff --git a/native/src/general/kernels.hpp b/native/src/general/kernels.hpp index 10e2d8562..9c8510287 100644 --- a/native/src/general/kernels.hpp +++ b/native/src/general/kernels.hpp @@ -1274,6 +1274,9 @@ struct StageTerms { ComplexView2 current, fine, whole, solution, error, stage_phase, stage_inverse, noise; ComplexView2 flow_input, flow_lab, flow_drive, general_scaled, general_source; ComplexView2 projected, absorber; + // A host source's spectral source (solprop SEP 49), `(n_columns, n_omega)`: the + // stage's source in place of the projected response when `hosted`. + ComplexView2 host_spectrum; ComplexView3 stages, phi_terms, prefactor_terms; RealView tableau, phi_delta, phi_at, prefactor_weights, window; IndexView crop, gaps; @@ -1284,6 +1287,7 @@ struct StageTerms { bool add_noise = false, analytic = false, general = false, flow = false; bool carried = false, keep_lab = false, zero_response = false; bool in_stage_absorber = false; + bool hosted = false; // Every coarse bin is the fine bin of its own index (`crop` is the identity). bool identity_crop = false; @@ -1524,15 +1528,22 @@ struct StageTerms { // A zero weight is skipped: a z table's terms are its values at a trial's // positions, one-hot weighted (SEP 29 G1). The sum's value is unchanged; at most // the sign of an exactly zero sum could differ. - Complex prefactor{}; - for (std::size_t term = 0; term < n_prefactor; ++term) { - const double weight = prefactor_weights(stage * n_prefactor + term); - if (weight != 0.0) - prefactor += weight * prefactor_terms(term, column, index); + Complex value{}; + if (hosted) { + // The host's source carries its own factors (solprop SEP 49 section 4): the + // collar's product alone, as the reference's `stage_output` with unit factors. + value = window(index) * host_spectrum(column, index); + } else { + Complex prefactor{}; + for (std::size_t term = 0; term < n_prefactor; ++term) { + const double weight = prefactor_weights(stage * n_prefactor + term); + if (weight != 0.0) + prefactor += weight * prefactor_terms(term, column, index); + } + value = + prefactor * window(index) * + (general ? general_source(column, index) : projected(column, crop(index))); } - Complex value = - prefactor * window(index) * - (general ? general_source(column, index) : projected(column, crop(index))); if (in_stage_absorber && active(index) != 0U) value += absorber(column, crop(index)); value *= stage_inverse(column, index); @@ -1555,7 +1566,7 @@ struct StageTerms { // below: a stage that projects a source into the stage rows, off a coupled flow and // the general route, whose stores the per-bin forms keep. [[nodiscard]] bool blocks_vectorise() const noexcept { - return !flow && !general && !zero_response; + return !flow && !general && !zero_response && !hosted; } // `finish` over the `count` bins of a block from `first`, each bin the same diff --git a/native/src/general/setup.cpp b/native/src/general/setup.cpp index f53b7a601..b8cf2f8f9 100644 --- a/native/src/general/setup.cpp +++ b/native/src/general/setup.cpp @@ -1689,6 +1689,11 @@ std::map GeneralWorkspace::memory_ledger() const { ? general_->dispersive()->memory_ledger() : DispersiveResponse::empty_ledger()); ledger.merge(base_memory_ledger()); + if (host_source_) + // solprop SEP 49: the host's field and source rows and the source's view. + ledger.emplace("host_source", (host_field_.size() + host_out_.size()) * + sizeof(std::complex) + + host_spectrum_.size() * sizeof(Complex)); return ledger; } diff --git a/native/src/general/state.cpp b/native/src/general/state.cpp index e16896fa4..2fc61bfcc 100644 --- a/native/src/general/state.cpp +++ b/native/src/general/state.cpp @@ -456,6 +456,18 @@ std::size_t GeneralState::rk_source_evaluations() const noexcept { std::size_t GeneralState::axis_scans() const noexcept { return workspace_ ? workspace_->axis_scans() : 0; } +void GeneralState::bind_host_source(std::shared_ptr source) { + with_team([&](GeneralWorkspace& workspace) { + workspace.bind_host_source(std::move(source)); + }); +} +HostSource* GeneralState::host_source() const noexcept { + return workspace_ ? workspace_->host_source() : nullptr; +} +void GeneralState::host_evaluation(const double z, const HostPurpose purpose) noexcept { + if (workspace_) + workspace_->host_evaluation(z, purpose); +} bool GeneralState::closed() const noexcept { return !workspace_; } void GeneralState::close() noexcept { workspace_.reset(); } diff --git a/native/src/general/trial.cpp b/native/src/general/trial.cpp index 9c69d1278..d9463720a 100644 --- a/native/src/general/trial.cpp +++ b/native/src/general/trial.cpp @@ -5654,6 +5654,11 @@ StageTerms GeneralWorkspace::stage_terms(const std::size_t stage) const { !problem_.raman.enabled && !absorbing() && !dispersive_; terms.in_stage_absorber = absorbing(); terms.identity_crop = identity_crop_; + // A host source is the stage's whole source (solprop SEP 49 section 6.2). + terms.hosted = static_cast(host_source_); + terms.host_spectrum = host_spectrum_; + if (terms.hosted) + terms.zero_response = false; return terms; } @@ -5843,6 +5848,10 @@ void GeneralWorkspace::evaluate_stage_source(const std::size_t stage, const doub // low-storage march's passes. Returns whether a source was evaluated: a stage without // a response has none to finish. bool GeneralWorkspace::modal_core(const std::size_t stage) { + if (host_source_) { + host_core(); + return true; + } if (general_) { const bool general_active = general_->responds() || problem_.absorber.in_stage; if (general_active) @@ -5969,6 +5978,58 @@ bool GeneralWorkspace::modal_core(const std::size_t stage) { return true; } +void GeneralWorkspace::bind_host_source(std::shared_ptr source) { + if (!source) { + host_source_.reset(); + host_field_ = {}; + host_out_ = {}; + host_spectrum_ = ComplexView2(); + return; + } +#if defined(KOKKOS_ENABLE_CUDA) || defined(KOKKOS_ENABLE_HIP) + throw BackendRefusal("a host source runs on the native CPU build only"); +#else + if (general_ || problem_.separable_geometry() || problem_.n_columns() != 1 || + problem_.dims.family != FieldFamily::field_real) + throw BackendRefusal("a host source takes one real-field column of a modal route"); + const auto n = problem_.dims.n_omega; + host_field_.assign(n, {}); + host_out_.assign(n, {}); + host_spectrum_ = ComplexView2("solprop_native.host_source.spectrum", 1, n); + host_source_ = std::move(source); +#endif +} + +// solprop SEP 49: the stage core of a host source. The lab-frame field is gathered +// from the fine spectrum's coarse bins before the inverse transform destroys them, the +// in-stage absorber is formed from the modal time field as on every route, and the +// host evaluates the source from the thread that called the march, outside every +// parallel region, into the view the finish reads. Host builds only. +void GeneralWorkspace::host_core() { +#if defined(KOKKOS_ENABLE_CUDA) || defined(KOKKOS_ENABLE_HIP) + throw BackendRefusal("a host source runs on the native CPU build only"); +#else + execution_space_.fence("solprop_native.host_source.entry"); + const auto n = problem_.dims.n_omega; + for (std::size_t index = 0; index < n; ++index) { + const Complex value = fine_spectrum_(0, crop_index_(index)); + host_field_[index] = {value.real(), value.imag()}; + } + if (absorbing()) { + graph::execute(*inverse_real_plan_, fine_spectrum_, modal_real_, + KokkosFFT::Normalization::none); + form_absorber(); + graph::execute(*absorber_real_plan_, absorber_real_, absorber_spectrum_, + KokkosFFT::Normalization::none); + execution_space_.fence("solprop_native.host_source.absorber"); + } + host_source_->evaluate(host_field_, host_z_, host_purpose_, host_out_); + for (std::size_t index = 0; index < n; ++index) + host_spectrum_(0, index) = + Complex(host_out_[index].real(), host_out_[index].imag()); +#endif +} + // The general modal route's stage core: the shared lab-frame input, the in-stage // absorber on the modal time field as on every route, and the general source in place // of the legacy node pipeline; the finish applies the shared normalisation and the diff --git a/native/src/general/workspace.hpp b/native/src/general/workspace.hpp index f76c8ed94..93584f725 100644 --- a/native/src/general/workspace.hpp +++ b/native/src/general/workspace.hpp @@ -184,6 +184,12 @@ class GeneralWorkspace { std::vector projected_to_host(bool coarse); std::map dispersive_path() const; std::map memory_ledger() const; + void bind_host_source(std::shared_ptr source); + [[nodiscard]] HostSource* host_source() const noexcept { return host_source_.get(); } + void host_evaluation(const double z, const HostPurpose purpose) noexcept { + host_z_ = z; + host_purpose_ = purpose; + } /** The team a compact problem's FFTW loops take (`CpuTeamScope`): its pools' widest instance, which every threaded kernel and transform of it uses. Zero for any other problem: the runtime's team. */ @@ -319,6 +325,7 @@ class GeneralWorkspace { void evaluate_stage_source(std::size_t stage, double dz, bool accumulate = true); bool modal_core(std::size_t stage); void general_core(std::size_t stage); + void host_core(); StageTerms stage_terms(std::size_t stage) const; std::vector> fine_gap_runs() const; void stage_entry(std::size_t stage, double dz); @@ -737,6 +744,13 @@ class GeneralWorkspace { ComplexView transform_work_; #endif std::unique_ptr slab_forward_, slab_inverse_; + // solprop SEP 49: a host source, the lab-frame field and source rows it reads and + // writes on the host, and the source's view the stage finish reads. + std::shared_ptr host_source_; + std::vector> host_field_, host_out_; + ComplexView2 host_spectrum_; + double host_z_ = 0.0; + HostPurpose host_purpose_ = HostPurpose::startup; ComplexView2 current_, advanced_, solution_, error_, residual_, recycled_, noise_; // solprop SEP 24 phase 3: the low-storage march's registers B and C (the state is // A) and its flows. diff --git a/native/src/solprop_native/_core.pyi b/native/src/solprop_native/_core.pyi index a57486e4a..b1e43ff97 100644 --- a/native/src/solprop_native/_core.pyi +++ b/native/src/solprop_native/_core.pyi @@ -448,6 +448,7 @@ class State: self, state: dict[str, object], bounds: dict[str, object] ) -> dict[str, Any]: ... def adams_march_fault(self, stage: str, after: int, category: str) -> None: ... + def bind_host_source(self, owner: object | None) -> None: ... def rk_march_configure( self, policy: dict[str, object], diff --git a/native/src/solprop_native/_host_source.py b/native/src/solprop_native/_host_source.py new file mode 100644 index 000000000..6fc6f621b --- /dev/null +++ b/native/src/solprop_native/_host_source.py @@ -0,0 +1,135 @@ +# SPDX-FileCopyrightText: 2026 John C. Travers and contributors +# +# SPDX-License-Identifier: GPL-3.0-or-later + +"""The TDSE response as the native Adams march's host source (solprop SEP 49). + +The march calls three entry points through the binding layer, which takes the +interpreter lock for each: ``begin(cost)`` before an attempt, ``evaluate`` at each +source evaluation, and ``end(outcome)`` after its decision. They are the reference's +binding calls at the same points of an attempt -- ``reserve``, the evaluation with +solprop's shared source assembly (``solprop.backends.tdse_source``), ``record_trial`` +and ``release_reservation`` -- and the source each evaluation returns is ``F`` of +SEP 49 section 4, which the engine finishes as it finishes every source. An exception +raised here is kept, and the march reports only that one was; the session raises the +kept exception unchanged, with every later one attached as a note. +""" + +from typing import Any + +import numpy as np +import numpy.typing as npt +from solprop.backends import tdse_source +from solprop.translate.problem import PropagationProblem +from solprop.translate.responses import prefactor_at +from solprop.translate.stage import StageScalars, StageTerms, combine, scalars +from solprop.translate.tdse import TDSEBinding, TDSEPurpose + +__all__ = ["TDSEHostSource"] + +# The march's attempt outcomes (`HostOutcome` in host_source.hpp). +_REJECTED, _ACCEPTED, _ABANDONED = 0, 1, 2 + + +class TDSEHostSource: + """The driver's TDSE binding as the native march's host source. + + - ``binding``: the driver's binding, which the backend never owns. + - ``terms``: the stage seam's fixed arrays, from which the prefactor is combined as + the reference combines it. + """ + + def __init__( + self, binding: TDSEBinding, problem: PropagationProblem, terms: StageTerms + ) -> None: + self._binding = binding + self._problem = problem + self._terms = terms + self.error: BaseException | None = None + # The last evaluation's purpose, position and projection diagnostics, as the + # reference records them (`tdse_last_evaluation`). + self.last: dict[str, object] = { + "purpose": None, + "z": None, + "projection": None, + } + + def begin(self, cost: int) -> None: + """Reserve every atomic call of the attempt's `cost` evaluations.""" + self._kept(lambda: self._binding.reserve(int(cost))) + + def evaluate( + self, + field: npt.NDArray[np.complex128], + z: float, + purpose: TDSEPurpose, + out: npt.NDArray[np.complex128], + ) -> None: + """Evaluate the source of the lab-frame `field` at `z`, m, into `out`. + + Both arrays are the engine's host rows, valid for this call alone. + """ + + def _run() -> None: + problem = self._problem + lab = np.array(field, dtype=np.complex128, copy=True).reshape(1, 1, -1) + bundle = scalars(problem, float(z), float(z)) + fields = tdse_source.node_fields(problem, lab, bundle) + result = self._binding.evaluate( + fields, z=bundle.z1, purpose=purpose, density=bundle.density + ) + polarisation, current = tdse_source.atomic_sources( + fields, result, problem, bundle + ) + source, projection = tdse_source.project_atomic_sources( + polarisation, + current, + result, + problem, + bundle, + self._prefactor(bundle), + edge=tdse_source.atomic_edge_source(result, problem, bundle), + ) + out[:] = source.reshape(-1) + self.last = {"purpose": purpose, "z": bundle.z1, "projection": projection} + + self._kept(_run) + + def end(self, outcome: int) -> None: + """Record a decided attempt's outcome, then release what it reserved. + + An abandoned attempt (a failure or a floor before its decision) is not + recorded, as the reference records none; its reservation is released all the + same. + """ + + def _run() -> None: + try: + if outcome != _ABANDONED: + self._binding.record_trial(accepted=outcome == _ACCEPTED) + finally: + self._binding.release_reservation() + + self._kept(_run) + + def take(self) -> BaseException | None: + """Return the kept exception, once.""" + error, self.error = self.error, None + return error + + def _prefactor(self, bundle: StageScalars) -> Any: + """Return the stage prefactor as the reference forms it (`HostDriver`).""" + if bundle.prefactor is None or self._terms.prefactor is None: + return prefactor_at(self._problem.responses, bundle.z1) + return combine(bundle.prefactor, self._terms.prefactor) + + def _kept(self, call: Any) -> None: + """Run an entry point, keeping its exception: the first, later ones as notes.""" + try: + call() + except BaseException as error: + if self.error is None: + self.error = error + elif error is not self.error: + self.error.add_note(f"then, on the host source's cleanup: {error!r}") + raise diff --git a/native/src/solprop_native/backend.py b/native/src/solprop_native/backend.py index 5628e2e94..6a343a6b7 100644 --- a/native/src/solprop_native/backend.py +++ b/native/src/solprop_native/backend.py @@ -23,6 +23,7 @@ from solprop import transverse_monitor from solprop._helpers import Helpers from solprop._process import acquire_host_blas, thread_budget +from solprop.backends import tdse_source from solprop.backends.exponential import ( CarrierReference, carrier_reference, @@ -120,6 +121,7 @@ ModalRoute, SpatialRoute, flux_of, + tdse_descriptor, transverse_of, ) from solprop.translate.run_request import RunRequest @@ -130,6 +132,8 @@ require_device_ready, terms, ) +from solprop.translate.tdse import AtomicCallLedger, TDSEBinding +from solprop.translate.tdse_source import TDSE_UNAVAILABLE_STATISTICS from solprop.transverse_monitor import ( TransverseEdge, _EdgeReading, # pyright: ignore[reportPrivateUsage] - released together @@ -153,6 +157,7 @@ ) from ._execution import ExecutionRecord from ._helpers import NATIVE +from ._host_source import TDSEHostSource from ._load import Loaded, StageCache, collar_rests, load, propagator_mode from ._probe import NativeProbe from ._statistics import ( @@ -314,6 +319,7 @@ def __init__( fft_planning: Mapping[str, object] | None = None, far_field: FarFieldTables | None = None, options: ExecutionOptions | None = None, + tdse_binding: TDSEBinding | None = None, ) -> None: check_build() # The private gated benchmark is captured where the backend is built, as the @@ -321,6 +327,7 @@ def __init__( self._gated_plasma = _BENCHMARK_GATED.get() stepper = request.stepper _refuse_unsupported(problem, stepper) + _refuse_tdse(problem, request, tdse_binding) if self._gated_plasma: _refuse_gated_route(problem) validate_run(problem, request) @@ -441,6 +448,13 @@ def __init__( self._native_state.cpu_execution ) self._native_state.set_crossing_workers(thread_budget().threads) + # solprop SEP 49: the TDSE response is the march's host source, bound here for + # the backend's life; the driver owns the binding and closes it. + self._tdse_binding = tdse_binding + self._host_source: TDSEHostSource | None = None + if tdse_binding is not None: + self._host_source = TDSEHostSource(tdse_binding, problem, self._terms) + self._native_state.bind_host_source(self._host_source) if fft_planning is None: _keep_plans(self._native_state, wisdom) self._wisdom_digest = ( @@ -580,8 +594,13 @@ def _restore(self, start: Start, loaded: Loaded) -> tuple[str | None, float]: if start.ledgers is not None: atomic = start.ledgers.get("atomic", {}) if atomic: - raise BackendError( - f"{self.name} does not restore atomic response call ledgers" + if self._tdse_binding is None: + raise BackendError( + f"{self.name} does not restore atomic response call ledgers " + "without a TDSE binding" + ) + self._tdse_binding.restore_ledger( + AtomicCallLedger.from_record(cast("Mapping[str, object]", atomic)) ) if adams is not None: adams.restore_ledger( @@ -1407,6 +1426,12 @@ def metadata(self) -> dict[str, object]: work["node_chunk_rule"] = ( "device" if _on_device() and self.chunk_plan.automatic else "host" ) + if self._tdse_binding is not None and self._host_source is not None: + # As the reference records them (solprop SEP 49 section 5.1). + work["atomic_calls"] = dataclasses.asdict(self._tdse_binding.ledger) + work["tdse_execution"] = dict(self._tdse_binding.metadata()) + work["tdse_last_evaluation"] = dict(self._host_source.last) + work["tdse_source"] = tdse_source.source_record(self._problem) return { **work, **facts.info, @@ -1500,6 +1525,11 @@ def memory_ledger(self) -> dict[str, int]: # ones, so that an estimate made before the run counts them. field = int(ledger.get("state", 0)) ledger["rk_dense"] = 6 * field + field // 16 + if self._tdse_binding is not None: + ledger |= { + f"tdse_{name}": int(size) + for name, size in self._tdse_binding.memory_ledger().items() + } return ledger def prepare( @@ -1561,6 +1591,8 @@ def prepare_group( def close(self) -> None: """Release native plans and views; repeated calls have no effect.""" if not self._closed: + if self._host_source is not None: + self._native_state.bind_host_source(None) self._native_state.close() _host_blas.release() self._closed = True @@ -2872,6 +2904,28 @@ def values(self, kind: Literal["phi", "prefactor"], z: float) -> ComplexArray: return self._native.table_terms_to_host(kind, slot) +def _refuse_tdse( + problem: PropagationProblem, request: RunRequest, binding: TDSEBinding | None +) -> None: + """Refuse a TDSE problem without its binding, or a binding without one. + + And the legacy statistics the TDSE response does not define, by name, as the + reference refuses them, before anything is allocated. + """ + if (tdse_descriptor(problem.route) is None) != (binding is None): + raise BackendError("TDSE problems require an explicit host response binding") + if binding is None: + return + stats = request.observation.stats + unavailable = TDSE_UNAVAILABLE_STATISTICS.intersection( + (*stats.names, *stats.periodic) + ) + if unavailable: + raise BackendError( + f"TDSE does not define legacy statistics {sorted(unavailable)}" + ) + + def _refuse_unknown_method(stepper: StepperSettings) -> None: """Refuse a method object the native marches do not step, by name.""" integrator = stepper.integrator @@ -2946,11 +3000,11 @@ def _refuse_unsupported(problem: PropagationProblem, stepper: StepperSettings) - # Read as any object: a variant this build does not know is refused by name too. route = cast("object", problem.route) match route: - case ModalRoute(tdse=tdse) if tdse is not None: + case ModalRoute(tdse=tdse) if tdse is not None and _on_device(): + space = str(_build_facts().info["device_backend"]).upper() raise BackendError( - "solprop-native does not execute TDSEGasResponse (solprop SEP 12); " - "use backend='numpy_ref' with the independent solprop-tdse atomic " - "engine" + "the TDSE response runs on the native CPU build; this build is " + f"{space} (solprop SEP 49)" ) case ( ModalRoute() diff --git a/native/src/solprop_native/session.py b/native/src/solprop_native/session.py index 2055fdbb1..d80bfb474 100644 --- a/native/src/solprop_native/session.py +++ b/native/src/solprop_native/session.py @@ -62,13 +62,16 @@ native monitor and the draw's readings by the native transverse edge, and the noise phase at each plane, formed on the host or, for a z table's propagator phase, read from its slot in the leased state; -- no group (:mod:`solprop_native.group` runs one native session per member); and no - TDSE response, which the backend refuses by name where it is built, before it - allocates (SEP 47 section 6.3). +- no group (:mod:`solprop_native.group` runs one native session per member); +- the TDSE response on a CPU build, through the march's host source (solprop SEP 49): + each ``advance`` opens an interruptible scope beside its commit, so a signal held + while the host source waits is raised there, and a host source's failure is the + exception it kept. """ from __future__ import annotations +import contextlib import dataclasses import itertools import logging @@ -78,7 +81,7 @@ import numpy as np import numpy.typing as npt -from solprop._interrupts import committing +from solprop._interrupts import committing, interruptible from solprop._ownership import hold from solprop.backends.events import ( Eligibility, @@ -113,6 +116,7 @@ BackendError, ConvergenceError, GridError, + ResponseBudgetError, StopRequested, ) from solprop.monitor import ( @@ -158,6 +162,8 @@ ) _LOG = logging.getLogger(__name__) +# A march failure a host source raised (`march::Category::host`, solprop SEP 49). +_HOST = 5 if TYPE_CHECKING: from types import TracebackType @@ -236,7 +242,8 @@ def refusal( Decided from the problem, the request and the start alone, before anything is allocated: the statistics the session does not evaluate. Everything else native - does not run -- a TDSE response, a coupled flow beside the interaction formulation, + does not run -- a TDSE response on a device build, a coupled flow beside the + interaction formulation, stage rows the native tables do not read -- the backend refuses by name where it is built (``NativeBackend``), still before any native state (SEP 47 section 6.3). @@ -858,8 +865,10 @@ def advance( # the commit would otherwise drop beside it. failure: BaseException | None = None translated: tuple[BackendHistory, Event] | None = None + # A host source's waits inside the call take a held signal (SEP 49, 6.7). + hosted = self._host_source() is not None try: - with committing(): + with committing(), interruptible() if hosted else contextlib.nullcontext(): try: batch = self._native.advance(limit, float(deadline)) except _core.SessionLifecycleError as error: @@ -1765,6 +1774,38 @@ def _budget_error(self, event: _core.SessionEvent) -> BaseException: field, ) + def _host_source(self) -> Any: + """Return the backend's TDSE host source, or None (solprop SEP 49).""" + if self._backend is None: + return None + return self._driver(self._backend)._host_source + + def _host_error( + self, kept: BaseException, event: _core.SessionEvent + ) -> BaseException: + """Return a host source's kept exception as the reference raises it. + + A reservation the atomic budget refused names the last accepted step and + carries its field, as `NumpyRefBackend._reserve_source_calls` does; any other + exception is returned unchanged. + """ + if not isinstance(kept, ResponseBudgetError): + return kept + field = np.empty(self._shape, dtype=np.complex128) + self._native.export(self._sequence, event.generation, False, False, None, field) + z = float(event.z) + error = ResponseBudgetError( + "the TDSE atomic-call budget cannot reserve the next trial " + f"at z = {z:.6g} m ({kept.calls} charged); " + "the run stops at its last accepted step, unfinished", + z=z, + field=field, + calls=kept.calls, + ) + error.__cause__ = None + error.__suppress_context__ = True + return error + def _abandoned_step(self, event: _core.SessionEvent) -> float: """Return the length of the step a poisoned terminal event left in flight. @@ -1836,6 +1877,14 @@ def _error(self, event: _core.SessionEvent, outcome: Outcome) -> BaseException: f"the replayed schedule ends at {end:.6g} m and cannot reach " f"z = {event.target:.6g} m" ) + # A host source's failure (SEP 49 section 6.3): the exception it kept, + # unchanged; a refused reservation as the reference reports it, from the last + # accepted step. + host = self._host_source() + if int(event.category) == _HOST and host is not None: + kept = host.take() + if kept is not None: + return self._host_error(kept, event) # A statistics evaluation the native plan refused: the reference's exception. failure = BackendError(event.message) refused = translated(failure) diff --git a/native/tests/test_backend.py b/native/tests/test_backend.py index 950c6a9e3..dfe6b809a 100644 --- a/native/tests/test_backend.py +++ b/native/tests/test_backend.py @@ -398,14 +398,22 @@ def test_tdse_is_refused_by_name_before_integrator_or_native_setup( source_filter: str, dense_output: bool, ) -> None: - """Refuse TDSE before native setup or either supported march is selected.""" + """Refuse TDSE without its binding before native setup or a march is selected. + + The CPU build runs the TDSE response through its host source (solprop SEP 49), + which needs the driver's binding; a problem without one is refused by name first. + """ problem, run_request, grid = phase_a_case # This deliberately bypasses scientific validation to isolate the adapter's # earliest refusal. Neither a valid atomic basis nor an optional provider is - # needed to reject a route that this backend never executes. + # needed to refuse a route that comes without its binding. requested = copy.copy(problem) route = copy.copy(problem.route) - object.__setattr__(route, "tdse", SimpleNamespace(source_filter=source_filter)) + object.__setattr__( + route, + "tdse", + SimpleNamespace(descriptor=SimpleNamespace(), source_filter=source_filter), + ) object.__setattr__(requested, "route", route) requested_request = run_request.with_stepper( dataclasses.replace( @@ -420,7 +428,7 @@ def forbidden(*_arguments: object, **_keywords: object) -> NoReturn: monkeypatch.setattr(solprop_native.backend, "require_device_ready", forbidden) monkeypatch.setattr(_core, "State", forbidden) - with pytest.raises(BackendError, match=r"TDSEGasResponse.*SEP 12"): + with pytest.raises(BackendError, match="explicit host response binding"): NativeBackend(requested, requested_request, grid=grid) diff --git a/src/solprop/backends/numpy_ref/backend.py b/src/solprop/backends/numpy_ref/backend.py index 3847294a8..706ad7471 100644 --- a/src/solprop/backends/numpy_ref/backend.py +++ b/src/solprop/backends/numpy_ref/backend.py @@ -122,9 +122,7 @@ from solprop.translate.stage import StageScalars, combine from solprop.translate.tdse import AtomicCallLedger, TDSEBinding, TDSEPurpose from solprop.translate.tdse_source import ( - STEADY_CURRENT_CERTIFICATE_LIMIT, TDSE_UNAVAILABLE_STATISTICS, - steady_current_certificate, ) __all__ = ["BlockResidual", "NumpyRefBackend", "SourceTables"] @@ -930,7 +928,7 @@ def metadata(self) -> dict[str, object]: "z": self._tdse_last_z, "projection": self._tdse_last_projection, } - record["tdse_source"] = _source_record(self._problem) + record["tdse_source"] = tdse_source.source_record(self._problem) return record def restore_call_ledgers( @@ -2947,29 +2945,6 @@ def _coupled_flow_maps(problem: PropagationProblem) -> _CoupledFlow | None: return held -def _source_record(problem: PropagationProblem) -> dict[str, object]: - """Record a TDSE run's source filter and, for the gated one, its certificate. - - The steady-current certificate (SEP 26 section 4.4) is refused above its limit at - translation. It is run metadata, never part of the descriptor or any identity. - - - Returns: ``gated``, and for a gated source the certificate, its limit and the - taper breakpoints in s. - """ - descriptor = tdse_descriptor(problem.route) - assert descriptor is not None - record: dict[str, object] = {"gated": descriptor.gated} - if descriptor.gated: - record |= { - "steady_current_certificate": steady_current_certificate( - descriptor, problem.grids.omega, problem.grids.active_mask - ), - "certificate_limit": STEADY_CURRENT_CERTIFICATE_LIMIT, - "gate_breakpoints": list(descriptor.source_breakpoints()), - } - return record - - def _harmonic_carrier( problem: PropagationProblem, ) -> Complex[np.ndarray, " n_to"] | None: diff --git a/src/solprop/backends/registry.py b/src/solprop/backends/registry.py index a02f53db6..be73404d6 100644 --- a/src/solprop/backends/registry.py +++ b/src/solprop/backends/registry.py @@ -36,7 +36,6 @@ from solprop.profiles import ZProfile from solprop.spatial import FAR_FIELD_STATISTICS from solprop.translate.problem import PropagationProblem -from solprop.translate.routes import tdse_descriptor from solprop.translate.run_request import RunRequest from solprop.translate.tdse import TDSEBinding @@ -354,7 +353,8 @@ def select( profiles : collections.abc.Mapping, optional The z-profiles keyed by statistic name, for the profile statistics. tdse_binding : solprop.translate.tdse.TDSEBinding or None, optional - Driver-owned host response binding for a TDSE problem on NumPy. + Driver-owned host response binding for a TDSE problem, on either backend + (SEP 49). options : solprop.backends.session.ExecutionOptions or None, optional The run's execution options, among them an explicit node budget, from which the backend plans its node chunk (SEP 47 section 6.5); ``None`` for the @@ -389,10 +389,6 @@ def select( ensure() _check_required_device(name) - if ( - tdse_descriptor(problem.route) is not None or tdse_binding is not None - ) and name != "numpy_ref": - raise BackendError(f"{name} does not execute TDSE responses; use numpy_ref") refusal = statistics_refusal(name, request.observation.stats.names) if refusal is not None: raise BackendError(refusal) @@ -468,8 +464,8 @@ def _construct( """Import and build the named backend, whose package is installed. Imported here, not at module scope: selecting a backend must not be a reason to - import the other or initialize a native runtime. Only the reference is given a TDSE - binding; :func:`select` refuses one elsewhere. + import the other or initialize a native runtime. A TDSE binding goes to either + backend, which refuses a problem without its binding by name. """ factory: Callable[..., Backend] if name == "native": diff --git a/src/solprop/backends/tdse_source.py b/src/solprop/backends/tdse_source.py index c860b08c8..075db7a45 100644 --- a/src/solprop/backends/tdse_source.py +++ b/src/solprop/backends/tdse_source.py @@ -36,7 +36,10 @@ from solprop.translate.routes import tdse_descriptor, tdse_of from solprop.translate.stage import StageScalars from solprop.translate.tdse import TDSEResult, lattice_times, waveform_identity -from solprop.translate.tdse_source import steady_current_certificate +from solprop.translate.tdse_source import ( + STEADY_CURRENT_CERTIFICATE_LIMIT, + steady_current_certificate, +) __all__ = [ "FFT_ERROR_CONSTANT", @@ -44,6 +47,7 @@ "atomic_sources", "node_fields", "project_atomic_sources", + "source_record", ] FFT_ERROR_CONSTANT = 8.0 @@ -935,3 +939,35 @@ def _current_operands( factor=source.current_numerator / power_weight_at(problem, bundle.z1), inverse=_inverse_i_omega(problem) if gated else None, ) + + +def source_record(problem: PropagationProblem) -> dict[str, object]: + """Record a TDSE run's source filter and, for the gated one, its certificate. + + The steady-current certificate (SEP 26 section 4.4) is refused above its limit at + translation. It is run metadata, never part of the descriptor or any identity; + both backends record it as ``tdse_source``. + + Parameters + ---------- + problem : PropagationProblem + A TDSE problem. + + Returns + ------- + dict + ``gated``, and for a gated source the certificate, its limit and the taper + breakpoints in s. + """ + descriptor = tdse_descriptor(problem.route) + assert descriptor is not None + record: dict[str, object] = {"gated": descriptor.gated} + if descriptor.gated: + record |= { + "steady_current_certificate": steady_current_certificate( + descriptor, problem.grids.omega, problem.grids.active_mask + ), + "certificate_limit": STEADY_CURRENT_CERTIFICATE_LIMIT, + "gate_breakpoints": list(descriptor.source_breakpoints()), + } + return record diff --git a/src/solprop/capabilities.py b/src/solprop/capabilities.py index d515d7256..e64b80fe9 100644 --- a/src/solprop/capabilities.py +++ b/src/solprop/capabilities.py @@ -597,12 +597,6 @@ class Rule: "tdse_execution is the resources of a TDSE response, which tdse declares.", _CAPILLARY, ), - Rule( - "tdse-numpy-ref", - ("tdse_execution", "backend"), - "The TDSE response runs on numpy_ref alone, named explicitly.", - _CAPILLARY, - ), Rule( "tdse-multistep", ("tdse_execution", "integrator"), @@ -683,7 +677,7 @@ def route(name: str) -> RouteCapabilities: >>> capillary.choices["shape"], capillary.choices["noise"] (('gauss', 'sech'), ('default', 'input', 'modified', 'none')) >>> [rule.id for rule in capillary.rules_for("tdse_execution")][:2] - ['tdse-execution-needs-tdse', 'tdse-numpy-ref'] + ['tdse-execution-needs-tdse', 'tdse-multistep'] """ # The recipe layer sits above this module, which the driver imports. from solprop.recipe.core import ROUTES # noqa: PLC0415 - layer above diff --git a/src/solprop/driver/simulation.py b/src/solprop/driver/simulation.py index 752ebbd4d..37447b001 100644 --- a/src/solprop/driver/simulation.py +++ b/src/solprop/driver/simulation.py @@ -1059,14 +1059,6 @@ def _admit_tdse_execution(settings: SimulationSettings) -> None: f"{type(settings.tdse_execution).__name__}" ) raise DriverError(msg, parameter="tdse_execution") - if settings.backend != "numpy_ref": - msg = ( - f"tdse_execution requires an explicit backend='numpy_ref', the one backend " - f"that runs the TDSE response; got backend={settings.backend!r}" - ) - raise DriverError( - msg, parameter=("tdse_execution", "backend"), rule="tdse-numpy-ref" - ) if not isinstance(settings.integrator, Multistep): msg = ( "tdse_execution requires integrator=Multistep(), the Adams method the TDSE " @@ -7624,13 +7616,11 @@ def _check_tdse_execution(simulation: Simulation, settings: SimulationSettings) if settings.tdse_execution is not None: raise DriverError("tdse_execution requires a TDSEGasResponse") return - if ( - settings.tdse_execution is None - or settings.backend != "numpy_ref" - or not isinstance(settings.integrator, Multistep) + if settings.tdse_execution is None or not isinstance( + settings.integrator, Multistep ): raise DriverError( - "TDSE requires explicit NumPy, Multistep and tdse_execution settings" + "TDSE requires explicit Multistep and tdse_execution settings" ) if simulation.noise is not None and simulation.noise.model == "modified": raise DriverError("TDSE refuses noise='modified'; use 'input' or None") @@ -9308,9 +9298,9 @@ def _tdse_entry( tdse, atomic_window=dataclasses.replace(window, duration=trange) ) duration_source = "trange" - resolved_backend = "numpy_ref" if backend == "auto" else backend - if backend == "auto": - logger.info("backend='auto' resolves to numpy_ref for a TDSE response") + # Both backends run the TDSE response; "auto" is native, as for every run + # (SEP 49). + resolved_backend = "native" if backend == "auto" else backend final_only = save_planes is None and n_save is None resolved_steps = 1 if step_planes is None else step_planes record: dict[str, JsonValue] = { diff --git a/tests/backends/test_tdse_march.py b/tests/backends/test_tdse_march.py index fcd3e7ab3..450060fe4 100644 --- a/tests/backends/test_tdse_march.py +++ b/tests/backends/test_tdse_march.py @@ -289,27 +289,26 @@ def test_a_failed_release_is_a_note_on_the_trial_s_own_error(march_problem): backend.close() -@pytest.mark.parametrize("backend_name", ["native"]) -def test_registry_refuses_tdse_before_constructing_a_backend( - march_problem, monkeypatch, backend_name -): - """An unsupported explicit backend never silently redirects the atomic problem.""" - problem, run_request, grid = march_problem - - def forbidden(*args, **kwargs): - raise AssertionError("no backend is constructed for an unsupported TDSE route") - - monkeypatch.setattr(registry, "_construct", forbidden) - with pytest.raises(BackendError, match=f"{backend_name} does not execute TDSE"): - registry.select(backend_name, problem, run_request, grid=grid) - - -def test_direct_native_construction_refuses_tdse_before_allocating(march_problem): - """The direct constructor enforces the same named refusal as the registry.""" - native = pytest.importorskip("solprop_native") +def test_registry_hands_a_tdse_binding_to_either_backend(march_problem, monkeypatch): + """Both backends run the TDSE response (SEP 49); the binding goes to the factory.""" problem, run_request, grid = march_problem - with pytest.raises(BackendError, match="native does not execute TDSE"): - native.NativeBackend(problem, run_request, grid=grid) + constructed = [] + + class ConstructedError(Exception): + pass + + def construct(name, *args, **kwargs): + constructed.append((name, kwargs.get("tdse_binding"))) + raise ConstructedError + + monkeypatch.setattr(registry, "_construct", construct) + monkeypatch.setattr(registry, "_native_installed", lambda: True) + monkeypatch.setattr("solprop.calibration.first_use.ensure", lambda: None) + binding = ManufacturedBinding(tdse_descriptor(problem.route), 10) + for name in ("numpy_ref", "native"): + with pytest.raises(ConstructedError): + registry.select(name, problem, run_request, grid=grid, tdse_binding=binding) + assert constructed == [("numpy_ref", binding), ("native", binding)] def test_tdse_problem_requires_a_binding_even_at_the_direct_backend(march_problem): diff --git a/tests/backends/test_tdse_native.py b/tests/backends/test_tdse_native.py new file mode 100644 index 000000000..61a8e99b3 --- /dev/null +++ b/tests/backends/test_tdse_native.py @@ -0,0 +1,432 @@ +# SPDX-FileCopyrightText: 2026 John C. Travers and contributors +# +# SPDX-License-Identifier: GPL-3.0-or-later + +"""The TDSE response on native against the reference (SEP 49 section 7.3). + +Manufactured bindings replace the driver's binding, not the provider, and need no +worker: ``edge_source_cases.ManufacturedBinding`` returns a manufactured remainder +beside a large linear shadow. Native evaluates the source through its host source +hook, the reference through its own evaluation; both call the shared assembly +(``solprop.backends.tdse_source``), so ``F`` agrees bit for bit for the same field and +position, and what each backend does after it is compared within its budget. +""" + +import dataclasses +import signal +from typing import Any + +import numpy as np +import pytest + +from bounds import EPS, assert_within +from edge_source_cases import ManufacturedBinding +from sessions.executors import IDENTITY, Started, drive +from solprop.backends import tdse_source +from solprop.backends.numpy_ref import NumpyRefBackend +from solprop.backends.session import Start +from solprop.backends.sessions import start_session +from solprop.driver import Simulation +from solprop.exceptions import ( + BackendError, + ResponseBudgetError, + RunTerminated, + TDSEArchiveError, +) +from solprop.nonlinear.tdse import ( + TDSEBasis, + TDSECalibration, + TDSEExecution, + TDSEGasResponse, + TDSESampling, +) +from solprop.stats import StatsSpec +from solprop.stepper.multistep import Multistep +from solprop.translate.routes import tdse_descriptor +from solprop.translate.tdse import TDSEResult, lattice_times, waveform_identity + +native = pytest.importorskip("solprop_native") + +pytestmark = pytest.mark.medium + +LENGTH = 1e-5 + + +def _law(times): + """A manufactured remainder ``S(t)``, A m per atom.""" + return 1e-27 * np.exp(-((times / 8e-15) ** 2)) * np.sin(2.4e15 * times) + + +def _case( + formulation: Any = "interaction", + kerr_model: Any = "tdse_native", + *, + length=LENGTH, + **changes, +): + """A short helium capillary with a manufactured TDSE response.""" + dz_max = changes.pop("dz_max", 1e-6) + parent = Simulation.from_capillary( + 256e-6, + length, + "He", + 0.1, + wavelength=800e-9, + wavelength_limits=(400e-9, 1.6e-6), + trange=60e-15, + fwhm=2e-15, + energy=1e-12, + modes=1, + n_r=4, + n_save=2, + noise=None, + plasma=False, + backend="numpy_ref", + integrator=Multistep(formulation=formulation), + dz_init=min(1e-6, dz_max), + dz_max=dz_max, + ) + basis = TDSEBasis( + "manufactured", (0.0, 1e-9), (8,), 5, 1e-16, source="native fixture" + ) + calibration = TDSECalibration( + "native-fixture", + "research", + basis.identity, + (400e-9, 2e-6), + source="native fixture", + limitations=("no atomic calculation",), + ) + responses = [ + TDSEGasResponse( + "helium", + basis, + 1e-19, + kerr_model, + 1.0 if kerr_model == "tdse_native" else 2.0, + calibration, + TDSESampling(), + ) + ] + simulation = Simulation( + parent.grid, + parent.modes, + responses, + parent._field0, + settings=dataclasses.replace( + parent.settings, + tdse_execution=TDSEExecution(), + stats=StatsSpec(names=("energy",)), + **changes, + ), + noise=None, + profiles=parent.profiles, + frame=parent.frame, + ) + return simulation.translate(), simulation._run_request(), simulation.grid + + +class Unbudgeted(ManufacturedBinding): + """The manufactured histories with no budget: a lone evaluation's, any purpose. + + A cubic replacement's third-order current rides beside them. + """ + + def evaluate(self, fields, *, z, purpose, density=None): + del z, purpose, density + first = self.linear * fields + times = lattice_times(self.descriptor.samples, self.descriptor.spacing) + remainder = np.broadcast_to(_law(times), fields.shape) + cubic = self.descriptor.kerr_model != "tdse_native" + third = 2.0**-120 * fields**3 if cubic else None + return TDSEResult( + first + remainder + (0.0 if third is None else third), + first, + input_identity=waveform_identity(fields), + configuration_identity=self.descriptor.identity, + valid=np.ones(fields.shape, dtype=bool), + shadow_orders=(1, 3) if cubic else (1,), + third_order_current=third, + ) + + +class Recording(ManufacturedBinding): + """Record every binding call, with a failure or a hang injected on request.""" + + def __init__(self, descriptor, limit, *, fail_evaluation=None, fail_record=None): + super().__init__(descriptor, limit, _law) + self.calls = [] + self.fail_evaluation = fail_evaluation + self.fail_record = fail_record + + def reserve(self, logical_calls): + self.calls.append(("reserve", logical_calls)) + super().reserve(logical_calls) + + def evaluate(self, fields, *, z, purpose, density=None): + self.calls.append(("evaluate", purpose, z)) + if ( + self.fail_evaluation is not None + and len(self.purposes) == self.fail_evaluation + ): + self.purposes.append(purpose) + raise self.failure() + return super().evaluate(fields, z=z, purpose=purpose, density=density) + + def failure(self) -> BaseException: + return BackendError("manufactured provider failure") + + def record_trial(self, *, accepted): + self.calls.append(("record_trial", accepted)) + trials = sum(1 for call in self.calls if call[0] == "record_trial") + if self.fail_record is not None and trials == self.fail_record: + raise TDSEArchiveError("manufactured archive failure") + + def release_reservation(self): + self.calls.append(("release",)) + super().release_reservation() + + +class Interrupted(Recording): + """Raise SIGTERM's exception from an evaluation, as the worker's wait raises a + signal it took (a held one, on native).""" + + def failure(self) -> BaseException: + return RunTerminated(signal.SIGTERM) + + +def _run(backend, problem, request, *, work_limit=256) -> Any: + """Drive a session of `backend` to its end; return the run and its record. + + A signal's exception that the session raises rather than reports, as a run loop + receives it, is returned in place of the run. + """ + session = start_session( + backend, problem, request, Start.entrance(problem), owns=False + ) + try: + run = drive( + Started(session), export=IDENTITY, work_limit=work_limit, raises=False + ) + record = dict(session.execution_record()) + except RunTerminated as signalled: + return signalled, {"z": session.status().z} + finally: + session.close() + return run, record + + +def _pair(case, binding_class=Recording, *, limit=1000, **options): + """Run the case on both backends with a fresh binding each.""" + problem, request, grid = case + descriptor = tdse_descriptor(problem.route) + runs = {} + for name, construct in ( + ("numpy_ref", NumpyRefBackend), + ("native", native.NativeBackend), + ): + bound = binding_class(descriptor, limit, **options) + backend = construct(problem, request, grid=grid, tdse_binding=bound) + try: + run, record = _run(backend, problem, request) + metadata = backend.metadata() + finally: + backend.close() + runs[name] = (run, record, bound, metadata) + return runs + + +def _reference_source(problem, request, grid, binding, field, z): + """``F`` as the reference forms it at `field` and `z`, its scalars and prefactor.""" + reference = NumpyRefBackend(problem, request, grid=grid, tdse_binding=binding) + try: + reference.set_initial(field, None) + bundle = reference._stage_scalars(z, z) + fields = tdse_source.node_fields(problem, field, bundle) + result = binding.evaluate( + fields, z=z, purpose="startup", density=bundle.density + ) + polarisation, current = tdse_source.atomic_sources( + fields, result, problem, bundle + ) + source, _ = tdse_source.project_atomic_sources( + polarisation, + current, + result, + problem, + bundle, + reference._prefactor(bundle), + edge=tdse_source.atomic_edge_source(result, problem, bundle), + ) + finally: + reference.close() + return source + + +@pytest.mark.parametrize("kerr_model", ["tdse_native", "optmat_cubic"]) +def test_one_evaluation_is_the_reference_s_f_and_stage(kerr_model): + """Item 1: ``F`` bit for bit; the stage output within the one-stage budget.""" + problem, request, grid = _case(kerr_model=kerr_model) + descriptor = tdse_descriptor(problem.route) + field = np.array(problem.initial.field0) + reference_binding = Unbudgeted(descriptor, 10, _law) + expected = _reference_source(problem, request, grid, reference_binding, field, 0.0) + binding = Unbudgeted(descriptor, 10, _law) + backend = native.NativeBackend(problem, request, grid=grid, tdse_binding=binding) + try: + hook = backend._host_source + out = np.empty(problem.grids.n_omega, dtype=np.complex128) + hook.evaluate(field.reshape(-1), 0.0, "startup", out) + assert hook.last["purpose"] == "startup" + finally: + backend.close() + # The same shared code on the same field and position: an exact identity. What + # each backend does after F is compared over whole runs below. + np.testing.assert_array_equal(out, expected.reshape(-1)) + assert np.any(out != 0) + + +@pytest.mark.parametrize("formulation", ["interaction", "exponential"]) +def test_the_call_sequence_ledgers_and_metadata_are_the_reference_s(formulation): + """Item 2 on a normal run, PEC: every call in the same order on both.""" + runs = _pair(_case(formulation)) + reference, ours = runs["numpy_ref"], runs["native"] + assert [call[:2] for call in ours[2].calls] == [ + call[:2] for call in reference[2].calls + ] + # The positions follow step proposals that are smooth functions of independently + # rounded defect norms (a rejection's defect cancels to a few digits), so they + # agree within an accuracy envelope, not bits; the sequence above is exact. + zs = np.array([call[2] for call in ours[2].calls if call[0] == "evaluate"]) + zr = np.array([call[2] for call in reference[2].calls if call[0] == "evaluate"]) + np.testing.assert_allclose(zs, zr, rtol=1e-9, atol=0) + assert ours[2].ledger == reference[2].ledger + for key in ("atomic_calls", "tdse_source"): + assert ours[3][key] == reference[3][key] + assert ours[3]["tdse_last_evaluation"]["purpose"] == "predictor" + assert ours[0].terminal.error is None and reference[0].terminal.error is None + + +def test_a_budget_refusal_stops_both_at_the_same_step_with_the_same_charges(): + """Item 2 and 3: a reservation the atomic budget refuses, before the attempt.""" + runs = _pair(_case(), limit=7) + reference, ours = runs["numpy_ref"], runs["native"] + for run, _, binding, _ in (reference, ours): + error = run.terminal.error + assert isinstance(error, ResponseBudgetError) + assert "atomic-call budget cannot reserve" in str(error) + assert binding.ledger.reserved == 0 + assert ours[2].calls == reference[2].calls + assert ours[0].terminal.error.z == reference[0].terminal.error.z + assert ours[0].terminal.error.calls == reference[0].terminal.error.calls + + +@pytest.mark.parametrize("binding_class", [Recording, Interrupted]) +def test_a_failed_evaluation_keeps_the_last_accepted_state(binding_class): + """Item 3: a provider failure, or a signal at the boundary, before commit.""" + runs = _pair(_case(), binding_class, fail_evaluation=12) + reference, ours = runs["numpy_ref"], runs["native"] + positions = [] + for run, _, binding, _ in (reference, ours): + if binding_class is Recording: + assert isinstance(run.terminal.error, BackendError) + positions.append(run.terminal.z) + elif isinstance(run, RunTerminated): + # The reference's session lets the signal through to the run loop... + pass + else: + # ...native's reports it as its terminal error, the batch booked and the + # state eligible for the checkpoint; the run loop raises it the same. + assert isinstance(run.terminal.error, RunTerminated) + assert run.terminal.eligibility.value == "eligible" + positions.append(run.terminal.z) + assert binding.ledger.reserved == 0 + # The failed evaluation's trial was not recorded; its reservation released. + assert binding.calls[-1] == ("release",) + # The last accepted step's end, to the steps' independent rounding (see above). + assert all(position > 0 for position in positions) + if len(positions) == 2: + assert positions[1] == pytest.approx(positions[0], rel=1e-9, abs=0) + assert [call[:2] for call in ours[2].calls] == [ + call[:2] for call in reference[2].calls + ] + assert ours[2].ledger == reference[2].ledger + + +def test_an_archive_failure_after_an_accepted_commit_keeps_the_new_state(): + """Item 3: the commit is the boundary; the archive's error is the run's.""" + runs = _pair(_case(), fail_record=2) + reference, ours = runs["numpy_ref"], runs["native"] + for run, _, binding, _ in (reference, ours): + assert isinstance(run.terminal.error, TDSEArchiveError) + assert binding.ledger.reserved == 0 + assert binding.calls[-1] == ("release",) + # The second accepted step stands: the run stops at its end, on both. + assert ours[0].terminal.z == reference[0].terminal.z + assert [call[:2] for call in ours[2].calls] == [ + call[:2] for call in reference[2].calls + ] + + +@pytest.mark.parametrize("work_limit", [1, 256]) +def test_a_short_adaptive_run_agrees_within_its_per_step_budget(work_limit): + """Item 4: one attempt per call and the production bound alike. + + The case's collar holds its steps to about a two-hundredth of its length, so a + run of 0.2 um is some four hundred attempts, the same on both backends: each step + is one stage of the one-step budget, 16 eps (tests/bounds.py's rung), accumulated + linearly, the in-stage absorber damping rather than amplifying. Measured 1.3e-13 + on an Apple M5 Max against a bound of 1.4e-12. + """ + problem, request, grid = _case(length=2e-7, dz_max=2e-7) + descriptor = tdse_descriptor(problem.route) + fields, ledgers = {}, {} + for name, construct in ( + ("numpy_ref", NumpyRefBackend), + ("native", native.NativeBackend), + ): + binding = ManufacturedBinding(descriptor, 10_000, _law) + backend = construct(problem, request, grid=grid, tdse_binding=binding) + try: + run, _ = _run(backend, problem, request, work_limit=work_limit) + finally: + backend.close() + assert run.terminal.error is None + fields[name] = run.fields[-1] + ledgers[name] = (binding.ledger, run.status.attempts, run.status.rejected) + # The same attempts and rejections, so no decision forked on this case. + assert ledgers["native"] == ledgers["numpy_ref"] + attempts = ledgers["native"][1] + assert attempts > 8 + assert_within( + fields["native"], fields["numpy_ref"], attempts * 16 * EPS, what="short run" + ) + + +def test_a_device_build_refuses_the_tdse_response_by_name(monkeypatch): + """Item 6, by the refusal's rule: no device build runs a host source.""" + problem, request, grid = _case() + monkeypatch.setattr(native.backend, "_on_device", lambda: True) + binding = ManufacturedBinding(tdse_descriptor(problem.route), 10, _law) + with pytest.raises(BackendError, match="runs on the native CPU build"): + native.NativeBackend(problem, request, grid=grid, tdse_binding=binding) + + +def test_a_tdse_problem_needs_its_binding_and_a_binding_its_problem(): + problem, request, grid = _case() + with pytest.raises(BackendError, match="explicit host response binding"): + native.NativeBackend(problem, request, grid=grid) + + +def test_the_memory_ledger_lists_the_host_rows_and_the_binding_s(): + problem, request, grid = _case() + binding = ManufacturedBinding(tdse_descriptor(problem.route), 10, _law) + binding.memory_ledger = lambda: {"transport.solprop": 64} + backend = native.NativeBackend(problem, request, grid=grid, tdse_binding=binding) + try: + ledger = backend.memory_ledger() + finally: + backend.close() + n = problem.grids.n_omega + assert ledger["host_source"] == 3 * 16 * n + assert ledger["tdse_transport.solprop"] == 64 diff --git a/tests/driver/test_initial_z_and_checkpoint.py b/tests/driver/test_initial_z_and_checkpoint.py index 7bf10d4da..1dd6028fe 100644 --- a/tests/driver/test_initial_z_and_checkpoint.py +++ b/tests/driver/test_initial_z_and_checkpoint.py @@ -711,8 +711,8 @@ def test_tdse_does_not_guess_how_to_resample_custom_absorbers(atomic_simulation) def test_tdse_refuses_implicit_execution_and_dense_saves(atomic_simulation): simulation = atomic_simulation - with pytest.raises(DriverError, match="explicit backend='numpy_ref'"): - dataclasses.replace(simulation.settings, backend="auto") + # backend="auto" runs a TDSE response on native (SEP 49): no refusal. + assert dataclasses.replace(simulation.settings, backend="auto").backend == "native" with pytest.raises(DriverError, match="tdse_execution"): simulation.settings = dataclasses.replace( simulation.settings, tdse_execution=None diff --git a/tests/driver/test_interface_defaults.py b/tests/driver/test_interface_defaults.py index 833bbfae9..4c9155ef1 100644 --- a/tests/driver/test_interface_defaults.py +++ b/tests/driver/test_interface_defaults.py @@ -387,14 +387,17 @@ def test_a_copy_takes_the_route_s_default(route): def test_a_tdse_run_s_refusals_name_the_backend_and_the_integrator(): with pytest.raises(DriverError, match=r"integrator=Multistep\(\).*RungeKutta"): _capillary(tdse=_sae(), integrator=RungeKutta()) - with pytest.raises(DriverError, match="explicit backend='numpy_ref'"): + # Both backends run the TDSE response (SEP 49): native is no refusal. + assert ( SimulationSettings( save_planes=[1.0], dz_max=1.0, dz_init=1e-3, backend="native", tdse_execution=TDSEExecution(), - ) + ).backend + == "native" + ) # Not a record at all: refused where the annotations are not checked at run time. settings = SimulationSettings( save_planes=[1.0], dz_max=1.0, dz_init=1e-3, backend="numpy_ref" diff --git a/tests/driver/test_tdse_entry.py b/tests/driver/test_tdse_entry.py index 6a441a0b0..c0503a967 100644 --- a/tests/driver/test_tdse_entry.py +++ b/tests/driver/test_tdse_entry.py @@ -87,10 +87,9 @@ def test_tdse_keyword_builds_the_one_response_run_in_one_step(caplog): simulation = _capillary(tdse=response, tdse_execution=execution) assert simulation.responses == (response,) assert simulation.settings.tdse_execution == execution - assert simulation.settings.backend == "numpy_ref" - assert "resolves to numpy_ref" in caplog.text + assert simulation.settings.backend == "native" record = plain(simulation.configuration["tdse_entry"]) - assert record["backend"] == {"requested": "auto", "resolved": "numpy_ref"} + assert record["backend"] == {"requested": "auto", "resolved": "native"} assert record["model"] == "sae" assert tdse_descriptor(simulation.translate().route) is not None # A default execution record is taken when none is given. @@ -183,9 +182,10 @@ def test_tdse_execution_alone_and_a_foreign_record_are_refused(): ) -def test_explicit_incompatible_settings_keep_their_existing_refusals(): - with pytest.raises(DriverError, match=r"explicit backend='numpy_ref'.*native"): - _capillary(tdse=_sae(), backend="native") +def test_an_explicit_backend_is_kept_for_a_tdse_run(): + # Both backends run the TDSE response (SEP 49). + for backend in ("native", "numpy_ref"): + assert _capillary(tdse=_sae(), backend=backend).settings.backend == backend def test_statistics_default_excludes_what_a_tdse_response_cannot_supply(): From 0625619d423f79bdf0d19533955c4f47309fca2f Mon Sep 17 00:00:00 2001 From: John Travers Date: Thu, 12 Nov 2026 15:44:47 +0000 Subject: [PATCH 12/20] SEP 49 phase 3: fix the review's defects in the native hook The host source now reaches the engine: the binding hands `evaluate` its two rows with a no-op capsule owner (nanobind copied the ownerless views), the field read-only, and the stage's entry kernel writes the unscaled lab field, noise included, into the hook's field row instead of the hook reading the fine spectrum (the field divided by n_to). The rows are two, as the SEP says, under the generic `host_source`. A failed `end` after an accepted commit keeps the step's proposal, count and replay cursor. `NativeBackend.close` no longer unbinds under a leased state. An abandoned attempt's failed release rides on the run's error as a note, and the kept error is cleared per attempt. The native TDSE tests go through the C++ gather and the binding (field, F and the history's first row against the reference), use a remainder strong enough that F moves every compared field far beyond its budget and check that it does, restore the eight-step 128 eps budget, and exercise a real held SIGTERM, the floor's note, the terminal proposal and close under a session. Each defect, put back, fails them. Part II records the review and closes Q12. A stale import in test_tdse_edge_free is fixed. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HAYtF5P9QqEd3HQx3WJkA3 --- docs/dev/kernel-contract.md | 12 +- docs/dev/seps/sep-0049-tdse-on-native.md | 125 ++++--- native/src/adams_march.cpp | 8 +- native/src/bindings.cpp | 17 +- native/src/general/kernels.hpp | 10 +- native/src/general/setup.cpp | 5 +- native/src/general/trial.cpp | 39 +-- native/src/general/workspace.hpp | 7 +- native/src/solprop_native/_host_source.py | 10 +- native/src/solprop_native/backend.py | 4 +- native/src/solprop_native/session.py | 15 + tests/backends/test_tdse_edge_free.py | 4 +- tests/backends/test_tdse_native.py | 408 ++++++++++++++++++---- 13 files changed, 502 insertions(+), 162 deletions(-) diff --git a/docs/dev/kernel-contract.md b/docs/dev/kernel-contract.md index 13d5e47d4..dd7988df8 100644 --- a/docs/dev/kernel-contract.md +++ b/docs/dev/kernel-contract.md @@ -1036,18 +1036,20 @@ native CPU build the binding reaches the resident Adams march as a host source: session binding outside the kernel problem, whose three entry points the march calls from the thread that called `advance`, outside every parallel region, holding the interpreter lock for each. `begin(cost)` reserves an attempt's calls after the march's -budget check; `evaluate(field, z, purpose, out)` receives the lab-frame spectrum the -engine would hand its own source pass and writes `F`, the projected source before the +budget check; `evaluate(field, z, purpose, out)` receives, read-only, the lab-frame +spectrum the engine would hand its own source pass, unscaled and with any noise added, +which the stage's entry kernel writes into the hook's field row, and writes `F`, the projected source before the spectral collar, which the shared assembly (`solprop.backends.tdse_source`) forms as the reference does; `end(outcome)` records an accepted or rejected attempt after its decision, and after its commit for an accepted one, then releases the reservation, and an attempt that leaves before a decision only releases it. The engine's own response pass does not run; its stage finish applies the spectral collar, the in-stage absorber -and the propagator. The hook's two host rows are allocated when it is bound and listed -as `host_source` in the memory ledger. An exception in an entry point is kept on the +and the propagator. The hook's two rows, the field and `F`, are allocated when it is +bound, handed to it in place, and listed as `host_source` in the memory ledger. An exception in an entry point is kept on the Python side and reported to the march as a host failure: before the commit the attempt is abandoned, after it the committed attempt is returned with the failure, -and the session raises the kept exception unchanged. A device build refuses a TDSE +and the session raises the kept exception unchanged; an abandoned attempt's failed +release rides on the run's own error as a note. A device build refuses a TDSE problem at preparation. The backend independently projects macroscopic P and J with the existing modal diff --git a/docs/dev/seps/sep-0049-tdse-on-native.md b/docs/dev/seps/sep-0049-tdse-on-native.md index acb2e1c99..dec0c624b 100644 --- a/docs/dev/seps/sep-0049-tdse-on-native.md +++ b/docs/dev/seps/sep-0049-tdse-on-native.md @@ -1150,24 +1150,27 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. `Category::host`, naming its stage, `none` for `begin`. `ResidentEngine` forwards both to the bound source and names each evaluation's position and purpose (the startup at `z`, the predictor at `z + dz`). `GeneralState::bind_host_source` - allocates the two host rows and the source's view once, listed as `host_source` - in the memory ledger; the workspace's stage core then gathers the lab-frame bins - from the fine spectrum, forms the in-stage absorber as every route does, calls the - source from the calling thread outside every parallel region, and the stage finish - (`StageTerms::finish`) takes the source with the collar alone, skipping the - engine's own response pass. Host builds only: a device build throws, and its + allocates the hook's two rows once, the field and the source, listed as + `host_source` in the memory ledger; the stage's entry kernel + (`StageTerms::combine`) writes the lab-frame field into the field row, unscaled + and with any noise added, the workspace's stage core forms the in-stage absorber + as every route does and calls the source from the calling thread outside every + parallel region, and the stage finish (`StageTerms::finish`) takes the source with + the collar alone, skipping the engine's own response pass. Host builds only: a device build throws, and its adapter refuses the problem first. - The binding layer (`bindings.cpp`): `PythonHostSource` calls a Python object's - three methods under the interpreter lock the march released, catches - `nb::python_error` there and reports it to the march as `HostSourceError`; the - state holds the object (`State.bind_host_source`), the engine borrows it. + three methods under the interpreter lock the march released, handing `evaluate` + the two rows in place (the field read-only), catches `nb::python_error` there and + reports it to the march as `HostSourceError`; the state holds the object + (`State.bind_host_source`), the engine borrows it. - Python adapter: `solprop_native/_host_source.py` (new), `TDSEHostSource`: `begin` reserves, `evaluate` is `node_fields`, the binding's evaluation and the shared assembly and projection with the reference's own scalars (`scalars(problem, z, z)`, plain helpers) and prefactor (`prefactor_at` or the combined z table), `end` records the outcome and releases in a `finally`; every exception is kept, the - first as the error and later ones as notes. `NativeBackend` takes `tdse_binding`, - binds the hook for its life and unbinds it on close, refuses a TDSE problem without + first as the error and later ones as notes, and a new attempt starts with none. + `NativeBackend` takes `tdse_binding`, binds the hook for its life (closing the + state drops it, a leased one too), refuses a TDSE problem without its binding (and the reverse) and the statistics the TDSE response does not define, records `atomic_calls`, `tdse_execution`, `tdse_last_evaluation` and `tdse_source` (now `tdse_source.source_record`, shared with the reference), lists the binding's @@ -1175,12 +1178,14 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. `_interrupts.interruptible()` beside its commit around `advance` when a source is bound, and reports a host failure as the kept exception, a reservation the budget refused rewrapped with the last accepted step's position and field as the reference - does. + does; a kept exception the march did not stop for, an abandoned attempt's failed + release, rides on the run's error as a note, as the reference's `_trial` attaches + it. - solprop: the registry hands a TDSE binding to either backend; the driver's `numpy_ref`-only rules for a TDSE run (`_admit_tdse_execution`, `_check_tdse_execution`) are gone, so `backend="auto"` runs it on native; the capability rule `tdse-numpy-ref` is removed. - - Tests: `tests/backends/test_tdse_native.py` (13); the registry, entry, defaults + - Tests: `tests/backends/test_tdse_native.py` (23); the registry, entry, defaults and checkpoint tests that held the old refusals rewritten. - Documentation: `docs/dev/kernel-contract.md` (§1's `TDSESource`, the source reservations, §6's host source hook), `docs/dev/backends.md`, `AGENTS.md` and @@ -1198,16 +1203,23 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. error, the batch booked and the state eligible for its checkpoint, where the reference's session lets the signal through to the run loop; the run loop raises it either way. - - §7.3 item 1's stage comparison is not made one stage at a time. The reference's - `_adams_lab_source` on a freshly loaded backend is not its run's startup sample - (about 47 against the 1.3e7, mostly the in-stage absorber's term, the run's - startup row holds), and the two engines' startup rows read after one attempt are - 1 % apart while whole runs agree to 1e-13, so what those rows hold is not the same - quantity and is left as Q12. `F` is compared bit for bit, and the finish through - the runs of items 2 to 4. - - Item 4 is a short adaptive run, not eight steps: the case's collar holds its steps - to about a two-hundredth of its length, so 0.2 um is some four hundred attempts; - its budget is the attempts times the one-step 16 eps. + - §7.3 item 1 reads each engine's history row zero after the first attempt, not + the stage output alone: native has no stage output outside a run. Without the + in-stage absorber the initial step is accepted on both, and the row, the startup + output carried to the step's end in the interaction formulation, is equal bit for + bit; with it, the collar rejects the initial step, the row is the startup output, + and it is within one transform's bound, peak-normalised. + - Item 4 is two runs: eight steps of 10 nm without the in-stage absorber, within the + eight-step 128 eps; and, with the absorber, whose collar holds the steps to about a + two-hundredth of the run, 0.2 um in some four hundred attempts within the + attempts times the one-step 16 eps. Each checks that the source moves its field a + millionfold beyond the budget. + - The memory row is the generic `host_source`, not §6.2's `tdse_host_source`: the + engine knows no TDSE (P10). It holds §6.2's two `n_omega` rows. + - The reference records an accepted replayed step after releasing its reservation; + native's `end` records, then releases, for every decided attempt. The hook is not + told that a step is replayed, and the two orders differ only when the record or + the release fails. - Item 5 is met through the driver: the TDSE run tests that ask for `auto` (`test_tdse_hacc_run.py`, `test_tdse_archive_run.py`, `test_tdse_entry.py`, the gated-derivative and natural-output runs) now run on native, unchanged; those @@ -1217,16 +1229,25 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. `Multistep()` only, and a zero-length Runge-Kutta trial with a source bound does not give the reference's stage (above), so a later guard should refuse it there. - **Verification.** Native CPU build (Apple clang, editable, the `estimate` planning), - Apple M5 Max. `tests/backends/test_tdse_native.py`: `F` bit for bit on the gated and - cubic descriptors; the call sequence (every reserve, evaluation purpose, record and - release, in order), the atomic ledger and the `atomic_calls` and `tdse_source` - records equal to the reference's on a whole run in both formulations, the - evaluations' positions within 1e-9 relative; a budget refusal, a provider failure - and a SIGTERM before the commit, and an archive failure after it, each leaving the - stated state with nothing reserved and the reference's calls; a 0.2 um adaptive run - within its budget, with one attempt per call and the production bound (measured - 1.3e-13 against 1.4e-12); a device build and a missing binding refused by name; the - memory rows. The TDSE, worker and interrupt modules of `tests/backends` and + Apple M5 Max. `tests/backends/test_tdse_native.py`, its manufactured remainder + 1e-18 A m, strong enough that `F` moves every compared field seven orders or + more beyond its budget: through the C++ gather and the binding, in both formulations, on + the gated and cubic descriptors, with and without input noise, the field the hook + receives and `F` bit for bit, the history's first row as above; the call sequence + (every reserve, evaluation purpose, record and release, in order), the atomic ledger + and the `atomic_calls` and `tdse_source` records equal to the reference's on a whole + run in both formulations, the evaluations' positions within 1e-9 relative and the + final fields within the per-attempt budget; a budget refusal, a provider failure, a + SIGTERM before the commit (raised as an exception, and a real signal the commit + holds, raised by the evaluation's `raise_held` inside the session's + `interruptible`), and an archive failure after it, each leaving the stated state + with nothing reserved and the reference's calls, the archive failure's terminal + state carrying the committed step's proposal; an abandoned attempt's failed release + as the reference's note on the floor's error; the eight-step and collar-held runs + within their budgets, with one attempt per call and the production bound (measured + 1e-4 eps against 128 eps, and 5.0 eps against 6,480); a device build and a missing + binding refused by name; the memory rows; closing a backend whose state a session + leases. The TDSE, worker and interrupt modules of `tests/backends` and `tests/driver`, the compiled-multistep, resume-ledger, capability, structure and SEP modules: 1,411 passed, 3 skipped; `native/tests/test_backend.py` and `test_session_failure.py`: 61 passed. Non-TDSE runs, before (the #770 @@ -1236,15 +1257,41 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. device-compile lanes. - **Limitations.** TDSE on a native device build is refused by name (§12, Q2). The hook is not exercised by a real provider (§7.4, phase 4). -- **Open questions.** Q12: what the two engines' Adams startup rows hold for a TDSE - problem, 1 % apart while their runs agree to rounding, and why the reference's - `_adams_lab_source` outside a run is not its run's sample; until settled, item 1 - compares `F` and the runs compare the finish (owner: phase 4). +- **Open questions.** Q12, closed by the review below: the rows had been read at + different points (the reference's after an acceptance had carried it, native's + after a rejection), and at matched points the reviewer found them 1.3e-11 apart. The + mismatch hid the true one: native's `F` never reached its history. - **Cost** (phase 3, `git diff --numstat` from `42e3b22b`): C++ +332 −11, native Python +251 −12, `src/solprop` +49 −58, tests +464 −30. §6.8 estimated +0.3k C++, +0.4k - native Python and +0.5k tests for the hook. + native Python and +0.5k tests for the hook. The review's fixes: C++ +51 −35, + native Python +25 −4, tests +333 −79. +- **Review** (independent, of `478b3808`; fixed in the following commit). + - Critical: the binding handed `evaluate` ownerless NumPy views, which nanobind + copies when it passes them to Python, so `out[:] = source` wrote into a copy and + native's TDSE stages were the window times zero plus the absorber. The views now + have an owner, a capsule that frees nothing; the field is read-only. + - Critical: the stage core gathered the field from the fine spectrum, which holds + it divided by `n_to` (and doubled on an analytic grid's interior), so the hook saw + the field divided by 384. The entry kernel now writes the unscaled stage input, + noise included, into the hook's field row; the stage core reads that row. + - High: the tests could not see either defect. Item 1 compared the Python adapter + with itself; the runs' remainder of 1e-27 A m moved the field by 581 eps against + a budget of 6,480 eps. The tests above replace them. + - Medium: `NativeBackend.close` unbound the hook first, which a leased state + refuses; it now only closes the state. + - Low: a kept exception the march did not stop for was neither taken nor reported, + and stayed for the next attempt; it is now a note on the run's error and cleared + at each attempt's start. A failed `end` after an accepted commit returned before + the step's proposal, count and replay cursor were kept; they now are. + - Each fix was checked by putting its defect back and running the module: the + ownerless views fail 14 tests (item 1 in every case, both run budgets, both call + sequences and the floor's note); the scaled field fails item 1 in all 8 cases + (the manufactured remainder does not depend on the field, so the runs cannot see + a scale); the unbind fails the close test; the missing note fails the note test; + the session without `interruptible` fails the held-signal test; and the lost + proposal fails the archive test. Non-TDSE runs stay bit for bit as before. - **Handoff.** Phase 4: the manual check of §7.4 through the worker on both backends, - the ADR, the remaining pages, Q12, and the size comparison. + the ADR, the remaining pages, and the size comparison. ## Part III — History diff --git a/native/src/adams_march.cpp b/native/src/adams_march.cpp index 014d8a973..1cb5ea808 100644 --- a/native/src/adams_march.cpp +++ b/native/src/adams_march.cpp @@ -639,10 +639,16 @@ Result run(const Policy& policy, State& state, Engine& engine, const Bounds& bou factor(policy, choice.norm, choice.control, policy.max_growth)); record(step.end, step.dz, norm, true, trial.order); // After the commit (SEP 49 section 6.3): a failure here returns the committed - // attempt with the error. + // attempt with the error, its proposal and count kept. It is the reason the + // march stops, before the floor and the watch, as the reference's record raises + // before its floor check. try { attempt.end(HostOutcome::accepted); } catch (const std::exception& error) { + state.proposal = next; + ++accepted; + if (bounds.replay) + ++state.cursor; fail(result, Stage::accept, error); return result; } diff --git a/native/src/bindings.cpp b/native/src/bindings.cpp index c1af5ec66..4fdbc99d7 100644 --- a/native/src/bindings.cpp +++ b/native/src/bindings.cpp @@ -87,6 +87,9 @@ using IndexInput = nb::ndarray, nb::c_contig, nb::device::cpu>; using RealOutput = nb::ndarray; +// A read-only NumPy view of the engine's memory (solprop SEP 49's host source field). +using HostField = + nb::ndarray, nb::c_contig, nb::device::cpu>; using ComplexStorage = std::vector>; using RealStorage = std::vector; @@ -109,12 +112,16 @@ class PythonHostSource final : public native::HostSource { const native::HostPurpose purpose, const std::span> source) override { call([&] { - // Views of the engine's two host rows, valid for the call alone. + // Views of the engine's two host rows, valid for the call alone: the field + // read-only, the source written in place. Each has an owner, a capsule that + // frees nothing, because nanobind copies an ownerless array it hands to + // Python, and a write into that copy would never reach the engine. const std::array in_shape{field.size()}; const std::array out_shape{source.size()}; - const ComplexOutput in(const_cast*>(field.data()), 1, - in_shape.data(), nb::handle()); - const ComplexOutput out(source.data(), 1, out_shape.data(), nb::handle()); + const nb::capsule borrowed_in(field.data(), [](void* /*unused*/) noexcept {}); + const nb::capsule borrowed_out(source.data(), [](void* /*unused*/) noexcept {}); + const HostField in(field.data(), 1, in_shape.data(), borrowed_in); + const ComplexOutput out(source.data(), 1, out_shape.data(), borrowed_out); owner_.attr("evaluate")( in, z, purpose == native::HostPurpose::startup ? "startup" : "predictor", out); @@ -2671,6 +2678,8 @@ void BoundState::close() noexcept { general_state_->close(); general_state_.reset(); } + // A host source's owner (SEP 49) stays until the state is destroyed: this close + // runs without the interpreter lock, and the engine that borrowed it is gone. } // solprop SEP 47 section 6.4: a statistics evaluation lowered once from the problem, diff --git a/native/src/general/kernels.hpp b/native/src/general/kernels.hpp index 9c8510287..28d35701b 100644 --- a/native/src/general/kernels.hpp +++ b/native/src/general/kernels.hpp @@ -1274,9 +1274,10 @@ struct StageTerms { ComplexView2 current, fine, whole, solution, error, stage_phase, stage_inverse, noise; ComplexView2 flow_input, flow_lab, flow_drive, general_scaled, general_source; ComplexView2 projected, absorber; - // A host source's spectral source (solprop SEP 49), `(n_columns, n_omega)`: the - // stage's source in place of the projected response when `hosted`. - ComplexView2 host_spectrum; + // A host source's rows (solprop SEP 49), `(1, n_omega)`: the lab-frame field the + // entry writes for it, and its source, read in place of the projected response, when + // `hosted`. + ComplexView2 host_spectrum, host_input; ComplexView3 stages, phi_terms, prefactor_terms; RealView tableau, phi_delta, phi_at, prefactor_weights, window; IndexView crop, gaps; @@ -1478,6 +1479,9 @@ struct StageTerms { const double analytic_scale = analytic && index > 0 && 2 * crop(index) < n_to ? 2.0 : 1.0; fine(column, crop(index)) = analytic_scale * inverse_n_to * value; + // The host source evaluates the field itself, unscaled (SEP 49 section 6.2). + if (hosted) + host_input(column, index) = value; if (keep_lab) flow_lab(column, index) = value; // The general route synthesises on the coarse grid: the pad scale and the diff --git a/native/src/general/setup.cpp b/native/src/general/setup.cpp index b8cf2f8f9..05e425bf0 100644 --- a/native/src/general/setup.cpp +++ b/native/src/general/setup.cpp @@ -1691,9 +1691,8 @@ std::map GeneralWorkspace::memory_ledger() const { ledger.merge(base_memory_ledger()); if (host_source_) // solprop SEP 49: the host's field and source rows and the source's view. - ledger.emplace("host_source", (host_field_.size() + host_out_.size()) * - sizeof(std::complex) + - host_spectrum_.size() * sizeof(Complex)); + ledger.emplace("host_source", + (host_input_.size() + host_spectrum_.size()) * sizeof(Complex)); return ledger; } diff --git a/native/src/general/trial.cpp b/native/src/general/trial.cpp index d9463720a..d2e0cfd1e 100644 --- a/native/src/general/trial.cpp +++ b/native/src/general/trial.cpp @@ -5657,6 +5657,7 @@ StageTerms GeneralWorkspace::stage_terms(const std::size_t stage) const { // A host source is the stage's whole source (solprop SEP 49 section 6.2). terms.hosted = static_cast(host_source_); terms.host_spectrum = host_spectrum_; + terms.host_input = host_input_; if (terms.hosted) terms.zero_response = false; return terms; @@ -5981,8 +5982,7 @@ bool GeneralWorkspace::modal_core(const std::size_t stage) { void GeneralWorkspace::bind_host_source(std::shared_ptr source) { if (!source) { host_source_.reset(); - host_field_ = {}; - host_out_ = {}; + host_input_ = ComplexView2(); host_spectrum_ = ComplexView2(); return; } @@ -5993,40 +5993,39 @@ void GeneralWorkspace::bind_host_source(std::shared_ptr source) { problem_.dims.family != FieldFamily::field_real) throw BackendRefusal("a host source takes one real-field column of a modal route"); const auto n = problem_.dims.n_omega; - host_field_.assign(n, {}); - host_out_.assign(n, {}); + host_input_ = ComplexView2("solprop_native.host_source.field", 1, n); host_spectrum_ = ComplexView2("solprop_native.host_source.spectrum", 1, n); host_source_ = std::move(source); #endif } -// solprop SEP 49: the stage core of a host source. The lab-frame field is gathered -// from the fine spectrum's coarse bins before the inverse transform destroys them, the -// in-stage absorber is formed from the modal time field as on every route, and the -// host evaluates the source from the thread that called the march, outside every -// parallel region, into the view the finish reads. Host builds only. +// solprop SEP 49: the stage core of a host source. The stage's entry kernel left the +// lab-frame field, unscaled and with any noise added, in the host source's field row +// (`StageTerms::combine`); the in-stage absorber is formed from the modal time field +// as on every route; and the host evaluates the source from the thread that called the +// march, outside every parallel region, into the row the finish reads. Both rows are +// host memory on the CPU builds that admit a host source. Host builds only. void GeneralWorkspace::host_core() { #if defined(KOKKOS_ENABLE_CUDA) || defined(KOKKOS_ENABLE_HIP) throw BackendRefusal("a host source runs on the native CPU build only"); #else - execution_space_.fence("solprop_native.host_source.entry"); - const auto n = problem_.dims.n_omega; - for (std::size_t index = 0; index < n; ++index) { - const Complex value = fine_spectrum_(0, crop_index_(index)); - host_field_[index] = {value.real(), value.imag()}; - } if (absorbing()) { graph::execute(*inverse_real_plan_, fine_spectrum_, modal_real_, KokkosFFT::Normalization::none); form_absorber(); graph::execute(*absorber_real_plan_, absorber_real_, absorber_spectrum_, KokkosFFT::Normalization::none); - execution_space_.fence("solprop_native.host_source.absorber"); } - host_source_->evaluate(host_field_, host_z_, host_purpose_, host_out_); - for (std::size_t index = 0; index < n; ++index) - host_spectrum_(0, index) = - Complex(host_out_[index].real(), host_out_[index].imag()); + // The entry kernel's row and the absorber's work, on every instance, are done. + Kokkos::fence("solprop_native.host_source.entry"); + const auto n = problem_.dims.n_omega; + // Kokkos::complex and std::complex share one layout, two doubles. + static_assert(sizeof(Complex) == sizeof(std::complex)); + const std::span> field( + reinterpret_cast*>(host_input_.data()), n); + const std::span> source( + reinterpret_cast*>(host_spectrum_.data()), n); + host_source_->evaluate(field, host_z_, host_purpose_, source); #endif } diff --git a/native/src/general/workspace.hpp b/native/src/general/workspace.hpp index 93584f725..274f04668 100644 --- a/native/src/general/workspace.hpp +++ b/native/src/general/workspace.hpp @@ -744,11 +744,10 @@ class GeneralWorkspace { ComplexView transform_work_; #endif std::unique_ptr slab_forward_, slab_inverse_; - // solprop SEP 49: a host source, the lab-frame field and source rows it reads and - // writes on the host, and the source's view the stage finish reads. + // solprop SEP 49: a host source and its two rows, `(1, n_omega)`: the lab-frame field + // the entry kernel writes and the source the host writes and the finish reads. std::shared_ptr host_source_; - std::vector> host_field_, host_out_; - ComplexView2 host_spectrum_; + ComplexView2 host_input_, host_spectrum_; double host_z_ = 0.0; HostPurpose host_purpose_ = HostPurpose::startup; ComplexView2 current_, advanced_, solution_, error_, residual_, recycled_, noise_; diff --git a/native/src/solprop_native/_host_source.py b/native/src/solprop_native/_host_source.py index 6fc6f621b..0b7ee0580 100644 --- a/native/src/solprop_native/_host_source.py +++ b/native/src/solprop_native/_host_source.py @@ -55,7 +55,12 @@ def __init__( } def begin(self, cost: int) -> None: - """Reserve every atomic call of the attempt's `cost` evaluations.""" + """Reserve every atomic call of the attempt's `cost` evaluations. + + A new attempt starts with no kept exception: one the session did not take was + the previous attempt's, and was reported with it. + """ + self.error = None self._kept(lambda: self._binding.reserve(int(cost))) def evaluate( @@ -67,7 +72,8 @@ def evaluate( ) -> None: """Evaluate the source of the lab-frame `field` at `z`, m, into `out`. - Both arrays are the engine's host rows, valid for this call alone. + Both arrays are the engine's host rows, valid for this call alone; `field` + is read-only. """ def _run() -> None: diff --git a/native/src/solprop_native/backend.py b/native/src/solprop_native/backend.py index 6a343a6b7..9aa235315 100644 --- a/native/src/solprop_native/backend.py +++ b/native/src/solprop_native/backend.py @@ -1591,8 +1591,8 @@ def prepare_group( def close(self) -> None: """Release native plans and views; repeated calls have no effect.""" if not self._closed: - if self._host_source is not None: - self._native_state.bind_host_source(None) + # Closing the state drops a host source with the engine that borrowed it, + # also while a session still leases the state (SEP 49). self._native_state.close() _host_blas.release() self._closed = True diff --git a/native/src/solprop_native/session.py b/native/src/solprop_native/session.py index d80bfb474..e75a188eb 100644 --- a/native/src/solprop_native/session.py +++ b/native/src/solprop_native/session.py @@ -1856,6 +1856,21 @@ def _log_finish(self) -> None: def _error(self, event: _core.SessionEvent, outcome: Outcome) -> BaseException: """Return the exception a stopped or failed session reports. + A host source's exception the march did not stop for, the abandoned attempt's + failed release, is attached to it as a note, never raised in its place (SEP 49 + section 6.3), as the reference's `_trial` attaches it. + """ + error = self._reported_error(event, outcome) + host = self._host_source() + if host is not None and (kept := host.take()) is not None and kept is not error: + error.add_note(f"releasing the source reservation failed: {kept}") + return error + + def _reported_error( + self, event: _core.SessionEvent, outcome: Outcome + ) -> BaseException: + """Return the exception the march's event reports. + A failure is formatted by the backend's own march adapter, as its native call would have raised it. """ diff --git a/tests/backends/test_tdse_edge_free.py b/tests/backends/test_tdse_edge_free.py index df0772ea5..bba32a756 100644 --- a/tests/backends/test_tdse_edge_free.py +++ b/tests/backends/test_tdse_edge_free.py @@ -681,8 +681,8 @@ def test_a_gate_with_no_sample_in_its_leading_taper_says_so(): def test_a_cubic_closure_records_an_ungated_source_and_no_certificate(): # The cubic replacements are never gated (the source filter's removal): the run # metadata says so and carries no certificate to qualify a gate. - from solprop.backends.numpy_ref.backend import _source_record # noqa: PLC0415 + from solprop.backends.tdse_source import source_record # noqa: PLC0415 case = make_case(kerr_model="optmat_cubic") assert not case.descriptor.gated - assert _source_record(case.problem) == {"gated": False} + assert source_record(case.problem) == {"gated": False} diff --git a/tests/backends/test_tdse_native.py b/tests/backends/test_tdse_native.py index 61a8e99b3..062a66337 100644 --- a/tests/backends/test_tdse_native.py +++ b/tests/backends/test_tdse_native.py @@ -9,26 +9,32 @@ beside a large linear shadow. Native evaluates the source through its host source hook, the reference through its own evaluation; both call the shared assembly (``solprop.backends.tdse_source``), so ``F`` agrees bit for bit for the same field and -position, and what each backend does after it is compared within its budget. +position, and what each backend does after it is compared within its budget. The +remainder is strong enough that ``F`` moves every compared field seven orders or more +above its budget, and each budgeted comparison checks that it does. """ import dataclasses +import math import signal -from typing import Any +from typing import Any, cast import numpy as np import pytest -from bounds import EPS, assert_within +from bounds import EPS, assert_within, fft_bound from edge_source_cases import ManufacturedBinding from sessions.executors import IDENTITY, Started, drive +from solprop import _interrupts from solprop.backends import tdse_source from solprop.backends.numpy_ref import NumpyRefBackend +from solprop.backends.numpy_ref.multistep import HostHistory from solprop.backends.session import Start from solprop.backends.sessions import start_session from solprop.driver import Simulation from solprop.exceptions import ( BackendError, + ConvergenceError, ResponseBudgetError, RunTerminated, TDSEArchiveError, @@ -53,8 +59,17 @@ def _law(times): - """A manufactured remainder ``S(t)``, A m per atom.""" - return 1e-27 * np.exp(-((times / 8e-15) ** 2)) * np.sin(2.4e15 * times) + """A manufactured remainder ``S(t)``, A m per atom. + + Nine orders above the accounting fixtures' 1e-27 A m, so that ``F`` changes a run's + field far above any budget here: a source that never reached the field would fail. + """ + return 1e-18 * np.exp(-((times / 8e-15) ** 2)) * np.sin(2.4e15 * times) + + +def _silent(times): + """No remainder: the same binding, its source zero.""" + return np.zeros_like(times) def _case( @@ -62,10 +77,12 @@ def _case( kerr_model: Any = "tdse_native", *, length=LENGTH, + noise=None, **changes, ): """A short helium capillary with a manufactured TDSE response.""" dz_max = changes.pop("dz_max", 1e-6) + dz_init = changes.pop("dz_init", min(1e-6, dz_max)) parent = Simulation.from_capillary( 256e-6, length, @@ -79,11 +96,12 @@ def _case( modes=1, n_r=4, n_save=2, - noise=None, + noise=noise, + seed=None if noise is None else 7, plasma=False, backend="numpy_ref", integrator=Multistep(formulation=formulation), - dz_init=min(1e-6, dz_max), + dz_init=dz_init, dz_max=dz_max, ) basis = TDSEBasis( @@ -119,7 +137,7 @@ def _case( stats=StatsSpec(names=("energy",)), **changes, ), - noise=None, + noise=parent.noise, profiles=parent.profiles, frame=parent.frame, ) @@ -153,8 +171,12 @@ def evaluate(self, fields, *, z, purpose, density=None): class Recording(ManufacturedBinding): """Record every binding call, with a failure or a hang injected on request.""" - def __init__(self, descriptor, limit, *, fail_evaluation=None, fail_record=None): + def __init__( + self, descriptor, limit, *, fail_evaluation=None, fail_record=None, signal=False + ): super().__init__(descriptor, limit, _law) + self.signal = signal + self.raised = None self.calls = [] self.fail_evaluation = fail_evaluation self.fail_record = fail_record @@ -170,6 +192,8 @@ def evaluate(self, fields, *, z, purpose, density=None): and len(self.purposes) == self.fail_evaluation ): self.purposes.append(purpose) + if self.signal: + self.held() raise self.failure() return super().evaluate(fields, z=z, purpose=purpose, density=density) @@ -178,14 +202,36 @@ def failure(self) -> BaseException: def record_trial(self, *, accepted): self.calls.append(("record_trial", accepted)) - trials = sum(1 for call in self.calls if call[0] == "record_trial") - if self.fail_record is not None and trials == self.fail_record: + commits = self.calls.count(("record_trial", True)) + if accepted and self.fail_record is not None and commits == self.fail_record: raise TDSEArchiveError("manufactured archive failure") def release_reservation(self): self.calls.append(("release",)) super().release_reservation() + def held(self) -> None: + """Take a real SIGTERM, then raise it as the worker's wait does between polls. + + The signal arrives inside the session's commit, which holds it; only the + session's ``interruptible`` scope lets ``raise_held`` raise it here. + """ + signal.raise_signal(signal.SIGTERM) + try: + _interrupts.raise_held() + except RunTerminated as error: + self.raised = error + raise + + +class ReleaseFails(Recording): + """Fail the first release, after releasing.""" + + def release_reservation(self): + super().release_reservation() + if self.calls.count(("release",)) == 1: + raise BackendError("manufactured release failure") + class Interrupted(Recording): """Raise SIGTERM's exception from an evaluation, as the worker's wait raises a @@ -236,54 +282,152 @@ def _pair(case, binding_class=Recording, *, limit=1000, **options): return runs -def _reference_source(problem, request, grid, binding, field, z): - """``F`` as the reference forms it at `field` and `z`, its scalars and prefactor.""" - reference = NumpyRefBackend(problem, request, grid=grid, tdse_binding=binding) +def _unsourced(case, budget, field, *, work_limit=256): + """Assert that `field` is far from the run with no source: the comparison holds `F`. + + - ``budget``: the comparison's relative bound, which the source's effect must + exceed a millionfold. + """ + problem, request, grid = case + binding = ManufacturedBinding(tdse_descriptor(problem.route), 10**6, _silent) + backend = native.NativeBackend(problem, request, grid=grid, tdse_binding=binding) try: - reference.set_initial(field, None) - bundle = reference._stage_scalars(z, z) - fields = tdse_source.node_fields(problem, field, bundle) - result = binding.evaluate( - fields, z=z, purpose="startup", density=bundle.density - ) - polarisation, current = tdse_source.atomic_sources( - fields, result, problem, bundle - ) - source, _ = tdse_source.project_atomic_sources( - polarisation, - current, - result, - problem, - bundle, - reference._prefactor(bundle), - edge=tdse_source.atomic_edge_source(result, problem, bundle), - ) + run, _ = _run(backend, problem, request, work_limit=work_limit) finally: - reference.close() - return source + backend.close() + effect = np.linalg.norm(field - run.fields[-1]) / np.linalg.norm(field) + assert effect > 1e6 * budget, f"the source moves the field by {effect:.3g} only" -@pytest.mark.parametrize("kerr_model", ["tdse_native", "optmat_cubic"]) -def test_one_evaluation_is_the_reference_s_f_and_stage(kerr_model): - """Item 1: ``F`` bit for bit; the stage output within the one-stage budget.""" - problem, request, grid = _case(kerr_model=kerr_model) - descriptor = tdse_descriptor(problem.route) - field = np.array(problem.initial.field0) - reference_binding = Unbudgeted(descriptor, 10, _law) - expected = _reference_source(problem, request, grid, reference_binding, field, 0.0) - binding = Unbudgeted(descriptor, 10, _law) +def _reference_startup(case, monkeypatch): + """The reference's startup: the field it evaluates, its ``F`` and row zero.""" + problem, request, grid = case + seen: dict[str, Any] = {} + nonlinear = NumpyRefBackend._nonlinear_tdse + project = tdse_source.project_atomic_sources + start = HostHistory.start + + def recording_nonlinear(self, lab, bundle, inverse, purpose): + seen.setdefault("field", np.array(lab).reshape(-1)) + seen.setdefault("z", bundle.z1) + return nonlinear(self, lab, bundle, inverse, purpose) + + def recording_project(*args, **kwargs): + result = project(*args, **kwargs) + seen.setdefault("source", np.array(result[0]).reshape(-1)) + return result + + def recording_start(self, z): + row = start(self, z) + seen.setdefault("row", np.array(row).reshape(-1)) + seen.setdefault("history", self) + return row + + with monkeypatch.context() as patch: + patch.setattr(NumpyRefBackend, "_nonlinear_tdse", recording_nonlinear) + patch.setattr(tdse_source, "project_atomic_sources", recording_project) + patch.setattr(HostHistory, "start", recording_start) + binding = Unbudgeted(tdse_descriptor(problem.route), 10**6, _law) + backend = NumpyRefBackend(problem, request, grid=grid, tdse_binding=binding) + session = start_session( + backend, problem, request, Start.entrance(problem), owns=False + ) + try: + while session.status().attempts < 1: + batch = session.advance(work_limit=1) + if batch.receipt is not None: + session.acknowledge(batch.receipt) + seen["after"] = np.array(seen["history"]._store[0]).reshape(-1) + status = session.status() + seen["attempts"], seen["position"] = status.attempts, status.z + finally: + session.close() + backend.close() + return seen + + +def _native_startup(case): + """Native's startup through the C++ gather and the binding: the hook's field and + source, and the history's row zero after the first attempt.""" + problem, request, grid = case + binding = Unbudgeted(tdse_descriptor(problem.route), 10**6, _law) backend = native.NativeBackend(problem, request, grid=grid, tdse_binding=binding) + seen: dict[str, Any] = {} + hook = backend._host_source + evaluate = hook.evaluate + + def recording(field, z, purpose, out): + evaluate(field, z, purpose, out) + if "field" not in seen: + seen.update( + field=np.array(field), + writable=field.flags.writeable, + source=np.array(out), + z=z, + purpose=purpose, + ) + + # The engine looks the entry point up on the object at each call. + hook.evaluate = recording + session = start_session( + backend, problem, request, Start.entrance(problem), owns=False + ) try: - hook = backend._host_source - out = np.empty(problem.grids.n_omega, dtype=np.complex128) - hook.evaluate(field.reshape(-1), 0.0, "startup", out) - assert hook.last["purpose"] == "startup" + raw = cast("Any", session)._native + batch = raw.advance(1, math.inf) + while raw.status().attempts < 1: + raw.acknowledge(batch.sequence) + batch = raw.advance(1, math.inf) + seen["attempts"], seen["position"] = raw.status().attempts, raw.status().z + snapshot = raw.march_snapshot(batch.sequence, raw.status().generation) + assert snapshot is not None + seen["row"] = np.array(snapshot[2]["store"][0]).reshape(-1) finally: + session.close() backend.close() - # The same shared code on the same field and position: an exact identity. What - # each backend does after F is compared over whole runs below. - np.testing.assert_array_equal(out, expected.reshape(-1)) - assert np.any(out != 0) + return seen + + +@pytest.mark.parametrize("noise", [None, "input"]) +@pytest.mark.parametrize("kerr_model", ["tdse_native", "optmat_cubic"]) +@pytest.mark.parametrize("formulation", ["interaction", "exponential"]) +def test_the_startup_evaluation_is_the_reference_s( + formulation, kerr_model, noise, monkeypatch +): + """Item 1: the field the engine gathers and ``F`` bit for bit, through the C++ + gather and the binding; the history's first row after the first attempt bit for + bit without the in-stage absorber, within one transform's bound with it.""" + for absorber in (False, True): + changes: dict[str, Any] = {} if absorber else {"absorber": None} + case = _case(formulation, kerr_model, noise=noise, **changes) + expected = _reference_startup(case, monkeypatch) + ours = _native_startup(case) + assert ours["purpose"] == "startup" + assert ours["z"] == expected["z"] + assert not ours["writable"] + np.testing.assert_array_equal(ours["field"], expected["field"]) + np.testing.assert_array_equal(ours["source"], expected["source"]) + assert np.any(ours["source"] != 0) + if not absorber: + # The first attempt, the initial step, is accepted on both: row zero is + # the startup output, carried to the step's end in the interaction + # formulation, kept as it is in the exponential one. + assert ours["attempts"] == expected["attempts"] == 1 + assert ours["position"] == expected["position"] > 0 + if formulation == "exponential": + np.testing.assert_array_equal(expected["after"], expected["row"]) + np.testing.assert_array_equal(ours["row"], expected["after"]) + else: + # The collar rejects the initial step: row zero is the startup output. + assert ours["position"] == case[0].initial.z + peak = np.max(np.abs(expected["row"])) + assert_within( + ours["row"], + expected["row"], + fft_bound(case[0].grids.n_to) * peak, + elementwise=True, + what="startup row with the in-stage absorber", + ) @pytest.mark.parametrize("formulation", ["interaction", "exponential"]) @@ -305,6 +449,10 @@ def test_the_call_sequence_ledgers_and_metadata_are_the_reference_s(formulation) assert ours[3][key] == reference[3][key] assert ours[3]["tdse_last_evaluation"]["purpose"] == "predictor" assert ours[0].terminal.error is None and reference[0].terminal.error is None + # The fields the sequences came from, within the per-attempt budget (below). + budget = ours[0].status.attempts * 16 * EPS + assert_within(ours[0].fields[-1], reference[0].fields[-1], budget, what="run") + _unsourced(_case(formulation), budget, ours[0].fields[-1]) def test_a_budget_refusal_stops_both_at_the_same_step_with_the_same_charges(): @@ -353,32 +501,99 @@ def test_a_failed_evaluation_keeps_the_last_accepted_state(binding_class): assert ours[2].ledger == reference[2].ledger +def test_an_abandoned_attempt_s_failed_release_is_a_note_on_the_run_s_error(): + """Section 6.3: the step-size floor abandons the first attempt, whose release + fails; that failure rides on the floor's error as a note, on both.""" + case = _case(length=1e-2, dz_max=1e-4, dz_init=1e-4, dz_min=9e-5, absorber=None) + runs = _pair(case, ReleaseFails) + errors = [runs[name][0].terminal.error for name in ("numpy_ref", "native")] + assert isinstance(errors[0], ConvergenceError) + assert type(errors[1]) is type(errors[0]) + assert str(errors[1]) == str(errors[0]) + assert ( + errors[1].__notes__ + == errors[0].__notes__ + == ["releasing the source reservation failed: manufactured release failure"] + ) + assert runs["native"][2].calls == runs["numpy_ref"][2].calls + + +def test_a_held_signal_stops_a_native_evaluation_before_its_commit(): + """Item 3 and section 6.7: a real SIGTERM inside the session's commit is held, and + the session's ``interruptible`` scope lets the evaluation's wait raise it.""" + problem, request, grid = case = _case() + binding = Recording( + tdse_descriptor(problem.route), 1000, fail_evaluation=12, signal=True + ) + backend = native.NativeBackend(problem, request, grid=grid, tdse_binding=binding) + try: + with _interrupts.handled(): + if signal.getsignal(signal.SIGTERM) is not _interrupts._handle: + pytest.skip("SIGTERM has a handler of its own in this process") + run, _ = _run(backend, problem, request) + finally: + backend.close() + # Raised by the evaluation's own `raise_held`, not after the commit ended. + assert isinstance(binding.raised, RunTerminated) + assert run.terminal.error is binding.raised + assert run.terminal.eligibility.value == "eligible" + assert binding.ledger.reserved == 0 + assert binding.calls[-1] == ("release",) + # The last accepted step, as a provider failure there leaves it. + failed = _pair(case, fail_evaluation=12)["native"][0] + assert run.terminal.z == failed.terminal.z > 0 + + def test_an_archive_failure_after_an_accepted_commit_keeps_the_new_state(): - """Item 3: the commit is the boundary; the archive's error is the run's.""" - runs = _pair(_case(), fail_record=2) - reference, ours = runs["numpy_ref"], runs["native"] - for run, _, binding, _ in (reference, ours): + """Item 3: the commit is the boundary; the archive's error is the run's, and the + terminal state is the committed step's, its next proposal included.""" + problem, request, grid = _case() + descriptor = tdse_descriptor(problem.route) + ends = {} + for name, construct in ( + ("numpy_ref", NumpyRefBackend), + ("native", native.NativeBackend), + ): + binding = Recording(descriptor, 1000, fail_record=2) + backend = construct(problem, request, grid=grid, tdse_binding=binding) + snapshots = [] + + def keep(session, batch, snapshots=snapshots): + if batch.terminal is not None: + snapshots.append(session.march_snapshot(batch.terminal.ticket)) + + session = start_session( + backend, problem, request, Start.entrance(problem), owns=False + ) + try: + run = drive(Started(session), export=IDENTITY, raises=False, between=keep) + finally: + session.close() + backend.close() assert isinstance(run.terminal.error, TDSEArchiveError) + assert run.terminal.eligibility.value == "eligible" assert binding.ledger.reserved == 0 - assert binding.calls[-1] == ("release",) + assert binding.calls[-2:] == [("record_trial", True), ("release",)] + ends[name] = (run.terminal.z, snapshots[0], binding.calls) + assert run.terminal.history is not None + step = float(run.terminal.history.dz[-1]) + (z, snapshot, calls), (reference_z, expected, reference_calls) = ( + ends["native"], + ends["numpy_ref"], + ) # The second accepted step stands: the run stops at its end, on both. - assert ours[0].terminal.z == reference[0].terminal.z - assert [call[:2] for call in ours[2].calls] == [ - call[:2] for call in reference[2].calls - ] + assert z == pytest.approx(reference_z, rel=1e-12, abs=0) and z > 0 + assert snapshot.z == z + # The committed step's own proposal, not the one it was taken with. + assert snapshot.proposal == pytest.approx(expected.proposal, rel=1e-9, abs=0) + assert expected.proposal != pytest.approx(step, rel=1e-6, abs=0) + assert [call[:2] for call in calls] == [call[:2] for call in reference_calls] -@pytest.mark.parametrize("work_limit", [1, 256]) -def test_a_short_adaptive_run_agrees_within_its_per_step_budget(work_limit): - """Item 4: one attempt per call and the production bound alike. - - The case's collar holds its steps to about a two-hundredth of its length, so a - run of 0.2 um is some four hundred attempts, the same on both backends: each step - is one stage of the one-step budget, 16 eps (tests/bounds.py's rung), accumulated - linearly, the in-stage absorber damping rather than amplifying. Measured 1.3e-13 - on an Apple M5 Max against a bound of 1.4e-12. - """ - problem, request, grid = _case(length=2e-7, dz_max=2e-7) +def _agreement(case, work_limit): + """Run `case` on both backends; return the native and reference final fields and + the attempts each took, which must be the same.""" + problem, request, grid = case descriptor = tdse_descriptor(problem.route) fields, ledgers = {}, {} for name, construct in ( @@ -396,11 +611,33 @@ def test_a_short_adaptive_run_agrees_within_its_per_step_budget(work_limit): ledgers[name] = (binding.ledger, run.status.attempts, run.status.rejected) # The same attempts and rejections, so no decision forked on this case. assert ledgers["native"] == ledgers["numpy_ref"] - attempts = ledgers["native"][1] - assert attempts > 8 - assert_within( - fields["native"], fields["numpy_ref"], attempts * 16 * EPS, what="short run" - ) + return fields["native"], fields["numpy_ref"], ledgers["native"][1] + + +@pytest.mark.parametrize("work_limit", [1, 256]) +def test_eight_steps_agree_within_the_eight_step_budget(work_limit): + """Item 4: eight steps of 10 nm, one attempt per call and the production bound. + + Without the in-stage absorber's collar the steps reach their cap: eight steps, each + one stage of the one-step budget, 16 eps, accumulated linearly to 128 eps. + """ + case = _case(length=8e-8, dz_max=1e-8, absorber=None) + ours, reference, attempts = _agreement(case, work_limit) + assert attempts == 9 # the startup's attempt and eight accepted steps + assert_within(ours, reference, 128 * EPS, what="eight steps") + _unsourced(case, 128 * EPS, ours, work_limit=work_limit) + + +def test_a_collar_held_run_agrees_within_its_per_attempt_budget(): + """Item 4 with the in-stage absorber, whose collar holds the steps to about a + two-hundredth of the run: some four hundred attempts, each one stage of 16 eps, + accumulated linearly, the absorber damping rather than amplifying.""" + case = _case(length=2e-7, dz_max=2e-7) + ours, reference, attempts = _agreement(case, 256) + assert attempts > 200 + budget = attempts * 16 * EPS + assert_within(ours, reference, budget, what="collar-held run") + _unsourced(case, budget, ours) def test_a_device_build_refuses_the_tdse_response_by_name(monkeypatch): @@ -428,5 +665,22 @@ def test_the_memory_ledger_lists_the_host_rows_and_the_binding_s(): finally: backend.close() n = problem.grids.n_omega - assert ledger["host_source"] == 3 * 16 * n + # The field and source rows, each one complex128 spectrum. + assert ledger["host_source"] == 2 * 16 * n assert ledger["tdse_transport.solprop"] == 64 + + +def test_closing_the_backend_under_an_open_session_drops_the_host_source(): + """Closing while a session leases the state closes it, host source and all.""" + problem, request, grid = _case() + binding = ManufacturedBinding(tdse_descriptor(problem.route), 10, _law) + backend = native.NativeBackend(problem, request, grid=grid, tdse_binding=binding) + session = start_session( + backend, problem, request, Start.entrance(problem), owns=False + ) + try: + backend.close() + assert backend.closed + backend.close() + finally: + session.close() From b9e04c5e43ca1ec5f1adbde4e047f0962e23a8a8 Mon Sep 17 00:00:00 2001 From: John Travers Date: Thu, 12 Nov 2026 16:12:11 +0000 Subject: [PATCH 13/20] SEP 49 phase 4: ADR-078, the remaining pages, size and the manual check The review's follow-ups to phase 3: NativeBackend.close drops the host source once the state is closed, since the state's reference to it is invisible to the collector and a cycle through a kept exception was never freed (the held-signal test now checks the binding is collected); and item 4's run is eight binary-exact steps, where 8e-8 m in 10 nm steps split its last step into two. ADR-078 records the host source hook, its scope and the device refusal, the provider worker and the failure boundary, amending ADR-077's exception for the TDSE response. testing.md states the oracle boundary for shared code and the worker-contract tests. Part II records phase 4, the size against section 6.8, and the maintainer's manual check with a real Pyroraptor as a pending, copy-pasteable procedure (install, a two-mode script, pass criteria), dry-run with the manufactured provider. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HAYtF5P9QqEd3HQx3WJkA3 --- docs/dev/backends.md | 4 +- docs/dev/decisions.md | 58 +++- docs/dev/kernel-contract.md | 6 +- docs/dev/seps/sep-0049-tdse-on-native.md | 328 ++++++++++++++++++++++- docs/dev/testing.md | 16 ++ native/src/bindings.cpp | 9 +- native/src/solprop_native/backend.py | 10 +- tests/backends/test_tdse_native.py | 42 ++- 8 files changed, 445 insertions(+), 28 deletions(-) diff --git a/docs/dev/backends.md b/docs/dev/backends.md index a7f1f3086..5499a576c 100644 --- a/docs/dev/backends.md +++ b/docs/dev/backends.md @@ -17,7 +17,7 @@ processor is at or above a build's baseline, and otherwise raises `BackendError` to do; it stores that name before execution or worker transfer. `auto` has no fallback. Native being installed is a packaging policy ([SEP 43](seps/archive/sep-0043-retire-numba-cpu.md) §5), so a run does not have to prove that native could execute here, and nothing loads a runtime to -enforce it. A TDSE response's `auto` resolves to `native` like every other run (SEP 49). +enforce it. A TDSE response's `auto` resolves to `native` like every other run (SEP 49, ADR-078). Explicit names are preserved, so `backend="numpy_ref"` runs where native is absent and `backend="native"` on a processor below the baseline raises the package's refusal. `select` also accepts `"auto"`, checks `SOLPROP_REQUIRE_DEVICE` against the actual build before @@ -368,7 +368,7 @@ every step of the anatomy in [architecture.md §8](architecture.md) is a named f docstring stating the formula. It is slow (tens of separate array operations per stage) and that is acceptable; it is the reference against which every other backend is validated, and the supported executor of any feature `native` does not run yet, of which there is none -since the TDSE response joined the native CPU build (SEP 49; +since the TDSE response joined the native CPU build (SEP 49, ADR-078; [the list](../guide/performance/backends.md#features-that-run-on-the-reference-backend)). A new feature joins the list only as an accepted exception (§10). diff --git a/docs/dev/decisions.md b/docs/dev/decisions.md index 30843ac53..5195d6098 100644 --- a/docs/dev/decisions.md +++ b/docs/dev/decisions.md @@ -2239,7 +2239,8 @@ Withdrawn (development-process record, not a design decision). - **Status:** accepted (maintainer), 2026-11-11. Supersedes ADR-054 and ADR-061; amends ADR-064 (single precision under `backend="auto"`) and ADR-068 (`numpy_ref` "stays comprehensive"); withdraws [SEP 48](seps/archive/sep-0048-julia-backend.md). Those - records, and the SEPs implemented under them, stay as written. + records, and the SEPs implemented under them, stay as written. Amended by ADR-078: + native runs the TDSE response too. - **Context:** single precision runs on `numpy_ref` alone since `mlx` was retired (SEP 47): no installed backend runs it quickly, and `backend="auto"` refuses it. [SEP 36](seps/archive/sep-0036-native-reduced-precision.md) measured single and mixed @@ -2275,3 +2276,58 @@ Withdrawn (development-process record, not a design decision). - The backend registry becomes a fixed table of the two backends, with no plugin rows. - The precision field and CF4's settings leave the hashed configuration in one release, so resume identity changes once: a run started on an earlier release finishes on it. + +## ADR-078 — The TDSE response runs on native through a host source; its provider in a worker + +- **Status:** proposed with [SEP 49](seps/sep-0049-tdse-on-native.md) (PR #771), for the + maintainer. Amends ADR-077's "every feature except the TDSE response, for now": native + implements every feature. +- **Context:** the TDSE response ran on `numpy_ref` alone, and native refused it by name. + Its source is a complete atomic calculation at every radial node, from 93 s to over + 1,300 s a call, so a native rewrite of the provider buys nothing; what native lacked + was a way to hand the provider a stage's field and take its source. The provider, + Pyroraptor, ran in solprop's process, beside the native wheel's Kokkos and OpenMP + runtimes, which rules out a GPU provider beside a CPU wheel, and a call of minutes + could not be stopped. +- **Decision:** + - *The host source hook.* The native resident Adams march takes one optional, generic + host source, `begin(cost)`, `evaluate(field, z, purpose, source)` and + `end(outcome)`, that knows nothing of the TDSE. When it is bound, each source + evaluation hands it the stage's lab-frame field, unscaled and with any noise + added, and takes the projected source in place of the engine's own response pass; + the stage finish applies the collar, the in-stage absorber and the propagator as on + every route. It is called from the thread that called `advance`, outside every + parallel region, with the interpreter lock taken for the call: the one Python call + in a native trial. A Python exception is caught at the binding layer, never inside + a Kokkos kernel. Its scope: the native CPU build, the adaptive Adams march, and one + real column of a modal route (the TDSE response's own scope). The source assembly, + `solprop.backends.tdse_source`, is shared by both backends. + - *Device builds refuse.* A native CUDA or HIP build refuses a TDSE problem by name: + a TDSE run gives the provider the GPU or every CPU thread, and solprop's optical + side is negligible beside an atomic call. + - *The provider worker.* The provider runs only in a worker process of its own (the + same interpreter, `subprocess`, length-prefixed frames, no pickle), for presets, + bases, diagnostics and runs alike, so solprop's process never imports it. A worker + is replaced after any provider failure. Ctrl-C, SIGTERM and a stop flag end it + during a call; the call stays charged. + - *The failure boundary.* An attempt reserves its atomic calls before it evaluates. + A failure before the step commits abandons the attempt (its reservation released, + nothing recorded, the state the last accepted one); a failure after the commit, in + the archive's record, leaves the committed step standing and is the run's error; + a release that fails after another error is a note on it. Both backends keep this + boundary. + - `backend="auto"` runs a TDSE problem on native; `backend="numpy_ref"` still runs it, + as the reference. +- **Why:** the response is the provider's arithmetic either way; what native adds is + every optical feature it already runs, at its speed, and one backend for production. + A generic hook keeps the engine free of the TDSE and of Python, except at that one + call. A process boundary is the simplest isolation of two native runtimes, and the + only one that lets a GPU provider sit beside a CPU wheel; a few milliseconds a call + are nothing against minutes. +- **Consequences:** per evaluation the hook adds a Python call, the worker a copy each + way, and the shared assembly 0.13–1.9 ms (SEP 49, phase 0), below 1e-4 of the + fastest recorded atomic call. Native runs without a host source are unchanged bit for + bit. The hook's two rows are listed as `host_source` in the memory ledger. CI tests + the hook and the worker with manufactured providers; compatibility with an installed + Pyroraptor is checked by hand ([SEP 49](seps/sep-0049-tdse-on-native.md), section 7.4), + and the worker's handshake refuses a mismatched revision or execution space. diff --git a/docs/dev/kernel-contract.md b/docs/dev/kernel-contract.md index dd7988df8..b5a4b7cde 100644 --- a/docs/dev/kernel-contract.md +++ b/docs/dev/kernel-contract.md @@ -1031,9 +1031,9 @@ it changes no step, charge or reservation. None of this is part of the descripto or the kernel problem, which compiled backends see, so it is outside the kernel schema. -**The native host source hook** ([SEP 49](seps/sep-0049-tdse-on-native.md)). On the -native CPU build the binding reaches the resident Adams march as a host source: a -session binding outside the kernel problem, whose three entry points the march calls +**The native host source hook** ([SEP 49](seps/sep-0049-tdse-on-native.md), +ADR-078). On the native CPU build the binding reaches the resident Adams march as a +host source: a session binding outside the kernel problem, whose three entry points the march calls from the thread that called `advance`, outside every parallel region, holding the interpreter lock for each. `begin(cost)` reserves an attempt's calls after the march's budget check; `evaluate(field, z, purpose, out)` receives, read-only, the lab-frame diff --git a/docs/dev/seps/sep-0049-tdse-on-native.md b/docs/dev/seps/sep-0049-tdse-on-native.md index dec0c624b..ae49ee711 100644 --- a/docs/dev/seps/sep-0049-tdse-on-native.md +++ b/docs/dev/seps/sep-0049-tdse-on-native.md @@ -15,8 +15,8 @@ | **Requires** | [SEP 12](archive/sep-0012-tdse-nonlinear-response.md), [SEP 20](archive/sep-0020-hacc-helium-tdse.md) and [SEP 26](archive/sep-0026-edge-free-tdse-source.md) (the TDSE response and its source); [SEP 39](archive/sep-0039-native-adams-march.md) (the native Adams march); [SEP 47](archive/sep-0047-run-ownership.md) (backend-owned sessions); [SEP 25](archive/sep-0025-rounding-level-reproducibility.md) (agreement to the error bound); ADR-077 and the change it calls for (§3.4) | | **Supersedes** | — | | **Superseded-by** | — | -| **Implementation** | In progress: Opus 5.5 (AI), on `feature/sep-0049-tdse-on-native`; phases 0–3 | -| **Manual pages** | — | +| **Implementation** | In progress: Opus 5.5 (AI), on `feature/sep-0049-tdse-on-native`; phases 0–4, the manual check of §7.4 pending | +| **Manual pages** | [TDSE response](../../guide/media/tdse.md), [Choosing a backend](../../guide/performance/backends.md) | ## Part I — Proposal @@ -1209,8 +1209,8 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. output carried to the step's end in the interaction formulation, is equal bit for bit; with it, the collar rejects the initial step, the row is the startup output, and it is within one transform's bound, peak-normalised. - - Item 4 is two runs: eight steps of 10 nm without the in-stage absorber, within the - eight-step 128 eps; and, with the absorber, whose collar holds the steps to about a + - Item 4 is two runs: eight steps of 2^-27 m without the in-stage absorber (cap and + length binary-exact, so no final step is split), within the eight-step 128 eps; and, with the absorber, whose collar holds the steps to about a two-hundredth of the run, 0.2 um in some four hundred attempts within the attempts times the one-step 16 eps. Each checks that the source moves its field a millionfold beyond the budget. @@ -1245,9 +1245,11 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. state carrying the committed step's proposal; an abandoned attempt's failed release as the reference's note on the floor's error; the eight-step and collar-held runs within their budgets, with one attempt per call and the production bound (measured - 1e-4 eps against 128 eps, and 5.0 eps against 6,480); a device build and a missing - binding refused by name; the memory rows; closing a backend whose state a session - leases. The TDSE, worker and interrupt modules of `tests/backends` and + 5e-5 eps against 128 eps, and 5 to 6 eps against 6,480); a device build and a + missing binding refused by name; the memory rows; closing a backend whose state a + session leases, after which the binding, held in a cycle through the kept + exception's traceback, is collected (the state's own reference, which the collector + cannot see, is dropped once the state is closed). The TDSE, worker and interrupt modules of `tests/backends` and `tests/driver`, the compiled-multistep, resume-ledger, capability, structure and SEP modules: 1,411 passed, 3 skipped; `native/tests/test_backend.py` and `test_session_failure.py`: 61 passed. Non-TDSE runs, before (the #770 @@ -1293,6 +1295,318 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. - **Handoff.** Phase 4: the manual check of §7.4 through the worker on both backends, the ADR, the remaining pages, and the size comparison. +### Phase 4 — Qualification and documentation + +- **Scope.** Phase 4, 2026-11-12, Opus 5.5 (AI): the review's two follow-ups to phase 3, + the ADR, the remaining pages of §8, the size report against §6.8, and the procedure + for the manual check of §7.4, which is pending. The SEP stays `In progress`: it + becomes `Final` after CI, an A/B and the manual check. +- **Delivered.** + - Phase 3 follow-ups: + - `NativeBackend.close` now drops the host source once the state is closed + (`bind_host_source(None)`, which a closed state allows). Before, the state kept a + reference that the garbage collector cannot see. A cycle through it (a kept + exception's traceback reaching the session) was never freed, and the held-signal + test leaked eight nanobind instances. + - That test now checks, with a weak reference after `gc.collect()`, that the + binding is freed, and it fails without the fix. A session that could not close + keeps the source it still uses. + - Item 4's eight-step run actually took nine attempts, all accepted: 8e-8 m in + 10 nm steps left a last step one ulp over the cap, which the march split in two. + The cap and length are now binary-exact (2^-27 m and 2^-24 m), so the run is + eight accepted steps. The comment no longer counts the startup as an attempt. + - ADR-078 (`decisions.md`) covers: + - the host source hook, its scope and the device refusal; + - the provider worker; + - the failure boundary; + - `auto`. + + It amends ADR-077's "every feature except the TDSE response, for now", and + ADR-077's status now says so. + - `testing.md` gains *Shared code and out-of-process providers*: the oracle boundary + for shared code and the worker-contract tests. `backends.md` and + `kernel-contract.md` now cite ADR-078. + - The other pages of §8 were done in phases 2 and 3: + - the TDSE manual and *Choosing a backend*; + - `backends.md`, `kernel-contract.md` §6 and `architecture.md`; + - `AGENTS.md` and `native/AGENTS.md`; + - the release notes: `tdse-on-native.changed.md` for `auto`, and + `tdse-provider-worker.changed.md` for the worker and cancellation during an + atomic call. +- **Size** against §6.8. Lines are `git diff --numstat` from the tip of #770 + (`1f00b8f7`, which contains `main`), so #770's own changes are not counted. Code lines + are from `tools/size_report.py`. + + | part | `src/solprop` | native Python | C++ | tests | + |---|---|---|---|---| + | estimate, total | +0.6k | +0.4k | +0.3k | +1.3k | + | lines | +2,448 −784 | +280 −12 | +349 −11 | +2,160 −66 | + | code lines | +1,052 | +166 | — | — | + + By part: + - The shared source is a move: +34 code lines with `node_fields`, and its tests +0.3k. + - The worker and adapter are `_source_worker.py` (522 code lines) and `_pyroraptor.py` + (605), less 136 from `driver/tdse.py`: about +1.0k code lines, which is the + `src/solprop` overrun. Their tests are +1.0k. + - The hook is +166 native Python code lines and +349 C++ lines, with tests of +0.8k. + + The worker is larger than §6.8 estimated. The binding's validation and normalisation + stay in solprop's process, as §6.5 asks, while the requests the parent's view of the + haCC model makes must be served in the worker. Phase 2's report records what was + examined for removal. +- **Pending: the manual check of §7.4.** For the maintainer, with Pyroraptor at the + pinned revision `876b72d10b5286478125b842ea45548a5051b033` + (`solprop.driver.tdse._PROVIDER_REVISION`). Use a Pyroraptor checkout that contains + that commit: `git -C cat-file -e 876b72d10b52` must succeed. A squashed + public history may not contain it. + + 1. Set up the environment on the CPU. On macOS this needs Homebrew LLVM, `libomp` and + FFTW; on Linux, `CXX=clang++`, as Pyroraptor's installation guide says. + + ```sh + cd # this branch + uv sync # solprop and its native CPU build + git -C checkout 876b72d10b5286478125b842ea45548a5051b033 + uv pip install --python .venv/bin/python # its CPU (OpenMP) build + uv run --no-sync python -c "import pyroraptor as p; print(p.build_info())" + ``` + + From here on, use `uv run --no-sync`: a plain `uv run` or `uv sync` removes + Pyroraptor. + 2. On a GPU machine, use the same solprop environment with its native CPU build (a + native device build refuses TDSE). Install Pyroraptor's device build into it in + place of step 1's, configured as its `scripts/device_build.sh` does: + + ```sh + cd + kokkos="$PWD/build/sources/kokkos-5.2.1" + test -f "$kokkos/CMakeLists.txt" || git clone --depth 1 --branch 5.2.1 \ + https://github.com/kokkos/kokkos.git "$kokkos" + # CUDA + arch=$(scripts/device_build.sh cuda arch auto) + NVCC_WRAPPER_DEFAULT_COMPILER=g++ CMAKE_ARGS="-DPYRORAPTOR_EXECUTION_SPACE=CUDA \ + -DPYRORAPTOR_DEVICE_ARCH=$arch -DCMAKE_CXX_COMPILER=$kokkos/bin/nvcc_wrapper \ + -DFETCHCONTENT_SOURCE_DIR_KOKKOS=$kokkos" \ + uv pip install --python /.venv/bin/python --reinstall . + # or HIP + arch=$(scripts/device_build.sh hip arch auto) + CMAKE_ARGS="-DPYRORAPTOR_EXECUTION_SPACE=HIP -DPYRORAPTOR_DEVICE_ARCH=$arch \ + -DCMAKE_CXX_COMPILER=$(command -v hipcc) \ + -DFETCHCONTENT_SOURCE_DIR_KOKKOS=$kokkos" \ + uv pip install --python /.venv/bin/python --reinstall . + ``` + + 3. Save this script as `sep49_check.py` in an empty directory outside the checkout: + + ```python + """SEP 49 section 7.4: a real Pyroraptor through the worker, native against numpy_ref. + + python sep49_check.py compare DEVICE one short haCC(1) fast segment on both backends + python sep49_check.py cancel DEVICE native, Ctrl-C (SIGINT) during an atomic call + + DEVICE is cpu, cuda or hip, matching the installed Pyroraptor build. + """ + + import json + import os + import signal + import subprocess + import sys + import threading + import time + import warnings + from pathlib import Path + + import numpy as np + + from solprop.driver import Simulation + from solprop.driver.tdse import hacc_preset + from solprop.exceptions import ResponseBudgetError + from solprop.nonlinear.tdse import ( + TDSEConsistency, + TDSEExecution, + TDSEGasResponse, + TDSESampling, + ) + from solprop.output import open_file + + LENGTH = 2.0**-24 # one step of 60 nm: a startup and a predictor call, then a retry + BUDGET = 3 # atomic calls, at most (SEP 49 section 7.4) + RTOL = 0.1 # loose: the one step is accepted, a check of the plumbing, not accuracy + BUSY = float(os.environ.get("SEP49_BUSY_SECONDS", "5")) # worker CPU time in a call + + + def simulation(backend, device, budget, length): + preset = hacc_preset(1, "fast") + response = TDSEGasResponse( + "helium", + preset.basis, + preset.max_atomic_step, + "tdse_native", + 1.0, + preset.calibration(), + TDSESampling(4), + model="hacc", + # No weak-field consistency call: the budget is the march's alone. + consistency=TDSEConsistency(enforce=False), + ) + return Simulation.from_capillary( + 256e-6, length, "He", 0.1, + wavelength=800e-9, wavelength_limits=(400e-9, 1.6e-6), trange=40e-15, + fwhm=6e-15, energy=20e-6, modes=1, radial_rule="field_scale", n_r=2, + noise=None, n_save=2, rtol=RTOL, dz_init=LENGTH, dz_max=LENGTH, backend=backend, + tdse=response, + tdse_execution=TDSEExecution(device=device, max_atomic_calls=budget), + ) + + + def run(backend, device, budget=BUDGET, length=LENGTH): + path = Path(f"sep49-{backend}-{device}.h5") + path.unlink(missing_ok=True) + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + try: + simulation(backend, device, budget, length).run(path, progress="none") + except ResponseBudgetError as error: # the budget ends the segment + print(f"{backend}: {error}") + return path + + + def summary(path): + with open_file(path) as output: + outcome = output.outcome + plain = lambda value: json.loads(json.dumps(value, default=str)) # noqa: E731 + return { + "status": outcome.status, + "z": float(outcome.z), + "atomic_calls": plain(outcome.atomic_calls), + "logical_calls": plain(outcome.logical_calls), + "provider": plain(outcome.execution["tdse"]["build"]), + "fields": np.asarray(output.fields(convention="fft")), + } + + + def workers(): + """This process's provider workers and their CPU seconds.""" + table = subprocess.run( + ["ps", "-A", "-o", "pid=,ppid=,time=,command="], capture_output=True, text=True, + check=True, + ).stdout + found = {} + for line in table.splitlines(): + pid, ppid, cpu, command = line.split(None, 3) + if int(ppid) == os.getpid() and "solprop.driver._source_worker" in command: + parts = [float(part) for part in cpu.replace("-", ":").split(":")] + found[int(pid)] = sum(v * 60**i for i, v in enumerate(reversed(parts))) + return found + + + def compare(device): + results = {name: summary(run(name, device)) for name in ("numpy_ref", "native")} + reference, ours = results["numpy_ref"], results["native"] + for name, result in results.items(): + print(name, result["status"], result["z"], result["atomic_calls"], + result["logical_calls"]) + print("provider:", reference["provider"]) + if reference["logical_calls"]["accepted_steps"] == 0: + print("no step was accepted, so nothing to compare: lower LENGTH") + return 1 + first, last = reference["fields"][0], reference["fields"][-1] + gap = np.linalg.norm(ours["fields"][-1] - last) + difference = gap / np.linalg.norm(last) + print(f"final field, native against numpy_ref: {difference:.3e} relative") + print(f"... and against the segment's change: {gap / np.linalg.norm(last - first):.3e}") + keys = ("status", "z", "atomic_calls", "logical_calls") + same = all(ours[key] == reference[key] for key in keys) + print("same status, position and call ledgers:", same) + return 0 if same and difference <= 1e-9 else 1 + + + def cancel(device): + sent = {} + + output = Path(f"sep49-native-{device}.h5") + output.unlink(missing_ok=True) + + def watch(): + # A worker of the run (its file is open) that has computed for BUSY seconds + # is inside an atomic call. + while True: + busy = {pid: cpu for pid, cpu in workers().items() if cpu >= BUSY} + if busy and output.exists(): + time.sleep(float(os.environ.get("SEP49_DELAY", "1"))) + sent.update(worker=next(iter(busy)), at=time.monotonic()) + os.kill(os.getpid(), signal.SIGINT) + return + time.sleep(0.5) + + threading.Thread(target=watch, daemon=True).start() + try: + # Many steps, so that a call is in flight whenever the signal comes. + run("native", device, budget=1000, length=1000 * LENGTH) + except KeyboardInterrupt: + stopped = time.monotonic() - sent["at"] + print(f"KeyboardInterrupt {stopped:.2f} s after SIGINT, during a call") + else: + print("the run finished before a call was interrupted: raise LENGTH") + return 1 + alive = sent["worker"] in workers() + print("worker", sent["worker"], "still running" if alive else "gone") + result = summary(output) + print(result["status"], result["z"], result["atomic_calls"], result["logical_calls"]) + return 0 if not alive and result["atomic_calls"].get("reserved") == 0 else 1 + + + if __name__ == "__main__": + mode, device = sys.argv[1], sys.argv[2] + sys.exit({"compare": compare, "cancel": cancel}[mode](device)) + ``` + + 4. Run it from that directory. With the GPU build, replace `cpu` with `cuda` or `hip`. + + ```sh + uv run --no-sync --project python sep49_check.py compare cpu + uv run --no-sync --project python sep49_check.py cancel cpu + ``` + + `compare` runs one 60 nm step of haCC(1) fast at about 3e13 W/cm², with two radial + nodes, through the worker: first on `numpy_ref`, then on native. Each run makes at + most three atomic calls: a startup, a predictor and one retry. + - It passes (exit status 0) when both runs end with the same status, position and + call ledgers, and native's final field is within 1e-9 of `numpy_ref`'s, + relative. + - It also prints the difference against the segment's own change. + + `cancel` runs native over a longer segment. It sends itself SIGINT once a worker + of the run has computed for 5 s (`SEP49_BUSY_SECONDS`). On a GPU build, whose + worker may wait rather than compute, lower that threshold. It passes when: + - the run raises `KeyboardInterrupt` within about a second; + - the worker is gone and nothing is reserved; + - the file records the call as charged and the last accepted step's position. + 5. Record here, for each run: + - the provider revision and build (the `provider:` line); + - the machine; + - the threads (`SOLPROP_THREADS` or the machine's) or the device; + - both modes' output and exit status. + + Both modes were dry-run on an Apple M5 Max. The worker served the manufactured haCC + provider of `tests/hacc_cases.py` in place of Pyroraptor + (`solprop.driver.tdse._ADAPTER = "source_adapters:manufactured_tdse"`). + - `compare` took one accepted step on each backend with two atomic calls. The ledgers + were the same, and the final fields were 1.3e-13 apart, relative. + - `cancel` was run with every call after the second hanging + (`MANUFACTURED_HANG_AFTER=2`). It raised `KeyboardInterrupt` 0.11 s after the + signal: the worker was gone, three calls were charged, none were reserved, and the + run stood at its one accepted step. + + This checks the script and the plumbing, not Pyroraptor. +- **Verification.** `tests/backends/test_tdse_native.py`: 23 passed. The held-signal test + no longer produces nanobind's leak report, and with the unbind removed it fails its + weak-reference check. The non-TDSE bitwise check is unaffected, because phase 4 + changes the native extension's comments only. +- **Handoff.** Next are the maintainer's manual check (above), CI and an A/B; then + `Final`. + ## Part III — History - 2026-11-12: Drafted. diff --git a/docs/dev/testing.md b/docs/dev/testing.md index 1087ee173..ed4ce9827 100644 --- a/docs/dev/testing.md +++ b/docs/dev/testing.md @@ -168,6 +168,22 @@ The two thread settings matter: eighteen xdist workers each initialising an eighteen-thread Kokkos pool oversubscribe the machine, and OpenMP binding pins each worker's threads to the same first cores. +### Shared code and out-of-process providers + +Parity between the backends checks what each computes independently. It cannot check +code both call: a defect in the TDSE response's source assembly +(`solprop.backends.tdse_source`, [SEP 49](seps/sep-0049-tdse-on-native.md)) is the same +on both. That module's tests state their expectations independently of it, analytic or +manufactured (`tests/backends/test_tdse_source.py`, `tests/edge_source_cases.py`), and a +deliberate defect in each term is shown to fail one of them; parity then covers what each +backend does with the source. The atomic provider runs in a worker process. Its contract +is tested with real workers serving manufactured providers (`tests/source_adapters.py`, +`tests/driver/test_source_worker.py`, `tests/driver/test_tdse_worker.py`): results equal +a direct call bit for bit, the handshake's refusals, every failure, cancellation by +signal and stop flag, and clean-up. These tests have hard deadlines, so a broken +cancellation fails instead of hanging; the production code has none. An installed +Pyroraptor is not tested in CI; it is checked by hand (SEP 49, section 7.4). + ### Budgets and timing assertions The medium tier's budget is tight: time it with `poe test-medium` before adding to it. Most diff --git a/native/src/bindings.cpp b/native/src/bindings.cpp index 4fdbc99d7..07c52b72b 100644 --- a/native/src/bindings.cpp +++ b/native/src/bindings.cpp @@ -2222,8 +2222,9 @@ class BoundState { } // Closes a session that leases this state first (below, after its class). void close() noexcept; - // solprop SEP 49: bind `owner` as the state's host source, or unbind with None. The - // state keeps the object; the engine borrows it. + // solprop SEP 49: bind `owner` as the state's host source, or unbind with None (on + // a closed state too, which no longer borrows it). The state keeps the object; the + // engine borrows it. void bind_host_source(nb::object owner) { require_unleased(); if (owner.is_none()) { @@ -2678,8 +2679,8 @@ void BoundState::close() noexcept { general_state_->close(); general_state_.reset(); } - // A host source's owner (SEP 49) stays until the state is destroyed: this close - // runs without the interpreter lock, and the engine that borrowed it is gone. + // A host source's owner (SEP 49) is not released here, without the interpreter + // lock: the engine that borrowed it is gone, and `bind_host_source(None)` drops it. } // solprop SEP 47 section 6.4: a statistics evaluation lowered once from the problem, diff --git a/native/src/solprop_native/backend.py b/native/src/solprop_native/backend.py index 9aa235315..1c5cafb45 100644 --- a/native/src/solprop_native/backend.py +++ b/native/src/solprop_native/backend.py @@ -6,6 +6,7 @@ from __future__ import annotations +import contextlib import dataclasses import functools import logging @@ -1592,8 +1593,15 @@ def close(self) -> None: """Release native plans and views; repeated calls have no effect.""" if not self._closed: # Closing the state drops a host source with the engine that borrowed it, - # also while a session still leases the state (SEP 49). + # also while a session still leases the state (SEP 49). The state's own + # reference to the source is invisible to the garbage collector, which + # could then never free a cycle through it (a kept exception's traceback + # reaching the session), so it is dropped once the state is closed. A + # session that could not close still uses the state, and keeps it. self._native_state.close() + if self._host_source is not None: + with contextlib.suppress(RuntimeError): + self._native_state.bind_host_source(None) _host_blas.release() self._closed = True diff --git a/tests/backends/test_tdse_native.py b/tests/backends/test_tdse_native.py index 062a66337..a8fb488b6 100644 --- a/tests/backends/test_tdse_native.py +++ b/tests/backends/test_tdse_native.py @@ -15,8 +15,10 @@ """ import dataclasses +import gc import math import signal +import weakref from typing import Any, cast import numpy as np @@ -518,10 +520,13 @@ def test_an_abandoned_attempt_s_failed_release_is_a_note_on_the_run_s_error(): assert runs["native"][2].calls == runs["numpy_ref"][2].calls -def test_a_held_signal_stops_a_native_evaluation_before_its_commit(): - """Item 3 and section 6.7: a real SIGTERM inside the session's commit is held, and - the session's ``interruptible`` scope lets the evaluation's wait raise it.""" - problem, request, grid = case = _case() +def _held_signal_run(case) -> tuple[float, weakref.ref]: + """Run `case` on native with a real SIGTERM held in an evaluation; check the stop. + + Returns the run's last position and a weak reference to its binding, which the + signal's exception reaches through its traceback (the binding keeps it). + """ + problem, request, grid = case binding = Recording( tdse_descriptor(problem.route), 1000, fail_evaluation=12, signal=True ) @@ -539,9 +544,23 @@ def test_a_held_signal_stops_a_native_evaluation_before_its_commit(): assert run.terminal.eligibility.value == "eligible" assert binding.ledger.reserved == 0 assert binding.calls[-1] == ("release",) + return run.terminal.z, weakref.ref(binding) + + +def test_a_held_signal_stops_a_native_evaluation_before_its_commit(): + """Item 3 and section 6.7: a real SIGTERM inside the session's commit is held, and + the session's ``interruptible`` scope lets the evaluation's wait raise it. + + The closed backend leaves nothing the collector cannot see: the binding, in a + cycle through the kept exception's traceback, is freed. + """ + case = _case() + z, binding = _held_signal_run(case) + gc.collect() + assert binding() is None # The last accepted step, as a provider failure there leaves it. failed = _pair(case, fail_evaluation=12)["native"][0] - assert run.terminal.z == failed.terminal.z > 0 + assert z == failed.terminal.z > 0 def test_an_archive_failure_after_an_accepted_commit_keeps_the_new_state(): @@ -616,14 +635,17 @@ def _agreement(case, work_limit): @pytest.mark.parametrize("work_limit", [1, 256]) def test_eight_steps_agree_within_the_eight_step_budget(work_limit): - """Item 4: eight steps of 10 nm, one attempt per call and the production bound. + """Item 4: eight steps, one attempt per call and the production bound. - Without the in-stage absorber's collar the steps reach their cap: eight steps, each - one stage of the one-step budget, 16 eps, accumulated linearly to 128 eps. + Without the in-stage absorber's collar the steps reach their cap. Cap and length + are binary-exact (2^-27 m, 2^-24 m), so eight steps of the cap land on the end + with no split final step: each one stage of the one-step budget, 16 eps, + accumulated linearly to 128 eps. """ - case = _case(length=8e-8, dz_max=1e-8, absorber=None) + case = _case(length=2.0**-24, dz_max=2.0**-27, absorber=None) ours, reference, attempts = _agreement(case, work_limit) - assert attempts == 9 # the startup's attempt and eight accepted steps + # Eight accepted steps; the startup sample is evaluated inside the first. + assert attempts == 8 assert_within(ours, reference, 128 * EPS, what="eight steps") _unsourced(case, 128 * EPS, ours, work_limit=work_limit) From 7a5ca1f9a7e25c28007c8ee4768f834b6ffa4d51 Mon Sep 17 00:00:00 2001 From: John Travers Date: Thu, 12 Nov 2026 16:27:38 +0000 Subject: [PATCH 14/20] SEP 49: accept ADR-078; pin no Pyroraptor revision MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Maintainer decisions of 2026-11-12. ADR-078 is accepted. Until Pyroraptor has a release, solprop uses whatever provider is installed (by default the latest main): `_PROVIDER_REVISION` and `pinned_revision` are removed, and the handshake refuses only another protocol version or an execution space other than `TDSEExecution.device`. The worker's reported revision, build and execution space remain the run's build record; presets and bases name that revision in their source, the consistency cache keys on it (`tdse.provider_revision`), and the archive contexts drop `provider_pin`. Tests check that another revision is recorded, not refused. The SEP's §6.5, §7.2, §7.4, §9.3, Q10 and Part II, dependencies.md and ADR-078 say so, with a release note. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HAYtF5P9QqEd3HQx3WJkA3 --- changelog.d/tdse-provider-unpinned.changed.md | 1 + docs/dev/decisions.md | 12 ++-- docs/dev/dependencies.md | 14 ++-- docs/dev/seps/sep-0049-tdse-on-native.md | 50 +++++++++----- native/ci/check_skips.py | 2 +- src/solprop/driver/_pyroraptor.py | 13 ++-- src/solprop/driver/simulation.py | 3 +- src/solprop/driver/tdse.py | 67 +++++++++---------- src/solprop/driver/tdse_consistency.py | 3 +- tests/driver/test_tdse_binding.py | 18 +++-- tests/driver/test_tdse_consistency.py | 3 +- tests/driver/test_tdse_hacc_binding.py | 34 +++++----- tests/driver/test_tdse_hacc_installed.py | 17 ++--- tests/driver/test_tdse_worker.py | 23 ++++--- tests/source_adapters.py | 1 + tests/test_check_skips.py | 2 +- 16 files changed, 146 insertions(+), 117 deletions(-) create mode 100644 changelog.d/tdse-provider-unpinned.changed.md diff --git a/changelog.d/tdse-provider-unpinned.changed.md b/changelog.d/tdse-provider-unpinned.changed.md new file mode 100644 index 000000000..05750aa8b --- /dev/null +++ b/changelog.d/tdse-provider-unpinned.changed.md @@ -0,0 +1 @@ +solprop no longer pins a Pyroraptor revision: a TDSE run uses whatever provider is installed (by default its latest `main`, until Pyroraptor has a release), and records the revision, build and execution space the provider reports; the provider's protocol version and its execution space against `TDSEExecution.device` are still checked (SEP 49). diff --git a/docs/dev/decisions.md b/docs/dev/decisions.md index 5195d6098..aefff55f3 100644 --- a/docs/dev/decisions.md +++ b/docs/dev/decisions.md @@ -2279,8 +2279,8 @@ Withdrawn (development-process record, not a design decision). ## ADR-078 — The TDSE response runs on native through a host source; its provider in a worker -- **Status:** proposed with [SEP 49](seps/sep-0049-tdse-on-native.md) (PR #771), for the - maintainer. Amends ADR-077's "every feature except the TDSE response, for now": native +- **Status:** accepted (maintainer), 2026-11-12, with + [SEP 49](seps/sep-0049-tdse-on-native.md) (PR #771). Amends ADR-077's "every feature except the TDSE response, for now": native implements every feature. - **Context:** the TDSE response ran on `numpy_ref` alone, and native refused it by name. Its source is a complete atomic calculation at every radial node, from 93 s to over @@ -2308,7 +2308,11 @@ Withdrawn (development-process record, not a design decision). - *The provider worker.* The provider runs only in a worker process of its own (the same interpreter, `subprocess`, length-prefixed frames, no pickle), for presets, bases, diagnostics and runs alike, so solprop's process never imports it. A worker - is replaced after any provider failure. Ctrl-C, SIGTERM and a stop flag end it + is replaced after any provider failure. Its handshake refuses another protocol + version or an execution space other than `TDSEExecution.device`. No provider + revision is pinned: until Pyroraptor has a release, solprop uses whatever provider + is installed, by default the latest `main`, and records the revision, build and + execution space the worker reports as the run's build record. Ctrl-C, SIGTERM and a stop flag end it during a call; the call stays charged. - *The failure boundary.* An attempt reserves its atomic calls before it evaluates. A failure before the step commits abandons the attempt (its reservation released, @@ -2330,4 +2334,4 @@ Withdrawn (development-process record, not a design decision). bit. The hook's two rows are listed as `host_source` in the memory ledger. CI tests the hook and the worker with manufactured providers; compatibility with an installed Pyroraptor is checked by hand ([SEP 49](seps/sep-0049-tdse-on-native.md), section 7.4), - and the worker's handshake refuses a mismatched revision or execution space. + and the worker's handshake refuses a mismatched protocol or execution space. diff --git a/docs/dev/dependencies.md b/docs/dev/dependencies.md index 1ea7326b4..1fa08044f 100644 --- a/docs/dev/dependencies.md +++ b/docs/dev/dependencies.md @@ -40,12 +40,14 @@ is prepared; ordinary imports and non-TDSE runs never initialise its runtime. A provider, an unsupported interpreter (the provider supports Python 3.14 only) or a missing capability is refused with a `BackendError`. -solprop pins one provider source revision, `solprop.driver.tdse._PROVIDER_REVISION` -(currently `876b72d10b5286478125b842ea45548a5051b033`); -the binding checks the installed provider against it, and the source identity and the -loaded native build identity are recorded separately with every run. A device build -(CUDA or HIP) is selected explicitly through `TDSEExecution.device` and checked against -the build's execution space, with no fallback. +solprop pins no provider revision: until Pyroraptor has a release, it uses whatever +provider is installed, by default the latest `main` +([SEP 49](seps/sep-0049-tdse-on-native.md), ADR-078). The provider runs in a worker +process, whose handshake refuses another protocol version; the revision, build and +execution space it reports are recorded with every run, and a preset's source names +that revision. A device build (CUDA or HIP) is selected explicitly through +`TDSEExecution.device` and checked against the build's execution space, with no +fallback. ## The native backend package diff --git a/docs/dev/seps/sep-0049-tdse-on-native.md b/docs/dev/seps/sep-0049-tdse-on-native.md index ae49ee711..301ddcf56 100644 --- a/docs/dev/seps/sep-0049-tdse-on-native.md +++ b/docs/dev/seps/sep-0049-tdse-on-native.md @@ -425,9 +425,11 @@ provider: a run's evaluations, `basis_from_preset`, `hacc_preset`, `hacc_basis` length is a protocol error before anything is allocated. - **Handshake.** The worker's first frame carries the protocol version and the provider's identity: its build record (source revision, execution space, Kokkos configuration) and - the worker's process id. The parent refuses a protocol mismatch, a revision other than - `_PROVIDER_REVISION` or an execution space other than `TDSEExecution.device` by name, - before any evaluation. The build record in a run's metadata is this report. + the worker's process id. The parent refuses a protocol mismatch or an execution space + other than `TDSEExecution.device` by name, before any evaluation. It pins no provider + revision: until Pyroraptor has a release, solprop uses whatever provider is installed, + by default the latest `main`, and records the revision it reports (maintainer, + 2026-11-12). The build record in a run's metadata is this report. - **What crosses.** Preparation sends the descriptor's plain records (basis, model, outputs, chunks); the worker builds the models and plans and replies with its memory rows and plan records. Each atomic call sends one chunk's drive rows and receives its @@ -562,7 +564,8 @@ worker: 1. evaluations through the worker equal calling the manufactured provider directly, bit for bit, and the binding's call sequence, ledgers and archive are unchanged; -2. the handshake refuses a protocol, revision or execution-space mismatch by name; +2. the handshake refuses a protocol or execution-space mismatch by name, and records + another provider revision; 3. a provider exception, a worker exit, a closed pipe, a malformed or oversized frame, and a failure during start-up and preparation each end as the named failure of §6.5, with the worker reaped and both pipe ends closed; @@ -611,8 +614,9 @@ three atomic calls on the smallest preset, native against `numpy_ref` (class T), the worker, with Pyroraptor on the CPU and, where a GPU build is in use, on a GPU; and one cancellation during a call. Each check is reported in Part II with its configuration (provider revision, build, machine, threads or device). CI establishes the manufactured -contracts, not compatibility with an installed provider; the handshake's revision and -execution-space check (§6.5) refuses a mismatched provider at run time. +contracts, not compatibility with an installed provider; the handshake's protocol and +execution-space checks (§6.5) refuse a mismatched provider at run time, and the +provider's revision is recorded with every run. ### 8. Documentation @@ -652,8 +656,8 @@ providers. differ on Linux, macOS and Windows. Mitigation: §7.2 in CI on every platform CI runs; the reader thread avoids platform-specific polling. - **Provider compatibility untested in CI** (Q10). Mitigation: the handshake refuses a - mismatched revision or execution space by name; manual evidence with its configuration - (§7.4). + mismatched protocol or execution space by name, and records the provider's revision; + manual evidence with its configuration (§7.4). - **Python exceptions through C++.** Mitigation: caught at the binding layer, never inside a Kokkos kernel; §7.3 items 2–3 cover every entry point and the boundary. - **A shared-code defect invisible to parity.** Mitigation: §7.1's independent @@ -722,8 +726,9 @@ Decided by the maintainer on 2026-11-12 (Part III, review follow-up): 4. **Q9. Delivery.** One implementation pull request; Pyroraptor's changes in a linked companion (§11). 5. **Q10. A real provider in CI.** None; by hand as used, reported with its configuration; - the optional import and revision check kept, and the worker's actual provider - identity reported (§6.5, §7.4). + the optional import kept, no revision pinned (the installed provider is used, by + default the latest `main`, until Pyroraptor has a release), and the worker's actual + provider identity recorded (§6.5, §7.4). 6. **Q11. The transport.** Pipes only, arrays unchanged, bounded frames, transport copies in the memory ledger, responsive to cancellation (§6.5, §6.7). @@ -1322,7 +1327,17 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. - `auto`. It amends ADR-077's "every feature except the TDSE response, for now", and - ADR-077's status now says so. + ADR-077's status now says so. The maintainer accepted it on 2026-11-12. + - No provider revision is pinned (maintainer, 2026-11-12): until Pyroraptor has a + release, solprop uses whatever provider is installed, by default the latest `main`. + `_PROVIDER_REVISION` and `pinned_revision` are gone, and the handshake no longer + refuses another revision. It still refuses another protocol version, and an + execution space other than `TDSEExecution.device`. The build record keeps the + revision, build and execution space the worker reports (no `source_pin`); a + preset's or a basis's `source` names the reported revision, the consistency cache + keys on it (`tdse.provider_revision`), and the archives' contexts drop + `provider_pin`. Tests now check that another revision is recorded, not refused; + the installed-provider test runs with any revision; a release note says so. - `testing.md` gains *Shared code and out-of-process providers*: the oracle boundary for shared code and the worker-contract tests. `backends.md` and `kernel-contract.md` now cite ADR-078. @@ -1354,11 +1369,8 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. stay in solprop's process, as §6.5 asks, while the requests the parent's view of the haCC model makes must be served in the worker. Phase 2's report records what was examined for removal. -- **Pending: the manual check of §7.4.** For the maintainer, with Pyroraptor at the - pinned revision `876b72d10b5286478125b842ea45548a5051b033` - (`solprop.driver.tdse._PROVIDER_REVISION`). Use a Pyroraptor checkout that contains - that commit: `git -C cat-file -e 876b72d10b52` must succeed. A squashed - public history may not contain it. +- **The manual check of §7.4: procedure.** With Pyroraptor's latest `main` (no revision + is pinned), installed into solprop's environment. 1. Set up the environment on the CPU. On macOS this needs Homebrew LLVM, `libomp` and FFTW; on Linux, `CXX=clang++`, as Pyroraptor's installation guide says. @@ -1366,7 +1378,7 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. ```sh cd # this branch uv sync # solprop and its native CPU build - git -C checkout 876b72d10b5286478125b842ea45548a5051b033 + git -C switch main && git -C pull uv pip install --python .venv/bin/python # its CPU (OpenMP) build uv run --no-sync python -c "import pyroraptor as p; print(p.build_info())" ``` @@ -1656,6 +1668,10 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. - 2026-11-12: Phase 2 (the provider worker) reported in Part II; implementation pull request opened as a draft. - 2026-11-12: Phase 3 (the native hook) reported in Part II. +- 2026-11-12: Phase 4 reported in Part II. ADR-078 accepted by John Travers. The + maintainer decided that solprop pins no Pyroraptor revision until Pyroraptor has a + release: the installed provider is used, by default the latest `main`, and its + reported revision recorded; §6.5, §7.2, §7.4, §9.3 and Q10 revised to match. ### AI review — 2026-11-12 diff --git a/native/ci/check_skips.py b/native/ci/check_skips.py index 856e8ae73..ce500f762 100644 --- a/native/ci/check_skips.py +++ b/native/ci/check_skips.py @@ -40,7 +40,7 @@ r"the real Windows query", r"NumPy's interp may fuse off x86-64", # Other backends and packages a lane doesn't install. - r"Pyroraptor at the pinned revision is not installed", + r"Pyroraptor is not installed", # A lane on native has its wheel; these are the no-native lane's. r"solprop-native is installed", # Parametrized cases that apply to one backend's route only. diff --git a/src/solprop/driver/_pyroraptor.py b/src/solprop/driver/_pyroraptor.py index 6de646399..0f3ade89c 100644 --- a/src/solprop/driver/_pyroraptor.py +++ b/src/solprop/driver/_pyroraptor.py @@ -102,7 +102,7 @@ def _load_provider() -> _Provider: except (ImportError, OSError) as error: raise BackendError( "TDSE response requires the optional pyroraptor dependency; " - "install the pinned Pyroraptor revision into the environment, " + "install Pyroraptor into the environment, " "with a matching CPU, CUDA or HIP build" ) from error @@ -141,15 +141,14 @@ def _is(value: object, kind: type) -> bool: return isinstance(value, kind) -def _hacc_surface(provider: _Provider, revision: str) -> _Native: +def _hacc_surface(provider: _Provider) -> _Native: """Return the provider's haCC namespace, refusing a build without it.""" hacc = getattr(provider, "hacc", None) missing = [name for name in _HACC_SURFACE if not hasattr(hacc, name)] if missing: raise BackendError( "the installed TDSE provider has no haCC consumer surface " - f"(missing {', '.join(missing)}); install Pyroraptor at the pinned " - f"revision {revision}" + f"(missing {', '.join(missing)}); install a Pyroraptor build with it" ) return hacc @@ -484,7 +483,7 @@ def _basis_from_preset(self, request: Mapping[str, Any]) -> object: def _hacc_preset(self, request: Mapping[str, Any]) -> object: """Return a qualified haCC preset's basis fields and its records.""" - hacc = _hacc_surface(self._provider, request["revision"]) + hacc = _hacc_surface(self._provider) preset = hacc.helium_preset(request["ions"], request["variant"]) return { "fields": _hacc_fields( @@ -500,7 +499,7 @@ def _hacc_preset(self, request: Mapping[str, Any]) -> object: def _hacc_basis(self, request: Mapping[str, Any]) -> object: """Return the basis fields of a caller's provider objects, sent as records.""" - hacc = _hacc_surface(self._provider, request["revision"]) + hacc = _hacc_surface(self._provider) return _hacc_fields( hacc, request["atom"], @@ -620,7 +619,7 @@ def _prepare_hacc(self, basis: HaccBasis) -> _HaccModel: "TDSEExecution(complement='deterministic'), recorded as not " "applicable" ) - hacc = _hacc_surface(self._provider, request["revision"]) + hacc = _hacc_surface(self._provider) atom, native_basis, gauge = _hacc_objects(self._provider, hacc, basis) preset: _Native = None if basis.preset is not None: diff --git a/src/solprop/driver/simulation.py b/src/solprop/driver/simulation.py index 37447b001..f8cdde2af 100644 --- a/src/solprop/driver/simulation.py +++ b/src/solprop/driver/simulation.py @@ -3429,7 +3429,7 @@ def _consistency_gate( binding, declared, reference_wavelength=self.grid.reference_wavelength, - provider_revision=tdse_runtime.pinned_revision(), + provider_revision=tdse_runtime.provider_revision(build), computed_on={ "execution_space": native.get("execution_space"), "build": build, @@ -7524,7 +7524,6 @@ def _open_tdse_archive( # noqa: PLR0917 - called positionally "node_weights_m2": None if weights is None else weights.tolist(), "quadrature": metadata.quadrature, "execution": dataclasses.asdict(execution), - "provider_pin": tdse_runtime.pinned_revision(), # The file's own record on a resume, which the session holds and a snapshot # compares with. "optical_configuration_hash": ( diff --git a/src/solprop/driver/tdse.py b/src/solprop/driver/tdse.py index 09aebed53..fe16a72af 100644 --- a/src/solprop/driver/tdse.py +++ b/src/solprop/driver/tdse.py @@ -87,11 +87,10 @@ "estimate_binding_memory", "hacc_basis", "hacc_preset", - "pinned_revision", "prepare_binding", + "provider_revision", ] -_PROVIDER_REVISION = "876b72d10b5286478125b842ea45548a5051b033" _Preset = Literal["production", "high_accuracy", "strong_field"] _HaccVariant = Literal["standard", "fast"] # The adapter module the worker runs (SEP 49 section 6.5). @@ -138,15 +137,26 @@ def kill(self) -> None: ... -def pinned_revision() -> str: - """Return the provider source revision this solprop is pinned to. +def provider_revision(build: Mapping[str, object]) -> str: + """Return the provider source revision a build record names. + + solprop pins no provider revision (SEP 49 section 6.5): it uses the installed + provider and records the revision its worker reports. + + Parameters + ---------- + build : collections.abc.Mapping + A binding's or diagnostic session's build record (``_build_record``). Returns ------- str - Full Git revision; an installed build must report a prefix of it. + The provider's reported source revision, or ``""`` for an empty record. """ - return _PROVIDER_REVISION + native = build.get("native") + if not isinstance(native, Mapping): + return "" + return str(cast("Mapping[str, object]", native).get("source_revision", "")) def _open_provider( @@ -197,19 +207,16 @@ def _local_error(error: RemoteError, context: str) -> SolpropError: def _build_record( identity: Mapping[str, object], execution: TDSEExecution | None ) -> dict[str, object]: - """Refuse a stale provider or an execution-space mismatch the worker reported. + """Refuse an execution-space mismatch the worker reported; return its build. + + No provider revision is refused: solprop uses the installed provider, whose + revision, build and execution space the record keeps (SEP 49 section 6.5). - ``identity``: the handshake's provider identity: its build and distribution. - ``execution``: required device family, or None when only resolving a preset. - Returns: serializable native build and installed distribution identities. """ build = cast("Mapping[str, object]", identity["native"]) - revision = str(build["source_revision"]) - if len(revision) < 7 or not _PROVIDER_REVISION.startswith(revision): - raise BackendError( - f"TDSE native source {revision!r} does not match pinned " - f"{_PROVIDER_REVISION}" - ) space = str(build["execution_space"]).lower() devices = {"cpu": {"openmp", "serial", "threads"}, "cuda": {"cuda"}, "hip": {"hip"}} if execution is not None and space not in devices[execution.device]: @@ -219,7 +226,6 @@ def _build_record( ) record: dict[str, object] = { "native": dict(build), - "source_pin": _PROVIDER_REVISION, "distribution": identity.get("distribution"), } if "direct_url" in identity: @@ -227,16 +233,15 @@ def _build_record( return record -def _query(kind: str, request: Mapping[str, object], *, checked: bool) -> Any: +def _query(kind: str, request: Mapping[str, object]) -> tuple[Any, str]: """Ask a short-lived worker one question (a preset, a basis) and end it. - - ``checked``: refuse a stale provider first, as resolving a preset does. + - Returns: the reply, and the provider revision the worker reported. """ worker = _open_provider() try: - if checked: - _build_record(worker.identity, None) - return worker.request(kind, {**request, "revision": _PROVIDER_REVISION}) + revision = provider_revision(_build_record(worker.identity, None)) + return worker.request(kind, dict(request)), revision except RemoteError as error: raise _local_error(error, "TDSE provider query failed") from error finally: @@ -251,7 +256,7 @@ def basis_from_preset(target: str, preset: _Preset) -> TDSEBasis: target : str Atomic target, currently ``helium`` only. preset : {'production', 'high_accuracy', 'strong_field'} - Name of the pinned provider's measured basis and spectral cut. + Name of the provider's measured basis and spectral cut. Returns ------- @@ -264,7 +269,7 @@ def basis_from_preset(target: str, preset: _Preset) -> TDSEBasis: MaterialError If the target or preset is unsupported. BackendError - If the optional provider is missing or its native revision is stale. + If the optional provider is missing. """ if target != "helium" or preset not in ( "production", @@ -272,13 +277,10 @@ def basis_from_preset(target: str, preset: _Preset) -> TDSEBasis: "strong_field", ): raise MaterialError("TDSE requires helium and a supported provider preset") - fields = dict( - _query("basis_from_preset", {"target": target, "preset": preset}, checked=True) - ) + reply, revision = _query("basis_from_preset", {"target": target, "preset": preset}) + fields = dict(reply) provider_source = fields.pop("provider_source") - return TDSEBasis( - **fields, source=f"pyroraptor@{_PROVIDER_REVISION}: {provider_source}" - ) + return TDSEBasis(**fields, source=f"pyroraptor@{revision}: {provider_source}") # ------------------------------------------------------------------- haCC records @@ -416,7 +418,7 @@ def hacc_preset(ions: Literal[1, 2], variant: _HaccVariant = "standard") -> Hacc If ``ions`` is not the integer 1 or 2, or ``variant`` is not ``'standard'`` or ``'fast'``. BackendError - If the optional provider is missing, stale or has no haCC surface, or its + If the optional provider is missing or has no haCC surface, or its preset's variant disagrees with the request. Examples @@ -430,14 +432,14 @@ def hacc_preset(ions: Literal[1, 2], variant: _HaccVariant = "standard") -> Hacc raise MaterialError("hacc_preset needs ions=1 (haCC(1)) or ions=2 (haCC(5))") if not isinstance(variant, str) or variant not in ("standard", "fast"): raise MaterialError("hacc_preset needs variant='standard' or variant='fast'") - reply = _query("hacc_preset", {"ions": ions, "variant": variant}, checked=True) + reply, revision = _query("hacc_preset", {"ions": ions, "variant": variant}) preset = cast("Mapping[str, Any]", reply["preset"]) if preset["variant"] != variant: raise BackendError( f"the provider returned the {preset['variant']!r} haCC preset for the " f"{variant!r} variant" ) - source = f"pyroraptor@{_PROVIDER_REVISION}: {preset['source']}" + source = f"pyroraptor@{revision}: {preset['source']}" basis = HaccBasis( **cast("dict[str, Any]", reply["fields"]), source=source, @@ -512,7 +514,7 @@ def hacc_basis( """ # The objects travel to the worker as records of their fields, and are checked # there against the provider's own classes (SEP 49 section 6.5). - fields = _query( + fields, _ = _query( "hacc_basis", { "atom": atom, @@ -521,7 +523,6 @@ def hacc_basis( "cut_energy": cut_energy, "max_step_au": max_step_au, }, - checked=False, ) return HaccBasis(**cast("dict[str, Any]", fields), source=source) @@ -1132,7 +1133,6 @@ def _prepare_request(self, basis: TDSEBasis | HaccBasis) -> dict[str, object]: "memory": dict(self._memory), "memory_limit": execution.memory_limit, "defer_plans": self._defer_plans, - "revision": _PROVIDER_REVISION, } def _ensure_worker(self) -> None: @@ -2399,7 +2399,6 @@ def __init__( "memory": {}, "memory_limit": execution.memory_limit, "defer_plans": True, - "revision": _PROVIDER_REVISION, }, ), ) diff --git a/src/solprop/driver/tdse_consistency.py b/src/solprop/driver/tdse_consistency.py index cc4433b1c..baf000fc9 100644 --- a/src/solprop/driver/tdse_consistency.py +++ b/src/solprop/driver/tdse_consistency.py @@ -902,7 +902,7 @@ def check_consistency( basis, response.consistency, reference_wavelength=reference_wavelength, - provider_revision=tdse_runtime.pinned_revision(), + provider_revision=tdse_runtime.provider_revision(build), computed_on={ "execution_space": native.get("execution_space") if isinstance(native, dict) @@ -946,7 +946,6 @@ def _open_check_archive( "label": basis.label, "protocol": PROTOCOL_VERSION, "execution": asdict(execution), - "provider_pin": tdse_runtime.pinned_revision(), }, ) diff --git a/tests/driver/test_tdse_binding.py b/tests/driver/test_tdse_binding.py index 29f3ab372..39a7c38ff 100644 --- a/tests/driver/test_tdse_binding.py +++ b/tests/driver/test_tdse_binding.py @@ -9,7 +9,7 @@ import sys from dataclasses import dataclass, replace from types import SimpleNamespace -from typing import cast +from typing import Any, cast import numpy as np import pytest @@ -448,7 +448,9 @@ def unexpected_start(**options): @pytest.mark.parametrize( "device,space", [("cpu", "OpenMP"), ("cuda", "Cuda"), ("hip", "HIP")] ) -def test_binding_requires_matching_device_and_revision(provider, device, space): +def test_binding_requires_a_matching_device_and_records_any_revision( + provider, device, space +): provider.build = _Build(space) binding = tdse.prepare_binding(_descriptor(), TDSEExecution(device=device), 1) assert "plan.1.state" in binding.memory_ledger() @@ -457,9 +459,12 @@ def test_binding_requires_matching_device_and_revision(provider, device, space): provider.build = _Build("other") with pytest.raises(BackendError, match="native build"): tdse.prepare_binding(_descriptor(), TDSEExecution(device=device), 1) + # No revision is pinned: another provider revision is used and recorded. provider.build = _Build(space, "old0000") - with pytest.raises(BackendError, match="pinned"): - tdse.prepare_binding(_descriptor(), TDSEExecution(device=device), 1) + binding = tdse.prepare_binding(_descriptor(), TDSEExecution(device=device), 1) + build = cast("Any", binding.metadata())["build"] + assert build["native"]["source_revision"] == "old0000" + binding.close() def test_binding_input_and_resume_contract(provider): @@ -537,7 +542,10 @@ def missing_distribution(name): binding = tdse.prepare_binding(_descriptor(), TDSEExecution(), 1) metadata = _saved_metadata(binding) assert metadata["build"]["distribution"] is None - assert metadata["build"]["source_pin"].startswith(provider.build.source_revision) + assert metadata["build"]["native"]["source_revision"] == ( + provider.build.source_revision + ) + assert "source_pin" not in metadata["build"] assert provider.assembled == 1 assert provider.calls == [] binding.close() diff --git a/tests/driver/test_tdse_consistency.py b/tests/driver/test_tdse_consistency.py index d58b66bc6..946afb455 100644 --- a/tests/driver/test_tdse_consistency.py +++ b/tests/driver/test_tdse_consistency.py @@ -21,6 +21,7 @@ import pytest from scipy.integrate import solve_ivp from tests.hacc_cases import ( + FakeBuild, alpha_law, hacc_response, helium_alpha, @@ -61,7 +62,7 @@ _LAMBDA = 800e-9 _SUPPORT = (0.0, 2 * math.pi * C / 60e-9) -_REVISION = tdse.pinned_revision() +_REVISION = FakeBuild().source_revision # the manufactured provider's def _reference(omega): diff --git a/tests/driver/test_tdse_hacc_binding.py b/tests/driver/test_tdse_hacc_binding.py index 69d775d2a..da2bc81f3 100644 --- a/tests/driver/test_tdse_hacc_binding.py +++ b/tests/driver/test_tdse_hacc_binding.py @@ -13,7 +13,7 @@ import inspect import json import warnings -from typing import Any +from typing import Any, cast import numpy as np import pytest @@ -296,22 +296,23 @@ def test_hacc_refuses_vendor_blas_device_and_missing_surface(fake): binding = _bind(_descriptor(preset_basis(1)), TDSEExecution(device="cuda")) binding.close() fake.hacc = object() - with pytest.raises(BackendError, match=r"no haCC consumer surface.*876b72d1"): + with pytest.raises(BackendError, match=r"no haCC consumer surface"): _bind(_descriptor(), TDSEExecution(device="cuda")) -@pytest.mark.parametrize( - ("build", "device", "message"), - [ - (FakeBuild("OpenMP", "c2833fa"), "cpu", "pinned"), - (FakeBuild("OpenMP"), "cuda", "native build"), - ], -) -def test_stale_revision_and_device_mismatch_are_refused(fake, build, device, message): - fake.build = build - with pytest.raises(BackendError, match=message): - _bind(_descriptor(), TDSEExecution(device=device)) +def test_a_device_mismatch_is_refused_and_any_revision_recorded(fake): + fake.build = FakeBuild("OpenMP") + with pytest.raises(BackendError, match="native build"): + _bind(_descriptor(), TDSEExecution(device="cuda")) assert fake.prepared == [] + # No revision is pinned (SEP 49 section 6.5): another one runs, and is recorded. + fake.build = FakeBuild("OpenMP", "c2833fa") + binding = _bind(_descriptor(), TDSEExecution(device="cpu")) + build = cast("Any", binding.metadata())["build"] + assert build["native"]["source_revision"] == "c2833fa" + binding.close() + preset = tdse.hacc_preset(1) + assert preset.basis.source.startswith("pyroraptor@c2833fa: ") def test_plan_construction_driven_error_is_a_preparation_failure(fake): @@ -656,7 +657,8 @@ def test_hacc_preset_returns_si_records_and_a_research_calibration(fake, ions, v assert fake.variant_requests == [variant] assert preset.basis.preset == preset.identity == expected.preset assert dataclasses.replace(preset.basis, source=expected.source) == expected - assert preset.basis.source.startswith("pyroraptor@876b72d1") + # The revision the provider's worker reported, not a pin. + assert preset.basis.source.startswith(f"pyroraptor@{fake.build.source_revision}: ") assert preset.variant == preset.basis.variant == variant assert preset.step_margin == (0.62 if variant == "standard" else 0.9) assert preset.max_atomic_step == expected.max_step_au * hacc_cases.AU_TIME @@ -782,10 +784,6 @@ def test_resolvers_refuse_a_provider_without_the_hacc_surface(fake): tdse.hacc_basis(None, None, None, 1.0, 1.0, source="x") -def test_pinned_revision_is_the_documented_pin(): - assert tdse.pinned_revision() == "876b72d10b5286478125b842ea45548a5051b033" - - def test_window_record_is_reconstructed_unchanged(fake): descriptor = _descriptor( preset_basis(1), atomic_window=TDSEAtomicWindow(6e-15, 1e-15) diff --git a/tests/driver/test_tdse_hacc_installed.py b/tests/driver/test_tdse_hacc_installed.py index c901d2cdb..045e61c83 100644 --- a/tests/driver/test_tdse_hacc_installed.py +++ b/tests/driver/test_tdse_hacc_installed.py @@ -4,9 +4,9 @@ """Exercise the real haCC provider surface when it is installed (SEP 20, 7.1). -Skipped unless Pyroraptor at the pinned revision is importable. The check runs in a -subprocess with one provider and one OpenMP thread, so the provider's native runtime -never shares this test process: it resolves both presets through ``hacc_preset``, +Skipped unless Pyroraptor is importable, at any revision: solprop pins none (SEP 49 +section 6.5). The check runs in a subprocess with one provider and one OpenMP thread, +so the provider's native runtime never shares this test process: it resolves both presets through ``hacc_preset``, checks their identities and SI conversion against the records the fake-provider tests use, and runs one call of the provider's own tiny consumer-example haCC(1) basis through the binding (C3 to C5). It qualifies nothing about the atomic physics. @@ -120,11 +120,9 @@ """ -def _pinned_provider_installed() -> bool: +def _provider_installed() -> bool: if importlib.util.find_spec("pyroraptor") is None: return False - from solprop.driver import tdse # noqa: PLC0415 - only after the spec check - probe = subprocess.run( [ sys.executable, @@ -137,8 +135,7 @@ def _pinned_provider_installed() -> bool: env=_environment(), check=False, ) - revision = probe.stdout.strip().splitlines()[-1] if probe.stdout.strip() else "" - return probe.returncode == 0 and tdse.pinned_revision().startswith(revision or "-") + return probe.returncode == 0 and bool(probe.stdout.strip()) def _environment() -> dict[str, str]: @@ -153,8 +150,8 @@ def _environment() -> dict[str, str]: @pytest.mark.skipif( - not _pinned_provider_installed(), - reason="Pyroraptor at the pinned revision is not installed", + not _provider_installed(), + reason="Pyroraptor is not installed", ) def test_installed_provider_resolves_presets_and_runs_a_tiny_custom_call(): completed = subprocess.run( diff --git a/tests/driver/test_tdse_worker.py b/tests/driver/test_tdse_worker.py index bcefb3aa3..302952a90 100644 --- a/tests/driver/test_tdse_worker.py +++ b/tests/driver/test_tdse_worker.py @@ -14,6 +14,7 @@ import sys import threading import warnings +from typing import Any, cast import numpy as np import pytest @@ -103,19 +104,23 @@ def test_evaluations_through_the_worker_equal_the_provider_called_directly(worke assert len(workers) == 1 and not workers[0].alive -@pytest.mark.parametrize( - ("factory", "message"), - [("stale_tdse", "does not match pinned"), ("cuda_tdse", "native build uses")], -) -def test_the_handshake_refuses_a_stale_or_other_provider_by_name( - monkeypatch, workers, factory, message -): - monkeypatch.setattr(tdse, "_ADAPTER", f"source_adapters:{factory}") - with pytest.raises(BackendError, match=message): +def test_the_handshake_refuses_another_execution_space_by_name(monkeypatch, workers): + monkeypatch.setattr(tdse, "_ADAPTER", "source_adapters:cuda_tdse") + with pytest.raises(BackendError, match="native build uses"): _bind(_descriptor(preset_basis(1))) assert len(workers) == 1 and not workers[0].alive +def test_the_handshake_records_another_provider_revision(monkeypatch, workers): + monkeypatch.setattr(tdse, "_ADAPTER", "source_adapters:stale_tdse") + binding = _bind(_descriptor(preset_basis(1))) + try: + build = cast("Any", binding.metadata())["build"] + assert build["native"]["source_revision"] == "old0000" + finally: + binding.close() + + def test_a_failure_in_preparation_is_named_and_ends_the_worker(workers): with pytest.raises(BackendError, match="no complement option") as raised: _bind( diff --git a/tests/source_adapters.py b/tests/source_adapters.py index 369dd4ee4..f20bb022a 100644 --- a/tests/source_adapters.py +++ b/tests/source_adapters.py @@ -158,6 +158,7 @@ def manufactured_tdse(): def stale_tdse(): + """Serve the manufactured provider under another source revision.""" from hacc_cases import FakeBuild # noqa: PLC0415 - worker only return _adapter(_provider(build=FakeBuild(source_revision="old0000"))) diff --git a/tests/test_check_skips.py b/tests/test_check_skips.py index 2c1b9aec7..2845c79f5 100644 --- a/tests/test_check_skips.py +++ b/tests/test_check_skips.py @@ -33,7 +33,7 @@ def test_an_expected_reason_and_a_pass_are_clean(tmp_path, capsys): path = junit( tmp_path, ("a", ""), - ("b", skip("Pyroraptor at the pinned revision is not installed")), + ("b", skip("Pyroraptor is not installed")), ) assert check_skips.main([str(path)]) == 0 assert "0 unexpected" in capsys.readouterr().out From 169a932f01413357d2a7daa319b1e3df92c13949 Mon Sep 17 00:00:00 2001 From: John Travers Date: Thu, 12 Nov 2026 17:36:27 +0000 Subject: [PATCH 15/20] SEP 49: record the manual check with a real Pyroraptor The manual check of section 7.4, run with Pyroraptor main (7658b61) on a Slurm cluster: on 32 EPYC 7763 cores, compare gave numpy_ref and native equal ledgers and final fields 4.1e-13 apart (about 100 s a call after 8.4 min of model preparation), and cancel ended a run 0.13 s after SIGINT mid-call with the worker gone, the call charged and nothing reserved; on an A40 with Pyroraptor's CUDA build beside native's CPU wheel, a device mismatch was refused by name and cancel ended the run 0.47 s after SIGINT mid-call with the device released. No GPU compare: an A40's FP64 makes a call hours long, and that comparison stays manual (Q10). The script gains a refusal mode and starts its cancellation 5 s into the first call. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HAYtF5P9QqEd3HQx3WJkA3 --- docs/dev/seps/sep-0049-tdse-on-native.md | 135 ++++++++++++++++++----- 1 file changed, 107 insertions(+), 28 deletions(-) diff --git a/docs/dev/seps/sep-0049-tdse-on-native.md b/docs/dev/seps/sep-0049-tdse-on-native.md index 301ddcf56..45aaf3cfa 100644 --- a/docs/dev/seps/sep-0049-tdse-on-native.md +++ b/docs/dev/seps/sep-0049-tdse-on-native.md @@ -15,7 +15,7 @@ | **Requires** | [SEP 12](archive/sep-0012-tdse-nonlinear-response.md), [SEP 20](archive/sep-0020-hacc-helium-tdse.md) and [SEP 26](archive/sep-0026-edge-free-tdse-source.md) (the TDSE response and its source); [SEP 39](archive/sep-0039-native-adams-march.md) (the native Adams march); [SEP 47](archive/sep-0047-run-ownership.md) (backend-owned sessions); [SEP 25](archive/sep-0025-rounding-level-reproducibility.md) (agreement to the error bound); ADR-077 and the change it calls for (§3.4) | | **Supersedes** | — | | **Superseded-by** | — | -| **Implementation** | In progress: Opus 5.5 (AI), on `feature/sep-0049-tdse-on-native`; phases 0–4, the manual check of §7.4 pending | +| **Implementation** | In progress: Opus 5.5 (AI), on `feature/sep-0049-tdse-on-native`; phases 0–4, with the manual check of §7.4 done | | **Manual pages** | [TDSE response](../../guide/media/tdse.md), [Choosing a backend](../../guide/performance/backends.md) | ## Part I — Proposal @@ -1303,9 +1303,9 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. ### Phase 4 — Qualification and documentation - **Scope.** Phase 4, 2026-11-12, Opus 5.5 (AI): the review's two follow-ups to phase 3, - the ADR, the remaining pages of §8, the size report against §6.8, and the procedure - for the manual check of §7.4, which is pending. The SEP stays `In progress`: it - becomes `Final` after CI, an A/B and the manual check. + the ADR, the remaining pages of §8, the size report against §6.8, and the manual + check of §7.4, its procedure and its results. The SEP stays `In progress`: it + becomes `Final` after CI and an A/B. - **Delivered.** - Phase 3 follow-ups: - `NativeBackend.close` now drops the host source once the state is closed @@ -1415,8 +1415,10 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. python sep49_check.py compare DEVICE one short haCC(1) fast segment on both backends python sep49_check.py cancel DEVICE native, Ctrl-C (SIGINT) during an atomic call + python sep49_check.py refuse DEVICE native, a device the build does not have: refused - DEVICE is cpu, cuda or hip, matching the installed Pyroraptor build. + DEVICE is cpu, cuda or hip, matching the installed Pyroraptor build (for refuse, one + it does not match). """ import json @@ -1431,9 +1433,9 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. import numpy as np - from solprop.driver import Simulation + from solprop.driver import Simulation, tdse from solprop.driver.tdse import hacc_preset - from solprop.exceptions import ResponseBudgetError + from solprop.exceptions import BackendError, ResponseBudgetError from solprop.nonlinear.tdse import ( TDSEConsistency, TDSEExecution, @@ -1445,7 +1447,7 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. LENGTH = 2.0**-24 # one step of 60 nm: a startup and a predictor call, then a retry BUDGET = 3 # atomic calls, at most (SEP 49 section 7.4) RTOL = 0.1 # loose: the one step is accepted, a check of the plumbing, not accuracy - BUSY = float(os.environ.get("SEP49_BUSY_SECONDS", "5")) # worker CPU time in a call + DELAY = float(os.environ.get("SEP49_DELAY", "5")) # seconds into the first call def simulation(backend, device, budget, length): @@ -1475,12 +1477,15 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. def run(backend, device, budget=BUDGET, length=LENGTH): path = Path(f"sep49-{backend}-{device}.h5") path.unlink(missing_ok=True) + started = time.monotonic() with warnings.catch_warnings(): warnings.simplefilter("ignore") try: simulation(backend, device, budget, length).run(path, progress="none") except ResponseBudgetError as error: # the budget ends the segment print(f"{backend}: {error}") + finally: + print(f"{backend}: {time.monotonic() - started:.1f} s", flush=True) return path @@ -1536,21 +1541,25 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. def cancel(device): sent = {} - output = Path(f"sep49-native-{device}.h5") output.unlink(missing_ok=True) + # The binding's evaluation in this process is where an atomic call is in flight + # (a private name: this script checks solprop, it is not an API). + calling = threading.Event() + evaluate = tdse._ProviderBinding.evaluate + + def marked(self, *args, **kwargs): + calling.set() + return evaluate(self, *args, **kwargs) + + tdse._ProviderBinding.evaluate = marked + def watch(): - # A worker of the run (its file is open) that has computed for BUSY seconds - # is inside an atomic call. - while True: - busy = {pid: cpu for pid, cpu in workers().items() if cpu >= BUSY} - if busy and output.exists(): - time.sleep(float(os.environ.get("SEP49_DELAY", "1"))) - sent.update(worker=next(iter(busy)), at=time.monotonic()) - os.kill(os.getpid(), signal.SIGINT) - return - time.sleep(0.5) + calling.wait() + time.sleep(DELAY) + sent.update(worker=next(iter(workers()), None), at=time.monotonic()) + os.kill(os.getpid(), signal.SIGINT) threading.Thread(target=watch, daemon=True).start() try: @@ -1565,16 +1574,31 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. alive = sent["worker"] in workers() print("worker", sent["worker"], "still running" if alive else "gone") result = summary(output) + print("provider:", result["provider"]) print(result["status"], result["z"], result["atomic_calls"], result["logical_calls"]) - return 0 if not alive and result["atomic_calls"].get("reserved") == 0 else 1 + charged = result["atomic_calls"].get("attempted", 0) >= 1 + reserved = result["atomic_calls"].get("reserved") + return 0 if not alive and charged and reserved == 0 else 1 + + + def refuse(device): + try: + run("native", device) + except BackendError as error: + print("refused:", error) + return 0 if "device=" in str(error) else 1 + print(f"device={device!r} was not refused") + return 1 if __name__ == "__main__": mode, device = sys.argv[1], sys.argv[2] - sys.exit({"compare": compare, "cancel": cancel}[mode](device)) + sys.exit({"compare": compare, "cancel": cancel, "refuse": refuse}[mode](device)) ``` - 4. Run it from that directory. With the GPU build, replace `cpu` with `cuda` or `hip`. + 4. Run it from that directory. On the CPU build, `compare cpu` and `cancel cpu`; on a + GPU build, `refuse cpu` and `cancel cuda` (or `hip`), and `compare` only on a GPU + whose FP64 rate makes an atomic call short (not an A40). ```sh uv run --no-sync --project python sep49_check.py compare cpu @@ -1589,12 +1613,14 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. relative. - It also prints the difference against the segment's own change. - `cancel` runs native over a longer segment. It sends itself SIGINT once a worker - of the run has computed for 5 s (`SEP49_BUSY_SECONDS`). On a GPU build, whose - worker may wait rather than compute, lower that threshold. It passes when: + `cancel` runs native over a longer segment and sends itself SIGINT 5 s + (`SEP49_DELAY`) into its first atomic call. It passes when: - the run raises `KeyboardInterrupt` within about a second; - the worker is gone and nothing is reserved; - the file records the call as charged and the last accepted step's position. + + `refuse` asks for a device the installed build does not have (`cpu` with a CUDA + build, say) and passes when the run is refused by name. 5. Record here, for each run: - the provider revision and build (the `provider:` line); - the machine; @@ -1612,12 +1638,63 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. run stood at its one accepted step. This checks the script and the plumbing, not Pyroraptor. +- **The manual check of §7.4: results** (2026-11-12, run by Opus 5.5 at the + maintainer's request, on a shared HPC cluster through Slurm). Done with this + procedure's script, and with `refuse` on the GPU. + - **Code.** solprop `7a5ca1f9a7e25c28007c8ee4768f834b6ffa4d51` (this branch). + Pyroraptor `7658b615ef99e169d30ff0b12526d2a8d8b638a8` (the latest `main`; reports + `source_revision='7658b61'`, version `0.0.1.dev26+g7658b615e`). + - **CPU builds.** + - solprop's native CPU wheel: LOCAL profile, Release, GCC 16.2, FFTW 3.3.11 with + OpenMP threads, OpenMP OpenBLAS, `-march=znver3`. + - Pyroraptor's OpenMP build: GCC 16.2, LOCAL, `x86-64-v3`, Kokkos 5.2.1. It was + installed into the same environment, so the worker ran the same interpreter. + - **CPU machine.** A node with two AMD EPYC 7763; a Slurm job with 32 cores + (`-c 32 --hint=nomultithread`). The worker reported `requested_threads=32`, the + run's whole budget. + - **`compare cpu`**: exit 0, wall 1,422 s. + - Both backends ended `complete` at z = 5.96e-8 m (2^-24 m). Each took one + accepted order-1 step: a startup call and a predictor call, 2 atomic calls, + none reserved. The call ledgers were equal. + - Native's final field was 4.1e-13 from `numpy_ref`'s, relative, and 1.4e-8 of + the segment's own change. + - Each backend's run took 707 s (`numpy_ref`) and 712 s (native). About 8.4 min of + that was the provider's model preparation in the worker; each atomic call took + about 100 s at 32 cores, below the 10 min bound set beforehand. The problem + therefore did not need reducing. + - **`cancel cpu`**: exit 0, wall 550 s. + - SIGINT came 5 s into the first atomic call (after the same preparation), and + `KeyboardInterrupt` was raised 0.13 s later. + - The worker was gone. The file records status `failed` at z = 0, with the startup + call charged (`attempted` 1) and nothing reserved. + - **GPU builds.** + - Pyroraptor's CUDA wheel: `PYRORAPTOR_EXECUTION_SPACE=CUDA`, `AMPERE86`, CUDA + 12.8.1 with `nvcc_wrapper` over system GCC 11.5, Kokkos 5.2.1. It was compiled on + the login node with 4 jobs, in 26 min. + - It was installed into a second environment beside the same solprop native CPU + wheel, whose `execution_space` is `OpenMP`. + - **GPU machine.** A node with NVIDIA A40s; a Slurm job with one A40, 16 cores and a + 20 min cap. The worker reported `execution_space='Cuda'`. + - **`refuse cpu`**: exit 0, wall 9.4 s. The run was refused by name ("TDSE requested + device='cpu', but the native build uses 'Cuda'"), and no worker remained. + - **`cancel cuda`**: exit 0, wall 681 s. + - The provider's model preparation on the A40 took about 11 min. SIGINT came 5 s + into the first atomic call, and `KeyboardInterrupt` was raised 0.47 s later. + - The worker was gone, and the device released: `nvidia-smi` listed no compute + process and 0 MiB used afterwards. + - The file records the startup call charged (`attempted` 1) and nothing reserved. + - **No GPU `compare`.** The A40's FP64 rate is about 1/64 of its FP32, so an A40 + atomic call could take hours, and SEP 49 does not change Pyroraptor's GPU + numerics. The GPU numerical comparison of native against `numpy_ref` is to be done + by hand when the provider is next used on a production GPU (Q10). + - **Logs.** The build logs, and both runs' logs with every command's output, are + kept with the maintainer's campaign records. - **Verification.** `tests/backends/test_tdse_native.py`: 23 passed. The held-signal test no longer produces nanobind's leak report, and with the unbind removed it fails its weak-reference check. The non-TDSE bitwise check is unaffected, because phase 4 changes the native extension's comments only. -- **Handoff.** Next are the maintainer's manual check (above), CI and an A/B; then - `Final`. +- **Handoff.** Next are CI and an A/B; then `Final`. The GPU numerical comparison stays + manual, for the provider's next production GPU use (Q10). ## Part III — History @@ -1668,7 +1745,9 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. - 2026-11-12: Phase 2 (the provider worker) reported in Part II; implementation pull request opened as a draft. - 2026-11-12: Phase 3 (the native hook) reported in Part II. -- 2026-11-12: Phase 4 reported in Part II. ADR-078 accepted by John Travers. The +- 2026-11-12: Phase 4 reported in Part II, with the manual check of §7.4 on the CPU + (compare and cancel) and on an A40 (device refusal and cancel). ADR-078 accepted by + John Travers. The maintainer decided that solprop pins no Pyroraptor revision until Pyroraptor has a release: the installed provider is used, by default the latest `main`, and its reported revision recorded; §6.5, §7.2, §7.4, §9.3 and Q10 revised to match. From b23f804511a165904cb04df7a3d573bb7af08028 Mon Sep 17 00:00:00 2001 From: John Travers Date: Thu, 12 Nov 2026 20:21:22 +0000 Subject: [PATCH 16/20] Bound the strict-FMA check by symbol size; format the SEP's script block CI's Linux wheel failed check_strict_fma.py with 36 fused multiply-adds "in" march::reached. In that wheel reached is 0x5d bytes with none; the 36 lie in 9 KB of unlabelled code LTO placed after it, the dispersive response's Kokkos kernels (calls to GeneralDispersive::respond, sin, tanh, pow, exp). The extension is stripped, so objdump labels only exported functions and the check counted that code as reached's. It now bounds each strict function by its symbol's size from `objdump -T` and `-t`, with a test; the CI wheel passes (333 strict functions, none fused). No native source changes; the non-TDSE runs are unchanged bit for bit. ruff formats the Python block embedded in the SEP's Part II; the block is formatted. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HAYtF5P9QqEd3HQx3WJkA3 --- docs/dev/kernel-contract.md | 3 +- docs/dev/seps/sep-0049-tdse-on-native.md | 54 ++++++++++++++++++++---- native/ci/check_strict_fma.py | 49 +++++++++++++++++---- tests/test_check_strict_fma.py | 32 ++++++++++++++ 4 files changed, 119 insertions(+), 19 deletions(-) diff --git a/docs/dev/kernel-contract.md b/docs/dev/kernel-contract.md index b5a4b7cde..c20f36d2d 100644 --- a/docs/dev/kernel-contract.md +++ b/docs/dev/kernel-contract.md @@ -1350,7 +1350,8 @@ library (#135): each strict file's probe, beside a control that fuses, in the in build (`tests/backends/test_native_strict_witness.py`, and the wheel stage's smoke test); the same probe in the native unit tests of an optimised build with fused multiply-add; and the wheel stage's scan of the strict files' disassembly for a fused instruction -(`native/ci/check_strict_fma.py`). The boundary checks above and the Runge–Kutta march +(`native/ci/check_strict_fma.py`, each function bounded by its symbol's size, since a +stripped extension labels only exported functions). The boundary checks above and the Runge–Kutta march (§12) are held the same way. ## 12. The Runge–Kutta march diff --git a/docs/dev/seps/sep-0049-tdse-on-native.md b/docs/dev/seps/sep-0049-tdse-on-native.md index 45aaf3cfa..7d6d40fe8 100644 --- a/docs/dev/seps/sep-0049-tdse-on-native.md +++ b/docs/dev/seps/sep-0049-tdse-on-native.md @@ -1465,10 +1465,24 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. consistency=TDSEConsistency(enforce=False), ) return Simulation.from_capillary( - 256e-6, length, "He", 0.1, - wavelength=800e-9, wavelength_limits=(400e-9, 1.6e-6), trange=40e-15, - fwhm=6e-15, energy=20e-6, modes=1, radial_rule="field_scale", n_r=2, - noise=None, n_save=2, rtol=RTOL, dz_init=LENGTH, dz_max=LENGTH, backend=backend, + 256e-6, + length, + "He", + 0.1, + wavelength=800e-9, + wavelength_limits=(400e-9, 1.6e-6), + trange=40e-15, + fwhm=6e-15, + energy=20e-6, + modes=1, + radial_rule="field_scale", + n_r=2, + noise=None, + n_save=2, + rtol=RTOL, + dz_init=LENGTH, + dz_max=LENGTH, + backend=backend, tdse=response, tdse_execution=TDSEExecution(device=device, max_atomic_calls=budget), ) @@ -1506,7 +1520,9 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. def workers(): """This process's provider workers and their CPU seconds.""" table = subprocess.run( - ["ps", "-A", "-o", "pid=,ppid=,time=,command="], capture_output=True, text=True, + ["ps", "-A", "-o", "pid=,ppid=,time=,command="], + capture_output=True, + text=True, check=True, ).stdout found = {} @@ -1522,8 +1538,13 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. results = {name: summary(run(name, device)) for name in ("numpy_ref", "native")} reference, ours = results["numpy_ref"], results["native"] for name, result in results.items(): - print(name, result["status"], result["z"], result["atomic_calls"], - result["logical_calls"]) + print( + name, + result["status"], + result["z"], + result["atomic_calls"], + result["logical_calls"], + ) print("provider:", reference["provider"]) if reference["logical_calls"]["accepted_steps"] == 0: print("no step was accepted, so nothing to compare: lower LENGTH") @@ -1532,7 +1553,9 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. gap = np.linalg.norm(ours["fields"][-1] - last) difference = gap / np.linalg.norm(last) print(f"final field, native against numpy_ref: {difference:.3e} relative") - print(f"... and against the segment's change: {gap / np.linalg.norm(last - first):.3e}") + print( + f"... and against the segment's change: {gap / np.linalg.norm(last - first):.3e}" + ) keys = ("status", "z", "atomic_calls", "logical_calls") same = all(ours[key] == reference[key] for key in keys) print("same status, position and call ledgers:", same) @@ -1575,7 +1598,9 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. print("worker", sent["worker"], "still running" if alive else "gone") result = summary(output) print("provider:", result["provider"]) - print(result["status"], result["z"], result["atomic_calls"], result["logical_calls"]) + print( + result["status"], result["z"], result["atomic_calls"], result["logical_calls"] + ) charged = result["atomic_calls"].get("attempted", 0) >= 1 reserved = result["atomic_calls"].get("reserved") return 0 if not alive and charged and reserved == 0 else 1 @@ -1689,6 +1714,17 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. by hand when the provider is next used on a production GPU (Q10). - **Logs.** The build logs, and both runs' logs with every command's output, are kept with the maintainer's campaign records. +- **The wheel's strict-FMA check.** On the merge with `main`, CI's Linux wheel failed + `native/ci/check_strict_fma.py` with 36 fused multiply-adds "in" `march::reached`. + The disassembly of that wheel shows `reached` itself (0x5d bytes, its symbol's size) + with none. The 36 lie in 9 KB of unlabelled code that the link-time optimisation + placed after it: Kokkos kernels of the dispersive response, which call + `GeneralDispersive::respond`, `sin`, `tanh`, `pow` and `exp`. The wheel is stripped, + so `objdump` labels only exported functions and counted that code as `reached`'s. + No strict file includes a contracting header, and no strict code changed. The check + now bounds each strict function by its symbol's size from `objdump -T` and `-t`, + with a test. It passes on that wheel (333 strict functions, none fused). The + non-TDSE runs stay bit for bit as before. - **Verification.** `tests/backends/test_tdse_native.py`: 23 passed. The held-signal test no longer produces nanobind's leak report, and with the unbind removed it fails its weak-reference check. The non-TDSE bitwise check is unaffected, because phase 4 diff --git a/native/ci/check_strict_fma.py b/native/ci/check_strict_fma.py index 5c513c070..fa97343cb 100644 --- a/native/ci/check_strict_fma.py +++ b/native/ci/check_strict_fma.py @@ -17,6 +17,13 @@ It also fails when it finds none of the namespaces' functions, so that a change in symbol names or visibility cannot make it pass by finding nothing. +A wheel's extension is stripped: its disassembly labels only the exported functions, +and the code a link placed after one of them without a symbol of its own (another +file's local functions, under link-time optimisation) would read as that function's. +Each strict function is therefore bounded by its symbol's size from the dynamic and +static symbol tables (``objdump -T`` and ``-t``); a label whose size is unknown keeps +every line up to the next label. + Standard library only; it needs GNU ``objdump`` and reads x86-64 and AArch64 code. """ @@ -29,7 +36,7 @@ import zipfile from pathlib import Path -__all__ = ["STRICT", "fused_functions", "main"] +__all__ = ["STRICT", "fused_functions", "main", "symbol_sizes"] STRICT = re.compile( r"^[0-9a-f]+ :$") +LABEL = re.compile(r"^([0-9a-f]+) <.*>:$") +INSTRUCTION = re.compile(r"^\s*([0-9a-f]+):") +SYMBOL = re.compile(r"^([0-9a-f]+)\s.*\s\.text\S*\s+([0-9a-f]+)\s") +"""A code symbol in ``objdump -T`` or ``-t`` output: its address and its size.""" FUSED = re.compile( # x86-64: the FMA3 and AVX-512 forms (vfmadd231sd, vfnmsub132pd, vfmaddsub213pd). r"\tv?fn?m(?:add|sub)(?:sub|add)?\d{3}[ps][sdh]\b" @@ -46,22 +56,43 @@ ) +def _objdump(*arguments: str) -> str: + """Return GNU ``objdump``'s output for `arguments`.""" + return subprocess.run( + ["objdump", *arguments], check=True, capture_output=True, text=True + ).stdout + + +def symbol_sizes(library: Path) -> dict[int, int]: + """Return each code symbol's size, by address, from both symbol tables.""" + sizes: dict[int, int] = {} + for table in ("-T", "-t"): + for line in _objdump(table, str(library)).splitlines(): + if match := SYMBOL.match(line): + address, size = int(match[1], 16), int(match[2], 16) + sizes[address] = max(size, sizes.get(address, 0)) + return sizes + + def fused_functions(library: Path) -> tuple[int, dict[str, int]]: """Return the strict functions' count and, for each that fuses, its FMA count.""" - listing = subprocess.run( - ["objdump", "-d", "-C", "--no-show-raw-insn", str(library)], - check=True, - capture_output=True, - text=True, - ).stdout + sizes = symbol_sizes(library) + listing = _objdump("-d", "-C", "--no-show-raw-insn", str(library)) strict = 0 fused: dict[str, int] = {} name: str | None = None + end: int | None = None for line in listing.splitlines(): - if LABEL.match(line): + if label := LABEL.match(line): name = line if STRICT.match(line) else None strict += name is not None + start = int(label[1], 16) + end = start + sizes[start] if sizes.get(start) else None elif name is not None and FUSED.search(line): + instruction = INSTRUCTION.match(line) + if end is not None and instruction and int(instruction[1], 16) >= end: + # Past the symbol's end: unlabelled code the link placed after it. + continue fused[name] = fused.get(name, 0) + 1 return strict, fused diff --git a/tests/test_check_strict_fma.py b/tests/test_check_strict_fma.py index 753c33227..37941efce 100644 --- a/tests/test_check_strict_fma.py +++ b/tests/test_check_strict_fma.py @@ -101,3 +101,35 @@ def test_the_native_sessions_unit_is_strict(tmp_path, monkeypatch, capsys): wheel = _wheel(tmp_path, monkeypatch, STRICT + session) assert check_strict_fma.main([wheel]) == 1 assert "solprop_native::session::apply_recipe" in capsys.readouterr().err + + +def test_code_past_a_strict_symbol_s_end_is_not_its_own(tmp_path, monkeypatch, capsys): + # A stripped wheel labels only exported functions: a link may place another file's + # unlabelled, contracting code right after a strict function (as LTO put a + # dispersive kernel after march::reached). The symbol's size bounds the function. + listing = ( + "0000000000001100 :\n" + " 1100:\tvmulsd %xmm1,%xmm0,%xmm0\n" + " 1104:\tretq\n" + " 1110:\tvfmadd231sd %xmm1,%xmm2,%xmm0\n" + ) + symbols = ( + "0000000000001100 g DF .text\t0000000000000005 Base " + "solprop_native::march::reached(double, double, int)\n" + ) + wheel = tmp_path / "solprop_native-0-cp314-abi3-linux_x86_64.whl" + with zipfile.ZipFile(wheel, "w") as archive: + archive.writestr("solprop_native/_core.abi3.so", b"") + + def objdump(command, **_): + table = "-T" in command + output = symbols if table else "" if "-t" in command else listing + return subprocess.CompletedProcess(command, 0, stdout=output, stderr="") + + monkeypatch.setattr(check_strict_fma.subprocess, "run", objdump) + assert check_strict_fma.main([str(wheel)]) == 0 + assert "1 strict functions, none fused" in capsys.readouterr().out + # Inside the symbol, the same instruction is the function's own. + listing = listing.replace(" 1110:", " 1102:") + assert check_strict_fma.main([str(wheel)]) == 1 + assert "1 fused multiply-add(s) in" in capsys.readouterr().err From adc76c0430bf4aa7f854c2430f4534e1e31ccafa Mon Sep 17 00:00:00 2001 From: John Travers Date: Thu, 12 Nov 2026 22:03:28 +0000 Subject: [PATCH 17/20] SEP 49: fix the CI test failures on the merged branch - test_snapshots: the TDSE snapshot test read the in-process haCC model and plans, which live in the provider's worker since phase 2; it now checks that the snapshots keep the same worker, model view and plans. The failure was the test's own hook, logged by the snapshot transaction; the snapshot path does not touch the model. - test_tdse_worker: the import-isolation claim sets aside provider modules another test of the process loaded, then checks that the run through the worker imports neither pyroraptor nor the adapter. - test_source_worker, test_tdse_worker: each test carries its own tier, so the slow ones are not also medium. - check_skips: the POSIX-only worker tests' reason is expected on Windows. - test_tdse_native: the interaction formulation's transported startup row is each engine's own complex product, which MSVC fuses on Arm64; it is compared within 4 eps of each value, the exponential row still bit for bit. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HAYtF5P9QqEd3HQx3WJkA3 --- docs/dev/seps/sep-0049-tdse-on-native.md | 14 ++++++++----- native/ci/check_skips.py | 1 + tests/backends/test_tdse_native.py | 19 +++++++++++++++--- tests/driver/test_snapshots.py | 8 ++++++-- tests/driver/test_source_worker.py | 25 ++++++++++++++++++++++-- tests/driver/test_tdse_worker.py | 23 +++++++++++++++++----- 6 files changed, 73 insertions(+), 17 deletions(-) diff --git a/docs/dev/seps/sep-0049-tdse-on-native.md b/docs/dev/seps/sep-0049-tdse-on-native.md index 7d6d40fe8..090605e9e 100644 --- a/docs/dev/seps/sep-0049-tdse-on-native.md +++ b/docs/dev/seps/sep-0049-tdse-on-native.md @@ -1210,12 +1210,16 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. it either way. - §7.3 item 1 reads each engine's history row zero after the first attempt, not the stage output alone: native has no stage output outside a run. Without the - in-stage absorber the initial step is accepted on both, and the row, the startup - output carried to the step's end in the interaction formulation, is equal bit for - bit; with it, the collar rejects the initial step, the row is the startup output, - and it is within one transform's bound, peak-normalised. + in-stage absorber the initial step is accepted on both. In the exponential + formulation the row is the startup output, equal bit for bit. In the interaction + formulation it is that output carried to the step's end, by each engine's own + complex product, so it is within 4 eps of each value: bit for bit in CI's macOS, + Linux and Windows x86-64 lanes, one ulp apart on Windows on Arm. + With the absorber, the collar rejects the initial step, the row is the startup + output, and it is within one transform's bound, peak-normalised. - Item 4 is two runs: eight steps of 2^-27 m without the in-stage absorber (cap and - length binary-exact, so no final step is split), within the eight-step 128 eps; and, with the absorber, whose collar holds the steps to about a + length binary-exact, so no final step is split), within the eight-step 128 eps; + and, with the absorber, whose collar holds the steps to about a two-hundredth of the run, 0.2 um in some four hundred attempts within the attempts times the one-step 16 eps. Each checks that the source moves its field a millionfold beyond the budget. diff --git a/native/ci/check_skips.py b/native/ci/check_skips.py index ce500f762..aa62aa944 100644 --- a/native/ci/check_skips.py +++ b/native/ci/check_skips.py @@ -29,6 +29,7 @@ r"the Slurm \(and its Queue\) executor is POSIX only and refuses Windows", r"the Queue and Slurm executors are POSIX only", r"POSIX process signal delivery", + r"POSIX signals and groups", r"POSIX SIGKILL", r"POSIX shared locks", r"Windows has no SIGKILL", diff --git a/tests/backends/test_tdse_native.py b/tests/backends/test_tdse_native.py index a8fb488b6..aaadfa35c 100644 --- a/tests/backends/test_tdse_native.py +++ b/tests/backends/test_tdse_native.py @@ -397,8 +397,10 @@ def test_the_startup_evaluation_is_the_reference_s( formulation, kerr_model, noise, monkeypatch ): """Item 1: the field the engine gathers and ``F`` bit for bit, through the C++ - gather and the binding; the history's first row after the first attempt bit for - bit without the in-stage absorber, within one transform's bound with it.""" + gather and the binding; the history's first row after the first attempt, without + the in-stage absorber, bit for bit where it is the startup output itself and + within one complex product where the interaction formulation transported it; with + the absorber, within one transform's bound.""" for absorber in (False, True): changes: dict[str, Any] = {} if absorber else {"absorber": None} case = _case(formulation, kerr_model, noise=noise, **changes) @@ -418,7 +420,18 @@ def test_the_startup_evaluation_is_the_reference_s( assert ours["position"] == expected["position"] > 0 if formulation == "exponential": np.testing.assert_array_equal(expected["after"], expected["row"]) - np.testing.assert_array_equal(ours["row"], expected["after"]) + np.testing.assert_array_equal(ours["row"], expected["after"]) + else: + # The transport is each engine's own complex product, which a + # compiler may fuse (MSVC on Arm64 does): within its rounding, a few + # eps of each value, not bits. + assert_within( + ours["row"], + expected["after"], + 4 * EPS * np.abs(expected["after"]), + elementwise=True, + what="transported startup row", + ) else: # The collar rejects the initial step: row zero is the startup output. assert ours["position"] == case[0].initial.z diff --git a/tests/driver/test_snapshots.py b/tests/driver/test_snapshots.py index 58f3f469b..99b7d511c 100644 --- a/tests/driver/test_snapshots.py +++ b/tests/driver/test_snapshots.py @@ -124,8 +124,11 @@ def watched(*args): id(backend), id(backend._adams), id(backend._tdse_binding), - id(binding._hacc.model), - tuple(id(plan) for plan in binding._plans.values()), + # The model and its plans live behind the provider's worker (SEP 49): + # the same worker, the same view of the model and the same plans. + id(binding._worker), + id(binding._hacc), + tuple(sorted(binding._plan_records)), ) ) before = (len(fake.calls), len(fake.prepared), len(fake.plan_arguments)) @@ -153,6 +156,7 @@ def watched(*args): assert controller.last_result.bytes_copied > 0 assert len(built) == 3 and all(item == built[0] for item in built) assert len(live) == 1 and all(item == identities[0] for item in identities) + assert identities[0][-1] # the plans were built before the snapshots assert len(fake.prepared) == 1 assert output.outcome is not None assert_no_extra_call(output.outcome) diff --git a/tests/driver/test_source_worker.py b/tests/driver/test_source_worker.py index 1fa0361ea..7820359c3 100644 --- a/tests/driver/test_source_worker.py +++ b/tests/driver/test_source_worker.py @@ -42,8 +42,6 @@ ) from solprop.exceptions import RunTerminated, StopRequested -pytestmark = pytest.mark.medium - # How long a cancellation may take before the test calls it broken: the poll interval # and a kill, with room for a loaded machine. DEADLINE = 10.0 @@ -78,6 +76,7 @@ class _Pair: right: object +@pytest.mark.medium def test_the_codec_round_trips_values_and_records_without_pickle(): value = { "arrays": [ @@ -108,6 +107,7 @@ def test_the_codec_round_trips_values_and_records_without_pickle(): assert back["opaque"].type_name == "builtins.object" +@pytest.mark.medium def test_a_record_s_computed_values_read_as_attributes_but_are_not_fields(): tree, arrays = encode(_Pair(1, 2), lambda value: {"ok": True}) record: Any = decode(tree, arrays) @@ -117,6 +117,7 @@ def test_a_record_s_computed_values_read_as_attributes_but_are_not_fields(): record.missing # noqa: B018 - the attribute access is the test +@pytest.mark.medium def test_the_worker_environment_drops_native_s_own_wait_settings_only(): native = { "SOLPROP_NATIVE_WAIT": "spin", @@ -138,6 +139,7 @@ def test_the_worker_environment_drops_native_s_own_wait_settings_only(): assert kept["GOMP_SPINCOUNT"] == "5" and "KMP_BLOCKTIME" not in kept +@pytest.mark.medium def test_the_native_wait_settings_are_solprop_native_s_own(monkeypatch): """The worker removes exactly what solprop-native sets (ADR-071).""" spec = importlib.util.find_spec("solprop_native") @@ -155,11 +157,13 @@ def test_the_native_wait_settings_are_solprop_native_s_own(monkeypatch): assert _source_worker._NATIVE_WAIT_SETTINGS == module._SETTINGS +@pytest.mark.medium def test_the_worker_environment_keeps_blas_idle(): environment = worker_environment({}, {}) assert "OPENBLAS_THREAD_TIMEOUT" in environment +@pytest.mark.medium def test_values_cross_a_real_worker_bit_for_bit(worker): rng = np.random.default_rng(49) sent = {"field": rng.standard_normal((3, 257)), "labels": (1, 3), "z": 0.25} @@ -171,11 +175,13 @@ def test_values_cross_a_real_worker_bit_for_bit(worker): assert worker.pid == worker.identity["pid"] != os.getpid() +@pytest.mark.medium def test_the_provider_s_printing_never_corrupts_a_frame(worker): assert worker.request("print") == "printed" assert worker.request("echo", [1, 2]) == [1, 2] +@pytest.mark.medium def test_a_provider_exception_is_reported_by_name_and_ends_the_worker(worker): with pytest.raises(RemoteError) as raised: worker.request("raise") @@ -188,6 +194,7 @@ def test_a_provider_exception_is_reported_by_name_and_ends_the_worker(worker): _assert_ended(worker) +@pytest.mark.medium def test_a_worker_that_exits_is_a_named_failure_with_its_status_and_stderr(worker): with pytest.raises(WorkerError, match="exit status 7") as raised: worker.request("exit") @@ -204,12 +211,14 @@ def test_a_worker_that_exits_is_a_named_failure_with_its_status_and_stderr(worke ("malformed", "malformed worker message"), ], ) +@pytest.mark.medium def test_a_closed_pipe_or_a_malformed_frame_ends_the_worker(worker, kind, message): with pytest.raises(WorkerError, match=message): worker.request(kind) _assert_ended(worker) +@pytest.mark.medium def test_an_oversized_frame_is_refused_before_it_is_read(): worker = _worker(limit=4096) with pytest.raises(WorkerError, match="over its bound of 4096"): @@ -217,6 +226,7 @@ def test_an_oversized_frame_is_refused_before_it_is_read(): _assert_ended(worker) +@pytest.mark.medium def test_a_failure_while_starting_is_reported_by_name(): worker = Worker( "source_adapters:failing_adapter", @@ -227,6 +237,7 @@ def test_a_failure_while_starting_is_reported_by_name(): _assert_ended(worker) +@pytest.mark.medium def test_a_protocol_mismatch_is_refused_before_any_request(monkeypatch): monkeypatch.setattr(_source_worker, "PROTOCOL_VERSION", 99) worker = Worker("source_adapters", environment=worker_environment(os.environ, {})) @@ -240,6 +251,7 @@ def test_a_protocol_mismatch_is_refused_before_any_request(monkeypatch): ['{"$d": []}', "[" * 100_000 + "]" * 100_000], ids=["bad tag", "too deep"], ) +@pytest.mark.medium def test_a_message_this_side_cannot_decode_is_a_protocol_error(body): control = f'{{"kind": "reply", "body": {body}, "arrays": []}}'.encode() stream = io.BytesIO(_source_worker._HEADER.pack(len(control)) + control) @@ -247,6 +259,7 @@ def test_a_message_this_side_cannot_decode_is_a_protocol_error(body): _source_worker.read_message(stream.read, len(control)) +@pytest.mark.medium @POSIX def test_a_worker_that_died_between_calls_is_a_named_failure(worker): assert worker._process is not None @@ -257,6 +270,7 @@ def test_a_worker_that_died_between_calls_is_a_named_failure(worker): _assert_ended(worker) +@pytest.mark.medium def test_an_interruption_during_close_still_ends_the_worker(worker, monkeypatch): process = worker._process assert process is not None @@ -273,6 +287,7 @@ def interrupted(timeout=None): assert process.returncode is not None and not worker.alive +@pytest.mark.medium def test_a_worker_its_owner_forgets_is_ended_when_collected(): worker = _worker() process = worker._process @@ -282,6 +297,7 @@ def test_a_worker_its_owner_forgets_is_ended_when_collected(): assert process.returncode is not None +@pytest.mark.medium def test_close_between_calls_ends_the_worker_in_order(worker): worker.close() _assert_ended(worker) @@ -297,6 +313,7 @@ def _timed(worker, kind, expected): return elapsed +@pytest.mark.medium def test_a_stop_flag_cancels_a_hanging_call_at_the_next_poll(worker): flag = threading.Event() worker.stop = flag @@ -309,6 +326,7 @@ def test_a_stop_flag_cancels_a_hanging_call_at_the_next_poll(worker): ("number", "expected"), [(signal.SIGINT, KeyboardInterrupt), (signal.SIGTERM, RunTerminated)], ) +@pytest.mark.medium def test_a_signal_cancels_a_hanging_call(worker, number, expected): defaults = { signal.SIGTERM: signal.SIG_DFL, @@ -321,6 +339,7 @@ def test_a_signal_cancels_a_hanging_call(worker, number, expected): assert _timed(worker, "hang", expected) < DEADLINE +@pytest.mark.medium @POSIX def test_a_signal_held_by_a_commit_is_taken_by_an_interruptible_wait(worker): """Native's ``advance`` waits inside ``committing``: the loop raises the signal.""" @@ -335,6 +354,7 @@ def test_a_signal_held_by_a_commit_is_taken_by_an_interruptible_wait(worker): _assert_ended(worker) +@pytest.mark.medium @POSIX def test_outside_an_interruptible_scope_the_commit_keeps_the_signal(worker): """The stop flag ends the wait; the signal stays held, dropped with the stop.""" @@ -350,6 +370,7 @@ def test_outside_an_interruptible_scope_the_commit_keeps_the_signal(worker): _assert_ended(worker) +@pytest.mark.medium def test_the_worker_imports_neither_backend_nor_the_parent_s_main(worker): loaded = worker.request("modules") names = loaded["modules"] diff --git a/tests/driver/test_tdse_worker.py b/tests/driver/test_tdse_worker.py index 302952a90..c6296354b 100644 --- a/tests/driver/test_tdse_worker.py +++ b/tests/driver/test_tdse_worker.py @@ -34,8 +34,6 @@ from solprop.nonlinear.tdse import TDSEExecution from solprop.output import open_file, resume -pytestmark = pytest.mark.medium - DEADLINE = 10.0 """A test-harness bound on a cancellation; the production code has none.""" @@ -86,9 +84,20 @@ def _plain(record): return json.loads(json.dumps(record, allow_nan=False)) +@pytest.mark.medium def test_evaluations_through_the_worker_equal_the_provider_called_directly(workers): descriptor = _descriptor(preset_basis(1)) - through, worker_binding = _evaluate_once(descriptor) + provider_modules = ("pyroraptor", "source_adapters") + with pytest.MonkeyPatch.context() as isolated: + # Another test of this process may have loaded a provider module (a fake + # one, in-process): set them aside, so that what follows shows what this + # run imports, whatever ran before it. + for name in [n for n in sys.modules if n.split(".")[0] in provider_modules]: + isolated.delitem(sys.modules, name) + through, worker_binding = _evaluate_once(descriptor) + # The provider lived in the worker alone. + loaded = [n for n in sys.modules if n.split(".")[0] in provider_modules] + assert not loaded with pytest.MonkeyPatch.context() as patch: fake = source_channel.install(patch, FakeHaccProvider()) fake.response = alpha_law(1.02, lambda5=2.0**-200) @@ -99,11 +108,10 @@ def test_evaluations_through_the_worker_equal_the_provider_called_directly(worke assert through.outcomes_json == direct.outcomes_json assert worker_binding.ledger == direct_binding.ledger assert _plain(worker_binding.metadata()) == _plain(direct_binding.metadata()) - # The provider lived in the worker alone. - assert "pyroraptor" not in sys.modules and "source_adapters" not in sys.modules assert len(workers) == 1 and not workers[0].alive +@pytest.mark.medium def test_the_handshake_refuses_another_execution_space_by_name(monkeypatch, workers): monkeypatch.setattr(tdse, "_ADAPTER", "source_adapters:cuda_tdse") with pytest.raises(BackendError, match="native build uses"): @@ -111,6 +119,7 @@ def test_the_handshake_refuses_another_execution_space_by_name(monkeypatch, work assert len(workers) == 1 and not workers[0].alive +@pytest.mark.medium def test_the_handshake_records_another_provider_revision(monkeypatch, workers): monkeypatch.setattr(tdse, "_ADAPTER", "source_adapters:stale_tdse") binding = _bind(_descriptor(preset_basis(1))) @@ -121,6 +130,7 @@ def test_the_handshake_records_another_provider_revision(monkeypatch, workers): binding.close() +@pytest.mark.medium def test_a_failure_in_preparation_is_named_and_ends_the_worker(workers): with pytest.raises(BackendError, match="no complement option") as raised: _bind( @@ -131,6 +141,7 @@ def test_a_failure_in_preparation_is_named_and_ends_the_worker(workers): assert len(workers) == 1 and not workers[0].alive +@pytest.mark.medium def test_the_transport_copies_are_in_the_ledger_and_admitted_before_start(workers): descriptor = _descriptor(preset_basis(1)) execution = TDSEExecution(max_batch_rows=2) @@ -150,6 +161,7 @@ def test_the_transport_copies_are_in_the_ledger_and_admitted_before_start(worker assert len(workers) == 1 +@pytest.mark.medium def test_a_stop_during_an_atomic_call_kills_the_worker_and_keeps_the_charge( monkeypatch, workers ): @@ -166,6 +178,7 @@ def test_a_stop_during_an_atomic_call_kills_the_worker_and_keeps_the_charge( binding.close() +@pytest.mark.medium def test_after_a_provider_exception_the_retry_gets_a_fresh_worker( monkeypatch, workers, tmp_path ): From 55abe4e0dd10fdad206c46e76f7c1f2323fb881b Mon Sep 17 00:00:00 2001 From: John Travers Date: Fri, 13 Nov 2026 00:37:54 +0000 Subject: [PATCH 18/20] SEP 49: cover the worker's own side and the binding's failure paths The coverage lane missed the source worker's child side, which runs in a process of its own, and several new failure paths of the TDSE binding. Tests: - The worker's main, its private streams and the module's entry block run in this process on pipes standing in for descriptors 0 and 1, with os._exit recorded: requests in order until close, an adapter exception and a reply the codec refuses reported as errors, provider printing kept out of the frames, SIGINT and SIGTERM ignored, a failed start reported in place of the handshake, and the watchdog's exit at end of file or on garbage. - Real workers: an answer before the handshake, a message out of turn, a close that overruns its grace, a grandchild holding standard error across a kill, an unstarted worker's close and kill. - Frames: truncated, oversized and mistyped messages; the reply reader's endings; numpy scalars, callables and re-sent records in the codec. - Binding: a provider that cannot start, a worker that dies in a call (a charged "worker" failure with its stderr attached), a stop during the consistency diagnostic's call, a missing result record, a refused shadow component, a fresh worker reporting another build, and the wrapping of worker and local preparation errors. - Adapter: requests outside its protocol, and a result info its model comparison cannot read. Code: - A result that is not a record is refused by name, as a contract failure, instead of failing on a missing attribute as "unexpected". - A worker's failure waits for its standard-error reader once it has exited, so the last lines it wrote are on the error (they could be lost to a race before). - The diagnostic's close is always sent, suppressed when the worker is already ended; a killed worker refuses it at once, so the "alive" check only added a branch no run could take. Removed as unreachable: - tdse._json_default's dataclass branch: provider records now cross the worker as Records, and nothing else the binding encodes is a dataclass. - tdse._component's "order must be one of" refusal: the worker evaluates every quantity at each order the shadow reports, and the binding asks only for orders it has checked against those labels. - tdse_source._cubic_polarisation's checks: PropagationProblem.validate, which both backends run first, admits optmat_cubic only with its real Kerr coefficient and no plasma, Raman or dispersive Kerr beside it. - Worker._exited's and _end's None checks: only a started worker, which always has its process and all three pipes, reaches them. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HAYtF5P9QqEd3HQx3WJkA3 --- src/solprop/backends/tdse_source.py | 15 +- src/solprop/driver/_source_worker.py | 23 +- src/solprop/driver/tdse.py | 31 +- tests/driver/test_source_worker.py | 393 ++++++++++++++++++++++++- tests/driver/test_tdse_binding.py | 92 ++++++ tests/driver/test_tdse_hacc_binding.py | 22 ++ tests/driver/test_tdse_worker.py | 48 ++- tests/hacc_cases.py | 5 + tests/source_adapters.py | 70 ++++- 9 files changed, 658 insertions(+), 41 deletions(-) diff --git a/src/solprop/backends/tdse_source.py b/src/solprop/backends/tdse_source.py index 075db7a45..234748c1c 100644 --- a/src/solprop/backends/tdse_source.py +++ b/src/solprop/backends/tdse_source.py @@ -32,7 +32,6 @@ from solprop.exceptions import BackendError from solprop.translate.linop import power_weight_at from solprop.translate.problem import PropagationProblem -from solprop.translate.responses import ResponseStage from solprop.translate.routes import tdse_descriptor, tdse_of from solprop.translate.stage import StageScalars from solprop.translate.tdse import TDSEResult, lattice_times, waveform_identity @@ -138,19 +137,11 @@ def _cubic_polarisation( ``P = eps0 chi3 rho E^3 w(t)`` in C/m², with the temporal collar ``w``: the real cubic of a field grid, the only term the TDSE route admits beside the atom. """ - stages = set(problem.responses.response_order) - {ResponseStage.ACCUMULATE} + # A validated problem (``PropagationProblem.validate``) holds this real cubic for + # ``optmat_cubic``, and no plasma, Raman or dispersive Kerr term beside it. kerr = problem.responses.kerr_coefficients + assert kerr is not None values = fields[:, None, :] - if not stages: - # Nothing but the accumulation compiled: an identically zero polarisation. - return np.zeros_like(values) - if ( - kerr is None - or kerr.cubic_form != "real" - or kerr.analytic_signal - or stages != {ResponseStage.KERR} - ): - raise BackendError("the cubic replacement needs the real Kerr term alone") polarisation = kerr.coefficient * bundle.kerr_density * values**2 * values return polarisation * problem.grids.t_window_fine diff --git a/src/solprop/driver/_source_worker.py b/src/solprop/driver/_source_worker.py index ae5915004..0ee8d9553 100644 --- a/src/solprop/driver/_source_worker.py +++ b/src/solprop/driver/_source_worker.py @@ -581,14 +581,18 @@ def _wait(self) -> tuple[str, object]: def _exited(self) -> WorkerError: """Return the failure of a worker whose pipe ended: its status, its stderr.""" + # Only a started worker is waited on, so its process and pipes exist. process = self._process + assert process is not None and process.stderr is not None self._usable = False - status: int | None = None - if process is not None: - try: - status = process.wait(timeout=CLOSE_GRACE) - except subprocess.TimeoutExpired: - status = None + try: + status = process.wait(timeout=CLOSE_GRACE) + except subprocess.TimeoutExpired: + status = None + if status is not None: + # Its last words may still be in the pipe, which ends now that it has + # exited (unless a grandchild holds it open: the grace bounds that). + self._threads[process.stderr].join(timeout=CLOSE_GRACE) tail = "\n".join(self._tail) error = WorkerError( f"the source worker ended (exit status {status}) without replying" @@ -614,9 +618,10 @@ def _end( # Its own session: the group holds anything the provider started. os.killpg(process.pid, signal.SIGKILL) process.wait() - if process.stdin is not None: - with contextlib.suppress(OSError): - process.stdin.close() + # Every worker is started with all three pipes. + assert process.stdin is not None + with contextlib.suppress(OSError): + process.stdin.close() # The worker's ends are closed now, so its readers see the end of their pipes. for stream, thread in threads.items(): thread.join(timeout=CLOSE_GRACE) diff --git a/src/solprop/driver/tdse.py b/src/solprop/driver/tdse.py index fe16a72af..3148bf4c9 100644 --- a/src/solprop/driver/tdse.py +++ b/src/solprop/driver/tdse.py @@ -27,7 +27,7 @@ import warnings from collections.abc import Mapping from contextlib import ExitStack -from dataclasses import asdict, dataclass, is_dataclass +from dataclasses import asdict, dataclass from typing import Any, Literal, Protocol, cast, runtime_checkable import numpy as np @@ -610,9 +610,7 @@ def _estimated_memory( def _json_default(value: object) -> object: - """Convert a public native record or array into JSON-compatible containers.""" - if is_dataclass(value) and not isinstance(value, type): - return asdict(value) + """Convert a provider record or array into JSON-compatible containers.""" if isinstance(value, Record): # A provider record from the worker: its fields, as the dataclass gave them. return dict(vars(value)) @@ -943,10 +941,12 @@ def _returned(reply: object) -> _Native: """ answer = cast("Mapping[str, Any]", reply) raw = answer["raw"] - if isinstance(raw, Record): - raw.computed["provider_type"] = raw.record_type.rpartition(".")[2] - raw.record_type = answer["kind"] - raw.computed["model_match"] = answer["model_match"] + if not isinstance(raw, Record): + # Nothing, or an object without fields, which the worker sends as its repr. + raise BackendError(f"the TDSE provider returned no result record: {raw!r}") + raw.computed["provider_type"] = raw.record_type.rpartition(".")[2] + raw.record_type = answer["kind"] + raw.computed["model_match"] = answer["model_match"] return raw @@ -971,10 +971,9 @@ def _component(shadow: _Native, quantity: str, order: int) -> np.ndarray: The worker evaluated the provider's method (``_pyroraptor``); a component it refused raises here as the provider raised it. """ - components = cast("Mapping[str, Any]", shadow.components) - found = components.get(f"{quantity}:{order}") - if found is None: - raise ValueError(f"order must be one of {shadow.orders}") + # The worker evaluated every quantity at each order the shadow reports, and the + # binding asks only for orders it has checked against those labels. + found = cast("Mapping[str, Any]", shadow.components)[f"{quantity}:{order}"] if isinstance(found, Mapping): refusal = cast("Mapping[str, str]", found) if refusal["type"] == "ValueError": @@ -2139,10 +2138,10 @@ def _diagnostic( finally: budget.release_reservation() finally: - # A failed or cancelled call has ended the worker already; nothing to close. - if worker.alive: - with contextlib.suppress(RemoteError, WorkerError): - worker.request("diagnostic_close", {}) + # A failed or cancelled call has ended the worker already, which then refuses + # the close at once as not running: nothing is left to close. + with contextlib.suppress(RemoteError, WorkerError): + worker.request("diagnostic_close", {}) return ( np.array(total[0], dtype=np.float64, copy=True), np.array(first[0], dtype=np.float64, copy=True), diff --git a/tests/driver/test_source_worker.py b/tests/driver/test_source_worker.py index 7820359c3..d268a3869 100644 --- a/tests/driver/test_source_worker.py +++ b/tests/driver/test_source_worker.py @@ -16,13 +16,16 @@ import io import json import os +import queue +import runpy import signal import subprocess import sys import threading import time +import warnings from pathlib import Path -from typing import Any +from typing import Any, cast import numpy as np import psutil @@ -31,7 +34,9 @@ from solprop._interrupts import committing, handled, interruptible from solprop.driver import _source_worker from solprop.driver._source_worker import ( + CONTROL_LIMIT, POLL_INTERVAL, + Opaque, Record, RemoteError, Worker, @@ -107,6 +112,26 @@ def test_the_codec_round_trips_values_and_records_without_pickle(): assert back["opaque"].type_name == "builtins.object" +@pytest.mark.medium +def test_the_codec_carries_numpy_scalars_and_sends_callables_as_opaque(): + tree, arrays = encode({"step": np.int32(3), "z": np.float64(0.5), "f": len}) + back: Any = decode(json.loads(json.dumps(tree)), arrays) + assert back["step"] == 3 and type(back["step"]) is int + assert back["z"] == 0.5 and type(back["z"]) is float + assert back["f"] == Opaque("builtins.builtin_function_or_method", repr(len)) + + +@pytest.mark.medium +def test_a_received_record_re_encodes_with_its_type_and_computed_values(): + """A record sent on keeps its type; the sender's computed values win.""" + tree, arrays = encode(_Pair(1, 2), lambda value: {"ok": True, "kept": 1}) + received: Any = decode(tree, arrays) + again: Any = decode(*encode(received, lambda value: {"ok": False})) + assert again.record_type == received.record_type + assert vars(again) == {"left": 1, "right": 2} + assert again.computed == {"ok": False, "kept": 1} + + @pytest.mark.medium def test_a_record_s_computed_values_read_as_attributes_but_are_not_fields(): tree, arrays = encode(_Pair(1, 2), lambda value: {"ok": True}) @@ -259,6 +284,103 @@ def test_a_message_this_side_cannot_decode_is_a_protocol_error(body): _source_worker.read_message(stream.read, len(control)) +def _frame(payload): + return _source_worker._HEADER.pack(len(payload)) + payload + + +def _control(*arrays, kind: object = "reply"): + message = {"kind": kind, "body": None, "arrays": list(arrays)} + return _frame(json.dumps(message).encode()) + + +_LIMIT = 4096 +_HEADER = _source_worker._HEADER +_FOUR = {"dtype": " Date: Fri, 13 Nov 2026 00:40:35 +0000 Subject: [PATCH 19/20] SEP 49: Final The A/B on non-TDSE cases shows no regression (geometric mean 0.986 against the limit 1.015), and the bitwise check is unchanged on all seven cases. CI's coverage lane now covers the worker's own side. Archive the SEP, move its index and toctree entries, drop its roadmap entry, link it from the TDSE manual page, and record the completion and the issues the open questions moved to (#785, #777). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HAYtF5P9QqEd3HQx3WJkA3 --- docs/dev/architecture.md | 2 +- docs/dev/decisions.md | 4 +- docs/dev/dependencies.md | 2 +- docs/dev/kernel-contract.md | 2 +- docs/dev/roadmap.md | 7 -- docs/dev/seps/archive/index.md | 1 + .../{ => archive}/sep-0049-tdse-on-native.md | 84 ++++++++++++++----- docs/dev/seps/index.md | 3 +- docs/dev/testing.md | 2 +- docs/guide/media/tdse.md | 3 +- 10 files changed, 72 insertions(+), 38 deletions(-) rename docs/dev/seps/{ => archive}/sep-0049-tdse-on-native.md (96%) diff --git a/docs/dev/architecture.md b/docs/dev/architecture.md index cc9dd740b..73fed6ebb 100644 --- a/docs/dev/architecture.md +++ b/docs/dev/architecture.md @@ -122,7 +122,7 @@ belong to the driver/binding, which runs the provider in a worker process of its provider's adapter): framed pipes, no pickle, cancellation while waiting; modal synthesis to the nodes, the assembly of the macroscopic sources and their projection belong to `backends/tdse_source.py`, which imports neither backend and which a backend calls from its source evaluation -([SEP 49](seps/sep-0049-tdse-on-native.md)). See the [TDSE manual](../guide/media/tdse.md). +([SEP 49](seps/archive/sep-0049-tdse-on-native.md)). See the [TDSE manual](../guide/media/tdse.md). Geometry: mode-averaged propagation is the single-node case of multimode propagation (the effective-area normalisation replaces the quadrature); the GNLSE is the mode-averaged diff --git a/docs/dev/decisions.md b/docs/dev/decisions.md index 954f0f06e..dfa0361e0 100644 --- a/docs/dev/decisions.md +++ b/docs/dev/decisions.md @@ -2280,7 +2280,7 @@ Withdrawn (development-process record, not a design decision). ## ADR-078 — The TDSE response runs on native through a host source; its provider in a worker - **Status:** accepted (maintainer), 2026-11-12, with - [SEP 49](seps/sep-0049-tdse-on-native.md) (PR #771). Amends ADR-077's "every feature + [SEP 49](seps/archive/sep-0049-tdse-on-native.md) (PR #771). Amends ADR-077's "every feature except the TDSE response, for now": native implements every feature. - **Context:** the TDSE response ran on `numpy_ref` alone, and native refused it by name. Its source is a complete atomic calculation at every radial node, from 93 s to over @@ -2333,5 +2333,5 @@ Withdrawn (development-process record, not a design decision). fastest recorded atomic call. Native runs without a host source are unchanged bit for bit. The hook's two rows are listed as `host_source` in the memory ledger. CI tests the hook and the worker with manufactured providers; compatibility with an installed - Pyroraptor is checked by hand ([SEP 49](seps/sep-0049-tdse-on-native.md), section 7.4), + Pyroraptor is checked by hand ([SEP 49](seps/archive/sep-0049-tdse-on-native.md), section 7.4), and the worker's handshake refuses a mismatched protocol or execution space. diff --git a/docs/dev/dependencies.md b/docs/dev/dependencies.md index 1fa08044f..c921ae2af 100644 --- a/docs/dev/dependencies.md +++ b/docs/dev/dependencies.md @@ -42,7 +42,7 @@ missing capability is refused with a `BackendError`. solprop pins no provider revision: until Pyroraptor has a release, it uses whatever provider is installed, by default the latest `main` -([SEP 49](seps/sep-0049-tdse-on-native.md), ADR-078). The provider runs in a worker +([SEP 49](seps/archive/sep-0049-tdse-on-native.md), ADR-078). The provider runs in a worker process, whose handshake refuses another protocol version; the revision, build and execution space it reports are recorded with every run, and a preset's source names that revision. A device build (CUDA or HIP) is selected explicitly through diff --git a/docs/dev/kernel-contract.md b/docs/dev/kernel-contract.md index c20f36d2d..e8a98f815 100644 --- a/docs/dev/kernel-contract.md +++ b/docs/dev/kernel-contract.md @@ -1031,7 +1031,7 @@ it changes no step, charge or reservation. None of this is part of the descripto or the kernel problem, which compiled backends see, so it is outside the kernel schema. -**The native host source hook** ([SEP 49](seps/sep-0049-tdse-on-native.md), +**The native host source hook** ([SEP 49](seps/archive/sep-0049-tdse-on-native.md), ADR-078). On the native CPU build the binding reaches the resident Adams march as a host source: a session binding outside the kernel problem, whose three entry points the march calls from the thread that called `advance`, outside every parallel region, holding the diff --git a/docs/dev/roadmap.md b/docs/dev/roadmap.md index 0e9a2ff4e..0e5065bf3 100644 --- a/docs/dev/roadmap.md +++ b/docs/dev/roadmap.md @@ -35,13 +35,6 @@ being implemented. routes, bound response and other geometries remain follow-up work; the first route is a single-mode capillary with node quadrature. -- [SEP 49 — The TDSE response on native](seps/sep-0049-tdse-on-native.md): - the TDSE response on the native CPU build through a host source hook, the one Python - call admitted inside a native trial (device builds refuse it), with the host source - assembly shared by both backends and the atomic provider in a worker process of its - own, which `numpy_ref`'s TDSE runs use too. `numpy_ref` stays a complete backend. - TDSE physics, a compiled TDSE source and device TDSE runs remain out of scope. - ## Draft enhancement proposals SEPs under discussion; their scope may change and implementation has not been accepted. diff --git a/docs/dev/seps/archive/index.md b/docs/dev/seps/archive/index.md index 69fe3ab95..7dd2e80a5 100644 --- a/docs/dev/seps/archive/index.md +++ b/docs/dev/seps/archive/index.md @@ -45,4 +45,5 @@ sep-0045-native-rk-march sep-0046-machine-calibration sep-0047-run-ownership sep-0048-julia-backend +sep-0049-tdse-on-native ``` diff --git a/docs/dev/seps/sep-0049-tdse-on-native.md b/docs/dev/seps/archive/sep-0049-tdse-on-native.md similarity index 96% rename from docs/dev/seps/sep-0049-tdse-on-native.md rename to docs/dev/seps/archive/sep-0049-tdse-on-native.md index 090605e9e..879e482e5 100644 --- a/docs/dev/seps/sep-0049-tdse-on-native.md +++ b/docs/dev/seps/archive/sep-0049-tdse-on-native.md @@ -6,17 +6,17 @@ | **Title** | The TDSE response on native | | **Author** | John Travers | | **Drafting agent** | Opus 5.5 (2026-11-12) | -| **Status** | In progress | +| **Status** | Final | | **Review status** | Reviewed | | **Reviewers** | GPT-6 (AI); John Travers (human) | | **Type** | Change | | **Created** | 2026-11-12 | | **Resolution** | Accepted 2026-11-12 by John Travers, in [#765](https://github.com/LupoLab/solprop/pull/765) | -| **Requires** | [SEP 12](archive/sep-0012-tdse-nonlinear-response.md), [SEP 20](archive/sep-0020-hacc-helium-tdse.md) and [SEP 26](archive/sep-0026-edge-free-tdse-source.md) (the TDSE response and its source); [SEP 39](archive/sep-0039-native-adams-march.md) (the native Adams march); [SEP 47](archive/sep-0047-run-ownership.md) (backend-owned sessions); [SEP 25](archive/sep-0025-rounding-level-reproducibility.md) (agreement to the error bound); ADR-077 and the change it calls for (§3.4) | +| **Requires** | [SEP 12](sep-0012-tdse-nonlinear-response.md), [SEP 20](sep-0020-hacc-helium-tdse.md) and [SEP 26](sep-0026-edge-free-tdse-source.md) (the TDSE response and its source); [SEP 39](sep-0039-native-adams-march.md) (the native Adams march); [SEP 47](sep-0047-run-ownership.md) (backend-owned sessions); [SEP 25](sep-0025-rounding-level-reproducibility.md) (agreement to the error bound); ADR-077 and the change it calls for (§3.4) | | **Supersedes** | — | | **Superseded-by** | — | -| **Implementation** | In progress: Opus 5.5 (AI), on `feature/sep-0049-tdse-on-native`; phases 0–4, with the manual check of §7.4 done | -| **Manual pages** | [TDSE response](../../guide/media/tdse.md), [Choosing a backend](../../guide/performance/backends.md) | +| **Implementation** | Opus 5.5 (AI) for John Travers, on `feature/sep-0049-tdse-on-native` ([#771](https://github.com/LupoLab/solprop/pull/771)); phases 0–4, started and complete 2026-11-12/13; the manual check of §7.4 done by hand on a CPU build and an NVIDIA A40 | +| **Manual pages** | [TDSE response](../../../guide/media/tdse.md), [Choosing a backend](../../../guide/performance/backends.md) | ## Part I — Proposal @@ -47,7 +47,7 @@ included. Slimming it to a test library, which this SEP first proposed, is defer (§10.1). Success means: a TDSE run on native agrees with the same run on `numpy_ref` within the -rounding budgets of [testing.md](../testing.md) §3 derived for its cases, with its atomic +rounding budgets of [testing.md](../../testing.md) §3 derived for its cases, with its atomic ledger, labels, reservations and archive records identical; the shared source meets independent analytic and manufactured expectations; the worker's contract holds in CI with a manufactured provider; and an explicit stop ends a run within the wait loop's @@ -115,9 +115,9 @@ native hook. Neither the hook nor the worker is specific to TDSE or Pyroraptor ( `numpy_ref` is a complete executor: its own Runge–Kutta, Adams and low-storage marches (`backends/host/`), sessions, group sessions, a lockstep batch and a registry row -([backends.md](../backends.md) §1–§2). Native runs every run through its native session +([backends.md](../../backends.md) §1–§2). Native runs every run through its native session (`solprop_native.session`), whose compiled marches advance to an event from one policy -record (SEP 47; [kernel-contract.md](../kernel-contract.md) §11–§12). Each `advance` call +record (SEP 47; [kernel-contract.md](../../kernel-contract.md) §11–§12). Each `advance` call returns after 256 attempts or 0.1 s of wall time; the session wraps it in `_interrupts.committing()`, so a SIGINT or SIGTERM arriving during the call is held and raised when the call returns. The run's stop flag is polled before each native call, and @@ -125,7 +125,7 @@ a `StopToken` also forwards a stop to the native march, which reads it between a #### 3.2 The TDSE response today -The TDSE response ([TDSE response](../../guide/media/tdse.md); SEPs 12, 20 and 26) is +The TDSE response ([TDSE response](../../../guide/media/tdse.md); SEPs 12, 20 and 26) is admitted on one route only, by `PropagationProblem._check_tdse` and `translate.problem._check_tdse_assembly_route`: @@ -189,14 +189,14 @@ is below $10^{-3}$ of a call (§6.8). The native Adams march runs the trial in its resident engine and evaluates the source natively; its rules forbid "allocation, planning, logging, Python call or unplanned fence -in a trial step" ([backends.md](../backends.md) §3.2, `native/AGENTS.md`). A TDSE source +in a trial step" ([backends.md](../../backends.md) §3.2, `native/AGENTS.md`). A TDSE source cannot be evaluated without the provider, and its evaluation is a field-dependent, minutes-long call that cannot be precomputed. With a trial lasting minutes, each `advance` call returns after one attempt anyway. #### 3.4 Decisions and changes this SEP builds on -- **ADR-077** (two backends, binary64 only; [decisions.md](../decisions.md)), and the +- **ADR-077** (two backends, binary64 only; [decisions.md](../../decisions.md)), and the change it calls for, which must be merged before implementation starts: in particular native importing nothing from `numpy_ref`. ADR-077 records that `numpy_ref` stays a complete, selectable backend (amended with the maintainer's decision in this SEP's @@ -240,7 +240,7 @@ where $P$ and $S = J - j_1$ are the macroscopic polarisation and remainder curre $n_{\text{nodes}}$ radial nodes (density and the declared scales applied once), $Q$ the modal projection with the taper scale, $g$ and $g'$ the gate and its analytic derivative (SEP 26), $p$ the stage prefactor and $c$ the independent current factor -([kernel-contract.md](../kernel-contract.md) §6). Each backend then forms what it forms +([kernel-contract.md](../../kernel-contract.md) §6). Each backend then forms what it forms for every source: the stage output $w\,F$ with the spectral collar $w$, the in-stage absorber term, and in the interaction formulation the inverse propagator $e^{-\Delta\Phi}$. That is the composition of steps 1–5 in §3.2, split after step 4. @@ -253,7 +253,7 @@ the collar, a sum with the absorber term, a phase factor) and its optical march own, so a TDSE stage and a short TDSE run on native agree with `numpy_ref`'s within the one-stage and eight-step budgets `tests/bounds.py` derives for the existing source comparisons ($16\varepsilon$ and $128\varepsilon$ peak-normalised on those cases, -[accuracy.md](../accuracy.md) §7). These are budgets derived for these cases, checked on +[accuracy.md](../../accuracy.md) §7). These are budgets derived for these cases, checked on them, not guarantees for every TDSE problem or variable-step history. Where independently rounded norms fall on either side of a decision threshold, the two runs fork; such a fork is reported and the pair is then compared to an accuracy envelope (class T), not counted @@ -505,9 +505,9 @@ GUI job's stop) and the run's `StopFlag` (a stop button, a scan's supervisor). **Time.** One TDSE evaluation is a complete atomic calculation at every radial node: 93 s (production basis, 17 nodes) to 615 s (strong-field basis, 33 nodes) on compressed -fields in [SEP 12](archive/sep-0012-tdse-nonlinear-response.md)'s node selection +fields in [SEP 12](sep-0012-tdse-nonlinear-response.md)'s node selection (Part II), and 366 s (haCC(1)) and 1,307 s (haCC(5)) on one AMD MI325X -([SEP 20](archive/sep-0020-hacc-helium-tdse.md), Part II). Per evaluation the hook adds +([SEP 20](sep-0020-hacc-helium-tdse.md), Part II). Per evaluation the hook adds a GIL acquisition and a Python call (microseconds), the worker a copy each way of a few MB and a context switch (milliseconds), and the host source assembly, which `numpy_ref` already performs (of order 10–100 ms at the response lattices in use, to be measured in @@ -534,7 +534,7 @@ is removed but `numpy_ref/tdse.py`, which moves): The kernel problem does not change: the TDSE descriptor already crosses it, and the hook is a session binding outside it, as the binding is today. The hook's binding joins the -members `check_build` looks up by name, and [kernel-contract.md](../kernel-contract.md) +members `check_build` looks up by name, and [kernel-contract.md](../../kernel-contract.md) §6's "the native backend refuses TDSE by name and never holds a binding" is rewritten, with a release-note fragment. @@ -620,14 +620,14 @@ provider's revision is recorded with every run. ### 8. Documentation -- **Manual.** [TDSE response](../../guide/media/tdse.md): runs on native's CPU build and +- **Manual.** [TDSE response](../../../guide/media/tdse.md): runs on native's CPU build and on `numpy_ref`; the provider in its worker, its threads or device, and stopping a run; - [Choosing a backend](../../guide/performance/backends.md): TDSE leaves the list of + [Choosing a backend](../../../guide/performance/backends.md): TDSE leaves the list of features that run only on the reference. -- **Design pages.** [backends.md](../backends.md) §1 (the `auto` rule) and §3.2 (the - hook's exception); [kernel-contract.md](../kernel-contract.md) §6; - [architecture.md](../architecture.md) (the shared source module, the worker); - [testing.md](../testing.md) (the oracle boundary for shared code; worker-contract +- **Design pages.** [backends.md](../../backends.md) §1 (the `auto` rule) and §3.2 (the + hook's exception); [kernel-contract.md](../../kernel-contract.md) §6; + [architecture.md](../../architecture.md) (the shared source module, the worker); + [testing.md](../../testing.md) (the oracle boundary for shared code; worker-contract tests); `AGENTS.md` and `native/AGENTS.md` (the hook's exception). - **Records.** One ADR: the host source hook (the one Python call in a native trial, its scope, its refusal on device builds), the provider worker, and the failure boundary; @@ -1736,6 +1736,46 @@ cancellation of P3 to work in every case; he accepted P6–P10 the same day. - **Handoff.** Next are CI and an A/B; then `Final`. The GPU numerical comparison stays manual, for the provider's next production GPU use (Q10). +### Completion — 2026-11-13 + +- **Coverage.** CI's coverage lane found the worker's own side uncovered: it runs in a + subprocess, which the parent's coverage does not see. Tests now run it in-process + (two pipes standing in for its standard streams, `os._exit` recorded): requests served + in order until `close`, adapter and codec errors reported, the provider's prints kept + out of the frames, signals ignored, a failed start reported in place of the handshake, + and the watchdog's exit on end of file. Real-worker tests cover a reply before the + handshake, a message out of turn, a close past its grace (killed), and a worker dying + mid-call (a charged `[worker]` failure with its stderr). Four defensive checks that + validation already guarantees became assertions, and a provider reply that is not a + record is now refused by name as a contract failure. `Worker._exited` waits for the + stderr reader once the worker has exited, so the last lines are never lost. +- **Performance (A/B, non-TDSE).** Baseline `main` against this branch, on one NUMA + domain of a 128-core AMD EPYC 7763 node, production threads, 3 rounds of 3 repeats, + noise ε from A/A runs on the same machine. Every case did the same work: + + | Case | Ratio | Min | Max | ε | Field difference | + |---|---|---|---|---|---| + | `argon_blueshift_envelope` | 0.964 | 0.958 | 0.989 | 0.064 | 0 | + | `rdw_one_mode` | 1.016 | 0.996 | 1.037 | 0.039 | 5.5e-15 | + | `kerr_beam_192` | 1.003 | 0.996 | 1.013 | 0.007 | 1.4e-16 | + | `radial_filament_256_saved_hdf5` | 0.973 | 0.967 | 1.021 | 0.032 | 0 | + | `he_rdw_averaged` | 0.998 | 0.992 | 0.999 | 0.020 | 5.6e-15 | + | `h2_raman_averaged` | 0.983 | 0.979 | 0.997 | 0.017 | 4.8e-15 | + | `ctc_reduced` | 0.949 | 0.911 | 1.048 | 0.052 | 3.6e-15 | + | `kerr_beam_384` | 1.002 | 1.000 | 1.019 | 0.007 | 0 | + + Geometric mean 0.986, against the limit 1 + mean(ε)/2 = 1.015: no regression. With + estimated FFT plans and calibration off, all seven cases of the bitwise check gave a + field difference of exactly 0, so non-TDSE runs are unchanged bit for bit. The + differences above come from measured FFT plans. +- **Open questions moved to issues.** The numerical comparison of native against + `numpy_ref` with the provider on a production GPU, left manual by Q10: + [#785](https://github.com/LupoLab/solprop/issues/785). The strict-FP check on object + files rather than the stripped wheel: + [#777](https://github.com/LupoLab/solprop/issues/777). +- **Status.** `Final`, set in the last commit of #771; merging it completes the SEP and + closes [#764](https://github.com/LupoLab/solprop/issues/764). + ## Part III — History - 2026-11-12: Drafted. @@ -1888,7 +1928,7 @@ no physical constant or TDSE model was requalified by this review. [`test_a_full_adaptive_run_meets_the_reference_s_accuracy`](https://github.com/LupoLab/solprop/blob/63201dca4ab69d21b6d4ae0cd4611c4203ff47b1/tests/backends/test_backend_ladder.py) compares against a tighter reference integration and checks a declared work band. Replaying native's own schedule cannot replace that claim. The analytic P0–P2, - three-level P3 and cross-scheme checks in [testing.md](../testing.md) must remain + three-level P3 and cross-scheme checks in [testing.md](../../testing.md) must remain separately identified class-T or analytic checks, not become matched-replay tests. **Recommendation and question:** compare a third option: a small, non-production diff --git a/docs/dev/seps/index.md b/docs/dev/seps/index.md index 96eda3347..f74ac4c0b 100644 --- a/docs/dev/seps/index.md +++ b/docs/dev/seps/index.md @@ -33,7 +33,6 @@ Proposals that are active, in draft, accepted, in progress or deferred. | [11](sep-0011-finite-density-gas-index.md) | Finite-density gas index and mixture optics | Draft | Change | | [17](sep-0017-gas-two-photon-ionisation.md) | Broadband two-photon continuum response beside strong-field ionisation | Accepted | Feature | | [32](sep-0032-raman-maxwell-bloch.md) | Raman Maxwell–Bloch dynamics | Draft | Feature | -| [49](sep-0049-tdse-on-native.md) | The TDSE response on native | In progress | Change | SEP 13, the standalone numerical prerequisite for SEP 12, SEP 14, which carried it to the compiled backends, and SEP 16, which unified the multistep family and made it the @@ -85,6 +84,7 @@ Proposals that are final, rejected, withdrawn or superseded; the | [46](archive/sep-0046-machine-calibration.md) | Calibrating the native backend to its machine | Final | Change | | [47](archive/sep-0047-run-ownership.md) | Backend-owned execution sessions | Final | Change | | [48](archive/sep-0048-julia-backend.md) | A Julia backend: one engine for CPUs and GPUs, in both precisions | Withdrawn | Feature | +| [49](archive/sep-0049-tdse-on-native.md) | The TDSE response on native | Final | Change | ```{toctree} :hidden: @@ -101,7 +101,6 @@ sep-0010-low-rank-raman sep-0011-finite-density-gas-index sep-0017-gas-two-photon-ionisation sep-0032-raman-maxwell-bloch -sep-0049-tdse-on-native template archive/index ``` diff --git a/docs/dev/testing.md b/docs/dev/testing.md index 087a4ac3c..20f2e41be 100644 --- a/docs/dev/testing.md +++ b/docs/dev/testing.md @@ -180,7 +180,7 @@ worker's threads to the same first cores. Parity between the backends checks what each computes independently. It cannot check code both call: a defect in the TDSE response's source assembly -(`solprop.backends.tdse_source`, [SEP 49](seps/sep-0049-tdse-on-native.md)) is the same +(`solprop.backends.tdse_source`, [SEP 49](seps/archive/sep-0049-tdse-on-native.md)) is the same on both. That module's tests state their expectations independently of it, analytic or manufactured (`tests/backends/test_tdse_source.py`, `tests/edge_source_cases.py`), and a deliberate defect in each term is shown to fail one of them; parity then covers what each diff --git a/docs/guide/media/tdse.md b/docs/guide/media/tdse.md index 36f0ca8c5..20a49a17b 100644 --- a/docs/guide/media/tdse.md +++ b/docs/guide/media/tdse.md @@ -334,4 +334,5 @@ against an exact solution. See also §10.6 of the primer's [ionisation chapter](../../primer/ionisation.md), [radial nodes for expensive responses](../waveguides/radial-quadrature.md), and the API reference of {mod}`solprop.nonlinear.tdse`, {mod}`solprop.driver.tdse` and -{mod}`solprop.driver.tdse_consistency`. +{mod}`solprop.driver.tdse_consistency`. The native backend's TDSE response and the provider's +worker process are [SEP 49](../../dev/seps/archive/sep-0049-tdse-on-native.md). From 33a239485d650c3016eaec703c2d79ea335d29eb Mon Sep 17 00:00:00 2001 From: John Travers Date: Fri, 13 Nov 2026 01:23:45 +0000 Subject: [PATCH 20/20] Retrigger the documentation build (Read the Docs ended the last one for inactivity) Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HAYtF5P9QqEd3HQx3WJkA3