diff --git a/AGENTS.md b/AGENTS.md index 5c9ebe5aa..f70af8fae 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -187,7 +187,7 @@ to it; a function that departs from it says so in the first line of its docstrin | Linear operator (internal) | `D = −i(β − β₀ − β₁(ω − ω₀)) − α/2`, `Im n_eff > 0` is loss | | Amplitudes | interface modal amplitude `b = sqrt(Re n_eff) a` in true √W; the internal `a` is vacuum-normalised (`conventions.md §4`) | | Kernel state layout | `(n_modes, n_pol, n_omega)` complex128, ω contiguous, one trajectory (a group is a sequence of problems); saved fields `(n_z, n_modes, n_pol, n_omega)` | -| Axis names | `n_batch n_modes n_pol n_omega n_omega_fine n_t n_to n_nodes n_z n_intervals n_deg n_windows`, reused verbatim in annotations | +| Axis names | `n_modes n_pol n_omega n_omega_fine n_t n_to n_nodes n_z n_intervals n_deg n_windows`, reused verbatim in annotations | ## Backends, kernels and hot paths diff --git a/changelog.d/sep-0052-run-records.changed.md b/changelog.d/sep-0052-run-records.changed.md new file mode 100644 index 000000000..4b973164e --- /dev/null +++ b/changelog.d/sep-0052-run-records.changed.md @@ -0,0 +1 @@ +Both backends read each run's records -- the method's policy, the low-storage settings, the schedule, the plane plan and the boundary monitor's configuration and continuation -- formed once by the new `solprop.backends.records` (SEP 52). `RungeKuttaPolicy` moves there from `solprop.backends.rk_core` and carries the march's landings; `rk_policy` becomes `records.runge_kutta_policy`. The engine protocols the marches are to call are declared in the new `solprop.backends.engines`. diff --git a/changelog.d/sep-0052-snapshot-layout.breaking.md b/changelog.d/sep-0052-snapshot-layout.breaking.md new file mode 100644 index 000000000..2327d27f9 --- /dev/null +++ b/changelog.d/sep-0052-snapshot-layout.breaking.md @@ -0,0 +1,7 @@ +The output schema is 3 (SEP 52): a file's march snapshot (`/march_snapshot`) now stores +its `state` as `(n_modes, n_pol, n_omega)` and its engine `rows` as `(n_slots, n_modes, +n_pol, n_omega)`, without the leading axis of one they kept before. A snapshot written in +the earlier layout is refused by name rather than squeezed, and a file of output schema 2, +which 0.2.0 wrote, is refused as any file of another schema is: resume or continue it with +the solprop version that wrote it. Saved fields, statistics and the run record are +otherwise unchanged. diff --git a/changelog.d/sep-0052-unit-axis.changed.md b/changelog.d/sep-0052-unit-axis.changed.md new file mode 100644 index 000000000..0f88e5e4c --- /dev/null +++ b/changelog.d/sep-0052-unit-axis.changed.md @@ -0,0 +1,9 @@ +The arrays of one trajectory carry no leading axis of one anywhere (SEP 52): the +`PropagationProblem` operator's value tables (`LinearOperator.phi_coefficients` is +`(n_intervals, n_deg, n_modes, n_pol, n_omega)`, `phi_constant` and an expansion's +`terms` lose the same axis), the stage terms (`StageTerms.phi`), the separable operator's +`value_shape`, the boundary monitor's and the split collar's readings +(`BoundaryMonitor.power`, `SplitCrossing.occupancy`), the design check's launch, the +carrier frame's chord (a scalar per position) and both backends' private state. No field +is added or removed and no value changes; code that indexed `[0]` into these arrays drops +the index. diff --git a/docs/conf.py b/docs/conf.py index 68cc60736..58c533faf 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -166,6 +166,9 @@ ("py:class", "solprop.transverse_monitor._EdgeTables"), ("py:class", "solprop.transverse_monitor._HostEdgeReduction"), ("py:class", "Helpers"), + # The transverse edge's verdict literal, named in a signature as the run records + # import it for type checking only (solprop.backends.records). + ("py:class", "TransverseStatus"), ] # Array annotations are written with jaxtyping on numpy arrays (ADR-018). Neither # jaxtyping nor numpy's private typing aliases publish an inventory Sphinx can resolve, diff --git a/docs/dev/architecture.md b/docs/dev/architecture.md index 1702a4ea0..43f17a9d8 100644 --- a/docs/dev/architecture.md +++ b/docs/dev/architecture.md @@ -218,7 +218,7 @@ against `src/solprop` by `tests/test_structure_figures.py`. | `modes/` | Mode protocols and derived quantities; capillary, antiresonant, tabulated, step-index, rectangular and planar (slab, uniform in y, per unit width) modes, with `PlanarGrid` for the planar guide resolved in y; shared gas fill, quadrature and collections; full-vector modal records, material laws and regions, bound step-index and radial graded-index providers, and the guide that pairs a provider with a mode collection, and the connection of a basis that moves along z | P | | `linear.py`, `boundaries.py`, `nonlinear/` | Linear assembly, absorbing rates, response definitions, ionisation, Raman, normalisation and the GNLSE mapping | P | | `translate/` | Validated `PropagationProblem` (`translate/problem.py`) and its route union (`translate/routes.py`), assembled by `assemble` (`translate/assembly.py`), with a capillary's coherent wall and taper coupling in `translate/walls.py` and the refusals and advisories assembly applies in `translate/validation.py`; parameter expansions and z tables; modal/radial/Cartesian transforms and the hybrid planar one (slab modes in x × FFT in y, `translate/planar.py`); dense tracked modal tables, the general modal route and the coupled-flow and metric records (`translate/general.py`, `translate/coupled.py`); the spatial plane measure (`translate/flux.py`, the `FluxPlan` of schema 35); the persisted frame integral and the filled input port of a problem's entrance (`translate/coupling.py`); what the general route and the coupled flow execute, checked once (`translate/admission.py`); response tables, masks and stage bundles | T | -| `backends/` | Registry and protocols, sessions, the Adams call ledger both backends keep (`ledger.py`), node chunks, scalar bodies, spatial and population statistics, and the TDSE response's host source assembly (`tdse_source.py`) | B | +| `backends/` | Registry and protocols, sessions, the run records both backends read (`records.py`, SEP 52) and the engine protocols the marches call (`engines.py`), the Adams call ledger both backends keep (`ledger.py`), node chunks, scalar bodies, spatial and population statistics, and the TDSE response's host source assembly (`tdse_source.py`) | B | | `backends/numpy_ref/` | Reference execution, its Adams kernels and tables (`multistep.py`) and its on-axis scan (`populations.py`) | B | | `backends/host/` | The reference's own step driver, Adams and low-storage marches and run session ([SEP 47](seps/archive/sep-0047-run-ownership.md) phase 2); the parts the Python sessions compose are in `backends/session_core.py` | B | | `stepper/`, `log.py`, `stats.py` | Tableaux, controllers, schedules, progress reporters and statistics specifications | D | @@ -495,7 +495,7 @@ checks. What it computes, once per problem: | product | from | representation | |---|---|---| -| Φ table | `linear.py` + `ZProfile`s | piecewise Chebyshev in z, coefficients `(n_intervals, n_deg, 1, n_modes, n_pol, n_omega)` as in `kernel-contract.md` §3, breakpoints, hard kinks; constant-D fast path (degree 1: one interval, or one per stretch between the kinks of a response path) | +| Φ table | `linear.py` + `ZProfile`s | piecewise Chebyshev in z, coefficients `(n_intervals, n_deg, n_modes, n_pol, n_omega)` as in `kernel-contract.md` §3, breakpoints, hard kinks; constant-D fast path (degree 1: one interval, or one per stretch between the kinks of a response path) | | unitary phase for noise | Im Φ | same table, imaginary part | | quadrature | `modes/quadrature.py` | nodes, weights, coarse weights, synthesis and projection matrices; taper scaling rule | | response spec | `nonlinear/responses.py` | compiled ordering of the fused pass, coefficients, ionisation-rate spline tables (log-spaced, converged, clip limits), Raman kernels on the doubled grid in the frequency domain, normalisation prefactors | diff --git a/docs/dev/backends.md b/docs/dev/backends.md index 38446e5c9..ac6ecc0ce 100644 --- a/docs/dev/backends.md +++ b/docs/dev/backends.md @@ -452,8 +452,8 @@ It consumes the contract only through `linop.delta_phi`, `linop.unitary_phase`, decay rates, so a change of the operator's representation cannot reach it. It refuses, by name and with `BackendError`: a spectral response other than the compiled envelope-plasma assembly; scalar-only Raman on a two-component -problem; general z-dependent response data; a transform and a state that -would both carry the transverse components; and a batch axis above one. +problem; general z-dependent response data; and a transform and a state that +would both carry the transverse components. Measured on the verification ladder: P0 exact to 4.8e-16 for a constant operator and 6.4e-16 for a z-varying one and independent of the step count; P1 order 5.10 and 5.82 over diff --git a/docs/dev/conventions.md b/docs/dev/conventions.md index 8a8a677b6..40b9fb992 100644 --- a/docs/dev/conventions.md +++ b/docs/dev/conventions.md @@ -335,8 +335,9 @@ Cartesian one). ## 6. Array layout and axis names Kernel state and canonical storage: `(n_modes, n_pol, n_omega)` complex128, ω -contiguous (C order): one trajectory. A backend may hold its state privately with a leading axis of one (the native build's four-axis -layout), and a march snapshot's file keeps that axis. Every backend computes in binary64 +contiguous (C order): one trajectory, with no leading axis of one anywhere: not in a +backend's state, the operator's value tables (§3 of the kernel contract), the memory +ledgers or a march snapshot's stored state (SEP 52). Every backend computes in binary64 (ADR-077). Saved fields: `(n_z, n_modes, n_pol, n_omega)`; a scan stores each point separately. Space-domain buffers: `(n_nodes_chunk, n_pol, n_to)`. Convenience accessors may squeeze trivial axes; the seam and the files never do. @@ -345,7 +346,6 @@ Standard axis names, reused verbatim in every jaxtyping annotation and docstring | name | meaning | |---|---| -| `n_batch` | the leading axis of one of a backend's private state layout, the operator's value tables (§3 of the kernel contract) and a march snapshot's stored state; never part of a single trajectory's seam | | `n_modes` | modes (1 for mode-averaged) | | `n_pol` | polarisation components of the kernel state (1 or 2) | | `n_components` | Cartesian components of a local physical field (1, 2 or 3); a general modal guide carries `n_pol = 1` and `n_components = 3` | diff --git a/docs/dev/decisions.md b/docs/dev/decisions.md index 1444bdffe..47361ac70 100644 --- a/docs/dev/decisions.md +++ b/docs/dev/decisions.md @@ -72,7 +72,10 @@ it is qualified, [backends.md](backends.md); for the current seam, - **Status:** accepted; amended by [SEP 47](seps/archive/sep-0047-run-ownership.md) §6.2, which removes the nominal `n_batch` from the single-trajectory seam: the state is - `(n_modes, n_pol, n_omega)` and a group is a sequence of member problems + `(n_modes, n_pol, n_omega)` and a group is a sequence of member problems; and by + [SEP 52](seps/sep-0052-backend-shape.md) §5.4, which removes the remaining leading axis + of one everywhere: the backends' state, the operator's value tables, the memory ledgers + and a march snapshot's stored state - **Context:** Luna has different array shapes per geometry and special code paths for scalar vs vector polarisation and for mode-averaged vs modal propagation. Polarisation support must not slow single-polarisation runs; batch mode must be a first-class leading axis. diff --git a/docs/dev/figures/build.py b/docs/dev/figures/build.py index da2d13dc0..5ad7b7a1e 100644 --- a/docs/dev/figures/build.py +++ b/docs/dev/figures/build.py @@ -722,6 +722,8 @@ class Layer(NamedTuple): ( "protocol", "registry", + "records", + "engines", "spatial", "populations", "tdse_source", diff --git a/docs/dev/figures/layers.svg b/docs/dev/figures/layers.svg index 70d7c5eea..ef64aef0c 100644 --- a/docs/dev/figures/layers.svg +++ b/docs/dev/figures/layers.svg @@ -1,6 +1,6 @@ + width="1200" height="1946.5" viewBox="0 0 1200 1946.5"> - + The layers of solprop, and every module in each Bands and colours from architecture.md §3 and decisions.md ADR-001; modules from src/solprop. Arrows point from the layer that imports to the layer it imports. @@ -174,148 +174,149 @@ text { font-family: Inter, "Helvetica Neue", Helvetica, Arial, sans-serif; fill: plasma_integration · plasma_source envelope_plasma · raman · masks tdse · tdse_source · run_request - - -B -backends -may import -numpy; the native package in native/ - + + +B +backends +may import +numpy; the native package in native/ + backends/ -protocol · registry · spatial -populations · tdse_source · memory -node_chunk · ledger · adams_core -rk_core · exponential · low_storage -session · events · export -session_core · sessions - +protocol · registry · records +engines · spatial · populations +tdse_source · memory · node_chunk +ledger · adams_core · rk_core +exponential · low_storage · session +events · export · session_core +sessions + backends/host/ driver · adams · low_storage session - + backends/numpy_ref/ backend · passes · multistep populations · envelope_plasma general_modal - - -D -driver -may import -P, T -B through the seam, h5py - -log.py - -stats.py - -stepper/ -tableau · runge_kutta · multistep -low_storage · controller · schedule - -output/ -serialisation · provenance -records · metadata · store -capability · memory · callback -hdf5 · tdse_archive · snapshots -scan · scan_artifacts · resumable - -scan.py - -calibration/ -command · controller · first_use -child · grain · problems · refine -selection - -driver/executors/ -local · pool · queue · slurm - -driver/routes/ -gnlse · hollow · general · spatial -materials · keywords - -driver/ -simulation · settings · inputs -metadata · resume · runner -tdse_run · run · summary -interface · scaffold · mixtures -material_output · tdse -tdse_consistency · scan · defaults -estimate · plane_view -session_loop - -recipe/ -core · document · errors · export -metadata · paths · pins · preview -provenance · quantity · registry - -cli.py - - -X -processing / plotting -may import -numpy, scipy -matplotlib, D (outputs) -T (table identities) - -processing/ -beams · axes · fields · spectra -time · coherence · compression -mixtures · modal · planes -plane_file · scans - -plotting/ -propagation · statistics -spectrograms · style · geometry - -sol.py - -capabilities.py - -service/ -agent · beam · beams · client -design · explore · figures · frames -handlers · install · job · kernel -launch · machines · messages -postprocess · recipes · record -reduce · results · run · runs -scan_reductions · sheet · states -waveguide · worker - -processing/ -windows + + +D +driver +may import +P, T +B through the seam, h5py + +log.py + +stats.py + +stepper/ +tableau · runge_kutta · multistep +low_storage · controller · schedule + +output/ +serialisation · provenance +records · metadata · store +capability · memory · callback +hdf5 · tdse_archive · snapshots +scan · scan_artifacts · resumable + +scan.py + +calibration/ +command · controller · first_use +child · grain · problems · refine +selection + +driver/executors/ +local · pool · queue · slurm + +driver/routes/ +gnlse · hollow · general · spatial +materials · keywords + +driver/ +simulation · settings · inputs +metadata · resume · runner +tdse_run · run · summary +interface · scaffold · mixtures +material_output · tdse +tdse_consistency · scan · defaults +estimate · plane_view +session_loop + +recipe/ +core · document · errors · export +metadata · paths · pins · preview +provenance · quantity · registry + +cli.py + + +X +processing / plotting +may import +numpy, scipy +matplotlib, D (outputs) +T (table identities) + +processing/ +beams · axes · fields · spectra +time · coherence · compression +mixtures · modal · planes +plane_file · scans + +plotting/ +propagation · statistics +spectrograms · style · geometry + +sol.py + +capabilities.py + +service/ +agent · beam · beams · client +design · explore · figures · frames +handlers · install · job · kernel +launch · machines · messages +postprocess · recipes · record +reduce · results · run · runs +scan_reductions · sheet · states +waveguide · worker + +processing/ +windows T → P physics objects B → T delta_phi, prefactor_at, the stage bundle - - -D → B backends.select, then the seam - - -X → D a saved run, through the Output protocol - + + +D → B backends.select, then the seam + + +X → D a saved run, through the Output protocol + - -1 - + +1 + - -2 - + +2 + - -3 - + +3 + - -4 -1 Simulation calls translate.assemble -2 the driver builds the grid, the modes, the pulse and the responses -3 the statistics path: a backend is handed grid, modes and profiles at construction: the one import of P that architecture.md §3 allows B, types and constants, never a physics object -4 processing.modal takes the run's own ModalTransformTables beside the handler: a general modal file records what its reference is and not the arrays, so the reference is rebuilt in T and read in X -The package-wide modules carry no layer letter: __init__.py · exceptions.py + +4 +1 Simulation calls translate.assemble +2 the driver builds the grid, the modes, the pulse and the responses +3 the statistics path: a backend is handed grid, modes and profiles at construction: the one import of P that architecture.md §3 allows B, types and constants, never a physics object +4 processing.modal takes the run's own ModalTransformTables beside the handler: a general modal file records what its reference is and not the arrays, so the reference is rebuilt in T and read in X +The package-wide modules carry no layer letter: __init__.py · exceptions.py \ No newline at end of file diff --git a/docs/dev/kernel-contract.md b/docs/dev/kernel-contract.md index 238ce03c6..72cb2e44a 100644 --- a/docs/dev/kernel-contract.md +++ b/docs/dev/kernel-contract.md @@ -361,14 +361,14 @@ and falls to the z table below. The names are the vocabulary of | `phi_representation` | `"constant" \| "expansion" \| "table" \| "separable"` | the representation consumed by `delta_phi` and `unitary_phase`; every kind except the z table has a device form | | `phi_breakpoints` | `(n_intervals + 1,) float64` | interval edges in z — of the table itself, or of the expansion's scalar tables | | `phi_degrees` | `(n_intervals,) intp` | degree used per interval (diagnostic) | -| `phi_coefficients` | `(n_intervals, n_deg, 1, n_modes, n_pol, n_omega) complex128` or `None` | Chebyshev coefficients of Φ(ω, z) = ∫D dz − ½ ln[w(z)/w(0)] along z (the leading axis of one is the value layout the native build reads; the state has none); present for `"table"` and for `"constant"`, whose degree-1 intervals are exact — one, or one per stretch between the hard kinks a response path declares — and absent for `"expansion"` | +| `phi_coefficients` | `(n_intervals, n_deg, n_modes, n_pol, n_omega) complex128` or `None` | Chebyshev coefficients of Φ(ω, z) = ∫D dz − ½ ln[w(z)/w(0)] along z; present for `"table"` and for `"constant"`, whose degree-1 intervals are exact — one, or one per stretch between the hard kinks a response path declares — and absent for `"expansion"` | | `phi_expansion` | frozen dataclass or `None` | Φ as an expansion in the mode set's scalar paths (`translate.expansion.ParameterExpansion`; a leaf, see below); present exactly when `phi_representation` is `"expansion"` | | `hard_kinks` | `(n_kinks,) float64` | z positions where D, or a scalar path a response consumes (a plasma channel's preionisation, an explicit density), jumps or kinks; steps land on them | -| `phi_constant` | `(1, n_modes, n_pol, n_omega) complex128` or `None` | `D` itself, present exactly when `phi_representation` is `"constant"`, so that `delta_phi` returns `D (z₂ − z₁)` to the last bit rather than the difference of two Clenshaw sums. As a bundle it is the one-term case: one array, and the distance as its weight | +| `phi_constant` | `(n_modes, n_pol, n_omega) complex128` or `None` | `D` itself, present exactly when `phi_representation` is `"constant"`, so that `delta_phi` returns `D (z₂ − z₁)` to the last bit rather than the difference of two Clenshaw sums. As a bundle it is the one-term case: one array, and the distance as its weight | | `power_weight_representation` | `"constant" \| "expansion" \| "table"` or `None` | modal power weight representation, present exactly for modal transforms and absent for spatial geometry | | `power_weight` | `PiecewiseChebyshev` or `None` | positive dimensionless `Re n_eff`, or the prefactor's index override, per `(n_modes, n_pol, n_omega)`; table and constant representations only; unit weight outside the active band | | `power_weight_expansion` | `ParameterExpansion` or `None` | the weight with plain basis-product weights; expansion representation only | -| `power_weight_constant` | `(n_modes, n_pol, n_omega) float64` or `None` | exact constant weight, with a matching degree-zero table; constant representation only. It carries no leading axis of one, unlike `phi_constant`'s value layout | +| `power_weight_constant` | `(n_modes, n_pol, n_omega) float64` or `None` | exact constant weight, with a matching degree-zero table; constant representation only | | `beta0`, `beta1` | `ZProfile` tables | frame quantities along z (diagnostic; already inside Φ) | | `separable` | `SeparableOperator` or `None` | shared constant spatial coefficients, present exactly for `"separable"`; described below and shared by identity with `responses.separable` | | `coupled_flow` | `CoupledFlowTables` or `None` | the generator of a guide whose channels couple, dense or factored, with its residual, conditioning and error certificate and one method per section; a leaf in `translate.coupled`, described in [the coupled-flow leaf](#the-coupled-flow-leaf) below | @@ -457,7 +457,7 @@ blocks' tensor sizes. Reuse reweights each block with its original weight kind. this document; they are `parameters` (the path names, in the order of the term multi-index), `ranges` (`(n_params, 2) float64`, the values each path reaches over the tabulated range), `degrees` (`(n_params,) intp`, the Chebyshev degree per parameter), -`terms` (`(n_terms, 1, n_modes, n_pol, n_omega) complex128`, with +`terms` (`(n_terms, n_modes, n_pol, n_omega) complex128`, with `n_terms = Π(degrees + 1) + Π(endpoint_degrees + 1)` when the endpoint block is present, otherwise just the first product, and each multi-index flattened in C order) and `weights`, the concatenated integral and endpoint tables along z as a `(n_terms,)`-valued `PiecewiseChebyshev`. Its @@ -1360,8 +1360,9 @@ one record and holds no default and no constant of its own (`native/AGENTS.md`). [SEP 45](seps/archive/sep-0045-native-rk-march.md) §4 is the algorithm in full; this section is its interface. -**The policy record** (`solprop.backends.rk_core.RungeKuttaPolicy`, formed once per run -by `rk_policy`, with the landings of the run): +**The policy record** (`solprop.backends.records.RungeKuttaPolicy`, formed once per run +by `runge_kutta_policy` with the landings of the run, one of the run records both +backends read): | field | meaning | |---|---| diff --git a/docs/dev/kernel-problem.md b/docs/dev/kernel-problem.md index ebd59760a..9b6cdafbd 100644 --- a/docs/dev/kernel-problem.md +++ b/docs/dev/kernel-problem.md @@ -260,7 +260,7 @@ mask. # A dispersion-specified fibre does not vary along z, so the prefactor is the array # of the fast path rather than a Clenshaw pass over its table. prefactor = np.abs(problem.responses.prefactor_constant[0, 0])[order] - operator = problem.linear.phi_constant[0, 0, 0][order] + operator = problem.linear.phi_constant[0, 0][order] fig, (left, right) = plt.subplots(1, 2, figsize=(7.0, 3.0), layout="constrained") left.plot(detuning, -np.imag(operator), lw=1.6, color="C0") diff --git a/docs/dev/seps/sep-0052-backend-shape.md b/docs/dev/seps/sep-0052-backend-shape.md index 877ee0677..78aa04b23 100644 --- a/docs/dev/seps/sep-0052-backend-shape.md +++ b/docs/dev/seps/sep-0052-backend-shape.md @@ -15,7 +15,7 @@ | **Requires** | SEPs 25, 39, 45 and 47 | | **Supersedes** | — | | **Superseded-by** | — | -| **Implementation** | Opus 5.5 (AI) for John Travers, through four stacked pull requests (D7): (1) phases 0–1, branch `feature/sep-0052-backend-shape-p1` ([#853](https://github.com/LupoLab/solprop/pull/853)); (2) phase 2 and phase 3's axis removal, `feature/sep-0052-backend-shape-p2`; (3) the rest of phase 3, `feature/sep-0052-backend-shape-p3`; (4) phases 4–5, `feature/sep-0052-backend-shape-p4`; started 2026-11-15 | +| **Implementation** | Opus 5.5 (AI) for John Travers, through four stacked pull requests (D7): (1) phases 0–1, branch `feature/sep-0052-backend-shape-p1` ([#853](https://github.com/LupoLab/solprop/pull/853)); (2) phase 2 and phase 3's axis removal, `feature/sep-0052-backend-shape-p2` ([#854](https://github.com/LupoLab/solprop/pull/854)); (3) the rest of phase 3, `feature/sep-0052-backend-shape-p3`; (4) phases 4–5, `feature/sep-0052-backend-shape-p4`; started 2026-11-15 | | **Manual pages** | — | ## Part I — Proposal @@ -1115,6 +1115,112 @@ comparison corpus of §7 from this tree, pins the split collar's rules, and remo unit axis. `ApertureSet.same_as` is kept because tests use it, though production no longer does. +### Milestone 2 (phase 2 and phase 3's axis removal) — 2026-11-16 + +Branch `feature/sep-0052-backend-shape-p2` +([#854](https://github.com/LupoLab/solprop/pull/854)), stacked on milestone 1, by Opus 5.5 +(AI) for John Travers. Three parts were developed by AI agents on separate branches and merged in +order: the corpus (`39fab778`), the run records and protocols (`32fe1130`) and the axis +removal (`e75271e8`). + +**Delivered.** + +- *The pinned corpus* (§7), recorded from the milestone 1 tree before anything moved: + `tests/backend_corpus_cases.py`, `tests/data/generate_backend_corpus.py` and + `tests/data/backend_corpus.npz` (1.2 MB), compared by + `tests/backends/test_backend_corpus.py`. Thirteen cases, each on both backends: + replayed, adaptive and low-storage Runge–Kutta on a fibre; fused Adams with transported + history, exponential Adams, and exponential Adams in the carrier frame with + interpolated planes on a capillary; envelope plasma; general modal; radial; Cartesian; + the line route with a real-space collar under exponential Adams and under low-storage; + and the planar hybrid. Each records the last field, the planes and their statistics, + the step history, the monitor and edge records (split removal and occupancies + included), a stop at the middle plane resumed from its file, and two probe trials or + the low-storage refusal. Fixed schedules are held to `march_bounds.march_bound` (SEP + 25), statistics to four times that, and adaptive runs to their accuracy envelope; a + bitwise comparison is reported as a diagnostic only. Moving a field or the removal by + ten budgets is detected in every case. The generator refuses to overwrite the corpus + without `--force`, which its docstring permits only under §4. +- *The split collar's rules pinned* (§5.2), in `tests/sessions/test_split_collar_rules.py`, + on both backends through sessions, each with a guard that what it rules out exceeds 100 + tolerances. +- *The run records,* `solprop.backends.records`: `run_records(problem, request, start)` + builds a `RunRecords` of the method's policy (`RungeKuttaPolicy`, lifted from + `rk_core` and now carrying the landings; `adams_policy` building the `MarchPolicy`, + formerly derived twice), `LowStorageSettings`, the schedule, the `PlanePlan` and the + `MonitorRecords` (`MonitorConfig`, `EdgeConfig`, `MonitorContinuation`). The monitor, + edge and notice records moved here from `solprop_native._watch` and `_transverse`, and + the quiet-collar rule `collar_rests`, formerly duplicated, lives here once. Native's + session builds every configuration dictionary from the records + (`solprop_native/_records.py`); the reference reads its controller policy, landings, + Adams policy, quiet-collar rule and plane bookkeeping from them. What crosses into + `_core` is unchanged. +- *The engine protocols,* `solprop.backends.engines`: `RungeKuttaEngine`, `AdamsEngine`, + `AcceptRecord`, `BoundaryReading`, `HistoryEntry` and `AttemptOutcome`, as §5.2 states + them, with the commit contract and the low-storage difference in the docstrings. The + old `AdamsEngine` and `LowStorageEngine` protocols remain in use until milestone 3. + `tests/backends/test_engines.py` pins the new members and signatures. +- *The unit axis removed everywhere* (§5.4, #618): translation's value tables + (`phi_coefficients` is `(n_intervals, n_deg, n_modes, n_pol, n_omega)`), the separable + operators, the stage terms, `SplitCrossing` and `BoundaryMonitor`, the reference's state, + passes, history and ledgers, native's load, readback, chord tables and history rows (in + the Python adapter; no C++ changed), the carrier chord, resume, and the stored + `MarchSnapshot`. `HOST_SURFACE` in `tests/backends/test_driver.py` loses `n_batch`; no + field was added or removed. `conventions.md` §6 drops `n_batch`, and ADR-003's status + records the change. +- *Output schema 3.* `SCHEMA_VERSION` goes from 2, which 0.2.0 wrote, to 3, the one bump + for the next release. A snapshot with a leading axis of one is refused by name ("…the + layout (1, …) of a file written before output schema 3…"), and a snapshot of any other + wrong shape is refused too, where it used to be skipped with a warning. +- Changelog fragments: `sep-0052-run-records.changed.md`, + `sep-0052-unit-axis.changed.md` (the seam fragment of ADR-068) and + `sep-0052-snapshot-layout.breaking.md`. + +**Verification.** On an Apple M5 Max, CPU editable build: the corpus passes unchanged +after the axis removal, without regeneration, so the layout change moved no number past +its budget; the corpus also passes on the milestone 1 tree, so phase 0 moved none. The +directly affected modules pass: `tests/translate` (1,420), `tests/backends`, `tests/driver`, +`tests/processing` and `tests/recipe` (about 4,200), `tests/sessions`, `tests/output`, +`tests/nonlinear`, the monitor, split-crossing and spatial-boundary modules, the decision +traces, and the affected `native/tests` files, apart from the thread-sizing failure noted +in milestone 1. After the three merges, the corpus, the collar rules, the records, +protocols, frozen tables, traces, snapshot layout and native session-history modules were +rerun together (391 passed). One run under load once failed +`test_native_session_history.py::test_a_noisy_run_on_a_z_table_meets_the_reference[input-runge-kutta]` +(accepted step ends 2.3e-6 relative against a 1e-6 tolerance); it passed on every rerun +and is likely the multithreaded reductions' order. Not run locally: the full and coverage +lanes, the docs build, the whole native suite. + +**Decisions.** + +- The dense-observation rule of §5.2 cannot be exercised: translation refuses dense + observation beside the split collar, so the rule holds because the combination cannot + run. The refusal is pinned instead. +- `BoundaryReading` follows §5.2 and leaves out the split sums, which travel in + `AcceptRecord`; native's C++ reading still carries them until phase 4. +- Schema 3, not a number above the pre-0.1.0 development schemas (which reached 28): a + development file that says 3 is still refused by its configuration hash and missing + fields, though not by name. +- The calibration store key keeps its `(1, …)` shape, because it is written on users' + disks and changing it would orphan their stored refinements. + +**Findings.** + +- A native run's bits depend on what ran before it in the process: at one build and + thread setting, a native run after a `numpy_ref` run differs in the last bits from one + in a fresh process. SEP 25 allows it, but it is worth an issue. + +**Cost** (`poe size --all`, Python). Total 266,928 → 267,882; `backends` 22,753 → 24,208 +(the records and protocols, mostly docstrings); `translate` 35,021 → 35,005. Tests: about +2,700 lines added, most of them the corpus and the record and collar tests. + +**Handoff.** Milestone 3 restructures the reference over the protocols and records, and +deletes `HostDriver`; the corpus is its comparison. `HostLowStorage` still derives its flow +treatment, `HostDriver` still owns its row period and step planes, and the reference's +monitor resumes through its own `BoundaryMonitor` path: all three go with the driver. The +private-import count is at its cap (87), because `records.py` reads two private monitor +names. + ## Part III — History - 2026-11-15: Drafted, from a review of the native binding's use in production and of the @@ -1131,6 +1237,7 @@ longer does. - 2026-11-15: Implementation started by Opus 5.5 (AI) for John Travers, on `feature/sep-0052-backend-shape-p1`. - 2026-11-16: Milestone 1 (phases 0–1) reported in Part II. +- 2026-11-16: Milestone 2 (phase 2 and the axis removal) reported in Part II. ### AI review — 2026-11-15 diff --git a/docs/dev/testing.md b/docs/dev/testing.md index c40b23e1a..67ae1d561 100644 --- a/docs/dev/testing.md +++ b/docs/dev/testing.md @@ -360,6 +360,16 @@ these records. compared; where the explicit value is solprop's own rather than Luna's, deliberately, to measure the difference, the test says so. A witness is retired, with its generator, only when it blocks a change, and that change says so. +- **The backend corpus** (`tests/data/backend_corpus.npz`, [SEP 52](seps/index.md) §7) + pins what every march and route family left on `numpy_ref` and the native CPU build + before that SEP restructured them: short fixed-schedule runs (and two adaptive ones), + their monitor and edge records, a resume and the timing probe + (`tests/backend_corpus_cases.py`). `tests/backends/test_backend_corpus.py` holds each + backend to its own record within the matched march's budget (class R) or an accuracy + envelope (class T), and reports bitwise agreement on the recording build and thread + setting as a diagnostic only. Its generator refuses to overwrite it unless a change + of the reference's numbers is recorded under SEP 52 §4. The split collar's session + rules are held beside it (`tests/sessions/test_split_collar_rules.py`). - **Analytic problems** (P1, P2) are truth and need no stored data. - **Converged solprop references** (P3) are produced by the three-level ladder with the order gate and stored with their observed order. diff --git a/docs/guide/numerics/linear-tables.md b/docs/guide/numerics/linear-tables.md index 79c2e2377..c2d3539df 100644 --- a/docs/guide/numerics/linear-tables.md +++ b/docs/guide/numerics/linear-tables.md @@ -23,7 +23,7 @@ $$ with the coefficients held in one rectangular array of shape `(n_intervals, n_deg, ...)`, the trailing axes being those of the value; for the kernel contract that value is -`(1, n_modes, n_pol, n_omega)`. Intervals that need a lower degree carry +`(n_modes, n_pol, n_omega)`. Intervals that need a lower degree carry zeros in the high-order slots, which the evaluation handles exactly and at no cost, so a single vectorised pass serves every interval. @@ -219,7 +219,7 @@ Everything above describes the *representation*. What {func}`solprop.translate.linop.build_phi` actually hands a backend is section 3 of the [kernel contract](../../dev/kernel-contract.md): a {class}`solprop.translate.linop.LinearOperator`, holding the interval edges, the Chebyshev -coefficients of $\Phi$ over `(1, n_modes, n_pol, n_omega)`, the degree used on +coefficients of $\Phi$ over `(n_modes, n_pol, n_omega)`, the degree used on each interval, the hard kinks steps must land on, and the frame quantities as scalar tables for the record. Every array is read-only and C-contiguous; nothing else about the mode, the gas or the frame survives the crossing. diff --git a/docs/guide/results/output.md b/docs/guide/results/output.md index fbcc3fc5d..71a098f6d 100644 --- a/docs/guide/results/output.md +++ b/docs/guide/results/output.md @@ -275,7 +275,7 @@ For a modal run: step_stats/ already available periodic statistics /march_snapshot/ optional; attrs: z (NaN while being written), method, order, rejections, last_rejected, proposal, identities (JSON) - state (1, n_modes, n_pol, n_omega) the accepted state, internal + state (n_modes, n_pol, n_omega) the accepted state, internal positions, slots (n_entries,) the history's entries rows/ (n_slots, ...) the engine's rows ``` @@ -428,13 +428,15 @@ it with `open_tdse_archive` ([the TDSE response](../media/tdse.md#the-raw-respon ## The schema version Every file records its `schema_version`; this release writes and reads output -schema **2** (`solprop.output.SCHEMA_VERSION`). Schema 1 was the first released layout, -that of 0.1.0; schema 2 adds `spectral_phase` to the record of a -{py:class}`~solprop.pulses.DataPulse`. A file of any other schema is refused by name -wherever it is opened, read or resumed: files written by an earlier release or by -development versions of solprop, which declared output schemas up to 28, are not read, -and nor is a file from a later release. Read such a file with the solprop version that -wrote it. A scan file is refused the same way, by the output schema each point's run +schema **3** (`solprop.output.SCHEMA_VERSION`). Schema 1 was the first released layout, +that of 0.1.0; schema 2, that of 0.2.0, adds `spectral_phase` to the record of a +{py:class}`~solprop.pulses.DataPulse`; schema 3 stores a march snapshot's `state` and +its engine `rows` without the leading axis of one they kept before, and a snapshot +written in that earlier layout is refused by name. A file of any other schema is refused +by name wherever it is opened, read or resumed: files written by an earlier release or +by development versions of solprop, which declared output schemas up to 28, are not +read, and nor is a file from a later release. Read such a file with the solprop version +that wrote it. A scan file is refused the same way, by the output schema each point's run record declares, whether it is read or resumed; its own `scan_schema_version` versions the scan container, not the records in it, and is still 1. A resume needs the file's schema to be this one, and checks that before it compares the records diff --git a/docs/guide/results/provenance.md b/docs/guide/results/provenance.md index 8e839c248..a293b2e54 100644 --- a/docs/guide/results/provenance.md +++ b/docs/guide/results/provenance.md @@ -96,7 +96,7 @@ A saved result needs the grid, field convention, material models, response choic strength, and a collar's strength, band, threshold and step limit; ``None`` without a transverse boundary. * - ``schema_version``, ``spectral_convention``, ``field_layout`` - - The version of this record layout (currently 2), the sign convention of the + - The version of this record layout (currently 3), the sign convention of the stored field (always ``"optics"``) and the axis order of the stored field. ``` diff --git a/native/src/solprop_native/_core.pyi b/native/src/solprop_native/_core.pyi index cd9fb9d4b..f5c48025c 100644 --- a/native/src/solprop_native/_core.pyi +++ b/native/src/solprop_native/_core.pyi @@ -257,10 +257,9 @@ class State: # `solprop.translate.problem.PropagationProblem`: its shared grid, operator, # response and source components, with the route variant the adapter visited # (solprop SEP 47 section 6.2); `n_modes` and `n_pol_state` are the state's - # `(n_modes, n_pol, n_omega)`. The workspace keeps its four-axis layout - # `(1, n_modes, n_pol, n_omega)`: `set_initial`, `field_to_host` and the - # Adams rows carry the leading axis of one, which the adapter adds and removes, - # and a single trajectory crosses the Python seam without it. + # `(n_modes, n_pol, n_omega)`. `set_initial` takes, and `field_to_host` and the + # Adams rows return, that layout as contiguous elements, which the adapter + # reshapes. def __init__(self, problem: Problem) -> None: ... def set_initial( self, diff --git a/native/src/solprop_native/_load.py b/native/src/solprop_native/_load.py index 5034f03bb..c24579b90 100644 --- a/native/src/solprop_native/_load.py +++ b/native/src/solprop_native/_load.py @@ -16,8 +16,8 @@ - :class:`StageCache`: a step's stage scalars (:func:`solprop.translate.stage.scalars_at`), the anchor hoisted once a step, kept until the next hoist. -- :func:`collar_rests` and :func:`propagator_mode`: the run's quiet-collar rule (SEP 21 - D15) and its flow treatment (SEP 24 section 4.3). +- :func:`propagator_mode`: the run's flow treatment (SEP 24 section 4.3); the + quiet-collar rule is solprop's (:func:`solprop.backends.records.collar_rests`). Each is the reference's arithmetic (``solprop.backends.host.driver.HostDriver``), operation for operation, as the step driver the native backend inherited until SEP 47 @@ -58,7 +58,6 @@ __all__ = [ "Loaded", "StageCache", - "collar_rests", "load", "propagator_mode", "schedule_index", @@ -78,7 +77,7 @@ class Loaded: in the record a backend keeps once its native state holds it, so no host copy outlives the load. noise : numpy.ndarray or None - The ``modified`` model's realisation, shape ``(1, n_modes, n_pol, n_omega)``, + The ``modified`` model's realisation, shape ``(n_modes, n_pol, n_omega)``, which every stage adds; ``None`` under the other models. floor : numpy.ndarray or None The ``input`` model's draw, the same shape, which rides in the state; the @@ -133,8 +132,6 @@ def load( f"{problem.shape} from the contract" ) raise BackendError(msg) - # The kernels' layout, a leading axis of one (SEP 47 section 6.2). - values = values[np.newaxis] position = _checked_position(problem, float(z)) drawn = _noise_field(problem, noise) model = problem.initial.noise_model @@ -146,9 +143,7 @@ def load( and position == problem.initial.z ): if transverse_of(problem.route) is not None: - clean = _clean_launch_columns( - values[0], drawn[0], problem.grids.active_mask - ) + clean = _clean_launch_columns(values, drawn, problem.grids.active_mask) values = values + drawn return Loaded( field=np.ascontiguousarray(values.reshape(-1)), @@ -195,13 +190,12 @@ def _checked_position(problem: PropagationProblem, z: float) -> float: def _noise_field( problem: PropagationProblem, noise: NoiseRealisation | None ) -> ComplexArray | None: - """Return the realisation to carry with a leading axis of one, or ``None``.""" + """Return the realisation to carry, or ``None``.""" if problem.initial.noise_model == "none": return None if noise is not None: - return noise_spectrum(noise, problem.shape)[np.newaxis] - noise0 = problem.initial.noise0 - return None if noise0 is None else noise0[np.newaxis] + return noise_spectrum(noise, problem.shape) + return problem.initial.noise0 def _clean_launch_columns( @@ -273,31 +267,6 @@ def scalars(self, z0: float, z1: float) -> StageScalars: return scalars(self._problem, z0, z1, at=held, helpers=self._helpers) -def collar_rests(problem: PropagationProblem, stepper: StepperSettings) -> bool: - """Return whether a run's stages may leave a quiet temporal collar out (SEP 21). - - A modal or radial stage with the in-stage (``"nonlinear"``) scheme and a temporal - collar alone. The low-storage march, coupled flows, the general modal route, a - real-space collar sharing the term, and the Cartesian, line and planar routes keep - the collar in every stage (SEP 21 D15). - - - Returns: True when the term may be left out while the collar is quiet. - """ - responses = problem.responses - geometry = problem.transform.geometry - resting_geometry = geometry is None or ( - geometry.kind == "radial" and not np.any(geometry.absorber_rate) - ) - return ( - not isinstance(stepper.integrator, LowStorageRungeKutta) - and responses.temporal_absorber is not None - and responses.absorber_scheme == "nonlinear" - and problem.linear.coupled_flow is None - and problem.transform.modal_tables is None - and resting_geometry - ) - - def propagator_mode( problem: PropagationProblem, stepper: StepperSettings, *, device: bool ) -> PropagatorMode: diff --git a/native/src/solprop_native/_probe.py b/native/src/solprop_native/_probe.py index b82fd4e0f..64ba56cdb 100644 --- a/native/src/solprop_native/_probe.py +++ b/native/src/solprop_native/_probe.py @@ -93,8 +93,8 @@ def field(self) -> npt.NDArray[np.complex128]: native = backend._native_state # pyright: ignore[reportPrivateUsage] kept = np.array(backend._observed_field(native)) # pyright: ignore[reportPrivateUsage] finished = backend._finish(native, self._length) # pyright: ignore[reportPrivateUsage] - field = np.array(backend._observed_field(finished)[0]) # pyright: ignore[reportPrivateUsage] + field = np.array(backend._observed_field(finished)) # pyright: ignore[reportPrivateUsage] # The kept state already carries any `input` draw: a resume adds none. - backend._load(kept[0], self._noise, z=self._loaded.z, resume=True) # pyright: ignore[reportPrivateUsage] + backend._load(kept, self._noise, z=self._loaded.z, resume=True) # pyright: ignore[reportPrivateUsage] self._length = None return field diff --git a/native/src/solprop_native/_records.py b/native/src/solprop_native/_records.py new file mode 100644 index 000000000..9849e794e --- /dev/null +++ b/native/src/solprop_native/_records.py @@ -0,0 +1,199 @@ +# SPDX-FileCopyrightText: 2026 John C. Travers and contributors +# +# SPDX-License-Identifier: GPL-3.0-or-later + +"""solprop's run records as ``_core`` reads them (solprop SEP 52 section 5.1). + +solprop forms each run's records once (:func:`solprop.backends.records.run_records`): +the method's policy, the low-storage settings, the schedule, the plane plan and the +boundary monitor's configuration and continuation. This module only converts them into +the dictionaries and arrays the extension's configuration calls take; it derives +nothing. +""" + +from __future__ import annotations + +import dataclasses +from typing import TYPE_CHECKING + +import numpy as np + +if TYPE_CHECKING: + from solprop.backends.records import ( + EdgeConfig, + EdgeResumption, + MonitorConfig, + MonitorContinuation, + PlanePlan, + Resumption, + RungeKuttaPolicy, + ) + from solprop.stepper.multistep import MarchPolicy + from solprop.stepper.schedule import FixedSchedule + +__all__ = [ + "NO_SCHEDULE", + "adams_policy", + "continuation", + "edge_config", + "edge_resumption", + "monitor_config", + "resumption", + "rk_policy", + "schedule_arrays", + "session_plan", +] + +NO_SCHEDULE = np.empty(0, dtype=np.float64) +"""The empty schedule of an adaptive march's configuration.""" + + +def adams_policy(policy: MarchPolicy) -> dict[str, object]: + """Return the Adams march's policy as ``adams_march_configure`` reads it.""" + return dataclasses.asdict(policy) + + +def rk_policy(policy: RungeKuttaPolicy) -> dict[str, object]: + """Return the Runge-Kutta march's policy as ``rk_march_configure`` reads it. + + The controller's fields flattened in: its gains, factors, lag, acceptance rule and + the equivalent norm's power and exponent (the kernel contract's section 12). + """ + controller = policy.controller + return { + "dz_min": policy.dz_min, + "dz_max": policy.dz_max, + "stretch": policy.stretch, + "z_end": policy.z_end, + "save_planes": list(policy.save_planes), + "interpolate": policy.interpolate, + "hard_kinks": list(policy.hard_kinks), + "breakpoints": list(policy.breakpoints), + "landing_ulps": policy.landing_ulps, + "quiet_interval": policy.quiet_interval, + "safety": float(controller.safety), + "beta1": float(controller.beta1), + "beta2": float(controller.beta2), + "min_factor": float(controller.min_factor), + "max_factor": float(controller.max_factor), + "lag": int(controller.lag), + "reject_free": bool(controller.reject_free), + "power": float(controller.estimator_order + 1), + "exponent": 1.0 / (controller.estimator_order + 1), + "start_growth": policy.start_growth, + "auto_init": policy.auto_init, + } + + +def schedule_arrays(schedule: FixedSchedule | None) -> tuple[np.ndarray, np.ndarray]: + """Return a replay's steps and edges, m, or an adaptive run's empty schedule.""" + if schedule is None: + return NO_SCHEDULE, NO_SCHEDULE + return ( + np.ascontiguousarray(schedule.steps, dtype=np.float64), + np.ascontiguousarray(schedule.edges, dtype=np.float64), + ) + + +def session_plan( + planes: PlanePlan, + *, + plane_statistics: object, + row_statistics: object, + monitor: dict[str, object] | None, + dense: dict[str, object] | None, +) -> dict[str, object]: + """Return the observation request and itinerary ``start_session`` reads. + + - ``plane_statistics``, ``row_statistics``: the lowered statistics plans + (``_core.StatisticsPlan``), each ``None`` for none. + - ``monitor``: the monitor's plan, or ``None`` without a monitor. + - ``dense``: the dense output's weights, or ``None`` without dense output. + """ + return { + "planes": planes.planes, + "next_plane": planes.next_plane, + "launch_pending": planes.launch_pending, + "step_planes": planes.step_planes, + "plane_statistics": plane_statistics, + "row_statistics": row_statistics, + "period": planes.period, + "attempt_capacity": planes.attempt_capacity, + "row_capacity": planes.row_capacity, + "replay": planes.replay_steps > 0, + "replay_steps": planes.replay_steps, + "replay_length": planes.replay_length, + "call_attempts": planes.call_attempts, + "call_seconds": planes.call_seconds, + "monitor": monitor, + "dense": dense, + } + + +def monitor_config(config: MonitorConfig) -> dict[str, object]: + """Return the monitor's configuration as ``watch::Config`` reads it.""" + return { + "policy": dict(config.policy), + "floor_share": config.floor_share, + "floor_margin": config.floor_margin, + "budget": config.budget, + "rate": config.rate, + "can_rest": config.can_rest, + } + + +def edge_config(config: EdgeConfig) -> dict[str, object]: + """Return the edge's configuration as ``transverse::Config`` reads it. + + Its fields by name, the arrays as they are (``dataclasses.asdict`` would copy them). + """ + values = {one.name: getattr(config, one.name) for one in dataclasses.fields(config)} + values["faces"] = list(config.faces) + return values + + +def resumption(held: Resumption) -> dict[str, object]: + """Return a monitor record's resumption: numbers flattened in, no absent list.""" + out: dict[str, object] = { + "record": held.record, + "collar_quiet": held.collar_quiet, + **held.numbers, + } + if held.unwatched is not None: + out["unwatched"] = [list(stretch) for stretch in held.unwatched] + if held.resumed_at is not None: + out["resumed_at"] = list(held.resumed_at) + if held.planes is not None: + out["planes"] = [list(column) for column in held.planes] + return out + + +def edge_resumption(held: EdgeResumption) -> dict[str, object]: + """Return an edge record's resumption: the numbers flattened in, no absent field.""" + out: dict[str, object] = {"record": held.record} + if not held.record: + return out + out.update(held.numbers) + for name in ("edge_baseline", "edge_max", "edge_max_raw"): + values = getattr(held, name) + if values is not None: + out[name] = list(values) + if held.reads is not None: + out["reads"] = held.reads + if held.transverse_status is not None: + out["transverse_status"] = held.transverse_status + return out + + +def continuation(held: MonitorContinuation) -> dict[str, object]: + """Return a monitored resume's continuation as the native session reads it. + + - Returns: the earlier record's resumption and its transverse edge's, whether the + run resumes at a saved plane, and the stretches run since without a monitor. + """ + return { + "edge_record": edge_resumption(held.edge), + "record": resumption(held.record), + "plane": held.plane, + "unwatched": [[start, end] for start, end in held.unwatched], + } diff --git a/native/src/solprop_native/_transverse.py b/native/src/solprop_native/_transverse.py index 314d9f367..fbc6bbeaa 100644 --- a/native/src/solprop_native/_transverse.py +++ b/native/src/solprop_native/_transverse.py @@ -2,39 +2,32 @@ # # SPDX-License-Identifier: GPL-3.0-or-later -"""The native session's transverse edge, as solprop configures and reads it. +"""The native transverse edge's launch readings, record and verdicts, for solprop. solprop SEP 47 section 6.4 and the maintainer's decision 4 at its acceptance: the native session owns a free-space run's transverse edge (SEP 40 section 5.3) with every decision and record (``transverse::Edge``, -``native/include/solprop_native/transverse.hpp``). This module only translates: it -forms the edge's configuration from solprop's -:class:`~solprop.transverse_monitor.TransverseEdge` and the problem, reads an earlier -execution's ``transverse`` record into what a resume restores, formats the native -record as :meth:`~solprop.transverse_monitor.TransverseEdge.summary` lays it out, and -formats a native verdict as the edge's warning or exception with the edge's own texts. -It decides nothing. +``native/include/solprop_native/transverse.hpp``). The edge's configuration and a +resume's continuation are solprop's run records (:mod:`solprop.backends.records`, SEP +52), which :mod:`solprop_native._records` converts. This module only translates the +rest: it forms the input noise's readings a launch carries, formats the native record +as :meth:`~solprop.transverse_monitor.TransverseEdge.summary` lays it out, and formats +a native verdict as the edge's warning or exception with the edge's own texts. It +decides nothing. """ from __future__ import annotations -import enum import math from collections.abc import Mapping -from dataclasses import dataclass from typing import TYPE_CHECKING, cast import numpy as np +from solprop.backends.records import STATUSES, EdgeNotice, EdgeNoticeKind from solprop.exceptions import BackendError, BoundaryError, BoundaryWarning -from solprop.spatial_absorber import RAMP_K_FLOOR -from solprop.translate.routes import transverse_of from solprop.transverse_monitor import ( _UNWATCHED_MESSAGE, # pyright: ignore[reportPrivateUsage] - the edge's texts _UNWATCHED_REASON, # pyright: ignore[reportPrivateUsage] - BACKSTOP_READS, - NOISE_FACTOR, - REACH_GROWTH, - READS, _clip_message, # pyright: ignore[reportPrivateUsage] _coarse_message, # pyright: ignore[reportPrivateUsage] _face_message, # pyright: ignore[reportPrivateUsage] @@ -48,193 +41,19 @@ _stop_message, # pyright: ignore[reportPrivateUsage] ) -from ._watch import _number # pyright: ignore[reportPrivateUsage] - one reading - if TYPE_CHECKING: - from solprop.translate.problem import PropagationProblem, StepperSettings - from solprop.transverse_monitor import TransverseEdge, TransverseStatus + from solprop.transverse_monitor import TransverseEdge __all__ = [ - "STATUSES", - "EdgeConfig", - "EdgeNotice", - "NoticeKind", - "config_of", - "declared", - "edge_config", + "edge_notice_of", "log_declared", "log_finish", "noise_readings", "notice_error", "notice_warning", "record_of", - "resumption", ] -STATUSES: tuple[TransverseStatus, ...] = ( - "clear", - "warned", - "stopped", - "insensitive", - "coarse", - "unwatched", -) -"""The transverse statuses in the native order (``transverse::Status``).""" - - -class NoticeKind(enum.IntEnum): - """A native edge notice's kind (``transverse::Notice::Kind``). - - Attributes - ---------- - NONE : int - No notice. - RAMP_BAND : int - The launch's share in the safety ramp's band. - UNWATCHED : int - The backend does not read the edge. - CLIP : int - A face's clip of the launch. - INSENSITIVE : int - The input noise's allowance exceeds the warning. - COARSE : int - A fixed schedule's longest step outruns the read spacing. - FACE : int - A face's judged share. - RETURNED : int - A requested collar's returned share. - RAMP_REMOVAL : int - The safety ramp's removal. - """ - - NONE = 0 - RAMP_BAND = 1 - UNWATCHED = 2 - CLIP = 3 - INSENSITIVE = 4 - COARSE = 5 - FACE = 6 - RETURNED = 7 - RAMP_REMOVAL = 8 - - -_NUMBERS = ( - "launch_energy", - "noise_allowance", - "raw_ramp_removed", - "noise_ramp_removed", - "noise_share_in_ramp", - "declared_share_in_ramp", - "spatial_removed", - "current", - "current_baseline", - "strip_baseline", - "strip_floor", - "strip_max", - "returned_max", - "largest_gap", - "warned_at", -) -"""The numbers a resume reads from an earlier record (``transverse::Resumption``).""" - - -@dataclass(frozen=True, eq=False) -class EdgeConfig: - """The native edge's configuration (``transverse::Config``). - - Attributes - ---------- - values : collections.abc.Mapping - Each field, as ``_core`` reads it. - """ - - values: Mapping[str, object] - - def native(self) -> dict[str, object]: - """Return the configuration as ``_core`` reads it.""" - return dict(self.values) - - -def declared(problem: PropagationProblem) -> bool: - """Return whether a run declares content above ``discard_above``. - - - Returns: True when its transverse specification sets ``discard_above``. - """ - design = transverse_of(problem.route) - return design is not None and design.specification.discard_above is not None - - -def edge_config( - edge: TransverseEdge, problem: PropagationProblem, stepper: StepperSettings -) -> EdgeConfig: - """Return the native edge's configuration for a run's fresh edge. - - - ``stepper``: the run's stepper section: its fixed schedule, if any. - """ - schedule = stepper.schedule - steps = None if schedule is None else np.asarray(schedule.steps) - return config_of(edge, steps=steps, declared=declared(problem)) - - -def config_of( - edge: TransverseEdge, - *, - steps: np.ndarray | None, - declared: bool, - reads: bool = True, -) -> EdgeConfig: - """Return the native edge's configuration for an edge and its launch's arguments. - - Every threshold is formed here as the edge forms it, so that the native edge - compares the same numbers: ``budget ** 2``, ``(stop_factor * budget) ** 2``, the - longest fixed step, and the input noise's share of the weights and count of modes. - """ - design = edge.design - faces = edge._faces # pyright: ignore[reportPrivateUsage] - weights = edge._weights # pyright: ignore[reportPrivateUsage] - first = faces[0].edge - modes = max(int(np.count_nonzero(first)), 1) * max(int(edge.band.size), 1) - return EdgeConfig( - { - "faces": [face.name for face in faces], - "warn": edge.budget**2, - "stop": (edge.stop_factor * edge.budget) ** 2, - "length": float(edge.length), - "guard": float(edge._guard), # pyright: ignore[reportPrivateUsage] - "k_min": float(edge.k_min), - "k_floor": float(RAMP_K_FLOOR), - "reads": float(READS), - "backstop_reads": float(BACKSTOP_READS), - "reach_growth": float(REACH_GROWTH), - "noise_factor": float(NOISE_FACTOR), - "columns": np.ascontiguousarray( - edge._columns, # pyright: ignore[reportPrivateUsage] - dtype=np.float64, - ), - "kappa": np.ascontiguousarray( - edge._kappa, # pyright: ignore[reportPrivateUsage] - dtype=np.float64, - ), - "kappa_order": np.ascontiguousarray( - edge._kappa_order, # pyright: ignore[reportPrivateUsage] - dtype=np.uint64, - ), - "design": design is not None, - "ramp_rate": np.empty(0, dtype=np.float64) - if design is None - else np.ascontiguousarray(design.ramp_rate, dtype=np.float64), - "collar": design is not None and design.has_collar, - "collar_rate": design is not None and design.collar_rate is not None, - "declared": bool(declared), - "reads_edge": bool(reads), - "longest_step": float(np.max(steps)) - if steps is not None and steps.size - else None, - "noise_share": float(np.sum(weights[first])) / float(np.sum(weights)), - "noise_modes": float(modes), - } - ) - def noise_readings( edge: TransverseEdge, noise: np.ndarray, clean: np.ndarray | None @@ -277,58 +96,6 @@ def noise_readings( } -def resumption(record: object, faces: int) -> dict[str, object]: - """Return what the native resume reads of an earlier execution's edge record. - - Only the record's types are read here, as :meth:`TransverseEdge.resume` reads - them: each number present, and, once the record's launch energy and baseline make - that resume read further, each per-face maximum that has a value per face and the - reads, made whole as it makes them and refused as it refuses them. Which fields are - restored, their defaults and the status a resume keeps are native. - - - ``record``: the ``transverse`` entry of the earlier monitor record, or anything - else when there is none. - - Returns: the resumption, as ``_core`` reads it. - """ - if not isinstance(record, Mapping): - return {"record": False} - mapping = cast("Mapping[str, object]", record) - out: dict[str, object] = {"record": True} - for name in _NUMBERS: - value = _number(mapping, name) - if value is not None: - out[name] = value - baseline = _per_face(mapping, "edge_baseline", faces) - launch = cast("float", out.get("launch_energy", math.nan)) - if launch > 0.0 and baseline is not None: - # `TransverseEdge.resume` reads the maxima and the count only past its test - # of the launch and the baseline: a record it ignores is not read further. - out["edge_baseline"] = baseline - for name in ("edge_max", "edge_max_raw"): - value = _per_face(mapping, name, faces) - if value is not None: - out[name] = value - reads = _number(mapping, "reads") - out["reads"] = int(0.0 if reads is None else reads) - status = mapping.get("transverse_status") - if isinstance(status, str): - out["transverse_status"] = status - return out - - -def _per_face( - record: Mapping[str, object], name: str, faces: int -) -> list[float] | None: - """Return a record's per-face list, or ``None`` when it has no value per face. - - As ``TransverseEdge.resume``'s ``numbers`` reads it. - """ - value = record.get(name) - if not isinstance(value, list) or len(cast("list[object]", value)) != faces: - return None - return [float(cast("float", one)) for one in cast("list[object]", value)] - - def record_of(edge: TransverseEdge, native: Mapping[str, object]) -> dict[str, object]: """Format the native edge's record as :meth:`TransverseEdge.summary` does. @@ -378,48 +145,17 @@ def numbers(name: str) -> list[float]: } -@dataclass(frozen=True) -class EdgeNotice: - """A native edge's verdict (``transverse::Notice``): a warning or a stop. - - Attributes - ---------- - kind : NoticeKind - What it is. - stop : bool - Whether it stops the run. - face : int - The face it names, by index. - value : float - The share, allowance or step it names. - z : float - Where, m. - spacing : float - A coarse schedule's read spacing, m. - warned_at : float - Where the edge first warned, for a face's stop, m; NaN where it did not. - """ - - kind: NoticeKind - stop: bool - face: int - value: float - z: float - spacing: float - warned_at: float - - @classmethod - def of(cls, native: Mapping[str, object]) -> EdgeNotice: - """Return the notice ``_core`` reports.""" - return cls( - NoticeKind(int(cast("int", native["kind"]))), - bool(native["stop"]), - int(cast("int", native["face"])), - float(cast("float", native["value"])), - float(cast("float", native["z"])), - float(cast("float", native["spacing"])), - float(cast("float", native["warned_at"])), - ) +def edge_notice_of(native: Mapping[str, object]) -> EdgeNotice: + """Return the edge notice ``_core`` reports (``SessionEvent.notice["edge"]``).""" + return EdgeNotice( + EdgeNoticeKind(int(cast("int", native["kind"]))), + bool(native["stop"]), + int(cast("int", native["face"])), + float(cast("float", native["value"])), + float(cast("float", native["z"])), + float(cast("float", native["spacing"])), + float(cast("float", native["warned_at"])), + ) def _face(edge: TransverseEdge, notice: EdgeNotice) -> str: @@ -430,22 +166,22 @@ def _face(edge: TransverseEdge, notice: EdgeNotice) -> str: def notice_warning(edge: TransverseEdge, notice: EdgeNotice) -> BoundaryWarning: """Return a native edge's warning, as the edge issues it.""" kind = notice.kind - if notice.stop or kind is NoticeKind.NONE: + if notice.stop or kind is EdgeNoticeKind.NONE: msg = f"the native transverse edge warned with a {kind.name.lower()} notice" raise BackendError(msg) - if kind is NoticeKind.RAMP_BAND: + if kind is EdgeNoticeKind.RAMP_BAND: text = _ramp_band_message(notice.value, stop=False) - elif kind is NoticeKind.UNWATCHED: + elif kind is EdgeNoticeKind.UNWATCHED: text = _UNWATCHED_MESSAGE - elif kind is NoticeKind.CLIP: + elif kind is EdgeNoticeKind.CLIP: text = _clip_message(notice.value, _face(edge, notice)) - elif kind is NoticeKind.INSENSITIVE: + elif kind is EdgeNoticeKind.INSENSITIVE: text = _insensitive_message(notice.value) - elif kind is NoticeKind.COARSE: + elif kind is EdgeNoticeKind.COARSE: text = _coarse_message(notice.value, notice.spacing) - elif kind is NoticeKind.FACE: + elif kind is EdgeNoticeKind.FACE: text = _face_message(_face(edge, notice), notice.z, notice.value) - elif kind is NoticeKind.RETURNED: + elif kind is EdgeNoticeKind.RETURNED: text = _return_message(notice.value, notice.z, None) else: text = _ramp_removal_message(notice.value, None) @@ -460,9 +196,9 @@ def notice_error(edge: TransverseEdge, notice: EdgeNotice) -> BaseException: return BackendError( f"the native transverse edge stopped on a {kind.name.lower()} warning" ) - if kind is NoticeKind.RAMP_BAND: + if kind is EdgeNoticeKind.RAMP_BAND: return BoundaryError(_ramp_band_message(notice.value, stop=True)) - if kind is NoticeKind.FACE: + if kind is EdgeNoticeKind.FACE: return BoundaryError( _stop_message( edge.space, @@ -474,9 +210,9 @@ def notice_error(edge: TransverseEdge, notice: EdgeNotice) -> BaseException: warned_at=notice.warned_at, ) ) - if kind is NoticeKind.RETURNED: + if kind is EdgeNoticeKind.RETURNED: return BoundaryError(_return_message(notice.value, notice.z, stop)) - if kind is NoticeKind.RAMP_REMOVAL: + if kind is EdgeNoticeKind.RAMP_REMOVAL: return BoundaryError(_ramp_removal_message(notice.value, stop)) return BackendError( f"the native transverse edge stopped on a {kind.name.lower()} notice" diff --git a/native/src/solprop_native/_watch.py b/native/src/solprop_native/_watch.py index df08ce089..2bb755145 100644 --- a/native/src/solprop_native/_watch.py +++ b/native/src/solprop_native/_watch.py @@ -2,112 +2,43 @@ # # SPDX-License-Identifier: GPL-3.0-or-later -"""The native session's boundary monitor, as solprop configures and reads it. +"""The native session's boundary monitor's record and verdicts, as solprop reads them. solprop SEP 47 section 6.4 and the maintainer's decision 4 at its acceptance: the native session owns the boundary monitor's every decision and record -(``watch::Monitor``, ``native/include/solprop_native/watch.hpp``). This module only -translates: it forms the monitor's configuration from solprop's -:class:`~solprop.monitor.BoundaryMonitor` and the problem, reads an earlier -execution's record into what a resume restores, formats the native record as -:meth:`~solprop.monitor.BoundaryMonitor.summary` lays it out, and formats a native -verdict as solprop's warning or exception. It decides nothing. +(``watch::Monitor``, ``native/include/solprop_native/watch.hpp``). The monitor's +configuration and a resume's continuation are solprop's run records +(:mod:`solprop.backends.records`, SEP 52), which :mod:`solprop_native._records` +converts. This module only translates what comes back: it formats the native record +as :meth:`~solprop.monitor.BoundaryMonitor.summary` lays it out, and a native verdict +as solprop's warning or exception. It decides nothing. """ from __future__ import annotations import dataclasses -import enum import math -from collections.abc import Mapping, Sequence -from dataclasses import dataclass, field +from collections.abc import Mapping from typing import TYPE_CHECKING, cast -import numpy as np +from solprop.backends.records import PLANE_COLUMNS, Notice, NoticeKind from solprop.exceptions import BackendError, BoundaryError from solprop.monitor import ( - _FLOOR, # pyright: ignore[reportPrivateUsage] - the module's constant - FLOOR_MARGIN, _boundary_warning, # pyright: ignore[reportPrivateUsage] - one warning - _count, # pyright: ignore[reportPrivateUsage] - one refusal _guard_message, # pyright: ignore[reportPrivateUsage] - one text _removal_message, # pyright: ignore[reportPrivateUsage] - one text ) if TYPE_CHECKING: - from solprop.boundaries import SplitCrossing - from solprop.monitor import BoundaryCheck, BoundaryMonitor + from solprop.monitor import BoundaryCheck __all__ = [ - "PLANE_COLUMNS", - "MonitorConfig", - "Notice", - "NoticeKind", - "Resumption", - "monitor_config", "notice_error", + "notice_of", "notice_warning", "record_of", - "resumption", - "unwatched_of", ] - -class NoticeKind(enum.IntEnum): - """A native notice's kind (``session::Notice::Kind``). - - Attributes - ---------- - NONE : int - No notice. - GUARD : int - A guard occupancy, at the launch or at a check. - REMOVAL : int - The collar's removal at the end of a run. - CROSSING : int - A replay's step passes the split collar's cap. - READING : int - The split found nothing where the state holds light. - DIVISION : int - Light after a launch that held none, which solprop's monitor divides by. - TRANSVERSE : int - The transverse edge's verdict, which :mod:`solprop_native._transverse` - formats. - """ - - NONE = 0 - GUARD = 1 - REMOVAL = 2 - CROSSING = 3 - READING = 4 - DIVISION = 5 - TRANSVERSE = 6 - - -PLANE_COLUMNS = ("z", "collar_occupancy", "guard_low", "guard_high", "removed_fraction") -"""The monitor record's plane rows, in the native order (``watch::plane_columns``).""" - -_NUMBERS = ( - "launch_energy", - "launch_guard_low", - "launch_guard_high", - "launch_collar", - "guard_low_max", - "guard_high_max", - "guard_low_at", - "guard_high_at", - "collar_max", - "raw_removed_fraction", - "removed_signed", - "removed_fraction", - "floor_removed_fraction", - "raw_guard_max", -) -"""The numbers a resume reads from an earlier record (``watch::Resumption``).""" - -_COUNTS = ("steps", "skipped", "collar_left_out_checks") -"""The counts it reads, each whole or refused (:func:`solprop.monitor._count`).""" - _UNMEASURED = ( "launch_energy", "launch_guard_low", @@ -119,237 +50,18 @@ class NoticeKind(enum.IntEnum): """The record's fields that read ``None`` where nothing was measured.""" -@dataclass(frozen=True) -class MonitorConfig: - """The native monitor's configuration (``watch::Config``). - - Attributes - ---------- - policy : collections.abc.Mapping - The predictions' policy (:meth:`BoundaryMonitor.watch_policy`'s fields); its - floor regions and launch floor are formed natively at the launch. - floor_share : float - ``_FLOOR``: no light is judged below this share of the launch energy. - floor_margin : float - :data:`solprop.monitor.FLOOR_MARGIN`. - budget : float - The budget the finish's removal is judged against. - rate : numpy.ndarray - The collar's power rate on the coarse axis, 1/m. - can_rest : bool - Whether the route's stages may leave a quiet collar out. - """ - - policy: Mapping[str, object] - floor_share: float - floor_margin: float - budget: float - rate: np.ndarray - can_rest: bool - - def native(self) -> dict[str, object]: - """Return the configuration as ``_core`` reads it.""" - return { - "policy": dict(self.policy), - "floor_share": self.floor_share, - "floor_margin": self.floor_margin, - "budget": self.budget, - "rate": self.rate, - "can_rest": self.can_rest, - } - - -@dataclass(frozen=True) -class Resumption: - """What a native resume reads of an earlier record (``watch::Resumption``). - - Attributes - ---------- - record : bool - Whether there is a record at all. - collar_quiet : bool - ``bool(record.get("collar_quiet", False))``. - numbers : collections.abc.Mapping - Each number present in the record, by field; the counts already whole. - unwatched : tuple or None - The record's ``unwatched`` stretches, when it lists them. - resumed_at : tuple or None - The record's ``resumed_at``, when it lists them. - planes : tuple or None - The record's plane rows by :data:`PLANE_COLUMNS` (NaN for ``None``), when they - have those five columns. - """ - - record: bool - collar_quiet: bool = False - numbers: Mapping[str, float] = field(default_factory=dict[str, float]) - unwatched: tuple[tuple[float, ...], ...] | None = None - resumed_at: tuple[float, ...] | None = None - planes: tuple[tuple[float, ...], ...] | None = None - - def native(self) -> dict[str, object]: - """Return the resumption as ``_core`` reads it. - - - Returns: its fields, the numbers flattened in and absent lists left out. - """ - out: dict[str, object] = { - "record": self.record, - "collar_quiet": self.collar_quiet, - **self.numbers, - } - if self.unwatched is not None: - out["unwatched"] = [list(stretch) for stretch in self.unwatched] - if self.resumed_at is not None: - out["resumed_at"] = list(self.resumed_at) - if self.planes is not None: - out["planes"] = [list(column) for column in self.planes] - return out - - -@dataclass(frozen=True) -class Notice: - """A native monitor's verdict (``session::Notice``): a warning or a stop. - - Attributes - ---------- - kind : NoticeKind - What it is. - launch : bool - For a guard: whether it is the launch's judgement. - z : float - Where, m. - value : float - A guard's amplitude, or the removed fraction. - cap : float - A crossing's cap, m. - step : float - A crossing's longest step before the next check, m. - """ - - kind: NoticeKind - launch: bool - z: float - value: float - cap: float - step: float - - @classmethod - def of(cls, native: Mapping[str, object]) -> Notice: - """Return the notice ``_core`` reports. - - - ``native``: ``SessionEvent.notice``. - """ - return cls( - NoticeKind(int(cast("int", native["kind"]))), - bool(native["launch"]), - float(cast("float", native["z"])), - float(cast("float", native["value"])), - float(cast("float", native["cap"])), - float(cast("float", native["step"])), - ) - - -def monitor_config( - monitor: BoundaryMonitor, crossing: SplitCrossing | None, *, can_rest: bool -) -> MonitorConfig: - """Return the native monitor's configuration for a run's fresh monitor. - - The predictions' policy is the monitor's own (:meth:`BoundaryMonitor.watch_policy`) - with the split collar's crossing limit the driver would give it - (``HostDriver.attach_monitor``). - """ - configured = dataclasses.replace(monitor, crossing=crossing) - policy = configured.watch_policy() - policy["floor_regions"] = [0.0, 0.0, 0.0, 0.0] - policy["launch_floor"] = 0.0 - return MonitorConfig( - policy=policy, - floor_share=_FLOOR, - floor_margin=FLOOR_MARGIN, - budget=float(monitor.check.budget), - rate=np.ascontiguousarray(monitor.weights[3], dtype=np.float64), - can_rest=bool(can_rest), - ) - - -def _number(record: Mapping[str, object], name: str) -> float | None: - """Return a record's number, or ``None`` when it is absent or not a number. - - As :func:`solprop.monitor._number` reads it, without its default, which the native - reconstruction applies. - """ - value = record.get(name) - if isinstance(value, bool) or not isinstance(value, int | float): - return None - return float(value) - - -def resumption(record: Mapping[str, object] | None) -> Resumption: - """Return what the native resume reads of an earlier execution's record. - - Only the record's types are read here, as :meth:`BoundaryMonitor.resume` reads - them: each number present, the lists, and the plane rows when they have the five - columns. A count that is not whole is refused as that resume refuses it, once the - record's launch energy makes it read the counts. Which numbers are restored, - their defaults and the fallbacks are native. - - - ``record``: the ``boundary_monitor`` record of the latest earlier execution, or - ``None``. - """ - if record is None: - return Resumption(record=False) - numbers: dict[str, float] = {} - for name in _NUMBERS: - value = _number(record, name) - if value is not None: - numbers[name] = value - launch = numbers.get("launch_energy", math.nan) - if launch > 0.0: - # `BoundaryMonitor.resume` reads the counts only past its launch test. - quiet = _number(record, "quiet_checks") - if quiet is not None and math.isfinite(quiet): - numbers["quiet_checks"] = float(_count(quiet, "quiet_checks")) - for name in _COUNTS: - value = _number(record, name) - numbers[name] = float(_count(0.0 if value is None else value, name)) - listed = record.get("unwatched") - resumed = record.get("resumed_at") - rows = record.get("planes") - planes = None - if isinstance(rows, Mapping) and set(cast("Mapping[str, object]", rows)) == set( - PLANE_COLUMNS - ): - columns = cast("Mapping[str, list[float | None]]", rows) - planes = tuple( - tuple(math.nan if one is None else float(one) for one in columns[name]) - for name in PLANE_COLUMNS - ) - return Resumption( - record=True, - collar_quiet=bool(record.get("collar_quiet", False)), - numbers=numbers, - unwatched=tuple( - tuple(float(one) for one in cast("list[float]", stretch)) - for stretch in cast("list[object]", listed) - ) - if isinstance(listed, list) - else None, - resumed_at=tuple(float(one) for one in cast("list[float]", resumed)) - if isinstance(resumed, list) - else None, - planes=planes, +def notice_of(native: Mapping[str, object]) -> Notice: + """Return the notice ``_core`` reports (``SessionEvent.notice``).""" + return Notice( + NoticeKind(int(cast("int", native["kind"]))), + bool(native["launch"]), + float(cast("float", native["z"])), + float(cast("float", native["value"])), + float(cast("float", native["cap"])), + float(cast("float", native["step"])), ) -def unwatched_of(stretches: Sequence[tuple[float, float]]) -> list[list[float]]: - """Return the stretches a resumed run ran without a monitor, as lists. - - - ``stretches``: ``(start, end)`` in m. - - Returns: ``[start, end]`` pairs. - """ - return [[float(start), float(end)] for start, end in stretches] - - def record_of(check: BoundaryCheck, native: Mapping[str, object]) -> dict[str, object]: """Format the native monitor's record as :meth:`BoundaryMonitor.summary` does. diff --git a/native/src/solprop_native/backend.py b/native/src/solprop_native/backend.py index 124ee6c7a..b0482fcb5 100644 --- a/native/src/solprop_native/backend.py +++ b/native/src/solprop_native/backend.py @@ -44,6 +44,7 @@ plan_node_chunk, ) from solprop.backends.protocol import TimingProbe +from solprop.backends.records import adams_policy from solprop.backends.session import ( ExecutionOptions, PreparedExecution, @@ -76,11 +77,8 @@ ) from solprop.stepper.controller import PIController from solprop.stepper.low_storage import LowStorageRungeKutta -from solprop.stepper.multistep import ( - MarchPolicy, - Multistep, - march_policy, -) +from solprop.stepper.multistep import MarchPolicy, Multistep +from solprop.stepper.schedule import FixedSchedule from solprop.translate.chebyshev import ( _EDGE_SLACK_FRACTION, # pyright: ignore[reportPrivateUsage] - the tables' own admitted span PiecewiseChebyshev, @@ -147,6 +145,7 @@ _core, _counters, _host_blas, + _records, _wisdom, build_info, check_build, @@ -154,7 +153,7 @@ from ._execution import ExecutionRecord from ._helpers import NATIVE from ._host_source import TDSEHostSource -from ._load import Loaded, StageCache, collar_rests, load, propagator_mode +from ._load import Loaded, StageCache, load, propagator_mode from ._probe import NativeProbe from ._statistics import ( StatisticsLowering, @@ -259,9 +258,9 @@ class NativeBackend: Parameters ---------- problem - Validated kernel problem. Accepts one batch member, - modal or spatial scalar or two-component propagation (radial, Cartesian, or - the hybrid planar kind of slab modes in x times an FFT in y), instantaneous + Validated kernel problem. Accepts modal or spatial scalar or two-component + propagation (radial, Cartesian, or the hybrid planar kind of slab modes in x + times an FFT in y), instantaneous Kerr, plasma, scalar time-domain Raman, weighted control, both absorber schemes, and device-owned statistics; and solprop's general modal route, on a constant guide or a moving one whose maps are combined per stage. Statistics @@ -511,16 +510,6 @@ def closed(self) -> bool: # numpydoc ignore=RT01 """Whether the backend has released its buffers and can run no other input.""" return self._closed - @property - def _layout(self) -> tuple[int, int, int, int]: - """Return the build's state layout: the shape behind a unit axis. - - The C++ kernels keep their four-axis layouts; a single trajectory crosses the - Python seam without the axis (solprop SEP 47 section 6.2). - """ - n_modes, n_pol, n_omega = self._problem.shape - return (1, n_modes, n_pol, n_omega) - def _load( self, field: ComplexArray, @@ -616,7 +605,7 @@ def _restore(self, start: Start, loaded: Loaded) -> tuple[str | None, float]: reason = adams.resume(snapshot, loaded.z) if reason is not None: return reason, proposal - self._install(np.array(snapshot.state, copy=True)[None]) + self._install(np.array(snapshot.state, copy=True)) return None, float(snapshot.proposal) def _adams_engine( @@ -654,7 +643,7 @@ def _adams_engine( native=self._native_state, method=method, controller=self._stepper.controller, - shape=self._layout, + shape=self._problem.shape, compensated=compensated, interpolate=self._stepper.interpolate_planes, carrier=carrier_reference(self._problem, method), @@ -944,7 +933,7 @@ def _delta_phi(self, bundle: StageScalars) -> ComplexArray: if tables is not None and tables.phi: here = tables.values("phi", bundle.z1) anchored = tables.values("phi", bundle.z0) - return (here - anchored).reshape(self._layout) + return (here - anchored).reshape(self._problem.shape) if bundle.phi_delta is None or self._terms.phi is None: return delta_phi( self._problem.linear, bundle.z0, bundle.z1, helpers=self.helpers @@ -959,7 +948,7 @@ def _unitary_phase(self, bundle: StageScalars) -> RealArray: """ tables = self._z_tables if tables is not None and tables.phi: - return np.imag(tables.values("phi", bundle.z1)).reshape(self._layout) + return np.imag(tables.values("phi", bundle.z1)).reshape(self._problem.shape) if bundle.phi_at is None or self._terms.phi is None: return unitary_phase(self._problem.linear, bundle.z1) return np.imag(combine(bundle.phi_at, self._terms.phi)) @@ -1152,10 +1141,6 @@ def _finish(self, state: _core.State, dz: float) -> _core.State: state.finish(dz) return state - def _collar_rests(self) -> bool: - """Return whether this run's stages may leave a quiet collar out (SEP 21).""" - return collar_rests(self._problem, self._stepper) - def _launch_clean_columns(self, edge: TransverseEdge) -> RealArray | None: """Return each column's launched energy without the input draw, or ``None``. @@ -1173,8 +1158,8 @@ def _launch_clean_columns(self, edge: TransverseEdge) -> RealArray | None: return None if loaded.clean_columns is not None: return loaded.clean_columns - state = np.asarray(self._observed_field(self._native_state)[0]) - return edge.reduction.column_power(state - np.asarray(floor[0])) + state = np.asarray(self._observed_field(self._native_state)) + return edge.reduction.column_power(state - np.asarray(floor)) def timing_probe(self, start: Start) -> TimingProbe: """Load a start and return single trials of this run's march from it. @@ -1206,7 +1191,7 @@ def timing_probe(self, start: Start) -> TimingProbe: def _observed_field(self, state: _core.State) -> ComplexArray: if state is not self._native_state: raise BackendError("the observed state does not belong to this backend") - field = state.field_to_host().reshape(self._layout) + field = state.field_to_host().reshape(self._problem.shape) if (counters := _counters.active) is not None: counters.crossing("plane", field.nbytes) return field @@ -1559,7 +1544,7 @@ class _ResidentAdams: method: Multistep controller: PIController shape: tuple[int, ...] - """The state layout, ``(1, n_modes, n_pol, n_omega)``.""" + """The state layout, ``(n_modes, n_pol, n_omega)``.""" interpolate: bool compensated: bool = False """Whether trials carry a varying operator's transported compensation.""" @@ -1605,8 +1590,8 @@ def snapshot_nbytes(self) -> int: def load_rows(self, rows: Mapping[str, np.ndarray], shape: tuple[int, ...]) -> None: """Install a snapshot's rows, for a state of `shape`. - The rows keep ``HostHistory``'s names and layout, ``(n_slots, n_batch, - n_modes, n_pol, n_omega)``: ``"store"`` and, compensated, ``"states"`` and + The rows keep ``HostHistory``'s names and layout, ``(n_slots, n_modes, + n_pol, n_omega)``: ``"store"`` and, compensated, ``"states"`` and ``"rates"``. A frozen-residual flow's history holds each sample's generator applied to its state, ``"applied"``, where the host engine holds the sample's weights (``"weights_"``); the two are not interchangeable, and each engine @@ -1649,9 +1634,6 @@ def load_rows(self, rows: Mapping[str, np.ndarray], shape: tuple[int, ...]) -> N _EMPTY = np.empty(0, dtype=np.complex128) """No modified-noise realisation: the native state adds none.""" -_NO_SCHEDULE = np.empty(0, dtype=np.float64) -"""The empty schedule of an adaptive march's configuration.""" - _STAGE_CONSTANT = 1 _STAGE_EXPANSION = 2 _STAGE_TABLE = 3 @@ -1696,8 +1678,8 @@ def _section_tables( def _chord_tables(carrier: CarrierReference | None) -> dict[str, object]: """Return the carrier frame's chord as the native stage tables take it. - The imaginary part of the reference element's operator for the one batch member - (`CarrierChord`): a constant's rate, an expansion's terms over the ``Phi`` weights + The imaginary part of the reference element's operator (`CarrierChord`): a + constant's rate, an expansion's terms over the ``Phi`` weights the tables already hold, or a z table's reference-element column as a scalar table of its own (solprop issue #255), packed as its batch pass reads it. None without an active frame. @@ -1706,12 +1688,12 @@ def _chord_tables(carrier: CarrierReference | None) -> dict[str, object]: if chord is not None and chord.constant is not None: return { "chord": 1, - "chord_rate": float(np.imag(chord.constant)[0]), + "chord_rate": float(np.imag(chord.constant)), "chord_terms": [], "chord_table": None, } if chord is not None and chord.expansion is not None: - terms = np.imag(np.asarray(chord.expansion.terms))[:, 0] + terms = np.imag(np.asarray(chord.expansion.terms)) return { "chord": 2, "chord_rate": 0.0, @@ -1720,9 +1702,9 @@ def _chord_tables(carrier: CarrierReference | None) -> dict[str, object]: } if chord is not None and chord.table is not None: edges, lower, upper, coefficients = chord.table.packed() - # The one batch member's column: the imaginary part a complex Clenshaw pass - # forms at a real position, by the same operations (`stage::chord`). - column = np.ascontiguousarray(np.imag(coefficients[:, :, :1])) + # The imaginary part a complex Clenshaw pass forms at a real position, by the + # same operations (`stage::chord`). + column = np.ascontiguousarray(np.imag(coefficients)) return { "chord": 3, "chord_rate": 0.0, @@ -1835,7 +1817,17 @@ def _low_storage_flows( """ if not isinstance(stepper.integrator, LowStorageRungeKutta): return None - flows = low_storage_flows(problem, stepper, "native") + return _resident_flows( + low_storage_flows(problem, stepper, "native"), stepper, device=device + ) + + +def _resident_flows(flows: int, stepper: StepperSettings, *, device: bool) -> int: + """Return the flow volumes the native state holds for a CPU's count `flows`. + + A device build recomputes under ``"auto"`` and holds none + (:class:`~solprop.backends.records.LowStorageSettings` counts a CPU's). + """ return 0 if device and stepper.propagator_mode == "auto" else flows @@ -1963,7 +1955,7 @@ class _NativeAdamsMarch: The native state steps solprop's exponential Adams march (solprop SEP 39 phase 2): the controller, the history, the landings and the call ledger's counts advance - natively, from one policy record (:func:`~solprop.stepper.multistep.march_policy`). + natively, from one policy record (:func:`~solprop.backends.records.adams_policy`). This record is what the run starts from: the policy, the history's entries (position, identity and engine row), the order, the rejection flags and the :class:`~solprop.backends.numpy_ref.multistep.CallLedger`, each fresh at a load @@ -1996,31 +1988,13 @@ def __init__( self._last_rejected = False self._order = 1 if method.variable_order else method.order self.ledger = CallLedger() - # An in-stage temporal collar's largest rate bounds each order's stable step - # in the exponential formulation (`adams_core`'s order choice). - responses = problem.responses - collar = responses.temporal_absorber - collar_rate = ( - 0.5 * float(np.max(collar)) - if collar is not None - and responses.absorber_scheme == "nonlinear" - and exponential_form - else 0.0 - ) - linear = problem.linear carrier = getattr(self._engine, "carrier", None) - # Every number the native controller and landings read (SEP 39 section 4.2). - self._policy: MarchPolicy = march_policy( + # Every number the native controller and landings read (SEP 39 section 4.2), + # formed as the reference forms it. + self._policy: MarchPolicy = adams_policy( + problem, + stepper, method, - dz_min=stepper.dz_min, - dz_max=stepper.dz_max, - stretch=stepper.stretch, - z_end=stepper.z_end, - save_planes=np.asarray(stepper.save_planes, dtype=np.float64).tolist(), - hard_kinks=np.asarray(linear.hard_kinks, dtype=np.float64).tolist(), - breakpoints=np.asarray(linear.phi_breakpoints, dtype=np.float64).tolist(), - interpolate=self._interpolate, - collar_rate=collar_rate, carrier=isinstance(carrier, CarrierReference) and carrier.active, ) # Whether the stages left a quiet collar out at the last attempt. @@ -2082,10 +2056,8 @@ def resume(self, snapshot: MarchSnapshot, z: float) -> str | None: reason = self._refusal(snapshot, z) if reason is not None: return reason - # The rows keep the engine's layout, a leading axis of one beside their own - # (the snapshot's disk format, SEP 47 section 6.7). try: - self._engine.load_rows(snapshot.rows, (1, *self._backend._problem.shape)) + self._engine.load_rows(snapshot.rows, self._backend._problem.shape) except BackendError as error: return str(error) self._history = [ @@ -2161,24 +2133,20 @@ def start_state(self, z: float, proposal: float, cursor: int) -> dict[str, objec "ledger": {name: getattr(ledger, name) for name in _MARCH_COUNTERS}, } - def _configure(self) -> None: - """Hand the native state the run's policy and a replay's schedule, once. + def _configure(self, policy: MarchPolicy, schedule: FixedSchedule | None) -> None: + """Hand the native state a run's policy and a replay's schedule, once. - The policy carries no quiet interval: a session's native monitor holds its - own (SEP 47 section 6.4). + - ``policy``: the run's record + (:attr:`~solprop.backends.records.RunRecords.policy`), with no quiet + interval: a session's native monitor holds its own (SEP 47 section 6.4). """ if self._configured: return - policy = dataclasses.replace(self._policy, quiet_interval=None) backend = self._backend engine = cast("_ResidentAdams", self._engine) - schedule = backend._stepper.schedule - steps = edges = _NO_SCHEDULE - if schedule is not None: - steps = np.ascontiguousarray(schedule.steps, dtype=np.float64) - edges = np.ascontiguousarray(schedule.edges, dtype=np.float64) + steps, edges = _records.schedule_arrays(schedule) backend._native_state.adams_march_configure( - dataclasses.asdict(policy), + _records.adams_policy(policy), self._fused, engine.compensated, engine.interpolate, diff --git a/native/src/solprop_native/session.py b/native/src/solprop_native/session.py index 833c9f59b..81fd18b23 100644 --- a/native/src/solprop_native/session.py +++ b/native/src/solprop_native/session.py @@ -48,7 +48,8 @@ each warning is its own :class:`~solprop.backends.events.ServiceRequired` barrier and a stop a terminal failure, the edge's readings and a collar's current are taken from the resident state with no return to Python, and a monitored resume continues - both records natively (:mod:`solprop_native._transverse`); + both records natively from solprop's run records + (:mod:`solprop.backends.records`, converted by :mod:`solprop_native._records`); - dense output (``interpolate_planes``), for Adams and the embedded Runge-Kutta tableau alike: a plane the march steps across is interpolated natively inside the step that crossed it, the committed state held aside, and observed in order before @@ -99,6 +100,13 @@ from solprop.backends.ledger import CallLedger, budget_error from solprop.backends.populations import axis_refusal from solprop.backends.protocol import BackendHistory, StepStats, StepSummary +from solprop.backends.records import ( + NoticeKind, + RungeKuttaPolicy, + monitor_continuation, + run_records, + runge_kutta_policy, +) from solprop.backends.session import ( DEFAULT_DEADLINE, DEFAULT_WORK_LIMIT, @@ -128,15 +136,14 @@ ) from solprop.output.records import MarchSnapshot from solprop.stats import weight_refusal_for -from solprop.stepper.controller import START_GROWTH, ControlRecord +from solprop.stepper.controller import ControlRecord from solprop.stepper.low_storage import LowStorageRungeKutta from solprop.stepper.multistep import Multistep -from solprop.stepper.schedule import LANDING_ULPS from solprop.stepper.tableau import continuous_extension from solprop.translate.linop import power_weight_at from solprop.translate.modal import QUADRATURE_ALARM, warn_quadrature -from . import _core, _counters, _transverse, _watch +from . import _core, _counters, _records, _transverse, _watch from ._load import Loaded from ._statistics import ( ADMITTED_GENERAL_STATISTICS, @@ -152,12 +159,11 @@ _DIVERGED, # pyright: ignore[reportPrivateUsage] _ERROR, # pyright: ignore[reportPrivateUsage] _FLOOR, # pyright: ignore[reportPrivateUsage] - _NO_SCHEDULE, # pyright: ignore[reportPrivateUsage] NativeBackend, _aperture_payload, # pyright: ignore[reportPrivateUsage] - _low_storage_flows, # pyright: ignore[reportPrivateUsage] _on_device, # pyright: ignore[reportPrivateUsage] _raise_native, # pyright: ignore[reportPrivateUsage] + _resident_flows, # pyright: ignore[reportPrivateUsage] ) _LOG = logging.getLogger(__name__) @@ -169,6 +175,7 @@ from solprop.backends.events import Event from solprop.backends.export import ExportRecipe + from solprop.backends.records import RunRecords from solprop.backends.session import RunSession, Start from solprop.grid import TimeGrid from solprop.log import StopFlag @@ -176,6 +183,8 @@ from solprop.monitor import BoundaryMonitor from solprop.profiles import ZProfile from solprop.stats import StatsSpec + from solprop.stepper.multistep import MarchPolicy + from solprop.stepper.schedule import FixedSchedule from solprop.translate.problem import PropagationProblem, StepperSettings from solprop.translate.run_request import RunRequest @@ -188,19 +197,6 @@ "refusal", ] -CALL_ATTEMPTS = 256 -"""A march call's most attempts, as the driver's native marches bound them (SEP 39).""" - -CALL_SECONDS = 0.1 -"""A march call's most wall time, s.""" - -ATTEMPT_CAPACITY = 4096 -"""The most attempts the native session holds between two events; a full buffer yields -a progress event carrying them (SEP 47 section 5.3).""" - -ROW_CAPACITY = 1024 -"""The most periodic rows it holds between two events, likewise.""" - _SESSION_IDS = itertools.count(1) _KINDS = ( ObservationKind.LAUNCH, @@ -417,6 +413,10 @@ def __init__( if refused is not None: msg = f"the native session does not run {refused}" raise BackendError(msg) + if request.boundary.monitor is not None: + _refuse_foreign_monitor( + NativeBackend.name, problem, request.boundary.monitor + ) owned = backend is None if backend is not None: adopted = self._driver(backend) @@ -439,10 +439,16 @@ def __init__( # The stepping policy as one record (`RunRequest.stepper`). self._stepper = request.stepper self._monitor: BoundaryMonitor | None = request.boundary.monitor + # Every record the run reads, formed once (SEP 52 section 5.1): the method's + # policy, the low-storage settings, the schedule, the plane plan and the + # monitor's configuration and continuation. + self._records: RunRecords = run_records(problem, request, start) self._finish_logged = False # A declared launch's share in the ramp's band is logged once it is launched. - edge = None if self._monitor is None else self._monitor.transverse - self._declared_logged = edge is None or not _transverse.declared(problem) + held = self._records.monitor + self._declared_logged = ( + held is None or held.edge is None or not held.edge.declared + ) self._sequence = 0 self._id = next(_SESSION_IDS) self._listener = listener @@ -486,12 +492,7 @@ def __init__( self._owns = owned or owns # Every position a plane statistic is taken at, where the request fixes them # all: its start and its save planes, with no step or interpolated plane. - observation = request.observation - self._planes: tuple[float, ...] | None = ( - None - if observation.dense or observation.step_planes - else (float(start.z), *(float(z) for z in observation.save_planes)) - ) + self._planes = self._records.planes.fixed_positions lowering = self._lower(union, grid, modes, profiles) self._statistics_tables = lowering.tables self._plane_lowering = lowering.subset(spec.names) @@ -524,7 +525,7 @@ def __init__( # count, less what earlier runs on the backend made (SEP 19). self._axis_scans = int(backend._native_state.axis_scans) # pyright: ignore[reportPrivateUsage] self._native = backend._native_state.start_session( # pyright: ignore[reportPrivateUsage] - kind, state, self._plan(start), float(start.z) + kind, state, self._plan(), float(start.z) ) except BaseException: if owned: @@ -593,15 +594,20 @@ def _prepare( ) self._loaded = loaded self._march_refusal, proposal = driver._restore(start, loaded) + records = self._records if isinstance(integrator, Multistep): march = driver._adams - march._configure() # pyright: ignore[reportPrivateUsage] + march._configure( # pyright: ignore[reportPrivateUsage] + cast("MarchPolicy", records.policy), records.schedule + ) else: # The Runge-Kutta march, and the low-storage replay, which is the native # Runge-Kutta march's over the stage tables the state reads each step's # scalars from (SEP 47 section 6.3). driver._staged_rows() - _NativeRungeKuttaMarch(backend)._configure() # pyright: ignore[reportPrivateUsage] + _NativeRungeKuttaMarch(backend)._configure( # pyright: ignore[reportPrivateUsage] + cast("RungeKuttaPolicy", records.policy), records.schedule + ) kind = "runge_kutta" if isinstance(integrator, Multistep): kind = "adams" @@ -609,18 +615,15 @@ def _prepare( kind = "low_storage" # The flows `_ResidentLowStorage` holds, and the operator's origin, from # which a constant operator's table is built. - flows = _low_storage_flows( - self._problem, self._stepper, device=_on_device() - ) - assert flows is not None # the march is the low-storage one + settings = records.low_storage + assert settings is not None # the march is the low-storage one backend._native_state.low_storage_configure( # pyright: ignore[reportPrivateUsage] - flows, - driver._terms.separable is not None, - float(self._problem.linear.z_span[0]), + _resident_flows(settings.flows, self._stepper, device=_on_device()), + settings.separable, + settings.origin, ) monitor = self._monitor if monitor is not None: - _refuse_foreign_monitor(driver.name, self._problem, monitor) # The monitor's weights where the state lives, read at every check, and the # split collar's walk-off buckets; the monitor stays solprop's and fresh: # the session owns its state natively. @@ -635,7 +638,7 @@ def _prepare( ledger = adams.ledger self._calls = dataclasses.replace(ledger, orders=list(ledger.orders)) self._method = adams._method - self._max_calls = adams._policy.max_calls + self._max_calls = cast("MarchPolicy", records.policy).max_calls # The history's persistent rows, sized while the state is not leased. self._rows_nbytes = int(adams._engine.snapshot_nbytes()) state: dict[str, object] = adams.start_state( @@ -649,10 +652,10 @@ def _prepare( "norms": norms, "cursor": int(loaded.cursor), } - self._edge_noise = self._noise_readings(driver, start) + self._edge_noise = self._noise_readings(driver) return kind, state - def _noise_readings(self, driver: Any, start: Start) -> dict[str, object] | None: + def _noise_readings(self, driver: Any) -> dict[str, object] | None: """Return the input noise's readings the transverse edge's launch carries. The edge's launch takes the input draw and the launch's columns without it @@ -661,7 +664,7 @@ def _noise_readings(self, driver: Any, start: Start) -> dict[str, object] | None before the native session leases it. A launch that resumes an earlier edge's record is not judged, so it takes no such columns and no state is read for them: the native edge resumes exactly when the record's resumption holds a - launch baseline (:func:`solprop_native._transverse.resumption`). + launch baseline (:func:`solprop.backends.records.edge_resumption`). - Returns: the readings (:func:`solprop_native._transverse.noise_readings`), or ``None`` without an edge or input noise. @@ -671,11 +674,13 @@ def _noise_readings(self, driver: Any, start: Start) -> dict[str, object] | None drawn = self._loaded.floor if edge is None or drawn is None: return None - resumed = start.monitor is not None and "edge_baseline" in cast( - "dict[str, object]", self._continuation(start.monitor)["edge_record"] + held = self._records.monitor + continuation = None if held is None else held.continuation + resumed = ( + continuation is not None and continuation.edge.edge_baseline is not None ) clean = None if resumed else driver._launch_clean_columns(edge) - return _transverse.noise_readings(edge, np.asarray(drawn[0]), clean) + return _transverse.noise_readings(edge, np.asarray(drawn), clean) @staticmethod def _driver(backend: NativeBackend | None) -> Any: @@ -689,31 +694,16 @@ def _driver(backend: NativeBackend | None) -> Any: assert backend is not None return backend - def _plan(self, start: Start) -> dict[str, object]: + def _plan(self) -> dict[str, object]: """Return the observation request and itinerary for the native session.""" - observation = self._request.observation - schedule = ( - None if self._request.replay is None else self._request.replay.schedule + planes = self._records.planes + return _records.session_plan( + planes, + plane_statistics=self._plane_lowering.plan, + row_statistics=self._row_lowering.plan, + monitor=self._monitor_plan(), + dense=self._dense_plan() if planes.dense else None, ) - rows = bool(self._row_names) - return { - "planes": np.ascontiguousarray(observation.save_planes, dtype=np.float64), - "next_plane": int(start.plane_index), - "launch_pending": bool(start.plane_pending), - "step_planes": int(observation.step_planes), - "plane_statistics": self._plane_lowering.plan, - "row_statistics": self._row_lowering.plan, - "period": int(self._spec.period) if rows else 0, - "attempt_capacity": ATTEMPT_CAPACITY, - "row_capacity": ROW_CAPACITY, - "replay": schedule is not None, - "replay_steps": 0 if schedule is None else int(schedule.n_steps), - "replay_length": 0.0 if schedule is None else float(schedule.total_length), - "call_attempts": CALL_ATTEMPTS, - "call_seconds": CALL_SECONDS, - "monitor": self._monitor_plan(start), - "dense": self._dense_plan() if observation.dense else None, - } def _dense_plan(self) -> dict[str, object]: """Return the dense output the native session interpolates planes with. @@ -729,22 +719,18 @@ def _dense_plan(self) -> dict[str, object]: assert weights is not None # the translation refuses dense output without one return {"weights": np.ascontiguousarray(weights, dtype=np.float64)} - def _monitor_plan(self, start: Start) -> dict[str, object] | None: + def _monitor_plan(self) -> dict[str, object] | None: """Return the native monitor's configuration and any record it continues. - Returns: the monitor's plan, or ``None`` without a monitor. """ monitor = self._monitor - if monitor is None: + held = self._records.monitor + if monitor is None or held is None: return None - backend = self._driver(self._backend) plan: dict[str, object] = { - "config": _watch.monitor_config( - monitor, - self._problem.split_crossing if monitor.split else None, - can_rest=bool(backend._collar_rests()), - ).native(), - "resume": start.monitor is not None, + "config": _records.monitor_config(held.config), + "resume": held.continuation is not None, } assert self._loaded is not None # `_prepare` loaded the start drawn = self._loaded.floor @@ -753,43 +739,13 @@ def _monitor_plan(self, start: Start) -> dict[str, object] | None: # `BoundaryMonitor.launch` reads it. floor = float(monitor.power(drawn).sum()) plan["floor"] = [floor, *(floor * monitor.weights.mean(axis=1)).tolist()] - if monitor.transverse is not None: - plan["edge"] = _transverse.edge_config( - monitor.transverse, self._problem, self._stepper - ).native() + if held.edge is not None: + plan["edge"] = _records.edge_config(held.edge) plan["edge_noise"] = self._edge_noise - if start.monitor is not None: - plan.update(self._continuation(start.monitor)) + if held.continuation is not None: + plan.update(_records.continuation(held.continuation)) return plan - def _continuation(self, continuation: Mapping[str, object]) -> dict[str, object]: - """Return a monitored resume's continuation as the native session reads it. - - - Returns: the earlier record's resumption and its transverse edge's, whether - the run resumes at a saved plane, and the stretches run since without a - monitor. - """ - record = continuation.get("record") - edge = None if self._monitor is None else self._monitor.transverse - faces = 0 if edge is None else len(edge._faces) # pyright: ignore[reportPrivateUsage] - return { - "edge_record": _transverse.resumption( - cast("Mapping[str, object]", record).get("transverse") - if isinstance(record, Mapping) - else None, - faces, - ), - "record": _watch.resumption( - cast("Mapping[str, object]", record) - if isinstance(record, Mapping) - else None - ).native(), - "plane": bool(continuation.get("plane", False)), - "unwatched": _watch.unwatched_of( - cast("Sequence[tuple[float, float]]", continuation.get("unwatched", ())) - ), - } - # -- protocol ------------------------------------------------------------------- @property @@ -1122,11 +1078,10 @@ def noise_phase(self, ticket: ExportTicket) -> np.ndarray | None: # Too many positions, one outside the table's span, or no table of # `Phi`: the backend's error, as `_ZTables.load` raises the first. raise BackendError(str(error)) from error - phase = np.imag(values).reshape(driver._layout) + phase = np.imag(values).reshape(self._problem.shape) else: phase = driver._unitary_phase(driver._stage_scalars(z, z)) - # The phase keeps the operator table's leading axis of one. - return np.array(np.broadcast_to(phase, (1, *self._problem.shape))[0]) + return np.array(np.broadcast_to(phase, self._problem.shape)) def request_snapshot(self) -> SnapshotReceipt: """Latch a snapshot for the next eligible accepted endpoint. @@ -1200,8 +1155,6 @@ def march_snapshot(self, ticket: ExportTicket) -> MarchSnapshot | None: history = state["history"] return MarchSnapshot( z=float(state["z"]), - # The trajectory's state; the history rows keep the engine's layout, a - # leading axis of one after the row axis (solprop SEP 47 section 6.7). state=field.reshape(shape), proposal=float(state["proposal"]), method=repr(method), @@ -1213,7 +1166,7 @@ def march_snapshot(self, ticket: ExportTicket) -> MarchSnapshot | None: "predicted" if entry[1] else "accepted" for entry in history ), slots=np.array([entry[2] for entry in history], dtype=np.int64), - rows={name: row.reshape(len(row), 1, *shape) for name, row in rows.items()}, + rows={name: row.reshape(len(row), *shape) for name, row in rows.items()}, collar_was_quiet=bool(state["collar_was_quiet"]), history_key=method.history_key, ) @@ -1394,7 +1347,7 @@ def continue_monitor(self, continuation: Mapping[str, object]) -> None: """ if self._monitor is None: return - held = self._continuation(continuation) + held = _records.continuation(monitor_continuation(self._monitor, continuation)) try: self._native.continue_monitor( cast("dict[str, object]", held["record"]), @@ -1615,7 +1568,8 @@ def _tally( if event.kind == _PROGRESS: # The native session's yield (`Session::advance`): a full chunk buffer # first, then the attempt bound, else the time bound. - if attempts >= ATTEMPT_CAPACITY or rows >= ROW_CAPACITY: + planes = self._records.planes + if attempts >= planes.attempt_capacity or rows >= planes.row_capacity: name += ":buffer" elif attempts >= limit: name += ":work_limit" @@ -1711,8 +1665,8 @@ def _event( # A boundary warning (SEP 47 section 5.5): the caller emits it, and its # filters may raise it; the committed field and the chunk are there for # that failure's checkpoint. - notice = _watch.Notice.of(event.notice) - if notice.kind is _watch.NoticeKind.REMOVAL: + notice = _watch.notice_of(event.notice) + if notice.kind is NoticeKind.REMOVAL: self._log_finish() warning = self._warning(event.notice) return ServiceRequired( @@ -1823,10 +1777,10 @@ def _warning(self, native: Mapping[str, object]) -> Warning: """ monitor = self._monitor assert monitor is not None - notice = _watch.Notice.of(native) - if notice.kind is _watch.NoticeKind.TRANSVERSE: + notice = _watch.notice_of(native) + if notice.kind is NoticeKind.TRANSVERSE: assert monitor.transverse is not None - edge = _transverse.EdgeNotice.of( + edge = _transverse.edge_notice_of( cast("Mapping[str, object]", native["edge"]) ) return _transverse.notice_warning(monitor.transverse, edge) @@ -1874,14 +1828,14 @@ def _reported_error( """ if outcome is Outcome.STOPPED: return StopRequested(self._stop) - notice = _watch.Notice.of(event.notice) - if notice.kind is _watch.NoticeKind.TRANSVERSE and self._monitor is not None: + notice = _watch.notice_of(event.notice) + if notice.kind is NoticeKind.TRANSVERSE and self._monitor is not None: assert self._monitor.transverse is not None - edge = _transverse.EdgeNotice.of( + edge = _transverse.edge_notice_of( cast("Mapping[str, object]", event.notice["edge"]) ) return _transverse.notice_error(self._monitor.transverse, edge) - if notice.kind is not _watch.NoticeKind.NONE and self._monitor is not None: + if notice.kind is not NoticeKind.NONE and self._monitor is not None: return _watch.notice_error(self._monitor.check, notice) if event.schedule_exhausted: schedule = self._stepper.schedule @@ -1944,7 +1898,8 @@ class _NativeRungeKuttaMarch: The adaptive loop -- the landing candidate, the trial on the staged rows, the equivalent norm and acceptance, the start-up rule, the next proposal or the rejection's, and the boundary checks -- runs natively in the session from one - policy record (the kernel contract's section 12), which this hands the native + policy record (the kernel contract's section 12, + :class:`~solprop.backends.records.RungeKuttaPolicy`), which this hands the native state once. """ @@ -1953,58 +1908,26 @@ def __init__(self, backend: Any) -> None: self._driver: Any = backend def _policy(self, interval: float | None = None) -> dict[str, object]: - """Return the run's controller and landing record (kernel contract 12). + """Return the backend's controller and landing record (kernel contract 12). - ``interval``: a resting collar's quiet interval, m; ``None`` for a session, whose native monitor holds its own. """ backend = self._driver - stepper = backend._stepper - controller = stepper.controller - breakpoints = np.asarray(backend._problem.linear.phi_breakpoints, np.float64) - if not stepper.interval_cap: - breakpoints = breakpoints[[0, -1]] - return { - "dz_min": float(stepper.dz_min), - "dz_max": float(stepper.dz_max), - "stretch": float(stepper.stretch), - "z_end": float(stepper.z_end), - # Interpolated planes are stepped across (SEP 14 section 4.3). - "save_planes": [] - if stepper.interpolate_planes - else [float(z) for z in stepper.save_planes], - "interpolate": bool(stepper.interpolate_planes), - "hard_kinks": [float(z) for z in backend._problem.linear.hard_kinks], - "breakpoints": [float(z) for z in breakpoints], - "landing_ulps": LANDING_ULPS, - "quiet_interval": interval, - "safety": float(controller.safety), - "beta1": float(controller.beta1), - "beta2": float(controller.beta2), - "min_factor": float(controller.min_factor), - "max_factor": float(controller.max_factor), - "lag": int(controller.lag), - "reject_free": bool(controller.reject_free), - "power": float(controller.estimator_order + 1), - "exponent": 1.0 / (controller.estimator_order + 1), - "start_growth": float(START_GROWTH), - "auto_init": bool(stepper.auto_init), - } + policy = runge_kutta_policy(backend._problem, backend._stepper) + return _records.rk_policy(dataclasses.replace(policy, quiet_interval=interval)) - def _configure(self) -> None: + def _configure( + self, policy: RungeKuttaPolicy, schedule: FixedSchedule | None + ) -> None: """Hand the native state the run's policy and a replay's schedule.""" backend = self._driver - stepper = backend._stepper - controller = stepper.controller + controller = policy.controller if controller.weights is not None: backend._rk_weights_ready() - schedule = stepper.schedule - steps = edges = _NO_SCHEDULE - if schedule is not None: - steps = np.ascontiguousarray(schedule.steps, dtype=np.float64) - edges = np.ascontiguousarray(schedule.edges, dtype=np.float64) + steps, edges = _records.schedule_arrays(schedule) backend._native_state.rk_march_configure( - self._policy(), + _records.rk_policy(policy), list(backend._rk_nodes), float(controller.rtol), float(controller.atol), diff --git a/native/tests/plasma_bounds.py b/native/tests/plasma_bounds.py index c27955c5e..a0b2882b2 100644 --- a/native/tests/plasma_bounds.py +++ b/native/tests/plasma_bounds.py @@ -241,7 +241,7 @@ def node_field_error( # within fft_bound of the field's 2-norm over every node and sample. size = grids.n_to * transform.n_nodes return np.full(fields.shape, fft_bound(size) * float(np.linalg.norm(fields))) - state = reference._require_state()[0] # pyright: ignore[reportPrivateUsage] + state = reference._require_state() # pyright: ignore[reportPrivateUsage] kind = reference._kind # pyright: ignore[reportPrivateUsage] bundle = reference._stage_scalars(0.0, 0.0) # pyright: ignore[reportPrivateUsage] taper = float(bundle.taper_scale) @@ -414,11 +414,11 @@ def plasma_stage_bound( state = reference._require_state() # pyright: ignore[reportPrivateUsage] bundle = reference._stage_scalars(0.0, 0.0) # pyright: ignore[reportPrivateUsage] stage = reference._nonlinear( # pyright: ignore[reportPrivateUsage] - state, bundle, np.ones((1, *problem.shape), complex) - )[0] + state, bundle, np.ones(problem.shape, complex) + ) fields = np.real( np.concatenate( - list(reference._plasma_nodes(state[0])) # pyright: ignore[reportPrivateUsage] + list(reference._plasma_nodes(state)) # pyright: ignore[reportPrivateUsage] ) ) field_error = node_field_error(reference, problem, fields) diff --git a/native/tests/test_backend.py b/native/tests/test_backend.py index 759a1c44e..bc2c258d2 100644 --- a/native/tests/test_backend.py +++ b/native/tests/test_backend.py @@ -891,8 +891,8 @@ def test_packed_real_blas_radial_envelope_matches_the_reference_when_eligible() reference._nonlinear( # pyright: ignore[reportPrivateUsage] reference._require_state(), # pyright: ignore[reportPrivateUsage] reference._stage_scalars(0.0, 0.0), # pyright: ignore[reportPrivateUsage] - np.ones((1, *problem.shape), complex), - )[0] + np.ones(problem.shape, complex), + ) ) finally: reference.close() diff --git a/native/tests/test_band_column_budget.py b/native/tests/test_band_column_budget.py index cd5f660fe..57c693b5a 100644 --- a/native/tests/test_band_column_budget.py +++ b/native/tests/test_band_column_budget.py @@ -231,8 +231,8 @@ def stages( expected = reference._nonlinear( # pyright: ignore[reportPrivateUsage] reference._require_state(), # pyright: ignore[reportPrivateUsage] reference._stage_scalars(0.0, 0.0), # pyright: ignore[reportPrivateUsage] - np.ones((1, *problem.shape), complex), - )[0] + np.ones(problem.shape, complex), + ) finally: reference.close() backend = NativeBackend(problem, run_request, grid=GRID) @@ -280,7 +280,7 @@ def launch_fraction_bound( bundle = reference._stage_scalars(0.0, 0.0) # pyright: ignore[reportPrivateUsage] fields = np.real( np.concatenate( - list(reference._plasma_nodes(state[0])) # pyright: ignore[reportPrivateUsage] + list(reference._plasma_nodes(state)) # pyright: ignore[reportPrivateUsage] ) ) field_error = node_field_error(reference, problem, fields) diff --git a/native/tests/test_band_pathological.py b/native/tests/test_band_pathological.py index ea94bdb29..29d4fc6a0 100644 --- a/native/tests/test_band_pathological.py +++ b/native/tests/test_band_pathological.py @@ -172,7 +172,7 @@ def synthesised( state = reference._require_state() # pyright: ignore[reportPrivateUsage] return np.real( np.concatenate( - list(reference._plasma_nodes(state[0])) # pyright: ignore[reportPrivateUsage] + list(reference._plasma_nodes(state)) # pyright: ignore[reportPrivateUsage] ) ) finally: @@ -190,8 +190,8 @@ def source_of_reference( return reference._nonlinear( # pyright: ignore[reportPrivateUsage] reference._require_state(), # pyright: ignore[reportPrivateUsage] reference._stage_scalars(0.0, 0.0), # pyright: ignore[reportPrivateUsage] - np.ones((1, *problem.shape), complex), - )[0] + np.ones(problem.shape, complex), + ) finally: reference.close() diff --git a/native/tests/test_band_plasma.py b/native/tests/test_band_plasma.py index 8aa185cb4..76eaa583b 100644 --- a/native/tests/test_band_plasma.py +++ b/native/tests/test_band_plasma.py @@ -437,8 +437,8 @@ def test_one_evaluation_matches_the_reference() -> None: expected = reference._nonlinear( # pyright: ignore[reportPrivateUsage] reference._require_state(), # pyright: ignore[reportPrivateUsage] reference._stage_scalars(0.0, 0.0), # pyright: ignore[reportPrivateUsage] - np.ones((1, *problem.shape), complex), - )[0] + np.ones(problem.shape, complex), + ) finally: reference.close() backend = NativeBackend(problem, run_request, grid=grid) @@ -485,7 +485,7 @@ def test_every_stage_of_a_trial_matches_the_reference( reference.set_initial(problem.initial.field0, None) reference._trial_step(0.0, dz) # pyright: ignore[reportPrivateUsage] - oracle expected = [ - np.asarray(stage)[0] + np.asarray(stage) for stage in reference._last_stages # pyright: ignore[reportPrivateUsage] ] finally: diff --git a/native/tests/test_band_statistics_per_node.py b/native/tests/test_band_statistics_per_node.py index 708c63609..48cf0b901 100644 --- a/native/tests/test_band_statistics_per_node.py +++ b/native/tests/test_band_statistics_per_node.py @@ -146,7 +146,7 @@ def fields_of( state = reference._require_state() # pyright: ignore[reportPrivateUsage] return np.real( np.concatenate( - list(reference._plasma_nodes(state[0])) # pyright: ignore[reportPrivateUsage] + list(reference._plasma_nodes(state)) # pyright: ignore[reportPrivateUsage] ) ) finally: @@ -174,7 +174,7 @@ def node_fields_and_error( reference._hoist(0.0) # pyright: ignore[reportPrivateUsage] - oracle probe fields = np.real( np.concatenate( - list(reference._plasma_nodes(state[0])) # pyright: ignore[reportPrivateUsage] + list(reference._plasma_nodes(state)) # pyright: ignore[reportPrivateUsage] ) ) return fields, node_field_error(reference, problem, fields) diff --git a/native/tests/test_band_stress.py b/native/tests/test_band_stress.py index 72ee4ea49..69af69c62 100644 --- a/native/tests/test_band_stress.py +++ b/native/tests/test_band_stress.py @@ -201,7 +201,7 @@ def node_fields( reference.set_initial(problem.initial.field0, None) state = reference._require_state() # pyright: ignore[reportPrivateUsage] return np.concatenate( - list(reference._plasma_nodes(state[0])) # pyright: ignore[reportPrivateUsage] + list(reference._plasma_nodes(state)) # pyright: ignore[reportPrivateUsage] ) finally: reference.close() @@ -277,8 +277,8 @@ def test_one_evaluation_matches_the_reference(kind: str) -> None: expected = reference._nonlinear( # pyright: ignore[reportPrivateUsage] reference._require_state(), # pyright: ignore[reportPrivateUsage] reference._stage_scalars(0.0, 0.0), # pyright: ignore[reportPrivateUsage] - np.ones((1, *problem.shape), complex), - )[0] + np.ones(problem.shape, complex), + ) finally: reference.close() backend = NativeBackend(problem, run_request, grid=grid) @@ -333,7 +333,7 @@ def test_the_launch_exposure_is_within_its_scan_bound(kind: str) -> None: bundle = reference._stage_scalars(0.0, 0.0) # pyright: ignore[reportPrivateUsage] fields = np.real( np.concatenate( - list(reference._plasma_nodes(state[0])) # pyright: ignore[reportPrivateUsage] + list(reference._plasma_nodes(state)) # pyright: ignore[reportPrivateUsage] ) ) field_error = node_field_error(reference, problem, fields) diff --git a/native/tests/test_dispersive.py b/native/tests/test_dispersive.py index 5a4232550..d14803a95 100644 --- a/native/tests/test_dispersive.py +++ b/native/tests/test_dispersive.py @@ -326,7 +326,7 @@ def test_one_evaluation_matches_the_reference(evaluator: str, thg: bool) -> None spectra.append(_testing.projected_to_host(state, False)) else: backend.set_initial(problem.initial.field0, None) - backend._adams_lab_source(problem.initial.field0[None], 0.0) # pyright: ignore[reportPrivateUsage] + backend._adams_lab_source(problem.initial.field0, 0.0) # pyright: ignore[reportPrivateUsage] projected = np.asarray(backend._last_projected) # pyright: ignore[reportPrivateUsage] spectra.append(np.asarray(np.fft.rfft(projected, axis=-1))) finally: diff --git a/native/tests/test_envelope_plasma.py b/native/tests/test_envelope_plasma.py index 23eb591ee..28af8e818 100644 --- a/native/tests/test_envelope_plasma.py +++ b/native/tests/test_envelope_plasma.py @@ -181,10 +181,10 @@ def test_envelope_current_streams_use_large_radial_projection() -> None: try: reference.set_initial(problem.initial.field0, None) expected = reference._nonlinear( # pyright: ignore[reportPrivateUsage] - problem.initial.field0[None], + problem.initial.field0, reference._stage_scalars(0.0, 0.0), # pyright: ignore[reportPrivateUsage] - np.ones((1, *problem.shape), complex), - )[0] + np.ones(problem.shape, complex), + ) state = load_native(actual, problem.initial.field0, clear_fsal=False) native_trial(actual, 0.0, 0.0) observed = np.asarray(_testing.stage_to_host(state, 0)).reshape(problem.shape) diff --git a/native/tests/test_envelope_subnormal.py b/native/tests/test_envelope_subnormal.py index 271468fbd..627f201d1 100644 --- a/native/tests/test_envelope_subnormal.py +++ b/native/tests/test_envelope_subnormal.py @@ -131,8 +131,8 @@ def evaluate( stage = backend._nonlinear( # pyright: ignore[reportAttributeAccessIssue, reportPrivateUsage, reportUnknownMemberType, reportUnknownVariableType] backend._require_state(), # pyright: ignore[reportArgumentType, reportPrivateUsage] backend._stage_scalars(0.0, 0.0), # pyright: ignore[reportPrivateUsage] - np.ones((1, *problem.shape), complex), - )[0] + np.ones(problem.shape, complex), + ) return stage, statistics # pyright: ignore[reportUnknownVariableType] finally: backend.close() @@ -147,7 +147,7 @@ def node_magnitude( reference.set_initial(problem.initial.field0, None) state = reference._require_state() # pyright: ignore[reportPrivateUsage] nodes = np.concatenate( - list(reference._plasma_nodes(state[0])) # pyright: ignore[reportPrivateUsage] + list(reference._plasma_nodes(state)) # pyright: ignore[reportPrivateUsage] ) return float(np.max(np.abs(nodes))) finally: diff --git a/native/tests/test_radial.py b/native/tests/test_radial.py index c2e793747..2bfcebda4 100644 --- a/native/tests/test_radial.py +++ b/native/tests/test_radial.py @@ -113,8 +113,8 @@ def test_radial_partial_chunks_preserve_the_reference_source( reference._nonlinear( # pyright: ignore[reportPrivateUsage] reference._require_state(), # pyright: ignore[reportPrivateUsage] reference._stage_scalars(0.0, 0.0), # pyright: ignore[reportPrivateUsage] - np.ones((1, *problem.shape), complex), - )[0] + np.ones(problem.shape, complex), + ) ) finally: reference.close() diff --git a/src/solprop/backends/engines.py b/src/solprop/backends/engines.py new file mode 100644 index 000000000..8de004f78 --- /dev/null +++ b/src/solprop/backends/engines.py @@ -0,0 +1,357 @@ +# SPDX-FileCopyrightText: 2026 John C. Travers and contributors +# +# SPDX-License-Identifier: GPL-3.0-or-later + +"""The engine protocols: what a march asks of the state it advances (SEP 52 §5.2). + +A march owns its controller, landing, history and call ledger; an engine does every +state-sized operation a march asks of the workspace that holds the state. The two +protocols here mirror the native marches' C++ interfaces one-to-one, every argument +included: :class:`RungeKuttaEngine` is ``rk::Engine`` +(``native/include/solprop_native/rk_march.hpp``), which the low-storage step implements +too, and :class:`AdamsEngine` is ``march::Engine`` +(``native/include/solprop_native/adams_march.hpp``). They are the target the +reference's engines take in SEP 52's phase 3; until then its Adams march reads +:class:`solprop.backends.numpy_ref.multistep.AdamsEngine` and its low-storage march +:class:`solprop.backends.low_storage.LowStorageEngine`. + +**The commit contract.** + +- **The Runge-Kutta trial** keeps its stages and end state in the workspace. It commits + nothing until ``accept``, and a rejected trial leaves the committed state as it was. +- **The low-storage step is the same engine, with one documented difference.** Its + trial overwrites the state, the register ``A``. From the start of ``trial`` until + ``accept`` returns, ``in_flight`` is true. An exception or a non-finite result in that + interval leaves no committed state at that position, so the session ends the run + failed with no checkpoint there. The engine owns the in-flight flag and the finish; + the march owns the committed position. The stiffness diagnostic is the engine's, a + field of its :class:`AcceptRecord` (NaN where the engine does not form it). The march + reads it from there, and the session reads the march's state, never a private field. +- **The Adams accept** receives the retained history entries, oldest first, which the + interaction formulation's transport uses. +- **step_carrier** returns the last trial's carrier rotations. The march stores them in + the accepted step's record, and the session's dense observation applies them to the + interpolation weights. +- **The dense hold, the dense observation and the snapshot rows** are the session's; no + engine holds, restores or observes. +- **The TDSE reservation and the trial's outcome** are the engine's, as + ``begin_attempt`` and ``end_attempt``. +- **The split collar's sums.** An accepted step's finish computes them in the + workspace, from the masked field before the split, and returns them in its + :class:`AcceptRecord`; the session accumulates and drains them. The removal adds over + every accepted step since the previous check, and the next boundary reading drains + it; the found occupancies are those of the latest accepted step; a rejected trial, a + dense observation and a probe trial contribute nothing, because none of them finishes + a step; loading a state empties the accumulator, and attaching a monitor, at a launch + or a resume, starts it from zero. + +A march holds its own state and touches no other object's fields. +""" + +import enum +import math +from collections.abc import Sequence +from dataclasses import dataclass +from typing import Literal, Protocol + +import numpy as np +from jaxtyping import Float + +from solprop.backends.adams_core import Stencil + +__all__ = [ + "AcceptRecord", + "AdamsEngine", + "AttemptOutcome", + "BoundaryReading", + "HistoryEntry", + "RungeKuttaEngine", +] + + +class AttemptOutcome(enum.IntEnum): + """How an attempt ended, for a host source's reservation (``HostOutcome``, SEP 49). + + Attributes + ---------- + REJECTED : int + The trial was rejected. + ACCEPTED : int + The trial was accepted and committed. + ABANDONED : int + The attempt ended without a decision: a failure or a stop. + """ + + REJECTED = 0 + ACCEPTED = 1 + ABANDONED = 2 + + +@dataclass(frozen=True) +class HistoryEntry: + """One entry of an Adams march's history (``march::Entry``). + + Attributes + ---------- + z : float + Where its sample was evaluated, m. + identity : {"accepted", "predicted"} + Whether the sample is the accepted state's, or the predicted state's that PEC + stores in its place. + row : int + The engine's history row that holds it. + """ + + z: float + identity: Literal["accepted", "predicted"] + row: int + + +@dataclass(frozen=True, eq=False) +class BoundaryReading: + """The boundary monitor's reading of the committed state (``monitor_reading``). + + The state's own sums (:class:`solprop.monitor.BoundaryEnergies`, less the split + collar's sums, which travel in each step's :class:`AcceptRecord`). Equality is by + identity (``eq=False``) because it holds arrays. + + Attributes + ---------- + total : float + The state's energy, ``sum_t P(t)``, in the proxy's units of energy. + regions : numpy.ndarray + Shape ``(4,)``: the low guard's, the high guard's and the collar's weighted + energies, and the collar's removal rate, energy per metre. + occupancy : numpy.ndarray or None + The state's energy per walk-off bucket of the split collar's crossing limit; + ``None`` without one. + """ + + total: float + regions: Float[np.ndarray, " 4"] + occupancy: Float[np.ndarray, " n_buckets"] | None = None + + +@dataclass(frozen=True, eq=False) +class AcceptRecord: + """What an accepted step's finish reports (SEP 52 §5.2). + + Equality is by identity (``eq=False``) because it holds an array. + + Attributes + ---------- + stiffness : float + The low-storage step's stiffness indicator ``h rho``, dimensionless (SEP 24 + section 6.3); NaN where the engine does not form it, on every other engine and + on the native low-storage engine. + removed : float + What the split temporal collar took at this step, from the masked field before + the split, in the proxy's units of energy; 0.0 under the in-stage scheme or + without a collar. + found : numpy.ndarray or None + Shape ``(4,)``: the energy and the first three weighted energies of the power + the step's split found, taken halfway through its factor; ``None`` under the + in-stage scheme or without a collar. + """ + + stiffness: float = math.nan + removed: float = 0.0 + found: Float[np.ndarray, " 4"] | None = None + + +class RungeKuttaEngine(Protocol): + """What a Runge-Kutta march asks of its engine (``rk::Engine``). + + The embedded Runge-Kutta trial and, with the documented difference of the module's + commit contract, the low-storage step. + """ + + def trial(self, z: float, dz: float) -> float: + """Form a trial from `z` over `dz`; return its scaled error norm. + + The low-storage step overwrites the state, and returns 0.0 ("not estimated"), + or a non-finite value when the field diverged. + + Parameters + ---------- + z : float + Where the trial starts, m. + dz : float + The step, m, positive. + + Returns + ------- + float + The scaled error norm. + """ + ... + + def accept(self, dz: float) -> AcceptRecord: + """Adopt the trial: promote its recycled stage, then finish and commit it. + + Parameters + ---------- + dz : float + The accepted step, m. + + Returns + ------- + AcceptRecord + The finish's stiffness diagnostic and split collar's sums. + """ + ... + + def extend(self, z: float, dz: float) -> None: + """Hold the accepted trial from `z` over `dz` as a dense extension. + + Called before the trial is adopted, for the session's dense observation. + + Parameters + ---------- + z : float + Where the trial started, m. + dz : float + The step, m. + """ + ... + + def reading(self) -> BoundaryReading: + """Return the committed state's boundary reading. + + Returns + ------- + BoundaryReading + The reading. + """ + ... + + @property + def in_flight(self) -> bool: # numpydoc ignore=RT01 + """Whether a step that overwrites the state in place was left partway.""" + ... + + +class AdamsEngine(Protocol): + """What an Adams march asks of its engine (``march::Engine``). + + The engine keeps the history's rows where the state lives; the march keeps the + entries (:class:`HistoryEntry`) and passes them in. + """ + + def start(self, z: float) -> None: + """Evaluate the history's first sample at the accepted state at `z` into row 0. + + Parameters + ---------- + z : float + The accepted position, m. + """ + ... + + def trial( + self, z: float, dz: float, stencil: Stencil + ) -> Float[np.ndarray, " n_orders"]: + """Form a trial from `z` over `dz`; return its scored orders' norms. + + Parameters + ---------- + z : float + Where the trial starts, m. + dz : float + The step, m. + stencil : solprop.backends.adams_core.Stencil + The history's nodes and rows, newest first, and the orders scored. + + Returns + ------- + numpy.ndarray + One scaled norm per scored order, the step's own first. + """ + ... + + def accept( + self, dz: float, free: int, stencil: Stencil, kept: Sequence[HistoryEntry] + ) -> AcceptRecord: + """Adopt the held trial; the new sample takes row `free`. + + Parameters + ---------- + dz : float + The accepted step, m. + free : int + The history row the new sample takes. + stencil : solprop.backends.adams_core.Stencil + The trial's stencil. + kept : collections.abc.Sequence of HistoryEntry + The history the step keeps, oldest first, which the interaction + formulation transports. + + Returns + ------- + AcceptRecord + The finish's split collar's sums. + """ + ... + + def chord_phase(self, z1: float, z2: float) -> float: + """Return the carrier chord's largest phase over ``[z1, z2]``. + + Parameters + ---------- + z1, z2 : float + The interval, m. + + Returns + ------- + float + The phase, rad. + """ + ... + + def step_carrier(self, order: int) -> Float[np.ndarray, " n_rotations"]: + """Return the last trial's carrier rotations over its `order` newest rows. + + Parameters + ---------- + order : int + The rows the dense observation's formula reads. + + Returns + ------- + numpy.ndarray + The shift's imaginary part, then each row's rotation as its real and + imaginary parts; empty without a carrier frame. + """ + ... + + def begin_attempt(self, cost: int) -> None: + """Start an attempt: a host source's reservation (SEP 49 section 6.2). + + Called after the budget check and before the attempt's first evaluation. + + Parameters + ---------- + cost : int + The attempt's logical response calls. + """ + ... + + def end_attempt(self, outcome: AttemptOutcome) -> None: + """End an attempt: a host source's outcome. + + Parameters + ---------- + outcome : AttemptOutcome + How it ended. + """ + ... + + def reading(self) -> BoundaryReading: + """Return the committed state's boundary reading. + + Returns + ------- + BoundaryReading + The reading. + """ + ... diff --git a/src/solprop/backends/exponential.py b/src/solprop/backends/exponential.py index 4860f9e59..b9d79c889 100644 --- a/src/solprop/backends/exponential.py +++ b/src/solprop/backends/exponential.py @@ -316,28 +316,27 @@ class CarrierChord: """The reference element's cumulative operator alone (SEP 35 sections 6.1, 6.4). The operator's own representation restricted to column 0, polarisation 0 and the - reference bin, every batch member kept: its chord over a step is the reference - element's exponent, to rounding the one the backend forms, at one scalar per - member. The step cap reads it for every proposal, and a backend that forms its - exponent elsewhere (the native engine, on a device too) takes its shift from it - with nothing read back. + reference bin: its chord over a step is the reference element's exponent, to + rounding the one the backend forms, at one scalar. The step cap reads it for every + proposal, and a backend that forms its exponent elsewhere (the native engine, on a + device too) takes its shift from it with nothing read back. Attributes ---------- constant : numpy.ndarray or None - ``D`` at the element, 1/m, shape ``(n_batch_c,)``, for a constant operator. + ``D`` at the element, 1/m, shape ``()``, for a constant operator. expansion : solprop.translate.expansion.ParameterExpansion or None The expansion with its terms at the element, for an expansion. table : solprop.translate.chebyshev.PiecewiseChebyshev or None The z table at the element, for a table. """ - constant: Complex[np.ndarray, " n_batch_c"] | None = None + constant: Complex[np.ndarray, ""] | None = None expansion: ParameterExpansion | None = None table: PiecewiseChebyshev | None = None - def exponent(self, z1: float, z2: float) -> Complex[np.ndarray, " n_batch_c"]: - """Return ``Phi(z2) - Phi(z1)`` at the element, per batch member. + def exponent(self, z1: float, z2: float) -> Complex[np.ndarray, ""]: + """Return ``Phi(z2) - Phi(z1)`` at the element. Parameters ---------- @@ -347,10 +346,10 @@ def exponent(self, z1: float, z2: float) -> Complex[np.ndarray, " n_batch_c"]: Returns ------- numpy.ndarray - Dimensionless, shape ``(n_batch_c,)``. + Dimensionless, shape ``()``. """ if self.constant is not None: - return self.constant * (z2 - z1) + return np.asarray(self.constant * (z2 - z1)) if self.expansion is not None: return np.asarray(self.expansion.delta(z1, z2), dtype=np.complex128) assert self.table is not None @@ -375,14 +374,14 @@ def carrier_chord(linear: LinearOperator, index: int) -> CarrierChord | None: """ def _element(values: np.ndarray) -> np.ndarray: - """Return ``values[..., b, 0, 0, index]`` over its leading axes and batch. + """Return ``values[..., 0, 0, index]`` over its leading axes. - - ``values``: a representation's array, its last four axes the state's layout. + - ``values``: a representation's array, its last three axes the state's layout. - Returns: column 0, polarisation 0 and the reference bin, every leading axis kept. """ column = min(index, values.shape[-1] - 1) - return np.ascontiguousarray(values[..., 0, 0, column]) + return np.array(values[..., 0, 0, column], order="C", copy=True) if linear.phi_constant is not None: return CarrierChord(constant=_element(linear.phi_constant)) @@ -403,7 +402,7 @@ def _element(values: np.ndarray) -> np.ndarray: class CarrierReference: """Where a run's carrier frame takes its shift, fixed for the run (SEP 35 6.1). - The shift of batch member ``b`` is ``sigma_b = i Im x[b, 0, 0, bin]``, ``x`` the + The shift is ``sigma = i Im x[0, 0, bin]``, ``x`` the trial's own exponent ``Phi(z + h) - Phi(z)``: column 0 is the frame's mode (``build_phi`` fixes the frame on the first mode; a coupled route's carrier is the same in every channel), polarisation 0. The change of variable is exact for any @@ -440,9 +439,9 @@ def active(self) -> bool: # numpydoc ignore=RT01 return self.bin is not None def shift( - self, exponent: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"] - ) -> Complex[np.ndarray, "n_batch 1 1 1"]: - """Return each batch member's shift for a trial's exponent. + self, exponent: Complex[np.ndarray, "n_modes n_pol n_omega"] + ) -> Complex[np.ndarray, "1 1 1"]: + """Return the shift for a trial's exponent. Parameters ---------- @@ -452,8 +451,7 @@ def shift( Returns ------- numpy.ndarray - Purely imaginary, shape ``(n_batch, 1, 1, 1)``, broadcasting against the - state. + Purely imaginary, shape ``(1, 1, 1)``, broadcasting against the state. Raises ------ @@ -464,13 +462,11 @@ def shift( msg = f"the carrier frame is inert for this run: {self.reason}" raise BackendError(msg) # The real part (loss) is never shifted (SEP 35 section 10). - phase = np.imag(exponent[:, 0, 0, self.bin]).reshape((-1, 1, 1, 1)) + phase = np.imag(exponent[0, 0, self.bin]).reshape((1, 1, 1)) return 1j * phase - def phase( - self, exponent: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"] - ) -> float: - """Return the largest carrier phase over the batch, for the step cap. + def phase(self, exponent: Complex[np.ndarray, "n_modes n_pol n_omega"]) -> float: + """Return the carrier phase, for the step cap. Parameters ---------- @@ -480,14 +476,14 @@ def phase( Returns ------- float - ``max_b |Im x[b, 0, 0, bin]|``, rad; 0 when the policy is inert. + ``|Im x[0, 0, bin]|``, rad; 0 when the policy is inert. """ if self.bin is None: return 0.0 - return float(np.max(np.abs(np.imag(exponent[:, 0, 0, self.bin])))) + return float(np.abs(np.imag(exponent[0, 0, self.bin]))) def chord_phase(self, z1: float, z2: float) -> float: - """Return the largest carrier phase over the batch from the chord alone. + """Return the carrier phase from the chord alone. Parameters ---------- @@ -497,7 +493,7 @@ def chord_phase(self, z1: float, z2: float) -> float: Returns ------- float - ``max_b |Im (Phi(z2) - Phi(z1))|`` at the reference element, rad; 0 when + ``|Im (Phi(z2) - Phi(z1))|`` at the reference element, rad; 0 when the policy is inert. Raises @@ -510,7 +506,7 @@ def chord_phase(self, z1: float, z2: float) -> float: if self.chord is None: msg = "the carrier frame's reference has no chord on this operator" raise BackendError(msg) - return float(np.max(np.abs(np.imag(self.chord.exponent(z1, z2))))) + return float(np.abs(np.imag(self.chord.exponent(z1, z2)))) def carrier_reference( diff --git a/src/solprop/backends/host/adams.py b/src/solprop/backends/host/adams.py index 4c5aa3c63..f202b5a0d 100644 --- a/src/solprop/backends/host/adams.py +++ b/src/solprop/backends/host/adams.py @@ -34,10 +34,11 @@ ExponentialKernels, HostHistory, ) +from solprop.backends.records import adams_policy from solprop.exceptions import BackendError, ConvergenceError, ResponseBudgetError from solprop.output.records import MarchSnapshot from solprop.stepper.controller import limit_step -from solprop.stepper.multistep import MarchPolicy, Multistep, march_policy +from solprop.stepper.multistep import MarchPolicy, Multistep from solprop.stepper.schedule import FixedSchedule, landing __all__ = ["HostAdamsMarch"] @@ -107,35 +108,15 @@ def __init__( self._order = 1 if method.variable_order else method.order self._ahead: tuple[float, object] | None = None self.ledger = CallLedger() - # An in-stage temporal collar adds -alpha(t) / 2 to the derivative: in the - # exponential formulation, whose linear flow is exact, it is the one stiff term - # the explicit history integrates, and its largest rate bounds each order's - # stable step, which the order choice weighs (`_choose_order`). - responses = backend._problem.responses - collar = responses.temporal_absorber - collar_rate = ( - 0.5 * float(np.max(collar)) - if collar is not None - and responses.absorber_scheme == "nonlinear" - and method.formulation == "exponential" - else 0.0 - ) # Every number the controller and the landings read, from one record (SEP 39 - # section 4.2); the decisions are `adams_core`'s functions of it. - settings = backend._stepper - linear = backend._problem.linear + # section 4.2); the decisions are `adams_core`'s functions of it. An in-stage + # temporal collar's largest rate, the one stiff term the exponential history + # integrates, bounds each order's stable step there (`_choose_order`). carrier = self._engine.carrier - self._policy: MarchPolicy = march_policy( + self._policy: MarchPolicy = adams_policy( + backend._problem, + backend._stepper, method, - dz_min=settings.dz_min, - dz_max=settings.dz_max, - stretch=settings.stretch, - z_end=settings.z_end, - save_planes=np.asarray(settings.save_planes, dtype=np.float64).tolist(), - hard_kinks=np.asarray(linear.hard_kinks, dtype=np.float64).tolist(), - breakpoints=np.asarray(linear.phi_breakpoints, dtype=np.float64).tolist(), - interpolate=self._interpolate, - collar_rate=collar_rate, carrier=isinstance(carrier, CarrierReference) and carrier.active, ) # Whether the stages left a quiet collar out at the last attempt: the step @@ -181,9 +162,7 @@ def snapshot(self) -> MarchSnapshot | None: backend = self._backend return MarchSnapshot( z=float(backend._z), - # The snapshot's state is the trajectory's own; the march holds it with a - # leading axis of one. - state=np.array(backend._adams_state()[0], dtype=np.complex128, copy=True), + state=np.array(backend._adams_state(), dtype=np.complex128, copy=True), proposal=float(backend._proposal), method=repr(self._method), order=int(self._order), @@ -236,14 +215,11 @@ def load(self, snapshot: MarchSnapshot) -> str | None: if reason is not None: return reason load_rows = self._engine.load_rows - # The engine's rows keep the march's layout, a leading axis of one beside - # their own (the snapshot's disk format, SEP 47 section 6.7). - shape = (1, *backend._problem.shape) try: - load_rows(snapshot.rows, shape) + load_rows(snapshot.rows, backend._problem.shape) except BackendError as error: return str(error) - backend._adams_restore(snapshot.z, np.array(snapshot.state, copy=True)[None]) + backend._adams_restore(snapshot.z, np.array(snapshot.state, copy=True)) self._history = [ Derivative(float(z), self._engine.value(int(slot)), identity, int(slot)) for z, identity, slot in zip( @@ -474,7 +450,7 @@ def _budget_error(self, cost: int) -> ResponseBudgetError: "the run stops at its last accepted step, unfinished" ) return ResponseBudgetError( - msg, z=backend._z, field=backend._adams_state()[0].copy(), calls=spent + msg, z=backend._z, field=backend._adams_state().copy(), calls=spent ) def _start(self) -> None: @@ -492,7 +468,7 @@ def _capped(self, proposal: float) -> float: Under a carrier frame no step covers more than :data:`~solprop.stepper.multistep.CARRIER_PHASE_CAP` of carrier phase (SEP 35 section 6.4), measured by one chord of the reference element's operator over - the proposal (:class:`CarrierChord`), the largest over the batch + the proposal (:class:`CarrierChord`) (:func:`~solprop.backends.adams_core.carrier_capped`). Applied before the landing rule, so a shortened step is not a landing; a fixed schedule's replay is not capped. diff --git a/src/solprop/backends/host/driver.py b/src/solprop/backends/host/driver.py index 5e3a3ad8d..0c406b8e0 100644 --- a/src/solprop/backends/host/driver.py +++ b/src/solprop/backends/host/driver.py @@ -46,7 +46,11 @@ StepSummary, TimingProbe, ) -from solprop.backends.rk_core import rk_policy +from solprop.backends.records import ( + RungeKuttaPolicy, + collar_rests, + runge_kutta_policy, +) from solprop.backends.session import ( ExecutionOptions, PreparedExecution, @@ -141,8 +145,7 @@ def append(self, z: float, step: int, values: Mapping[str, np.ndarray]) -> None: """Add one row. - ``z``: position the row was taken at, m. - - ``values``: what the backend's ``stats`` returned, keyed by name, without a - batch axis. + - ``values``: what the backend's ``stats`` returned, keyed by name. """ if self._capacity == 0: self._allocate(values) @@ -267,7 +270,7 @@ def field(self) -> Complex[np.ndarray, "n_modes n_pol n_omega"]: if self._trial is None: raise BackendError("take a trial before reading its field") state, length = self._trial - field = np.array(driver._observed_field(driver._finish(state, length))[0]) + field = np.array(driver._observed_field(driver._finish(state, length))) self._trial = None return field @@ -376,7 +379,7 @@ def __init__( self._profiles = profiles self._logger = logging.getLogger(type(self).__module__) self._state: _StateT | None = None - self._noise: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"] | None = None + self._noise: Complex[np.ndarray, "n_modes n_pol n_omega"] | None = None self._z = 0.0 self._proposal = self._stepper.dz_init self._clean_columns: Float[np.ndarray, " n_columns"] | None = None @@ -437,7 +440,7 @@ def _trial_step(self, z: float, dz: float) -> tuple[float, _StateT]: @abstractmethod def _apply_mask( - self, state: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"] + self, state: Complex[np.ndarray, "n_modes n_pol n_omega"] ) -> _StateT: """Apply the hard band mask to a state that has not been through a step. @@ -470,7 +473,7 @@ def _finish( """ del dz return self._apply_mask( - cast(Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], state) + cast(Complex[np.ndarray, "n_modes n_pol n_omega"], state) ) def _reserve_source_calls(self, count: int) -> None: @@ -487,14 +490,14 @@ def _release_source_calls(self) -> None: def _adams_source( self, - lab: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], + lab: Complex[np.ndarray, "n_modes n_pol n_omega"], anchor: float, position: float, *, purpose: TDSEPurpose = "predictor", ) -> tuple[ - Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], - Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], + Complex[np.ndarray, "n_modes n_pol n_omega"], + Complex[np.ndarray, "n_modes n_pol n_omega"], ]: """Evaluate the source once for the low-call Adams march (SEP 13, SEP 14). @@ -517,11 +520,11 @@ def _adams_source( def _adams_lab_source( self, - lab: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], + lab: Complex[np.ndarray, "n_modes n_pol n_omega"], position: float, *, purpose: TDSEPurpose = "predictor", - ) -> Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]: + ) -> Complex[np.ndarray, "n_modes n_pol n_omega"]: """Evaluate the source once, untransported, for exponential Adams (SEP 16). The complete source assembly ``F`` -- the nonlinear response, the run's noise @@ -562,7 +565,7 @@ def _adams_block(self, z: float, dz: float) -> object: def _adams_propagator( self, anchor: float, position: float - ) -> Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]: + ) -> Complex[np.ndarray, "n_modes n_pol n_omega"]: """Return the linear propagator of an Adams step, ``U(position, anchor)``. - ``anchor``, ``position``: the step's anchor and its end, m. @@ -572,7 +575,7 @@ def _adams_propagator( bundle = self._stage_scalars(anchor, position) return np.exp(self._delta_phi(bundle)) - def _adams_state(self) -> Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]: + def _adams_state(self) -> Complex[np.ndarray, "n_modes n_pol n_omega"]: """Return the accepted state as the Adams march reads it, a host array. The default is the protected state itself; a backend whose state lives @@ -580,23 +583,21 @@ def _adams_state(self) -> Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]: - Returns: the accepted state in the canonical layout. """ - return cast( - Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], self._require_state() - ) + return cast(Complex[np.ndarray, "n_modes n_pol n_omega"], self._require_state()) def _adams_adopt( - self, advanced: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], dz: float - ) -> Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]: + self, advanced: Complex[np.ndarray, "n_modes n_pol n_omega"], dz: float + ) -> Complex[np.ndarray, "n_modes n_pol n_omega"]: """Adopt an accepted Adams step: its boundary treatments, then the state. - ``dz``: the step, m, which the split-step absorber needs. """ finished = self._finish(cast(_StateT, advanced), dz) self._state = finished - return cast(Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], finished) + return cast(Complex[np.ndarray, "n_modes n_pol n_omega"], finished) def _adams_restore( - self, z: float, state: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"] + self, z: float, state: Complex[np.ndarray, "n_modes n_pol n_omega"] ) -> None: """Restore an accepted Adams state after interpolation or a failed correction. @@ -607,7 +608,7 @@ def _adams_restore( def _observed_field( self, state: _StateT - ) -> Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]: + ) -> Complex[np.ndarray, "n_modes n_pol n_omega"]: """Return an owning host view of a backend's protected state. Array-backed CPU implementations use the identity. A device backend overrides @@ -616,7 +617,7 @@ def _observed_field( - Returns: an owning host ``complex128`` field in canonical layout. """ - return cast(Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], state) + return cast(Complex[np.ndarray, "n_modes n_pol n_omega"], state) @abstractmethod def stats(self, spec: StatsSpec) -> dict[str, np.ndarray]: @@ -635,7 +636,7 @@ def stats(self, spec: StatsSpec) -> dict[str, np.ndarray]: Returns ------- dict - Small host arrays keyed by statistic name, without a batch axis. + Small host arrays keyed by statistic name. """ def _on_accepted_step(self) -> None: # noqa: B027 - optional by design @@ -968,7 +969,7 @@ def _stage_scalars(self, z0: float, z1: float) -> StageScalars: def _delta_phi( self, bundle: StageScalars - ) -> Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]: + ) -> Complex[np.ndarray, "n_modes n_pol n_omega"]: """Return the stage's propagator exponent from the bundle. One linear combination of the terms this backend holds over the bundle's @@ -997,7 +998,7 @@ def _prefactor( def _unitary_phase( self, bundle: StageScalars - ) -> Float[np.ndarray, "n_batch n_modes n_pol n_omega"]: + ) -> Float[np.ndarray, "n_modes n_pol n_omega"]: """Return ``Im Phi`` at the stage's position, from the same terms. An expansion of ``Phi`` is cumulative and vanishes at the start of the range, @@ -1081,9 +1082,6 @@ def set_initial( f"{self._problem.shape} from the contract" ) raise BackendError(msg) - # The march holds its state with a leading axis of one, privately: the layout - # its kernels and the native build's read (SEP 47 section 6.2). - values = values[np.newaxis] position = self._checked_position(float(z)) drawn = self._noise_field(noise) model = self._problem.initial.noise_model @@ -1096,7 +1094,7 @@ def set_initial( ): if transverse_of(self._problem.route) is not None: self._clean_columns = _clean_launch_columns( - values[0], drawn[0], self._problem.grids.active_mask + values, drawn, self._problem.grids.active_mask ) values = values + drawn self._state = self._apply_mask(values) @@ -1331,7 +1329,7 @@ def observe(self, *, field: bool = True) -> Observation: hold(whole) plane = Observation( z=self._z, - field=None if whole is None else whole[0], + field=whole, noise_phase=self._noise_phase() if field else None, step_history=history, n_steps=history.n_steps, @@ -1448,31 +1446,25 @@ def _march_to(self, target: float) -> None: - ``target``: position to reach, m. """ - settings = self._stepper policy = self._controller_policy() - dense = settings.interpolate_planes + dense = policy.interpolate if dense and self._dense_ahead is not None: # The march is ahead of the last interpolated plane; resume it there. self._adams_restore(*self._dense_ahead) self._dense_ahead = None if self._natural_stop_due(target): return - breakpoints = self._problem.linear.phi_breakpoints - if not settings.interval_cap: - # The cumulative table makes the flow exact across its intervals, so only - # the table's two ends bound a step (SEP 14 section 4.1). - breakpoints = breakpoints[[0, -1]] + # The policy's landings (SEP 45 section 4.2): only the operator table's two + # ends bound a step without the interval cap (SEP 14 section 4.1), and dense + # output steps across the save planes, still landing on the end of the run and + # the hard kinks (SEP 14 section 4.3). targets = StepTargets( - # Dense output steps across the save planes; the end of the run and the - # hard kinks are still landed on (SEP 14 section 4.3). - save_planes=np.empty(0) - if dense - else np.asarray(settings.save_planes, dtype=np.float64), - hard_kinks=self._problem.linear.hard_kinks, - breakpoints=breakpoints, - dz_max=settings.dz_max, - z_end=settings.z_end if dense else target, - stretch=settings.stretch, + save_planes=np.asarray(policy.save_planes, dtype=np.float64), + hard_kinks=np.asarray(policy.hard_kinks, dtype=np.float64), + breakpoints=np.asarray(policy.breakpoints, dtype=np.float64), + dz_max=policy.dz_max, + z_end=policy.z_end if dense else target, + stretch=policy.stretch, ) while not reached(self._z, target): self._poll_stop() @@ -1543,7 +1535,7 @@ def _dense_record( - Returns: the anchor, the step and the five coefficients, host arrays. """ start = self._observed_field(cast(_StateT, self._state)).reshape( - (1, *self._problem.shape) + self._problem.shape ) coefficients = extension_coefficients( self._stepper.tableau, start, self._dense_stages(), dz @@ -1570,7 +1562,7 @@ def _observe_inside(self, position: float) -> None: self._dense_ahead = ( self._z, self._observed_field(cast(_StateT, self._state)).reshape( - (1, *self._problem.shape) + self._problem.shape ), ) self._state = self._apply_mask(self._adams_propagator(z, position) * value) @@ -1585,9 +1577,9 @@ def _dense_stages(self) -> list[np.ndarray]: msg = f"{self.name} does not observe densely" raise BackendError(msg) - def _controller_policy(self) -> rk_core.RungeKuttaPolicy: - """Return the Runge-Kutta march's controller policy (SEP 45).""" - return rk_policy(self._stepper) + def _controller_policy(self) -> RungeKuttaPolicy: + """Return the Runge-Kutta march's controller and landing policy (SEP 45).""" + return runge_kutta_policy(self._problem, self._stepper) def _start_up(self, norm: float) -> None: """Take the step after the first from the first step's own estimate. @@ -1765,7 +1757,7 @@ def _natural_stop_due(self, target: float) -> bool: _monitor: BoundaryMonitor | None = None """The run's boundary monitor (SEP 21), fed every committed step; ``None``.""" - _input_floor: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"] | None = None + _input_floor: Complex[np.ndarray, "n_modes n_pol n_omega"] | None = None """The input-noise draw the launch carries, the monitor's floor; ``None``.""" _split_removed: float = 0.0 @@ -1846,7 +1838,7 @@ def attach_monitor(self, monitor: BoundaryMonitor) -> None: current=edge.every_step and edge.resumed, ), self._z, - noise=None if floor is None else np.asarray(floor[0]), + noise=None if floor is None else np.asarray(floor), clean_columns=self._edge_clean_columns(edge), steps=None if schedule is None else np.asarray(schedule.steps), reads=self.reads_transverse_edge, @@ -1914,7 +1906,7 @@ def _edge_reading( - ``full``: whether every node is wanted, not only the rows a read uses. """ return edge.reduction.reading( - np.asarray(self._observed_field(state)[0]), + np.asarray(self._observed_field(state)), faces=faces, current=current, full=full, @@ -1938,8 +1930,8 @@ def _edge_clean_columns( return None if self._clean_columns is not None: return self._clean_columns - state = np.asarray(self._observed_field(self._require_state())[0]) - return edge.reduction.column_power(state - np.asarray(floor[0])) + state = np.asarray(self._observed_field(self._require_state())) + return edge.reduction.column_power(state - np.asarray(floor)) def _next_step(self) -> float: """Return the step the march takes next, m. @@ -2151,28 +2143,10 @@ def _quiet_capped(self, proposal: float) -> float: def _collar_can_rest(self) -> bool: """Return whether this run's stages may leave a quiet temporal collar out. - One rule for every backend: a modal or radial stage with the in-stage - (``"nonlinear"``) scheme and a temporal collar alone. The low-storage march, - coupled flows, the general modal route, a real-space collar sharing the term, - and the Cartesian, line and planar routes, whose stages keep the term whatever - the monitor says, keep the collar in every stage (SEP 21 D15). - - - Returns: True when the term may be left out while the collar is quiet. + The one rule every backend reads (:func:`solprop.backends.records.collar_rests`, + SEP 21 D15). """ - problem = self._problem - responses = problem.responses - geometry = problem.transform.geometry - resting_geometry = geometry is None or ( - geometry.kind == "radial" and not np.any(geometry.absorber_rate) - ) - return ( - self._low_storage is None - and responses.temporal_absorber is not None - and responses.absorber_scheme == "nonlinear" - and problem.linear.coupled_flow is None - and problem.transform.modal_tables is None - and resting_geometry - ) + return collar_rests(self._problem, self._stepper) def _collar_changed(self) -> None: # noqa: B027 - optional by design """Drop whatever was evaluated under the other collar setting. @@ -2262,7 +2236,7 @@ def _record(self, z: float, dz: float, norm: float, accepted: bool) -> None: def _noise_field( self, noise: NoiseRealisation | None - ) -> Complex[np.ndarray, "n_batch n_modes n_pol n_omega"] | None: + ) -> Complex[np.ndarray, "n_modes n_pol n_omega"] | None: """Return the noise realisation to carry, in the canonical layout. - Returns: the noise spectrum, or ``None`` when no noise model is active. @@ -2270,13 +2244,12 @@ def _noise_field( if self._problem.initial.noise_model == "none": return None if noise is not None: - return noise_spectrum(noise, self._problem.shape)[np.newaxis] - noise0 = self._problem.initial.noise0 - return None if noise0 is None else noise0[np.newaxis] + return noise_spectrum(noise, self._problem.shape) + return self._problem.initial.noise0 def _noise_at( self, bundle: StageScalars - ) -> Complex[np.ndarray, "n_batch n_modes n_pol n_omega"] | None: + ) -> Complex[np.ndarray, "n_modes n_pol n_omega"] | None: """Return the shot-noise field advanced to a stage's position. ``N(z) = N0 exp(i Im Phi(omega, z))``: dispersion yes, loss and gain no, from @@ -2300,9 +2273,7 @@ def _noise_phase( if self._problem.initial.noise_model == "none": return None phase = self._unitary_phase(self._stage_scalars(self._z, self._z)) - return np.ascontiguousarray( - np.broadcast_to(phase, (1, *self._problem.shape))[0] - ) + return np.ascontiguousarray(np.broadcast_to(phase, self._problem.shape)) # --- guards -------------------------------------------------------------------- @@ -2311,13 +2282,13 @@ def _axis_statistics( field: Complex[np.ndarray, "n_modes n_pol n_omega"] | None, scan: tuple[np.ndarray, np.ndarray] | None, ) -> dict[str, np.ndarray]: - """Return the four on-axis plasma statistics of one batch member (SEP 19). + """Return the four on-axis plasma statistics of the state (SEP 19). A rule with a node on the axis hands over that node's row from this observation's node scan; otherwise the axis row is scanned once (:func:`solprop.backends.numpy_ref.populations.axis_populations`). - - ``field``: one batch member at this plane, shape (n_modes, n_pol, n_omega), or + - ``field``: the state at this plane, shape (n_modes, n_pol, n_omega), or ``None`` from a backend whose :meth:`_axis_row_populations` reads its own state. - ``scan``: this observation's channel fractions and peak rates at every node, diff --git a/src/solprop/backends/host/session.py b/src/solprop/backends/host/session.py index fafc65830..77f03ca64 100644 --- a/src/solprop/backends/host/session.py +++ b/src/solprop/backends/host/session.py @@ -53,6 +53,7 @@ StepStats, StepSummary, ) +from solprop.backends.records import plane_plan from solprop.backends.session import ( DEFAULT_DEADLINE, DEFAULT_WORK_LIMIT, @@ -152,10 +153,14 @@ def __init__( self._problem = problem self._request = request self._ledger = SessionLedger() - self._planes = np.asarray(request.observation.save_planes, dtype=np.float64) + # The run's plane plan (SEP 52 section 5.1): its planes, the first beyond the + # start, whether the start's own plane is owed, and the step planes. + plan = plane_plan(request, start) + self._planes = plan.planes + self._step_planes = plan.step_planes self._owns = owns - self._index = start.plane_index - self._launch_pending = start.plane_pending + self._index = plan.next_plane + self._launch_pending = plan.launch_pending # The session takes over the request's fresh monitor and feeds it. self._monitor: BoundaryMonitor | None = request.boundary.monitor self._monitor_resume = start.monitor @@ -299,9 +304,7 @@ def advance( # noqa: PLR0911 - one return per barrier kind return self._issue([Progress(self._z, history, rows)]) self._summary = None return self._observation( - ObservationKind.STEP - if self._request.observation.step_planes - else ObservationKind.NATURAL, + ObservationKind.STEP if self._step_planes else ObservationKind.NATURAL, self._index, summary, ) diff --git a/src/solprop/backends/ledger.py b/src/solprop/backends/ledger.py index 1b0710985..c33a8934e 100644 --- a/src/solprop/backends/ledger.py +++ b/src/solprop/backends/ledger.py @@ -24,7 +24,7 @@ __all__ = ["CallLedger", "Derivative", "budget_error", "source_absent"] -_State = Complex[np.ndarray, "n_batch n_modes n_pol n_omega"] +_State = Complex[np.ndarray, "n_modes n_pol n_omega"] @dataclass @@ -257,7 +257,7 @@ class Derivative: Where the derivative was evaluated, m. value : numpy.ndarray or None The history store's row, complex128 of shape - ``(n_batch, n_modes, n_pol, n_omega)``, or ``None`` for an engine that keeps + ``(n_modes, n_pol, n_omega)``, or ``None`` for an engine that keeps its rows resident (the native state). identity : str Which field it was evaluated at: ``"accepted"``, or ``"predicted"`` for the diff --git a/src/solprop/backends/low_storage.py b/src/solprop/backends/low_storage.py index 46b454daa..81ffdbbd1 100644 --- a/src/solprop/backends/low_storage.py +++ b/src/solprop/backends/low_storage.py @@ -81,7 +81,7 @@ logger = logging.getLogger(__name__) -_State = Complex[np.ndarray, "n_batch n_modes n_pol n_omega"] +_State = Complex[np.ndarray, "n_modes n_pol n_omega"] def table_steps(schedule: FixedSchedule) -> Float[np.ndarray, " n_steps"]: @@ -180,7 +180,9 @@ class LowStorageEngine(Protocol): Part of the kernel seam (schema 29): the march keeps the schedule, the commit and the records; an engine keeps the registers and the flow table and does every - state-sized operation of a step. :class:`HostLowStorage` is the reference. + state-sized operation of a step. :class:`HostLowStorage` is the reference. SEP 52's + phase 3 replaces it with :class:`solprop.backends.engines.RungeKuttaEngine`, which + the native low-storage engine already implements. """ stiffness: float diff --git a/src/solprop/backends/numpy_ref/backend.py b/src/solprop/backends/numpy_ref/backend.py index 3de4f9ed3..ffdb4de6f 100644 --- a/src/solprop/backends/numpy_ref/backend.py +++ b/src/solprop/backends/numpy_ref/backend.py @@ -130,7 +130,7 @@ logger = logging.getLogger(__name__) -class NumpyRefBackend(HostDriver[Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]]): +class NumpyRefBackend(HostDriver[Complex[np.ndarray, "n_modes n_pol n_omega"]]): """Propagate one Kerr and plasma problem with interaction-picture DOPRI5. The oracle of ``docs/dev/backends.md`` section 2. It implements @@ -314,7 +314,7 @@ def __init__( # The lab-frame source's unit "propagator": one element broadcast to the state's # shape, so it holds no volume beside the state. self._lab_frame = np.broadcast_to( - np.ones((), dtype=np.complex128), (1, *problem.shape) + np.ones((), dtype=np.complex128), problem.shape ) if logger.isEnabledFor(logging.DEBUG): logger.debug( @@ -364,9 +364,9 @@ def _envelope_populations( def _error_norm( self, - error: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], - state: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], - advanced: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], + error: Complex[np.ndarray, "n_modes n_pol n_omega"], + state: Complex[np.ndarray, "n_modes n_pol n_omega"], + advanced: Complex[np.ndarray, "n_modes n_pol n_omega"], ) -> float: """Return the controller's scaled norm of a trial step's embedded difference.""" return passes.error_norm( @@ -380,8 +380,12 @@ def _error_norm( def stats(self, spec: StatsSpec) -> dict[str, np.ndarray]: """Collect the requested statistics at the current position. - The trajectory's statistics, without the leading axis of one the march holds - its state with (SEP 47 section 6.2), which ``_stacked_statistics`` evaluates. + The statistics of the saved field go to :func:`solprop.stats.collect`; there is + exactly one definition of each in the package and that is it. The statistics + :class:`solprop.stats.StatDefinition` marks ``backend`` are of quantities a + saved plane does not carry -- the electron density, the peak ionisation rate, + the on-axis intensity, the quadrature error -- and are formed here from the + backend's own buffers. Parameters ---------- @@ -393,25 +397,6 @@ def stats(self, spec: StatsSpec) -> dict[str, np.ndarray]: dict Small host arrays keyed by statistic name. """ - return { - name: np.asarray(value[0]) - for name, value in self._stacked_statistics(spec).items() - } - - def _stacked_statistics(self, spec: StatsSpec) -> dict[str, np.ndarray]: - """Collect the statistics with the leading axis of one of the held state. - - The statistics of the saved field go to :func:`solprop.stats.collect`, one call - per row of the held state, stacked along its leading axis; there is exactly one - definition of each in the package and that is it. The - statistics :class:`solprop.stats.StatDefinition` marks ``backend`` are of - quantities a saved plane does not carry -- the electron density, the peak - ionisation rate, the on-axis intensity, the quadrature error -- and are formed - here from the backend's own buffers. - - - Returns: small host arrays keyed by statistic name, each with a leading axis - of one. - """ state = self._require_state() grid = self._require_grid() if self._tdse_binding is not None: @@ -436,24 +421,20 @@ def _stacked_statistics(self, spec: StatsSpec) -> dict[str, np.ndarray]: periodic=(), ) tables = self._far_field.for_spec(spatial_spec) - members = [ - collect_spatial( - spatial_spec, - grid, - self._problem.transform, - one, - z=self._z, - flux=flux_of(self._problem.route), - profiles=self._profiles, - tables=tables, - operator=self._problem.linear.separable, - node_chunk=self.node_chunk, - ) - for one in state - ] + spatial = collect_spatial( + spatial_spec, + grid, + self._problem.transform, + state, + z=self._z, + flux=flux_of(self._problem.route), + profiles=self._profiles, + tables=tables, + operator=self._problem.linear.separable, + node_chunk=self.node_chunk, + ) return { - name: np.stack([one[name] for one in members]) - for name in spatial_spec.names + name: np.array(spatial[name]) for name in spatial_spec.names } | self._backend_statistics( tuple(name for name in spec.names if name in response_names), state ) @@ -481,14 +462,9 @@ def _stacked_statistics(self, spec: StatsSpec) -> dict[str, np.ndarray]: raise BackendError(msg) axis = self._on_axis_synthesis(self._modes) power_values = ( - [ - _poynting_statistics( - spec, grid, member, self._statistics_power_weight, axis - ) - for member in state - ] + _poynting_statistics(spec, grid, state, self._statistics_power_weight, axis) if weighted - else [] + else {} ) owned = tuple( name @@ -508,7 +484,7 @@ def _stacked_statistics(self, spec: StatsSpec) -> dict[str, np.ndarray]: if name not in owned and name not in power_names ), ) - # The basis metric at this plane, evaluated once for every member: the + # The basis metric at this plane: the # statistics of `METRIC_WEIGHTED_STATISTICS` are functionals of the physical # same-frequency flux `a.H W a`, and on a declared change of basis W is not # the identity. `metric_at` returns None for an identity metric, which is @@ -519,39 +495,27 @@ def _stacked_statistics(self, spec: StatsSpec) -> dict[str, np.ndarray]: if set(delegated.names) & METRIC_WEIGHTED_STATISTICS else None ) - members = [ - collect( - delegated, - grid, - SavedPlane(z=self._z, field=state[member]), - modes=self._modes, - profiles=self._profiles, - metric=metric, - ) - for member in range(state.shape[0]) - ] - values = { - name: np.stack([member[name] for member in members]) - for name in delegated.names - } - values.update( - { - name: np.stack([member[name] for member in power_values]) - for name in power_names - } + collected = collect( + delegated, + grid, + SavedPlane(z=self._z, field=state), + modes=self._modes, + profiles=self._profiles, + metric=metric, ) + values = {name: np.array(collected[name]) for name in delegated.names} + values.update({name: np.array(power_values[name]) for name in power_names}) return values | self._backend_statistics(owned, state) def _backend_statistics( self, names: tuple[str, ...], - state: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], + state: Complex[np.ndarray, "n_modes n_pol n_omega"], ) -> dict[str, np.ndarray]: """Share each fresh envelope population scan among the requested statistics. - - ``state``: accepted state at this plane with its leading axis of one. - - Returns: requested values with a leading axis of one and no retained field - cache. + - ``state``: accepted state at this plane. + - Returns: requested values, with no retained field cache. """ population_names = ( tuple( @@ -574,10 +538,10 @@ def _backend_statistics( population_names = tuple( name for name in names if name.startswith("species_") ) - # Node scans of this observation only, by batch member: a rule with a node on - # the axis hands its row to the on-axis statistics, and nothing outlives the - # call, so no stale stage or rejected step can be reused. - scans: dict[int, tuple[np.ndarray, np.ndarray]] = {} + # Node scans of this observation only: a rule with a node on the axis hands + # its row to the on-axis statistics, and nothing outlives the call, so no + # stale stage or rejected step can be reused. + scans: list[tuple[np.ndarray, np.ndarray]] = [] result = { name: self._backend_stat(name, state) for name in names @@ -586,74 +550,45 @@ def _backend_statistics( and name not in axis_names } if integration_names: - diagnostics = [self._integration_diagnostics(member) for member in state] + diagnostics = self._integration_diagnostics(state) result.update( - { - name: np.stack([member[name] for member in diagnostics]) - for name in integration_names - } + {name: np.array(diagnostics[name]) for name in integration_names} ) if population_names: - members = [ - self._species_statistics(member, scans, index) - for index, member in enumerate(state) - ] - result.update( - { - name: np.stack([member[name] for member in members]) - for name in population_names - } - ) + species = self._species_statistics(state, scans) + result.update({name: np.array(species[name]) for name in population_names}) if axis_names: - axis = [ - self._axis_statistics(member, scans.get(index)) - for index, member in enumerate(state) - ] - result.update( - { - name: np.stack([member[name] for member in axis]) - for name in axis_names - } - ) + axis = self._axis_statistics(state, scans[0] if scans else None) + result.update({name: np.array(axis[name]) for name in axis_names}) return result def _backend_stat( self, name: str, - state: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], + state: Complex[np.ndarray, "n_modes n_pol n_omega"], ) -> np.ndarray: - """Compute one backend-owned statistic over the batch. - - - Returns: the value with a leading axis of one. - """ + """Compute one backend-owned statistic.""" if name == "quadrature_error": - return np.full(state.shape[0], self._alarm) + return np.full((), self._alarm) if name == "peak_intensity": - return np.stack( - [self._peak_intensity(state[member]) for member in range(len(state))] - ) + return np.array(self._peak_intensity(state)) if ( name.startswith("species_") or self._problem.flags.n_channels > 1 or self._problem.responses.envelope_plasma is not None ): - return np.stack( - [self._species_statistics(member)[name] for member in state] - ) - ionised = [self._ionisation(state[member]) for member in range(len(state))] - index = 0 if name == "electron_density" else 1 - return np.stack([one[index] for one in ionised]) + return np.array(self._species_statistics(state)[name]) + return np.array(self._ionisation(state)[0 if name == "electron_density" else 1]) def _species_statistics( self, field: Complex[np.ndarray, "n_modes n_pol n_omega"], - scans: dict[int, tuple[np.ndarray, np.ndarray]] | None = None, - member: int = 0, + scans: list[tuple[np.ndarray, np.ndarray]] | None = None, ) -> dict[str, np.ndarray]: """Evaluate independent populations on the actual nonlinear nodes. - ``scans``: where to leave this observation's channel fractions and peak rates - at every node, keyed by `member`, for the on-axis statistics to reuse. + at every node, for the on-axis statistics to reuse. - Returns: the five species statistics and the two total plasma statistics. """ responses = self._problem.responses @@ -686,7 +621,7 @@ def _species_statistics( nodes, tuple(float(c.preionisation(self._z)) for c in channels) ) if scans is not None: - scans[member] = (fractions, peaks) + scans.append((fractions, peaks)) return population_statistics(self._problem, self._z, fractions, peaks) rates, population = passes.plasma_channel_fractions( nodes, @@ -696,7 +631,7 @@ def _species_statistics( responses.plasma_integral, ) if scans is not None: - scans[member] = (population[:, :, -1], rates.max(axis=-1)) + scans.append((population[:, :, -1], rates.max(axis=-1))) return population_statistics( self._problem, self._z, population[:, :, -1], rates.max(axis=-1) ) @@ -1107,7 +1042,7 @@ def _reserve_source_calls(self, count: int) -> None: f"at z = {self._z:.6g} m ({error.calls} charged); " "the run stops at its last accepted step, unfinished", z=self._z, - field=self._adams_state()[0].copy(), + field=self._adams_state().copy(), calls=error.calls, ) from None @@ -1118,14 +1053,14 @@ def _release_source_calls(self) -> None: def _adams_source( self, - lab: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], + lab: Complex[np.ndarray, "n_modes n_pol n_omega"], anchor: float, position: float, *, purpose: TDSEPurpose = "predictor", ) -> tuple[ - Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], - Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], + Complex[np.ndarray, "n_modes n_pol n_omega"], + Complex[np.ndarray, "n_modes n_pol n_omega"], ]: """Evaluate the source once through the reference passes. @@ -1192,11 +1127,11 @@ def _adams_block(self, z: float, dz: float) -> object: def _adams_lab_source( self, - lab: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], + lab: Complex[np.ndarray, "n_modes n_pol n_omega"], position: float, *, purpose: TDSEPurpose = "predictor", - ) -> Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]: + ) -> Complex[np.ndarray, "n_modes n_pol n_omega"]: """Evaluate the source once through the reference passes, untransported. The same passes as :meth:`_adams_source`, with the stage's inverse propagator @@ -1225,7 +1160,7 @@ def _adams_lab_source( def _adams_propagator( self, anchor: float, position: float - ) -> Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]: + ) -> Complex[np.ndarray, "n_modes n_pol n_omega"]: """Return ``U(position, anchor)``, held for the Adams step's source call. Only an Adams march reads it back (:meth:`_adams_source`), so only an Adams @@ -1242,7 +1177,7 @@ def _adams_propagator( def _trial_step( self, z: float, dz: float - ) -> tuple[float, Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]]: + ) -> tuple[float, Complex[np.ndarray, "n_modes n_pol n_omega"]]: """Take one interaction-picture Runge-Kutta step and score it. Every stage is evaluated. The tableau's first-same-as-last property would let @@ -1283,7 +1218,7 @@ def _trial_step( exponents = self._exponents(z, dz) step = self._stage_scalars(z, z + dz) actions = self._flow_actions(step.flow_section, dz) - stages: list[Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]] = [] + stages: list[Complex[np.ndarray, "n_modes n_pol n_omega"]] = [] solution = np.zeros_like(state) error = np.zeros_like(state) for index, node in enumerate(tableau.nodes): @@ -1337,9 +1272,9 @@ def _trial_step( def _channels( self, - values: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], + values: Complex[np.ndarray, "n_modes n_pol n_omega"], action: Complex[np.ndarray, "n_active_c n_modes_c_out n_modes_c_in"] | None, - ) -> Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]: + ) -> Complex[np.ndarray, "n_modes n_pol n_omega"]: """Mix the channel axis by one coupled-flow action, or return the values. - ``values``: any array of the state's layout. @@ -1352,10 +1287,10 @@ def _channels( def _residual( self, - lab: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], + lab: Complex[np.ndarray, "n_modes n_pol n_omega"], section: int, bundle: StageScalars, - ) -> Complex[np.ndarray, "n_batch n_modes n_pol n_omega"] | None: + ) -> Complex[np.ndarray, "n_modes n_pol n_omega"] | None: """Return the explicit residual's lab-frame source term, or nothing. ``None`` for every problem but a ``"frozen_residual"`` coupled flow: the @@ -1439,8 +1374,8 @@ def action( return action def _apply_mask( - self, state: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"] - ) -> Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]: + self, state: Complex[np.ndarray, "n_modes n_pol n_omega"] + ) -> Complex[np.ndarray, "n_modes n_pol n_omega"]: """Zero the state outside the active band. - ``state``: the state in the canonical layout. @@ -1474,9 +1409,9 @@ def _record(self, z: float, dz: float, norm: float, accepted: bool) -> None: def _finish( self, - state: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], + state: Complex[np.ndarray, "n_modes n_pol n_omega"], dz: float, - ) -> Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]: + ) -> Complex[np.ndarray, "n_modes n_pol n_omega"]: """Apply the once-per-accepted-step boundary treatments to a kept step. - ``dz``: length of the step, m. @@ -1503,26 +1438,26 @@ def _finish( # A (p, y) row is a modal amplitude whose time signal is a real field's, # and the rate is the same at every node, so it is applied row by row. transform = self._problem.transform - physical = slab_rows(transform, masked[0]) + physical = slab_rows(transform, masked) chunk = self.node_chunk for first in range(0, len(physical), chunk): rows = slice(first, first + chunk) physical[rows] = passes.apply_temporal_rate( - physical[rows][None], rate, dz, kind=self._kind - )[0] - return self._apply_mask(slab_columns(transform, physical)[None]) + physical[rows], rate, dz, kind=self._kind + ) + return self._apply_mask(slab_columns(transform, physical)) if geometry is not None and geometry.kind == "free": nx, ny = geometry.shape - shape = (ny, nx, *masked.shape[2:]) - physical = np.fft.ifft2(masked[0].reshape(shape), axes=(0, 1)).reshape( - masked.shape[1:] + shape = (ny, nx, *masked.shape[1:]) + physical = np.fft.ifft2(masked.reshape(shape), axes=(0, 1)).reshape( + masked.shape ) chunk = self.node_chunk for first in range(0, len(physical), chunk): rows = slice(first, first + chunk) physical[rows] = passes.apply_temporal_rate( - physical[rows][None], rate, dz, kind=self._kind - )[0] + physical[rows], rate, dz, kind=self._kind + ) result = np.fft.fft2(physical.reshape(shape), axes=(0, 1)).reshape( masked.shape ) @@ -1599,12 +1534,12 @@ def _update_alarm(self) -> None: def _nonlinear( self, - lab: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], + lab: Complex[np.ndarray, "n_modes n_pol n_omega"], bundle: StageScalars, - inverse_propagator: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], + inverse_propagator: Complex[np.ndarray, "n_modes n_pol n_omega"], *, purpose: TDSEPurpose = "predictor", - ) -> Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]: + ) -> Complex[np.ndarray, "n_modes n_pol n_omega"]: """Evaluate one stage derivative through the transform passes. The pipeline of ``architecture.md`` section 8 for any node count: one batched @@ -1636,17 +1571,15 @@ def _nonlinear( # would otherwise be dropped from a problem that has no response at all. if not self._in_stage_absorber: return np.zeros_like(lab) - alone = passes.to_time( - lab[0], grids.crop_index, grids.n_to, kind=self._kind - ) + alone = passes.to_time(lab, grids.crop_index, grids.n_to, kind=self._kind) return passes.stage_output( - np.zeros_like(lab[0]), - np.zeros_like(lab[0]), + np.zeros_like(lab), + np.zeros_like(lab), self._tables.omega_window, inverse_propagator, self._absorber_term(alone, may_quiet=True), ) - modal_t = passes.to_time(lab[0], grids.crop_index, grids.n_to, kind=self._kind) + modal_t = passes.to_time(lab, grids.crop_index, grids.n_to, kind=self._kind) geometry = self._problem.transform.geometry if ( geometry is not None @@ -1684,11 +1617,11 @@ def _nonlinear( def _nonlinear_tdse( self, - lab: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], + lab: Complex[np.ndarray, "n_modes n_pol n_omega"], bundle: StageScalars, - inverse_propagator: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], + inverse_propagator: Complex[np.ndarray, "n_modes n_pol n_omega"], purpose: TDSEPurpose, - ) -> Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]: + ) -> Complex[np.ndarray, "n_modes n_pol n_omega"]: """Evaluate the declared atomic binding and combine independent P/J sources. Steps 1 to 4 are :mod:`solprop.backends.tdse_source`'s, shared with native; @@ -1698,8 +1631,8 @@ def _nonlinear_tdse( grids = problem.grids binding = self._tdse_binding assert binding is not None and tdse_of(problem.route) is not None - modal_t = passes.to_time(lab[0], grids.crop_index, grids.n_to, kind="field") - fields = tdse_source.node_fields(problem, lab[0], bundle) + modal_t = passes.to_time(lab, grids.crop_index, grids.n_to, kind="field") + fields = tdse_source.node_fields(problem, lab, bundle) result = binding.evaluate( fields, z=bundle.z1, purpose=purpose, density=bundle.density ) @@ -1727,10 +1660,10 @@ def _nonlinear_tdse( def _nonlinear_general( self, - lab: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], + lab: Complex[np.ndarray, "n_modes n_pol n_omega"], bundle: StageScalars, - inverse_propagator: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], - ) -> Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]: + inverse_propagator: Complex[np.ndarray, "n_modes n_pol n_omega"], + ) -> Complex[np.ndarray, "n_modes n_pol n_omega"]: """Evaluate one stage derivative through a general modal guide's maps. The route's own execution path, in @@ -1759,7 +1692,7 @@ def _nonlinear_general( # is not dropped with it. spectrum = ( general_source( - lab[0], + lab, self._general, grids.crop_index, n_to=grids.n_to, @@ -1768,12 +1701,12 @@ def _nonlinear_general( dispersive_densities=bundle.dispersive_kerr_densities, ) if self._general_responds - else np.zeros_like(lab[0]) + else np.zeros_like(lab) ) absorber = None if self._in_stage_absorber: absorber = self._absorber_term( - passes.to_time(lab[0], grids.crop_index, grids.n_to, kind=self._kind) + passes.to_time(lab, grids.crop_index, grids.n_to, kind=self._kind) ) return passes.stage_output( spectrum, @@ -1884,7 +1817,7 @@ def _nonlinear_free( ) -> np.ndarray: """Apply transverse FFTs before bounded temporal response transforms. - - ``lab``: Lab-frame stage spectrum, with the state's leading axis of one. + - ``lab``: Lab-frame stage spectrum. """ problem = self._problem transform, grids, responses = ( @@ -1896,15 +1829,13 @@ def _nonlinear_free( if geometry is None: msg = "Cartesian nonlinear evaluation requires spatial geometry" raise BackendError(msg) - shape = (geometry.shape[1], geometry.shape[0], *lab.shape[2:]) + shape = (geometry.shape[1], geometry.shape[0], *lab.shape[1:]) result = np.zeros_like(lab) if len(responses.response_order) == 1 and not self._in_stage_absorber: return result - scratch = np.empty_like(lab[0]) + scratch = np.empty_like(lab) if len(responses.response_order) > 1: - scratch.reshape(shape)[...] = np.fft.ifft2( - lab[0].reshape(shape), axes=(0, 1) - ) + scratch.reshape(shape)[...] = np.fft.ifft2(lab.reshape(shape), axes=(0, 1)) raman = self._raman_spectrum(bundle.raman_rates, bundle.raman_densities) for first in range(0, transform.n_nodes, self.node_chunk): rows = slice(first, first + self.node_chunk) @@ -1934,13 +1865,11 @@ def _nonlinear_free( for first in range(0, transform.n_nodes, self.node_chunk): rows = slice(first, first + self.node_chunk) _, prefactor = separable_values(data, rows) - result[0, rows] = prefactor * self._tables.omega_window * scratch[rows] + result[rows] = prefactor * self._tables.omega_window * scratch[rows] if self._in_stage_absorber: # The rate is additive to the normalised material term. Reusing the # transverse scratch avoids a second retained physical spectral volume. - scratch.reshape(shape)[...] = np.fft.ifft2( - lab[0].reshape(shape), axes=(0, 1) - ) + scratch.reshape(shape)[...] = np.fft.ifft2(lab.reshape(shape), axes=(0, 1)) temporal = self._tables.temporal_rate timed = temporal is not None and responses.absorber_scheme == "nonlinear" for first in range(0, transform.n_nodes, self.node_chunk): @@ -1961,7 +1890,7 @@ def _nonlinear_free( ) view = scratch.reshape(shape) view[...] = np.fft.fft2(view, axes=(0, 1)) - result[0] += grids.active_mask * scratch + result += grids.active_mask * scratch return result * inverse_propagator def _nonlinear_slab( @@ -1983,7 +1912,7 @@ def _nonlinear_slab( (kernel schema 15). The y collar's in-stage rate depends on y alone, so it acts on the rows directly. - - ``lab``: Lab-frame stage spectrum, with the state's leading axis of one. + - ``lab``: Lab-frame stage spectrum. """ problem = self._problem transform, grids, responses = ( @@ -1998,7 +1927,7 @@ def _nonlinear_slab( return result n_x, n_y = geometry.shape n_p = transform.synthesis.shape[1] - rows_f = slab_rows(transform, lab[0]) + rows_f = slab_rows(transform, lab) temporal = responses.temporal_absorber_fine timed = temporal is not None and responses.absorber_scheme == "nonlinear" rows_t = ( @@ -2070,9 +1999,9 @@ def _nonlinear_slab( for first in range(0, transform.n_modes, self.node_chunk): rows = slice(first, first + self.node_chunk) _, prefactor = separable_values(data, rows) - result[0, rows] = prefactor * grids.omega_window * columns[rows] + result[rows] = prefactor * grids.omega_window * columns[rows] if self._in_stage_absorber: - result[0] += self._slab_absorber(rows_f, rows_t) + result += self._slab_absorber(rows_f, rows_t) return result * inverse_propagator def _slab_absorber( @@ -2309,9 +2238,7 @@ def _raman_spectrum( def _exponents( self, z: float, dz: float - ) -> Callable[ - [float, StageScalars], Complex[np.ndarray, "n_batch n_modes n_pol n_omega"] - ]: + ) -> Callable[[float, StageScalars], Complex[np.ndarray, "n_modes n_pol n_omega"]]: """Return the memoised ``dPhi`` of a step, keyed by the stage fraction. The anchor is hoisted once per step and each exponent is evaluated once and @@ -2328,11 +2255,11 @@ def _exponents( del z, dz if self._propagator_mode == "recompute": return lambda fraction, bundle: self._delta_phi(bundle) # noqa: ARG005 - cache: dict[float, Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]] = {} + cache: dict[float, Complex[np.ndarray, "n_modes n_pol n_omega"]] = {} def exponent( fraction: float, bundle: StageScalars - ) -> Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]: + ) -> Complex[np.ndarray, "n_modes n_pol n_omega"]: """Return ``dPhi`` at a stage fraction, computing it at most once. - ``fraction``: the tableau node ``c_i``, dimensionless, in ``[0, 1]``. @@ -2754,9 +2681,9 @@ def __init__(self, flow: _CoupledFlow) -> None: def __call__( self, section: int, - state: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], + state: Complex[np.ndarray, "n_modes n_pol n_omega"], weights: Float[np.ndarray, " n_flow_terms"], - ) -> Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]: + ) -> Complex[np.ndarray, "n_modes n_pol n_omega"]: """Return ``R_s(z) state`` for the flow weights of a position. Parameters @@ -2787,11 +2714,11 @@ def __call__( def _residual_drive( - lab: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], + lab: Complex[np.ndarray, "n_modes n_pol n_omega"], action: Complex[np.ndarray, "n_active_d n_modes_d_out n_modes_d_in"], active: Bool[np.ndarray, " n_omega"], scored: Bool[np.ndarray, " n_omega_scored"], -) -> Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]: +) -> Complex[np.ndarray, "n_modes n_pol n_omega"]: """Return the explicit residual's contribution to one stage's lab-frame source. Unlike :func:`_apply_flow`, which *transforms* a state and therefore leaves the @@ -2811,17 +2738,17 @@ def _residual_drive( - Returns: ``R_s(z_1) u`` in the state's layout, zero off the scored active bins. """ drive = np.zeros_like(lab) - drive[:, :, :, scored] = np.einsum( - "wij,bjpw->bipw", action[scored[active]], lab[:, :, :, scored] + drive[:, :, scored] = np.einsum( + "wij,jpw->ipw", action[scored[active]], lab[:, :, scored] ) return drive def _apply_flow( - state: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], + state: Complex[np.ndarray, "n_modes n_pol n_omega"], action: Complex[np.ndarray, "n_active_a n_modes_a_out n_modes_a_in"], active: Bool[np.ndarray, " n_omega"], -) -> Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]: +) -> Complex[np.ndarray, "n_modes n_pol n_omega"]: """Apply a per-frequency channel action to the canonical state. The identity case is the caller's: a problem with no coupled flow, or the zero @@ -2832,9 +2759,7 @@ def _apply_flow( off them, where the block is zero and the action is the identity. """ carried = state.copy() - carried[:, :, :, active] = np.einsum( - "wij,bjpw->bipw", action, state[:, :, :, active] - ) + carried[:, :, active] = np.einsum("wij,jpw->ipw", action, state[:, :, active]) return carried diff --git a/src/solprop/backends/numpy_ref/multistep.py b/src/solprop/backends/numpy_ref/multistep.py index 3c79baaeb..fe3c2805e 100644 --- a/src/solprop/backends/numpy_ref/multistep.py +++ b/src/solprop/backends/numpy_ref/multistep.py @@ -86,11 +86,11 @@ logger = logging.getLogger(__name__) -_State = Complex[np.ndarray, "n_batch n_modes n_pol n_omega"] -_StoreRows = Complex[np.ndarray, "n_slots n_batch n_modes n_pol n_omega"] -_Shift = Complex[np.ndarray, "#n_batch 1 1 #n_omega"] -"""A carrier frame's shift: one per batch member (SEP 35); a bin axis broadcasts.""" -_Factors = Complex[np.ndarray, "n_used #n_batch 1 1 #n_omega"] +_State = Complex[np.ndarray, "n_modes n_pol n_omega"] +_StoreRows = Complex[np.ndarray, "n_slots n_modes n_pol n_omega"] +_Shift = Complex[np.ndarray, "1 1 #n_omega"] +"""A carrier frame's shift (SEP 35); a bin axis broadcasts.""" +_Factors = Complex[np.ndarray, "n_used 1 1 #n_omega"] """A carrier frame's rotation of each history row a trial reads, in its slots' order.""" @@ -148,7 +148,7 @@ def predict( # noqa: PLR0917 - called positionally self, state: _State, propagator: _State, - store: Complex[np.ndarray, "n_slots n_batch n_modes n_pol n_omega"], + store: Complex[np.ndarray, "n_slots n_modes n_pol n_omega"], slots: np.ndarray, weights: np.ndarray, dz: float, @@ -185,7 +185,7 @@ def correct( # noqa: PLR0917 - buffers are a kernel's parameters state: _State, propagator: _State, source: _State, - store: Complex[np.ndarray, "n_slots n_batch n_modes n_pol n_omega"], + store: Complex[np.ndarray, "n_slots n_modes n_pol n_omega"], slots: np.ndarray, corrector: np.ndarray, rows: np.ndarray, @@ -240,7 +240,7 @@ def correct( # noqa: PLR0917 - buffers are a kernel's parameters def transport( self, - store: Complex[np.ndarray, "n_slots n_batch n_modes n_pol n_omega"], + store: Complex[np.ndarray, "n_slots n_modes n_pol n_omega"], slots: np.ndarray, propagator: _State, new_slot: int, @@ -315,7 +315,7 @@ class _MatrixMoments: Off the block's bins the step's flow and moments are the diagonal ones, per element; on them each is a matrix in the channel axis, applied to the channel - vector of every batch row and polarisation. ``matrices`` and ``flow_matrix`` are + vector of every polarisation. ``matrices`` and ``flow_matrix`` are laid out by bin, ``(..., n_bins, n_modes, n_modes)``; the products read the same matrices with the bin axis last, as the state holds it. """ @@ -349,12 +349,9 @@ def _product( """ out = scalar * values selected = values[..., self.active] - block = columns[:, 0][np.newaxis, :, np.newaxis] * selected[:, np.newaxis, 0] + block = columns[:, 0][:, np.newaxis] * selected[np.newaxis, 0] for column in range(1, columns.shape[1]): - block += ( - columns[:, column][np.newaxis, :, np.newaxis] - * selected[:, np.newaxis, column] - ) + block += columns[:, column][:, np.newaxis] * selected[np.newaxis, column] out[..., self.active] = block return out @@ -477,7 +474,7 @@ def coupled_moments( ) flow_scalar = np.exp(unshifted) n_modes = block.triangular.shape[-1] - shift = exponent[0, 0, 0, active] + shift = exponent[0, 0, active] matrices = np.empty((count, *block.triangular.shape), dtype=np.complex128) flow_matrix = np.empty(block.triangular.shape, dtype=np.complex128) size = n_modes * (count + 1) @@ -507,7 +504,7 @@ def coupled_moments( @ adjoint ) if carrier is not None: - factor = np.exp(np.broadcast_to(carrier, unshifted.shape)[0, 0, 0, active]) + factor = np.exp(np.broadcast_to(carrier, unshifted.shape)[0, 0, active]) flow_matrix = factor[:, np.newaxis, np.newaxis] * flow_matrix return _MatrixMoments(scalar, flow_scalar, matrices, flow_matrix, active) @@ -563,7 +560,7 @@ def _times_flow(self, flow: Any, values: _State) -> _State: def moments( self, exponent: _State, count: int, *, shift: _Shift | None = None - ) -> Complex[np.ndarray, "n_k n_batch n_modes n_pol n_omega"]: + ) -> Complex[np.ndarray, "n_k n_modes n_pol n_omega"]: """Return ``k! phi_{k+1}(x)`` for ``k < count`` along a new leading axis. Parameters @@ -591,11 +588,11 @@ def carry( # noqa: PLR0917 - called positionally self, exponent: _State, dz: float, - rates: Complex[np.ndarray, "n_slots n_batch n_modes n_pol n_omega"], - states: Complex[np.ndarray, "n_slots n_batch n_modes n_pol n_omega"], + rates: Complex[np.ndarray, "n_slots n_modes n_pol n_omega"], + states: Complex[np.ndarray, "n_slots n_modes n_pol n_omega"], slots: np.ndarray, spans: np.ndarray, - ) -> Complex[np.ndarray, "n_used n_batch n_modes n_pol n_omega"]: + ) -> Complex[np.ndarray, "n_used n_modes n_pol n_omega"]: """Return each history sample's compensation carried to the step's end. ``exp(s_j x) (D(z_j) - x / dz) u_j`` with ``s_j = 1 - theta_j``, the @@ -644,7 +641,7 @@ def flow(self, exponent: _State) -> _State: def apply( self, - moments: Complex[np.ndarray, "n_k n_batch n_modes n_pol n_omega"], + moments: Complex[np.ndarray, "n_k n_modes n_pol n_omega"], coefficients: np.ndarray, values: list[_State], dz: float, @@ -693,8 +690,8 @@ def predict( # noqa: PLR0917 - buffers are a kernel's parameters self, state: _State, flow: _State, - moments: Complex[np.ndarray, "n_k n_batch n_modes n_pol n_omega"], - store: Complex[np.ndarray, "n_slots n_batch n_modes n_pol n_omega"], + moments: Complex[np.ndarray, "n_k n_modes n_pol n_omega"], + store: Complex[np.ndarray, "n_slots n_modes n_pol n_omega"], slots: np.ndarray, coefficients: np.ndarray, dz: float, @@ -737,7 +734,7 @@ def _predicted( # noqa: PLR0917 - buffers are a kernel's parameters state: _State, flow: Any, moments: Any, - store: Complex[np.ndarray, "n_slots n_batch n_modes n_pol n_omega"], + store: Complex[np.ndarray, "n_slots n_modes n_pol n_omega"], slots: np.ndarray, coefficients: np.ndarray, dz: float, @@ -779,9 +776,9 @@ def correct( # noqa: PLR0917 - buffers are a kernel's parameters self, state: _State, flow: _State, - moments: Complex[np.ndarray, "n_k n_batch n_modes n_pol n_omega"], + moments: Complex[np.ndarray, "n_k n_modes n_pol n_omega"], source: _State, - store: Complex[np.ndarray, "n_slots n_batch n_modes n_pol n_omega"], + store: Complex[np.ndarray, "n_slots n_modes n_pol n_omega"], slots: np.ndarray, corrector: np.ndarray, rows: np.ndarray, @@ -957,7 +954,7 @@ def carry_coupled( # noqa: PLR0917 - called positionally slots: np.ndarray, spans: np.ndarray, block: CoupledBlock, - ) -> Complex[np.ndarray, "n_used n_batch n_modes n_pol n_omega"]: + ) -> Complex[np.ndarray, "n_used n_modes n_pol n_omega"]: """Return each sample's compensation carried to the step's end (SEP 16 4.5). ``C_j = (D(z_j) - x / h) u_j + R_s(z_j) u_j``: the diagonal chord's @@ -1004,9 +1001,8 @@ def predict( # noqa: PLR0917 - buffers are a kernel's parameters self, state: _State, flow: _State | _MatrixMoments, - moments: Complex[np.ndarray, "n_k n_batch n_modes n_pol n_omega"] - | _MatrixMoments, - store: Complex[np.ndarray, "n_slots n_batch n_modes n_pol n_omega"], + moments: Complex[np.ndarray, "n_k n_modes n_pol n_omega"] | _MatrixMoments, + store: Complex[np.ndarray, "n_slots n_modes n_pol n_omega"], slots: np.ndarray, coefficients: np.ndarray, dz: float, @@ -1045,10 +1041,9 @@ def correct( # noqa: PLR0917 - buffers are a kernel's parameters self, state: _State, flow: _State | _MatrixMoments, - moments: Complex[np.ndarray, "n_k n_batch n_modes n_pol n_omega"] - | _MatrixMoments, + moments: Complex[np.ndarray, "n_k n_modes n_pol n_omega"] | _MatrixMoments, source: _State, - store: Complex[np.ndarray, "n_slots n_batch n_modes n_pol n_omega"], + store: Complex[np.ndarray, "n_slots n_modes n_pol n_omega"], slots: np.ndarray, corrector: np.ndarray, rows: np.ndarray, @@ -1173,7 +1168,7 @@ class _Compensation: moments: Any """The trial's moments, read by the untransported witness only.""" nodes: list[float] - carried: Complex[np.ndarray, "n_used n_batch n_modes n_pol n_omega"] + carried: Complex[np.ndarray, "n_used n_modes n_pol n_omega"] dz: float treatment: str helpers: Helpers @@ -1273,6 +1268,8 @@ class AdamsEngine(Protocol): The march keeps the controller; an engine keeps the history's rows and does every state-sized operation, so that a backend can keep them where its state lives. :class:`HostHistory` is the reference; ``docs/dev/stepper.md`` states the contract. + SEP 52's phase 3 replaces it with :class:`solprop.backends.engines.AdamsEngine`, + which mirrors the native ``march::Engine``. """ dense: Any @@ -2270,7 +2267,7 @@ def _carrier_factors(nodes: list[float], count: int, shift: _Shift) -> _Factors: def _rows_read( - store: Complex[np.ndarray, "n_slots n_batch n_modes n_pol n_omega"], + store: Complex[np.ndarray, "n_slots n_modes n_pol n_omega"], slots: np.ndarray, factors: _Factors | None, ) -> list[_State]: diff --git a/src/solprop/backends/numpy_ref/passes.py b/src/solprop/backends/numpy_ref/passes.py index 329383d03..b59ee9c53 100644 --- a/src/solprop/backends/numpy_ref/passes.py +++ b/src/solprop/backends/numpy_ref/passes.py @@ -99,12 +99,12 @@ def stage_input( - state: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], - stage_sum: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], + state: Complex[np.ndarray, "n_modes n_pol n_omega"], + stage_sum: Complex[np.ndarray, "n_modes n_pol n_omega"], dz: float, - propagator: Complex[np.ndarray, "n_batch_p n_modes_p n_pol_p n_omega_p"], - noise: Complex[np.ndarray, "n_batch_n n_modes_n n_pol_n n_omega_n"] | None, -) -> Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]: + propagator: Complex[np.ndarray, "n_modes_p n_pol_p n_omega_p"], + noise: Complex[np.ndarray, "n_modes_n n_pol_n n_omega_n"] | None, +) -> Complex[np.ndarray, "n_modes n_pol n_omega"]: r"""Combine the stages, return to the lab frame and add the noise. ``u = exp(dPhi_i) (y + h sum_j a_ij k_j) + N(z_n + c_i h)``, the first pass of @@ -117,15 +117,14 @@ def stage_input( ---------- state : numpy.ndarray The state at the anchor, complex128, shape - ``(n_batch, n_modes, n_pol, n_omega)``, in the square root of watts. + ``(n_modes, n_pol, n_omega)``, in the square root of watts. stage_sum : numpy.ndarray ``sum_j a_ij k_j`` in the interaction variable, same shape and dtype, in the square root of watts per metre. dz : float Length of the step, m, positive. propagator : numpy.ndarray - ``exp(dPhi_i)``, dimensionless, broadcastable against the state, whose leading - axis of one it shares. + ``exp(dPhi_i)``, dimensionless, broadcastable against the state. noise : numpy.ndarray or None The shot-noise field already advanced to this position, ``N0 exp(i Im Phi(omega, z))``, in the square root of watts, or ``None`` when no @@ -140,7 +139,7 @@ def stage_input( -------- >>> import numpy as np >>> from solprop.backends.numpy_ref.passes import stage_input - >>> state = np.ones((1, 1, 1, 2), dtype=complex) + >>> state = np.ones((1, 1, 2), dtype=complex) >>> zero = np.zeros_like(state) >>> stage_input(state, zero, 0.1, 2.0 * np.ones_like(state), None).ravel() array([2.+0.j, 2.+0.j]) @@ -1780,9 +1779,9 @@ def stage_output( polarisation: Complex[np.ndarray, "n_modes n_pol n_omega"], prefactor: Complex[np.ndarray, "n_modes_p n_pol_p n_omega_p"], omega_window: Float[np.ndarray, " n_window"], - inverse_propagator: Complex[np.ndarray, "n_batch_i n_modes_i n_pol_i n_omega_i"], + inverse_propagator: Complex[np.ndarray, "n_modes_i n_pol_i n_omega_i"], absorber: Complex[np.ndarray, "n_modes_a n_pol_a n_omega_a"] | None = None, -) -> Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]: +) -> Complex[np.ndarray, "n_modes n_pol n_omega"]: r"""Normalise the polarisation and return it to the interaction picture. ``k_i = exp(-dPhi_i) (p(omega, z) W(omega) crop(Ptilde) + A_i)``, pass 7 of @@ -1817,7 +1816,7 @@ def stage_output( ------- numpy.ndarray The stage derivative in the interaction variable, complex128, shape - ``(n_batch, n_modes, n_pol, n_omega)``, in the square root of watts per metre. + ``(n_modes, n_pol, n_omega)``, in the square root of watts per metre. Examples -------- @@ -1825,7 +1824,7 @@ def stage_output( >>> from solprop.backends.numpy_ref.passes import stage_output >>> polarisation = np.ones((1, 1, 2), dtype=complex) >>> prefactor = np.array([[[-2.0j, 0.0j]]]) - >>> unit = np.ones((1, 1, 1, 2), dtype=complex) + >>> unit = np.ones((1, 1, 2), dtype=complex) >>> stage_output(polarisation, prefactor, np.ones(2), unit).ravel().tolist() [-2j, 0j] """ @@ -1836,10 +1835,10 @@ def stage_output( def accumulate( - total: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], - stage: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], + total: Complex[np.ndarray, "n_modes n_pol n_omega"], + stage: Complex[np.ndarray, "n_modes n_pol n_omega"], weight: float, -) -> Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]: +) -> Complex[np.ndarray, "n_modes n_pol n_omega"]: r"""Add one weighted stage derivative to a running accumulator. ``total + w k_i``. The solution and the embedded error are running accumulators @@ -1850,7 +1849,7 @@ def accumulate( ---------- total : numpy.ndarray The accumulator so far, complex128, shape - ``(n_batch, n_modes, n_pol, n_omega)``, in the square root of watts per metre. + ``(n_modes, n_pol, n_omega)``, in the square root of watts per metre. stage : numpy.ndarray The stage derivative ``k_i``, same shape and units. weight : float @@ -1865,7 +1864,7 @@ def accumulate( -------- >>> import numpy as np >>> from solprop.backends.numpy_ref.passes import accumulate - >>> total = np.zeros((1, 1, 1, 2), dtype=complex) + >>> total = np.zeros((1, 1, 2), dtype=complex) >>> accumulate(total, np.ones_like(total), 0.5).ravel() array([0.5+0.j, 0.5+0.j]) """ @@ -1873,9 +1872,9 @@ def accumulate( def error_norm( - error: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], - state: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], - state_new: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], + error: Complex[np.ndarray, "n_modes n_pol n_omega"], + state: Complex[np.ndarray, "n_modes n_pol n_omega"], + state_new: Complex[np.ndarray, "n_modes n_pol n_omega"], norm_mask: Bool[np.ndarray, " n_norm"], controller: PIController, ) -> float: @@ -1892,7 +1891,7 @@ def error_norm( ---------- error : numpy.ndarray The embedded difference of the propagated state, complex128, shape - ``(n_batch, n_modes, n_pol, n_omega)``, in the square root of watts. + ``(n_modes, n_pol, n_omega)``, in the square root of watts. state, state_new : numpy.ndarray The state before and after the trial step, same shape and units. norm_mask : numpy.ndarray @@ -1911,7 +1910,7 @@ def error_norm( >>> import numpy as np >>> from solprop.stepper import PIController >>> from solprop.backends.numpy_ref.passes import error_norm - >>> state = np.ones((1, 1, 1, 2), dtype=complex) + >>> state = np.ones((1, 1, 2), dtype=complex) >>> mask = np.ones(2, dtype=bool) >>> control = PIController(rtol=1e-2, atol=0.0) >>> round(error_norm(1e-2 * state, state, state, mask, control), 12) @@ -1930,11 +1929,11 @@ def error_norm( def accept_step( - state: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], - propagator: Complex[np.ndarray, "n_batch_p n_modes_p n_pol_p n_omega_p"], + state: Complex[np.ndarray, "n_modes n_pol n_omega"], + propagator: Complex[np.ndarray, "n_modes_p n_pol_p n_omega_p"], dz: float, - accumulated: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], -) -> Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]: + accumulated: Complex[np.ndarray, "n_modes n_pol n_omega"], +) -> Complex[np.ndarray, "n_modes n_pol n_omega"]: r"""Advance the state over a completed step. ``y <- exp(dPhi) (y + h sum_i b_i k_i)``. The interaction picture is anchored at the @@ -1947,7 +1946,7 @@ def accept_step( ---------- state : numpy.ndarray The state at the anchor, complex128, shape - ``(n_batch, n_modes, n_pol, n_omega)``, in the square root of watts. + ``(n_modes, n_pol, n_omega)``, in the square root of watts. propagator : numpy.ndarray ``exp(dPhi)`` over the whole step, dimensionless, broadcastable against the state. @@ -1966,7 +1965,7 @@ def accept_step( -------- >>> import numpy as np >>> from solprop.backends.numpy_ref.passes import accept_step - >>> state = np.ones((1, 1, 1, 2), dtype=complex) + >>> state = np.ones((1, 1, 2), dtype=complex) >>> accept_step(state, 2.0 * np.ones_like(state), 0.5, state).ravel() array([3.+0.j, 3.+0.j]) """ @@ -1974,9 +1973,9 @@ def accept_step( def apply_hard_mask( - state: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], + state: Complex[np.ndarray, "n_modes n_pol n_omega"], active_mask: Bool[np.ndarray, " n_active"], -) -> Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]: +) -> Complex[np.ndarray, "n_modes n_pol n_omega"]: r"""Zero the state outside the active band. ``y <- y * active_mask``, once per accepted step. The hard band limit is the one @@ -1989,7 +1988,7 @@ def apply_hard_mask( Parameters ---------- state : numpy.ndarray - The state, complex128, shape ``(n_batch, n_modes, n_pol, n_omega)``, in the + The state, complex128, shape ``(n_modes, n_pol, n_omega)``, in the square root of watts. active_mask : numpy.ndarray The active band, ``omega_min/2 < omega < 1.1 omega_max``, Boolean, shape @@ -2004,7 +2003,7 @@ def apply_hard_mask( -------- >>> import numpy as np >>> from solprop.backends.numpy_ref.passes import apply_hard_mask - >>> state = np.ones((1, 1, 1, 3), dtype=complex) + >>> state = np.ones((1, 1, 3), dtype=complex) >>> apply_hard_mask(state, np.array([True, True, False])).ravel() array([1.+0.j, 1.+0.j, 0.+0.j]) """ @@ -2012,12 +2011,12 @@ def apply_hard_mask( def apply_temporal_rate( - state: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], + state: Complex[np.ndarray, "n_modes n_pol n_omega"], rate: Float[np.ndarray, " n_rate"], dz: float, *, kind: GridKind, -) -> Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]: +) -> Complex[np.ndarray, "n_modes n_pol n_omega"]: r"""Apply the temporal absorber as an exact rate over one accepted step. ``y <- F[F^-1[y] exp(-alpha(t) h / 2)]``, the split-step absorber scheme. The @@ -2033,7 +2032,7 @@ def apply_temporal_rate( ---------- state : numpy.ndarray The state, complex128, shape - ``(n_batch, n_modes, n_pol, n_omega)``, in the square root of watts. On a + ``(n_modes, n_pol, n_omega)``, in the square root of watts. On a field grid its DC and Nyquist bins are real (above). rate : numpy.ndarray ``alpha(t)`` on the coarse time grid, float64, shape ``(n_t,)``, in 1/m, @@ -2052,7 +2051,7 @@ def apply_temporal_rate( -------- >>> import numpy as np >>> from solprop.backends.numpy_ref.passes import apply_temporal_rate - >>> state = np.array([[[[4.0 + 0.0j, 0.0j, 0.0j]]]]) + >>> state = np.array([[[4.0 + 0.0j, 0.0j, 0.0j]]]) >>> absorbed = apply_temporal_rate(state, np.zeros(4), 1.0, kind="field") >>> bool(np.allclose(absorbed, state)) True diff --git a/src/solprop/backends/numpy_ref/populations.py b/src/solprop/backends/numpy_ref/populations.py index 46c493a60..5264fda8b 100644 --- a/src/solprop/backends/numpy_ref/populations.py +++ b/src/solprop/backends/numpy_ref/populations.py @@ -36,7 +36,7 @@ def axis_populations( problem : PropagationProblem The translated problem. field : numpy.ndarray - One batch member's state, shape (n_modes, n_pol, n_omega). + The state, shape (n_modes, n_pol, n_omega). z : float Plane position in m. axis : numpy.ndarray diff --git a/src/solprop/backends/protocol.py b/src/solprop/backends/protocol.py index ec7733dc5..b578ae50d 100644 --- a/src/solprop/backends/protocol.py +++ b/src/solprop/backends/protocol.py @@ -912,7 +912,7 @@ def stats(self, spec: StatsSpec) -> dict[str, np.ndarray]: Returns ------- dict - Small host arrays keyed by statistic name, without a batch axis. + Small host arrays keyed by statistic name. """ ... diff --git a/src/solprop/backends/records.py b/src/solprop/backends/records.py new file mode 100644 index 000000000..e1e4cc59f --- /dev/null +++ b/src/solprop/backends/records.py @@ -0,0 +1,1274 @@ +# SPDX-FileCopyrightText: 2026 John C. Travers and contributors +# +# SPDX-License-Identifier: GPL-3.0-or-later + +"""The run records: what one run's march, session and monitor read (SEP 52 §5.1). + +Each record is a frozen dataclass formed once per run by the shared builders here, from +the :class:`~solprop.translate.problem.PropagationProblem`, the +:class:`~solprop.translate.run_request.RunRequest` and the +:class:`~solprop.backends.session.Start`, so that every number either backend uses has +one source. :func:`run_records` forms them all: + +- the method's policy: :class:`~solprop.stepper.multistep.MarchPolicy` for the Adams + march (:func:`adams_policy`), :class:`RungeKuttaPolicy` for the Runge-Kutta march and + the low-storage replay (:func:`runge_kutta_policy`), and the low-storage march's + :class:`LowStorageSettings`; +- the schedule, when one is fixed (a :class:`~solprop.stepper.schedule.FixedSchedule`); +- the :class:`PlanePlan`: the planes, step planes, statistics cadence, replay length and + the native session's capacities; +- the boundary monitor's :class:`MonitorRecords`: its :class:`MonitorConfig`, a + transverse edge's :class:`EdgeConfig`, and a resumed run's + :class:`MonitorContinuation`. + +The native adapter converts them into what its extension reads; the reference reads +them where it would otherwise derive the same facts. :class:`Notice` and +:class:`EdgeNotice` are the backend-neutral form of a session's boundary verdicts. +The statistics lowering stays each backend's own: the native plan is lowered in C++, +and the reference evaluates the :class:`~solprop.stats.StatsSpec` directly. + +Nothing here imports a backend, so the records are available without the native +package installed. +""" + +import dataclasses +import enum +import math +from collections.abc import Mapping, Sequence +from dataclasses import dataclass, field +from typing import TYPE_CHECKING, cast + +import numpy as np +from jaxtyping import Float, UInt + +from solprop.backends.exponential import carrier_reference +from solprop.backends.memory import low_storage_flows +from solprop.monitor import ( + _FLOOR, # pyright: ignore[reportPrivateUsage] - the floor the monitor judges by + FLOOR_MARGIN, + _count, # pyright: ignore[reportPrivateUsage] - one refusal of a count +) +from solprop.spatial_absorber import RAMP_K_FLOOR +from solprop.stepper.controller import START_GROWTH, PIController +from solprop.stepper.low_storage import LowStorageRungeKutta +from solprop.stepper.multistep import MarchPolicy, Multistep, march_policy +from solprop.stepper.schedule import LANDING_ULPS, FixedSchedule +from solprop.translate.routes import transverse_of +from solprop.transverse_monitor import BACKSTOP_READS, NOISE_FACTOR, REACH_GROWTH, READS + +if TYPE_CHECKING: + from solprop.backends.session import Start + from solprop.boundaries import SplitCrossing + from solprop.monitor import BoundaryMonitor + from solprop.translate.problem import PropagationProblem, StepperSettings + from solprop.translate.run_request import RunRequest + from solprop.transverse_monitor import TransverseEdge, TransverseStatus + +__all__ = [ + "ATTEMPT_CAPACITY", + "CALL_ATTEMPTS", + "CALL_SECONDS", + "PLANE_COLUMNS", + "ROW_CAPACITY", + "STATUSES", + "EdgeConfig", + "EdgeNotice", + "EdgeNoticeKind", + "EdgeResumption", + "LowStorageSettings", + "MonitorConfig", + "MonitorContinuation", + "MonitorRecords", + "Notice", + "NoticeKind", + "PlanePlan", + "Resumption", + "RunRecords", + "RungeKuttaPolicy", + "adams_policy", + "collar_rests", + "declared", + "edge_config", + "edge_resumption", + "low_storage_settings", + "monitor_config", + "monitor_continuation", + "monitor_records", + "plane_plan", + "resumption", + "run_records", + "runge_kutta_policy", +] + +CALL_ATTEMPTS = 256 +"""A march call's most attempts, as the native marches bound them (SEP 39).""" + +CALL_SECONDS = 0.1 +"""A march call's most wall time, s.""" + +ATTEMPT_CAPACITY = 4096 +"""The most attempts the native session holds between two events; a full buffer yields +a progress event carrying them (SEP 47 section 5.3).""" + +ROW_CAPACITY = 1024 +"""The most periodic rows it holds between two events, likewise.""" + +PLANE_COLUMNS = ("z", "collar_occupancy", "guard_low", "guard_high", "removed_fraction") +"""The monitor record's plane rows, in the native order (``watch::plane_columns``).""" + +STATUSES: tuple["TransverseStatus", ...] = ( + "clear", + "warned", + "stopped", + "insensitive", + "coarse", + "unwatched", +) +"""The transverse statuses in the native order (``transverse::Status``).""" + +_NUMBERS = ( + "launch_energy", + "launch_guard_low", + "launch_guard_high", + "launch_collar", + "guard_low_max", + "guard_high_max", + "guard_low_at", + "guard_high_at", + "collar_max", + "raw_removed_fraction", + "removed_signed", + "removed_fraction", + "floor_removed_fraction", + "raw_guard_max", +) +"""The numbers a resume reads from an earlier monitor record (``watch::Resumption``).""" + +_COUNTS = ("steps", "skipped", "collar_left_out_checks") +"""The counts it reads, each whole or refused (``solprop.monitor._count``).""" + +_EDGE_NUMBERS = ( + "launch_energy", + "noise_allowance", + "raw_ramp_removed", + "noise_ramp_removed", + "noise_share_in_ramp", + "declared_share_in_ramp", + "spatial_removed", + "current", + "current_baseline", + "strip_baseline", + "strip_floor", + "strip_max", + "returned_max", + "largest_gap", + "warned_at", +) +"""The numbers a resume reads from an earlier edge record (``transverse::Resumption``). +""" + + +# -- the method's policy ------------------------------------------------------------ + + +@dataclass(frozen=True) +class RungeKuttaPolicy: + """Every number the Runge-Kutta march reads and never changes during a run (SEP 45). + + The controller's decisions read the controller, the floor, the cap and the start-up + flag (:mod:`solprop.backends.rk_core`); the landings read the rest, the fields of + :class:`~solprop.stepper.multistep.MarchPolicy` of the same names. The native march + receives it whole, with the controller's fields, and holds no default of its own. + + Attributes + ---------- + controller : solprop.stepper.controller.PIController + The controller, whose gains, factors, lag, estimator order, tolerances and + acceptance rule the decisions read. + dz_min, dz_max : float + The floor and the cap on a proposal, m. + auto_init : bool + Whether the first accepted step takes the start-up rule. + stretch, z_end : float + The landing stretch (a ratio) and the run's end, m. + save_planes : tuple of float + The planes to land on, m; empty when the planes are interpolated. + interpolate : bool + Whether the save planes are observed densely. + hard_kinks : tuple of float + The operator's hard kinks, m. + breakpoints : tuple of float + The edges whose intervals cap a step, m: the operator's breakpoints with the + interval cap, otherwise only its first and last. + landing_ulps : int + :data:`~solprop.stepper.schedule.LANDING_ULPS`. + start_growth : float + :data:`~solprop.stepper.controller.START_GROWTH`. + quiet_interval : float or None + The boundary monitor's interval, m, which caps a step while the collar rests; + ``None`` for a session, whose monitor holds its own. + """ + + controller: PIController + dz_min: float + dz_max: float + auto_init: bool + stretch: float + z_end: float + save_planes: tuple[float, ...] + interpolate: bool + hard_kinks: tuple[float, ...] + breakpoints: tuple[float, ...] + landing_ulps: int + start_growth: float + quiet_interval: float | None + + +def runge_kutta_policy( + problem: "PropagationProblem", stepper: "StepperSettings" +) -> RungeKuttaPolicy: + """Return the Runge-Kutta march's policy of one run. + + Parameters + ---------- + problem : solprop.translate.problem.PropagationProblem + The problem, whose operator's breakpoints and hard kinks the landings read. + stepper : solprop.translate.problem.StepperSettings + The run's stepper settings. + + Returns + ------- + RungeKuttaPolicy + The record, with no quiet interval. + """ + breakpoints = np.asarray(problem.linear.phi_breakpoints, dtype=np.float64) + if not stepper.interval_cap: + # The cumulative table makes the flow exact across its intervals, so only the + # table's two ends bound a step (SEP 14 section 4.1). + breakpoints = breakpoints[[0, -1]] + interpolate = bool(stepper.interpolate_planes) + return RungeKuttaPolicy( + controller=stepper.controller, + dz_min=float(stepper.dz_min), + dz_max=float(stepper.dz_max), + auto_init=bool(stepper.auto_init), + stretch=float(stepper.stretch), + z_end=float(stepper.z_end), + # Interpolated planes are stepped across (SEP 14 section 4.3). + save_planes=() + if interpolate + else tuple(float(z) for z in np.asarray(stepper.save_planes)), + interpolate=interpolate, + hard_kinks=tuple(float(z) for z in np.asarray(problem.linear.hard_kinks)), + breakpoints=tuple(float(z) for z in breakpoints), + landing_ulps=LANDING_ULPS, + start_growth=float(START_GROWTH), + quiet_interval=None, + ) + + +def adams_policy( + problem: "PropagationProblem", + stepper: "StepperSettings", + method: Multistep, + *, + carrier: bool, +) -> MarchPolicy: + """Return the Adams march's policy of one run (SEP 39 section 4.2). + + Parameters + ---------- + problem : solprop.translate.problem.PropagationProblem + The problem: the operator's breakpoints and kinks, and the in-stage temporal + collar, whose largest rate bounds each order's stable step in the exponential + formulation. + stepper : solprop.translate.problem.StepperSettings + The run's stepper settings. + method : solprop.stepper.multistep.Multistep + The method. + carrier : bool + Whether the engine's carrier frame is active + (:class:`~solprop.backends.exponential.CarrierReference`). + + Returns + ------- + solprop.stepper.multistep.MarchPolicy + The record, with no quiet interval. + """ + responses = problem.responses + collar = responses.temporal_absorber + collar_rate = ( + 0.5 * float(np.max(collar)) + if collar is not None + and responses.absorber_scheme == "nonlinear" + and method.formulation == "exponential" + else 0.0 + ) + linear = problem.linear + return march_policy( + method, + dz_min=stepper.dz_min, + dz_max=stepper.dz_max, + stretch=stepper.stretch, + z_end=stepper.z_end, + save_planes=np.asarray(stepper.save_planes, dtype=np.float64).tolist(), + hard_kinks=np.asarray(linear.hard_kinks, dtype=np.float64).tolist(), + breakpoints=np.asarray(linear.phi_breakpoints, dtype=np.float64).tolist(), + interpolate=bool(stepper.interpolate_planes), + collar_rate=collar_rate, + carrier=carrier, + ) + + +@dataclass(frozen=True) +class LowStorageSettings: + """What the low-storage replay reads beside the Runge-Kutta policy (SEP 24). + + Attributes + ---------- + flows : int + The flow volumes a step holds on a CPU + (:func:`~solprop.backends.memory.low_storage_flows`): 0 recomputes each flow, + 1 holds a constant operator's table ``exp(h_t D / 2)``, 2 a varying operator's + ``P`` and ``Q``. + separable : bool + Whether the operator is a separable geometry's, whose increments are the + distances themselves. + origin : float + The operator's origin, ``linear.z_span[0]``, m, from which a table is built. + """ + + flows: int + separable: bool + origin: float + + +def low_storage_settings( + problem: "PropagationProblem", stepper: "StepperSettings" +) -> LowStorageSettings | None: + """Return the low-storage replay's settings, or ``None`` for another march. + + Parameters + ---------- + problem : solprop.translate.problem.PropagationProblem + The problem. + stepper : solprop.translate.problem.StepperSettings + The run's stepper settings. + + Returns + ------- + LowStorageSettings or None + The settings, when the run's integrator is the low-storage march. + """ + if not isinstance(stepper.integrator, LowStorageRungeKutta): + return None + return LowStorageSettings( + flows=low_storage_flows(problem, stepper, "native"), + separable=problem.linear.separable is not None, + origin=float(problem.linear.z_span[0]), + ) + + +def collar_rests(problem: "PropagationProblem", stepper: "StepperSettings") -> bool: + """Return whether a run's stages may leave a quiet temporal collar out (SEP 21). + + One rule for every backend: a modal or radial stage with the in-stage + (``"nonlinear"``) scheme and a temporal collar alone. The low-storage march, coupled + flows, the general modal route, a real-space collar sharing the term, and the + Cartesian, line and planar routes keep the collar in every stage (SEP 21 D15). + + Parameters + ---------- + problem : solprop.translate.problem.PropagationProblem + The problem. + stepper : solprop.translate.problem.StepperSettings + The run's stepper settings. + + Returns + ------- + bool + True when the term may be left out while the collar is quiet. + """ + responses = problem.responses + geometry = problem.transform.geometry + resting_geometry = geometry is None or ( + geometry.kind == "radial" and not np.any(geometry.absorber_rate) + ) + return ( + not isinstance(stepper.integrator, LowStorageRungeKutta) + and responses.temporal_absorber is not None + and responses.absorber_scheme == "nonlinear" + and problem.linear.coupled_flow is None + and problem.transform.modal_tables is None + and resting_geometry + ) + + +# -- the plane plan ------------------------------------------------------------------- + + +@dataclass(frozen=True, eq=False) +class PlanePlan: + """Where a run is observed and how its session buffers (SEP 47 section 5.3). + + Equality is by identity (``eq=False``) because ``planes`` is an array. + + Attributes + ---------- + z : float + Where the run starts, m. + planes : numpy.ndarray + The save planes, m, shape ``(n_z,)``. + next_plane : int + The first save plane strictly beyond `z`. + launch_pending : bool + Whether the plane at `z` is owed. + step_planes : int + Save every this many accepted steps; 0 for none. + dense : bool + Whether the planes are interpolated rather than landed on. + period : int + Accepted steps between the periodic statistics rows; 0 without rows. + replay_steps : int + A fixed schedule's steps; 0 without one. + replay_length : float + A fixed schedule's total length, m; 0.0 without one. + attempt_capacity, row_capacity : int + The most attempts and periodic rows the native session holds between two + events (:data:`ATTEMPT_CAPACITY`, :data:`ROW_CAPACITY`). + call_attempts : int + A march call's most attempts (:data:`CALL_ATTEMPTS`). + call_seconds : float + A march call's most wall time, s (:data:`CALL_SECONDS`). + """ + + z: float + planes: Float[np.ndarray, " n_z"] + next_plane: int + launch_pending: bool + step_planes: int + dense: bool + period: int + replay_steps: int + replay_length: float + attempt_capacity: int + row_capacity: int + call_attempts: int + call_seconds: float + + @property + def fixed_positions(self) -> tuple[float, ...] | None: # numpydoc ignore=RT01 + """Every position a plane is observed at, m, when the plan fixes them all. + + The start and the save planes; ``None`` with step planes or interpolated + planes, whose positions the march decides. + """ + if self.dense or self.step_planes: + return None + return (self.z, *(float(z) for z in self.planes)) + + +def plane_plan(request: "RunRequest", start: "Start") -> PlanePlan: + """Return a run's plane plan. + + Parameters + ---------- + request : solprop.translate.run_request.RunRequest + The run's request. + start : solprop.backends.session.Start + Where the run starts. + + Returns + ------- + PlanePlan + The plan, with the native session's capacities. + """ + observation = request.observation + spec = observation.stats + schedule = None if request.replay is None else request.replay.schedule + return PlanePlan( + z=float(start.z), + planes=np.ascontiguousarray(observation.save_planes, dtype=np.float64), + next_plane=int(start.plane_index), + launch_pending=bool(start.plane_pending), + step_planes=int(observation.step_planes), + dense=bool(observation.dense), + period=int(spec.period) if spec.periodic else 0, + replay_steps=0 if schedule is None else int(schedule.n_steps), + replay_length=0.0 if schedule is None else float(schedule.total_length), + attempt_capacity=ATTEMPT_CAPACITY, + row_capacity=ROW_CAPACITY, + call_attempts=CALL_ATTEMPTS, + call_seconds=CALL_SECONDS, + ) + + +# -- the boundary monitor ------------------------------------------------------------- + + +@dataclass(frozen=True, eq=False) +class MonitorConfig: + """A session's boundary monitor's configuration (``watch::Config``). + + Equality is by identity (``eq=False``) because ``rate`` is an array. + + Attributes + ---------- + policy : collections.abc.Mapping + The predictions' policy (:meth:`~solprop.monitor.BoundaryMonitor.watch_policy`'s + fields), its floor regions and launch floor zero: they are formed at the + launch. + floor_share : float + No light is judged below this share of the launch energy. + floor_margin : float + :data:`solprop.monitor.FLOOR_MARGIN`. + budget : float + The budget the finish's removal is judged against. + rate : numpy.ndarray + The collar's power rate on the coarse axis, 1/m, shape ``(n_t,)``. + can_rest : bool + Whether the route's stages may leave a quiet collar out (:func:`collar_rests`). + """ + + policy: Mapping[str, object] + floor_share: float + floor_margin: float + budget: float + rate: Float[np.ndarray, " n_t"] + can_rest: bool + + +def monitor_config( + monitor: "BoundaryMonitor", crossing: "SplitCrossing | None", *, can_rest: bool +) -> MonitorConfig: + """Return a fresh monitor's configuration for a run. + + The predictions' policy is the monitor's own with the split collar's crossing limit + the run gives it. + + Parameters + ---------- + monitor : solprop.monitor.BoundaryMonitor + The run's fresh monitor. + crossing : solprop.boundaries.SplitCrossing or None + The split collar's crossing limit, under the split scheme. + can_rest : bool + Whether the route's stages may leave a quiet collar out. + + Returns + ------- + MonitorConfig + The configuration. + """ + configured = dataclasses.replace(monitor, crossing=crossing) + policy = configured.watch_policy() + policy["floor_regions"] = [0.0, 0.0, 0.0, 0.0] + policy["launch_floor"] = 0.0 + return MonitorConfig( + policy=policy, + floor_share=_FLOOR, + floor_margin=FLOOR_MARGIN, + budget=float(monitor.check.budget), + rate=np.ascontiguousarray(monitor.weights[3], dtype=np.float64), + can_rest=bool(can_rest), + ) + + +@dataclass(frozen=True, eq=False) +class EdgeConfig: + """A session's transverse edge's configuration (``transverse::Config``, SEP 40). + + Every threshold is formed as :class:`~solprop.transverse_monitor.TransverseEdge` + forms it, so that a session compares the same numbers. Equality is by identity + (``eq=False``) because it holds arrays. + + Attributes + ---------- + faces : tuple of str + The faces judged, in the edge's order. + warn, stop : float + ``budget ** 2`` and ``(stop_factor * budget) ** 2``. + length : float + The run's length, m. + guard : float + The guard's width. + k_min : float + The safety ramp's least transverse wavenumber, 1/m. + k_floor : float + :data:`~solprop.spatial_absorber.RAMP_K_FLOOR`. + reads, backstop_reads, reach_growth, noise_factor : float + :mod:`solprop.transverse_monitor`'s constants of those names. + columns : numpy.ndarray + The spectral columns' weights, shape ``(n_columns,)``. + kappa : numpy.ndarray + Each column's transverse wavenumber, 1/m, shape ``(n_kappa,)``. + kappa_order : numpy.ndarray + Its ascending order, ``uint64``, shape ``(n_kappa,)``. + design : bool + Whether a safety ramp is designed. + ramp_rate : numpy.ndarray + The ramp's rate by column, 1/m; empty without a design. + collar, collar_rate : bool + Whether the design has a collar, and a collar rate. + declared : bool + Whether the run declares content above ``discard_above`` (:func:`declared`). + reads_edge : bool + Whether the backend reads the edge. + longest_step : float or None + A fixed schedule's longest step, m; ``None`` without one. + noise_share : float + The input noise's share of the weights on the first face. + noise_modes : float + The input noise's count of modes. + """ + + faces: tuple[str, ...] + warn: float + stop: float + length: float + guard: float + k_min: float + k_floor: float + reads: float + backstop_reads: float + reach_growth: float + noise_factor: float + columns: Float[np.ndarray, " n_columns"] + kappa: Float[np.ndarray, " n_kappa"] + kappa_order: UInt[np.ndarray, " n_kappa"] + design: bool + ramp_rate: Float[np.ndarray, " n_ramp"] + collar: bool + collar_rate: bool + declared: bool + reads_edge: bool + longest_step: float | None + noise_share: float + noise_modes: float + + +def declared(problem: "PropagationProblem") -> bool: + """Return whether a run declares content above ``discard_above``. + + Parameters + ---------- + problem : solprop.translate.problem.PropagationProblem + The problem. + + Returns + ------- + bool + True when its transverse specification sets ``discard_above``. + """ + design = transverse_of(problem.route) + return design is not None and design.specification.discard_above is not None + + +def edge_config( + edge: "TransverseEdge", + *, + steps: Float[np.ndarray, " n_steps"] | None, + declared: bool, + reads: bool = True, +) -> EdgeConfig: + """Return a fresh transverse edge's configuration for its launch's arguments. + + Parameters + ---------- + edge : solprop.transverse_monitor.TransverseEdge + The run's fresh edge. + steps : numpy.ndarray or None + A fixed schedule's steps, m; ``None`` for an adaptive run. + declared : bool + Whether the run declares content above ``discard_above``. + reads : bool, optional + Whether the backend reads the edge. + + Returns + ------- + EdgeConfig + The configuration. + """ + design = edge.design + faces = edge._faces # pyright: ignore[reportPrivateUsage] - the edge's own geometry + weights = edge._weights # pyright: ignore[reportPrivateUsage] + first = faces[0].edge + modes = max(int(np.count_nonzero(first)), 1) * max(int(edge.band.size), 1) + return EdgeConfig( + faces=tuple(face.name for face in faces), + warn=edge.budget**2, + stop=(edge.stop_factor * edge.budget) ** 2, + length=float(edge.length), + guard=float(edge._guard), # pyright: ignore[reportPrivateUsage] + k_min=float(edge.k_min), + k_floor=float(RAMP_K_FLOOR), + reads=float(READS), + backstop_reads=float(BACKSTOP_READS), + reach_growth=float(REACH_GROWTH), + noise_factor=float(NOISE_FACTOR), + columns=np.ascontiguousarray( + edge._columns, # pyright: ignore[reportPrivateUsage] + dtype=np.float64, + ), + kappa=np.ascontiguousarray( + edge._kappa, # pyright: ignore[reportPrivateUsage] + dtype=np.float64, + ), + kappa_order=np.ascontiguousarray( + edge._kappa_order, # pyright: ignore[reportPrivateUsage] + dtype=np.uint64, + ), + design=design is not None, + ramp_rate=np.empty(0, dtype=np.float64) + if design is None + else np.ascontiguousarray(design.ramp_rate, dtype=np.float64), + collar=design is not None and design.has_collar, + collar_rate=design is not None and design.collar_rate is not None, + declared=bool(declared), + reads_edge=bool(reads), + longest_step=float(np.max(steps)) if steps is not None and steps.size else None, + noise_share=float(np.sum(weights[first])) / float(np.sum(weights)), + noise_modes=float(modes), + ) + + +@dataclass(frozen=True) +class Resumption: + """What a resume reads of an earlier monitor record (``watch::Resumption``). + + Only the record's types are read: which numbers are restored, their defaults and the + fallbacks are the session's. + + Attributes + ---------- + record : bool + Whether there is a record at all. + collar_quiet : bool + ``bool(record.get("collar_quiet", False))``. + numbers : collections.abc.Mapping + Each number present in the record, by field; the counts already whole. + unwatched : tuple or None + The record's ``unwatched`` stretches, when it lists them. + resumed_at : tuple or None + The record's ``resumed_at``, when it lists them. + planes : tuple or None + The record's plane rows by :data:`PLANE_COLUMNS` (NaN for ``None``), when they + have those five columns. + """ + + record: bool + collar_quiet: bool = False + numbers: Mapping[str, float] = field(default_factory=dict[str, float]) + unwatched: tuple[tuple[float, ...], ...] | None = None + resumed_at: tuple[float, ...] | None = None + planes: tuple[tuple[float, ...], ...] | None = None + + +def _number(record: Mapping[str, object], name: str) -> float | None: + """Return a record's number, or ``None`` when it is absent or not a number. + + As ``solprop.monitor._number`` reads it, without its default, which the session's + reconstruction applies. + """ + value = record.get(name) + if isinstance(value, bool) or not isinstance(value, int | float): + return None + return float(value) + + +def resumption(record: Mapping[str, object] | None) -> Resumption: + """Return what a resume reads of an earlier execution's monitor record. + + Read as :meth:`~solprop.monitor.BoundaryMonitor.resume` reads it: each number + present, the lists, and the plane rows when they have the five columns. A count + that is not whole is refused as that resume refuses it, once the record's launch + energy makes it read the counts. + + Parameters + ---------- + record : collections.abc.Mapping or None + The ``boundary_monitor`` record of the latest earlier execution, or ``None``. + + Returns + ------- + Resumption + What the session restores from. + """ + if record is None: + return Resumption(record=False) + numbers: dict[str, float] = {} + for name in _NUMBERS: + value = _number(record, name) + if value is not None: + numbers[name] = value + launch = numbers.get("launch_energy", math.nan) + if launch > 0.0: + # `BoundaryMonitor.resume` reads the counts only past its launch test. + quiet = _number(record, "quiet_checks") + if quiet is not None and math.isfinite(quiet): + numbers["quiet_checks"] = float(_count(quiet, "quiet_checks")) + for name in _COUNTS: + value = _number(record, name) + numbers[name] = float(_count(0.0 if value is None else value, name)) + listed = record.get("unwatched") + resumed = record.get("resumed_at") + rows = record.get("planes") + planes = None + if isinstance(rows, Mapping) and set(cast("Mapping[str, object]", rows)) == set( + PLANE_COLUMNS + ): + columns = cast("Mapping[str, list[float | None]]", rows) + planes = tuple( + tuple(math.nan if one is None else float(one) for one in columns[name]) + for name in PLANE_COLUMNS + ) + return Resumption( + record=True, + collar_quiet=bool(record.get("collar_quiet", False)), + numbers=numbers, + unwatched=tuple( + tuple(float(one) for one in cast("list[float]", stretch)) + for stretch in cast("list[object]", listed) + ) + if isinstance(listed, list) + else None, + resumed_at=tuple(float(one) for one in cast("list[float]", resumed)) + if isinstance(resumed, list) + else None, + planes=planes, + ) + + +@dataclass(frozen=True) +class EdgeResumption: + """What a resume reads of an earlier edge record (``transverse::Resumption``). + + Attributes + ---------- + record : bool + Whether there is a record at all. + numbers : collections.abc.Mapping + Each number present in the record, by field. + edge_baseline, edge_max, edge_max_raw : tuple of float or None + The per-face lists, once the record's launch energy and baseline make the + resume read them; each ``None`` otherwise or without a value per face. + reads : int or None + The reads, made whole, under the same condition. + transverse_status : str or None + The record's status, when it names one. + """ + + record: bool + numbers: Mapping[str, float] = field(default_factory=dict[str, float]) + edge_baseline: tuple[float, ...] | None = None + edge_max: tuple[float, ...] | None = None + edge_max_raw: tuple[float, ...] | None = None + reads: int | None = None + transverse_status: str | None = None + + +def _per_face( + record: Mapping[str, object], name: str, faces: int +) -> tuple[float, ...] | None: + """Return a record's per-face list, or ``None`` when it has no value per face. + + As ``TransverseEdge.resume``'s ``numbers`` reads it. + """ + value = record.get(name) + if not isinstance(value, list) or len(cast("list[object]", value)) != faces: + return None + return tuple(float(cast("float", one)) for one in cast("list[object]", value)) + + +def edge_resumption(record: object, faces: int) -> EdgeResumption: + """Return what a resume reads of an earlier execution's transverse edge record. + + Read as :meth:`~solprop.transverse_monitor.TransverseEdge.resume` reads it: each + number present, and, once the record's launch energy and baseline make that resume + read further, each per-face maximum that has a value per face and the reads, made + whole as it makes them. + + Parameters + ---------- + record : object + The ``transverse`` entry of the earlier monitor record, or anything else when + there is none. + faces : int + The edge's face count. + + Returns + ------- + EdgeResumption + What the session restores from. + """ + if not isinstance(record, Mapping): + return EdgeResumption(record=False) + mapping = cast("Mapping[str, object]", record) + numbers: dict[str, float] = {} + for name in _EDGE_NUMBERS: + value = _number(mapping, name) + if value is not None: + numbers[name] = value + baseline = _per_face(mapping, "edge_baseline", faces) + launch = numbers.get("launch_energy", math.nan) + status = mapping.get("transverse_status") + if not (launch > 0.0 and baseline is not None): + # `TransverseEdge.resume` reads the maxima and the count only past its test of + # the launch and the baseline: a record it ignores is not read further. + return EdgeResumption( + record=True, + numbers=numbers, + transverse_status=status if isinstance(status, str) else None, + ) + reads = _number(mapping, "reads") + return EdgeResumption( + record=True, + numbers=numbers, + edge_baseline=baseline, + edge_max=_per_face(mapping, "edge_max", faces), + edge_max_raw=_per_face(mapping, "edge_max_raw", faces), + reads=int(0.0 if reads is None else reads), + transverse_status=status if isinstance(status, str) else None, + ) + + +@dataclass(frozen=True) +class MonitorContinuation: + """A monitored resume's continuation, as a session reads it. + + Attributes + ---------- + record : Resumption + The earlier monitor record's resumption. + edge : EdgeResumption + Its transverse edge record's. + plane : bool + Whether the run resumes at a saved plane. + unwatched : tuple of tuple of float + The ``(start, end)`` stretches, m, run since without a monitor. + """ + + record: Resumption + edge: EdgeResumption + plane: bool + unwatched: tuple[tuple[float, float], ...] + + +def monitor_continuation( + monitor: "BoundaryMonitor", continuation: Mapping[str, object] +) -> MonitorContinuation: + """Return a monitored resume's continuation records. + + Parameters + ---------- + monitor : solprop.monitor.BoundaryMonitor + The run's fresh monitor, whose transverse edge gives the face count. + continuation : collections.abc.Mapping + As :attr:`solprop.backends.session.Start.monitor`. + + Returns + ------- + MonitorContinuation + The records. + """ + record = continuation.get("record") + mapping = ( + cast("Mapping[str, object]", record) if isinstance(record, Mapping) else None + ) + edge = monitor.transverse + faces = 0 if edge is None else len(edge._faces) # pyright: ignore[reportPrivateUsage] + stretches = cast("Sequence[tuple[float, float]]", continuation.get("unwatched", ())) + return MonitorContinuation( + record=resumption(mapping), + edge=edge_resumption( + None if mapping is None else mapping.get("transverse"), faces + ), + plane=bool(continuation.get("plane", False)), + unwatched=tuple((float(start), float(end)) for start, end in stretches), + ) + + +@dataclass(frozen=True, eq=False) +class MonitorRecords: + """A monitored run's boundary records. + + Attributes + ---------- + config : MonitorConfig + The temporal monitor's configuration. + edge : EdgeConfig or None + A free-space run's transverse edge's, or ``None`` without one. + continuation : MonitorContinuation or None + A monitored resume's continuation, or ``None`` when the start carries none + (a session may still be continued before it watches). + """ + + config: MonitorConfig + edge: EdgeConfig | None + continuation: MonitorContinuation | None + + +def monitor_records( + problem: "PropagationProblem", request: "RunRequest", start: "Start" +) -> MonitorRecords | None: + """Return a run's boundary records, or ``None`` without a monitor. + + Parameters + ---------- + problem : solprop.translate.problem.PropagationProblem + The problem: the split collar's crossing limit, the quiet-collar rule and a + declared transverse specification. + request : solprop.translate.run_request.RunRequest + The run's request, with its fresh monitor. + start : solprop.backends.session.Start + Where the run starts, with any monitor continuation. + + Returns + ------- + MonitorRecords or None + The records. + """ + monitor = request.boundary.monitor + if monitor is None: + return None + stepper = request.stepper + edge = monitor.transverse + schedule = stepper.schedule + return MonitorRecords( + config=monitor_config( + monitor, + problem.split_crossing if monitor.split else None, + can_rest=collar_rests(problem, stepper), + ), + edge=None + if edge is None + else edge_config( + edge, + steps=None if schedule is None else np.asarray(schedule.steps), + declared=declared(problem), + ), + continuation=None + if start.monitor is None + else monitor_continuation(monitor, start.monitor), + ) + + +# -- the boundary verdicts ------------------------------------------------------------ + + +class NoticeKind(enum.IntEnum): + """A boundary monitor's notice's kind (``session::Notice::Kind``). + + Attributes + ---------- + NONE : int + No notice. + GUARD : int + A guard occupancy, at the launch or at a check. + REMOVAL : int + The collar's removal at the end of a run. + CROSSING : int + A replay's step passes the split collar's cap. + READING : int + The split found nothing where the state holds light. + DIVISION : int + Light after a launch that held none, which the monitor divides by. + TRANSVERSE : int + The transverse edge's verdict (:class:`EdgeNotice`). + """ + + NONE = 0 + GUARD = 1 + REMOVAL = 2 + CROSSING = 3 + READING = 4 + DIVISION = 5 + TRANSVERSE = 6 + + +@dataclass(frozen=True) +class Notice: + """A boundary monitor's verdict (``session::Notice``): a warning or a stop. + + Attributes + ---------- + kind : NoticeKind + What it is. + launch : bool + For a guard: whether it is the launch's judgement. + z : float + Where, m. + value : float + A guard's amplitude, or the removed fraction. + cap : float + A crossing's cap, m. + step : float + A crossing's longest step before the next check, m. + """ + + kind: NoticeKind + launch: bool + z: float + value: float + cap: float + step: float + + +class EdgeNoticeKind(enum.IntEnum): + """A transverse edge's notice's kind (``transverse::Notice::Kind``). + + Attributes + ---------- + NONE : int + No notice. + RAMP_BAND : int + The launch's share in the safety ramp's band. + UNWATCHED : int + The backend does not read the edge. + CLIP : int + A face's clip of the launch. + INSENSITIVE : int + The input noise's allowance exceeds the warning. + COARSE : int + A fixed schedule's longest step outruns the read spacing. + FACE : int + A face's judged share. + RETURNED : int + A requested collar's returned share. + RAMP_REMOVAL : int + The safety ramp's removal. + """ + + NONE = 0 + RAMP_BAND = 1 + UNWATCHED = 2 + CLIP = 3 + INSENSITIVE = 4 + COARSE = 5 + FACE = 6 + RETURNED = 7 + RAMP_REMOVAL = 8 + + +@dataclass(frozen=True) +class EdgeNotice: + """A transverse edge's verdict (``transverse::Notice``): a warning or a stop. + + Attributes + ---------- + kind : EdgeNoticeKind + What it is. + stop : bool + Whether it stops the run. + face : int + The face it names, by index. + value : float + The share, allowance or step it names. + z : float + Where, m. + spacing : float + A coarse schedule's read spacing, m. + warned_at : float + Where the edge first warned, for a face's stop, m; NaN where it did not. + """ + + kind: EdgeNoticeKind + stop: bool + face: int + value: float + z: float + spacing: float + warned_at: float + + +# -- one run's records ---------------------------------------------------------------- + + +@dataclass(frozen=True, eq=False) +class RunRecords: + """Every record of one run, formed once by :func:`run_records` (SEP 52 §5.1). + + Attributes + ---------- + policy : solprop.stepper.multistep.MarchPolicy or RungeKuttaPolicy + The method's policy: the Adams march's, or the Runge-Kutta march's, which the + low-storage replay reads too. + low_storage : LowStorageSettings or None + The low-storage replay's settings; ``None`` for another march. + schedule : solprop.stepper.schedule.FixedSchedule or None + The fixed schedule, when the run replays one. + planes : PlanePlan + The plane plan. + monitor : MonitorRecords or None + The boundary records; ``None`` without a monitor. + """ + + policy: MarchPolicy | RungeKuttaPolicy + low_storage: LowStorageSettings | None + schedule: FixedSchedule | None + planes: PlanePlan + monitor: MonitorRecords | None + + +def run_records( + problem: "PropagationProblem", request: "RunRequest", start: "Start" +) -> RunRecords: + """Return one run's records. + + Parameters + ---------- + problem : solprop.translate.problem.PropagationProblem + The translated problem. + request : solprop.translate.run_request.RunRequest + The run's request. + start : solprop.backends.session.Start + Where the run starts; its field is not read. + + Returns + ------- + RunRecords + The records. + + Examples + -------- + >>> from solprop.backends.records import run_records + >>> from solprop.backends.session import Start + >>> from solprop.grid import EnvelopeGrid + >>> from solprop.nonlinear.gnlse import problem_from + >>> from solprop.pulses import SechPulse + >>> from solprop.stepper import PIController + >>> from solprop.translate.assembly import assemble + >>> grid = EnvelopeGrid.from_limits(1.0, 835e-9, (500e-9, 2e-6), 1e-12) + >>> gnlse = problem_from([0.0, 0.0, -1.183e-26], omega_ref=grid.omega0, + ... gamma=0.11, shock=False) + >>> launch = SechPulse(wavelength=835e-9, width=100e-15, peak_power=9.8) + >>> problem, request = assemble(grid, gnlse.mode, [gnlse.kerr], launch.build(grid), + ... controller=PIController(rtol=1e-8, atol=1e-16), + ... save_planes=[0.01], dz_max=0.01, dz_init=1e-3, + ... density=1.0, shock=False, prefactor_index=1.0) + >>> records = run_records(problem, request, Start.entrance(problem)) + >>> type(records.policy).__name__, records.planes.planes.tolist() + ('RungeKuttaPolicy', [0.01]) + >>> records.low_storage is None, records.monitor is None + (True, True) + """ + stepper = request.stepper + method = stepper.integrator + policy: MarchPolicy | RungeKuttaPolicy + if isinstance(method, Multistep): + reference = carrier_reference(problem, method) + policy = adams_policy( + problem, + stepper, + method, + carrier=reference is not None and reference.active, + ) + else: + policy = runge_kutta_policy(problem, stepper) + return RunRecords( + policy=policy, + low_storage=low_storage_settings(problem, stepper), + schedule=stepper.schedule, + planes=plane_plan(request, start), + monitor=monitor_records(problem, request, start), + ) diff --git a/src/solprop/backends/rk_core.py b/src/solprop/backends/rk_core.py index b5bd67914..fe45141f2 100644 --- a/src/solprop/backends/rk_core.py +++ b/src/solprop/backends/rk_core.py @@ -4,13 +4,13 @@ """The Runge-Kutta march's controller as functions of its policy and state (SEP 45). -The decisions of ``HostDriver._march_to`` -- the -equivalent norm and acceptance of an attempt, the start-up rule, the next proposal -after an accepted step and the shrink after a rejected one -- each as a function of the -run's :class:`RungeKuttaPolicy` and the values it decides on, with no backend or field -in reach. The march calls them, and SEP 45 section 4 specifies them; the native march -executes the same algorithm and is held to these functions by the decision traces of -``tests/data/rk_traces.json.gz``. +The decisions of ``HostDriver._march_to`` -- the equivalent norm and acceptance of an +attempt, the start-up rule, the next proposal after an accepted step and the shrink +after a rejected one -- each as a function of the run's +:class:`~solprop.backends.records.RungeKuttaPolicy` and the values it decides on, with +no backend or field in reach. The march calls them, and SEP 45 section 4 specifies +them; the native march executes the same algorithm and is held to these functions by +the decision traces of ``tests/data/rk_traces.json.gz``. Every function performs the operations of the code it was extracted from, in the same order. The controller's state is its own proposal: a cap or a landing shortens the @@ -22,63 +22,18 @@ """ from collections.abc import Sequence -from dataclasses import dataclass -from solprop.stepper.controller import START_GROWTH, PIController, limit_step -from solprop.translate.problem import StepperSettings +from solprop.backends.records import RungeKuttaPolicy +from solprop.stepper.controller import START_GROWTH, limit_step __all__ = [ - "RungeKuttaPolicy", "decide", "propose", "reject", - "rk_policy", "start_up", ] -@dataclass(frozen=True) -class RungeKuttaPolicy: - """Every number the Runge-Kutta march's controller reads during a run (SEP 45). - - Attributes - ---------- - controller : solprop.stepper.controller.PIController - The controller, whose gains, factors, lag, estimator order and acceptance rule - the decisions read. - dz_min, dz_max : float - The floor and the cap on a proposal, m. - auto_init : bool - Whether the first accepted step takes the start-up rule. - """ - - controller: PIController - dz_min: float - dz_max: float - auto_init: bool - - -def rk_policy(stepper: StepperSettings) -> RungeKuttaPolicy: - """Return the run's controller policy from its stepper settings. - - Parameters - ---------- - stepper : solprop.translate.problem.StepperSettings - The translated stepper settings. - - Returns - ------- - RungeKuttaPolicy - Its controller, floor, cap and start-up flag. - """ - return RungeKuttaPolicy( - controller=stepper.controller, - dz_min=stepper.dz_min, - dz_max=stepper.dz_max, - auto_init=stepper.auto_init, - ) - - def decide( policy: RungeKuttaPolicy, norm: float, dz: float, proposal: float ) -> tuple[float, bool]: diff --git a/src/solprop/boundaries.py b/src/solprop/boundaries.py index 5566c965c..062e9278e 100644 --- a/src/solprop/boundaries.py +++ b/src/solprop/boundaries.py @@ -966,8 +966,8 @@ class SplitCrossing: ---------- buckets : numpy.ndarray Each bin's bucket, ``uint8``, shape ``(n_modes, n_omega)``: the fastest of its - batch members, polarisations and positions along the guide; evanescent and - inactive bins in the last. + polarisations and positions along the guide; evanescent and inactive bins in + the last. width : float The collar's width at its narrower edge, s. launch_limit : float @@ -986,7 +986,7 @@ class SplitCrossing: >>> from solprop.boundaries import SplitCrossing, split_rate_buckets >>> buckets = split_rate_buckets(np.array([[1e-12, 1e-10]])) >>> crossing = SplitCrossing(buckets, 1e-13, launch_limit=0.05) - >>> occupancy = crossing.occupancy(np.array([[[[1.0, 1e-4]]]], dtype=complex)) + >>> occupancy = crossing.occupancy(np.array([[[1.0, 1e-4]]], dtype=complex)) >>> # The fast bin holds 1e-8 of the energy and is set aside: the slow bin's limit, >>> # 50 mm at 1e-12 s/m, a little shorter at its bucket's edge. >>> round(crossing.limit(occupancy) * 1e3, 1) @@ -1008,7 +1008,7 @@ def __post_init__(self) -> None: def occupancy( self, - field_spectrum: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], + field_spectrum: Complex[np.ndarray, "n_modes n_pol n_omega"], columns: Float[np.ndarray, " n_modes"] | None = None, ) -> Float[np.ndarray, " n_buckets"]: """Return a state's energy per bucket, on the host: the reference reduction. @@ -1033,8 +1033,8 @@ def occupancy( def _block(rows: slice) -> Float[np.ndarray, " n_buckets"]: """Return a block of modes' energy per bucket.""" - energy = np.einsum("bmpw,bmpw->mw", re[:, rows], re[:, rows]) + np.einsum( - "bmpw,bmpw->mw", im[:, rows], im[:, rows] + energy = np.einsum("mpw,mpw->mw", re[rows], re[rows]) + np.einsum( + "mpw,mpw->mw", im[rows], im[rows] ) if weights is not None: energy = energy * weights[rows, None] @@ -1044,7 +1044,7 @@ def _block(rows: slice) -> Float[np.ndarray, " n_buckets"]: minlength=SPLIT_RATE_BUCKETS, ).astype(np.float64) - return np.sum(parallel_rows(spectrum.shape[1], spectrum.nbytes, _block), axis=0) + return np.sum(parallel_rows(spectrum.shape[0], spectrum.nbytes, _block), axis=0) def limit(self, occupancy: Float[np.ndarray, " n_buckets"]) -> float: """Return the crossing limit of an occupancy, m. diff --git a/src/solprop/driver/resume.py b/src/solprop/driver/resume.py index 011e61d3e..f8ad2edd4 100644 --- a/src/solprop/driver/resume.py +++ b/src/solprop/driver/resume.py @@ -119,8 +119,8 @@ def _start( field = state.plane.field if problem.transform.geometry is None: field = power_amplitude( - field[np.newaxis], power_weight_at(problem, state.plane.z), inverse=True - )[0] + field, power_weight_at(problem, state.plane.z), inverse=True + ) integrator = request.integration.method record, march = _continue_march( simulation, diff --git a/src/solprop/monitor.py b/src/solprop/monitor.py index 8e04a7af1..63f91ddb8 100644 --- a/src/solprop/monitor.py +++ b/src/solprop/monitor.py @@ -505,7 +505,7 @@ def build( return cls(check, weights, grid.kind, columns, split, grid.length / CHECKS) def power( - self, field_spectrum: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"] + self, field_spectrum: Complex[np.ndarray, "n_modes n_pol n_omega"] ) -> Float[np.ndarray, " n_t"]: """Return the time-domain energy density summed over everything but time. @@ -526,11 +526,11 @@ def power( ) density = np.abs(samples) ** 2 if self.columns is not None: - density = density * self.columns[None, :, None, None] + density = density * self.columns[:, None, None] return density.reshape(-1, density.shape[-1]).sum(axis=0) def energies( - self, field_spectrum: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"] + self, field_spectrum: Complex[np.ndarray, "n_modes n_pol n_omega"] ) -> BoundaryEnergies: """Return a state's energy and its four weighted energies, on the host. @@ -555,7 +555,7 @@ def energies( >>> grid = EnvelopeGrid.from_limits(1.0, 800e-9, (600e-9, 1.2e-6), 1e-12) >>> monitor = BoundaryMonitor.build(grid, None, BoundaryCheck()) >>> pulse = np.exp(-((grid.t / 50e-15) ** 2)).astype(complex) - >>> energies = monitor.energies(np.fft.fft(pulse)[None, None, None, :]) + >>> energies = monitor.energies(np.fft.fft(pulse)[None, None, :]) >>> bool(np.isclose(energies.total, np.sum(np.abs(pulse) ** 2))) True """ @@ -564,7 +564,7 @@ def energies( def split_sums( self, - field_spectrum: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], + field_spectrum: Complex[np.ndarray, "n_modes n_pol n_omega"], step: float, ) -> Float[np.ndarray, " 5"]: """Return what one accepted step's split collar takes, and what it finds. @@ -607,7 +607,7 @@ def split_sums( >>> edge = 0.5 * grid.n_t * grid.dt >>> pulse = np.exp(-(((grid.t - 0.98 * edge) / 10e-15) ** 2)).astype(complex) >>> taken, found, low, high, collar = monitor.split_sums( - ... np.fft.fft(pulse)[None, None, None, :], 0.01 + ... np.fft.fft(pulse)[None, None, :], 0.01 ... ) >>> bool(taken > 0.0), bool(low < 1e-20 * found), bool(high > 0.5 * found) (True, True, True) @@ -650,7 +650,7 @@ def launch( energies: BoundaryEnergies, z: float, *, - floor: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"] | None = None, + floor: Complex[np.ndarray, "n_modes n_pol n_omega"] | None = None, ) -> None: """Record the launch, and warn or refuse if it already fills a guard. diff --git a/src/solprop/output/hdf5.py b/src/solprop/output/hdf5.py index 5f7c35462..e1d020774 100644 --- a/src/solprop/output/hdf5.py +++ b/src/solprop/output/hdf5.py @@ -79,7 +79,7 @@ from solprop._display import Displayed, line, tail from solprop.exceptions import OutputError from solprop.output._writer_lock import WriterLock -from solprop.output.metadata import RunMetadata +from solprop.output.metadata import SCHEMA_VERSION, RunMetadata from solprop.output.records import ( HISTORY_FIELDS, STEP_STATS_INDEX, @@ -197,11 +197,7 @@ def _overwrite_text(file: h5py.File, name: str, text: str) -> None: def _write_march(group: h5py.Group, snapshot: MarchSnapshot) -> None: """Store a march snapshot, its position last, so that a torn write reads as none.""" group.attrs["z"] = math.nan - # The dataset keeps the leading axis of one it has always had, so that every - # reader of the format reads it (SEP 47 section 6.7). - _overwrite( - group, "state", np.asarray(snapshot.state, dtype=np.complex128)[np.newaxis] - ) + _overwrite(group, "state", np.asarray(snapshot.state, dtype=np.complex128)) _overwrite(group, "positions", np.asarray(snapshot.positions, dtype=np.float64)) _overwrite(group, "slots", np.asarray(snapshot.slots, dtype=np.int64)) rows = group.require_group("rows") @@ -233,11 +229,18 @@ def _read_march(file: h5py.File, z: float) -> MarchSnapshot | None: stored = float(np.asarray(group.attrs.get("z", math.nan))) if stored != z: return None + try: + held = np.asarray(_dataset(group, "state")[()]) + except (KeyError, ValueError) as error: + logger.warning("ignoring the unreadable march snapshot in the file: %s", error) + return None + # Outside the tolerant reads: an earlier layout is refused, not ignored. + state = _trajectory_state(held) try: rows = _group(group, "rows") return MarchSnapshot( z=stored, - state=_trajectory_state(np.asarray(_dataset(group, "state")[()])), + state=state, proposal=float(np.asarray(group.attrs["proposal"])), method=str(group.attrs["method"]), order=int(np.asarray(group.attrs["order"])), @@ -260,20 +263,25 @@ def _read_march(file: h5py.File, z: float) -> MarchSnapshot | None: def _trajectory_state( stored: Complex[np.ndarray, "*shape"], ) -> Complex[np.ndarray, "n_modes n_pol n_omega"]: - """Return a stored march state without the leading axis of one its file keeps. + """Return a stored march state, ``(n_modes, n_pol, n_omega)``, refusing any other. - Every file writes ``(1, n_modes, n_pol, n_omega)``; ``(n_modes, n_pol, n_omega)`` - is taken as it is. A leading axis longer than one is a stacked group, which a - single trajectory's snapshot never is, and is refused rather than squeezed. + Before output schema 3 a file kept the state with a leading axis of one, ``(1, + n_modes, n_pol, n_omega)``; such a snapshot is refused by name, not squeezed. """ - if stored.ndim == 4 and stored.shape[0] == 1: - return stored[0] if stored.ndim == 3: return stored - msg = ( - f"the march snapshot's state has shape {stored.shape}, expected one " - "trajectory's (1, n_modes, n_pol, n_omega)" - ) + if stored.ndim == 4 and stored.shape[0] == 1: + msg = ( + f"the march snapshot's state has the layout {stored.shape} of a file " + f"written before output schema {SCHEMA_VERSION}, with a leading axis of " + "one; this version of solprop reads (n_modes, n_pol, n_omega) only. " + "Resume the file with the solprop version that wrote it" + ) + else: + msg = ( + f"the march snapshot's state has shape {stored.shape}, expected one " + "trajectory's (n_modes, n_pol, n_omega)" + ) raise OutputError(msg) diff --git a/src/solprop/output/metadata.py b/src/solprop/output/metadata.py index f9ffe3373..f2bd1a229 100644 --- a/src/solprop/output/metadata.py +++ b/src/solprop/output/metadata.py @@ -87,17 +87,18 @@ _ARRAY_PART = re.compile(r"\.__(?:ndarray|imaginary)__$") """The key of an array record's real or imaginary part, dropped from its stem.""" -SCHEMA_VERSION = 2 +SCHEMA_VERSION = 3 """Version of the saved-output layout; incremented when a stored field changes meaning. Schema 1 is the first released layout, 0.1.0's: the record types of this module, written by :class:`solprop.output.memory.MemoryOutput` and :class:`solprop.output.hdf5.HDF5Output` as ``docs/guide/results/output.md`` describes -them. Schema 2 records ``DataPulse.spectral_phase``. A record that declares any other -version is refused by name (:class:`RunMetadata`): files written by an earlier release -or by development versions of solprop, which declared output schemas up to 28, are not -read, and nor is a file from a later release. Read such a file with the solprop version -that wrote it. +them. Schema 2 records ``DataPulse.spectral_phase``. Schema 3 stores a march snapshot's +state and engine rows without the leading axis of one they kept before. A record that +declares any other version is refused by name (:class:`RunMetadata`): files written by +an earlier release or by development versions of solprop before 0.1.0, which declared +output schemas up to 28, are not read, and nor is a file from a later release. Read such +a file with the solprop version that wrote it. """ PEAK_FIELDS_ANNOTATION = "material_peak_fields" diff --git a/src/solprop/output/records.py b/src/solprop/output/records.py index 9278fbf9a..b3e4850c4 100644 --- a/src/solprop/output/records.py +++ b/src/solprop/output/records.py @@ -911,7 +911,7 @@ def _adopted(values: np.ndarray) -> np.ndarray: The driver's route (:func:`solprop.driver.run.run_problem`) for what an observation holds. Only for an array that nothing writes to again, as :meth:`solprop.backends.protocol.HostStepping.observe` promises, and never for a - batch member, whose view would pin the whole batch. + view into a larger array, which it would pin. - Returns: the same array, read-only. """ @@ -1310,8 +1310,7 @@ class MarchSnapshot: # numpydoc ignore=PR02 The accepted position, m. state : numpy.ndarray The accepted state, shape ``(n_modes, n_pol, n_omega)``, in the internal FFT - convention, exactly as the backend held it. A file stores it with a leading - axis of one, the layout it has always had (SEP 47 section 6.7). + convention, exactly as the backend held it. proposal : float The step the controller proposed next, m. method : str @@ -1331,8 +1330,8 @@ class MarchSnapshot: # numpydoc ignore=PR02 rows : collections.abc.Mapping The engine's rows by name: the samples (``"store"``) and, for a compensated exponential march, the sampled states and rates and each sample's flow - weights, in the engine's own layout, which keeps a leading axis of one after - the row axis. + weights, in the engine's own layout, the row axis first: a state-shaped row + is ``(n_slots, n_modes, n_pol, n_omega)``. history_key : str What defines the history (:attr:`solprop.stepper.multistep.Multistep.history_key`), which the resuming @@ -1355,7 +1354,7 @@ class MarchSnapshot: # numpydoc ignore=PR02 ... z=0.5, state=np.ones((1, 1, 4), complex), proposal=0.1, method="M()", ... order=2, rejections=0, last_rejected=False, ... positions=np.array([0.4, 0.5]), identities=("accepted", "predicted"), - ... slots=np.array([0, 1]), rows={"store": np.zeros((4, 1, 1, 1, 4), complex)}, + ... slots=np.array([0, 1]), rows={"store": np.zeros((4, 1, 1, 4), complex)}, ... history_key="adams", ... ) >>> snapshot.nbytes diff --git a/src/solprop/translate/assembly.py b/src/solprop/translate/assembly.py index 3a6a0a5fa..d0a657164 100644 --- a/src/solprop/translate/assembly.py +++ b/src/solprop/translate/assembly.py @@ -381,7 +381,7 @@ def _split_crossing( crossing = SplitCrossing(buckets, width, fastest=fastest) return replace( crossing, - launch_limit=crossing.limit(crossing.occupancy(launch[np.newaxis], columns)), + launch_limit=crossing.limit(crossing.occupancy(launch, columns)), ) @@ -880,7 +880,7 @@ def _general_prefactor( - Returns: one interval of degree zero with value shape ``(n_modes, n_pol, n_omega)``. """ - n_modes = linear.value_shape[1] + n_modes = linear.value_shape[0] values = np.zeros((1, 1, n_modes, n_pol, grid.n_omega), dtype=np.complex128) values[:, :, :, :, grid.active_mask] = grid.crop_scale return PiecewiseChebyshev( diff --git a/src/solprop/translate/collar_check.py b/src/solprop/translate/collar_check.py index 4354faa42..5d97773a6 100644 --- a/src/solprop/translate/collar_check.py +++ b/src/solprop/translate/collar_check.py @@ -520,7 +520,7 @@ def _half( # noqa: PLR0917 - called positionally def design_check( # noqa: PLR0917 - called positionally space: RadialGrid | FreeGrid | PlanarGrid, rate: Float[np.ndarray, " n_nodes"], - launch: Complex[np.ndarray, "n_batch n_columns n_pol n_omega"], + launch: Complex[np.ndarray, "n_columns n_pol n_omega"], dispersion: SpatialDispersion, z_max: float, width: float, @@ -534,7 +534,7 @@ def design_check( # noqa: PLR0917 - called positionally rate : numpy.ndarray The collar's power rate on the nodes, 1/m. launch : numpy.ndarray - The noise-free launch, canonical layout; its first batch member is checked. + The noise-free launch, canonical layout. dispersion : solprop.spatial.SpatialDispersion The medium and the temporal grid. z_max : float @@ -548,7 +548,7 @@ def design_check( # noqa: PLR0917 - called positionally The verdict, ``"pending"`` for a launch that is zero everywhere. """ started = time.perf_counter() - columns = np.asarray(launch[0], dtype=np.complex128) + columns = np.asarray(launch, dtype=np.complex128) if not np.any(columns): return CheckResult("pending", reason="no launch to check yet") returned, nodes, planes, outgrown = _estimate( diff --git a/src/solprop/translate/expansion.py b/src/solprop/translate/expansion.py index 5e2c87388..77c0ccfdb 100644 --- a/src/solprop/translate/expansion.py +++ b/src/solprop/translate/expansion.py @@ -154,8 +154,8 @@ class ParameterExpansion: present, contributes the rest. terms : numpy.ndarray The expansion coefficients, shape ``(n_terms, *value_shape)``. For the kernel - contract's section 3 the value shape is ``(1, n_modes, n_pol, - n_omega)`` and the array is complex128. + contract's section 3 the value shape is ``(n_modes, n_pol, n_omega)`` and + the array is complex128. weights : solprop.translate.chebyshev.PiecewiseChebyshev The scalar tables along z, value shape ``(n_terms,)``, float64: the products ``T_j(p_hat(z))`` for a plain quantity and their integrals from the start of the diff --git a/src/solprop/translate/linop.py b/src/solprop/translate/linop.py index 1e27c085f..208695521 100644 --- a/src/solprop/translate/linop.py +++ b/src/solprop/translate/linop.py @@ -199,9 +199,9 @@ _FLOOR_PROBES = 3 _FLOOR_STEP_FRACTION = 1e-9 -_OperatorAt = Callable[[float], Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]] +_OperatorAt = Callable[[float], Complex[np.ndarray, "n_modes n_pol n_omega"]] _OperatorOf = Callable[ - [Mapping[str, float]], Complex[np.ndarray, "n_batch n_modes n_pol n_omega"] + [Mapping[str, float]], Complex[np.ndarray, "n_modes n_pol n_omega"] ] @@ -229,11 +229,9 @@ class LinearOperator: the schedule read the same field either way. phi_coefficients : numpy.ndarray or None Chebyshev coefficients of ``Phi``, complex128, shape ``(n_intervals, n_deg, - 1, n_modes, n_pol, n_omega)``, present when `phi_representation` is - ``"table"`` and also for ``"constant"``, whose one interval of degree 1 is - exact; absent for ``"expansion"``. The axis of one is the value layout the - native build reads; the state has none. Units are dimensionless: ``Phi`` is an - exponent. + n_modes, n_pol, n_omega)``, present when `phi_representation` is ``"table"`` + and also for ``"constant"``, whose one interval of degree 1 is exact; absent + for ``"expansion"``. Units are dimensionless: ``Phi`` is an exponent. phi_degrees : numpy.ndarray Degree used on each interval, ``numpy.intp``, shape ``(n_intervals,)``. Diagnostic: the zero padding makes it unnecessary for evaluation. @@ -242,7 +240,7 @@ class LinearOperator: Steps land on them; the far more numerous split edges are not tracked here and nothing lands on those. phi_constant : numpy.ndarray or None - ``D`` itself, complex128, shape ``(1, n_modes, n_pol, n_omega)``, + ``D`` itself, complex128, shape ``(n_modes, n_pol, n_omega)``, present exactly when `phi_representation` is ``"constant"``, so that :func:`delta_phi` returns ``D (z2 - z1)`` to the last bit rather than the difference of two Clenshaw sums. @@ -297,18 +295,18 @@ class LinearOperator: >>> table = constant_phi(np.full(4, -1j), 2.0) >>> table.phi_representation, table.phi_degrees.tolist() ('constant', [1]) - >>> bool(np.array_equal(delta_phi(table, 0.0, 0.5), np.full((1, 1, 1, 4), -0.5j))) + >>> bool(np.array_equal(delta_phi(table, 0.0, 0.5), np.full((1, 1, 4), -0.5j))) True """ phi_representation: PhiRepresentation phi_breakpoints: Float[np.ndarray, " n_edges"] phi_coefficients: ( - Complex[np.ndarray, "n_intervals n_deg n_batch n_modes n_pol n_omega"] | None + Complex[np.ndarray, "n_intervals n_deg n_modes n_pol n_omega"] | None ) phi_degrees: Int[np.ndarray, " n_degrees"] hard_kinks: Float[np.ndarray, " n_kinks"] - phi_constant: Complex[np.ndarray, "n_batch_c n_modes_c n_pol_c n_omega_c"] | None + phi_constant: Complex[np.ndarray, "n_modes_c n_pol_c n_omega_c"] | None phi_expansion: ParameterExpansion | None = field(default=None) beta0: PiecewiseChebyshev | None = field(default=None) beta1: PiecewiseChebyshev | None = field(default=None) @@ -450,7 +448,7 @@ def _check_representation(self) -> None: def value_shape(self) -> tuple[int, ...]: # numpydoc ignore=RT01 """Shape of ``Phi`` at one position. - ``(1, n_modes, n_pol, n_omega)``, whichever representation + ``(n_modes, n_pol, n_omega)``, whichever representation carries it. Examples @@ -458,7 +456,7 @@ def value_shape(self) -> tuple[int, ...]: # numpydoc ignore=RT01 >>> import numpy as np >>> from solprop.translate.linop import constant_phi >>> constant_phi(np.full(4, -1j), 2.0).value_shape - (1, 1, 1, 4) + (1, 1, 4) """ if self.separable is not None: return self.separable.value_shape @@ -584,31 +582,24 @@ def nbytes(self) -> int: # numpydoc ignore=RT01 def _value_shape( operator: Complex[np.ndarray, "*value n_omega"], n_pol: int -) -> Complex[np.ndarray, "1 n_modes n_pol n_omega"]: - """Bring an operator to the value layout ``(1, n_modes, n_pol, n_omega)``. +) -> Complex[np.ndarray, "n_modes n_pol n_omega"]: + """Bring an operator to the state layout ``(n_modes, n_pol, n_omega)``. - The operator's values keep a leading broadcast axis of one, the layout the native - build reads; the state itself has none (SEP 47 section 6.2). - - - ``operator``: ``D`` as ``(n_omega,)``, in the state's ``(n_modes, n_pol, - n_omega)`` or already in the value layout. - - Returns: the operator with four axes. + - ``operator``: ``D`` as ``(n_omega,)`` or already in the state layout. """ values = np.asarray(operator) - if values.ndim == 4: - return values if values.ndim == 3: - return values[np.newaxis] + return values if values.ndim != 1: msg = ( - "the operator must have shape (n_omega,), (n_modes, n_pol, n_omega) or " - f"(1, n_modes, n_pol, n_omega), got {values.shape}" + "the operator must have shape (n_omega,) or (n_modes, n_pol, n_omega), " + f"got {values.shape}" ) raise TranslationError(msg) if n_pol not in (1, 2): msg = f"n_pol must be 1 or 2 (conventions.md section 5), got {n_pol!r}" raise TranslationError(msg) - return np.broadcast_to(values, (1, 1, n_pol, values.size)) + return np.broadcast_to(values, (1, n_pol, values.size)) def tabulate( @@ -788,8 +779,8 @@ def constant_phi( Parameters ---------- operator : numpy.ndarray - ``D(omega)`` in 1/m, complex, shape ``(n_omega,)`` or ``(n_batch, n_modes, - n_pol, n_omega)``. + ``D(omega)`` in 1/m, complex, shape ``(n_omega,)`` or ``(n_modes, n_pol, + n_omega)``. z_max : float Length the run covers, m, positive and finite. n_pol : int, optional @@ -822,7 +813,7 @@ def constant_phi( >>> from solprop.translate.linop import constant_phi >>> table = constant_phi(np.array([0.0, -1j, -2j]), 1.0) >>> table.phi_coefficients.shape, table.z_span - ((1, 2, 1, 1, 1, 3), (0.0, 1.05)) + ((1, 2, 1, 1, 3), (0.0, 1.05)) >>> table.phi_representation 'constant' """ @@ -1264,7 +1255,7 @@ def with_power_weight( LinearOperator Operator with a constant, parameter-expansion or z-table weight row. """ - n_pol = linear.value_shape[2] + n_pol = linear.value_shape[1] sample = _weight_sampler(modes, grid, n_pol, index) if index is not None or all(mode.z_constant for mode in modes): constant = hold(sample(0.0)) @@ -1464,14 +1455,14 @@ def _log_weight_sampler( - Returns: minus one half the logarithmic weight in canonical state shape. """ - def sample(z: float) -> Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]: + def sample(z: float) -> Complex[np.ndarray, "n_modes n_pol n_omega"]: """Sample the index at the requested position. - ``z``: positions within the translated span, in metres. """ - columns = np.zeros((1, len(modes), n_pol, grid.n_omega), dtype=complex) + columns = np.zeros((len(modes), n_pol, grid.n_omega), dtype=complex) for number, mode in enumerate(modes): - columns[0, number][:, grid.active_mask] = -0.5 * log_power_weight( + columns[number][:, grid.active_mask] = -0.5 * log_power_weight( mode, grid.omega[grid.active_mask], z, index=index ) return columns @@ -1491,12 +1482,12 @@ def _power_phi( """ if linear.phi_representation == "constant" or index is not None: return linear - n_pol = linear.value_shape[2] + n_pol = linear.value_shape[1] frozen = [mode for mode in modes if isinstance(mode, ParameterisedMode)] def value_at( values: Mapping[str, float], - ) -> Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]: + ) -> Complex[np.ndarray, "n_modes n_pol n_omega"]: """Sample the weight inside the declared parameter box. - Returns: weight values in the mode and frequency layout. @@ -1597,7 +1588,7 @@ def endpoint_phi( def _correction( z: Float[np.ndarray, " n_z"], - ) -> Complex[np.ndarray, "n_z n_batch n_modes n_pol n_omega"]: + ) -> Complex[np.ndarray, "n_z n_modes n_pol n_omega"]: """Sample the endpoint function along z. - ``z``: positions within the translated span, in metres. @@ -1809,7 +1800,7 @@ def _z_table_operator( def sample( z: Float[np.ndarray, " n_z"], *_: object - ) -> Complex[np.ndarray, "n_z n_batch n_modes n_pol n_omega"]: + ) -> Complex[np.ndarray, "n_z n_modes n_pol n_omega"]: """Return the operator at every position of a Chebyshev node set. - ``z``: positions along the waveguide, m, shape ``(n_z,)``. @@ -1889,14 +1880,14 @@ def _operator_sampler( # noqa: PLR0917 - called positionally ) -> _OperatorAt: """Return a callable giving ``D`` in the canonical layout at one position. - - Returns: ``z -> D`` of shape ``(1, n_modes, n_pol, n_omega)``. + - Returns: ``z -> D`` of shape ``(n_modes, n_pol, n_omega)``. """ - def operator_at(z: float) -> Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]: + def operator_at(z: float) -> Complex[np.ndarray, "n_modes n_pol n_omega"]: """Return the linear operator at one position, in the canonical layout. - ``z``: position along the waveguide, m. - - Returns: ``D(omega, z)`` in 1/m, shape ``(1, n_modes, n_pol, n_omega)``. + - Returns: ``D(omega, z)`` in 1/m, shape ``(n_modes, n_pol, n_omega)``. """ shared = _shared_frame(modes[0], frame, z) columns = [ @@ -1932,16 +1923,16 @@ def _operator_of_parameters( # noqa: PLR0917 - called positionally - ``frame``: its ``beta_0`` and ``beta_1`` follow the frozen reference mode, so the frame subtraction is a function of the same parameters and needs no dimension of its own. - - Returns: ``values -> D`` of shape ``(1, n_modes, n_pol, n_omega)``. + - Returns: ``values -> D`` of shape ``(n_modes, n_pol, n_omega)``. """ parameterised = [one for one in modes if isinstance(one, ParameterisedMode)] def operator_of( values: Mapping[str, float], - ) -> Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]: + ) -> Complex[np.ndarray, "n_modes n_pol n_omega"]: """Return the linear operator at one point of the parameter space. - - Returns: ``D(omega; values)`` in 1/m, shape ``(1, n_modes, n_pol, n_omega)``. + - Returns: ``D(omega; values)`` in 1/m, shape ``(n_modes, n_pol, n_omega)``. """ frozen = [one.at_parameters(values) for one in parameterised] # One frame for the whole set, from the reference mode at *these* parameter @@ -1962,7 +1953,7 @@ def operator_of( def _canonical( columns: Sequence[Complex[np.ndarray, " n_omega"]], n_pol: int -) -> Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]: +) -> Complex[np.ndarray, "n_modes n_pol n_omega"]: """Stack one operator per mode into the canonical state layout. - ``n_pol``: polarisation components; ``D`` is the same for both components of one @@ -1970,8 +1961,8 @@ def _canonical( """ # The repeat makes the array that owns the result's memory, so that a caller can # hold it (solprop._ownership.hold), as it could not hold a view of it. - stacked = np.stack(columns)[np.newaxis, :, np.newaxis, :] - return np.repeat(stacked, n_pol, axis=2) + stacked = np.stack(columns)[:, np.newaxis, :] + return np.repeat(stacked, n_pol, axis=1) def _shared_frame(reference: Mode, frame: Frame, z: float) -> Frame: @@ -2302,7 +2293,7 @@ def _frame_tables( def delta_phi( table: LinearOperator, z1: float, z2: float, *, helpers: Helpers | None = None -) -> Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]: +) -> Complex[np.ndarray, "n_modes n_pol n_omega"]: """Return the exponent of the exact linear flow between two positions. ``Phi(z2) - Phi(z1)``, so that ``exp(delta_phi(table, z1, z2))`` is the linear @@ -2326,7 +2317,7 @@ def delta_phi( Returns ------- numpy.ndarray - Dimensionless exponent, complex, shape ``(n_batch, n_modes, n_pol, n_omega)``. + Dimensionless exponent, complex, shape ``(n_modes, n_pol, n_omega)``. Raises ------ @@ -2361,7 +2352,7 @@ def phi_rate( *, before: bool = False, helpers: Helpers | None = None, -) -> Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]: +) -> Complex[np.ndarray, "n_modes n_pol n_omega"]: """Return the rate ``D(z) = dPhi/dz`` of the representation that defines the flow. A constant or separable operator returns ``D`` itself. A table's interval edges @@ -2385,7 +2376,7 @@ def phi_rate( Returns ------- numpy.ndarray - ``D(z)`` in 1/m, shape ``(n_batch, n_modes, n_pol, n_omega)``. + ``D(z)`` in 1/m, shape ``(n_modes, n_pol, n_omega)``. Examples -------- @@ -2510,10 +2501,10 @@ def _separable_phase( # Six complex-equivalent arrays cover square root, masks and products. # This independent host evaluation uses the same 64 MiB workspace ceiling. chunk = max(1, (64 << 20) // (8 * 16 * len(coefficients.k_squared))) - for first in range(0, coefficients.value_shape[1], chunk): + for first in range(0, coefficients.value_shape[0], chunk): rows = slice(first, first + chunk) operator, _ = separable_values(coefficients, rows) - result[:, rows] = (np.imag(operator) if imaginary else operator) * distance + result[rows] = (np.imag(operator) if imaginary else operator) * distance return result @@ -2643,11 +2634,11 @@ def _rates(phase: np.ndarray, valid: np.ndarray) -> np.ndarray: widest: np.ndarray | None = None for z, before, section in _rate_positions(table): rate = _channel_rate(table, z, before, section) - n_batch, n_modes, n_pol, n_omega = rate.shape + n_modes, n_pol, n_omega = rate.shape phase = -np.imag(rate.reshape(-1, n_omega))[:, order] valid = np.broadcast_to(sorted_active[None, :], phase.shape) - each = _rates(phase, valid).reshape(n_batch, n_modes, n_pol, n_omega) - each = each.max(axis=(0, 2)) + each = _rates(phase, valid).reshape(n_modes, n_pol, n_omega) + each = each.max(axis=1) widest = each if widest is None else np.maximum(widest, each) assert widest is not None buckets = np.empty(widest.shape, dtype=np.uint8) @@ -2695,7 +2686,7 @@ def _rate_positions( def _channel_rate( table: LinearOperator, z: float, before: bool, section: int | None -) -> Complex[np.ndarray, "n_batch n_modes n_pol n_omega"]: +) -> Complex[np.ndarray, "n_modes n_pol n_omega"]: """Return each channel's own operator at one position, ``D`` plus its flow's. A coupled flow's channels share the carrier's ``D``: each one's own propagation @@ -2720,12 +2711,12 @@ def _channel_rate( diagonal = np.diagonal(flow.generator, axis1=2, axis2=3) # (i, d, w, m) flat = diagonal.reshape(-1, *diagonal.shape[2:]) own = np.tensordot(flow_weights(flow, z, section), flat, axes=1).T - return rate + own[None, :, None, :] + return rate + own[:, None, :] def unitary_phase( table: LinearOperator, z: float -) -> Float[np.ndarray, "n_batch n_modes n_pol n_omega"]: +) -> Float[np.ndarray, "n_modes n_pol n_omega"]: """Return the phase that advances the shot-noise field to a position. The modified noise realisation is drawn once and carried along the fibre by the @@ -2745,7 +2736,7 @@ def unitary_phase( Returns ------- numpy.ndarray - Phase in radians, shape ``(n_batch, n_modes, n_pol, n_omega)``. + Phase in radians, shape ``(n_modes, n_pol, n_omega)``. Raises ------ diff --git a/src/solprop/translate/problem.py b/src/solprop/translate/problem.py index 488e77fcb..6390caf08 100644 --- a/src/solprop/translate/problem.py +++ b/src/solprop/translate/problem.py @@ -1167,14 +1167,9 @@ def _check_shapes(self) -> None: "match the state" ) raise TranslationError(msg) - # The operator's values keep a leading broadcast axis of one, the layout the - # native build reads; the state has none (SEP 47 section 6.2). value = self.linear.value_shape - if value != (1, n_modes, n_pol, n_omega): - msg = ( - f"linear carries values of shape {value}, expected " - f"{(1, n_modes, n_pol, n_omega)}" - ) + if value != self.shape: + msg = f"linear carries values of shape {value}, expected {self.shape}" raise TranslationError(msg) value = self.responses.prefactor_value_shape if value != (n_modes, n_pol, n_omega): diff --git a/src/solprop/translate/radial.py b/src/solprop/translate/radial.py index e6b3ac47b..57edd225d 100644 --- a/src/solprop/translate/radial.py +++ b/src/solprop/translate/radial.py @@ -517,7 +517,7 @@ def _admitted(self, model: SpatialDispersion, z: float) -> np.ndarray: def _log_ratio( self, model: SpatialDispersion, z: float - ) -> Complex[np.ndarray, "1 n_r n_pol n_omega"]: + ) -> Complex[np.ndarray, "n_r n_pol n_omega"]: """Return ``-1/2 ln(Re k_z / Re k_z(0))`` in the canonical layout. - ``z``: position in m at which to evaluate it. @@ -534,10 +534,10 @@ def _log_ratio( np.log(wave / reference), ) values = np.where(self._mask, -0.5 * ratio, 0.0) - shape = (1, values.shape[0], self._n_pol, values.shape[1]) - return np.broadcast_to(values[None, :, None, :], shape).astype(np.complex128) + shape = (values.shape[0], self._n_pol, values.shape[1]) + return np.broadcast_to(values[:, None, :], shape).astype(np.complex128) - def sample(self, z: float) -> Complex[np.ndarray, "1 n_r n_pol n_omega"]: + def sample(self, z: float) -> Complex[np.ndarray, "n_r n_pol n_omega"]: """Evaluate the term's endpoint function along the run path. - ``z``: position in m. @@ -547,7 +547,7 @@ def sample(self, z: float) -> Complex[np.ndarray, "1 n_r n_pol n_omega"]: def value_at( self, values: Mapping[str, float] - ) -> Complex[np.ndarray, "1 n_r n_pol n_omega"]: + ) -> Complex[np.ndarray, "n_r n_pol n_omega"]: """Evaluate the endpoint function at one point of the parameter box. - Returns: dimensionless endpoint function, canonical layout. @@ -605,7 +605,7 @@ def _linear_roundoff_target( errors, rounding * (conditioning + absolute + np.abs(frame)) ) operator, _ = sampled(dispersion, z) - peaks = np.maximum(peaks, np.abs(operator[0, :, 0])) + peaks = np.maximum(peaks, np.abs(operator[:, 0])) denominator = np.maximum(peaks, ENVELOPE_FLOOR * float(peaks.max())) active = dispersion.grid.active_mask[None, :] & (denominator > 0) relative = np.zeros_like(peaks) diff --git a/src/solprop/translate/responses.py b/src/solprop/translate/responses.py index ebf610a99..a30445756 100644 --- a/src/solprop/translate/responses.py +++ b/src/solprop/translate/responses.py @@ -2060,7 +2060,7 @@ def prefactor_value_shape(self) -> tuple[int, ...]: # numpydoc ignore=RT01 ``(n_modes, n_pol, n_omega)``, whichever representation carries it. """ if self.separable is not None: - return self.separable.value_shape[1:] + return self.separable.value_shape if self.prefactor_expansion is not None: return self.prefactor_expansion.value_shape # `_check_prefactor_representation` has already made the narrowing true; a cast diff --git a/src/solprop/translate/spatial.py b/src/solprop/translate/spatial.py index f9d559ad3..7636898b9 100644 --- a/src/solprop/translate/spatial.py +++ b/src/solprop/translate/spatial.py @@ -499,12 +499,12 @@ def n_channels(self) -> int: # numpydoc ignore=RT01 return 1 if self.kappa_sq is None else int(self.kappa_sq.shape[0]) @property - def value_shape(self) -> tuple[int, int, int, int]: # numpydoc ignore=RT01 + def value_shape(self) -> tuple[int, int, int]: # numpydoc ignore=RT01 """Full state shape represented by these shared coefficients. - ``(1, n_modes, n_pol, n_omega)`` without allocating that volume. + ``(n_modes, n_pol, n_omega)`` without allocating that volume. """ - return (1, self.kperp_sq.size, self.n_pol, self.k_squared.size) + return (self.kperp_sq.size, self.n_pol, self.k_squared.size) @dataclass(frozen=True) @@ -641,7 +641,7 @@ def separable_values( coefficients: SeparableOperator, columns: slice | None = None, ) -> tuple[ - Complex[np.ndarray, "1 n_modes n_pol n_omega"], + Complex[np.ndarray, "n_modes n_pol n_omega"], Complex[np.ndarray, "n_modes n_pol n_omega"], ]: """Evaluate a separable operator and prefactor for the host reference. @@ -659,7 +659,7 @@ def separable_values( Returns ------- tuple of numpy.ndarray - Linear rate in m^-1, shape ``(1, n_modes, n_pol, n_omega)``, and nonlinear + Linear rate in m^-1, shape ``(n_modes, n_pol, n_omega)``, and nonlinear prefactor, shape ``(n_modes, n_pol, n_omega)``. """ root, propagating, attenuation = longitudinal_root(coefficients, columns) @@ -681,7 +681,7 @@ def separable_values( prefactor *= coefficients.active shape = (root.shape[0], coefficients.n_pol, len(coefficients.k_squared)) return ( - np.broadcast_to(operator[:, None, :], shape)[None], + np.broadcast_to(operator[:, None, :], shape), np.broadcast_to(prefactor[:, None, :], shape), ) diff --git a/src/solprop/translate/stage.py b/src/solprop/translate/stage.py index c45eaae30..be77037ab 100644 --- a/src/solprop/translate/stage.py +++ b/src/solprop/translate/stage.py @@ -80,8 +80,8 @@ class StageTerms: Attributes ---------- phi : numpy.ndarray or None - The terms of ``Phi``, complex128, shape ``(n_phi_terms, 1, n_modes, - n_pol, n_omega)``. One term for a constant operator, ``n_terms`` for an + The terms of ``Phi``, complex128, shape ``(n_phi_terms, n_modes, n_pol, + n_omega)``. One term for a constant operator, ``n_terms`` for an expansion, and ``None`` for the z table, which has no such form. prefactor : numpy.ndarray or None The terms of the prefactor, complex128, shape ``(n_pre_terms, n_modes, n_pol, @@ -97,10 +97,10 @@ class StageTerms: >>> from solprop.translate.stage import StageTerms >>> table = constant_phi(np.array([-1j, -2j]), 1.0) >>> StageTerms(phi=table.phi_constant[np.newaxis], prefactor=None).phi.shape - (1, 1, 1, 1, 2) + (1, 1, 1, 2) """ - phi: Complex[np.ndarray, "n_phi_terms n_batch n_modes n_pol n_omega"] | None + phi: Complex[np.ndarray, "n_phi_terms n_modes n_pol n_omega"] | None prefactor: Complex[np.ndarray, "n_pre_terms n_modes_p n_pol_p n_omega_p"] | None separable: SeparableOperator | None = None @@ -403,7 +403,7 @@ def terms(problem: PropagationProblem) -> StageTerms: def _phi_terms( linear: LinearOperator, -) -> Complex[np.ndarray, "n_phi_terms n_batch n_modes n_pol n_omega"] | None: +) -> Complex[np.ndarray, "n_phi_terms n_modes n_pol n_omega"] | None: """Return the terms of ``Phi``, or ``None`` for the z table.""" if linear.phi_constant is not None: return linear.phi_constant[np.newaxis] diff --git a/src/solprop/translate/transverse.py b/src/solprop/translate/transverse.py index 65bd1a5aa..b52726e25 100644 --- a/src/solprop/translate/transverse.py +++ b/src/solprop/translate/transverse.py @@ -152,7 +152,7 @@ def band_wavenumbers( def _column_power( space: RadialGrid | FreeGrid | PlanarGrid, - field0: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], + field0: Complex[np.ndarray, "n_modes n_pol n_omega"], mask: np.ndarray, ) -> Float[np.ndarray, " n_columns"]: """Return each spectral column's launch energy over the requested band. @@ -160,14 +160,14 @@ def _column_power( - ``field0``: the noise-free launch, canonical layout. - Returns: energy per column in the field's own units. """ - density = np.empty(field0.shape[1]) + density = np.empty(field0.shape[0]) def columns(rows: slice) -> None: """Sum one block of columns; each column's own sum is unchanged by blocking.""" - density[rows] = np.sum(np.abs(field0[:, rows][..., mask]) ** 2, axis=(0, 2, 3)) + density[rows] = np.sum(np.abs(field0[rows][..., mask]) ** 2, axis=(1, 2)) # A large launch's single pass took 2 s of translation (solprop issue #347). - parallel_rows(field0.shape[1], field0.nbytes, columns) + parallel_rows(field0.shape[0], field0.nbytes, columns) return np.asarray(space.spectral_weights) * density @@ -192,7 +192,7 @@ def _intermediate_return( # noqa: PLR0917 - the design and its weights widths: tuple[float, ...], starts: list[float], alpha0: float, - field0: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], + field0: Complex[np.ndarray, "n_modes n_pol n_omega"], space: RadialGrid | FreeGrid | PlanarGrid, grid: TimeGrid, k: Float[np.ndarray, " n"], @@ -212,7 +212,7 @@ def _intermediate_return( # noqa: PLR0917 - the design and its weights """ band = np.nonzero(grid.inband_mask & grid.active_mask)[0] power = np.asarray(space.spectral_weights)[:, None] * np.sum( - np.abs(np.asarray(field0[0])[..., band]) ** 2, axis=1 + np.abs(np.asarray(field0)[..., band]) ** 2, axis=1 ) total = float(np.sum(power)) if not total > 0.0: @@ -291,8 +291,7 @@ def resolve_transverse( # noqa: PLR0912, PLR0915 - one resolution, in order grid = dispersion.grid k, omega = band_wavenumbers(space, dispersion, z_max) mask = grid.inband_mask & grid.active_mask - # The column reductions read a leading axis of one, a view. - launch = np.asarray(field0)[np.newaxis] + launch = np.asarray(field0) power = _column_power(space, launch, mask) attenuation = specification.attenuation axes = _axes(space) @@ -470,12 +469,12 @@ def _trial_grid( def _relaunched( space: RadialGrid | FreeGrid | PlanarGrid, trial: RadialGrid | FreeGrid | PlanarGrid, - field0: Complex[np.ndarray, "n_batch n_modes n_pol n_omega"], -) -> Complex[np.ndarray, "1 n_trial n_pol n_omega"]: + field0: Complex[np.ndarray, "n_modes n_pol n_omega"], +) -> Complex[np.ndarray, "n_trial n_pol n_omega"]: """Return the launch on a trial grid: its field, zero beyond the design's grid.""" - first = np.asarray(field0[:1], dtype=np.complex128) - _, n_cols, n_pol, n_omega = first.shape - flat = first[0].reshape(n_cols, n_pol * n_omega) + first = np.asarray(field0, dtype=np.complex128) + n_cols, n_pol, n_omega = first.shape + flat = first.reshape(n_cols, n_pol * n_omega) if isinstance(space, RadialGrid): assert isinstance(trial, RadialGrid) series = j0(np.outer(trial.r, space.k_r)) * ( @@ -510,7 +509,7 @@ def _relaunched( columns = np.fft.fftn(np.fft.ifftshift(bigger, axes=axes), axes=axes).reshape( -1, n_pol * n_omega ) - return columns.reshape(1, -1, n_pol, n_omega) + return columns.reshape(-1, n_pol, n_omega) def checked_design( @@ -553,8 +552,7 @@ def checked_design( if not design.has_collar: return design assert design.collar_rate is not None - # The check and the trial launches read a leading axis of one, a view. - held = np.asarray(field0)[np.newaxis] + held = np.asarray(field0) result = design_check( space, design.collar_rate, held, dispersion, z_max, min(design.width) ) @@ -569,7 +567,7 @@ def checked_design( launch = _relaunched(space, trial, held) try: candidate = resolve_transverse( - design.specification, trial, dispersion, launch[0], z_max + design.specification, trial, dispersion, launch, z_max ) except TranslationError: continue diff --git a/src/solprop/translate/walls.py b/src/solprop/translate/walls.py index ae69ed1aa..a326879b5 100644 --- a/src/solprop/translate/walls.py +++ b/src/solprop/translate/walls.py @@ -829,7 +829,7 @@ def _check_wall_total_varying( # noqa: PLR0917 - called positionally budget = (64 * count * float(np.finfo(float).eps) + 2.0 * tail) * scale for index in range(edges.size - 1): lo, hi = float(edges[index]), float(edges[index + 1]) - carrier = np.real(delta_phi(linear, lo, hi))[0, :, 0, :] / (hi - lo) + carrier = np.real(delta_phi(linear, lo, hi))[:, 0, :] / (hi - lo) frozen = np.tensordot(means, coefficients[index], axes=1) allowance = budget + _normalisation_rate(block, grid, lo, hi) total = ( @@ -933,7 +933,7 @@ def _check_wall_total( - ``generator``: the stored internal-convention generator in 1/m. """ assert linear.phi_constant is not None - total = np.real(linear.phi_constant[0, :, 0, :]) + np.real( + total = np.real(linear.phi_constant[:, 0, :]) + np.real( generator[np.arange(count), np.arange(count)] ) scale = float(np.max(abs(generator))) + float(np.max(abs(linear.phi_constant))) diff --git a/tests/backend_corpus_cases.py b/tests/backend_corpus_cases.py new file mode 100644 index 000000000..09fe2d42c --- /dev/null +++ b/tests/backend_corpus_cases.py @@ -0,0 +1,715 @@ +# SPDX-FileCopyrightText: 2026 John C. Travers and contributors +# +# SPDX-License-Identifier: GPL-3.0-or-later + +"""The pinned backend comparison corpus: its cases and how one is recorded. + +Before the reference is restructured and native's seam rewritten (SEP 52 phases 3 and +4), every march and every route family is run on short fixed schedules, on both +backends, and what each run leaves is recorded (``tests/data/backend_corpus.npz``, +written by ``tests/data/generate_backend_corpus.py``): + +- the last saved field, the planes' positions and their statistics, and the step + history; +- the boundary monitor's record, the split collar's removal and occupancies and the + transverse edge's record among them; +- a resume: the run stopped at its middle plane and continued from its file, with the + continuation's last field and monitor record; +- the timing probe's trials and their read-backs, or the low-storage march's refusal of + a read-back. + +Each backend is compared with its own record +(``tests/backends/test_backend_corpus.py``). Everything here runs through the public +paths -- :meth:`Simulation.run`, a file resumed by :meth:`Simulation.run`, and +:meth:`Backend.timing_probe` on a backend selected by the registry -- so that a +restructure behind them leaves the cases valid. +""" + +from __future__ import annotations + +import dataclasses +import hashlib +import json +import os +import platform +import warnings +from collections.abc import Callable, Mapping +from dataclasses import dataclass +from pathlib import Path +from typing import Any + +import numpy as np + +from capability_routes import general_constant +from solprop.backends.registry import select +from solprop.backends.session import Start +from solprop.boundaries import SpatialAbsorber +from solprop.constants import C +from solprop.driver import Simulation, SimulationSettings +from solprop.exceptions import BackendError, SolpropWarning +from solprop.grid import EnvelopeGrid, FieldGrid, FreeGrid, LineGrid, RadialGrid +from solprop.log import NullReporter +from solprop.modes import PlanarGrid, SlabMode +from solprop.monitor import BoundaryCheck +from solprop.nonlinear.responses import KerrResponse +from solprop.output.hdf5 import open_file, resume +from solprop.output.records import HISTORY_FIELDS +from solprop.output.store import PlaneStore +from solprop.physdata import gamma3 +from solprop.pulses import GaussianPulse, PlanarPulse, SpatialPulse +from solprop.spatial import SpatialMedium +from solprop.stepper import FixedSchedule, LowStorageRungeKutta, RungeKutta +from solprop.stepper.multistep import Multistep + +__all__ = [ + "CASES", + "THREAD_VARIABLES", + "Case", + "Recorded", + "identity", + "record", +] + +STEPS = 10 +"""Fixed steps of every scheduled case (``AGENTS.md``: about ten steps).""" + +RELAXED = BoundaryCheck(budget=1.0, stop_factor=1.0, transverse_budget=1.0) +"""A check that records everything and neither warns nor stops: the record is the +point, and the cases put light in the collar on purpose.""" + +HISTORY = tuple(name for name in HISTORY_FIELDS if name != "wall_time") +"""The step history's columns a run is compared on; its wall time is not a result.""" + + +def _schedule(length: float, planes: tuple[float, ...]) -> FixedSchedule: + """Return :data:`STEPS` equal steps over `length` that land on `planes`.""" + return FixedSchedule.uniform(0.0, length, length / STEPS, save_planes=planes) + + +# ----------------------------------------------------------------------- the routes + +FIBRE = 0.02 +"""The fibre's length, m: ten steps of 2 mm, a fiftieth of a metre's run.""" + +FIBRE_BETA2 = 2e-25 +"""s^2/m: the walk comes from a carrier offset, and a 0.07 m dispersion length for the +200 fs pulse keeps it compact over the fibre.""" + +SPARSE = 100.0 +"""How much longer the fibre's grid is declared than the fibre, so that the boundary +monitor checks a quiet window every ``SPARSE FIBRE / 256`` (``monitor.CHECKS``): about +every fourth step, and the saved planes' states regardless. The split collar's removal +then adds over several accepted steps between two readings. The declared length also +sets the collar's default reference length, which weakens its rate.""" + + +def walking_fibre( + backend: str, + integrator: RungeKutta | Multistep | LowStorageRungeKutta, + *, + planes: tuple[float, ...] = (FIBRE / 2, FIBRE), + scheduled: bool = True, + interpolated: bool = False, + rtol: float = 1e-7, +) -> Simulation: + """Return a fibre whose pulse walks into the split collar, checked sparsely. + + The GNLSE route on an envelope grid. Launched 0.1 ps late at a carrier offset under + :data:`FIBRE_BETA2`, the pulse's group delay walks a further 0.3 ps of its 1 ps + window over the fibre, so the ``"split_step"`` collar takes some of it at every + step from the first and the high guard fills: the split collar's removal and + occupancies are not empty. Its + nonlinear phase is about one radian. The schedule is :data:`STEPS` equal steps + landing on `planes`, or with `interpolated` on its own edges only, the planes then + observed by dense output; unscheduled, the march is adaptive at `rtol`. + """ + grid = EnvelopeGrid.from_limits(SPARSE * FIBRE, 1e-6, (0.8e-6, 1.25e-6), 1e-12) + omega0 = 2 * np.pi * C / 1e-6 + offset = 0.3e-12 / (FIBRE_BETA2 * FIBRE) + extra: dict[str, Any] = {"interpolate_planes": interpolated} + if not scheduled: + extra |= {"rtol": rtol, "atol": 0.0, "dz_init": FIBRE / 4} + elif interpolated: + extra["schedule"] = FixedSchedule(np.full(STEPS, FIBRE / STEPS)) + else: + extra["schedule"] = _schedule(FIBRE, planes[:-1]) + return Simulation.from_gnlse( + 50.0, FIBRE, [0.0, 0.0, FIBRE_BETA2], wavelength=1e-6, grid=grid, + pulses=[GaussianPulse( + wavelength=2 * np.pi * C / (omega0 + offset), fwhm=200e-15, + peak_power=1.0, phase=[0.0, 0.1e-12], + )], + noise=None, raman=False, shock=False, save_planes=list(planes), + backend=backend, integrator=integrator, absorber_scheme="split_step", + boundary_check=RELAXED, **extra, + ) # fmt: skip + + +CAPILLARY = 2e-3 +"""The capillary's length, m.""" + + +def capillary( + backend: str, + integrator: Multistep | RungeKutta, + *, + modes: int = 1, + envelope: bool = False, + plasma: bool = False, + interpolated: bool = False, + rtol: float = 1e-7, +) -> Simulation: + """Return a 2 mm argon capillary whose short pulse reaches the split collar. + + A 10 uJ, 15 fs pulse in a 50 um capillary of 2 bar of argon, on an 80 fs window: + the pulse's tails and, with several modes, their group delays put light in the + ``"split_step"`` collar within the ten steps. With `interpolated` the march is + adaptive at `rtol` and saves at planes inside its steps, observed by dense output, + which a fixed schedule does not offer. + """ + limits = (300e-9, 2e-6) if not envelope else (500e-9, 1.3e-6) + window = 80e-15 + grid = ( + EnvelopeGrid.from_limits(CAPILLARY, 800e-9, limits, window) + if envelope + else FieldGrid.from_limits(CAPILLARY, 800e-9, limits, window) + ) + extra: dict[str, Any] + if interpolated: + # Dense observation is refused beside the split collar, whose factor is + # applied once per accepted step: the in-stage collar, the default, instead. + planes = (0.35 * CAPILLARY, 0.75 * CAPILLARY, CAPILLARY) + extra = {"rtol": rtol, "atol": 1e-12, "interpolate_planes": True} + else: + planes = (CAPILLARY / 2, CAPILLARY) + extra = { + "schedule": _schedule(CAPILLARY, planes[:1]), + "absorber_scheme": "split_step", + } + with warnings.catch_warnings(): + warnings.simplefilter("ignore", SolpropWarning) + return Simulation.from_capillary( + 50e-6, CAPILLARY, "Ar", 2.0, grid=grid, wavelength=800e-9, fwhm=15e-15, + energy=10e-6, modes=modes, envelope=envelope, raman=False, plasma=plasma, + noise=None, save_planes=list(planes), backend=backend, + integrator=integrator, boundary_check=RELAXED, **extra, + ) # fmt: skip + + +SPATIAL = 2e-3 +"""The free-space and planar cases' length, m.""" + + +def _spatial_settings( + backend: str, + integrator: Multistep | RungeKutta | LowStorageRungeKutta, + **overrides: Any, +) -> SimulationSettings: + """Return :data:`STEPS` fixed steps over :data:`SPATIAL`, its middle plane saved.""" + planes = (SPATIAL / 2, SPATIAL) + schedule = _schedule(SPATIAL, planes[:1]) + keywords: dict[str, Any] = { + "save_planes": list(planes), + "schedule": schedule, + "dz_max": schedule.max_step, + "dz_init": schedule.max_step, + "backend": backend, + "integrator": integrator, + "absorber_scheme": "split_step", + "boundary_check": RELAXED, + } + return SimulationSettings(**(keywords | overrides)) + + +def _beam(energy: float, waist: float, propz: float = 0.0) -> SpatialPulse: + """Return a 40 fs Gaussian beam at 800 nm, `propz` from its waist (m).""" + return SpatialPulse( + GaussianPulse(wavelength=800e-9, fwhm=40e-15, energy=energy), + waist, + propz=propz, + ) + + +def radial( + backend: str, integrator: Multistep | RungeKutta | LowStorageRungeKutta +) -> Simulation: + """Return a 20 uJ beam of 80 um waist in 5 bar of argon on 16 Hankel nodes. + + Kerr self-focusing over 2 mm; the pulse's spreading in time reaches the split + collar of its 120 fs field window. + """ + grid = FieldGrid.from_limits(SPATIAL, 800e-9, (600e-9, 1.2e-6), 120e-15) + return Simulation.from_free( + grid, + RadialGrid(4e-4, 16), + SpatialMedium("Ar", 5.0), + [KerrResponse(gamma3("Ar", wavelength=800e-9), thg=False)], + _beam(20e-6, 80e-6), + settings=_spatial_settings(backend, integrator, spatial_absorber=False), + noise=None, + ) + + +def cartesian( + backend: str, integrator: Multistep | RungeKutta | LowStorageRungeKutta +) -> Simulation: + """Return a 2 uJ Kerr beam of 60 um waist in 5 bar of argon on an 8 by 8 plane. + + An envelope grid of a narrow band keeps the state small; no transverse collar is + asked for (the coarse plane launches into the default safety ramp's band, as the + catalogue's Cartesian route does). + """ + grid = EnvelopeGrid.from_limits(SPATIAL, 800e-9, (700e-9, 900e-9), 100e-15) + return Simulation.from_free( + grid, + FreeGrid(3e-4, 8, 3e-4, 8, collar_fraction=0), + SpatialMedium("Ar", 5.0), + [KerrResponse(gamma3("Ar", wavelength=800e-9), thg=False)], + _beam(2e-6, 60e-6), + settings=_spatial_settings(backend, integrator, spatial_absorber=False), + noise=None, + ) + + +def line_edge( + backend: str, integrator: Multistep | RungeKutta | LowStorageRungeKutta +) -> Simulation: + """Return a converging beam that reaches a real-space collar on the line route. + + SEP 40's requested collar outside a 300 um aperture of 40 points + (``backends/edge_collar_cases.py``, whose Cartesian plane would be too large to + record): the beam converges from 24 mm before its 60 um waist, so its wings reach + the collar and the transverse edge's record holds an inward current above + rounding. Linear, as the edge cases are. + """ + grid = EnvelopeGrid.from_limits(SPATIAL, 800e-9, (700e-9, 900e-9), 100e-15) + return Simulation.from_free( + grid, + LineGrid(300e-6, 40, collar_fraction=1.0), + SpatialMedium(1.0), + [], + _beam(1e-9, 60e-6, propz=-2.4e-2), + settings=_spatial_settings( + backend, integrator, spatial_absorber=SpatialAbsorber(kappa=8e4) + ), + noise=None, + ) + + +def planar( + backend: str, integrator: Multistep | RungeKutta | LowStorageRungeKutta +) -> Simulation: + """Return the hybrid planar route: two slab modes of an argon gap on 8 y points. + + ``capability_routes.planar`` on a 300 fs window and over 2 mm. + """ + grid = EnvelopeGrid.from_limits(SPATIAL, 800e-9, (650e-9, 1000e-9), 300e-15) + half_gap = 63.5e-6 + modes = [SlabMode(half_gap, "Ar", 1.0, order=p) for p in (1, 3)] + pulse = PlanarPulse( + GaussianPulse(wavelength=800e-9, fwhm=60e-15, energy=1e-3), + 400e-6, + waist_x=0.735 * half_gap, + ) + return Simulation.from_planar( + grid, + PlanarGrid(modes, 1e-3, 8), + [KerrResponse(gamma3("Ar", wavelength=800e-9), thg=False)], + pulse, + settings=_spatial_settings(backend, integrator, spatial_absorber=False), + ) + + +def general_modal( + backend: str, integrator: Multistep | RungeKutta | LowStorageRungeKutta +) -> Simulation: + """Return the general modal route: ``capability_routes.general_constant`` scheduled. + + One HE11 channel of a 1.5 um strand with a Kerr core, over the catalogue's 0.2 mm. + """ + simulation = general_constant(None) + length = float(simulation.settings.save_planes[-1]) + planes = (length / 2, length) + schedule = _schedule(length, planes[:1]) + simulation.settings = dataclasses.replace( + simulation.settings, + backend=backend, + integrator=integrator, + schedule=schedule, + save_planes=np.asarray(planes), + dz_max=schedule.max_step, + dz_init=schedule.max_step, + absorber_scheme="split_step", + boundary_check=RELAXED, + ) + return simulation + + +FIXED_ADAMS = Multistep(variable_order=False) +"""The exponential Adams method at its top order: no order choice to fork.""" + +CARRIER = Multistep(carrier_frame="scalar") +"""The exponential Adams method under the carrier frame (SEP 35), adaptive.""" + + +@dataclass(frozen=True) +class Case: + """One corpus case: a route on a march, and how its run is recorded. + + Attributes + ---------- + name : str + Its key in the corpus. + route : str + The route family it covers. + march : str + The march it covers. + build : Callable + The simulation, given the backend's registry name. + adaptive : bool + Whether the march chooses its own steps; then a tighter run is recorded beside + it, the accuracy envelope's reference. + tighter : Callable or None + The tighter run of an adaptive case, given the backend. + resume : bool or str + How its file is resumed (:meth:`Simulation.run`'s `resume`): + ``"interpolated"`` where the middle plane lies inside a step. + """ + + name: str + route: str + march: str + build: Callable[[str], Simulation] + adaptive: bool = False + tighter: Callable[[str], Simulation] | None = None + resume: bool | str = True + + +CASES: tuple[Case, ...] = ( + Case( + "fibre_rk_replay", + "gnlse", + "replayed Runge-Kutta", + lambda backend: walking_fibre(backend, RungeKutta()), + ), + Case( + "fibre_rk_adaptive", + "gnlse", + "adaptive Runge-Kutta", + lambda backend: walking_fibre(backend, RungeKutta(), scheduled=False), + adaptive=True, + tighter=lambda backend: walking_fibre( + backend, RungeKutta(), scheduled=False, rtol=1e-9 + ), + ), + Case( + "fibre_low_storage", + "gnlse", + "low-storage", + lambda backend: walking_fibre(backend, LowStorageRungeKutta()), + ), + Case( + "capillary_adams_interaction", + "capillary (three modes)", + "fused Adams, interaction formulation (retained-history transport)", + lambda backend: capillary( + backend, + Multistep(variable_order=False, formulation="interaction"), + modes=3, + ), + ), + Case( + "capillary_adams_exponential", + "capillary", + "exponential Adams (untransported history)", + lambda backend: capillary(backend, FIXED_ADAMS), + ), + Case( + "capillary_adams_carrier", + "capillary", + "exponential Adams, carrier frame, dense output at interpolated planes", + lambda backend: capillary(backend, CARRIER, interpolated=True), + adaptive=True, + tighter=lambda backend: capillary( + backend, CARRIER, interpolated=True, rtol=1e-9 + ), + resume="interpolated", + ), + Case( + "capillary_envelope_plasma", + "capillary (envelope plasma)", + "exponential Adams", + lambda backend: capillary(backend, FIXED_ADAMS, envelope=True, plasma=True), + ), + Case( + "general_modal_adams", + "general modal", + "exponential Adams", + lambda backend: general_modal(backend, FIXED_ADAMS), + ), + Case( + "radial_rk", + "free space, radial", + "replayed Runge-Kutta", + lambda backend: radial(backend, RungeKutta()), + ), + Case( + "cartesian_adams", + "free space, Cartesian", + "exponential Adams", + lambda backend: cartesian(backend, FIXED_ADAMS), + ), + Case( + "line_edge_adams", + "free space, line, real-space edge", + "exponential Adams", + lambda backend: line_edge(backend, FIXED_ADAMS), + ), + Case( + "line_edge_low_storage", + "free space, line, real-space edge", + "low-storage", + lambda backend: line_edge(backend, LowStorageRungeKutta()), + ), + Case( + "planar_adams", + "planar (hybrid slab)", + "exponential Adams", + lambda backend: planar(backend, FIXED_ADAMS), + ), +) +"""Every case, keyed by :attr:`Case.name`.""" + + +# ------------------------------------------------------------------- the recording + +THREAD_VARIABLES = ( + "SOLPROP_THREADS", + "SOLPROP_NATIVE_THREADS", + "VECLIB_MAXIMUM_THREADS", + "OPENBLAS_NUM_THREADS", + "MKL_NUM_THREADS", + "OMP_NUM_THREADS", +) +"""The thread settings of the default lane (``pyproject.toml``'s ``test`` task), each +pinned to one by the generator.""" + + +def identity(backend: str) -> str: + """Return a digest of what a backend's bits depend on beyond solprop's own code. + + The platform, NumPy's and SciPy's versions, the thread settings and, for native, + its build: its source, compiler, libraries and thread count. Two runs with the same + digest are expected to agree bit for bit where the order of operations is + unchanged (SEP 25 allows no more as a diagnostic). Hashed so that no host path is + recorded. + """ + import scipy # noqa: PLC0415 + + found = { + "system": platform.system(), + "machine": platform.machine(), + "python": platform.python_version(), + "numpy": np.__version__, + "scipy": scipy.__version__, + } | {name: os.environ.get(name, "") for name in THREAD_VARIABLES} + if backend == "native": + from solprop_native import ( # noqa: PLC0415 # pyright: ignore[reportMissingImports] + build_info, + ) + + info = build_info() + found |= { + "source": info.source_digest, + "compiler": info.compiler, + "flags": info.compiler_flags, + "fft": info.fft_library, + "blas": info.blas_library, + "vector_math": info.vector_math, + "threads": str(info.runtime_threads), + } + text = json.dumps(found, sort_keys=True).encode("utf-8") + return hashlib.sha256(text).hexdigest() + + +class _StopError(RuntimeError): + """Raised by :class:`_StopAtPlane` to interrupt a run, leaving its file behind.""" + + +class _StopAtPlane(NullReporter): + """Interrupt a run at its `n`-th saved plane after the launch.""" + + def __init__(self, n: int) -> None: + self.n, self.seen = n, 0 + + def plane(self, z: float, summary: object) -> None: + del z, summary + self.seen += 1 + if self.seen == self.n: + raise _StopError + + +@dataclass +class Recorded: + """What one case's run on one backend left. + + Attributes + ---------- + arrays : dict + Named arrays: ``field`` (the last saved plane, optics convention, as a stack + of one plane), ``z``, + ``stats/``, ``step_history/``, ``resumed/field`` and + ``resumed/z``, ``probe/fields`` (the read-back of each trial, stacked, in + internal units and the FFT convention), and for an adaptive case + ``tighter/field``. + records : dict + The monitor's records (``monitor``, ``resumed/monitor``), how many executions + the resumed file holds, the work counters, the probe's trials and refusal. + """ + + arrays: dict[str, np.ndarray] + records: dict[str, Any] + + +def _monitor(execution: Mapping[str, object] | None) -> dict[str, Any]: + """Return an execution record's boundary-monitor record, which every case keeps.""" + assert execution is not None + found = execution["boundary_monitor"] + assert isinstance(found, dict) + return dict(found) + + +def _run(simulation: Simulation, arrays: dict[str, np.ndarray]) -> dict[str, Any]: + """Run a case into memory; add its arrays and return its records.""" + with warnings.catch_warnings(): + warnings.simplefilter("ignore", SolpropWarning) + output = simulation.run(progress="none") + # The last plane, kept as a stack of one so that it is compared as one state. + arrays["field"] = np.asarray(output["field"][-1:]) + arrays["z"] = np.asarray(output["z"]) + for name in output.stat_names: + arrays[f"stats/{name}"] = np.asarray(output[f"stats/{name}"]) + for name in (*HISTORY, "counts"): + arrays[f"step_history/{name}"] = np.asarray(output[f"step_history/{name}"]) + assert isinstance(output, PlaneStore) + outcome = output.outcome + assert outcome is not None + execution = outcome.execution + assert execution is not None + return { + "monitor": _monitor(execution), + "source_evaluations": execution.get("source_evaluations"), + } + + +def _resumed( + case: Case, simulation: Simulation, path: Path, arrays: dict[str, np.ndarray] +) -> dict[str, Any]: + """Stop the case at its middle plane, resume its file to the end; return records. + + The stop leaves the file with the middle plane and its checkpoint (the march's + history among it), which the second run continues from. + """ + with warnings.catch_warnings(): + warnings.simplefilter("ignore", SolpropWarning) + try: + simulation.run(path, progress=_StopAtPlane(1)) + except _StopError: + pass + else: # pragma: no cover - a case with no middle plane is a corpus defect + raise AssertionError(f"{case.name} was not interrupted") + simulation.run(path, resume=case.resume, progress="none") # pyright: ignore[reportArgumentType] + with open_file(path) as handle: + arrays["resumed/field"] = np.asarray(handle["field"][-1:]) + arrays["resumed/z"] = np.asarray(handle["z"]) + outcome = resume(path).outcome + assert outcome is not None + return { + "monitor": _monitor(outcome.execution), + "executions": len(outcome.previous_executions) + 1, + } + + +def _probed(simulation: Simulation, arrays: dict[str, np.ndarray]) -> dict[str, Any]: + """Take the timing probe's trials from the launch; return what was asked. + + Two trials, at the default length and at half of it, each read back; the + low-storage march's probe refuses the read-back, and that refusal is recorded. + """ + problem = simulation.translate() + run_request = simulation._run_request() + backend = select( + simulation.settings.backend, + problem, + run_request, + grid=simulation.grid, + modes=simulation.modes, + profiles=simulation.profiles, + ) + z = float(problem.initial.z) + dz = float(run_request.stepper.dz_init) + trials = [(z, dz), (z, 0.5 * dz)] + fields = [] + refusal = None + try: + probe = backend.timing_probe(Start.entrance(problem)) + for start, length in trials: + probe.trial(start, length) + try: + fields.append(np.asarray(probe.field())) + except BackendError as error: + refusal = str(error) + break + finally: + backend.close() + if fields: + arrays["probe/fields"] = np.stack(fields) + return {"trials": [list(one) for one in trials], "refusal": refusal} + + +def record( + case: Case, + backend: str, + directory: Path, + *, + simulation: Simulation | None = None, + tighter: bool = True, +) -> Recorded: + """Run one case on one backend and return what it left. + + Parameters + ---------- + case : Case + The case. + backend : str + The backend's registry name. + directory : pathlib.Path + Where the resumed run's file is written. + simulation : solprop.driver.Simulation or None, optional + The case's simulation on `backend`, built once and run for each record, its + translation reused; built here by default. + tighter : bool, optional + Whether an adaptive case's tighter run is made too; a comparison reads the + recorded one. + + Returns + ------- + Recorded + Its arrays and records. + """ + if simulation is None: + simulation = case.build(backend) + arrays: dict[str, np.ndarray] = {} + records: dict[str, Any] = {"run": _run(simulation, arrays)} + path = directory / f"{case.name}-{backend}.h5" + records["resumed"] = _resumed(case, simulation, path, arrays) + records["probe"] = _probed(simulation, arrays) + if tighter and case.tighter is not None: + reference: dict[str, np.ndarray] = {} + _run(case.tighter(backend), reference) + arrays["tighter/field"] = reference["field"] + for name in ("field", "resumed/field", "probe/fields", "tighter/field"): + # A statistic may be undefined (NaN) by design; a field may not. + if name in arrays: + assert np.all(np.isfinite(arrays[name])), f"{case.name} {backend}: {name}" + return Recorded(arrays, records) diff --git a/tests/backends/adams_traces.py b/tests/backends/adams_traces.py index d5b428002..e39296425 100644 --- a/tests/backends/adams_traces.py +++ b/tests/backends/adams_traces.py @@ -149,7 +149,7 @@ def __init__(self, harness: _Scripted, method: Multistep) -> None: reason="scripted", # A constant operator whose reference element turns by `rate` per # metre: the chord's phase over a step is `rate` times its length. - chord=CarrierChord(constant=np.array([1j * rate])), + chord=CarrierChord(constant=np.array(1j * rate)), ) ) @@ -351,7 +351,7 @@ def stats(self, spec: Any) -> Any: # The driver surface the march uses. def _adams_state(self) -> np.ndarray: - return np.zeros((1, 1, 1, 1), dtype=np.complex128) + return np.zeros((1, 1, 1), dtype=np.complex128) def _reserve_source_calls(self, count: int) -> None: pass diff --git a/tests/backends/conftest.py b/tests/backends/conftest.py index 1073167d6..19b51df1c 100644 --- a/tests/backends/conftest.py +++ b/tests/backends/conftest.py @@ -361,7 +361,7 @@ def build_operator( build(weights_of, z0, z1, tol=tol, mandatory_breakpoints=kinks) ) stacked = np.stack( - [np.asarray(one, dtype=complex).reshape(1, 1, 1, -1) for one in terms] + [np.asarray(one, dtype=complex).reshape(1, 1, -1) for one in terms] ) expansion = ParameterExpansion( parameters=("synthetic",), diff --git a/tests/backends/geometry_rows.py b/tests/backends/geometry_rows.py index 2a174340e..f12bdd029 100644 --- a/tests/backends/geometry_rows.py +++ b/tests/backends/geometry_rows.py @@ -613,7 +613,7 @@ def the_leaf_is_pathological(built: Built) -> bool: assert leaf is not None leaves.assert_the_leaf_is_pathological(leaf) if leaf.kappa_sq is not None and built.run_request.stepper.integrator is None: - operator = leaves.separable_values(leaf)[0][0] + operator = leaves.separable_values(leaf)[0] assert not np.allclose(operator[0], operator[1]) return True diff --git a/tests/backends/low_storage_cases.py b/tests/backends/low_storage_cases.py index 3d3404ed9..d095c1bde 100644 --- a/tests/backends/low_storage_cases.py +++ b/tests/backends/low_storage_cases.py @@ -161,10 +161,7 @@ def parity_bound(problem, run_request, *, growth=2.0, transverse=1, depth=0): for a, b in itertools.pairwise(edges) ) final = np.abs(delta_phi(problem.linear, float(edges[0]), float(edges[-1]))) - # The exponent keeps the operator tables' leading axis of one. - argument = ( - EPS * ARGUMENT_FACTOR * np.broadcast_to(swept + final, (1, *problem.shape))[0] - ) + argument = EPS * ARGUMENT_FACTOR * np.broadcast_to(swept + final, problem.shape) bins = populated([problem.initial.field0], 1e-3) separable = problem.linear.separable roots = ( diff --git a/tests/backends/native_mixture_launch_probe.py b/tests/backends/native_mixture_launch_probe.py index c136ffb29..95e213b4a 100644 --- a/tests/backends/native_mixture_launch_probe.py +++ b/tests/backends/native_mixture_launch_probe.py @@ -73,7 +73,7 @@ def launch_case(kerr, n_pol): reference.set_initial(problem.initial.field0, None) reference._hoist(0.0) state = reference._require_state() - peak = max(float(np.max(abs(one))) for one in reference._plasma_nodes(state[0])) + peak = max(float(np.max(abs(one))) for one in reference._plasma_nodes(state)) finally: reference.close() assert peak > 0.0 diff --git a/tests/backends/native_trace_marches.py b/tests/backends/native_trace_marches.py index e690023d4..63b1a69ff 100644 --- a/tests/backends/native_trace_marches.py +++ b/tests/backends/native_trace_marches.py @@ -165,7 +165,7 @@ def __init__(self, backend: Any, method: Multistep, *, runner: Any) -> None: self._runner = runner self._ahead: tuple[float, object] | None = None - def _configure(self) -> None: + def _configure(self, *_: object) -> None: """Configure nothing: the scripted runner takes the policy itself.""" def _observe(self, position: float) -> object: @@ -193,7 +193,7 @@ def _budget_error(self, cost: int) -> ResponseBudgetError: self.ledger.total, backend._z, cost, - backend._adams_state()[0].copy(), + backend._adams_state().copy(), ) def march_to(self, target: float) -> None: @@ -460,7 +460,7 @@ def _policy(self, interval: float | None = None) -> dict[str, object]: """Return the record with the harness's quiet interval.""" return super()._policy(self._interval() if interval is None else interval) - def _configure(self) -> None: + def _configure(self, *_: object) -> None: """Configure nothing: the scripted runner takes the policy itself.""" MAX_ATTEMPTS: int = 256 diff --git a/tests/backends/rk_traces.py b/tests/backends/rk_traces.py index a8d5b41fc..7cdd88efa 100644 --- a/tests/backends/rk_traces.py +++ b/tests/backends/rk_traces.py @@ -34,7 +34,6 @@ import numpy as np from solprop.backends.host.driver import HostDriver -from solprop.backends.rk_core import RungeKuttaPolicy from solprop.exceptions import BackendError, ConvergenceError from solprop.stepper.controller import PIController from solprop.stepper.schedule import FixedSchedule, reached @@ -266,15 +265,6 @@ def _trial_step(self, z: float, dz: float) -> tuple[float, object]: return noise, "advanced" return noise * (dz / h0) ** self.order, "advanced" - def _controller_policy(self) -> RungeKuttaPolicy: - stepper = self.scenario - return RungeKuttaPolicy( - controller=self.controller, - dz_min=stepper.dz_min, - dz_max=stepper.dz_max, - auto_init=stepper.auto_init, - ) - def _finish(self, state: Any, dz: float) -> Any: return state diff --git a/tests/backends/session_rows.py b/tests/backends/session_rows.py index 9d2bd76cd..d59d9b5b3 100644 --- a/tests/backends/session_rows.py +++ b/tests/backends/session_rows.py @@ -670,7 +670,7 @@ def launch_phase(problem: PropagationProblem, run_request: Any, grid: Any) -> fl """ backend = NumpyRefBackend(problem, run_request, grid=grid) try: - state = problem.initial.field0[None] + state = problem.initial.field0 backend._hoist(0.0) source = backend._nonlinear( state, backend._stage_scalars(0.0, 0.0), np.ones_like(state) diff --git a/tests/backends/synthetic_leaf_cases.py b/tests/backends/synthetic_leaf_cases.py index f13db4229..b853cb96c 100644 --- a/tests/backends/synthetic_leaf_cases.py +++ b/tests/backends/synthetic_leaf_cases.py @@ -197,8 +197,8 @@ def nonlinear_rate(problem, run_request, grid): stage = reference._nonlinear( state, reference._stage_scalars(0.0, 0.0), - np.ones((1, *problem.shape), complex), - )[0] + np.ones(problem.shape, complex), + ) return 3.0 * float(np.max(np.abs(stage)) / np.max(np.abs(state))) finally: reference.close() diff --git a/tests/backends/test_absorber_schemes.py b/tests/backends/test_absorber_schemes.py index 2863a9f16..dbc4f679b 100644 --- a/tests/backends/test_absorber_schemes.py +++ b/tests/backends/test_absorber_schemes.py @@ -386,9 +386,7 @@ def test_the_split_step_absorber_leaves_nothing_outside_the_active_band( # above can see). rate = problem.responses.temporal_absorber assert rate is not None - absorbed = apply_temporal_rate( - field[None], rate, length / BAND_STEPS, kind=kind - ) + absorbed = apply_temporal_rate(field, rate, length / BAND_STEPS, kind=kind) assert outside[0] assert np.abs(absorbed[..., 0]).max() > 1e-3 * np.abs(absorbed).max() diff --git a/tests/backends/test_adams_natural_kink.py b/tests/backends/test_adams_natural_kink.py index 538a46ef6..4d8d67ac7 100644 --- a/tests/backends/test_adams_natural_kink.py +++ b/tests/backends/test_adams_natural_kink.py @@ -86,7 +86,7 @@ def observed(session, batch): } assert orders[steps_before_restart] == 1 start = problem.initial.field0[0, 0] * problem.grids.active_mask - exact = np.exp(delta_phi(problem.linear, 0.0, end))[0, 0, 0] * start + exact = np.exp(delta_phi(problem.linear, 0.0, end))[0, 0] * start # Four exact linear steps differ only by propagator multiplication roundoff. assert_allclose( result.fields[-1][0, 0], diff --git a/tests/backends/test_backend_corpus.py b/tests/backends/test_backend_corpus.py new file mode 100644 index 000000000..4b2a99cfb --- /dev/null +++ b/tests/backends/test_backend_corpus.py @@ -0,0 +1,367 @@ +# SPDX-FileCopyrightText: 2026 John C. Travers and contributors +# +# SPDX-License-Identifier: GPL-3.0-or-later + +"""Both backends against the pinned corpus (SEP 52 section 7, "A pinned corpus first"). + +``tests/data/backend_corpus.npz`` holds what every case of +``tests/backend_corpus_cases.py`` left on ``numpy_ref`` and on the native CPU build, +recorded before SEP 52 restructured the reference and native's seam +(``tests/data/generate_backend_corpus.py``): the last saved field, the planes' +statistics and step history, the boundary monitor's and transverse edge's records, a +resume from the middle plane, and the timing probe's read-backs or refusal. Each +backend's rerun is held to its own record. + +**Bitwise is a diagnostic.** On the recording build and thread setting (the same +:func:`backend_corpus_cases.identity`) the names whose bits moved are reported as the +test's ``bitwise`` property, never failed on (SEP 25: no comparison needs bitwise +agreement across builds, thread counts or changes of operation order). + +**What fails** is a departure beyond the computation's SEP 25 budget, fixed before the +comparison from the problem's operation counts: + +- a fixed schedule is a matched discrete march (class R), held to + :func:`march_bounds.march_bound` at the launch's nonlinear phase and the source's + conditioning, both read by the reference when the corpus was written: the fields at + that budget, the statistics at :data:`march_bounds.STATISTIC_FACTOR` times it (a rate + statistic at :data:`RATE_FACTOR` times), the prescribed positions and steps, flags, + counts and texts exactly, and every monitor reading within + :func:`march_bounds.reading_tolerance`; +- the probe's read-backs are one trial each, held to the same budget for one step; +- an adaptive march (class T) meets the accuracy envelope of + :func:`march_bounds.assert_in_envelope`: four times the recorded run's departure from + the recorded run at a hundredth of its tolerance, and its accepted steps a work band + (:func:`bounds.assert_work_within`); its decisions are compared bit for bit only in + the diagnostic. +""" + +from __future__ import annotations + +import functools +import json +from pathlib import Path +from typing import Any + +import numpy as np +import pytest + +from backend_corpus_cases import CASES, Case, Recorded, identity, record +from bounds import AccuracyEnvelope, assert_within, assert_work_within, departure +from march_bounds import ( + ENVELOPE_FACTOR, + assert_matched, + assert_record_within, + assert_stored_matched, + evaluation_rounding, + march_bound, +) +from solprop.backends.registry import available +from solprop.driver import Simulation + +CORPUS = Path(__file__).parents[1] / "data" / "backend_corpus.npz" + +BACKENDS = [ + "numpy_ref", + pytest.param( + "native", + marks=pytest.mark.skipif( + "native" not in available(), reason="native is not installed" + ), + ), +] + +RATE_FACTOR = 2 * 2 * 11 +"""How many times the field's relative departure an ionisation statistic may move: its +rate's ``d ln W / d ln |E|`` is at most ``2 K`` in the multiphoton limit, with +``K = 11`` photons of 800 nm to ionise argon (15.76 eV), and a factor of two of margin; +the tunnelling regime is less sensitive (``bounds.rate_log_sensitivity``).""" + +QUADRATURE = "stats/quadrature_error" +"""The transverse quadrature's relative error on the polarisation: a rounding residue +where the quadrature is exact (about 1e-15 on these cases), held to an absolute bound +(:func:`_assert_quadrature`).""" + +EXACT = ("z", "resumed/z", "step_history/z", "step_history/h") +"""What a prescribed schedule fixes whatever the arithmetic (class P).""" + +WORK_FACTOR = 1.5 +"""The work band of an adaptive rerun's accepted steps: a rounding difference may flip +one decision, which on these runs of a dozen or a few hundred steps moves the count by +far less than half (SEP 25 section 4.4).""" + + +@functools.cache +def _corpus() -> tuple[dict[str, np.ndarray], dict[str, Any]]: + """Return the corpus's arrays by name and its records.""" + with np.load(CORPUS, allow_pickle=False) as data: + arrays = {name: data[name] for name in data.files} + records = json.loads(arrays.pop("records").tobytes().decode("utf-8")) + return arrays, records + + +def _stored(case: Case, backend: str) -> Recorded: + """Return what the corpus recorded of a case on a backend.""" + arrays, records = _corpus() + prefix = f"{case.name}/{backend}/" + own = { + name.removeprefix(prefix): value + for name, value in arrays.items() + if name.startswith(prefix) + } + return Recorded(own, records["cases"][case.name]["backends"][backend]) + + +CROSS_BUILD = 2.0 +"""The factor on a matched march's budget for a record made by another build. + +The corpus is recorded once, on one machine, and compared on every platform CI runs. +Two builds evaluate the same operations with different vector-maths libraries and +contraction, which :func:`march_bounds.march_bound` (two runs of one build) does not +count: the general modal case was measured on x86-64 Linux at 1.01 of the single-build +budget against a record made on arm64 macOS. +""" + + +def _budget(case: Case, simulation: Simulation, steps: int | None = None) -> float: + """Return a case's rounding budget for two matched runs (class R). + + :func:`march_bounds.march_bound` at the recorded launch phase (no smaller than a + tenth, as :func:`march_bounds.simulation_bound` takes it) and conditioning, over + the schedule's steps or `steps`, times :data:`CROSS_BUILD`. + """ + entry = _corpus()[1]["cases"][case.name] + return CROSS_BUILD * march_bound( + simulation.translate(), + simulation._run_request(), + steps, + phi=max(float(entry["phi"]), 0.1), + kappa=float(entry["kappa"]), + backend=simulation.settings.backend, + ) + + +def _assert_quadrature( + case: Case, simulation: Simulation, got: Recorded, expected: Recorded +): + """Hold the quadrature's error to the rounding of the evaluations it compares. + + It is the relative difference of two evaluations of the polarisation, each within + :func:`march_bounds.evaluation_rounding` of its value relative to it (both runs' + roundings counted there); so a rerun's may move by the four evaluations' sum, an + absolute bound on a dimensionless number that is itself rounding on an exact rule. + Where it is undefined (the launch) it stays undefined. + """ + if QUADRATURE not in expected.arrays: + return + one, other = got.arrays[QUADRATURE], expected.arrays[QUADRATURE] + assert np.array_equal(np.isnan(one), np.isnan(other)), case.name + problem = simulation.translate() + limit = 4.0 * evaluation_rounding( + int(problem.grids.n_to), + int(problem.transform.n_nodes), + int(problem.shape[0]), + plasma=bool(problem.flags.has_plasma), + ) + found = np.nan_to_num(np.abs(one - other)) + assert np.all(found <= limit), (case.name, float(np.max(found)), limit) + + +def _bitwise(got: Recorded, expected: Recorded) -> list[str]: + """Return the names whose bits differ between a rerun and its record.""" + differ = [] + # An adaptive case's tighter run is read from the record, not rerun. + for name in sorted((set(got.arrays) | set(expected.arrays)) - {"tighter/field"}): + one, other = got.arrays.get(name), expected.arrays.get(name) + if one is None or other is None or one.shape != other.shape: + differ.append(name) + continue + nan = one.dtype.kind in "fc" + if not np.array_equal(one, other, equal_nan=nan): + differ.append(name) + for name in sorted(expected.records): + mine = json.dumps(got.records.get(name), sort_keys=True) + if mine != json.dumps(expected.records[name], sort_keys=True): + differ.append(f"records/{name}") + return differ + + +def _compare_probe( + case: Case, simulation: Simulation, got: Recorded, expected: Recorded +): + """Hold the probe's read-backs to one step's budget, its refusal exactly.""" + assert got.records["probe"]["trials"] == expected.records["probe"]["trials"] + refusal = expected.records["probe"]["refusal"] + if refusal is not None: + # The low-storage march has no trial field (SEP 52 section 5.5). + assert "no trial field" in refusal + assert got.records["probe"]["refusal"] is not None + assert "probe/fields" not in got.arrays + return + assert got.records["probe"]["refusal"] is None + assert_matched( + got.arrays["probe/fields"], + expected.arrays["probe/fields"], + _budget(case, simulation, steps=1), + what=f"{case.name} probe", + ) + + +def _compare_matched( + case: Case, simulation: Simulation, got: Recorded, expected: Recorded +): + """Hold a fixed-schedule case to its matched march's budget (class R).""" + bound = _budget(case, simulation) + # The error norms of a replay are a difference of nearly equal stage sums: their + # relative rounding is the field's over the estimate's own size, which no budget + # of the field bounds. They decide nothing on a schedule; the diagnostic reads + # their bits. + left_out = ("probe/fields", "step_history/norm", QUADRATURE) + names = set(expected.arrays) - set(left_out) + assert names == set(got.arrays) - set(left_out), case.name + factors = { + name: RATE_FACTOR for name in names if name.startswith("stats/species_") + } | { + name: RATE_FACTOR + for name in ("stats/electron_density", "stats/peak_ionisation_rate") + if name in names + } + assert_stored_matched( + {name: expected.arrays[name] for name in names}, + {name: got.arrays[name] for name in names}, + bound, + exact=EXACT, + factors=factors, + what=case.name, + ) + steps = int(np.sum(expected.arrays["step_history/accepted"])) + for part in ("run", "resumed"): + assert_record_within( + got.records[part]["monitor"], + expected.records[part]["monitor"], + bound, + steps, + what=f"{case.name} {part}", + work=False, + ) + # Accounting: the source evaluations a run made, where a backend counts them. + assert ( + got.records["run"]["source_evaluations"] + == expected.records["run"]["source_evaluations"] + ) + + +def _compare_adaptive(case: Case, got: Recorded, expected: Recorded): + """Hold an adaptive case to its accuracy envelope (class T).""" + reference = expected.arrays["tighter/field"] + error = max( + departure(expected.arrays["field"], reference), + departure(expected.arrays["resumed/field"], reference), + ) + assert error > 0.0, "the tighter run is the run itself" + envelope = AccuracyEnvelope( + reference=f"{case.name} at a hundredth of its tolerance", + error=error, + factor=ENVELOPE_FACTOR, + ) + for name in ("field", "resumed/field"): + assert_within( + got.arrays[name], reference, envelope.bound, what=f"{case.name} {name}" + ) + for name in ("z", "resumed/z"): + # The planes are prescribed even where the steps are not. + assert np.array_equal(got.arrays[name], expected.arrays[name]), name + assert_work_within( + int(np.sum(got.arrays["step_history/accepted"])), + int(np.sum(expected.arrays["step_history/accepted"])), + factor=WORK_FACTOR, + what=f"{case.name} accepted steps", + ) + # Two runs each inside the envelope differ by at most twice it. + bound = 2.0 * envelope.bound + steps = int(np.sum(expected.arrays["step_history/accepted"])) + stats = [ + name + for name in expected.arrays + if name.startswith("stats/") and name != QUADRATURE + ] + assert_stored_matched( + {name: expected.arrays[name] for name in stats}, + {name: got.arrays[name] for name in stats}, + bound, + factors={name: RATE_FACTOR for name in stats if "ionisation" in name}, + what=case.name, + ) + for part in ("run", "resumed"): + assert_record_within( + got.records[part]["monitor"], + expected.records[part]["monitor"], + bound, + steps, + what=f"{case.name} {part}", + work=True, + ) + + +# Medium: each case runs three short propagations and a probe, under a second on +# native and up to a few seconds on the reference. +@pytest.mark.medium +@pytest.mark.parametrize("backend", BACKENDS) +@pytest.mark.parametrize("case", CASES, ids=[case.name for case in CASES]) +def test_a_rerun_reproduces_its_recorded_corpus( + case: Case, backend: str, tmp_path: Path, record_property +) -> None: + expected = _stored(case, backend) + simulation = case.build(backend) + got = record(case, backend, tmp_path, simulation=simulation, tighter=False) + if identity(backend) == _corpus()[1]["identity"][backend]: + differ = _bitwise(got, expected) + report = "identical" if not differ else "differs: " + ", ".join(differ) + else: + report = "not compared: another build or thread setting" + record_property("bitwise", report) + print(f"{case.name} on {backend}, bitwise: {report}") # noqa: T201 - the diagnostic + assert got.records["resumed"]["executions"] == 2 + _compare_probe(case, simulation, got, expected) + _assert_quadrature(case, simulation, got, expected) + if case.adaptive: + _compare_adaptive(case, got, expected) + else: + _compare_matched(case, simulation, got, expected) + + +def test_the_corpus_holds_what_it_was_recorded_for() -> None: + # Every case on both backends, and the features the restructure must keep: the + # split collar's removal added over several steps between readings, a filled guard, + # a transverse edge read at every step, a resume, the probe's read-backs and the + # low-storage refusal, and the adaptive marches' rejections. + _, records = _corpus() + assert set(records["cases"]) == {case.name for case in CASES} + marches = {case.march for case in CASES} + assert any("adaptive Runge-Kutta" in one for one in marches) + assert any("replayed Runge-Kutta" in one for one in marches) + assert any("interaction formulation" in one for one in marches) + assert any("carrier frame" in one for one in marches) + assert any("low-storage" in one for one in marches) + for case in CASES: + for backend in ("numpy_ref", "native"): + stored = _stored(case, backend) + monitor = stored.records["run"]["monitor"] + assert stored.records["resumed"]["executions"] == 2 + assert stored.records["resumed"]["monitor"]["resumed_at"] + assert monitor["launch_energy"] > 0.0 + assert ("tighter/field" in stored.arrays) == case.adaptive + low_storage = case.march == "low-storage" + assert (stored.records["probe"]["refusal"] is not None) == low_storage + assert ("probe/fields" in stored.arrays) != low_storage + fibre = _stored(CASES[0], "numpy_ref").records["run"]["monitor"] + assert fibre["skipped"] > fibre["steps"] > 0 + assert fibre["raw_removed_fraction"] > 1e-5 + assert fibre["guard_high_max"] > 1e-6 + edge = _stored( + next(case for case in CASES if case.name == "line_edge_adams"), "native" + ).records["run"]["monitor"]["transverse"] + assert edge["reads"] > 0 and edge["ramp_removed"] > 0.0 + adaptive = _stored( + next(case for case in CASES if case.name == "fibre_rk_adaptive"), "numpy_ref" + ) + assert not np.all(adaptive.arrays["step_history/accepted"]) diff --git a/tests/backends/test_backend_ladder.py b/tests/backends/test_backend_ladder.py index af4bdceac..8b7856ac4 100644 --- a/tests/backends/test_backend_ladder.py +++ b/tests/backends/test_backend_ladder.py @@ -377,6 +377,6 @@ def test_the_observation_hook_is_the_identity_on_the_reference( backend.set_initial(problem.initial.field0, None) state = backend._require_state() assert backend._observed_field(state) is state - assert np.array_equal(backend.observe().require_field(), state[0]) + assert np.array_equal(backend.observe().require_field(), state) finally: backend.close() diff --git a/tests/backends/test_carrier_frame.py b/tests/backends/test_carrier_frame.py index 06c16eed9..5230dd943 100644 --- a/tests/backends/test_carrier_frame.py +++ b/tests/backends/test_carrier_frame.py @@ -84,7 +84,7 @@ def reference_rate(problem) -> float: """Return ``Im D`` at the reference bin of a constant operator, 1/m (``-kappa``).""" grids = problem.grids index = _nearest_bin(grids.omega, grids.omega0) - return float(problem.linear.phi_constant[0, 0, 0, index].imag) + return float(problem.linear.phi_constant[0, 0, index].imag) @dataclasses.dataclass(frozen=True) @@ -400,22 +400,21 @@ def test_the_policy_is_inert_where_no_carrier_phase_is_kept(): assert inactive is not None and not inactive.active assert f"({nearest}) is inactive" in inactive.reason with pytest.raises(BackendError, match="inert"): - inactive.shift(np.zeros((1, 1, 1, 3), dtype=complex)) - assert inactive.phase(np.ones((1, 1, 1, 3), dtype=complex)) == 0.0 + inactive.shift(np.zeros((1, 1, 3), dtype=complex)) + assert inactive.phase(np.ones((1, 1, 3), dtype=complex)) == 0.0 -def test_the_shift_is_per_member_and_the_cap_reads_the_largest(): - exponent = np.zeros((2, 3, 2, 4), dtype=complex) - exponent[0, 0, 0, 1] = -0.1 + 0.3j - exponent[1, 0, 0, 1] = -0.2 - 0.7j +def test_the_shift_and_the_cap_read_the_reference_element(): + exponent = np.zeros((3, 2, 4), dtype=complex) + exponent[0, 0, 1] = -0.2 - 0.7j # Other modes and polarisations never set the shift. - exponent[:, 1:, :, :] = 5j - exponent[:, :, 1, :] = 9j + exponent[1:, :, :] = 5j + exponent[:, 1, :] = 9j reference = CarrierReference("scalar", 1, 1.0, "") shift = reference.shift(exponent) - assert shift.shape == (2, 1, 1, 1) + assert shift.shape == (1, 1, 1) # Imaginary only: the loss is never shifted (section 10). - assert shift.ravel().tolist() == [0.3j, -0.7j] + assert shift.ravel().tolist() == [-0.7j] assert reference.phase(exponent) == pytest.approx(0.7) @@ -478,10 +477,10 @@ def test_the_dense_formula_is_the_integral_it_stands_for(): # nodes, a lossy rotating exponent and an imaginary shift, at interior fractions # and the endpoint: the production moments and passes. nodes = np.array([1.0, 0.0, -0.7, -1.9]) - x = np.array([-0.4 + 7.0j, -1.5 + 0.3j]).reshape((1, 1, 1, 2)) + x = np.array([-0.4 + 7.0j, -1.5 + 0.3j]).reshape((1, 1, 2)) sigma = 2.3j rng = np.random.default_rng(4) - samples = rng.normal(size=(4, 1, 1, 1, 2)) + 1j * rng.normal(size=(4, 1, 1, 1, 2)) + samples = rng.normal(size=(4, 1, 1, 2)) + 1j * rng.normal(size=(4, 1, 1, 2)) rotated = [ np.exp(sigma * (1 - node)) * one for node, one in zip(nodes, samples, strict=True) @@ -493,18 +492,18 @@ def test_the_dense_formula_is_the_integral_it_stands_for(): def polynomial(theta, element): """The rotated samples' interpolant at ``theta``.""" basis = np.array([theta**k for k in range(count)]) - values = np.array([one[0, 0, 0, element] for one in rotated]) + values = np.array([one[0, 0, element] for one in rotated]) return (coefficients @ basis) @ values for xi in (0.2, 0.63, 1.0): moments = kernels.moments(xi * (x - sigma), count) powers = np.array([xi ** (k + 1) for k in range(count)]) - scaled = moments * powers.reshape((count, 1, 1, 1, 1)) + scaled = moments * powers.reshape((count, 1, 1, 1)) found = np.exp(-sigma * (1 - xi)) * kernels.apply( scaled, coefficients, rotated, 1.0 ) for element in range(2): - rate = complex(x[0, 0, 0, element]) + rate = complex(x[0, 0, element]) def integrand(s, part, rate=rate, element=element, xi=xi): value = ( @@ -518,7 +517,7 @@ def integrand(s, part, rate=rate, element=element, xi=xi): quad(integrand, 0.0, xi, args=(0,), epsabs=1e-15, epsrel=1e-13)[0], quad(integrand, 0.0, xi, args=(1,), epsabs=1e-15, epsrel=1e-13)[0], ) - assert abs(found[0, 0, 0, element] - exact) <= 1e-13 * max(1.0, abs(exact)) + assert abs(found[0, 0, element] - exact) <= 1e-13 * max(1.0, abs(exact)) # --- aliasing and resolution (section 4.3) -------------------------------------------- @@ -900,8 +899,8 @@ def test_the_shifted_block_keeps_the_true_flow(): generator[np.tril_indices(3, -1)] *= 0.05 vectors, triangular = schur_blocks(generator[:, :, np.newaxis]) diagonal = -0.4 + 2.5j - exponent = np.full((1, 3, 1, 1), diagonal) - sigma = np.full((1, 1, 1, 1), 2.1j) + exponent = np.full((3, 1, 1), diagonal) + sigma = np.full((1, 1, 1), 2.1j) block = CoupledBlock( active=np.array([True]), vectors=vectors, triangular=triangular, section=0 ) @@ -1035,14 +1034,12 @@ def test_the_rounding_budget_bounds_one_shifted_increment(): count = len(nodes) exponent = ( rng.normal(size=6) * 0.3 - 0.05 + 1j * rng.uniform(-40.0, 40.0, size=6) - ).reshape((1, 1, 1, 6)) + ).reshape((1, 1, 6)) sigma = 0.9j # under the cap - samples = rng.normal(size=(count, 1, 1, 1, 6)) + 1j * rng.normal( - size=(count, 1, 1, 1, 6) - ) + samples = rng.normal(size=(count, 1, 1, 6)) + 1j * rng.normal(size=(count, 1, 1, 6)) shifted = exponent - sigma moments = ExponentialKernels().moments(shifted, count) - rotated = np.exp(sigma * (1.0 - nodes)).reshape((count, 1, 1, 1, 1)) * samples + rotated = np.exp(sigma * (1.0 - nodes)).reshape((count, 1, 1, 1)) * samples found = sum( moments[k] * np.tensordot(coefficients[:, k], rotated, axes=1) for k in range(count) @@ -1051,15 +1048,15 @@ def test_the_rounding_budget_bounds_one_shifted_increment(): exact = np.array( [ exact_increment( - complex(exponent[0, 0, 0, element]) - sigma, + complex(exponent[0, 0, element]) - sigma, sigma, nodes, coefficients, - samples[:, 0, 0, 0, element], + samples[:, 0, 0, element], ) for element in range(6) ] - ).reshape((1, 1, 1, 6)) + ).reshape((1, 1, 6)) budget = carrier_increment_budget( coefficients, moments, samples, nodes, sigma, shifted ) @@ -1176,7 +1173,7 @@ def test_the_chord_is_the_engine_s_exponent_at_the_reference(backend_name, opera # backend's own exponent of them. bundle = scalars(problem, z1, z2, helpers=backend.helpers) full = backend._delta_phi(bundle) - expected = np.asarray(full)[:, 0, 0, reference.bin] + expected = np.asarray(full)[0, 0, reference.bin] found = reference.chord.exponent(z1, z2) # The same representation evaluated at one element: a few roundings of the # exponent (an expansion's combination, a constant's product). diff --git a/tests/backends/test_driver.py b/tests/backends/test_driver.py index 4aa3d721b..6897491ca 100644 --- a/tests/backends/test_driver.py +++ b/tests/backends/test_driver.py @@ -99,7 +99,7 @@ def test_a_rejected_step_is_recorded_and_not_kept(driver): # 5.0 is far above the controller's threshold of one, so the first step is refused # and the second, from the same position, is taken. made = driver(norms=[5.0, 0.1], factor=2.0) - before = present(made._state)[0].copy() + before = present(made._state).copy() made.propagate_to(LENGTH) observed = made.observe() assert not bool(observed.step_history.accepted[0]) @@ -290,8 +290,8 @@ def test_the_input_noise_model_adds_its_realisation_once( resumed = scripted(problem, run_request, grid=narrow_grid) resumed.set_initial(problem.initial.field0, None, z=LENGTH / 2) masked = problem.initial.field0 * problem.grids.active_mask - assert not np.array_equal(present(entrance._state)[0], masked) - assert np.array_equal(present(resumed._state)[0], masked) + assert not np.array_equal(present(entrance._state), masked) + assert np.array_equal(present(resumed._state), masked) assert entrance._noise is None @@ -308,7 +308,7 @@ def test_the_modified_noise_model_keeps_its_realisation_apart( made = scripted(problem, run_request, grid=narrow_grid) made.set_initial(problem.initial.field0, None) assert np.array_equal( - present(made._state)[0], problem.initial.field0 * problem.grids.active_mask + present(made._state), problem.initial.field0 * problem.grids.active_mask ) drawn = present(made._noise) advanced = present(made._noise_at(made._stage_scalars(0.0, LENGTH / 2))) @@ -444,7 +444,7 @@ def test_started_is_announced_once_per_load_and_a_resume_announces_it_again(driv assert reporter.order.count("started") == 1 assert reporter.extent == (0.0, LENGTH, 2) - made.set_initial(present(made._state)[0], None, z=LENGTH / 2) + made.set_initial(present(made._state), None, z=LENGTH / 2) made.propagate_to(LENGTH) assert reporter.order.count("started") == 2 assert reporter.extent == (LENGTH / 2, LENGTH, 2) @@ -946,26 +946,25 @@ def test_the_bundles_a_compiled_backend_packs_are_frozen(): "_adams_block": "(self, z: float, dz: float) -> object", "_adams_lab_source": ( "(self, lab: jaxtyping.Complex[ndarray, " - "'n_batch n_modes n_pol n_omega'], position: float, *, " + "'n_modes n_pol n_omega'], position: float, *, " "purpose: Literal['startup', 'predictor', 'corrected', " "'diagnostic'] = 'predictor') -> jaxtyping.Complex[ndarray, " - "'n_batch n_modes n_pol n_omega']" + "'n_modes n_pol n_omega']" ), "_adams_propagator": ( "(self, anchor: float, position: float) -> jaxtyping.Complex[ndarray, " - "'n_batch n_modes n_pol n_omega']" + "'n_modes n_pol n_omega']" ), "_adams_source": ( "(self, lab: jaxtyping.Complex[ndarray, " - "'n_batch n_modes n_pol n_omega'], anchor: float, position: float, *, " + "'n_modes n_pol n_omega'], anchor: float, position: float, *, " "purpose: Literal['startup', 'predictor', 'corrected', " "'diagnostic'] = 'predictor') -> tuple[jaxtyping.Complex[ndarray, " - "'n_batch n_modes n_pol n_omega'], jaxtyping.Complex[ndarray, " - "'n_batch n_modes n_pol n_omega']]" + "'n_modes n_pol n_omega'], jaxtyping.Complex[ndarray, " + "'n_modes n_pol n_omega']]" ), "_apply_mask": ( - "(self, state: jaxtyping.Complex[ndarray, " - "'n_batch n_modes n_pol n_omega']) -> _StateT" + "(self, state: jaxtyping.Complex[ndarray, 'n_modes n_pol n_omega']) -> _StateT" ), "_begin_plane": "(self) -> None", "_dense_stages": "(self) -> list[numpy.ndarray]", diff --git a/tests/backends/test_engines.py b/tests/backends/test_engines.py new file mode 100644 index 000000000..04d8df3c0 --- /dev/null +++ b/tests/backends/test_engines.py @@ -0,0 +1,118 @@ +# SPDX-FileCopyrightText: 2026 John C. Travers and contributors +# +# SPDX-License-Identifier: GPL-3.0-or-later + +"""The engine protocols, frozen (SEP 52 section 5.2). + +``solprop.backends.engines`` mirrors the native marches' engines one-to-one: +``RungeKuttaEngine`` is ``rk::Engine`` and ``AdamsEngine`` is ``march::Engine``. A +change to a member or a signature here changes what every engine implements, and is made +with the C++ interface in the same commit. +""" + +import dataclasses +import inspect +import math + +import pytest + +from solprop.backends import engines +from solprop.backends.engines import ( + AcceptRecord, + AdamsEngine, + AttemptOutcome, + BoundaryReading, + HistoryEntry, + RungeKuttaEngine, +) + +RUNGE_KUTTA_ENGINE = { + "trial": "(self, z: float, dz: float) -> float", + "accept": "(self, dz: float) -> solprop.backends.engines.AcceptRecord", + "extend": "(self, z: float, dz: float) -> None", + "reading": "(self) -> solprop.backends.engines.BoundaryReading", + "in_flight": "(self) -> bool", +} +"""``rk::Engine`` (``native/include/solprop_native/rk_march.hpp``), member by member.""" + +ADAMS_ENGINE = { + "start": "(self, z: float) -> None", + "trial": ( + "(self, z: float, dz: float, stencil: solprop.backends.adams_core.Stencil)" + " -> jaxtyping.Float[ndarray, 'n_orders']" + ), + "accept": ( + "(self, dz: float, free: int, stencil: solprop.backends.adams_core.Stencil," + " kept: collections.abc.Sequence[solprop.backends.engines.HistoryEntry])" + " -> solprop.backends.engines.AcceptRecord" + ), + "chord_phase": "(self, z1: float, z2: float) -> float", + "step_carrier": "(self, order: int) -> jaxtyping.Float[ndarray, 'n_rotations']", + "begin_attempt": "(self, cost: int) -> None", + "end_attempt": "(self, outcome: solprop.backends.engines.AttemptOutcome) -> None", + "reading": "(self) -> solprop.backends.engines.BoundaryReading", +} +"""``march::Engine`` (``native/include/solprop_native/adams_march.hpp``).""" + +RECORDS = { + AcceptRecord: ("stiffness", "removed", "found"), + BoundaryReading: ("total", "regions", "occupancy"), + HistoryEntry: ("z", "identity", "row"), +} +"""The records the protocols pass, field by field.""" + +OUTCOMES = {"REJECTED": 0, "ACCEPTED": 1, "ABANDONED": 2} +"""``HostOutcome`` (``native/include/solprop_native/host_source.hpp``), by value.""" + + +@pytest.mark.parametrize( + ("protocol", "surface"), + [(RungeKuttaEngine, RUNGE_KUTTA_ENGINE), (AdamsEngine, ADAMS_ENGINE)], + ids=["runge_kutta", "adams"], +) +def test_the_engine_protocols_are_frozen(protocol, surface): + members = {name for name in vars(protocol) if not name.startswith("_")} + assert members == set(surface) + for name, signature in surface.items(): + member = vars(protocol)[name] + if isinstance(member, property): + assert member.fget is not None + member = member.fget + assert str(inspect.signature(member)) == signature, name + + +def test_only_in_flight_is_a_property(): + # The low-storage step's interval (SEP 52 section 5.2) is read, never called. + assert isinstance(vars(RungeKuttaEngine)["in_flight"], property) + assert not any( + isinstance(member, property) for member in vars(AdamsEngine).values() + ) + + +@pytest.mark.parametrize(("record", "fields"), RECORDS.items()) +def test_the_records_the_engines_pass_are_frozen(record, fields): + assert tuple(field.name for field in dataclasses.fields(record)) == fields + assert record.__dataclass_params__.frozen # pyright: ignore[reportAttributeAccessIssue] + + +def test_the_attempt_outcomes_are_the_native_values(): + assert {outcome.name: outcome.value for outcome in AttemptOutcome} == OUTCOMES + + +def test_an_accept_record_knows_nothing_by_default(): + # The stiffness diagnostic is NaN where the engine does not form it, and a step + # with no split collar reports no removal and no found occupancies. + record = AcceptRecord() + assert math.isnan(record.stiffness) + assert (record.removed, record.found) == (0.0, None) + + +def test_the_module_declares_the_protocols_and_their_records(): + assert set(engines.__all__) == { + "AcceptRecord", + "AdamsEngine", + "AttemptOutcome", + "BoundaryReading", + "HistoryEntry", + "RungeKuttaEngine", + } diff --git a/tests/backends/test_envelope_plasma_host.py b/tests/backends/test_envelope_plasma_host.py index 3d02c246e..9fd007d34 100644 --- a/tests/backends/test_envelope_plasma_host.py +++ b/tests/backends/test_envelope_plasma_host.py @@ -107,10 +107,10 @@ def test_envelope_population_values_do_not_depend_on_retaining_node_scans(host_c backend.set_initial(problem.initial.field0, None) field = problem.initial.field0 plain = backend._species_statistics(field) - scans = {} - retained = backend._species_statistics(field, scans, member=3) - assert set(scans) == {3} - fractions, peaks = scans[3] + scans = [] + retained = backend._species_statistics(field, scans) + assert len(scans) == 1 + fractions, peaks = scans[0] assert ( fractions.shape == peaks.shape @@ -165,7 +165,7 @@ def test_prescribed_density_off_carrier_drude_assembly(host_case): backend = NumpyRefBackend(problem, run_request, grid=grid) try: bundle = scalars(problem, 0.0, 0.0) - lab = problem.initial.field0[None] + lab = problem.initial.field0 actual = backend._nonlinear(lab, bundle, np.ones_like(lab)) assembly = problem.responses.envelope_plasma assert assembly is not None diff --git a/tests/backends/test_exponential_coupled.py b/tests/backends/test_exponential_coupled.py index 4b15f16dc..a1e913891 100644 --- a/tests/backends/test_exponential_coupled.py +++ b/tests/backends/test_exponential_coupled.py @@ -52,7 +52,7 @@ def test_the_matrix_moments_are_the_integrals_they_stand_for(): generator[np.tril_indices(3, -1)] *= 0.05 vectors, triangular = schur_blocks(generator[:, :, np.newaxis]) shift = -0.4 + 2.5j - exponent = np.full((1, 3, 1, 1), shift) + exponent = np.full((3, 1, 1), shift) block = CoupledBlock( active=np.array([True]), vectors=vectors, triangular=triangular, section=0 ) @@ -81,7 +81,7 @@ def test_the_eigenbasis_route_matches_the_augmented_exponential(): vectors, triangular = schur_blocks(np.moveaxis(generator, 0, -1)) basis = schur_eigenbasis(vectors, triangular) assert basis[3].tolist() == [True, False] - exponent = np.full((1, 3, 1, 2), -0.3 + 4.0j) + exponent = np.full((3, 1, 2), -0.3 + 4.0j) reference = CoupledBlock( active=np.array([True, True]), vectors=vectors, @@ -128,7 +128,7 @@ def test_the_reference_carry_keeps_a_span_zero_sample_on_and_off_the_route(): ), "no route": plain, } - exponent = np.full((1, 3, 1, 2), -0.3 + 4.0j) + exponent = np.full((3, 1, 2), -0.3 + 4.0j) shape = (3, *exponent.shape) rates = rng.normal(size=shape) + 1j * rng.normal(size=shape) states = rng.normal(size=shape) + 1j * rng.normal(size=shape) @@ -145,12 +145,10 @@ def test_the_reference_carry_keeps_a_span_zero_sample_on_and_off_the_route(): for row, span in enumerate(spans): for index in range(generator.shape[0]): action = expm( - span * (exponent[0, 0, 0, 0] * np.eye(3) + dz * generator[index]) + span * (exponent[0, 0, 0] * np.eye(3) + dz * generator[index]) ) - term = (rates[row] - exponent / dz)[0, :, 0, index] * states[row][ - 0, :, 0, index - ] - unrouted[row, 0, :, 0, index] = action @ term + term = (rates[row] - exponent / dz)[:, 0, index] * states[row][:, 0, index] + unrouted[row, :, 0, index] = action @ term for name, block in blocks.items(): expected = reference.carry_coupled(exponent, dz, rows, slots, spans, block) # The route forms W f(x + h lambda) Y where its absence forms the augmented diff --git a/tests/backends/test_exponential_march.py b/tests/backends/test_exponential_march.py index c21dbe800..a1dd17e25 100644 --- a/tests/backends/test_exponential_march.py +++ b/tests/backends/test_exponential_march.py @@ -140,7 +140,7 @@ def test_exponential_is_the_exact_linear_flow_without_a_source( ) field, march, _ = run(problem, run_request, narrow_grid) start = problem.initial.field0[0, 0] * problem.grids.active_mask - exact = np.exp(delta_phi(problem.linear, 0.0, length))[0, 0, 0] * start + exact = np.exp(delta_phi(problem.linear, 0.0, length))[0, 0] * start # A zero source leaves the flows' products: round-off per step. np.testing.assert_allclose(field, exact, rtol=0, atol=1e-12 * np.abs(start).max()) assert ledger_of(march).rejected_steps == 0 @@ -372,7 +372,7 @@ def test_exponential_dense_observation_is_the_linear_flow_without_a_source( observed = backend.observe().require_field()[0, 0] backend.close() start = problem.initial.field0[0, 0] * problem.grids.active_mask - exact = np.exp(delta_phi(problem.linear, 0.0, planes[0]))[0, 0, 0] * start + exact = np.exp(delta_phi(problem.linear, 0.0, planes[0]))[0, 0] * start np.testing.assert_allclose( observed, exact, rtol=0, atol=1e-12 * np.abs(start).max() ) @@ -574,7 +574,7 @@ def test_a_backend_without_the_lab_frame_source_refuses_the_formulation(driver): # Every shipped backend implements the hook; an external one that does not is # told which formulation to use instead. stub = SimpleNamespace(name="external") - lab = np.zeros((1, 1, 1, 4), dtype=complex) + lab = np.zeros((1, 1, 4), dtype=complex) with pytest.raises(BackendError, match="external does not execute"): driver._adams_lab_source(stub, lab, 0.0, purpose="startup") diff --git a/tests/backends/test_exponential_routes.py b/tests/backends/test_exponential_routes.py index edfdf7cea..61967d959 100644 --- a/tests/backends/test_exponential_routes.py +++ b/tests/backends/test_exponential_routes.py @@ -74,7 +74,7 @@ def test_the_lab_frame_sample_is_the_route_source(route): ) try: rng = np.random.default_rng(5) - field = problem.initial.field0[None] * (1.0 + 0.1 * rng.standard_normal()) + field = problem.initial.field0 * (1.0 + 0.1 * rng.standard_normal()) position = 0.3 * float(run_request.stepper.save_planes[-1]) lab = backend._adams_lab_source(field, position) transported, propagator = backend._adams_source(field, position, position) diff --git a/tests/backends/test_exponential_varying.py b/tests/backends/test_exponential_varying.py index 09b0d40bc..d2e0b2636 100644 --- a/tests/backends/test_exponential_varying.py +++ b/tests/backends/test_exponential_varying.py @@ -167,7 +167,7 @@ def test_a_structurally_absent_source_takes_the_exact_flow(): field, adams, _ = march(simulation) assert adams is not None and not adams._engine.compensated start = problem.initial.field0 * problem.grids.active_mask - exact = np.exp(delta_phi(problem.linear, 0.0, LENGTH))[0] * start + exact = np.exp(delta_phi(problem.linear, 0.0, LENGTH)) * start # Round-off of the flows' products, step by step. np.testing.assert_allclose(field, exact, rtol=0, atol=1e-12 * np.abs(exact).max()) diff --git a/tests/backends/test_exponential_weights.py b/tests/backends/test_exponential_weights.py index 1fce4c66b..124a7bc4f 100644 --- a/tests/backends/test_exponential_weights.py +++ b/tests/backends/test_exponential_weights.py @@ -151,8 +151,8 @@ def test_phi_functions_agree_with_the_augmented_exponential(x): def test_kernel_moments_agree_with_the_augmented_exponential(x): # The moments k! phi_{k+1}(x) the reference's trial takes. count = MAX_PHI_ORDER - exponent = np.full((1, 1, 1, 1), x, dtype=np.complex128) - moments = ExponentialKernels().moments(exponent, count)[:, 0, 0, 0, 0] + exponent = np.full((1, 1, 1), x, dtype=np.complex128) + moments = ExponentialKernels().moments(exponent, count)[:, 0, 0, 0] reference = augmented_phi(complex(x), count) for k in range(count): scale = math.factorial(k) diff --git a/tests/backends/test_far_field_statistics.py b/tests/backends/test_far_field_statistics.py index e516b5eab..91eb4bcc8 100644 --- a/tests/backends/test_far_field_statistics.py +++ b/tests/backends/test_far_field_statistics.py @@ -408,7 +408,7 @@ def test_each_absorbed_channel_is_its_share_of_the_linear_decay( * np.sum( geometry.spectral_weights[:, None, None] * np.real(axial_admittance(data, grid.omega, slice(None)))[:, None] - * (-2 * np.real(operator[0])) + * (-2 * np.real(operator)) * np.abs(problem.initial.field0) ** 2 * energy_weights(grid.n_omega, grid.dt, kind=grid.kind) ) diff --git a/tests/backends/test_free_absorber.py b/tests/backends/test_free_absorber.py index ed4424385..0294de244 100644 --- a/tests/backends/test_free_absorber.py +++ b/tests/backends/test_free_absorber.py @@ -243,7 +243,7 @@ def test_cartesian_split_step_temporal_absorber_reconstructs_physical_field( actual = state.field_to_host().reshape(start.shape) else: backend.set_initial(problem.initial.field0, None) - actual = backend._finish(problem.initial.field0[None].copy(), dz)[0] + actual = backend._finish(problem.initial.field0.copy(), dz) assert np.max(np.abs(actual - expected)) < bound record_metric( request, diff --git a/tests/backends/test_free_boundaries.py b/tests/backends/test_free_boundaries.py index 19efead0c..5d55be636 100644 --- a/tests/backends/test_free_boundaries.py +++ b/tests/backends/test_free_boundaries.py @@ -44,9 +44,9 @@ def test_cartesian_reference_stage_names_missing_geometry(free_backend_problem): object.__setattr__(problem, "route", route) with pytest.raises(BackendError, match=r"Cartesian nonlinear.*geometry"): backend._nonlinear( - problem.initial.field0[None], + problem.initial.field0, bundle, - np.ones((1, *problem.shape), complex), + np.ones(problem.shape, complex), ) finally: backend.close() diff --git a/tests/backends/test_free_noise.py b/tests/backends/test_free_noise.py index ec08b3461..b86d5c1bb 100644 --- a/tests/backends/test_free_noise.py +++ b/tests/backends/test_free_noise.py @@ -75,10 +75,10 @@ def test_cartesian_modified_noise_enters_the_rotated_nonlinear_field( reference.set_initial(problem.initial.field0, None) reference._hoist(position) expected = reference._nonlinear( - rotated[None], + rotated, reference._stage_scalars(position, position), - np.ones((1, *problem.shape), complex), - )[0] + np.ones(problem.shape, complex), + ) # Zero distance isolates one nonlinear evaluation while still exercising # each backend's real stage-input/noise path at nonzero absolute z. if backend.name == "native": diff --git a/tests/backends/test_general_modal.py b/tests/backends/test_general_modal.py index eb9569800..9735fc836 100644 --- a/tests/backends/test_general_modal.py +++ b/tests/backends/test_general_modal.py @@ -592,7 +592,7 @@ def test_a_millimetre_of_one_channel_accumulates_the_self_phase( ) amplitude = fourier.analytic_signal(fourier.inverse(state[0], kind="field")) power = float(np.abs(amplitude[0, 0]) ** 2) - linear = np.exp(delta_phi(problem.linear, 0.0, length))[0, 0, 0, index] + linear = np.exp(delta_phi(problem.linear, 0.0, length))[0, 0, index] turned = final / (state[0, 0, index] * linear) expected = -gamma * power * length assert abs(expected) > 1.0 diff --git a/tests/backends/test_held_plasma_routes.py b/tests/backends/test_held_plasma_routes.py index bbf1f9e88..35ba59276 100644 --- a/tests/backends/test_held_plasma_routes.py +++ b/tests/backends/test_held_plasma_routes.py @@ -54,10 +54,10 @@ def test_source_chunking_and_zero_after_a_nonzero_evaluation(name, rule): ) try: bundle = backend._stage_scalars(0.0, 0.0) - state = problem.initial.field0[None] + state = problem.initial.field0 ones = np.ones_like(state) backend._hoist(0.0) - result = backend._nonlinear(state, bundle, ones)[0] + result = backend._nonlinear(state, bundle, ones) allowance = relative * np.max(abs(reference)) assert allowance > 0 assert np.max(abs(result - reference)) <= allowance @@ -168,8 +168,8 @@ def test_mixture_production_assembles_both_current_channels(): weight = backend._prefactor(bundle) * grids.omega_window expected *= weight actual = backend._nonlinear( - problem.initial.field0[None], bundle, np.ones((1, *problem.shape), complex) - )[0] + problem.initial.field0, bundle, np.ones(problem.shape, complex) + ) # Projection chunk order, three FFTs and assembly of identical currents. allowance = gamma(64 * grids.n_to + 32 * transform.n_nodes) * np.linalg.norm( expected diff --git a/tests/backends/test_low_storage.py b/tests/backends/test_low_storage.py index b55975add..03d59e6e4 100644 --- a/tests/backends/test_low_storage.py +++ b/tests/backends/test_low_storage.py @@ -121,12 +121,11 @@ def linear_bound(problem, schedule): for a, b in itertools.pairwise(edges) ) final = np.abs(delta_phi(problem.linear, float(edges[0]), float(edges[-1]))) - # The exponent keeps the operator tables' leading axis of one. - shape = (1, *problem.shape) + shape = problem.shape return EPS * ( 4 * steps + ARGUMENT_FACTOR - * (np.broadcast_to(swept, shape)[0] + np.broadcast_to(final, shape)[0]) + * (np.broadcast_to(swept, shape) + np.broadcast_to(final, shape)) ) @@ -259,7 +258,7 @@ def test_the_cartesian_diffraction_reproduces_its_width_and_gouy_phase(fraction) def driven_phase(problem, step, bins): """Return ``max |Im D| h / 2`` over `bins`: section 4.4's asymptotic range.""" exponent = delta_phi(problem.linear, 0.0, step) - rate = np.broadcast_to(exponent / step, (1, *problem.shape))[0] + rate = np.broadcast_to(exponent / step, problem.shape) selected = np.broadcast_to(bins, problem.shape) return float(np.max(np.abs(np.imag(rate))[selected]) * step / 2) @@ -488,9 +487,9 @@ def evaluation_difference(problem, run_request, grid, gnlse): backend = NumpyRefBackend(problem, run_request, grid=grid) backend.set_initial(problem.initial.field0, None) state = backend._require_state() - march = backend._adams_lab_source(state, 0.0)[0, 0, 0] + march = backend._adams_lab_source(state, 0.0)[0, 0] backend.close() - scaffold = gnlse_source(gnlse, grid)(state[0, 0, 0]) + scaffold = gnlse_source(gnlse, grid)(state[0, 0]) assert np.linalg.norm(scaffold) > 0 return float(np.linalg.norm(march - scaffold) / np.linalg.norm(scaffold)), state @@ -543,7 +542,7 @@ def test_whole_runs_agree_with_the_scaffold_far_below_the_truncation_error( difference, start = evaluation_difference(problem, run_request, soliton_grid, gnlse) final, backend = run(problem, run_request, soliton_grid) backend.close() - start = start[0, 0, 0] + start = start[0, 0] scaffold = propagate_fixed_rk4(gnlse, soliton_grid, start, period, steps)[1][-1] gap = float(np.linalg.norm(final[0, 0] - scaffold) / np.linalg.norm(start)) exact = np.exp(1j * scales.recurrence_phase) * start @@ -1300,7 +1299,7 @@ def capture(lab, bundle, inverse): ] * np.exp(1j * phase[:, None, :]) bound = gamma(128 * (space.n_columns + grid.n_to)) bound += gamma(64) * position * float(grid.omega.max()) / C - drawn = (field_in - lab)[0] + drawn = field_in - lab scale = np.max(np.abs(expected)) assert np.max(np.abs(drawn - expected)) <= bound * scale @@ -1518,7 +1517,7 @@ def test_a_linear_step_is_rk4_s_stability_function_and_reads_its_stiffness(rate) roundings of the passes. """ rng = np.random.default_rng(5) - state = rng.standard_normal((1, 1, 1, 64)) + 1j * rng.standard_normal((1, 1, 1, 64)) + state = rng.standard_normal((1, 1, 64)) + 1j * rng.standard_normal((1, 1, 64)) start = state.copy() step = 0.05 backend = LinearSource(rate, state) diff --git a/tests/backends/test_mixture_statistics.py b/tests/backends/test_mixture_statistics.py index e437634cb..5cc9bf0ca 100644 --- a/tests/backends/test_mixture_statistics.py +++ b/tests/backends/test_mixture_statistics.py @@ -100,7 +100,7 @@ def test_numpy_species_statistics_match_the_independent_node_scans(): "peak_ionisation_rate", ) values = backend.stats(StatsSpec(names=names, periodic=names[:2])) - nodes = backend._nodes_at(backend._require_state()[0]) + nodes = backend._nodes_at(backend._require_state()) rates, fractions = passes.plasma_channel_fractions( nodes, problem.responses.plasma_channels, diff --git a/tests/backends/test_monitor_owner.py b/tests/backends/test_monitor_owner.py index 457fd5bf3..4c2514b6c 100644 --- a/tests/backends/test_monitor_owner.py +++ b/tests/backends/test_monitor_owner.py @@ -36,6 +36,7 @@ from hypothesis import given, settings from hypothesis import strategies as st +from solprop.backends import records from solprop.backends.registry import available from solprop.boundaries import SPLIT_RATE_BUCKETS, SplitCrossing from solprop.exceptions import BackendError, BoundaryError, BoundaryWarning @@ -58,6 +59,10 @@ def _modules() -> tuple[Any, Any]: ) +def _native_records() -> Any: + return importlib.import_module("solprop_native._records") + + def _grid() -> EnvelopeGrid: return EnvelopeGrid.from_limits(1.0, 1e-6, (0.8e-6, 1.25e-6), 1e-12) @@ -86,7 +91,9 @@ def __init__(self, monitor: BoundaryMonitor, *, can_rest: bool) -> None: self.python = monitor self.watch = watch self.native = core.Monitor( - watch.monitor_config(monitor, monitor.crossing, can_rest=can_rest).native() + _native_records().monitor_config( + records.monitor_config(monitor, monitor.crossing, can_rest=can_rest) + ) ) self.can_rest = can_rest self.steps = 0 @@ -160,9 +167,9 @@ def collar(self) -> None: def resume(self, record: dict | None, *, plane: bool, unwatched=()) -> None: self.python.resume(record, plane=plane, unwatched=unwatched) self.native.resume( - self.watch.resumption(record).native(), + _native_records().resumption(records.resumption(record)), plane, - self.watch.unwatched_of(unwatched), + [[float(start), float(end)] for start, end in unwatched], ) def launch(self, energies: BoundaryEnergies, z: float, floor=None) -> int: @@ -282,7 +289,7 @@ def _launch(pair: Pair, *, floor: np.ndarray | None = None) -> int: """Launch a centred pulse, its reading the host reduction's.""" grid = _grid() pulse = np.exp(-((grid.t / 20e-15) ** 2)).astype(complex) - energies = pair.python.energies(np.fft.fft(pulse)[None, None, None, :]) + energies = pair.python.energies(np.fft.fft(pulse)[None, None, :]) if pair.python.split: energies = BoundaryEnergies( energies.total, @@ -387,7 +394,7 @@ def test_a_resume_continues_the_record_as_solprop_does( pair.resume(record, plane=plane, unwatched=[(z, z + 1e-3), (z, z)]) grid = _grid() pulse = np.exp(-((grid.t / 20e-15) ** 2)).astype(complex) - energies = pair.python.energies(np.fft.fft(pulse)[None, None, None, :]) + energies = pair.python.energies(np.fft.fft(pulse)[None, None, :]) pair.launch(energies, z) _, ended = _run(pair, then, z) pair.records() @@ -480,7 +487,7 @@ def test_the_floor_s_split_removal_is_within_its_bound(steps, seed): floor = 1e-6 * ( rng.standard_normal(grid.n_omega) + 1j * rng.standard_normal(grid.n_omega) ) - if _launch(pair, floor=floor[None, None, None, :]) == STOP: + if _launch(pair, floor=floor[None, None, :]) == STOP: return _run(pair, steps) pair.records(floor_steps=pair.steps) @@ -495,7 +502,7 @@ def test_light_after_a_launch_that_held_none_fails_alike(): assert pair.launch(empty, 0.0) == NONE light = _energies(pair.python, 1.0, [0.0, 0.0, 0.0, 1e-3], 1.0) assert pair.step(light, 1e-3) == DIVIDE - notice = watch.Notice(watch.NoticeKind.DIVISION, False, 1e-3, 0.0, 0.0, 0.0) + notice = records.Notice(records.NoticeKind.DIVISION, False, 1e-3, 0.0, 0.0, 0.0) error = watch.notice_error(pair.python.check, notice) with pytest.raises(ZeroDivisionError) as raised: _ = 1.0 / 0.0 @@ -508,7 +515,6 @@ def test_light_after_a_launch_that_held_none_fails_alike(): ) def test_a_record_s_counts_are_read_alike(name, value): # A count that is not whole is refused by both resumes alike; a fraction truncates. - _, watch = _modules() earlier = Pair(_python(split=False, quiet=False), can_rest=False) _launch(earlier) record = earlier.python.summary() @@ -522,5 +528,5 @@ def test_a_record_s_counts_are_read_alike(name, value): with pytest.raises(BoundaryError) as python: _python(split=False, quiet=False).resume(record, plane=False) with pytest.raises(BoundaryError) as native: - watch.resumption(record) + records.resumption(record) assert str(native.value) == str(python.value) diff --git a/tests/backends/test_monitor_reduction.py b/tests/backends/test_monitor_reduction.py index 5a139323b..13b6c892b 100644 --- a/tests/backends/test_monitor_reduction.py +++ b/tests/backends/test_monitor_reduction.py @@ -587,7 +587,7 @@ def test_successive_occupancies_are_the_reference_s(backend: str, route: str) -> for third in (1, 2, 3): stepping(theirs).propagate_to(LENGTH * third / 3) expected = reading(theirs).occupancy - state = as_driver(theirs)._require_state()[0] + state = as_driver(theirs)._require_state() got = native_reading(ours, monitor, np.array(state)).occupancy assert expected is not None and got is not None np.testing.assert_allclose( diff --git a/tests/backends/test_monitor_watch.py b/tests/backends/test_monitor_watch.py index e9c5f8a1d..202694d0e 100644 --- a/tests/backends/test_monitor_watch.py +++ b/tests/backends/test_monitor_watch.py @@ -57,7 +57,7 @@ def _monitor(*, split: bool, quiet: bool, budget: float = 1e-3) -> BoundaryMonit buckets[0, :4] = np.arange(4) monitor.crossing = SplitCrossing(buckets, 1e-13, launch_limit=0.05) pulse = np.exp(-((grid.t / 20e-15) ** 2)).astype(complex) - spectrum = np.fft.fft(pulse)[None, None, None, :] + spectrum = np.fft.fft(pulse)[None, None, :] energies = monitor.energies(spectrum) if split: energies = BoundaryEnergies( diff --git a/tests/backends/test_multistep_march.py b/tests/backends/test_multistep_march.py index e6f1d5a71..45cdb9caf 100644 --- a/tests/backends/test_multistep_march.py +++ b/tests/backends/test_multistep_march.py @@ -96,7 +96,7 @@ def test_adams_is_the_exact_linear_flow_without_a_source(narrow_grid, linear_pro ) field, ledger = run(problem, run_request, narrow_grid) start = problem.initial.field0[0, 0] * problem.grids.active_mask - exact = np.exp(delta_phi(problem.linear, 0.0, length))[0, 0, 0] * start + exact = np.exp(delta_phi(problem.linear, 0.0, length))[0, 0] * start # A zero source leaves only the propagators' products: round-off per step. np.testing.assert_allclose(field, exact, rtol=0, atol=1e-12 * np.abs(start).max()) assert ledger is not None and ledger.rejected_steps == 0 @@ -190,7 +190,7 @@ def test_dense_observation_is_the_exact_linear_flow_without_a_source( for plane in planes: backend.propagate_to(plane) field = backend.observe().require_field()[0, 0] - exact = np.exp(delta_phi(problem.linear, 0.0, plane))[0, 0, 0] * start + exact = np.exp(delta_phi(problem.linear, 0.0, plane))[0, 0] * start # With no source the interaction variable is constant across the step, so # the interpolant reproduces it: round-off in the propagators only. np.testing.assert_allclose( @@ -283,7 +283,7 @@ def test_call_budget_stops_before_overspending(narrow_grid, soliton_problem): assert ledger.total == budget assert 0.0 < caught.value.z < SOLITON_PERIOD assert backend._state is not None - np.testing.assert_array_equal(caught.value.field, backend._state[0]) + np.testing.assert_array_equal(caught.value.field, backend._state) backend.close() diff --git a/tests/backends/test_native_error_norm.py b/tests/backends/test_native_error_norm.py index dd0737889..4043878fc 100644 --- a/tests/backends/test_native_error_norm.py +++ b/tests/backends/test_native_error_norm.py @@ -83,10 +83,10 @@ def _native_trial(problem, run_request, grid, dz): def _operands(backend, reference, problem, run_request, dz): - """Return native's stages and state, and the reference's propagator, 4-D.""" + """Return native's stages and state, and the reference's propagator.""" tableau = run_request.stepper.tableau state = backend._native_state - shape = (1, *problem.shape) + shape = problem.shape stages = [ np.asarray(_testing.stage_to_host(state, index)).reshape(shape) for index in range(len(tableau.nodes)) @@ -126,7 +126,7 @@ def test_the_rk_norm_is_the_reference_norm_of_native_s_operands( before = problem.initial.field0 backend._native_state.finish(dz) after = _field(backend, problem) - wanted = reference._error_norm(carried, before[None], after[None]) + wanted = reference._error_norm(carried, before, after) controller = run_request.stepper.controller mask = problem.grids.norm_mask scale = float( diff --git a/tests/backends/test_native_exponent_and_estimate.py b/tests/backends/test_native_exponent_and_estimate.py index d60bfb518..7242cc285 100644 --- a/tests/backends/test_native_exponent_and_estimate.py +++ b/tests/backends/test_native_exponent_and_estimate.py @@ -72,8 +72,7 @@ def test_a_separable_exponent_is_the_host_volume(): backend.close() assert [event.z for event in landed.observations] == [0.0, end] loaded, got = landed.fields[0], landed.fields[-1] - # The exponent keeps the operator tables' leading axis of one. - exponent = delta_phi(problem.linear, 0.0, end)[0] + exponent = delta_phi(problem.linear, 0.0, end) assert_within( got, np.exp(exponent) * loaded, diff --git a/tests/backends/test_native_held_projection_budget.py b/tests/backends/test_native_held_projection_budget.py index 2f98961e8..05683c155 100644 --- a/tests/backends/test_native_held_projection_budget.py +++ b/tests/backends/test_native_held_projection_budget.py @@ -124,7 +124,7 @@ def test_embedded_current_projection_accumulates_partial_chunks( ) for collar in collars ] - reference._nonlinear(state[None], bundle, np.ones_like(state[None])) + reference._nonlinear(state, bundle, np.ones_like(state)) assert reference._last_plasma is not None _, _, coarse_gate, coarse_edge = reference._last_plasma assert coarse_gate is not None and coarse_edge is not None diff --git a/tests/backends/test_native_mixture_fractions.py b/tests/backends/test_native_mixture_fractions.py index b82659c10..949c08ec2 100644 --- a/tests/backends/test_native_mixture_fractions.py +++ b/tests/backends/test_native_mixture_fractions.py @@ -55,7 +55,7 @@ def oracle(kerr, n_pol): reference._hoist(0.0) bundle = reference._stage_scalars(0.0, 0.0) fields = np.real( - np.concatenate(list(reference._plasma_nodes(reference._require_state()[0]))) + np.concatenate(list(reference._plasma_nodes(reference._require_state()))) ) finally: reference.close() diff --git a/tests/backends/test_native_planar_statistics.py b/tests/backends/test_native_planar_statistics.py index eb684c50a..8ba5d110e 100644 --- a/tests/backends/test_native_planar_statistics.py +++ b/tests/backends/test_native_planar_statistics.py @@ -168,7 +168,7 @@ def finished(problem, run_request, grid, rows, dz, options=None): try: reference.set_initial(problem.initial.field0, None) reference._hoist(0.0) - expected = np.array(reference._finish(rows[None].copy(), dz))[0] + expected = np.array(reference._finish(rows.copy(), dz)) finally: reference.close() backend = NativeBackend(problem, run_request, grid=grid, options=options) diff --git a/tests/backends/test_numpy_ref_plasma.py b/tests/backends/test_numpy_ref_plasma.py index a86a9e1a4..52830757f 100644 --- a/tests/backends/test_numpy_ref_plasma.py +++ b/tests/backends/test_numpy_ref_plasma.py @@ -319,7 +319,7 @@ def reference_nonlinear(problem, kerr, plasma, state, z): fine = np.zeros( (transform.n_modes, fourier.n_omega_for(grids.n_to, "field")), dtype=complex ) - fine[..., grids.crop_index] = state[0, :, 0, :] + fine[..., grids.crop_index] = state[:, 0, :] modal = fourier.inverse(fine, kind="field") nodes = transform.synthesis @ modal plane_plasma = dataclasses.replace( @@ -370,7 +370,7 @@ def test_one_stage_matches_the_responses_composed_by_hand( problem, run_request, kerr, plasma = ionising_capillary(preionisation) backend = NumpyRefBackend(problem, run_request) backend.set_initial(problem.initial.field0, None) - state = problem.initial.field0[None] + state = problem.initial.field0 ones = np.ones_like(state) got = backend._nonlinear(state, backend._stage_scalars(0.0, 0.05), ones) want = reference_nonlinear(problem, kerr, plasma, state, 0.05) diff --git a/tests/backends/test_numpy_ref_raman.py b/tests/backends/test_numpy_ref_raman.py index e876a5c71..eee7f7ccd 100644 --- a/tests/backends/test_numpy_ref_raman.py +++ b/tests/backends/test_numpy_ref_raman.py @@ -303,7 +303,7 @@ def reference_nonlinear(problem, grid, kerr, raman, state, z): fine = np.zeros( (transform.n_modes, fourier.n_omega_for(grids.n_to, "field")), dtype=complex ) - fine[..., grids.crop_index] = state[0, :, 0, :] + fine[..., grids.crop_index] = state[:, 0, :] modal = fourier.inverse(fine, kind="field") nodes = transform.synthesis @ modal density = float(responses.density(z)) @@ -338,7 +338,7 @@ def test_one_stage_matches_the_responses_composed_by_hand( problem, run_request, kerr, raman = raman_capillary() backend = NumpyRefBackend(problem, run_request) backend.set_initial(problem.initial.field0, None) - state = problem.initial.field0[None] + state = problem.initial.field0 got = backend._nonlinear(state, backend._stage_scalars(0.0, z), np.ones_like(state)) want = reference_nonlinear(problem, raman_field_grid, kerr, raman, state, z) # The Raman half is one convolution's rounding -- its kernel spectrum and its @@ -617,10 +617,10 @@ def stage_derivative(problem, grid, field): ) projected = passes.project(polarisation, transform.projection, 1.0, transform.n_pol) coarse = passes.to_frequency(projected, grids.crop_index, kind=grid.kind) - unit = np.ones((1, 1, 1, grid.n_omega), dtype=complex) + unit = np.ones((1, 1, grid.n_omega), dtype=complex) return passes.stage_output( coarse, prefactor_at(responses, 0.0), grids.omega_window, unit - )[0, 0, 0] + )[0, 0] @pytest.mark.parametrize(("trange", "bound"), [(3.5e-11, 2.5e-2), (7.0e-11, 1.0e-2)]) diff --git a/tests/backends/test_optmat_cases.py b/tests/backends/test_optmat_cases.py index 3082b3077..0b9fc7216 100644 --- a/tests/backends/test_optmat_cases.py +++ b/tests/backends/test_optmat_cases.py @@ -59,7 +59,7 @@ def test_the_lab_frame_sample_is_the_hoisted_derivative_on_the_masked_launch(): ) try: hoisted = np.array(launch_derivative(backend, problem)) - launch = problem.initial.field0[None] + launch = problem.initial.field0 masked = apply_hard_mask(launch, problem.grids.active_mask) assert np.any(masked != launch) backend._hoist(1e-3) diff --git a/tests/backends/test_polarised_runs.py b/tests/backends/test_polarised_runs.py index 626eca5c9..cbceee6a1 100644 --- a/tests/backends/test_polarised_runs.py +++ b/tests/backends/test_polarised_runs.py @@ -150,9 +150,9 @@ def problem_derivative(backend_class, problem, run_request, grid, modes): backend._hoist(0.0) bundle = backend._stage_scalars(0.0, 0.0) derivative = backend._nonlinear( - problem.initial.field0[None], + problem.initial.field0, bundle, - np.ones((1, *problem.shape), complex), + np.ones(problem.shape, complex), ) else: # The first stage of a trial is exactly N(initial, 0). Observation reads diff --git a/tests/backends/test_radial_absorber.py b/tests/backends/test_radial_absorber.py index d8f4d0040..40021b7f9 100644 --- a/tests/backends/test_radial_absorber.py +++ b/tests/backends/test_radial_absorber.py @@ -118,10 +118,10 @@ def test_spatial_absorber_derivative_matches_independent_bessel_operator( backend.set_initial(problem.initial.field0, None) backend._hoist(0.0) measured = backend._nonlinear( - problem.initial.field0[None], + problem.initial.field0, backend._stage_scalars(0.0, 0.0), - np.ones((1, *problem.shape), complex), - )[0] + np.ones(problem.shape, complex), + ) else: # The native state's own trial (``rk_trial`` on the uploaded stage # tables), loaded with the launch: its first stage is N(E, z=0). diff --git a/tests/backends/test_radial_nonlinear.py b/tests/backends/test_radial_nonlinear.py index d477303b5..0e9037930 100644 --- a/tests/backends/test_radial_nonlinear.py +++ b/tests/backends/test_radial_nonlinear.py @@ -114,7 +114,7 @@ def derivative(spatial_backend_class, problem, run_request, grid, options=None): backend = spatial_backend_class(problem, run_request, grid=grid, options=options) try: if isinstance(backend, NumpyRefBackend): - return launch_derivative(backend, problem)[0] + return launch_derivative(backend, problem) return _native_first_stage(backend, problem) finally: backend.close() diff --git a/tests/backends/test_radial_runs.py b/tests/backends/test_radial_runs.py index a4d36d3d7..5b23ae76b 100644 --- a/tests/backends/test_radial_runs.py +++ b/tests/backends/test_radial_runs.py @@ -97,7 +97,7 @@ def test_radial_operator_and_prefactor_match_vacuum_dispersion(radial_case): assert not problem.flags.mode_averaged assert problem.flags.constant_linop assert_allclose( - delta_phi(problem.linear, 0.0, grid.length)[0, :, 0], + delta_phi(problem.linear, 0.0, grid.length)[:, 0], expected * grid.length, rtol=0, atol=bound, diff --git a/tests/backends/test_records.py b/tests/backends/test_records.py new file mode 100644 index 000000000..0039ba3bf --- /dev/null +++ b/tests/backends/test_records.py @@ -0,0 +1,608 @@ +# SPDX-FileCopyrightText: 2026 John C. Travers and contributors +# +# SPDX-License-Identifier: GPL-3.0-or-later + +"""The run records, formed once per run from the problem, the request and the start. + +SEP 52 section 5.1: both backends read the same records, built by shared code +(``solprop.backends.records``) and importable without the native package. These tests +hold each record to the facts it is built from, for each march -- Adams, the adaptive +and the replayed Runge-Kutta march, the low-storage replay -- for a resumed start and a +monitored and an edged run; hold the reference's own derivations to them; and, where +the native backend is installed, hold its conversion to what its extension reads. +""" + +import dataclasses +import math +import subprocess +import sys +import textwrap +from typing import Any + +import numpy as np +import pytest + +from backends.test_transverse_backends import radial_collar +from solprop.backends.memory import low_storage_flows +from solprop.backends.records import ( + ATTEMPT_CAPACITY, + CALL_ATTEMPTS, + CALL_SECONDS, + PLANE_COLUMNS, + ROW_CAPACITY, + EdgeResumption, + LowStorageSettings, + PlanePlan, + Resumption, + RungeKuttaPolicy, + RunRecords, + adams_policy, + collar_rests, + declared, + edge_resumption, + monitor_continuation, + resumption, + run_records, + runge_kutta_policy, +) +from solprop.backends.registry import available, select +from solprop.backends.session import Start +from solprop.constants import C +from solprop.driver import Simulation +from solprop.exceptions import BoundaryError +from solprop.grid import EnvelopeGrid +from solprop.monitor import BoundaryCheck +from solprop.nonlinear.gnlse import problem_from +from solprop.pulses import GaussianPulse, SechPulse +from solprop.stats import StatsSpec +from solprop.stepper import FixedSchedule, Multistep, PIController, RungeKutta +from solprop.stepper.controller import START_GROWTH +from solprop.stepper.low_storage import LowStorageRungeKutta +from solprop.stepper.multistep import MarchPolicy, march_policy +from solprop.stepper.schedule import LANDING_ULPS +from solprop.translate.assembly import assemble +from solprop.translate.problem import Translation + +LENGTH = 0.02 +"""The fibre's length, m: a few steps.""" + +PLANES = (0.01, 0.02) +"""Its save planes, m.""" + +native_available = "native" in available() +needs_native = pytest.mark.skipif( + not native_available, reason="the native backend is not installed" +) + + +def _gnlse(**options: Any) -> Translation: + """Return a short GNLSE soliton's problem and request, with `options` assembled.""" + grid = EnvelopeGrid.from_limits(LENGTH, 835e-9, (500e-9, 2e-6), 1e-12) + gnlse = problem_from( + [0.0, 0.0, -1.183e-26], omega_ref=grid.omega0, gamma=0.11, shock=False + ) + launch = SechPulse(wavelength=835e-9, width=100e-15, peak_power=9.8) + settings: dict[str, Any] = { + "controller": PIController(rtol=1e-8, atol=1e-16), + "save_planes": list(PLANES), + "dz_max": LENGTH / 4, + "dz_init": 1e-4, + "density": 1.0, + "shock": False, + "prefactor_index": 1.0, + } + settings.update(options) + return assemble(grid, gnlse.mode, [gnlse.kerr], launch.build(grid), **settings) + + +def _uniform(step: float = LENGTH / 8) -> FixedSchedule: + """Return a uniform schedule landing on the planes.""" + return FixedSchedule.uniform(0.0, LENGTH, step, save_planes=list(PLANES)) + + +def _monitored(**options: Any) -> Simulation: + """Return a linear pulse walking towards its window's edge, monitored.""" + grid = EnvelopeGrid.from_limits(1.0, 1e-6, (0.8e-6, 1.25e-6), 1e-12) + omega0 = 2 * np.pi * C / 1e-6 + simulation = Simulation.from_gnlse( + 1e-30, 1.0, [0.0, 0.0, -2e-26], wavelength=1e-6, grid=grid, + pulses=[GaussianPulse( + wavelength=2 * np.pi * C / (omega0 + 1e12), fwhm=200e-15, peak_power=1.0, + )], + noise=None, raman=False, shock=False, backend="numpy_ref", n_save=3, + **options, + ) # fmt: skip + simulation.settings = dataclasses.replace( + simulation.settings, boundary_check=BoundaryCheck() + ) + return simulation + + +def _entrance(problem: Any) -> Start: + return Start.entrance(problem) + + +def test_the_records_import_and_build_without_the_native_package(): + # The reference reads the records with no native package installed (SEP 52 D4): + # importing them, and forming a run's, loads nothing of it. + code = textwrap.dedent( + """ + import sys + + class Refused: + def find_spec(self, name, path=None, target=None): + if name.partition(".")[0] == "solprop_native": + raise ImportError(name) + + sys.meta_path.insert(0, Refused()) + from solprop.backends import engines, records + from solprop.backends.session import Start + from solprop.grid import EnvelopeGrid + from solprop.nonlinear.gnlse import problem_from + from solprop.pulses import SechPulse + from solprop.stepper import PIController + from solprop.translate.assembly import assemble + + grid = EnvelopeGrid.from_limits(0.02, 835e-9, (500e-9, 2e-6), 1e-12) + gnlse = problem_from([0.0, 0.0, -1.183e-26], omega_ref=grid.omega0, + gamma=0.11, shock=False) + launch = SechPulse(wavelength=835e-9, width=100e-15, peak_power=9.8) + problem, request = assemble(grid, gnlse.mode, [gnlse.kerr], + launch.build(grid), save_planes=[0.02], + controller=PIController(rtol=1e-8, atol=1e-16), + dz_max=0.005, dz_init=1e-4, density=1.0, + shock=False, prefactor_index=1.0) + built = records.run_records(problem, request, Start.entrance(problem)) + assert isinstance(built.policy, records.RungeKuttaPolicy) + assert engines.AcceptRecord().removed == 0.0 + assert not [name for name in sys.modules if name.startswith("solprop_native")] + print("built") + """ + ) + done = subprocess.run( + [sys.executable, "-c", code], capture_output=True, text=True, check=False + ) + assert done.returncode == 0, done.stderr + assert done.stdout.strip() == "built" + + +# -- the method's policy ------------------------------------------------------------ + + +def test_an_adams_run_s_policy_is_the_march_policy_of_its_settings(): + problem, request = _gnlse(integrator=Multistep()) + records = run_records(problem, request, _entrance(problem)) + stepper = request.stepper + method = stepper.integrator + assert isinstance(method, Multistep) + # An envelope grid without the third harmonic takes no carrier shift, and the + # problem has no collar: the policy is the method's with the settings' landings. + expected = march_policy( + method, + dz_min=stepper.dz_min, + dz_max=stepper.dz_max, + stretch=stepper.stretch, + z_end=stepper.z_end, + save_planes=list(PLANES), + hard_kinks=list(problem.linear.hard_kinks), + breakpoints=list(problem.linear.phi_breakpoints), + interpolate=False, + ) + assert records.policy == expected + assert records.policy.quiet_interval is None + assert (records.low_storage, records.schedule, records.monitor) == ( + None, + None, + None, + ) + + +def test_the_reference_s_adams_march_reads_the_shared_policy(): + problem, request = _gnlse(integrator=Multistep()) + records = run_records(problem, request, _entrance(problem)) + backend: Any = select("numpy_ref", problem, request) + try: + assert backend._adams._policy == records.policy + finally: + backend.close() + + +def test_an_adams_policy_takes_its_carrier_and_collar_from_its_arguments(): + problem, request = _gnlse(integrator=Multistep()) + stepper = request.stepper + method = stepper.integrator + assert isinstance(method, Multistep) + policy = adams_policy(problem, stepper, method, carrier=True) + assert policy.carrier + assert policy.collar_rate == 0.0 # no in-stage temporal collar + + +def test_an_adaptive_runge_kutta_run_s_policy_holds_its_controller_and_landings(): + problem, request = _gnlse() + stepper = request.stepper + records = run_records(problem, request, _entrance(problem)) + breakpoints = np.asarray(problem.linear.phi_breakpoints) + assert records.policy == RungeKuttaPolicy( + controller=stepper.controller, + dz_min=stepper.dz_min, + dz_max=LENGTH / 4, + auto_init=stepper.auto_init, + stretch=stepper.stretch, + z_end=LENGTH, + save_planes=PLANES, + interpolate=False, + hard_kinks=tuple(float(z) for z in problem.linear.hard_kinks), + # Without the interval cap only the operator table's ends bound a step. + breakpoints=(float(breakpoints[0]), float(breakpoints[-1])), + landing_ulps=LANDING_ULPS, + start_growth=START_GROWTH, + quiet_interval=None, + ) + assert isinstance(records.policy, RungeKuttaPolicy) + assert records.policy.controller is stepper.controller + assert (records.low_storage, records.schedule) == (None, None) + + +def test_the_reference_s_runge_kutta_march_reads_the_shared_policy(): + problem, request = _gnlse() + backend: Any = select("numpy_ref", problem, request) + try: + assert ( + backend._controller_policy() + == run_records(problem, request, _entrance(problem)).policy + ) + assert backend._collar_can_rest() is collar_rests(problem, request.stepper) + finally: + backend.close() + + +def test_interpolated_planes_are_stepped_across(): + problem, request = _gnlse(interpolate_planes=True) + policy = runge_kutta_policy(problem, request.stepper) + assert (policy.save_planes, policy.interpolate) == ((), True) + records = run_records(problem, request, _entrance(problem)) + assert records.planes.dense + assert records.planes.fixed_positions is None + + +def test_a_replayed_run_carries_its_schedule_and_its_length(): + schedule = _uniform() + problem, request = _gnlse(schedule=schedule) + records = run_records(problem, request, _entrance(problem)) + assert isinstance(records.policy, RungeKuttaPolicy) + assert records.schedule is request.stepper.schedule + plan = records.planes + assert (plan.replay_steps, plan.replay_length) == ( + schedule.n_steps, + float(schedule.total_length), + ) + + +def test_a_low_storage_replay_carries_its_flows_and_its_operator_s_origin(): + problem, request = _gnlse(integrator=LowStorageRungeKutta(), schedule=_uniform()) + stepper = request.stepper + records = run_records(problem, request, _entrance(problem)) + assert isinstance(records.policy, RungeKuttaPolicy) + assert records.low_storage == LowStorageSettings( + flows=low_storage_flows(problem, stepper, "native"), + separable=False, + origin=float(problem.linear.z_span[0]), + ) + # The GNLSE's operator is constant: the replay tabulates one flow under "auto". + assert records.low_storage is not None and records.low_storage.flows == 1 + # The low-storage march keeps its collar in every pass (SEP 21 D15). + assert not collar_rests(problem, stepper) + + +# -- the plane plan ----------------------------------------------------------------- + + +def test_a_fresh_start_owes_its_launch_plane_and_fixes_every_position(): + problem, request = _gnlse() + plan = run_records(problem, request, _entrance(problem)).planes + assert isinstance(plan, PlanePlan) + assert plan.planes.tolist() == list(PLANES) + assert (plan.z, plan.next_plane, plan.launch_pending) == (0.0, 0, True) + assert (plan.step_planes, plan.dense, plan.period) == (0, False, 0) + assert (plan.replay_steps, plan.replay_length) == (0, 0.0) + assert plan.fixed_positions == (0.0, *PLANES) + assert (plan.attempt_capacity, plan.row_capacity) == ( + ATTEMPT_CAPACITY, + ROW_CAPACITY, + ) + assert (plan.call_attempts, plan.call_seconds) == (CALL_ATTEMPTS, CALL_SECONDS) + + +def test_a_resumed_start_continues_past_its_saved_plane(): + problem, request = _gnlse() + resumed = Start( + field=problem.initial.field0, + z=PLANES[0], + plane_index=1, + plane_pending=False, + resume=True, + ) + plan = run_records(problem, request, resumed).planes + assert (plan.z, plan.next_plane, plan.launch_pending) == (PLANES[0], 1, False) + assert plan.fixed_positions == (PLANES[0], *PLANES) + # The start's field is not read: a session takes it afterwards. + assert resumed.field is not None + + +def test_periodic_rows_and_step_planes_set_the_cadence(): + problem, request = _gnlse( + step_planes=3, + stats_spec=StatsSpec(names=("energy",), periodic=("energy",), period=4), + ) + plan = run_records(problem, request, _entrance(problem)).planes + assert (plan.step_planes, plan.period) == (3, 4) + # A step plane's position is the march's. + assert plan.fixed_positions is None + + +# -- the boundary monitor ----------------------------------------------------------- + + +def _monitored_records(start: Start | None = None) -> tuple[Any, Any, RunRecords]: + simulation = _monitored() + problem = simulation.translate() + request = simulation._run_request().for_run(monitor=simulation._boundary_monitor()) + return problem, request, run_records(problem, request, start or _entrance(problem)) + + +def test_a_monitored_run_carries_its_monitor_s_configuration(): + problem, request, records = _monitored_records() + monitor = request.boundary.monitor + held = records.monitor + assert held is not None + config = held.config + expected = dataclasses.replace( + monitor, crossing=problem.split_crossing if monitor.split else None + ).watch_policy() + # The launch's floor is formed natively: the record carries zeros for it. + expected["floor_regions"] = [0.0, 0.0, 0.0, 0.0] + expected["launch_floor"] = 0.0 + assert config.policy == expected + assert config.budget == monitor.check.budget + assert config.can_rest is collar_rests(problem, request.stepper) + np.testing.assert_array_equal(config.rate, monitor.weights[3]) + assert config.rate.flags.c_contiguous + assert (held.edge, held.continuation) == (None, None) + + +def _earlier_record() -> dict[str, Any]: + """Return an earlier execution's monitor record, as the output stores it.""" + return { + "launch_energy": 2.0, + "guard_low_max": 1e-6, + "removed_fraction": 1e-4, + "steps": 12, + "skipped": 3.0, + "quiet_checks": 2, + "collar_quiet": True, + "unwatched": [[0.1, 0.2]], + "resumed_at": [0.1], + "planes": {name: [0.5, None] for name in PLANE_COLUMNS}, + "transverse": {"launch_energy": 1.0, "edge_baseline": [0.1, 0.2]}, + "noise_floor": "not a number", + } + + +def test_a_monitored_resume_carries_its_continuation(): + record = _earlier_record() + simulation = _monitored() + problem = simulation.translate() + start = Start( + field=problem.initial.field0, + z=0.5, + plane_index=1, + plane_pending=False, + resume=True, + monitor={"record": record, "plane": True, "unwatched": [(0.2, 0.5)]}, + ) + _, _, records = _monitored_records(start) + assert records.monitor is not None + continuation = records.monitor.continuation + assert continuation is not None + assert continuation.plane + assert continuation.unwatched == ((0.2, 0.5),) + assert continuation.record == resumption(record) + # Without a transverse edge, no face reads a per-face list. + assert continuation.edge == edge_resumption(record["transverse"], 0) + + +def test_a_resumption_reads_only_the_record_s_types(): + record = _earlier_record() + held = resumption(record) + assert held.record and held.collar_quiet + # The counts are whole past the launch test; text is not a number. + assert held.numbers == { + "launch_energy": 2.0, + "guard_low_max": 1e-6, + "removed_fraction": 1e-4, + "quiet_checks": 2.0, + "steps": 12.0, + "skipped": 3.0, + "collar_left_out_checks": 0.0, + } + assert held.unwatched == ((0.1, 0.2),) + assert held.resumed_at == (0.1,) + assert held.planes is not None + assert held.planes[0][0] == 0.5 + assert all(math.isnan(column[1]) for column in held.planes) + assert resumption(None) == Resumption(record=False) + + +def test_a_record_without_a_launch_reads_no_counts(): + record = {"steps": 2.5, "launch_energy": 0.0} + assert resumption(record).numbers == {"launch_energy": 0.0} + + +def test_a_count_that_is_not_whole_is_refused_as_the_monitor_refuses_it(): + record = {"launch_energy": 1.0, "steps": -1.0} + with pytest.raises(BoundaryError, match="steps"): + resumption(record) + + +def test_an_edge_resumption_reads_further_only_past_its_launch_and_baseline(): + full = { + "launch_energy": 1.0, + "warned_at": 0.3, + "edge_baseline": [0.1, 0.2], + "edge_max": [0.3, 0.4], + "edge_max_raw": [0.5], + "reads": 7.0, + "transverse_status": "warned", + } + assert edge_resumption(full, 2) == EdgeResumption( + record=True, + numbers={"launch_energy": 1.0, "warned_at": 0.3}, + edge_baseline=(0.1, 0.2), + edge_max=(0.3, 0.4), + # One value for two faces is not a value per face. + edge_max_raw=None, + reads=7, + transverse_status="warned", + ) + dark = dict(full, launch_energy=0.0) + assert edge_resumption(dark, 2) == EdgeResumption( + record=True, + numbers={"launch_energy": 0.0, "warned_at": 0.3}, + transverse_status="warned", + ) + assert edge_resumption(None, 2) == EdgeResumption(record=False) + + +def test_an_edged_run_carries_its_edge_s_configuration(): + simulation = radial_collar("numpy_ref", RungeKutta()) + problem = simulation.translate() + monitor = simulation._boundary_monitor() + assert monitor is not None and monitor.transverse is not None + request = simulation._run_request().for_run(stats=StatsSpec(), monitor=monitor) + records = run_records(problem, request, _entrance(problem)) + assert records.monitor is not None + edge = records.monitor.edge + assert edge is not None + transverse = monitor.transverse + assert edge.faces == tuple(face.name for face in transverse._faces) + assert edge.warn == transverse.budget**2 + assert edge.stop == (transverse.stop_factor * transverse.budget) ** 2 + assert edge.length == transverse.length + # An adaptive run has no longest fixed step; the edge is read. + assert (edge.longest_step, edge.reads_edge) == (None, True) + assert edge.declared is declared(problem) + assert edge.design is (transverse.design is not None) + assert edge.kappa_order.dtype == np.uint64 + np.testing.assert_array_equal(edge.kappa_order, np.argsort(transverse._kappa)) + # The edge's faces decide whether a list holds a value per face. + faces = len(edge.faces) + for count, read in ((faces, True), (faces + 1, False)): + transverse_record = {"launch_energy": 1.0, "edge_baseline": [0.1] * count} + continuation = monitor_continuation( + monitor, {"record": {"transverse": transverse_record}} + ) + assert (continuation.edge.edge_baseline is not None) is read + assert continuation.unwatched == () + assert not continuation.plane + + +# -- the native conversion ---------------------------------------------------------- + + +@needs_native +def test_native_reads_the_runge_kutta_policy_with_the_controller_s_fields(): + native_records = pytest.importorskip("solprop_native._records") + problem, request = _gnlse() + stepper = request.stepper + controller = stepper.controller + policy = runge_kutta_policy(problem, stepper) + breakpoints = np.asarray(problem.linear.phi_breakpoints) + assert native_records.rk_policy(policy) == { + "dz_min": stepper.dz_min, + "dz_max": stepper.dz_max, + "stretch": stepper.stretch, + "z_end": stepper.z_end, + "save_planes": list(PLANES), + "interpolate": False, + "hard_kinks": [float(z) for z in problem.linear.hard_kinks], + "breakpoints": [float(breakpoints[0]), float(breakpoints[-1])], + "landing_ulps": LANDING_ULPS, + "quiet_interval": None, + "safety": controller.safety, + "beta1": controller.beta1, + "beta2": controller.beta2, + "min_factor": controller.min_factor, + "max_factor": controller.max_factor, + "lag": controller.lag, + "reject_free": controller.reject_free, + "power": float(controller.estimator_order + 1), + "exponent": 1.0 / (controller.estimator_order + 1), + "start_growth": START_GROWTH, + "auto_init": stepper.auto_init, + } + + +@needs_native +def test_native_s_adams_march_holds_the_shared_policy(): + native_backend = pytest.importorskip("solprop_native.backend") + problem, request = _gnlse(integrator=Multistep()) + records = run_records(problem, request, _entrance(problem)) + backend = native_backend.NativeBackend(problem, request) + try: + policy: MarchPolicy = backend._adams._policy # pyright: ignore[reportPrivateUsage, reportOptionalMemberAccess] + assert policy == records.policy + finally: + backend.close() + + +@needs_native +def test_native_reads_the_plane_plan_and_the_continuation(): + native_records = pytest.importorskip("solprop_native._records") + record = _earlier_record() + problem, request = _gnlse(schedule=_uniform()) + start = Start(field=None, z=PLANES[0], plane_index=1, plane_pending=False) + plan = native_records.session_plan( + run_records(problem, request, start).planes, + plane_statistics=None, + row_statistics=None, + monitor=None, + dense=None, + ) + assert plan["replay"] is True + assert (plan["next_plane"], plan["launch_pending"]) == (1, False) + assert plan["planes"].tolist() == list(PLANES) + simulation = _monitored() + monitor = simulation._boundary_monitor() + assert monitor is not None + held = native_records.continuation( + monitor_continuation( + monitor, {"record": record, "plane": True, "unwatched": [(0.2, 0.5)]} + ) + ) + assert held["plane"] is True + assert held["unwatched"] == [[0.2, 0.5]] + assert held["edge_record"] == {"record": True, "launch_energy": 1.0} + flat = held["record"] + assert flat["record"] and flat["collar_quiet"] and flat["steps"] == 12.0 + assert flat["unwatched"] == [[0.1, 0.2]] and flat["resumed_at"] == [0.1] + assert len(flat["planes"]) == len(PLANE_COLUMNS) + + +@needs_native +def test_native_reads_the_edge_s_configuration_by_field(): + native_records = pytest.importorskip("solprop_native._records") + simulation = radial_collar("numpy_ref", RungeKutta()) + problem = simulation.translate() + request = simulation._run_request().for_run( + stats=StatsSpec(), monitor=simulation._boundary_monitor() + ) + held = run_records(problem, request, _entrance(problem)).monitor + assert held is not None and held.edge is not None + values = native_records.edge_config(held.edge) + assert set(values) == {field.name for field in dataclasses.fields(held.edge)} + assert values["faces"] == list(held.edge.faces) + # The arrays cross as they are, not copied. + assert values["kappa"] is held.edge.kappa + config = native_records.monitor_config(held.config) + assert config["rate"] is held.config.rate + assert config["policy"] == held.config.policy diff --git a/tests/backends/test_transverse_owner.py b/tests/backends/test_transverse_owner.py index 80b8c73bd..714371d91 100644 --- a/tests/backends/test_transverse_owner.py +++ b/tests/backends/test_transverse_owner.py @@ -36,6 +36,7 @@ from hypothesis import given, settings from hypothesis import strategies as st +from solprop.backends import records from solprop.backends.registry import available from solprop.boundaries import SpatialAbsorber, SpatialAbsorberDesign from solprop.exceptions import BoundaryError, BoundaryWarning @@ -61,6 +62,10 @@ def _modules() -> tuple[Any, Any]: ) +def _native_records() -> Any: + return importlib.import_module("solprop_native._records") + + def _space(geometry: str) -> LineGrid | FreeGrid | RadialGrid: """Return a small grid of each kind of edge: two faces, four, or one.""" if geometry == "line": @@ -133,9 +138,11 @@ def __init__( space, _design(space, design), budget, 10.0, 0.01, np.arange(1), K800 ) self.native = core.TransverseEdge( - transverse.config_of( - self.python, steps=steps, declared=declared, reads=reads - ).native() + _native_records().edge_config( + records.edge_config( + self.python, steps=steps, declared=declared, reads=reads + ) + ) ) self.steps = steps self.declared = declared @@ -223,8 +230,8 @@ def _python_call(call: Any) -> list[tuple[str, str]]: def _native_call(self, first: dict[str, object]) -> list[tuple[str, str]]: """Serve a native call's notices, each warning then the rest of its point.""" out: list[tuple[str, str]] = [] - notice = self.transverse.EdgeNotice.of(first) - while notice.kind != self.transverse.NoticeKind.NONE: + notice = self.transverse.edge_notice_of(first) + while notice.kind != records.EdgeNoticeKind.NONE: if notice.stop: out.append( ("stop", str(self.transverse.notice_error(self.python, notice))) @@ -234,7 +241,7 @@ def _native_call(self, first: dict[str, object]) -> list[tuple[str, str]]: warning = self.transverse.notice_warning(self.python, notice) assert type(warning) is BoundaryWarning out.append(("warn", str(warning))) - notice = self.transverse.EdgeNotice.of(self.native.proceed()) + notice = self.transverse.edge_notice_of(self.native.proceed()) return out def state(self, z: float, proposal: float) -> None: @@ -244,7 +251,7 @@ def state(self, z: float, proposal: float) -> None: assert native.ceiling() == python.ceiling(z) assert native.every_step == python.every_step assert native.resumed == python.resumed - assert self.transverse.STATUSES[native.status] == python.status + assert records.STATUSES[native.status] == python.status for ahead in (0.0, proposal, 2 * proposal, python.ceiling(z)): if math.isfinite(ahead): assert native.due(z, ahead) == python.due(z, ahead) @@ -426,7 +433,9 @@ def test_a_resume_continues_the_record_as_solprop_does( pair = Pair(geometry, design) faces = len(pair.python._faces) # pyright: ignore[reportPrivateUsage] pair.python.resume(record) - pair.native.resume(pair.transverse.resumption(record, faces)) + pair.native.resume( + _native_records().edge_resumption(records.edge_resumption(record, faces)) + ) pair.launch(launch, z) pair.state(z, 1e-4) pair.record() @@ -505,7 +514,7 @@ def test_a_warning_the_caller_raises_leaves_the_rest_of_its_point_undone(): with pytest.raises(BoundaryWarning): pair.python.read(python_reading, 2e-3) notice = pair.native.read(values, 2e-3, sums) - assert notice["kind"] == pair.transverse.NoticeKind.FACE and not notice["stop"] + assert notice["kind"] == records.EdgeNoticeKind.FACE and not notice["stop"] assert pair.native.pending pair.record() @@ -602,7 +611,9 @@ def test_a_record_solprop_s_resume_ignores_is_ignored_alike(record): # reads nothing more: the maxima and the reads are not converted, nor refused. pair = Pair("line", "none") pair.python.resume(record) - pair.native.resume(pair.transverse.resumption(record, 2)) + pair.native.resume( + _native_records().edge_resumption(records.edge_resumption(record, 2)) + ) assert not pair.python.resumed and not pair.native.resumed pair.launch(Readings([0.0] * 4, [0.0] * 4, 0, 0, [1.0], [0.0], 0.0)) pair.record() diff --git a/tests/builders.py b/tests/builders.py index 281facc7d..7fd9c6f05 100644 --- a/tests/builders.py +++ b/tests/builders.py @@ -62,7 +62,7 @@ def launch_derivative(backend: NumpyRefBackend, problem: PropagationProblem) -> return backend._nonlinear( backend._require_state(), backend._stage_scalars(0.0, 0.0), - np.ones((1, *problem.shape), complex), + np.ones(problem.shape, complex), ) diff --git a/tests/data/backend_corpus.npz b/tests/data/backend_corpus.npz new file mode 100644 index 000000000..e37191ae9 Binary files /dev/null and b/tests/data/backend_corpus.npz differ diff --git a/tests/data/generate_backend_corpus.py b/tests/data/generate_backend_corpus.py new file mode 100644 index 000000000..ad24186dc --- /dev/null +++ b/tests/data/generate_backend_corpus.py @@ -0,0 +1,142 @@ +# SPDX-FileCopyrightText: 2026 John C. Travers and contributors +# +# SPDX-License-Identifier: GPL-3.0-or-later + +"""Generate ``tests/data/backend_corpus.npz``: both backends' runs, pinned. + +Recorded from the tree as it stood before SEP 52 restructured the reference and native's +seam (its phase 2, on top of phase 1's removal of the groups), and committed. It is what +the restructured backends are compared with (SEP 52 section 7, "A pinned corpus +first"; ``tests/backends/test_backend_corpus.py``): every case of +``tests/backend_corpus_cases.py`` on ``numpy_ref`` and on the native CPU build, each +backend held to its own record. + +Regenerating it would bless whatever the tree then computes. SEP 52 section 4 allows it +only for a change of the reference's numbers that its phase report records -- the +computation it changes, the largest difference on these fixed schedules against that +computation's SEP 25 budget, and whether native follows -- and that is accepted on that +evidence. So the script refuses to overwrite the file unless given ``--force``, and the +commit that uses it names the change. It needs the native package installed. + +Every thread pool is pinned to one thread, the default lane's setting, so that the +record's bits are those a one-thread run repeats; the comparison reports bitwise +agreement as a diagnostic only on the same build and thread setting +(:func:`backend_corpus_cases.identity`), and fails only beyond a rounding budget or an +accuracy envelope. + +Usage:: + + uv run python tests/data/generate_backend_corpus.py [--force] +""" + +from __future__ import annotations + +import os +import sys +from pathlib import Path + +TESTS = Path(__file__).resolve().parents[1] +sys.path.insert(0, str(TESTS)) + +# Before anything reads them: one thread per pool, as the default lane runs, and no +# machine calibration (the suite's own setting, tests/conftest.py). +os.environ["SOLPROP_CALIBRATION"] = "off" +for _name in ( + "SOLPROP_THREADS", + "SOLPROP_NATIVE_THREADS", + "VECLIB_MAXIMUM_THREADS", + "OPENBLAS_NUM_THREADS", + "MKL_NUM_THREADS", + "OMP_NUM_THREADS", +): + os.environ[_name] = "1" + +import importlib # noqa: E402 - after the environment above + +# The native runtime loads before solprop's FFTs, as in tests/conftest.py. +importlib.import_module("solprop_native") + +import json # noqa: E402 +import subprocess # noqa: E402 +import tempfile # noqa: E402 +from typing import Any # noqa: E402 + +import numpy as np # noqa: E402 + +from backend_corpus_cases import ( # noqa: E402 + CASES, + THREAD_VARIABLES, + identity, + record, +) +from march_bounds import measured_scales # noqa: E402 + +OUT = Path(__file__).with_name("backend_corpus.npz") + +BACKENDS = ("numpy_ref", "native") + + +def _commit() -> str: + """Return the checkout's head commit, marked when its sources are modified.""" + root = TESTS.parent + head = subprocess.run( + ["git", "rev-parse", "--short=12", "HEAD"], + cwd=root, + capture_output=True, + text=True, + check=True, + ).stdout.strip() + dirty = subprocess.run( + ["git", "status", "--porcelain", "--", "src", "native/src"], + cwd=root, + capture_output=True, + text=True, + check=True, + ).stdout.strip() + return head + ("-dirty" if dirty else "") + + +def main() -> int: + """Write the corpus, refusing to overwrite it without ``--force``.""" + if OUT.exists() and "--force" not in sys.argv[1:]: + sys.stderr.write( + f"{OUT} exists; pass --force only for a change SEP 52 section 4 records\n" + ) + return 1 + assert set(THREAD_VARIABLES) <= set(os.environ) + arrays: dict[str, Any] = {} + cases: dict[str, Any] = {} + with tempfile.TemporaryDirectory() as directory: + for case in CASES: + # The launch's nonlinear phase and the source's conditioning, which the + # comparison's rounding budget is formed from (march_bounds.march_bound), + # read once by the reference before anything is compared. + phi, kappa = measured_scales(case.build("numpy_ref")) + entry: dict[str, Any] = { + "route": case.route, + "march": case.march, + "phi": phi, + "kappa": kappa, + "backends": {}, + } + for backend in BACKENDS: + found = record(case, backend, Path(directory)) + for name, value in found.arrays.items(): + arrays[f"{case.name}/{backend}/{name}"] = value + entry["backends"][backend] = found.records + sys.stdout.write(f"{case.name} on {backend}\n") + cases[case.name] = entry + records = { + "generated_from": _commit(), + "identity": {backend: identity(backend) for backend in BACKENDS}, + "cases": cases, + } + text = json.dumps(records, sort_keys=True, allow_nan=False) + arrays["records"] = np.frombuffer(text.encode("utf-8"), dtype=np.uint8) + np.savez_compressed(OUT, **arrays) + sys.stdout.write(f"wrote {OUT}: {len(cases)} cases, {OUT.stat().st_size} bytes\n") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tests/driver/test_capillary_components.py b/tests/driver/test_capillary_components.py index 75bb8b94c..1ab104f13 100644 --- a/tests/driver/test_capillary_components.py +++ b/tests/driver/test_capillary_components.py @@ -264,7 +264,7 @@ def test_a_negligible_wall_beside_a_two_component_set_is_the_uncoupled_run(kind) constant = problem.linear.phi_constant assert constant is not None folded = constant.copy() - folded[0, :, 0, :] += diagonal[index, index] + folded[:, 0, :] += diagonal[index, index] inside = replace( problem, linear=replace(uncoupled.linear, phi_constant=folded), diff --git a/tests/driver/test_coherent_capillary.py b/tests/driver/test_coherent_capillary.py index 036c67d07..ca4272d8f 100644 --- a/tests/driver/test_coherent_capillary.py +++ b/tests/driver/test_coherent_capillary.py @@ -425,7 +425,7 @@ def test_the_route_records_its_flow_on_the_modal_identity(): assert record["residual_norm_bound"] == 0.0 assert record["radius"] == RADIUS assert 0.0 < float(cast("float", record["quasi_static"])) < 0.25 - assert metadata.schema_version == 2 + assert metadata.schema_version == 3 assert capability_of(metadata) == "modal" assert "mean impedance coherent wall" in str(metadata) # Nothing that is not a scalar reaches the resume key. @@ -444,9 +444,7 @@ def test_a_run_without_a_wall_declares_no_flow_and_no_identity(): assert simulation.metadata().modal is None assert problem.linear.phi_constant is not None constant = problem.linear.phi_constant - assert not np.array_equal( - constant, np.broadcast_to(constant[:, :1], constant.shape) - ) + assert not np.array_equal(constant, np.broadcast_to(constant[:1], constant.shape)) LATTICE = TubeLattice(180e-9, 10.38e-6, 6) @@ -854,7 +852,7 @@ def test_the_reference_refuses_a_problem_the_flow_cannot_sit_beside(): constant = problem.linear.phi_constant assert constant is not None varied = constant.copy() - varied[0, 1] *= 1.0000001 + varied[1] *= 1.0000001 with pytest.raises(TranslationError, match="mode-independent Phi"): NumpyRefBackend( replace(problem, linear=replace(problem.linear, phi_constant=varied)), @@ -1059,7 +1057,7 @@ def test_a_diagonal_generator_matches_the_same_diagonal_inside_phi(): constant = problem.linear.phi_constant assert constant is not None folded = constant.copy() - folded[0, :, 0, :] += diagonal[index, index] + folded[:, 0, :] += diagonal[index, index] inside = replace( problem, linear=replace(problem.linear, phi_constant=folded, coupled_flow=None), @@ -1110,7 +1108,7 @@ def test_ten_steps_apply_the_block_the_translation_layer_declared(): assert flow is not None index, launched = linear_launch(problem, simulation.grid) final = propagated(launched, simulation._run_request(), simulation.grid) - carrier = complex(delta_phi(problem.linear, 0.0, LENGTH)[0, 0, 0, index]) + carrier = complex(delta_phi(problem.linear, 0.0, LENGTH)[0, 0, index]) assert flow.generator is not None rate = complex(flow.generator[0, 0, 0, 0, index]) expected = 1e-9 * np.exp(carrier + rate * LENGTH) @@ -1168,7 +1166,7 @@ def test_each_channel_keeps_its_own_rate_in_the_weakly_coupled_limit(): assert flow is not None index, launched = linear_launch(problem, simulation.grid) final = propagated(launched, simulation._run_request(), simulation.grid) - carrier = complex(delta_phi(problem.linear, 0.0, LENGTH)[0, 0, 0, index]) + carrier = complex(delta_phi(problem.linear, 0.0, LENGTH)[0, 0, index]) assert flow.generator is not None rate = complex(flow.generator[0, 0, 0, 0, index]) expected = abs(1e-9 * np.exp(carrier + rate * LENGTH)) @@ -1250,7 +1248,7 @@ def test_the_transmission_of_one_physical_launch_converges_with_the_channel_coun problem, initial=replace(problem.initial, field0=state, noise0=None) ) final = propagated(launched, simulation._run_request(), simulation.grid) - carrier = float(np.real(delta_phi(problem.linear, 0.0, LENGTH)[0, 0, 0, index])) + carrier = float(np.real(delta_phi(problem.linear, 0.0, LENGTH)[0, 0, index])) transmissions[count] = float(np.sum(abs(final[:, 0, index]) ** 2)) / ( 1e-18 * total * math.exp(2.0 * carrier) ) @@ -2618,7 +2616,7 @@ def test_ten_steps_of_mixed_family_propagation_match_an_independent_solve(): assert flow is not None and flow.generator is not None index, launched = linear_launch(problem, simulation.grid) final = propagated(launched, simulation._run_request(), simulation.grid) - carrier = complex(delta_phi(problem.linear, 0.0, LENGTH)[0, 0, 0, index]) + carrier = complex(delta_phi(problem.linear, 0.0, LENGTH)[0, 0, index]) block = np.asarray(flow.generator[0, 0, :, :, index], dtype=complex) start = np.zeros(4, dtype=complex) start[0] = 1e-9 diff --git a/tests/driver/test_coherent_wall_phase.py b/tests/driver/test_coherent_wall_phase.py index 19e0909f0..1812f29b7 100644 --- a/tests/driver/test_coherent_wall_phase.py +++ b/tests/driver/test_coherent_wall_phase.py @@ -91,7 +91,7 @@ def predicted(problem, grid, sign): flow = problem.linear.coupled_flow assert flow is not None and flow.generator is not None index = int(np.argmin(abs(grid.omega - grid.omega0))) - carrier = delta_phi(problem.linear, 0.0, LENGTH)[0, :, 0, index] + carrier = delta_phi(problem.linear, 0.0, LENGTH)[:, 0, index] assert np.all(carrier == carrier[0]) start = np.zeros(flow.n_modes, dtype=complex) start[:2] = AMPLITUDE / np.sqrt(2.0) * np.array([1.0, sign]) diff --git a/tests/driver/test_envelope_plasma.py b/tests/driver/test_envelope_plasma.py index c5a53d44a..9d496469f 100644 --- a/tests/driver/test_envelope_plasma.py +++ b/tests/driver/test_envelope_plasma.py @@ -254,7 +254,7 @@ def test_envelope_hdf5_resume_and_reconstruction( budget = max(budget, fresh) np.testing.assert_allclose(actual, expected, rtol=0, atol=budget * scale) assert restored.metadata is not None - assert restored.metadata.schema_version == 2 + assert restored.metadata.schema_version == 3 assert restored.metadata.configuration["statistic_rate_conventions"] == { "peak_ionisation_rate": "cycle-averaged", "species_peak_ionisation_rate": "cycle-averaged", diff --git a/tests/driver/test_estimate.py b/tests/driver/test_estimate.py index 487b6c581..2b689366e 100644 --- a/tests/driver/test_estimate.py +++ b/tests/driver/test_estimate.py @@ -276,7 +276,7 @@ def test_a_timing_probe_commits_nothing_and_reads_back_its_trial(backend): # to a few roundings of the field's largest magnitude bounds it (SEP 25). stepping = cast("Any", fresh) # the driver's own trial, as a kernel test _, state = stepping._trial_step(float(problem.initial.z), dz) - trial = np.asarray(stepping._observed_field(stepping._finish(state, dz)))[0] + trial = np.asarray(stepping._observed_field(stepping._finish(state, dz))) scale = float(np.max(np.abs(trial))) np.testing.assert_allclose(whole, trial, rtol=0.0, atol=64e-16 * scale) # A copy the probe owns: a later trial does not write to it. diff --git a/tests/driver/test_flow_collar.py b/tests/driver/test_flow_collar.py index 053957df2..c444d74c6 100644 --- a/tests/driver/test_flow_collar.py +++ b/tests/driver/test_flow_collar.py @@ -62,7 +62,7 @@ from march_bounds import FIXED_ORDER from modal_provider_cases import GENERAL_Z_END from sessions.runs import bare, final_field, to_planes -from solprop.backends import available +from solprop.backends import available, records from solprop.backends.numpy_ref import NumpyRefBackend from solprop.backends.session import Start from solprop.backends.sessions import start_session @@ -320,22 +320,18 @@ def test_a_coupled_flow_keeps_its_collar_in_every_stage(transient, name, monkeyp # The native session's monitor holds the collar's state natively: the rule # reaches it in the monitor's configuration, which a monitored session of the # flow is built with (SEP 47 phase 5). - from solprop_native import ( # noqa: PLC0415 # pyright: ignore[reportMissingImports] - _watch, - ) - simulation = _transient_simulation() monitor = simulation._boundary_monitor() assert monitor is not None request = run_request.for_run(stats=StatsSpec(), monitor=monitor) asked: list[bool] = [] - configured = _watch.monitor_config + configured = records.monitor_config def spied(monitor, crossing, *, can_rest): asked.append(can_rest) return configured(monitor, crossing, can_rest=can_rest) - monkeypatch.setattr(_watch, "monitor_config", spied) + monkeypatch.setattr(records, "monitor_config", spied) session = start_session( backend(problem, request, grid=grid), problem, request, Start.entrance(problem) ) diff --git a/tests/driver/test_general_modal.py b/tests/driver/test_general_modal.py index 569528d74..e7d15d0dd 100644 --- a/tests/driver/test_general_modal.py +++ b/tests/driver/test_general_modal.py @@ -1191,7 +1191,7 @@ def test_an_hdf5_file_carries_the_labels_and_the_identities(guide, grid, tmp_pat simulation = build(guide, grid) simulation.run(path, progress="none") stored = resume(path).metadata - assert stored.schema_version == 2 + assert stored.schema_version == 3 assert stored.modal is not None assert dict(stored.modal)["basis"] == "general_modal" labels = [dict(one.label) for one in stored.modes if one.label is not None] diff --git a/tests/driver/test_material_manifest.py b/tests/driver/test_material_manifest.py index 76ed022bc..b0b309a63 100644 --- a/tests/driver/test_material_manifest.py +++ b/tests/driver/test_material_manifest.py @@ -566,7 +566,7 @@ def test_ten_steps_write_the_manifest_and_a_changed_identity_is_refused(tmp_path **{"noise": None, **options}, ) stored: Any = open_file(path).metadata - assert stored.schema_version == 2 + assert stored.schema_version == 3 assert names(stored.materials) == [ "eos/Ar", "fit/Ar", diff --git a/tests/driver/test_moving_guide.py b/tests/driver/test_moving_guide.py index 8d0d06779..3f92fb519 100644 --- a/tests/driver/test_moving_guide.py +++ b/tests/driver/test_moving_guide.py @@ -134,7 +134,7 @@ def test_the_problem_carries_moving_maps_a_frozen_flow_and_the_kink(kinked_probl assert problem.linear.hard_kinks.tolist() == [KINK] phi = problem.linear.phi_expansion assert phi is not None - assert np.array_equal(phi.terms[:, :, 0], phi.terms[:, :, 1]) + assert np.array_equal(phi.terms[:, 0], phi.terms[:, 1]) bundle = scalars(problem, 0.0, 0.55 * LENGTH) assert bundle.transform_weights is not None assert bundle.transform_weights.shape == (10,) diff --git a/tests/driver/test_native_flow_refusals.py b/tests/driver/test_native_flow_refusals.py index d9335e098..19fb85e9c 100644 --- a/tests/driver/test_native_flow_refusals.py +++ b/tests/driver/test_native_flow_refusals.py @@ -196,7 +196,7 @@ def test_every_flow_the_reference_refuses_native_refuses_in_the_same_words( constant = problem.linear.phi_constant assert constant is not None varied = constant.copy() - varied[0, 1] *= 1.0000001 + varied[1] *= 1.0000001 metric = orthonormal_metric( problem.grids.omega, problem.linear.phi_breakpoints[[0, -1]], diff --git a/tests/driver/test_native_flow_session.py b/tests/driver/test_native_flow_session.py index 23d388805..7dd880454 100644 --- a/tests/driver/test_native_flow_session.py +++ b/tests/driver/test_native_flow_session.py @@ -402,11 +402,11 @@ def _block_departure( )[1].shape[0] assert bool(shift.size) is carrier expected = coupled_moments( - _testing.adams_exponent_to_host(state).reshape(1, *shape), + _testing.adams_exponent_to_host(state).reshape(shape), count, block, dz, - carrier=np.full((1, 1, 1, 1), 1j * float(shift[0])) if carrier else None, + carrier=np.full((1, 1, 1), 1j * float(shift[0])) if carrier else None, ) active = int(np.count_nonzero(block.active)) native = _testing.adams_block_to_host(state).reshape(-1, active, shape[0], shape[0]) @@ -641,7 +641,7 @@ def test_a_carrier_frame_off_the_route(parity_cases, monkeypatch): steps = request.stepper.schedule.steps anchors = np.concatenate(([0.0], np.cumsum(steps)[:-1])) carrier = max( - float(np.max(np.abs(np.imag(reference.chord.exponent(z, z + dz))))) + float(np.abs(np.imag(reference.chord.exponent(z, z + dz)))) for z, dz in zip(anchors, steps, strict=True) ) assert carrier > 0.0 diff --git a/tests/driver/test_native_march_call_bound.py b/tests/driver/test_native_march_call_bound.py index 2082cd17f..24fb0818b 100644 --- a/tests/driver/test_native_march_call_bound.py +++ b/tests/driver/test_native_march_call_bound.py @@ -12,8 +12,8 @@ clock's every attempt, and never. Every native run marches in the native execution session (SEP 47), whose march calls -take the session's call bound (``solprop_native.session.CALL_ATTEMPTS`` and -``CALL_SECONDS``), dense output and a transverse edge included. +take the session's call bound (``solprop.backends.records.CALL_ATTEMPTS`` and +``CALL_SECONDS``, the plane plan's), dense output and a transverse edge included. """ import dataclasses @@ -29,7 +29,7 @@ from backends.test_exponential_varying import EXPONENTIAL, gradient from backends.test_transverse_backends import radial_collar from legacy_geometry import LEGACY_GRID, bare_absorber -from solprop.backends import available +from solprop.backends import available, records from solprop.driver import Simulation from solprop.grid import EnvelopeGrid from solprop.monitor import BoundaryCheck @@ -235,8 +235,8 @@ def _counted_attempts(patch: pytest.MonkeyPatch, most: int, seconds: float) -> C """Bound the session's native calls; return the record that fills with them.""" # The session's own bound: a call's attempts and time; its advances are lifted, # so that only the call's bound returns it. - patch.setattr(native_session, "CALL_ATTEMPTS", most) - patch.setattr(native_session, "CALL_SECONDS", seconds if seconds > 0 else 1e-300) + patch.setattr(records, "CALL_ATTEMPTS", most) + patch.setattr(records, "CALL_SECONDS", seconds if seconds > 0 else 1e-300) advance = native_session.NativeSession.advance calls = Calls() @@ -335,5 +335,5 @@ def test_the_rejections_case_rejects_steps(build_simulation, tmp_path): def test_the_call_bound_is_the_documented_one(): # `docs/guide/results/resuming.md` and `docs/dev/kernel-contract.md` state these. - bound = (native_session.CALL_ATTEMPTS, native_session.CALL_SECONDS) + bound = (records.CALL_ATTEMPTS, records.CALL_SECONDS) assert bound == (256, 0.1) diff --git a/tests/driver/test_native_march_lifecycle.py b/tests/driver/test_native_march_lifecycle.py index e41bce8c2..f4adbb1fb 100644 --- a/tests/driver/test_native_march_lifecycle.py +++ b/tests/driver/test_native_march_lifecycle.py @@ -67,8 +67,8 @@ def with_fault(monkeypatch, stage: str, after: int, category: str) -> list[Any]: armed: list[Any] = [] configure = NativeAdamsMarch._configure - def arming(self): - configure(self) + def arming(self, *args): + configure(self, *args) if not armed: _testing.adams_march_fault( self._backend._native_state, stage, after, category diff --git a/tests/driver/test_native_moving_guide_bound.py b/tests/driver/test_native_moving_guide_bound.py index 4c721d806..c910fa89c 100644 --- a/tests/driver/test_native_moving_guide_bound.py +++ b/tests/driver/test_native_moving_guide_bound.py @@ -69,9 +69,7 @@ def source_at(implementation, problem, run_request, grid, positions): backend = implementation(problem, run_request, grid=grid) try: backend.set_initial(state, None) - # The backend's private hook keeps the state's leading axis of one. - held = state[np.newaxis] - return [np.array(backend._adams_lab_source(held, z)) for z in positions] + return [np.array(backend._adams_lab_source(state, z)) for z in positions] finally: backend.close() # Native's resident Adams engine, on the same problem under the Adams march: its diff --git a/tests/driver/test_native_resume_history.py b/tests/driver/test_native_resume_history.py index 88025632d..fc2032715 100644 --- a/tests/driver/test_native_resume_history.py +++ b/tests/driver/test_native_resume_history.py @@ -235,8 +235,7 @@ def test_the_resident_history_is_restored_exactly_and_continued(build_simulation problem, request, first = opened(simulation, "native") engine = first._resident assert type(engine).__name__ == "_ResidentAdams" - # The engine's own layout keeps a leading axis of one after the row axis. - shape = (engine.n_slots, 1, *problem.shape) + shape = (engine.n_slots, *problem.shape) whole = Snapshots() baseline = run_session(first, problem, request, owns=True, between=whole) stored = whole.at_plane(0) diff --git a/tests/driver/test_plasma_band_propagation.py b/tests/driver/test_plasma_band_propagation.py index ba313e024..46dbe9fb1 100644 --- a/tests/driver/test_plasma_band_propagation.py +++ b/tests/driver/test_plasma_band_propagation.py @@ -300,7 +300,7 @@ def test_the_reference_s_plasma_statistics_refuse_a_problem_with_no_plasma( "numpy_ref", problem, simulation._run_request(), grid=simulation.grid ) try: - state = simulation._field0[np.newaxis] + state = simulation._field0 with pytest.raises(BackendError, match=message): cast("NumpyRefBackend", backend)._backend_statistics((name,), state) finally: diff --git a/tests/driver/test_prop_free.py b/tests/driver/test_prop_free.py index 9099012b5..11c14a9cc 100644 --- a/tests/driver/test_prop_free.py +++ b/tests/driver/test_prop_free.py @@ -123,7 +123,7 @@ def test_free_interface_separable_flow_and_metadata(free_case, request, backend) grid, space, SpatialMedium(), [], pulse, settings=settings, noise=None ) start, finish = output.fields("fft") - expected = start * np.exp(delta_phi(problem.linear, 0, grid.length))[0] + expected = start * np.exp(delta_phi(problem.linear, 0, grid.length)) defect = np.max(np.abs(finish - expected)) / np.max(np.abs(start)) assert defect < field_bound energy_error = abs( @@ -218,7 +218,7 @@ def test_free_simulation_records_noise_model_and_seed( # With no nonlinear source, both noise models obey the exact linear flow # of their saved entrance. The phase subtraction and complex exponential # cost fewer than 64 rounded operations per optical phase magnitude. - phase = delta_phi(simulation.translate().linear, 0.0, grid.length)[0] + phase = delta_phi(simulation.translate().linear, 0.0, grid.length) bound = 64 * EPS * (1 + float(np.max(np.abs(phase)))) fields = simulation.run().fields("fft") expected = np.exp(phase) * fields[0] @@ -346,7 +346,7 @@ def test_cartesian_resume_distinguishes_legacy_cutoff_arithmetic(free_case): grid, space, SpatialMedium(), [], pulse, settings=settings, noise=None ) current = simulation.metadata() - assert current.schema_version == 2 + assert current.schema_version == 3 assert current.configuration["spatial_arithmetic"] == "split-square-v1" legacy_configuration = dict(current.configuration) del legacy_configuration["spatial_arithmetic"] diff --git a/tests/driver/test_prop_radial.py b/tests/driver/test_prop_radial.py index b3ae5bc6e..35e821f1c 100644 --- a/tests/driver/test_prop_radial.py +++ b/tests/driver/test_prop_radial.py @@ -73,7 +73,7 @@ def test_radial_interface_caches_and_saves_geometry(case, backend, request): # Preserve the compact pulse specification; serializing the built transverse # volume here would duplicate the input and make Cartesian metadata enormous. assert metadata.configuration["initial"] is pulse - assert metadata.schema_version == 2 + assert metadata.schema_version == 3 assert metadata.simulation_kind == "radial" assert metadata.spatial is not None assert metadata.spatial["stored_field_units"] == "V m" @@ -173,7 +173,7 @@ def test_radial_simulation_records_noise_model_and_seed(case, model, backend, re # With no nonlinear source, both noise models obey the exact linear flow # of their saved entrance. The phase subtraction and complex exponential # cost fewer than 64 rounded operations per optical phase magnitude. - phase = delta_phi(simulation.translate().linear, 0.0, grid.length)[0] + phase = delta_phi(simulation.translate().linear, 0.0, grid.length) bound = 64 * EPS * (1 + float(np.max(np.abs(phase)))) fields = simulation.run().fields("fft") expected = np.exp(phase) * fields[0] diff --git a/tests/driver/test_prop_slab.py b/tests/driver/test_prop_slab.py index f2466bead..79974a90e 100644 --- a/tests/driver/test_prop_slab.py +++ b/tests/driver/test_prop_slab.py @@ -243,7 +243,7 @@ def test_the_run_records_its_measure_and_labels_per_unit_width(): back = RunMetadata.from_json(metadata.to_json()) assert back.transverse_measure == "per_unit_width" assert back.configuration_hash() == metadata.configuration_hash() - assert json.loads(metadata.to_json())["schema_version"] == 2 + assert json.loads(metadata.to_json())["schema_version"] == 3 units = transverse_units(output) assert (units.power, units.energy, units.intensity) == ("W/m", "J/m", "W/m^2") figures = plot_stats(output) diff --git a/tests/driver/test_resume.py b/tests/driver/test_resume.py index 57c861e50..e9ece8a77 100644 --- a/tests/driver/test_resume.py +++ b/tests/driver/test_resume.py @@ -365,13 +365,11 @@ def test_the_input_draw_is_added_at_a_launch_and_not_to_a_resumed_state( rtol=0.0, atol=1e-12 * np.linalg.norm(draw), ) - # The floor is the draw itself, behind the march's private axis of one. - assert engine._input_floor is not None - assert engine._input_floor.base is draw + # The floor is the draw itself. + assert engine._input_floor is draw engine.set_initial(launched, None, z=0.0, resume=True) np.testing.assert_array_equal(engine.observe().require_field(), launched) - assert engine._input_floor is not None - assert engine._input_floor.base is draw + assert engine._input_floor is draw finally: engine.close() diff --git a/tests/driver/test_scan_stop_progress.py b/tests/driver/test_scan_stop_progress.py index a869b7d59..641ce44d8 100644 --- a/tests/driver/test_scan_stop_progress.py +++ b/tests/driver/test_scan_stop_progress.py @@ -18,6 +18,7 @@ from legacy_geometry import LEGACY_GRID, bare_absorber from march_bounds import FIXED_ORDER +from solprop.backends import records from solprop.backends.registry import available from solprop.backends.session import DEFAULT_WORK_LIMIT from solprop.driver._scan_run import _collect, _Job, _Options, _ScanEvents, _Work @@ -262,8 +263,7 @@ def test_local_stop_mid_point_is_not_committed_and_a_rerun_computes_it(tmp_path) @pytest.mark.skipif("native" not in available(), reason="native is not installed") def test_local_stop_reaches_a_one_plane_native_adams_point(tmp_path): - native = pytest.importorskip("solprop_native.session") - most = native.CALL_ATTEMPTS + most = records.CALL_ATTEMPTS def build(point): # More steps than a native call's bound, and no plane until the last of them. diff --git a/tests/driver/test_specified_paths.py b/tests/driver/test_specified_paths.py index 5267c8493..6d8f5f1bf 100644 --- a/tests/driver/test_specified_paths.py +++ b/tests/driver/test_specified_paths.py @@ -157,7 +157,7 @@ def test_a_linear_beta1_path_accumulates_its_exact_walk_off(grid): worst = 0.0 for fraction in (0.25, 0.5, 0.75, 1.0): distance = fraction * LENGTH - accumulated = delta_phi(table, 0.0, distance)[0, 0, 0][active] + accumulated = delta_phi(table, 0.0, distance)[0, 0][active] expected = -1j * GRADIENT * distance**2 / 2.0 * detuning assert_allclose(accumulated[outer], expected[outer], rtol=rtol, atol=0.0) scale = np.max(np.abs(expected)) @@ -197,7 +197,7 @@ def test_a_kinked_beta1_path_lands_the_schedule_on_its_corner(grid): # That triangle's area: half its height times its base, the two legs together. delay = 0.5 * GRADIENT * corner * LENGTH outer = np.abs(detuning) > 0.5 * np.abs(detuning).max() - accumulated = delta_phi(table, 0.0, LENGTH)[0, 0, 0][active] + accumulated = delta_phi(table, 0.0, LENGTH)[0, 0][active] expected = -1j * delay * detuning assert_allclose(accumulated[outer], expected[outer], rtol=rtol, atol=0.0) diff --git a/tests/driver/test_taper_reciprocity.py b/tests/driver/test_taper_reciprocity.py index 539a740d2..66b8e9d6e 100644 --- a/tests/driver/test_taper_reciprocity.py +++ b/tests/driver/test_taper_reciprocity.py @@ -104,7 +104,7 @@ def transfer(radii, steps, name="numpy_ref"): driver = backend(launched, run_request, grid=grid) final = final_field(driver, launched, bare(run_request), ended=True, owns=True) columns.append(final[:, 0, index] / 1e-9) - carrier = delta_phi(problem.linear, 0.0, LENGTH)[0, :, 0, index] + carrier = delta_phi(problem.linear, 0.0, LENGTH)[:, 0, index] assert np.all(carrier == carrier[0]) root = {z: np.sqrt(power_weight_at(problem, z)[:, 0, index]) for z in (0, LENGTH)} vacuum = np.stack(columns, axis=1) diff --git a/tests/driver/test_tdse_hacc_run.py b/tests/driver/test_tdse_hacc_run.py index 80ab4f25a..33276006f 100644 --- a/tests/driver/test_tdse_hacc_run.py +++ b/tests/driver/test_tdse_hacc_run.py @@ -112,7 +112,7 @@ def test_hacc_run_records_model_window_consistency_and_ledger(fake, cache, windo simulation = _simulation(atomic_window=window) output = _run(simulation) assert output.metadata is not None - assert output.metadata.schema_version == SCHEMA_VERSION == 2 + assert output.metadata.schema_version == SCHEMA_VERSION == 3 descriptor = json.loads(plain(output.metadata.tdse)["configuration_json"]) assert descriptor["schema"] == "solprop.tdse-descriptor.v5" assert descriptor["response"]["model"] == "hacc" diff --git a/tests/driver/test_temperature_paths.py b/tests/driver/test_temperature_paths.py index 4e084a182..3a9ef7cb6 100644 --- a/tests/driver/test_temperature_paths.py +++ b/tests/driver/test_temperature_paths.py @@ -226,7 +226,7 @@ def test_three_populations_on_the_native_z_table_with_each_march(integrator): def test_the_run_records_the_temperature_path(): metadata = build(PAIR, WARM).metadata() configuration = cast("dict", serialise(metadata.configuration)) - assert metadata.schema_version == 2 + assert metadata.schema_version == 3 # The constructor's record keeps the path as the dataclass it is, and the # medium's record samples it with its breakpoints, as it does a density. assert configuration["capillary"]["temperature"] == { diff --git a/tests/driver/test_vector_raman_output.py b/tests/driver/test_vector_raman_output.py index 20c1b9a77..179ed7af4 100644 --- a/tests/driver/test_vector_raman_output.py +++ b/tests/driver/test_vector_raman_output.py @@ -85,6 +85,6 @@ def simulation(): # The same stored realisation carried by one march: its rounding budget. assert_matched(actual, expected, march, what=key) metadata = resume(part).metadata - assert metadata.schema_version == 2 + assert metadata.schema_version == 3 record = repr(metadata.configuration) assert "tensor_terms" in record and "rank2" in record and "isotropic" in record diff --git a/tests/driver/test_wall_production.py b/tests/driver/test_wall_production.py index ad055b289..db80736e0 100644 --- a/tests/driver/test_wall_production.py +++ b/tests/driver/test_wall_production.py @@ -100,7 +100,7 @@ def pinned_final(): assert flow is not None assert flow.generator is not None span = float(problem.linear.phi_breakpoints[-1]) - rate = complex(delta_phi(problem.linear, 0.0, span)[0, 0, 0, index]) / span + rate = complex(delta_phi(problem.linear, 0.0, span)[0, 0, index]) / span single = np.zeros_like(grid.active_mask) single[index] = True block = wall_operator( diff --git a/tests/flow_parity.py b/tests/flow_parity.py index 15621d961..0cb2663d8 100644 --- a/tests/flow_parity.py +++ b/tests/flow_parity.py @@ -670,8 +670,7 @@ def flow_scales( generator = flow.n_deg * float(np.max(norms)) + float(np.max(norms[:, 0])) backend = NumpyRefBackend(problem, run_request, grid=grid) try: - # The backend's private state keeps its leading axis of one. - state = problem.initial.field0[np.newaxis] + state = problem.initial.field0 bundle = backend._stage_scalars(0.0, 0.0) source = backend._nonlinear(state, bundle, np.ones_like(state)) finally: diff --git a/tests/march_bounds.py b/tests/march_bounds.py index 5d5a6ea53..017e30d07 100644 --- a/tests/march_bounds.py +++ b/tests/march_bounds.py @@ -50,6 +50,7 @@ UNIT_ROUNDOFF, AccuracyEnvelope, assert_within, + assert_work_within, fft_bound, propagation_growth, step_evaluations, @@ -179,8 +180,7 @@ def measured_scales(simulation) -> tuple[float, float]: run_request = simulation._run_request() backend = NumpyRefBackend(problem, run_request, grid=simulation.grid) try: - # The backend's private state keeps its leading axis of one. - state = problem.initial.field0[np.newaxis] + state = problem.initial.field0 bundle = backend._stage_scalars(0.0, 0.0) source = backend._nonlinear(state, bundle, np.ones_like(state)) finally: @@ -190,9 +190,7 @@ def measured_scales(simulation) -> tuple[float, float]: maps = _general_maps(problem) kappa = 1.0 if maps is not None: - kappa = conditioning( - maps.at(bundle.transform_weights), state[0], simulation.grid - ) + kappa = conditioning(maps.at(bundle.transform_weights), state, simulation.grid) return phi, kappa @@ -343,3 +341,62 @@ def assert_points_matched( factors=factors, what=f"{what} {point.index}", ) + + +def reading_tolerance(reference: float, bound: float, steps: int) -> float: + """Return how far a monitor reading may move when the field moves by `bound`. + + A reading is an energy share ``s = ||a||_w^2`` of a weighted region, mostly as a + fraction of the launch's energy, so a departure of the state, normwise `bound` of + it, moves it by at most ``2 sqrt(s) bound + bound^2``: the cross term and the + square. The split collar's removal adds such readings over a run's `steps`, whose + square roots add to at most ``sqrt(steps s)`` (Cauchy-Schwarz), and a ratio of two + readings carries :data:`STATISTIC_FACTOR` of the relative departure. So + ``STATISTIC_FACTOR bound |s| + 2 sqrt(steps |s|) bound + steps bound^2``, which + holds an empty guard's rounding residue (of order ``n eps^2``) as well as a filled + guard. + """ + size = abs(reference) + return ( + STATISTIC_FACTOR * bound * size + + 2.0 * math.sqrt(steps * size) * bound + + steps * bound**2 + ) + + +def assert_record_within( + got, expected, bound: float, steps: int, *, what: str = "", work: bool = False +) -> None: + """Assert a monitor or edge record agrees with another to a field's departure. + + The keys (in any order), flags, texts and ``None`` exactly; each real number within + :func:`reading_tolerance` of `bound` over `steps`; each integer exactly (the checked + and skipped steps, the reads), or with `work` -- an adaptive march, whose checks + fall on its own steps -- within a factor of two (:func:`bounds.assert_work_within`). + """ + if isinstance(expected, dict): + assert isinstance(got, dict) and sorted(got) == sorted(expected), what + for key, value in expected.items(): + assert_record_within( + got[key], value, bound, steps, what=f"{what}.{key}", work=work + ) + return + if isinstance(expected, list | tuple): + assert isinstance(got, list | tuple) and len(got) == len(expected), what + for index, (one, other) in enumerate(zip(got, expected, strict=True)): + assert_record_within( + one, other, bound, steps, what=f"{what}[{index}]", work=work + ) + return + if expected is None or isinstance(expected, bool | str): + assert got == expected, (what, got, expected) + return + if isinstance(expected, int): + if work and expected > 0: + assert_work_within(int(got), expected, factor=2.0, what=what) + else: + assert got == expected, (what, got, expected) + return + tolerance = reading_tolerance(float(expected), bound, steps) + error = abs(float(got) - float(expected)) + assert error <= tolerance, (what, got, expected, tolerance) diff --git a/tests/nonlinear/test_kerr_coefficient_per_true_watt.py b/tests/nonlinear/test_kerr_coefficient_per_true_watt.py index 220773dfa..df3a71496 100644 --- a/tests/nonlinear/test_kerr_coefficient_per_true_watt.py +++ b/tests/nonlinear/test_kerr_coefficient_per_true_watt.py @@ -30,18 +30,17 @@ def true_watt_slope(problem, run_request, grid): ) backend = NumpyRefBackend(problem, run_request, grid=grid) try: - # The backend's private hook keeps the state's leading axis of one. - state = np.zeros((1, *problem.shape), complex) + state = np.zeros(problem.shape, complex) carrier = int(np.argmin(abs(grid.omega - grid.omega0))) state[..., carrier] = grid.n_t / (2 if grid.kind == "field" else 1) state /= np.sqrt(power_weight_at(problem, 0.0)) derivative = backend._nonlinear( state, stage.scalars(problem, 0.0, 0.0), - np.ones((1, *problem.shape), complex), + np.ones(problem.shape, complex), ) - amplitude = fourier.inverse(state[0, 0, 0], kind=grid.kind) - response = fourier.inverse(derivative[0, 0, 0], kind=grid.kind) + amplitude = fourier.inverse(state[0, 0], kind=grid.kind) + response = fourier.inverse(derivative[0, 0], kind=grid.kind) if grid.kind == "field": amplitude = fourier.analytic_signal(amplitude) response = fourier.analytic_signal(response) diff --git a/tests/output/test_handlers.py b/tests/output/test_handlers.py index 09351fc5f..7df08f226 100644 --- a/tests/output/test_handlers.py +++ b/tests/output/test_handlers.py @@ -1270,9 +1270,9 @@ def test_a_plane_whose_channel_changes_its_statistics_is_refused(grid, metadata) def test_a_run_record_declares_the_current_output_schema(metadata): - """Schema 2 is schema 1, the first released layout, plus a recorded field.""" - assert SCHEMA_VERSION == 2 - assert metadata.schema_version == 2 + """Schema 3 is schema 1, the first released layout, plus two changes.""" + assert SCHEMA_VERSION == 3 + assert metadata.schema_version == 3 def test_five_columns_continue_as_before(grid, metadata, tmp_path): @@ -1308,7 +1308,7 @@ def test_a_file_of_another_output_schema_is_refused_by_name( "meta/json", data=json.dumps(document), dtype=h5py.string_dtype() ) file.attrs["schema_version"] = declared - refusal = rf"schema_version {declared}.*reads output schema 2 only" + refusal = rf"schema_version {declared}.*reads output schema 3 only" with pytest.raises(OutputError, match=refusal): hdf5_module.open_file(path) with pytest.raises(OutputError, match=refusal): diff --git a/tests/output/test_march_snapshot_layout.py b/tests/output/test_march_snapshot_layout.py index 99d8a6858..ab5cfc3da 100644 --- a/tests/output/test_march_snapshot_layout.py +++ b/tests/output/test_march_snapshot_layout.py @@ -2,20 +2,21 @@ # # SPDX-License-Identifier: GPL-3.0-or-later -"""A march snapshot's state is one trajectory's; its file keeps the leading axis of one. +"""A march snapshot's state is one trajectory's, on disk as at run time. -SEP 47 section 6.7: the runtime snapshot carries ``(n_modes, n_pol, n_omega)``, and the -stored dataset keeps the ``(1, n_modes, n_pol, n_omega)`` layout every file has been -written with, so old files stay readable and a stacked state is never squeezed. +SEP 52 section 5.4: from output schema 3 the stored dataset is ``(n_modes, n_pol, +n_omega)``, as the runtime snapshot carries it. A snapshot written before, with the +leading axis of one it then kept, is refused by name rather than squeezed, and so is a +state of any other shape. """ -import logging - import h5py import numpy as np import pytest +from solprop.exceptions import OutputError from solprop.output.hdf5 import _read_march, _write_march +from solprop.output.metadata import SCHEMA_VERSION from solprop.output.records import MarchSnapshot @@ -32,26 +33,29 @@ def snapshot(state): positions=np.array([0.4, 0.5]), identities=("accepted", "predicted"), slots=np.array([0, 1]), - rows={"store": np.zeros((4, 1, 1, 1, 4), complex)}, + rows={"store": np.zeros((4, 1, 1, 4), complex)}, history_key="adams", ) -def test_a_snapshot_is_stored_with_its_leading_axis_and_read_back_without(tmp_path): +def test_a_snapshot_is_stored_and_read_back_in_the_trajectory_s_layout(tmp_path): state = np.arange(4, dtype=complex).reshape(1, 1, 4) with h5py.File(tmp_path / "run.h5", "w") as file: _write_march(file.require_group("march_snapshot"), snapshot(state)) stored = file["march_snapshot/state"] assert isinstance(stored, h5py.Dataset) - assert stored.shape == (1, 1, 1, 4) + assert stored.shape == (1, 1, 4) + rows = file["march_snapshot/rows/store"] + assert isinstance(rows, h5py.Dataset) + assert rows.shape == (4, 1, 1, 4) read = _read_march(file, 0.5) assert read is not None assert read.state.shape == (1, 1, 4) np.testing.assert_array_equal(read.state, state) -def _old_file(path, stored): - """Write a snapshot group as the files before SEP 47 hold it, with `stored`.""" +def _stored_as(path, stored): + """Write a snapshot group whose state dataset is replaced by `stored`.""" with h5py.File(path, "w") as file: _write_march( file.require_group("march_snapshot"), snapshot(np.zeros((1, 1, 4), complex)) @@ -62,36 +66,33 @@ def _old_file(path, stored): group.create_dataset("state", data=stored) -def test_an_existing_file_s_four_axis_state_stays_readable(tmp_path): - stored = (1.0 + 2.0j) * np.ones((1, 2, 1, 4)) - _old_file(tmp_path / "old.h5", stored) - with h5py.File(tmp_path / "old.h5", "r") as file: - read = _read_march(file, 0.5) - assert read is not None - np.testing.assert_array_equal(read.state, stored[0]) +def test_a_snapshot_with_the_earlier_leading_axis_is_refused_by_name(tmp_path): + _stored_as(tmp_path / "old.h5", (1.0 + 2.0j) * np.ones((1, 2, 1, 4))) + with ( + h5py.File(tmp_path / "old.h5", "r") as file, + pytest.raises(OutputError, match=f"before output schema {SCHEMA_VERSION}"), + ): + _read_march(file, 0.5) -def test_a_stacked_state_is_refused_rather_than_squeezed(tmp_path, caplog): - _old_file(tmp_path / "stacked.h5", np.ones((2, 1, 1, 4), complex)) +@pytest.mark.parametrize("shape", [(2, 1, 1, 4), (1, 4), (1, 1, 1, 1, 4)]) +def test_a_state_of_another_shape_is_refused_rather_than_squeezed(tmp_path, shape): + _stored_as(tmp_path / "rank.h5", np.ones(shape, complex)) with ( - caplog.at_level(logging.WARNING, logger="solprop.output.hdf5"), - h5py.File(tmp_path / "stacked.h5", "r") as file, + h5py.File(tmp_path / "rank.h5", "r") as file, + pytest.raises(OutputError, match="one trajectory's"), ): - assert _read_march(file, 0.5) is None - assert any("one trajectory's" in record.message for record in caplog.records) + _read_march(file, 0.5) -@pytest.mark.parametrize("shape", [(1, 4), (1, 1, 1, 1, 4)]) -def test_a_state_of_another_rank_is_refused(tmp_path, shape): - _old_file(tmp_path / "rank.h5", np.ones(shape, complex)) - with h5py.File(tmp_path / "rank.h5", "r") as file: +def test_a_snapshot_whose_state_was_never_written_is_ignored(tmp_path, caplog): + """A write cut short before the state leaves a snapshot to skip, not a refusal.""" + with h5py.File(tmp_path / "cut.h5", "w") as file: + _write_march( + file.require_group("march_snapshot"), snapshot(np.zeros((1, 1, 4), complex)) + ) + group = file["march_snapshot"] + assert isinstance(group, h5py.Group) + del group["state"] assert _read_march(file, 0.5) is None - - -def test_a_state_stored_without_the_axis_is_read_as_it_is(tmp_path): - stored = np.ones((3, 1, 4), complex) - _old_file(tmp_path / "bare.h5", stored) - with h5py.File(tmp_path / "bare.h5", "r") as file: - read = _read_march(file, 0.5) - assert read is not None - np.testing.assert_array_equal(read.state, stored) + assert "ignoring the unreadable march snapshot" in caplog.text diff --git a/tests/output/test_material_resume.py b/tests/output/test_material_resume.py index 086896063..f4a1c1eb8 100644 --- a/tests/output/test_material_resume.py +++ b/tests/output/test_material_resume.py @@ -77,7 +77,7 @@ def selected(): def test_the_current_schema_carries_the_manifest_through_memory_and_json(selected): - assert SCHEMA_VERSION == 2 == selected.schema_version + assert SCHEMA_VERSION == 3 == selected.schema_version memory = MemoryOutput() memory.open(selected) stored: Any = memory.metadata diff --git a/tests/output/test_modal_resume.py b/tests/output/test_modal_resume.py index 6108bb189..c4be1416a 100644 --- a/tests/output/test_modal_resume.py +++ b/tests/output/test_modal_resume.py @@ -136,7 +136,7 @@ def written(guide, grid, tmp_path_factory): def test_the_record_carries_every_modal_identity(written, guide): """Labels, tables, metric, responses, guide, frame, band, noise and accessors.""" stored = resume(written).metadata - assert stored.schema_version == SCHEMA_VERSION == 2 + assert stored.schema_version == SCHEMA_VERSION == 3 modal = stored.modal assert modal is not None assert set(modal) == { @@ -271,7 +271,7 @@ def test_the_printed_block_prints_every_modal_identity_field_with_its_value(writ assert value in block, (line, value) # The long `model_id` itself is on the record, and not in the printed row. assert str(at(modal, "tables", "model_id")) not in block - assert "schema 2" in block + assert "schema 3" in block # Twelve rows for thirteen fields: the basis row carries `n_components` too. printed = {key for key, _ in stored._describe()[1]} assert len(printed & MODAL_ROWS) == len(MODAL_ROWS) @@ -638,7 +638,7 @@ def test_a_scan_point_writes_its_spool_and_reads_the_identity_back( results = capture.results() assert float(results["reduced/spooled_flux"]) > 0.0 stored = resume(capture.custom_path).metadata - assert stored.schema_version == SCHEMA_VERSION == 2 + assert stored.schema_version == SCHEMA_VERSION == 3 assert stored.modal is not None assert at(stored.modal, "basis") == "general_modal" assert [one.label for one in stored.modes] != [None] diff --git a/tests/output/test_scan.py b/tests/output/test_scan.py index eb1bc8633..19992fa7e 100644 --- a/tests/output/test_scan.py +++ b/tests/output/test_scan.py @@ -23,7 +23,7 @@ from solprop.output._file_lock import lock, unlock from solprop.output._scan_lock import lock_path, read_scan from solprop.output.hdf5 import _dataset, _group -from solprop.output.metadata import _example_metadata +from solprop.output.metadata import SCHEMA_VERSION, _example_metadata from solprop.output.scan import PointResult, ScanWriter, _hash, _json, read_point from solprop.processing.scans import ScanOutput from solprop.processing.scans import open as open_scan @@ -298,7 +298,7 @@ def test_scan_of_another_output_schema_is_refused_by_every_route(tmp_path): status = np.asarray(_dataset(file, "points/status").asstr()[...]) assert status.tolist() == ["done", "failed", "todo"] # The point of this release's schema is still read alone. - assert cast(Mapping, read_point(path, 0)["metadata"])["schema_version"] == 2 + assert cast(Mapping, read_point(path, 0)["metadata"])["schema_version"] == 3 def test_scan_of_this_output_schema_reads_as_before(tmp_path): @@ -308,7 +308,7 @@ def test_scan_of_this_output_schema_reads_as_before(tmp_path): writer.commit(result(0)) writer.commit(result(1)) writer.close() - _declare_schema(path, 0, 2) + _declare_schema(path, 0, SCHEMA_VERSION) reader = ScanOutput(path) assert reader.status.tolist() == ["done", "done"] assert reader.as_grid("stats/energy").shape == (2, 3, 2) diff --git a/tests/output/test_schema.py b/tests/output/test_schema.py index 648ce5cdf..3f2e086ad 100644 --- a/tests/output/test_schema.py +++ b/tests/output/test_schema.py @@ -547,11 +547,11 @@ def test_metadata_hash_changes_with_any_physics_field( def test_the_output_schema_is_the_current_one(envelope_grid, stepper): - assert SCHEMA_VERSION == 2 + assert SCHEMA_VERSION == 3 metadata = make_metadata(envelope_grid, stepper) - assert metadata.schema_version == 2 - assert metadata.hashed()["schema_version"] == 2 - assert json.loads(metadata.to_json())["schema_version"] == 2 + assert metadata.schema_version == 3 + assert metadata.hashed()["schema_version"] == 3 + assert json.loads(metadata.to_json())["schema_version"] == 3 @pytest.mark.parametrize( @@ -567,7 +567,7 @@ def test_a_record_of_another_output_schema_is_refused_by_name( # file is not read either. A boolean is not an integer, whatever it equals. refusal = ( rf"schema_version {re.escape(repr(declared))}, and this release of solprop " - r"reads output schema 2 only: .*earlier release.*development version.*" + r"reads output schema 3 only: .*earlier release.*development version.*" r"later release.*" r"solprop version that wrote it" ) @@ -1032,7 +1032,7 @@ def test_raman_tensor_identity_survives_metadata_round_trip_and_controls_resume( assert original.configuration_hash() != other.configuration_hash() with pytest.raises(OutputError, match=r"configuration\.raman"): resolve_metadata_write(restored, other) - assert original.schema_version == 2 + assert original.schema_version == 3 def test_spatial_metadata_names_its_flux_model(envelope_grid, stepper): diff --git a/tests/parity_ladder.py b/tests/parity_ladder.py index f0c30d36a..450d10fa3 100644 --- a/tests/parity_ladder.py +++ b/tests/parity_ladder.py @@ -263,7 +263,7 @@ def masked_launch(problem: PropagationProblem) -> np.ndarray: the raw launch would compare two inputs on a launch with content outside the band (issue #824); the mask is idempotent, so native loads the same field. """ - return apply_hard_mask(problem.initial.field0[None], problem.grids.active_mask)[0] + return apply_hard_mask(problem.initial.field0, problem.grids.active_mask) def source(backend_class: type, built: Built, *, z: float = 0.0) -> np.ndarray: @@ -279,7 +279,7 @@ def source(backend_class: type, built: Built, *, z: float = 0.0) -> np.ndarray: if backend.name == "native": return native_source(backend, launch, z) backend.set_initial(launch, None) - return np.array(backend._adams_lab_source(launch[None], z)) + return np.array(backend._adams_lab_source(launch, z)) finally: backend.close() @@ -345,7 +345,7 @@ def projected(backend_class: type, built: Built) -> list[np.ndarray | None]: # The launch native loads (:func:`masked_launch`). launch = masked_launch(problem) backend.set_initial(launch, None) - backend._adams_lab_source(launch[None], 0.0) + backend._adams_lab_source(launch, 0.0) coarse = backend._last_coarse return [ forward(np.asarray(backend._last_projected), axis=-1), diff --git a/tests/processing/test_mixtures.py b/tests/processing/test_mixtures.py index 3bb32356e..e5f4c6d6a 100644 --- a/tests/processing/test_mixtures.py +++ b/tests/processing/test_mixtures.py @@ -201,7 +201,7 @@ def test_saved_record_contains_ordered_species_and_complete_optics(case, outputs simulation, _ = case memory, reloaded = outputs for output in (memory, reloaded): - assert output.metadata.schema_version == 2 + assert output.metadata.schema_version == 3 material = output.metadata.configuration["medium"] assert material["labels"] == ["population-0", "population-1"] assert ( diff --git a/tests/processing/test_planes.py b/tests/processing/test_planes.py index 6e1547333..3362053b7 100644 --- a/tests/processing/test_planes.py +++ b/tests/processing/test_planes.py @@ -154,7 +154,7 @@ def test_the_record_survives_a_file(capillary_run, tmp_path): simulation.run(path, progress="none") with open_file(path) as stored: assert stored.metadata is not None and memory.metadata is not None - assert stored.metadata.schema_version == 2 + assert stored.metadata.schema_version == 3 assert stored.metadata.reference == memory.metadata.reference assert ( stored.metadata.configuration_hash() == memory.metadata.configuration_hash() diff --git a/tests/sessions/runs.py b/tests/sessions/runs.py index 75a434561..70d59b535 100644 --- a/tests/sessions/runs.py +++ b/tests/sessions/runs.py @@ -823,7 +823,7 @@ def native_source(backend: Any, field: np.ndarray, z: float = 0.0) -> np.ndarray Returns ------- numpy.ndarray - The source, shape ``(1, n_modes, n_pol, n_omega)`` as the reference's. + The source, shape ``(n_modes, n_pol, n_omega)`` as the reference's. """ from solprop_native._core import ( # noqa: PLC0415 # pyright: ignore[reportMissingImports, reportMissingModuleSource] _testing, @@ -831,7 +831,7 @@ def native_source(backend: Any, field: np.ndarray, z: float = 0.0) -> np.ndarray state = load_native(backend, field) native_trial(backend, z, 0.0) - return np.array(_testing.stage_to_host(state, 0)).reshape(1, *np.shape(field)) + return np.array(_testing.stage_to_host(state, 0)).reshape(np.shape(field)) def bind_native_monitor( @@ -1112,7 +1112,7 @@ def native_exponential_trial( if reference is not None and reference.active: # (Im sigma, Re f_0, Im f_0, ...), f_t = exp(sigma (1 - theta_t)), over the # rows read; sigma is the reference element's chord over the step (SEP 35). - shift = float(np.imag(reference.chord.exponent(z, z + dz))[0]) + shift = float(np.imag(reference.chord.exponent(z, z + dz))) spans = 1.0 - np.asarray(nodes[:rows], dtype=np.float64) carrier = np.empty(1 + 2 * rows, dtype=np.float64) carrier[0] = shift diff --git a/tests/sessions/test_native_load.py b/tests/sessions/test_native_load.py index 5dbc05256..8857fcf5d 100644 --- a/tests/sessions/test_native_load.py +++ b/tests/sessions/test_native_load.py @@ -59,7 +59,7 @@ def test_an_input_draw_is_added_at_the_launch_alone(): host = cast("Any", reference) assert loaded.floor is not None and loaded.noise is None assert np.array_equal(loaded.floor, host._input_floor) - expected = np.asarray(field)[np.newaxis] + loaded.floor + expected = np.asarray(field) + loaded.floor assert np.array_equal(loaded.field, expected.reshape(-1)) assert loaded.clean_columns is not None assert np.array_equal(loaded.clean_columns, host._clean_columns) diff --git a/tests/sessions/test_native_session.py b/tests/sessions/test_native_session.py index 855abe305..3a9531106 100644 --- a/tests/sessions/test_native_session.py +++ b/tests/sessions/test_native_session.py @@ -42,6 +42,7 @@ trace, window, ) +from solprop.backends import records from solprop.backends.events import ( ObservationReady, Outcome, @@ -645,8 +646,8 @@ def test_a_full_chunk_buffer_yields_and_changes_nothing(monkeypatch): problem, run_request = _case(rtol=1e-12, step_planes=0, period=1, dz_init=1e-2) request = run_request baseline = drive(_native_session(problem, request), export=CONSUMER) - monkeypatch.setattr(native_session, "ATTEMPT_CAPACITY", 3) - monkeypatch.setattr(native_session, "ROW_CAPACITY", 2) + monkeypatch.setattr(records, "ATTEMPT_CAPACITY", 3) + monkeypatch.setattr(records, "ROW_CAPACITY", 2) run = drive(_native_session(problem, request), export=CONSUMER) assert_same_run(run, baseline) chunks = [ diff --git a/tests/sessions/test_native_session_carrier_table.py b/tests/sessions/test_native_session_carrier_table.py index 9cade4848..0d41e584c 100644 --- a/tests/sessions/test_native_session_carrier_table.py +++ b/tests/sessions/test_native_session_carrier_table.py @@ -340,7 +340,7 @@ def test_the_chord_is_a_varying_table_the_session_admits(): assert reference.chord is not None and reference.chord.table is not None assert reference.chord.table.coefficients.shape[0] > 1 rates = [ - float(np.imag(reference.chord.exponent(z, z + 1e-4))[0]) / 1e-4 + float(np.imag(reference.chord.exponent(z, z + 1e-4))) / 1e-4 for z in (0.0, LENGTH - 1e-4) ] assert abs(rates[0] - rates[1]) > 1e-6 * abs(rates[0]) diff --git a/tests/sessions/test_native_session_history.py b/tests/sessions/test_native_session_history.py index f954b8d5e..b0e2420de 100644 --- a/tests/sessions/test_native_session_history.py +++ b/tests/sessions/test_native_session_history.py @@ -500,8 +500,8 @@ def _armed(monkeypatch, stage: str, after: int, category: str) -> None: march = native_backend._NativeAdamsMarch configure = march._configure - def arming(self) -> None: - configure(self) + def arming(self, *args: Any) -> None: + configure(self, *args) _testing.adams_march_fault(self._backend._native_state, stage, after, category) monkeypatch.setattr(march, "_configure", arming) diff --git a/tests/sessions/test_split_collar_rules.py b/tests/sessions/test_split_collar_rules.py new file mode 100644 index 000000000..7c90a10a3 --- /dev/null +++ b/tests/sessions/test_split_collar_rules.py @@ -0,0 +1,376 @@ +# SPDX-FileCopyrightText: 2026 John C. Travers and contributors +# +# SPDX-License-Identifier: GPL-3.0-or-later + +"""The split collar's sums across a session: SEP 52 section 5.2's rules, pinned. + +Under ``absorber_scheme="split_step"`` each accepted step's finish reduces the field +after its first band mask and before the split: what the split takes, and the power it +finds halfway through its factor, which the guards are judged on (SEP 52 section 4, +``BoundaryMonitor.split_sums``). Today the reference keeps those sums on its driver +(``_finish`` into ``_split_removed`` and ``_split_found``, drained by +``_boundary_energies``, reset by ``_bind_monitor``) and native in its state and session. +SEP 52 phase 3 moves their ownership into the session; these tests hold the rules it +must keep, through sessions on both backends, before anything moves: + +- the removal adds over every accepted step since the previous reading, and the next + boundary reading drains it; +- the occupancies a reading judges are those the latest accepted step found; +- a rejected trial and a probe trial contribute nothing, and a dense observation cannot, + since it is refused beside the split collar; +- loading a state empties the sums, and a monitor attached at a launch or a resume + starts from zero. + +The case is the corpus's walking fibre (``backend_corpus_cases.walking_fibre``): ten +fixed steps on a grid declared a hundred times longer than the fibre, so that a quiet +window is read about every fourth step and at the saved planes, and the split takes +some of the pulse at every step. Saving a plane at every step edge reads every step, +which gives each step's removal and occupancies; the run is otherwise the same discrete +march. Comparisons are to :func:`march_bounds.reading_tolerance` at the matched march's +budget (:func:`march_bounds.simulation_bound`), and each claim is shown non-vacuous: +the removal or occupancy it would add or confuse is more than a hundred tolerances. +``tests/test_monitor.py::test_a_deep_collar_reads_no_removal_from_residue`` holds the +measurement point itself. +""" + +from __future__ import annotations + +import dataclasses +import functools +from collections.abc import Mapping +from typing import Any + +import numpy as np +import pytest + +from backend_corpus_cases import FIBRE, FIXED_ADAMS, STEPS, walking_fibre +from march_bounds import assert_record_within, reading_tolerance, simulation_bound +from sessions.executors import IDENTITY +from solprop.backends.events import ObservationReady, Terminal +from solprop.backends.registry import available, select +from solprop.backends.session import Start +from solprop.backends.sessions import start_session +from solprop.driver import Simulation +from solprop.exceptions import TranslationError +from solprop.output.store import PlaneStore +from solprop.stepper import FixedSchedule, LowStorageRungeKutta, RungeKutta + +pytestmark = pytest.mark.medium + +BACKENDS = [ + "numpy_ref", + pytest.param( + "native", + marks=pytest.mark.skipif( + "native" not in available(), reason="native is not installed" + ), + ), +] + +MARCHES = { + "rk": RungeKutta(), + "adams": FIXED_ADAMS, + "low-storage": LowStorageRungeKutta(), +} +"""The fixed-schedule marches, each of which finishes its accepted steps itself.""" + +EVERY = tuple(FIBRE * step / STEPS for step in range(1, STEPS + 1)) +"""A save plane at every step edge: every step's state is read.""" + +DETECTION = 100.0 +"""How many tolerances what a rule rules out must exceed, for the test to see it.""" + + +def _backend(simulation: Simulation) -> tuple[Any, Any, Any]: + """Return a simulation's problem, its request with its monitor, and a backend.""" + problem = simulation.translate() + request = simulation._run_request() + backend = select( + simulation.settings.backend, + problem, + request, + grid=simulation.grid, + modes=simulation.modes, + profiles=simulation.profiles, + ) + return problem, request, backend + + +def _watched(simulation: Simulation, request: Any, start_plane: int = 0) -> Any: + """Return the request with a fresh boundary monitor, as a run attaches one.""" + return request.for_run( + monitor=simulation._boundary_monitor(), start_plane=start_plane + ) + + +def _drive( + backend: Any, + problem: Any, + request: Any, + start: Start, + *, + advances: int | None = None, + planes: int | None = None, +) -> tuple[dict[str, Any], list[tuple[float, np.ndarray]]]: + """Drive a session an attempt at a time; return its monitor record and planes. + + The session stops after `advances` yields (the launch's observation is the first) + or once it observed `planes` save planes, or at its end; the observations' fields + are exported as the session holds them. + """ + session = start_session(backend, problem, request, start, owns=False) + observed: list[tuple[float, np.ndarray]] = [] + try: + count = 0 + while not session.finished: + batch = session.advance(work_limit=1) + for event in batch.events: + if isinstance(event, Terminal) and event.error is not None: + raise event.error + if isinstance(event, ObservationReady) and event.plane_index >= 0: + exported = session.export(event.ticket, IDENTITY) + observed.append((float(event.z), np.array(exported))) + count += 1 + if (advances is not None and count >= advances) or ( + planes is not None and len(observed) >= planes + ): + break + session.acknowledge(batch.receipt) + return dict(session.monitor_record() or {}), observed + finally: + session.close() + + +def _record(simulation: Simulation) -> dict[str, Any]: + """Return the monitor record of a whole run of `simulation` on a fresh backend.""" + problem, request, backend = _backend(simulation) + try: + record, _ = _drive( + backend, problem, _watched(simulation, request), Start.entrance(problem) + ) + finally: + backend.close() + return record + + +@functools.cache +def _runs(backend: str, march: str) -> tuple[dict[str, Any], dict[str, Any], float]: + """Return a sparsely read run's record, the every-step run's, and their budget.""" + sparse = walking_fibre(backend, MARCHES[march]) + return ( + _record(sparse), + _record(walking_fibre(backend, MARCHES[march], planes=EVERY)), + simulation_bound(sparse), + ) + + +def _per_step(every: Mapping[str, Any]) -> np.ndarray: + """Return each step's removal from a run read at every step's plane.""" + rows = every["planes"] + # The run carries no input noise, so the removal above the floor is the whole. + return np.diff(np.asarray(rows["removed_fraction"], dtype=np.float64)) + + +@pytest.mark.parametrize("march", sorted(MARCHES)) +@pytest.mark.parametrize("backend", BACKENDS) +def test_the_removal_adds_every_step_since_the_last_reading(backend, march): + sparse, every, bound = _runs(backend, march) + rows = every["planes"] + # Every step of the every-step run is a plane read when it was reached... + assert len(rows["z"]) == STEPS + 1 + assert None not in rows["guard_high"] + # ...while the sparse run read its state at fewer than half its steps (its checked + # steps and at most its two planes'), so some readings drained several steps. + assert sparse["steps"] + 2 < STEPS // 2 + assert sparse["skipped"] > 0 + total = float(every["raw_removed_fraction"]) + tolerance = reading_tolerance(total, bound, STEPS) + steps = _per_step(every) + # A step dropped, or a drained step counted again, moves the total by a step's + # removal: at least a hundred tolerances. + assert np.all(steps > DETECTION * tolerance), (steps.min(), tolerance) + assert abs(float(sparse["raw_removed_fraction"]) - total) <= tolerance + # The middle plane's reading drained the five steps before it. + middle = STEPS // 2 + row = float(sparse["planes"]["removed_fraction"][1]) + expected = float(rows["removed_fraction"][middle]) + assert abs(row - expected) <= reading_tolerance(expected, bound, middle) + + +@pytest.mark.parametrize("march", sorted(MARCHES)) +@pytest.mark.parametrize("backend", BACKENDS) +def test_a_reading_judges_the_latest_step_s_occupancies(backend, march): + sparse, every, bound = _runs(backend, march) + rows, last = every["planes"], sparse["planes"] + for name in ("guard_low", "guard_high", "collar_occupancy"): + series = np.asarray(rows[name], dtype=np.float64) + expected = float(series[-1]) + tolerance = reading_tolerance(expected, bound, 1) + # The walk fills the collar and the guards step by step, so the steps since + # the middle plane found less than the last did: the first of them, or their + # sum, is no reading of the last. + late = series[STEPS // 2 + 1 :] + assert np.all(np.diff(late) > DETECTION * tolerance), (name, late) + assert abs(float(last[name][-1]) - expected) <= tolerance, name + + +@pytest.mark.parametrize("backend", BACKENDS) +def test_a_rejected_trial_adds_nothing(backend): + adaptive = walking_fibre(backend, RungeKutta(), scheduled=False) + output = adaptive.run(progress="none") + accepted = np.asarray(output["step_history/accepted"], dtype=bool) + assert not np.all(accepted), "the adaptive run rejected no trial" + assert isinstance(output, PlaneStore) + outcome = output.outcome + assert outcome is not None and outcome.execution is not None + found = outcome.execution["boundary_monitor"] + assert isinstance(found, dict) + # The same accepted steps, replayed with no rejection and read at every step. + schedule = FixedSchedule.from_history( + output["step_history/z"], output["step_history/h"], accepted + ) + replay = walking_fibre(backend, RungeKutta()) + replay.settings = dataclasses.replace( + replay.settings, + schedule=schedule, + save_planes=np.asarray(schedule.edges[1:]), + dz_max=schedule.max_step, + ) + every = _record(replay) + bound = simulation_bound(replay) + total = float(every["raw_removed_fraction"]) + tolerance = reading_tolerance(total, bound, schedule.n_steps) + # A rejected trial is longer than the step taken after it from the same state, so + # it would take more than that step's removal: at least a hundred tolerances. + assert np.all(_per_step(every) > DETECTION * tolerance) + assert abs(float(found["raw_removed_fraction"]) - total) <= tolerance + + +@pytest.mark.parametrize("backend", BACKENDS) +def test_a_dense_observation_is_refused_beside_the_split_collar(backend): + # Today the rule holds because the combination cannot run: a plane inside a step + # would be read from a state the split never finished. Translation refuses it on + # both backends alike. + simulation = walking_fibre( + backend, RungeKutta(), scheduled=False, interpolated=True + ) + with pytest.raises(TranslationError, match="dense observation interpolates"): + simulation.translate() + + +@pytest.mark.parametrize("march", ["adams", "rk"]) +@pytest.mark.parametrize("backend", BACKENDS) +def test_a_probe_trial_adds_nothing(backend, march): + # The low-storage march's probe closes its backend, so no session can follow it. + simulation = walking_fibre(backend, MARCHES[march]) + problem, request, own = _backend(simulation) + try: + probe = own.timing_probe(Start.entrance(problem)) + # Each trial is the schedule's first step from the launch, finished as a kept + # step: had it counted, the run's first reading would carry one step more. + probe.trial() + probe.field() + probe.trial(float(problem.initial.z), float(request.stepper.dz_init)) + probe.field() + after, _ = _drive( + own, problem, _watched(simulation, request), Start.entrance(problem) + ) + finally: + own.close() + _, every, bound = _runs(backend, march) + fresh = _record(walking_fibre(backend, MARCHES[march])) + tolerance = reading_tolerance(float(fresh["raw_removed_fraction"]), bound, STEPS) + assert _per_step(every)[0] > DETECTION * tolerance + assert_record_within(after, fresh, bound, STEPS, what="after a probe") + + +def _dirty(backend: Any, simulation: Simulation, problem: Any, request: Any) -> None: + """Leave three accepted steps' removal undrained in a backend's sums. + + A session is driven through three steps, none of them read (the grid's interval + reaches the fourth), and closed there. + """ + partial, _ = _drive( + backend, + problem, + _watched(simulation, request), + Start.entrance(problem), + advances=4, + ) + assert partial["steps"] == 0, "the interrupted session read a step" + assert partial["raw_removed_fraction"] == 0.0 + + +@pytest.mark.parametrize("march", sorted(MARCHES)) +@pytest.mark.parametrize("backend", BACKENDS) +def test_a_launch_starts_the_sums_from_zero(backend, march): + simulation = walking_fibre(backend, MARCHES[march]) + problem, request, own = _backend(simulation) + try: + _dirty(own, simulation, problem, request) + after, _ = _drive( + own, problem, _watched(simulation, request), Start.entrance(problem) + ) + finally: + own.close() + _, every, bound = _runs(backend, march) + fresh = _record(walking_fibre(backend, MARCHES[march])) + tolerance = reading_tolerance(float(fresh["raw_removed_fraction"]), bound, STEPS) + # Carried over, the three steps would add their removal to the first reading. + assert float(np.sum(_per_step(every)[:3])) > DETECTION * tolerance + assert_record_within(after, fresh, bound, STEPS, what="after an interrupted run") + + +@pytest.mark.parametrize("march", sorted(MARCHES)) +@pytest.mark.parametrize("backend", BACKENDS) +def test_a_resume_starts_the_sums_from_zero(backend, march): + simulation = walking_fibre(backend, MARCHES[march]) + # The state and the monitor's record at the middle plane, as a file keeps them. + problem, request, own = _backend(simulation) + try: + middle, observed = _drive( + own, + problem, + _watched(simulation, request), + Start.entrance(problem), + planes=1, + ) + finally: + own.close() + [(z, field)] = observed + + def resumed() -> Start: + """Return the start at the middle plane, the monitor's record continued.""" + return Start( + field=field.copy(), + z=z, + plane_index=1, + monitor={"record": middle, "plane": True, "unwatched": []}, + plane_pending=False, + resume=True, + ) + + problem, request, own = _backend(simulation) + try: + fresh, _ = _drive(own, problem, _watched(simulation, request, 1), resumed()) + finally: + own.close() + problem, request, own = _backend(simulation) + try: + _dirty(own, simulation, problem, request) + after, _ = _drive(own, problem, _watched(simulation, request, 1), resumed()) + finally: + own.close() + _, every, bound = _runs(backend, march) + tolerance = reading_tolerance(float(fresh["raw_removed_fraction"]), bound, STEPS) + assert float(np.sum(_per_step(every)[:3])) > DETECTION * tolerance + assert fresh["resumed_at"] == [z] + assert_record_within(after, fresh, bound, STEPS, what="a resume after a run") + if march != "adams": + # A march with no history continues the uninterrupted run's discrete steps: + # the resumed record adds only the steps after the resume to the record it + # continues. + total = float(every["raw_removed_fraction"]) + assert abs(float(fresh["raw_removed_fraction"]) - total) <= reading_tolerance( + total, bound, STEPS + ) diff --git a/tests/test_cartesian_precision.py b/tests/test_cartesian_precision.py index 334d566de..44f0a81dc 100644 --- a/tests/test_cartesian_precision.py +++ b/tests/test_cartesian_precision.py @@ -76,7 +76,7 @@ def test_cartesian_square_and_prefactor_meet_the_predeclared_certificate( # frame) or of the value itself (the prefactor). Fixed at 16 eps of each. wavenumber = float(np.sqrt(np.max(np.abs(high)))) np.testing.assert_allclose( - operator[0, :, 0], + operator[:, 0], dispersion.operator(space), rtol=0, atol=16 * epsilon * wavenumber, diff --git a/tests/test_design_check.py b/tests/test_design_check.py index de492177b..2f0048498 100644 --- a/tests/test_design_check.py +++ b/tests/test_design_check.py @@ -352,11 +352,10 @@ def test_a_refined_estimate_that_disagrees_is_inconclusive() -> None: SpatialAbsorber(kappa=0.45 * top), space, dispersion, launch, 2e-3 ) assert design.collar_rate is not None - # The check reads the launch behind a leading axis of one, as checked_design does. result = design_check( space, 0.1 * design.collar_rate, - launch[np.newaxis], + launch, dispersion, 2e-3, min(design.width), @@ -369,9 +368,9 @@ def test_a_refined_estimate_that_disagrees_is_inconclusive() -> None: def test_a_polarisation_that_carries_nothing_changes_nothing() -> None: space, dispersion, launch = _line(68, 5e-3, 20e-6) rate = 1e3 * (1 - np.asarray(space.window)) - one = design_check(space, rate, launch[np.newaxis], dispersion, 5e-3, 100e-6) + one = design_check(space, rate, launch, dispersion, 5e-3, 100e-6) both = np.concatenate([launch, np.zeros_like(launch)], axis=1) - two = design_check(space, rate, both[np.newaxis], dispersion, 5e-3, 100e-6) + two = design_check(space, rate, both, dispersion, 5e-3, 100e-6) assert two.returned == pytest.approx(one.returned, rel=1e-12) assert two.status == one.status == "passed" @@ -383,9 +382,7 @@ def test_a_launch_with_nothing_in_the_requested_band_returns_nothing() -> None: band = dispersion.grid.inband_mask & dispersion.grid.active_mask outside = np.where(band, 0.0, 1.0) * np.abs(launch).max() rate = 1e3 * (1 - np.asarray(space.window)) - result = design_check( - space, rate, (launch * 0 + outside)[np.newaxis], dispersion, 5e-3, 1e-4 - ) + result = design_check(space, rate, launch * 0 + outside, dispersion, 5e-3, 1e-4) assert result.status == "passed" assert result.returned == 0.0 @@ -418,10 +415,10 @@ def beam(axis: np.ndarray) -> np.ndarray: assert trial.spectral_weights.size == 2 ** (2 if kind == "box" else 1) * len( np.asarray(space.spectral_weights) ) - launch = columns.reshape(1, -1, 1, 1) + launch = columns.reshape(-1, 1, 1) carried = _relaunched(space, trial, launch) - before = float(np.sum(space.spectral_weights * np.abs(launch[0, :, 0, 0]) ** 2)) - after = float(np.sum(trial.spectral_weights * np.abs(carried[0, :, 0, 0]) ** 2)) + before = float(np.sum(space.spectral_weights * np.abs(launch[:, 0, 0]) ** 2)) + after = float(np.sum(trial.spectral_weights * np.abs(carried[:, 0, 0]) ** 2)) assert after == pytest.approx(before, rel=1e-12) diff --git a/tests/test_display.py b/tests/test_display.py index 7a9772a91..858764bfb 100644 --- a/tests/test_display.py +++ b/tests/test_display.py @@ -670,7 +670,8 @@ def test_the_run_record_is_what_it_was_before_the_printing(examples): metadata = quiet(examples["Simulation"].metadata) assert not metadata.materials.empty hashed: dict[str, Any] = metadata.hashed() - # Output schema 2 adds a field only to `DataPulse`, which this run does not hold. + # Output schemas 2 and 3 change only `DataPulse` and a march snapshot, which this + # run does not hold. hashed["schema_version"] = 1 for later in ("initial_z", "tdse", "quadrature"): del hashed[later] @@ -704,7 +705,7 @@ def test_the_run_record_names_its_power_convention(examples): # The amplitude label is part of what a file means, so the block prints it beside # the spectral convention and the output schema. metadata = quiet(examples["Simulation"].metadata) - assert "optics spectra, poynting amplitudes, schema 2" in str(metadata) + assert "optics spectra, poynting amplitudes, schema 3" in str(metadata) def test_a_mode_built_over_a_mixture_names_its_fill(): diff --git a/tests/test_identity.py b/tests/test_identity.py index 22400625c..9d8c71ea1 100644 --- a/tests/test_identity.py +++ b/tests/test_identity.py @@ -61,7 +61,7 @@ def test_a_numpy_scalar_is_written_as_the_number_it_holds(): [ ( "metadata", - "9e1f945c0ab5246cc011e9bb713242ac19d8fb4e76b58ba4e95f23af58fc6964", + "7cf89527a88bf97f031f79b4447ec19cb89639e78db71e9c930b8fbd7717d8fe", ), ("scan", "80f9c7ee2c67bb3246215ab42ce1d49fa0cb9f93e774cd9b2d543229b4ae335b"), ("basis", "d7b11decb56fec6a504280a7c9e26f83c5f33d84fa111aa9a9b27f9e87481587"), diff --git a/tests/test_mixture_baseline/test_m2_problem_keeps_every_recorded_array.npz b/tests/test_mixture_baseline/test_m2_problem_keeps_every_recorded_array.npz index bfa58f69f..1267363ca 100644 Binary files a/tests/test_mixture_baseline/test_m2_problem_keeps_every_recorded_array.npz and b/tests/test_mixture_baseline/test_m2_problem_keeps_every_recorded_array.npz differ diff --git a/tests/test_monitor.py b/tests/test_monitor.py index c39296d57..2eb0d2c94 100644 --- a/tests/test_monitor.py +++ b/tests/test_monitor.py @@ -711,7 +711,7 @@ def pulse(centre: float) -> np.ndarray: return np.exp(-(((t - centre) / 10e-15) ** 2)).astype(complex) state = math.sqrt(1.0 - share) * pulse(0.0) + math.sqrt(share) * pulse(at * t.max()) - return grid, np.fft.fft(state)[None, None, None, :] + return grid, np.fft.fft(state)[None, None, :] def _launch_with(share: float, at: float) -> BoundaryMonitor: @@ -788,16 +788,16 @@ def test_a_field_absorbed_to_nothing_is_not_judged() -> None: grid = EnvelopeGrid.from_limits(LENGTH, 1e-6, (0.8e-6, 1.25e-6), TRANGE) pulse = np.exp(-((grid.t / 50e-15) ** 2)).astype(complex) monitor = BoundaryMonitor.build(grid, None, BoundaryCheck()) - monitor.launch(monitor.energies(np.fft.fft(pulse)[None, None, None, :]), 0.0) + monitor.launch(monitor.energies(np.fft.fft(pulse)[None, None, :]), 0.0) residue = np.full(grid.n_t, 1e-12, dtype=complex) - monitor.step(monitor.energies(np.fft.fft(residue)[None, None, None, :]), 0.1) + monitor.step(monitor.energies(np.fft.fft(residue)[None, None, :]), 0.1) assert monitor.summary()["guard_low_max"] == 0.0 def test_a_launch_with_no_light_is_not_judged() -> None: grid = EnvelopeGrid.from_limits(LENGTH, 1e-6, (0.8e-6, 1.25e-6), TRANGE) monitor = BoundaryMonitor.build(grid, None, BoundaryCheck()) - dark = np.zeros((1, 1, 1, grid.n_t), dtype=complex) + dark = np.zeros((1, 1, grid.n_t), dtype=complex) monitor.launch(monitor.energies(dark), 0.0) monitor.step(monitor.energies(dark), 0.1) summary = monitor.summary() @@ -814,9 +814,9 @@ def _split_monitor( check = BoundaryCheck(budget=1.0, stop_factor=1.0) monitor = BoundaryMonitor.build(grid, rate, check, split=True) pulse = np.exp(-((grid.t / 50e-15) ** 2)).astype(complex) - spectrum = np.fft.fft(pulse)[None, None, None, :] + spectrum = np.fft.fft(pulse)[None, None, :] rng = np.random.default_rng(5) - draw = 1e-3 * np.exp(2j * np.pi * rng.random(grid.n_t))[None, None, None, :] + draw = 1e-3 * np.exp(2j * np.pi * rng.random(grid.n_t))[None, None, :] reading = monitor.energies(spectrum) monitor.launch(reading, 0.0, floor=draw if floor else None) # A step's reading carries what its split found: a centred pulse, clear of the @@ -891,7 +891,7 @@ def test_the_in_stage_scheme_ignores_what_a_split_would_have_found() -> None: grid, np.full(grid.n_t, 10.0), BoundaryCheck(budget=1.0, stop_factor=1.0) ) pulse = np.exp(-((grid.t / 50e-15) ** 2)).astype(complex) - reading = monitor.energies(np.fft.fft(pulse)[None, None, None, :]) + reading = monitor.energies(np.fft.fft(pulse)[None, None, :]) monitor.launch(reading, 0.0) found = reading.total * np.array([1.0, 0.0, 0.1, 0.2]) monitor.step(dataclasses.replace(reading, found=found), 0.1) @@ -929,7 +929,7 @@ def test_a_plane_is_recorded_once() -> None: grid = EnvelopeGrid.from_limits(LENGTH, 1e-6, (0.8e-6, 1.25e-6), TRANGE) monitor = BoundaryMonitor.build(grid, None, BoundaryCheck()) pulse = np.exp(-((grid.t / 50e-15) ** 2)).astype(complex) - monitor.launch(monitor.energies(np.fft.fft(pulse)[None, None, None, :]), 0.0) + monitor.launch(monitor.energies(np.fft.fft(pulse)[None, None, :]), 0.0) monitor.plane(0.0) monitor.plane(0.0) planes = monitor.summary()["planes"] @@ -978,7 +978,7 @@ def test_with_noise_the_removed_fraction_is_still_the_signal_lost() -> None: def test_a_launch_of_the_floor_alone_is_not_judged() -> None: grid = EnvelopeGrid.from_limits(LENGTH, 1e-6, (0.8e-6, 1.25e-6), TRANGE) rng = np.random.default_rng(3) - draw = np.exp(2j * np.pi * rng.random(grid.n_t))[None, None, None, :] + draw = np.exp(2j * np.pi * rng.random(grid.n_t))[None, None, :] monitor = BoundaryMonitor.build(grid, None, BoundaryCheck()) with warnings.catch_warnings(): warnings.simplefilter("error", BoundaryWarning) @@ -1121,7 +1121,7 @@ def test_a_check_is_due_every_step_while_light_is_near_an_edge() -> None: centred = np.exp(-((grid.t / 50e-15) ** 2)).astype(complex) edge = np.exp(-(((grid.t - 0.97 * grid.t.max()) / 30e-15) ** 2)).astype(complex) spectrum = lambda samples: monitor.energies( # noqa: E731 - np.fft.fft(samples)[None, None, None, :] + np.fft.fft(samples)[None, None, :] ) assert monitor.checked_at is None monitor.launch(spectrum(centred), 0.0) @@ -1154,7 +1154,7 @@ def test_a_collar_returns_at_once_and_rests_again_only_after_quiet_checks() -> N # warning at this relaxed budget. near = 1e-3 * np.exp(-(((grid.t - 0.93 * grid.t.max()) / 30e-15) ** 2)) spectrum = lambda samples: monitor.energies( # noqa: E731 - np.fft.fft(samples)[None, None, None, :] + np.fft.fft(samples)[None, None, :] ) monitor.launch(spectrum(centred), 0.0) verdicts = [] @@ -1173,7 +1173,7 @@ def test_a_resting_collar_adds_no_removal() -> None: check = BoundaryCheck(budget=1.0, stop_factor=1.0) monitor = BoundaryMonitor.build(grid, rate, check) pulse = np.fft.fft(np.exp(-((grid.t / 50e-15) ** 2)).astype(complex)) - state = pulse[None, None, None, :] + state = pulse[None, None, :] monitor.launch(monitor.energies(state), 0.0) monitor.note_collar(True) monitor.step(monitor.energies(state), 0.1) diff --git a/tests/test_safety_ramp.py b/tests/test_safety_ramp.py index 96c80370b..c092d566b 100644 --- a/tests/test_safety_ramp.py +++ b/tests/test_safety_ramp.py @@ -173,7 +173,7 @@ def test_the_rate_bounds_bound_the_operator_it_forms(evanescent_rate): operator = separable(evanescent_rate=evanescent_rate) rate, _ = separable_values(operator) bounds = rate_bounds(operator) - real = np.real(rate[0, :, 0, :]) + real = np.real(rate[:, 0, :]) assert bounds.loss >= float(np.max(-real)) assert bounds.gain >= float(np.max(real)) assert bounds.gain > 0 # the second bin's medium has gain diff --git a/tests/test_spatial_boundaries.py b/tests/test_spatial_boundaries.py index 941f7d193..8482573ed 100644 --- a/tests/test_spatial_boundaries.py +++ b/tests/test_spatial_boundaries.py @@ -137,7 +137,7 @@ def test_direct_spatial_constructor_derives_a_default_frame(case): @pytest.mark.parametrize( ("changes", "refusal"), [ - ({"schema_version": 3}, "reads output schema 2 only"), + ({"schema_version": 4}, "reads output schema 3 only"), ({"spatial": None}, "requires a spatial geometry record"), ], ) diff --git a/tests/test_split_crossing.py b/tests/test_split_crossing.py index 7fd20f2de..64f0c604f 100644 --- a/tests/test_split_crossing.py +++ b/tests/test_split_crossing.py @@ -149,8 +149,7 @@ def test_the_launch_s_limit_holds_dz_max() -> None: # pulse's, within a bucket's 18 % and the pulse's spectral width. rate = SPLIT_CROSSING * width / crossing.launch_limit assert GRID.n_t * GRID.dt < rate < 2.0 * GRID.n_t * GRID.dt - # The occupancy reads the state behind its private leading axis of one. - limit = crossing.limit(crossing.occupancy(problem.initial.field0[np.newaxis])) + limit = crossing.limit(crossing.occupancy(problem.initial.field0)) assert crossing.launch_limit == limit assert run_request.stepper.dz_max == limit assert run_request.stepper.dz_init <= limit @@ -341,7 +340,7 @@ def test_every_polarisation_is_read() -> None: # the second polarisation for one mode and in the first for the other. omega = np.linspace(1.0e15, 3.0e15, 201) quadratic = -1j * 0.5 * (omega - 2.0e15) ** 2 - beta2 = np.array([[[1e-26, 2e-26], [4e-26, 1e-26]]]) + beta2 = np.array([[1e-26, 2e-26], [4e-26, 1e-26]]) table = constant_phi(beta2[..., None] * quadratic, 1.0) buckets, _ = walk_off_buckets(table, omega, np.ones(201, dtype=bool)) rates = split_bucket_rates()[buckets[:, 150]] # 5e14 rad/s from the reference @@ -378,7 +377,7 @@ def test_a_rate_past_the_fastest_edge_is_counted_at_its_own() -> None: # first bucket takes that rate, so its limit is no longer than its own. buckets = split_rate_buckets(np.array([[1e-6, 1e-12]])) crossing = SplitCrossing(buckets, 1e-13, fastest=1e-6) - occupancy = crossing.occupancy(np.array([[[[1.0, 1.0]]]], dtype=complex)) + occupancy = crossing.occupancy(np.array([[[1.0, 1.0]]], dtype=complex)) assert crossing.limit(occupancy) == pytest.approx(split_crossing_step(1e-6, 1e-13)) @@ -487,7 +486,7 @@ def test_a_split_reading_without_the_occupancy_is_refused() -> None: monitor = simulation._boundary_monitor() assert monitor is not None monitor.crossing = problem.split_crossing - reading = monitor.energies(problem.initial.field0[np.newaxis]) + reading = monitor.energies(problem.initial.field0) with pytest.raises(BackendError, match="occupancy"): monitor.launch(reading, 0.0) @@ -521,11 +520,10 @@ def test_a_coupled_taper_s_channels_walk_at_their_own_rates() -> None: inband = coupled.grids.inband_mask assert np.min(crossing.buckets[2][inband]) < np.min(crossing.buckets[0][inband]) # Light in the third mode alone bounds the cap below light in the first alone. - # The occupancy reads the state behind its private leading axis of one. - launch = np.asarray(coupled.initial.field0)[np.newaxis] + launch = np.asarray(coupled.initial.field0) alone = [np.zeros_like(launch) for _ in range(2)] - alone[0][:, 0] = launch[:, 0] - alone[1][:, 2] = launch[:, 0] + alone[0][0] = launch[0] + alone[1][2] = launch[0] first, third = (crossing.limit(crossing.occupancy(state)) for state in alone) assert third < 0.2 * first diff --git a/tests/translate/test_free.py b/tests/translate/test_free.py index a4d116ed8..5feacc8db 100644 --- a/tests/translate/test_free.py +++ b/tests/translate/test_free.py @@ -44,7 +44,7 @@ def test_free_transform_and_separable_storage(free_inputs): # A chunk is the same elementwise evaluation on fewer columns; a vector loop may # round its tail differently, so it is held to a few roundings of the values. np.testing.assert_allclose( - chunk, full[:, 2:7], rtol=0, atol=gamma(4) * np.max(np.abs(full)) + chunk, full[2:7], rtol=0, atol=gamma(4) * np.max(np.abs(full)) ) np.testing.assert_allclose( selected, prefactor[2:7], rtol=0, atol=gamma(4) * np.max(np.abs(prefactor)) diff --git a/tests/translate/test_linop.py b/tests/translate/test_linop.py index 8a2964080..20d0b6951 100644 --- a/tests/translate/test_linop.py +++ b/tests/translate/test_linop.py @@ -124,7 +124,7 @@ def test_a_constant_operator_is_one_interval_of_degree_one(): assert table.phi_degrees.tolist() == [1] assert table.phi_representation == "constant" assert table.phi_coefficients is not None - assert table.phi_coefficients.shape == (1, 2, 1, 1, 1, 3) + assert table.phi_coefficients.shape == (1, 2, 1, 1, 3) assert table.z_span == (0.0, 2.0 * (1.0 + DEFAULT_MARGIN)) @@ -155,14 +155,14 @@ def test_a_z_constant_mode_takes_the_constant_path(): def test_the_constant_path_broadcasts_over_polarisation(): table = constant_phi(np.array([-1j, -2j]), 1.0, n_pol=2) - assert table.value_shape == (1, 1, 2, 2) + assert table.value_shape == (1, 2, 2) assert table.phi_constant is not None - assert np.array_equal(table.phi_constant[0, 0, 0], table.phi_constant[0, 0, 1]) + assert np.array_equal(table.phi_constant[0, 0], table.phi_constant[0, 1]) def test_the_constant_path_accepts_the_canonical_layout(): - operator = np.zeros((1, 2, 1, 4), dtype=complex) - assert constant_phi(operator, 1.0).value_shape == (1, 2, 1, 4) + operator = np.zeros((2, 1, 4), dtype=complex) + assert constant_phi(operator, 1.0).value_shape == (2, 1, 4) def test_the_unitary_phase_of_a_constant_operator_is_the_imaginary_exponent(): @@ -378,7 +378,7 @@ def test_the_comoving_frame_holds_the_pulse_in_the_window(field_grid, gradient_m def centroid(frame): table = build_phi(gradient_mode, field_grid, frame, z_max=0.2) - state = np.exp(delta_phi(table, 0.0, 0.2)[0, 0, 0]) * start[0, 0] + state = np.exp(delta_phi(table, 0.0, 0.2)[0, 0]) * start[0, 0] signal = fourier.analytic_signal(fourier.inverse(state, kind="field")) power = np.abs(signal) ** 2 return float(np.sum(field_grid.t * power) / np.sum(power)) @@ -435,7 +435,7 @@ def test_every_mode_of_a_set_is_written_in_the_first_mode_s_frame(field_grid): modes = tuple(MarcatiliMode(125e-6, "He", 1.0, m=m) for m in (1, 2, 3)) frame = Frame.comoving(field_grid) sampler = _operator_sampler(modes, field_grid, frame, 1, DEFAULT_LOSS_POLICY, None) - operator = sampler(0.0)[0, :, 0, :] + operator = sampler(0.0)[:, 0, :] active = field_grid.active_mask omega = field_grid.omega[active] shared = frame_quantities(modes[0], frame, 0.0)[1] @@ -479,7 +479,7 @@ def test_an_envelope_set_shares_the_first_mode_s_carrier_phase_too(): assert frame.subtract_beta0 operator = _operator_sampler(modes, grid, frame, 1, DEFAULT_LOSS_POLICY, None)(0.0) carrier = int(np.argmin(abs(grid.omega - grid.omega0))) - columns = operator[0, :, 0, carrier] + columns = operator[:, 0, carrier] # The reference mode is exactly in its own frame at the carrier; the loss is all # that is left of it, and the bound is the round-off of beta ~ 8e6 - beta_0. assert abs(np.imag(columns[0])) < 1e-6 @@ -499,7 +499,7 @@ def test_a_frame_given_both_quantities_is_used_as_it_is(field_grid): fixed = Frame(field_grid.omega0, subtract_beta0=False, beta0=0.0, beta1=0.0) operator = _operator_sampler( modes, field_grid, fixed, 1, DEFAULT_LOSS_POLICY, None - )(0.0)[0, :, 0, :] + )(0.0)[:, 0, :] active = field_grid.active_mask omega = field_grid.omega[active] for index, mode in enumerate(modes): @@ -532,13 +532,13 @@ def test_the_tolerance_is_raised_to_the_measured_round_off_floor( def test_the_round_off_floor_of_an_exact_operator_is_zero(): - constant = np.array([[[[1.0 + 0j, 2.0]]]]) + constant = np.array([[[1.0 + 0j, 2.0]]]) assert _roundoff_floor(lambda z: constant, 1.0) == 0.0 def test_the_round_off_floor_of_a_vanishing_operator_is_zero(): """A probe where the operator is identically zero has no scale, and is skipped.""" - zeros = np.zeros((1, 1, 1, 4), dtype=complex) + zeros = np.zeros((1, 1, 4), dtype=complex) assert _roundoff_floor(lambda z: zeros, 1.0) == 0.0 @@ -681,13 +681,13 @@ def test_a_square_root_vacuum_end_keeps_its_fitted_table(): def test_verify_phi_reports_agreement_for_an_exact_table(): - operator = np.array([[[[-1j, -2j]]]]) + operator = np.array([[[-1j, -2j]]]) table = constant_phi(operator, 1.0) assert verify_phi(table, lambda z: operator, [(0.0, 1.0), (0.2, 0.6)]) < 1e-14 def test_verify_phi_reports_the_absolute_deviation_when_the_operator_vanishes(): - zeros = np.zeros((1, 1, 1, 2), dtype=complex) + zeros = np.zeros((1, 1, 2), dtype=complex) table = constant_phi(zeros, 1.0) assert verify_phi(table, lambda z: zeros, [(0.0, 1.0)]) == 0.0 @@ -784,18 +784,18 @@ def test_the_tables_translation_builds_are_held_and_not_copied_again(): [ ({"phi_breakpoints": np.array([0.0, 0.5, 1.0])}, "phi_breakpoints has"), ( - {"phi_coefficients": np.zeros((2, 2, 1, 1, 1, 3), dtype=complex)}, + {"phi_coefficients": np.zeros((2, 2, 1, 1, 3), dtype=complex)}, "phi_degrees has", ), ({"phi_representation": "constant"}, "phi_constant must be present"), ( - {"phi_constant": np.zeros((1, 1, 1, 3), dtype=complex)}, + {"phi_constant": np.zeros((1, 1, 3), dtype=complex)}, "phi_constant must be present", ), ( { "phi_representation": "constant", - "phi_constant": np.zeros((1, 1, 1, 4), dtype=complex), + "phi_constant": np.zeros((1, 1, 4), dtype=complex), }, "phi_constant has shape", ), @@ -805,7 +805,7 @@ def test_the_group_refuses_inconsistent_members(changes, message): members = { "phi_representation": "table", "phi_breakpoints": np.array([0.0, 1.0]), - "phi_coefficients": np.zeros((1, 2, 1, 1, 1, 3), dtype=complex), + "phi_coefficients": np.zeros((1, 2, 1, 1, 3), dtype=complex), "phi_degrees": np.array([1]), "hard_kinks": np.empty(0), "phi_constant": None, @@ -850,7 +850,7 @@ def test_a_path_shared_by_two_modes_is_resolved_once(field_grid): # table carries the operator of both modes. assert len(paths) == 2 table = build_phi(modes, field_grid, Frame.comoving(field_grid), z_max=0.2, n_pol=1) - assert table.value_shape[1] == 2 + assert table.value_shape[0] == 2 same = declared_paths([modes[0], modes[0]]) assert len(same) == 1 @@ -1019,7 +1019,7 @@ def test_two_modes_of_one_fibre_share_one_parameter_axis(field_grid): assert sorted(varying) == ["density"] table = build_phi(modes, field_grid, Frame.comoving(field_grid), z_max=0.2) assert table.phi_representation == "expansion" - assert table.value_shape[1] == 2 + assert table.value_shape[0] == 2 def test_an_expansion_that_cannot_be_built_is_refused_when_it_is_demanded(field_grid): @@ -1067,8 +1067,8 @@ def test_the_floor_is_measured_in_the_acceptance_tests_own_norm(): small = 1.0 def operator_at(z): - noise = rng.normal(scale=1e-4, size=(1, 1, 1, 2)) - return np.array([[[[large, small]]]]) * (1.0 + 1e-9 * z) + noise + noise = rng.normal(scale=1e-4, size=(1, 1, 2)) + return np.array([[[large, small]]]) * (1.0 + 1e-9 * z) + noise floor = _roundoff_floor(operator_at, 1.0) # The small column's own noise is 1e-4 of itself; the large column's is 1e-8 of diff --git a/tests/translate/test_mixture_expansion.py b/tests/translate/test_mixture_expansion.py index 3ae6dc4ce..5c4776202 100644 --- a/tests/translate/test_mixture_expansion.py +++ b/tests/translate/test_mixture_expansion.py @@ -97,7 +97,7 @@ def test_independent_radial_profiles_use_the_general_table(optical_population): integral - wave_numbers * index_integral ) assert_allclose( - delta_phi(operator, 0.0, grid.length)[0, :, 0], + delta_phi(operator, 0.0, grid.length)[:, 0], reference, atol=bound + 1e-9 * float(np.max(np.abs(amplitude))), rtol=0, diff --git a/tests/translate/test_planar.py b/tests/translate/test_planar.py index 72fd143cf..cc655a6b0 100644 --- a/tests/translate/test_planar.py +++ b/tests/translate/test_planar.py @@ -151,7 +151,7 @@ def test_the_operator_at_ky_zero_is_each_slab_mode_to_its_rounding(space, proble - 0.5 * separable.spectral_rate[active] ) assert_allclose( - operator[0, channel, 0, active], + operator[channel, 0, active], expected, rtol=0, atol=8 * EPS * float(np.max(np.abs(beta))), @@ -167,8 +167,8 @@ def test_the_operator_at_ky_zero_is_each_slab_mode_to_its_rounding(space, proble rows = slice(5, 19) part_operator, part_prefactor = separable_values(separable, rows) assert_allclose( - part_operator[0], - operator[0, rows], + part_operator, + operator[rows], rtol=0, atol=4 * EPS * float(np.max(np.abs(operator))), ) @@ -196,7 +196,7 @@ def test_a_high_ky_column_is_evanescent_and_clamped(space, problem): # k_y = pi/dy far beyond every beta: the evanescent cap and the literal clamp. column = 2 * 2 active = GRID.active_mask - assert_allclose(np.real(operator[0, column, 0, active]), -data.evanescent_rate) + assert_allclose(np.real(operator[column, 0, active]), -data.evanescent_rate) assert_array_equal(prefactor[column, 0, active], data.prefactor_clamped[active]) diff --git a/tests/translate/test_power_boundaries.py b/tests/translate/test_power_boundaries.py index 1cd25c1dd..2b1548d79 100644 --- a/tests/translate/test_power_boundaries.py +++ b/tests/translate/test_power_boundaries.py @@ -244,7 +244,7 @@ def test_a_taper_with_a_floored_vacuum_end_translates(): phi = linear.phi_expansion assert phi is not None and phi.endpoint_degrees is not None assert phi.parameters == ("core_radius", "density") - n_pol = linear.value_shape[2] + n_pol = linear.value_shape[1] (mode,) = simulation.modes assert isinstance(mode, ParameterisedMode) sample = linop._log_weight_sampler((mode,), simulation.grid, n_pol, None) diff --git a/tests/translate/test_problem.py b/tests/translate/test_problem.py index ccfbf3ca2..785fd7200 100644 --- a/tests/translate/test_problem.py +++ b/tests/translate/test_problem.py @@ -268,7 +268,7 @@ def test_every_page_states_the_schema_the_code_writes(): is not a statement of the current one; the SEP records are history throughout. The kernel contract no longer has a number (ADR-068): it names the last one as history. """ - assert OUTPUT_SCHEMA_VERSION == 2 + assert OUTPUT_SCHEMA_VERSION == 3 stated: dict[str, list[str]] = {} for page in sorted(DOCS.rglob("*.md")): name = page.relative_to(DOCS).as_posix() diff --git a/tests/translate/test_radial.py b/tests/translate/test_radial.py index 169ce40ac..32499b7db 100644 --- a/tests/translate/test_radial.py +++ b/tests/translate/test_radial.py @@ -138,7 +138,7 @@ def test_varying_radial_medium_uses_cumulative_expansion(case, comoving): bound = ( rate_bound * (right - left) + quad_budget + endpoint_budget(amplitude) ) - actual = delta_phi(operator, left, right)[0] + actual = delta_phi(operator, left, right) assert_allclose(actual[:, 0], expected, rtol=0, atol=bound) # The polarisation columns hold identical terms, but the increment is one # BLAS product (np.tensordot) whose kernel may round each output column @@ -152,7 +152,7 @@ def test_varying_radial_medium_uses_cumulative_expansion(case, comoving): assert_within( actual[:, 1], actual[:, 0], - gamma(2 * weights.size) * magnitude[0][:, 0], + gamma(2 * weights.size) * magnitude[:, 0], elementwise=True, what="polarisation columns", ) @@ -255,7 +255,7 @@ def test_a_constant_radial_medium_carries_no_adiabatic_term(case): # The uniform medium's rate is the material operator itself, element by element: # no real endpoint term enters, so the exactly integrated D z is unchanged. np.testing.assert_array_equal( - operator.phi_constant[0, :, 0], model.operator(space.k_r**2) + operator.phi_constant[:, 0], model.operator(space.k_r**2) ) @@ -295,7 +295,7 @@ def attenuation(z): assert error <= quad_budget expected = amplitude.copy() expected[:, grid.active_mask] -= loss - actual = np.real(delta_phi(operator, left, right)[0, :, 0]) + actual = np.real(delta_phi(operator, left, right)[:, 0]) # Propagating bins: Re Phi is the endpoint term less the material loss # integral, which is zero for argon; the integral contributes no real part. assert_allclose( @@ -388,7 +388,7 @@ def test_a_varying_medium_with_no_propagating_column_carries_no_amplitude(case): assert operator.phi_expansion.endpoint_degrees is None # Every active bin is the capped attenuation alone, to the integral's # representation bound (relative 8 DEFAULT_TOL on the cap). - actual = np.real(delta_phi(operator, 0.0, grid.length)[0, :, 0]) + actual = np.real(delta_phi(operator, 0.0, grid.length)[:, 0]) cap = model.evanescent_rate * grid.length assert_allclose( actual[:, grid.active_mask], -cap, rtol=0, atol=8 * DEFAULT_TOL * cap @@ -422,7 +422,7 @@ def test_reused_radial_expansion_carries_the_adiabatic_term(case): amplitude, _ = adiabatic_oracle(medium, grid, space, left, right) budget = endpoint_budget(amplitude) for operator in (reused, fresh): - actual = np.real(delta_phi(operator, left, right)[0, :, 0]) + actual = np.real(delta_phi(operator, left, right)[:, 0]) assert_allclose(actual, amplitude, rtol=0, atol=budget) diff --git a/tests/translate/test_response_kinks.py b/tests/translate/test_response_kinks.py index 77d6e878a..08ac59ac6 100644 --- a/tests/translate/test_response_kinks.py +++ b/tests/translate/test_response_kinks.py @@ -64,7 +64,7 @@ def test_constant_phi_split_at_kinks_stays_exact_and_keeps_its_constant(): z = np.array([0.0, 0.1, 0.25, 0.4, 0.61, 0.9, 1.05]) values = np.stack([split.polynomial(float(one)) for one in z]) assert whole.phi_constant is not None - expected = z[:, None, None, None, None] * whole.phi_constant[None] + expected = z[:, None, None, None] * whole.phi_constant[None] np.testing.assert_allclose(values, expected, rtol=0, atol=8 * np.finfo(float).eps) diff --git a/tests/translate/test_spatial_contract.py b/tests/translate/test_spatial_contract.py index e6b1a6c50..1a75077a9 100644 --- a/tests/translate/test_spatial_contract.py +++ b/tests/translate/test_spatial_contract.py @@ -224,10 +224,10 @@ def test_separable_values_cover_propagating_cutoff_loss_and_gain(coefficients): prefactor[i, j] = -2j / root[i, j].real prefactor *= coefficients.active actual, measured_prefactor = separable_values(coefficients) - assert actual.shape == coefficients.value_shape == (1, 3, 2, 4) - assert_allclose(actual[0, :, 0], expected, rtol=gamma(16), atol=0) + assert actual.shape == coefficients.value_shape == (3, 2, 4) + assert_allclose(actual[:, 0], expected, rtol=gamma(16), atol=0) assert_allclose(measured_prefactor[:, 0], prefactor, rtol=gamma(16), atol=0) - np.testing.assert_array_equal(actual[0, :, 0], actual[0, :, 1]) + np.testing.assert_array_equal(actual[:, 0], actual[:, 1]) @pytest.mark.parametrize( diff --git a/tests/vector_raman_cases.py b/tests/vector_raman_cases.py index 5be24742e..5d0453b60 100644 --- a/tests/vector_raman_cases.py +++ b/tests/vector_raman_cases.py @@ -139,9 +139,9 @@ def result(backend_class, problem, run_request, grid, modes, options=CHUNK): backend._hoist(0.0) bundle = backend._stage_scalars(0.0, 0.0) rhs = backend._nonlinear( - problem.initial.field0[None], + problem.initial.field0, bundle, - np.ones((1, *problem.shape), complex), + np.ones(problem.shape, complex), ) else: assert backend.name == "native" diff --git a/tests/wall_gas_cases.py b/tests/wall_gas_cases.py index fd10c56bc..5d963b63e 100644 --- a/tests/wall_gas_cases.py +++ b/tests/wall_gas_cases.py @@ -102,8 +102,7 @@ def launch_source(problem, run_request, grid) -> tuple[np.ndarray, float]: """Return the reference's source ``N(a_0)`` at the launch, and ``||a_0||``.""" backend = NumpyRefBackend(problem, run_request, grid=grid) try: - # The backend's private state keeps its leading axis of one. - state = problem.initial.field0[np.newaxis] + state = problem.initial.field0 bundle = backend._stage_scalars(0.0, 0.0) source = backend._nonlinear(state, bundle, np.ones_like(state)) return np.array(source), float(np.linalg.norm(state))