Skip to content

Native backend architecture cleanup: layout, header boundaries, file summaries - #864

Draft
jtravs wants to merge 23 commits into
feature/sep-0051-save-planesfrom
refactor/native-architecture
Draft

jtravs wants to merge 23 commits into
feature/sep-0051-save-planesfrom
refactor/native-architecture

Conversation

@jtravs

@jtravs jtravs commented Nov 16, 2026

Copy link
Copy Markdown
Contributor

Architecture review and cleanup of the native backend (native/): reorganise for navigability with unchanged numerics and performance. No change to the _core API, the kernel contract or solprop_native.__all__ (SEP 52's interface).

Stacked on #851 (and so #859); retargets to main once they merge.

Stages

  • S1 dead code and stale references: fft, multiply_phase, GeneralState::adams_source/adams_magnitude, dead adapter constants, a timing script in native/tests/, stale _records citations.
  • S2 header boundaries: core.hpp/core.cpp split into execution, readings, cpu_execution, reference_timing, problem, general_state; the host FFTW wisdom moved out of fft_plan.hpp into fftw_wisdom.hpp/.cpp (the _core binding target no longer compiles KokkosFFT or fftw3.h); run_records.hpp; session_resident.hpp; setup_data.hpp; named spatial statistics flags; renames (general_workspace.hpp, session_dense.hpp/dense.hpp, statistics_plan.cpp, bindings_helpers.*).
  • S4c adapter split: backend.py → backend.py + _lowering.py + _march.py + _device.py; __init__.py → __init__.py + _build.py; monkeypatches retargeted (each checked to still bite).
  • S3 domain directory layout (runtime/, transforms/, problem/, response/, march/, session/, src/bindings/)
  • S4 trial.cpp as one TU in topic fragments; bindings split; native test organisation; CMake modules
  • S5 file-level summaries for every C++ file; source-layout map in native/README.md
  • Bitwise and A/B checks (DMOG + macOS), report

Draft; the full report (changes, deleted code, validation, risks) lands here at the end.

🤖 Generated with Claude Code

jtravs and others added 21 commits November 16, 2026 11:24
fft() and multiply_phase() were reached only by their own smoke tests;
GeneralState::adams_source and adams_magnitude (and their workspace
definitions) had no caller since the SEP 52 seam removal.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
core.hpp uses nothing from launch.hpp; plasma.hpp, z_table.hpp and
core.cpp, which relied on it transitively, now include it themselves.
dense_output.hpp gets adams_march.hpp through dense.hpp.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
backend.py's statistic flag tables had no reader; the deleted
solprop_native._records is cited as the C++ converter or solprop's
functions instead; tests/general_modal_benchmark.py was a timing script
in tests/ that nothing ran.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
core.hpp held the execution aliases, the monitor and edge readings,
the CPU execution policy, the SEP 28 timing references, the Problem
record and GeneralState in one 1450-line header. Each block moves,
verbatim and in its old order, to its own file:

- execution.hpp: ExecutionSpace, Layout, Range2, Complex, the view
  aliases and general_vector_lanes (from general_modal.hpp, so that
  slab.hpp no longer pulls KokkosFFT through general_modal.hpp)
- readings.hpp: the reduction sums with their reduction identities,
  EdgeLines with its field table (from workspace.hpp)
- cpu_execution.hpp/.cpp: ThreadPolicy, CpuExecution, FftPlanning,
  CpuExecutionPool, CpuTeamScope, the spin and the test hooks
- reference_timing.hpp/.cpp: reference_fft3_pairs and
  reference_stream_copies, with the KokkosFFT include
- problem.hpp/.cpp: FieldFamily, StatisticsFlag, Problem with its
  field tables, Problem::validate and compact_problem
- general_state.hpp: the Adams weights, TrialScalars, TableKind with
  its names (from workspace.hpp) and GeneralState, which forward-
  declares the host source types it only passes on
- refuse.hpp: EnvelopeConvergenceError and BackendRefusal

Every former includer now includes the headers it uses. The orphaned
adams_max_samples comment rejoins its constant. No code changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fft_plan.hpp held the plan policy, the workspace's planning and the
wisdom calls inline beside the device row plans, so every includer,
bindings.cpp among them, compiled KokkosFFT and fftw3.h for them. The
host-only bodies (fftw_seed_effort, fftw_effort_flag, the wisdom
export and import, FftPlanningScope, the seeds' bookkeeping,
fft_plan_key, time_fft_plan) and seed_fftw_wisdom from
general/kernels.hpp move verbatim to fftw_wisdom.cpp, declared in
fftw_wisdom.hpp, which includes neither. detail::planning and the
helpers only the seeds use become file-local. The row seed template
stays in fft_plan.hpp, which needs fftw3.h and KokkosFFT for it.

bindings.cpp now includes fftw_wisdom.hpp alone, and its wisdom-forget
test hook calls the new forget_fftw_wisdom: no binding unit compiles
KokkosFFT or fftw3.h. Plan-time code only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
workspace.hpp held the Setup structs, the run records that mirror
solprop.backends.records, their field tables and to_native beside the
Run, Probe and Workspace classes. The records move verbatim, with the
file comment's paragraph on the binding's conversion, to
run_records.hpp; workspace.hpp includes it and drops the headers only
the records used. No code changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
engines.hpp included all of session.hpp (the session, its plans and
methods, dense output, the monitor) for the one interface its
GeneralResident implements. session::Resident and the
StatisticsRecipe it takes move verbatim to session_resident.hpp, which
session.hpp and engines.hpp include. No code changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
heap_bytes and device_free_bytes are host code run once around
planning, but lived inline in the header every general/ unit and its
kernels include, with the allocator headers they need. They move
verbatim to general/memory.cpp, declared in general/memory.hpp, which
setup.cpp and statistics.cpp, their callers, include. No code changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
general/kernels.hpp's checked_product, checked_sum, require_extent and
require_allocation, general_modal.cpp's finite and the host-vector
upload that general_modal.cpp and coupled_flow.cpp each defined move to
setup_data.hpp. dispersive.cpp's product_of, the same check with the
same message as checked_product, gives way to it. The helpers whose
messages name their unit (coupled flow's and general modal's product
and require_size, dispersive's require_size and its typed uploads)
stay where they are. Setup-time code only; no behaviour changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
session_statistics.cpp named spatial_statistics' flag bits while
general/statistics.cpp wrote them as literals, and its plane_moments
repeated general/workspace.hpp's plane_moment_slots. The named flags,
a mask of them all and plane_moment_slots now sit beside StatisticsFlag
in problem.hpp, and both units and statistics_plan.hpp's comments use
them. Constants only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Its class is GeneralWorkspace, and the name no longer reads as the
public include/solprop_native/workspace.hpp. Includes and its strict-TU
refusal follow.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
It declares the session's dense-output interface; the name frees
dense.hpp for the dense:: weights and engines.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
It declares the dense:: weights and the AdamsDense and RungeKuttaDense
engines that dense.cpp defines.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
It implements statistics_plan.hpp (namespace statistics::). It stays a
strict file: SOLPROP_NATIVE_STRICT_SOURCES, native/AGENTS.md's list and
the licence test's copy of it follow; the namespace, which
ci/check_strict_fma.py matches, is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
They are nanobind bindings, part of the _core module target, beside
bindings.cpp and bindings_transverse.cpp.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
backend.py keeps NativeBackend, admission, refusals, the coupled flow's
host factors and FFT planning. The code moves verbatim:

- _lowering.py: _LowerPlan, _lower and its _lower_* helpers, the table
  packers and _ZTables.
- _march.py: the march protocol (reasons, error categories, FRESH_WATCH,
  _envelope_errors, _raise_native) and the Adams record _NativeAdamsMarch.
- _device.py: the build facts, _on_device, the node-chunk lowering and the
  resident flow volumes, which both backend and the lowering read.

_flow_guard stays in backend beside the coupled flow's guard constants.
Each module lists what its siblings import in __all__. The session takes
the march protocol from _march; tests that read, import or patch a moved
name now target the module where it is looked up.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
BuildInfo, build_info, _source_digest, _checkout_digest and check_build
move verbatim from the package's __init__ and are re-exported from it, so
solprop_native.__all__ and every public name are unchanged. The process
start (the processor check, thread budget, profile, wait policy, binding
and the _core import) and refresh_profile stay in __init__ in their order,
and NumPy still loads before them. _checkout_digest finds the checkout two
levels above _build.py, as it did above __init__.py; the digest tests now
patch the build module, where check_build looks the names up.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Since the rename the unit's main header is statistics_plan.hpp.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@read-the-docs-community

Copy link
Copy Markdown

Documentation build overview

📚 solprop | 🛠️ Build #35926355 | 📁 Comparing 373b1a1 against latest (33dda98)

  🔍 Preview build  

47 files changed · + 1 added · ± 45 modified · - 1 deleted

+ Added

± Modified

- Deleted

jtravs and others added 2 commits November 16, 2026 12:48
Every C++ file under native/include/solprop_native/ and native/src/ (the
Python package excepted) now lives in one domain directory: runtime/,
transforms/, problem/, response/, march/, session/ and general/ for the
headers, the same names plus bindings/ for the sources; src/general/
stays where it is. A header include/solprop_native/X.hpp moves to
include/solprop_native/<dir>/X.hpp and a source src/X to src/<dir>/X,
with every reference rewritten: the includes, the CMake source lists
(the strict list among them, with the licence test's copy), and the
path mentions in present-tense docs, comments and docstrings (SEPs and
the changelog are left as written). Docstring lines the longer paths
pushed past 88 columns are rewrapped, and clang-format re-sorts the
includes. Pure moves and reference rewrites: no code changes.

Generated by a re-runnable layout script that refuses any C++ file the
mapping does not place.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MSVC's library does not bring <variant> in transitively as libc++ does,
which broke the Windows arm64 build after the run records left
workspace.hpp. Also point a test docstring at cpu_execution.cpp.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant