Skip to content

WIP: SEP 52 milestone 3 — reference restructured (workspace, engines, marches, session); HostDriver deleted - #858

Merged
jtravs merged 18 commits into
mainfrom
feature/sep-0052-backend-shape-p3
Nov 16, 2026
Merged

jtravs merged 18 commits into
mainfrom
feature/sep-0052-backend-shape-p3

Conversation

@jtravs

@jtravs jtravs commented Nov 16, 2026

Copy link
Copy Markdown
Contributor

Milestone 3 of SEP 52, tracked in #849. Stacked on #854 (milestone 2, itself on #853): merge those first. The details are in the SEP's Part II report for milestone 3.

The reference (numpy_ref) now has native's shape: workspace → engines → marches → session. HostDriver is gone.

  • NumpyWorkspace (numpy_ref/workspace.py) holds the state and every state-sized operation. The finish computes the split collar's sums on the pre-split field, exactly where they were taken before, and returns them in an AcceptRecord. The _adams_* hooks become march-neutral operations (Low-storage march: the planar route's stage register and two hook names #344). numpy_ref/backend.py goes from about 2,900 lines to 550.
  • Engines implementing SEP §5.2: NumpyRungeKutta commits only at accept. The low-storage step is the same RungeKuttaEngine, with the documented destructive trial. HostHistory is the new AdamsEngine, taking retained history, step_carrier and the TDSE reservation as begin_attempt/end_attempt. The old engine protocols are deleted.
  • Marches own their state: a new RungeKuttaMarch plus the rewritten Adams and low-storage marches. None writes another object's fields.
  • HostSession owns the schedule in session.cpp's order: planes, step planes, natural stops, yields, halts, the dense hold and observation (numpy_ref/dense.py), periodic rows, snapshots, monitor and edge checks, the quadrature report, events and exports.
  • Deleted: host/driver.py (2,530 lines), HostStepping, stepping() and the periodic-stats storage hooks.
  • Boundary warnings are WARNING barrier events on the reference too, and the service loop's HostSession special case is gone. User-visible warning behaviour is unchanged.
  • Tests are ported to sessions, the workspace and engines, or scripted engines that host the real marches. Each test deleted with its subject is listed in the report.

Numbers: none moved. All 13 pinned corpus cases are bitwise identical on numpy_ref before and after, on one build and one thread. The decision traces, the split-collar rules and the deep-collar residue test pass unchanged. No corpus was regenerated.

Checked: targeted modules locally, plus full tests/backends, tests/driver, tests/sessions, tests/output, tests/stepper and native/tests sweeps on a 32-core Linux node; the only failures there were the four already fixed on #854.

🤖 Generated with Claude Code

jtravs and others added 14 commits November 16, 2026 03:12
SEP 52 phase 3, step 1 of 2. NumpyWorkspace (numpy_ref/workspace.py) holds the
state and every state-sized operation that NumpyRefBackend and HostDriver held:
load, the stage scalars and the sources, the finish (which returns the split
collar's sums, taken on the masked field before the split, in an AcceptRecord),
the mask, the statistics, the boundary and edge readings, the quadrature alarm as
a reading, the host source's reservation, the memory ledger. The `_adams_*` hooks
become its march-neutral operations (source, lab_source, propagator,
coupled_block).

The engines implement the protocols of solprop.backends.engines over it:
NumpyRungeKutta keeps its trial and commits only at accept; HostLowStorage is the
same protocol with the destructive trial (in flight from trial to accept, the
stiffness in its AcceptRecord); HostHistory takes the retained entries at accept,
and the TDSE reservation as begin_attempt/end_attempt.

The marches own their state: RungeKuttaMarch (new), HostAdamsMarch and
HostLowStorageMarch each keep position, proposal, cursor and norms in a
MarchState and ask the driver only through the narrow MarchHost. HostDriver keeps
the scheduling and accumulates the split sums from each AcceptRecord.

No number moves: the pinned corpus passes unchanged and its numpy_ref arrays are
bitwise those of the parent commit on this build; the decision traces pass
unchanged. Tests that called the moved private methods use the workspace and the
engines; the trace harnesses and the scripted driver drive the new marches.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… calls

Engine-contract cases of SEP 52 section 7: a Runge-Kutta trial commits nothing
until accept; the low-storage trial is in flight until its accept returns, and a
failure in the finish leaves no committed state; an Adams attempt reserves, then
records its decision and releases, an abandoned one only releases; the accept
takes the retained entries oldest first; the finish takes the split collar's sums
on the masked field before the split, and the driver adds the removal and keeps
the latest occupancies. The marches' tests drive them over recording engines and
sessions. The design pages name the workspace, engines and marches.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The low-storage failure and stiffness probes wrap the engine's trial and its
passes, the general route's linear short-circuit reads the workspace, the
fused-predictor integration reads the march's engine, and the scripted driver,
now the reference backend, refuses atomic ledgers with its TDSE message. Casts
where a test reaches past the engine protocol.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eted

HostSession schedules the reference's runs and drives the marches directly:
planes, step planes, halts, yields, the dense hold and observation (through the
backend's dense output, applying the march's carrier rotations), periodic rows,
the snapshot rows, the boundary monitor and transverse edge with the split
collar's sums, the quadrature report and the load. Boundary warnings go into a
sink and are reported as WARNING barrier events. NumpyRefBackend builds a fresh
march per run and keeps lowering, prepare, metadata, the ledger, the probe and
close. HostStepping, stepping() and the statistics storage hooks are removed.
The scripted executor and the decision-trace harnesses host the real marches.

numpy_ref is bitwise unchanged on the pinned corpus. Other tests are ported in
the following commits.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The tests that loaded and stepped the reference's step driver now run its
session, read its workspace or build its engines; the scripted executor and
the trace harnesses host the real marches. Tests of the deleted structure go
(HostDriver inheritance, the HostStepping seam, the statistics storage hooks,
target refusals a session cannot reach). Native tests that use the reference
as their oracle load its workspace. The benchmark worker times the reference's
workspace load. The structure figures name the session's parts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
backends.md (the seam, the five parts, the marches, boundary warnings as
barrier events), architecture.md section 9, kernel-contract.md (the statistics
rows and the march's bounds name the session), stepper.md, structure.md,
algorithms.md and testing.md (scripted engines) describe the code without the
step driver. The timing probe's private docstrings take the private style.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tale

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Native-against-reference warning tests now compare both sessions' warning
barriers; the march loads no engine; the TDSE startup reads the history's rows;
the stiffness warning's test holds the plane's replay to one march call.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… ci]

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@read-the-docs-community

read-the-docs-community Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

…tests

test_transverse_monitor reads the clean launch columns from the workspace. The
backend's doctest lists the barriers and the field-free test's plane advance is
unbounded: a bounded yield on a slow host adds a Progress batch before a plane,
on the reference before and after this restructure and on native alike.
Stale step-driver names in docstrings, and two long lines.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

jtravs and others added 3 commits November 16, 2026 07:49
…p ci]

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…on made dead

Tests reach a backend closed under its session, a start whose plane lies behind
it, a probe start of the wrong shape, the dense outputs' refusals before a step,
the engine's stages before a trial, each engine's protocol reading and the
low-storage extend refusal, and a low-storage replay that runs out. A field-free
drain no longer needs a loaded, open backend, so the closed backend's failure
reaches its terminal event. Removed as dead: adams_core.quiet_capped (the session's
step_cap), the low-storage march's snapshot, the workspace's quadrature property,
and the target guard's finiteness and table-end checks, which the validated
request already makes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Base automatically changed from feature/sep-0052-backend-shape-p2 to main November 16, 2026 09:07
@jtravs
jtravs marked this pull request as ready for review November 16, 2026 10:05
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-16T10:10:50.009118Z 6eb0de3 Draft marked ready
🔒 Security Review ✅ Completed 2026-11-16T10:09:12.426404Z 6eb0de3 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@jtravs
jtravs merged commit 33dda98 into main Nov 16, 2026
44 checks passed
@jtravs
jtravs deleted the feature/sep-0052-backend-shape-p3 branch November 16, 2026 10:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant