Skip to content

test(suite): declare the boost oracle's platform scope - #64

Merged
LoganAMorrison merged 2 commits into
masterfrom
claude/infallible-benz-ddc979
Aug 12, 2026
Merged

test(suite): declare the boost oracle's platform scope#64
LoganAMorrison merged 2 commits into
masterfrom
claude/infallible-benz-ddc979

Conversation

@LoganAMorrison

@LoganAMorrison LoganAMorrison commented Aug 12, 2026

Copy link
Copy Markdown
Owner

Summary

  • test/test_core_boost.py's cross-implementation gate was vacuous off macOS, and this replaces the mechanism that made it so. It scoped its bit-equality claims against the Cython twin with cython_contracts() — a probe comparing the compiled kernel against an unfused Python transcription. cython-to-rust Task 4.1 showed that mechanism unsound in test/test_core_positron_muon.py (PR #63, runs 31562223329 and 31564747071): it tests one contraction mechanism and is blind to the others, so it claims bit-equality where none holds and voids the comparison where its own mechanism is absent. This module was resolving the second way — on Linux/x86-64 the probe returned False and all 19 of its cross-implementation claims skipped, on every CI entry but macOS since PR #61. Replaced with the two declared modes Task 4.1 established: ON_THE_CAPTURING_PLATFORM, read from test/parity/data/manifest.json so this module's scope cannot drift from the corpus's, bit-equality there, and a measured budget elsewhere. Nothing in the module skips on any platform now.
  • Both budgets were measured, not guessed. This worktree was built for linux/amd64 (Debian, gcc, glibc, CPython 3.12.13, NumPy 2.5.1) and the two implementations compared directly over 40,000 delta-function draws and 16,800 tabulated points. OFF_PLATFORM_BUDGET = 1e-10 is scaled to the peak of the compared array rather than applied pointwise, and OFF_PLATFORM_EDGE_ULPS = 4096 bounds the support boundaries in ulps — a value tolerance cannot express those, since across the boundary one implementation returns a finite number and the other zero. Figures and headroom in the table below.
  • TestFusedArithmetic stopped needing a platform scope at all. It now discriminates against a fused Python reference — fma at exactly the twelve sites rust/src/boost.rs spells mul_add — instead of against the Cython. Both references are pure Python/NumPy and f64::mul_add is correctly rounded on every target Rust supports, so "the port fuses here and only here" is a claim that holds everywhere, and the class now outlives the .pyx rather than dying with it in Phase 06 Task 6.4.
  • No public value changes. git diff origin/master -- hazma is empty; this diff touches only test/, docs/, and projects/.
  • Docs: docs/agents/lessons.md's [platform-scoped-oracle-asserted-globally] and the two project docs that still prescribed "probe at import and skip" now prescribe the declared scope — that guidance would otherwise have reproduced the bug in the next task. test/test_core_interp.py carries the identical retired mechanism and still skips 9 claims off macOS; filed as docs/followups/todo/interp-oracle-scoped-by-an-unsound-probe.md rather than fixed here.

How the vacuous gate was established

Two independent ways, because the capturing platform cannot see this failure — there the probe answers correctly by accident.

  1. Directly: build this worktree for linux/amd64, run the module with -rs. The probe returns False; 19 claims skip (11 carrying requires_a_contracting_cython, 8 reaching a mid-test assert_matches_cython).
  2. From CI: master run 31619425557 reports 767 passed, 41 skipped on Linux against 1424 passed, 14 skipped on macOS. Once the corpus's own skip drops out with --ignore=test/parity, that is a 28-skip excess — exactly this module's 19 plus test_core_interp.py's 9.

The measurement the budgets come from

comparison max relative max |Δ|/peak
boost_delta_function (40,000 draws) 1.9e-13 7.3e-17
…_interp, eta 9.9e-13 1.2e-15
…_interp, eta_prime 6.3e-13 7.2e-16
…_interp, charged_kaon 3.6e-13 3.2e-15
…_interp, long_kaon 1.3e-13 3.1e-15
…_interp, short_kaon 1.2e-13 3.4e-15
…_interp, omega 2.6e-13 3.3e-15
…_interp, phi 9.1e-14 5.9e-16

No sign flip, no NaN, and no disagreement anywhere about which energies are zero — the zeros are compared first and on every platform, since where a spectrum vanishes is structural and no rounding excuses it.

OFF_PLATFORM_BUDGET = 1e-10 is 100× the worst relative reading and 2.9e4× the worst peak-relative one. The peak scaling is the load-bearing choice, and the two columns are why: they differ by three orders of magnitude, because the worst relative gaps land where the integrand cancels. On the eta table the two maxima together bound the value carrying the worst relative gap at 1.2e-3 of the peak, so a pointwise rtol wide enough to admit it would be a thousand times wider than the spectrum is — the shape Task 3.4 rejected a tolerance over. A wrong branch or dropped term still lands at O(1) against the peak; TestDroppedInteriorCell measures the one defect this module knows the size of at 53%.

Support edges: measured at 55 ulps worst over 800 bracketed edges (547 on the identical double). 4096 leaves 74× headroom and is a relative displacement of 9.1e-13 — eleven orders below anything a wrong edge formula would produce.

Why there are three guards

On the capturing platform every comparison takes its exact branch, so neither budget nor the mode dispatch is exercised by anything else and all three could rot unnoticed. Each was verified to fail under a mutation of the constant it protects — and the first two drafts of these guards survived their mutations, because they derived their perturbations from the constants they were testing. Both now use fixed absolute figures (BUDGET_PROBE_ERROR, EDGE_PROBE_ERROR_ULPS), and the mode guard re-derives the expected mode from sys/platform rather than reading ON_THE_CAPTURING_PLATFORM back out — reading it back would agree with the dispatcher by construction and assert nothing.

Project

projects/cython-to-rust/ — follow-up to Phase 03 Task 3.4 and Phase 04 Task 4.1, not a numbered task of its own.

projects/cython-to-rust/task-notes/phase-03/task-3.4-interp-boost.md carries a dated superseding note in §Findings (the diagnosis holds; the remedy does not) plus the skip arithmetic above. projects/cython-to-rust/task-notes/phase-04/task-4.1-positron-muon.md §Findings is the reasoning this copies.

Review round 1 (fd6090b)

Reviewer found durable verification counts contradicting the PR head. Documentation only — no file under hazma/ or test/ changed, so nothing in the Summary above moves.

The root cause was sectioning, not arithmetic: footnoting a single row of task-3.4-interp-boost.md's ## Verification table as "left as taken" implicitly upgraded every unlabeled measurement beside it into a claim about the current tree, including the mutation campaign's 102 passed baseline sixty lines below. Fixed at the section level with a dated banner. The genuine miss was that same footnote's 81 passed — measured correctly, then invalidated by my own later self-review edits (two tautological tests dropped, one mode guard added) and never re-derived; the head is 80, now shown as a derivation from diffed collected test ids (+12 −1 = +11) rather than a bare number.

Two apparent hits were deliberately left, because a stale-looking number can be right for what it claims: phase-03/README.md:572's cargo test … (69 units) describes the foundation's units, so "fixing" it to the current 80 would fold in Task 4.1's kernel and make a true sentence false. The rule applied throughout — a measurement in a dated block is a record and stays as taken; a measurement in the present tense must match the PR head.

Lessons ledger: this PR cited on [measurement-taken-before-the-task-ended] as its third shape, plus a new [partial-historical-labeling] entry.

Test plan

  • scripts/agents/preflight.sh --paths test/test_core_boost.py --md "<the six changed docs>"RESULT: PASS, every gate green:

    PASS   black --check           test/test_core_boost.py
    PASS   isort --check-only      test/test_core_boost.py
    PASS   ruff check              test/test_core_boost.py
    PASS   cargo fmt --check       rust/
    PASS   cargo clippy            rust/
    PASS   cargo test              rust/
    PASS   pytest                  1435 passed, 14 skipped, 5 warnings in 538.01s (0:08:58)
    PASS   import hazma            version 2.1.0
    PASS   markdownlint            <the six changed docs>
    PASS   forbidden tokens        none added
    

    1435 passed, 14 skipped against 1424 passed, 14 skipped at Task 4.1. The arithmetic closes: +11 passes, all of them this module (69 tests → 80), and the skip count is unchanged because this module never skipped on macOS — the whole point is that it skipped everywhere else.

  • macOS/arm64, the capturing platform: pytest test/test_core_boost.py -q -rs80 passed, nothing skipped.

  • linux/amd64, built from clean in a container: pytest test/test_core_boost.py -q -rs80 passed, nothing skipped — the budgets hold against a genuinely different compiled Cython, which is the thing that could not be checked before. Same run confirms test_core_interp.py at 24 passed, 9 skipped, the follow-up's evidence.

  • Both modes on one machine: the suite re-run with platform.machine forced to x86_6480 passed, so the budget branch is exercised on the capturing platform too.

  • Guard mutations, each run from a green baseline: OFF_PLATFORM_BUDGET → inf fails test_the_value_budget_rejects_a_real_error; OFF_PLATFORM_EDGE_ULPS → 2**60 fails test_the_edge_budget_rejects_a_real_error; ON_THE_CAPTURING_PLATFORM → False fails test_this_platform_gets_the_mode_it_is_supposed_to. Baseline green again after.

  • The fused reference reproduces the port bit-for-bit on both platforms (0 misses in 1,800 tabulated points and 40,000 delta draws), while the unfused form misses 980 and 5,025 respectively. Its worst delta-function miss is 1.868e-13 — the same number, to four digits, as the Linux Cython's measured 1.8683e-13, which is the direct evidence that a compiler with no hardware FMA simply computes the unfused reference.

  • scripts/agents/check_doc_citations.py over the six changed docs → out-of-range or ambiguous: NONE.

  • Full CI matrix green (run 31638401939), and the Linux skip counts are the direct evidence the gate is no longer vacuous:

    before (master run 31619425557) this PR
    Linux ×5 (py3.10–3.14) 767 passed, 41 skipped 797 passed, 22 skipped
    macOS py3.14 1424 passed, 14 skipped 1435 passed, 14 skipped

    +30 / −19 on Linux is exactly what the change predicts: the 19 previously-skipped claims now run (+19 passed, −19 skipped) plus the module's +11 net new tests. The 22 remaining Linux skips are 13 pre-existing plus test_core_interp.py's 9 — the population docs/followups/todo/interp-oracle-scoped-by-an-unsound-probe.md tracks.

  • Round-1 fixes re-gated: scripts/agents/preflight.shRESULT: PASS (1435 passed, 14 skipped in 570.32s). Every count now asserted as current re-derived on the head — test_core_{constants,special,quad,interp,dispatch}.py at 25/65/58/33/118 unchanged, boost 80, cargo test 80, the Task 3.4 module pair 113.

🤖 Generated with Claude Code

LoganAMorrison and others added 2 commits August 12, 2026 12:20
`test/test_core_boost.py` scoped its bit-equality claims against the
Cython twin with `cython_contracts()` — a probe comparing the compiled
kernel against an unfused Python transcription. Task 4.1 showed that
mechanism to be unsound (PR #63, runs 31562223329 and 31564747071): it
tests one contraction mechanism and is blind to the others.

This module was resolving the *silent* way. On Linux/x86-64 the probe
returned False and all 19 of its cross-implementation claims skipped, so
that gate had been vacuous on every CI entry but macOS since PR #61.
Confirmed two ways: by building this worktree for linux/amd64 and reading
`-rs`, and from master run 31619425557, whose `767 passed, 41 skipped` on
Linux against `1424 passed, 14 skipped` on macOS leaves a 28-skip excess
once the corpus drops out — this module's 19 plus `test_core_interp.py`'s
9.

Replaced with the two declared modes `test/test_core_positron_muon.py`
established: `ON_THE_CAPTURING_PLATFORM`, read from the parity corpus's
own manifest so the scopes cannot drift, bit-equality there, and a
measured budget elsewhere.

The budgets were measured, not guessed — this worktree built for
linux/amd64 (Debian, gcc, glibc, CPython 3.12.13, NumPy 2.5.1) and
compared directly over 40,000 delta-function draws and 16,800 tabulated
points. Worst disagreement 9.9e-13 relative and 3.4e-15 of the peak, with
no sign flip, no NaN and no disagreement anywhere about which energies are
zero. `OFF_PLATFORM_BUDGET = 1e-10` is 100x the first reading and 2.9e4x
the second, scaled to the peak of the compared array rather than applied
pointwise: the two readings differ by three orders because the worst
relative gaps land where the integrand cancels.

Support edges get a second budget, in ulps, because a value tolerance
cannot span them — across the boundary one implementation returns a finite
number and the other zero. Measured at 55 ulps worst over 800 bracketed
edges; `OFF_PLATFORM_EDGE_ULPS = 4096` leaves 74x headroom and is still
eleven orders below any wrong edge formula.

`TestFusedArithmetic` no longer needs a scope at all. It now discriminates
against a *fused* Python reference — `fma` at exactly the twelve sites
`rust/src/boost.rs` spells `mul_add` — instead of against the Cython. Both
references are pure Python and NumPy and `f64::mul_add` is correctly
rounded on every target, so "the port fuses here and only here" holds
everywhere, and the class now outlives the `.pyx`.

Both budgets and the mode dispatch carry guards, since on the capturing
platform nothing else exercises them; each was verified to fail under a
mutation of the constant it protects.

No public value changes: this diff touches no code under `hazma/`.

Docs: the `[platform-scoped-oracle-asserted-globally]` lesson and the two
project docs that still prescribed the probe now prescribe the declared
scope. `test/test_core_interp.py` carries the same retired mechanism and
still skips 9 claims off macOS; filed as
`docs/followups/todo/interp-oracle-scoped-by-an-unsound-probe.md`.

Verification: `scripts/agents/preflight.sh` RESULT: PASS (`1435 passed, 14
skipped`, against `1424 passed, 14 skipped` at Task 4.1 — all +11 are this
module). 80 passed / 0 skipped on both macOS/arm64 and linux/amd64.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Review round 1 on PR #64: durable verification counts contradicted the PR
head. Three sites, one root cause and one genuine miss.

The root cause was sectioning, not arithmetic. Footnoting a single row of
task-3.4's `## Verification` table as "left as taken" implicitly upgraded
every unlabeled measurement beside it into a claim about the current tree
— including the mutation campaign's `102 passed` baseline sixty lines
below, which review read as live. Fixed at the section level with a dated
banner, plus an inline marker and the current figure (`113 passed`) at the
`102` line.

The miss: that footnote said `81 passed`. It was measured correctly and
then invalidated by my own later self-review edits — two tautological
tests dropped, one mode guard added — and never re-derived. The PR head is
`80`. Replaced with a derivation from diffed collected test ids rather
than a bare number: `+12 −1 = +11`, `69 + 11 = 80`. The old prose also
blamed the delta on the 19 rescoped claims, which is wrong; those were
re-gated, not added. The +11 is `TestFusedArithmetic` 1 → 9 and
`TestOffPlatformBudgets`'s three guards.

`learnings/phase-03-numerics-foundation.md`'s test inventory read as
current. Not rewritten to 80: that bullet records what Phase 03 closed
with, so 69 is right for what it claims. Dated the inventory instead and
added the one count that has moved since. The same edit fixes a sibling
review did not cite — `cargo test carries 69 units`, now 80 since Task
4.1 — credited there rather than to this PR.

Two apparent hits deliberately left. `phase-03/README.md:572`'s
`(69 units)` describes the *foundation's* units, so "fixing" it to 80
would fold in a later phase's kernel and make a true sentence false;
`phase-03/README.md:515` and `task-notes/README.md:1433` already sit under
explicitly dated headers. The rule applied throughout: a measurement in a
dated block is a record and stays as taken, a measurement in the present
tense must match the PR head.

Every count now asserted as current was re-derived on this tree:
test_core_{constants,special,quad,interp,dispatch}.py at 25/65/58/33/118
unchanged, boost 80, `cargo test` 80, the Task 3.4 module pair 113.

Lessons: cited this PR on [measurement-taken-before-the-task-ended] as its
third shape (a count of your own tests, taken while the diff was still
moving), and added [partial-historical-labeling], which no entry covered —
including its reverse, that a stale-looking number can be correct for what
it claims and blind number-chasing breaks it.

Documentation only: no file under `hazma/` or `test/` changed, so no
public value moves and the numerical impact of PR #64 is unaltered.

Verification: `scripts/agents/preflight.sh` RESULT: PASS (`1435 passed, 14
skipped in 570.32s`).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@LoganAMorrison
LoganAMorrison merged commit 707b07c into master Aug 12, 2026
8 checks passed
@LoganAMorrison
LoganAMorrison deleted the claude/infallible-benz-ddc979 branch August 12, 2026 22:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant