test(suite): declare the boost oracle's platform scope - #64
Merged
Conversation
`test/test_core_boost.py` scoped its bit-equality claims against the Cython twin with `cython_contracts()` — a probe comparing the compiled kernel against an unfused Python transcription. Task 4.1 showed that mechanism to be unsound (PR #63, runs 31562223329 and 31564747071): it tests one contraction mechanism and is blind to the others. This module was resolving the *silent* way. On Linux/x86-64 the probe returned False and all 19 of its cross-implementation claims skipped, so that gate had been vacuous on every CI entry but macOS since PR #61. Confirmed two ways: by building this worktree for linux/amd64 and reading `-rs`, and from master run 31619425557, whose `767 passed, 41 skipped` on Linux against `1424 passed, 14 skipped` on macOS leaves a 28-skip excess once the corpus drops out — this module's 19 plus `test_core_interp.py`'s 9. Replaced with the two declared modes `test/test_core_positron_muon.py` established: `ON_THE_CAPTURING_PLATFORM`, read from the parity corpus's own manifest so the scopes cannot drift, bit-equality there, and a measured budget elsewhere. The budgets were measured, not guessed — this worktree built for linux/amd64 (Debian, gcc, glibc, CPython 3.12.13, NumPy 2.5.1) and compared directly over 40,000 delta-function draws and 16,800 tabulated points. Worst disagreement 9.9e-13 relative and 3.4e-15 of the peak, with no sign flip, no NaN and no disagreement anywhere about which energies are zero. `OFF_PLATFORM_BUDGET = 1e-10` is 100x the first reading and 2.9e4x the second, scaled to the peak of the compared array rather than applied pointwise: the two readings differ by three orders because the worst relative gaps land where the integrand cancels. Support edges get a second budget, in ulps, because a value tolerance cannot span them — across the boundary one implementation returns a finite number and the other zero. Measured at 55 ulps worst over 800 bracketed edges; `OFF_PLATFORM_EDGE_ULPS = 4096` leaves 74x headroom and is still eleven orders below any wrong edge formula. `TestFusedArithmetic` no longer needs a scope at all. It now discriminates against a *fused* Python reference — `fma` at exactly the twelve sites `rust/src/boost.rs` spells `mul_add` — instead of against the Cython. Both references are pure Python and NumPy and `f64::mul_add` is correctly rounded on every target, so "the port fuses here and only here" holds everywhere, and the class now outlives the `.pyx`. Both budgets and the mode dispatch carry guards, since on the capturing platform nothing else exercises them; each was verified to fail under a mutation of the constant it protects. No public value changes: this diff touches no code under `hazma/`. Docs: the `[platform-scoped-oracle-asserted-globally]` lesson and the two project docs that still prescribed the probe now prescribe the declared scope. `test/test_core_interp.py` carries the same retired mechanism and still skips 9 claims off macOS; filed as `docs/followups/todo/interp-oracle-scoped-by-an-unsound-probe.md`. Verification: `scripts/agents/preflight.sh` RESULT: PASS (`1435 passed, 14 skipped`, against `1424 passed, 14 skipped` at Task 4.1 — all +11 are this module). 80 passed / 0 skipped on both macOS/arm64 and linux/amd64. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Review round 1 on PR #64: durable verification counts contradicted the PR head. Three sites, one root cause and one genuine miss. The root cause was sectioning, not arithmetic. Footnoting a single row of task-3.4's `## Verification` table as "left as taken" implicitly upgraded every unlabeled measurement beside it into a claim about the current tree — including the mutation campaign's `102 passed` baseline sixty lines below, which review read as live. Fixed at the section level with a dated banner, plus an inline marker and the current figure (`113 passed`) at the `102` line. The miss: that footnote said `81 passed`. It was measured correctly and then invalidated by my own later self-review edits — two tautological tests dropped, one mode guard added — and never re-derived. The PR head is `80`. Replaced with a derivation from diffed collected test ids rather than a bare number: `+12 −1 = +11`, `69 + 11 = 80`. The old prose also blamed the delta on the 19 rescoped claims, which is wrong; those were re-gated, not added. The +11 is `TestFusedArithmetic` 1 → 9 and `TestOffPlatformBudgets`'s three guards. `learnings/phase-03-numerics-foundation.md`'s test inventory read as current. Not rewritten to 80: that bullet records what Phase 03 closed with, so 69 is right for what it claims. Dated the inventory instead and added the one count that has moved since. The same edit fixes a sibling review did not cite — `cargo test carries 69 units`, now 80 since Task 4.1 — credited there rather than to this PR. Two apparent hits deliberately left. `phase-03/README.md:572`'s `(69 units)` describes the *foundation's* units, so "fixing" it to 80 would fold in a later phase's kernel and make a true sentence false; `phase-03/README.md:515` and `task-notes/README.md:1433` already sit under explicitly dated headers. The rule applied throughout: a measurement in a dated block is a record and stays as taken, a measurement in the present tense must match the PR head. Every count now asserted as current was re-derived on this tree: test_core_{constants,special,quad,interp,dispatch}.py at 25/65/58/33/118 unchanged, boost 80, `cargo test` 80, the Task 3.4 module pair 113. Lessons: cited this PR on [measurement-taken-before-the-task-ended] as its third shape (a count of your own tests, taken while the diff was still moving), and added [partial-historical-labeling], which no entry covered — including its reverse, that a stale-looking number can be correct for what it claims and blind number-chasing breaks it. Documentation only: no file under `hazma/` or `test/` changed, so no public value moves and the numerical impact of PR #64 is unaltered. Verification: `scripts/agents/preflight.sh` RESULT: PASS (`1435 passed, 14 skipped in 570.32s`). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
test/test_core_boost.py's cross-implementation gate was vacuous off macOS, and this replaces the mechanism that made it so. It scoped its bit-equality claims against the Cython twin withcython_contracts()— a probe comparing the compiled kernel against an unfused Python transcription. cython-to-rust Task 4.1 showed that mechanism unsound intest/test_core_positron_muon.py(PR #63, runs 31562223329 and 31564747071): it tests one contraction mechanism and is blind to the others, so it claims bit-equality where none holds and voids the comparison where its own mechanism is absent. This module was resolving the second way — on Linux/x86-64 the probe returnedFalseand all 19 of its cross-implementation claims skipped, on every CI entry but macOS since PR #61. Replaced with the two declared modes Task 4.1 established:ON_THE_CAPTURING_PLATFORM, read fromtest/parity/data/manifest.jsonso this module's scope cannot drift from the corpus's, bit-equality there, and a measured budget elsewhere. Nothing in the module skips on any platform now.OFF_PLATFORM_BUDGET = 1e-10is scaled to the peak of the compared array rather than applied pointwise, andOFF_PLATFORM_EDGE_ULPS = 4096bounds the support boundaries in ulps — a value tolerance cannot express those, since across the boundary one implementation returns a finite number and the other zero. Figures and headroom in the table below.TestFusedArithmeticstopped needing a platform scope at all. It now discriminates against a fused Python reference —fmaat exactly the twelve sitesrust/src/boost.rsspellsmul_add— instead of against the Cython. Both references are pure Python/NumPy andf64::mul_addis correctly rounded on every target Rust supports, so "the port fuses here and only here" is a claim that holds everywhere, and the class now outlives the.pyxrather than dying with it in Phase 06 Task 6.4.git diff origin/master -- hazmais empty; this diff touches onlytest/,docs/, andprojects/.docs/agents/lessons.md's[platform-scoped-oracle-asserted-globally]and the two project docs that still prescribed "probe at import and skip" now prescribe the declared scope — that guidance would otherwise have reproduced the bug in the next task.test/test_core_interp.pycarries the identical retired mechanism and still skips 9 claims off macOS; filed asdocs/followups/todo/interp-oracle-scoped-by-an-unsound-probe.mdrather than fixed here.How the vacuous gate was established
Two independent ways, because the capturing platform cannot see this failure — there the probe answers correctly by accident.
-rs. The probe returnsFalse; 19 claims skip (11 carryingrequires_a_contracting_cython, 8 reaching a mid-testassert_matches_cython).767 passed, 41 skippedon Linux against1424 passed, 14 skippedon macOS. Once the corpus's own skip drops out with--ignore=test/parity, that is a 28-skip excess — exactly this module's 19 plustest_core_interp.py's 9.The measurement the budgets come from
boost_delta_function(40,000 draws)…_interp, eta…_interp, eta_prime…_interp, charged_kaon…_interp, long_kaon…_interp, short_kaon…_interp, omega…_interp, phiNo sign flip, no NaN, and no disagreement anywhere about which energies are zero — the zeros are compared first and on every platform, since where a spectrum vanishes is structural and no rounding excuses it.
OFF_PLATFORM_BUDGET = 1e-10is 100× the worst relative reading and 2.9e4× the worst peak-relative one. The peak scaling is the load-bearing choice, and the two columns are why: they differ by three orders of magnitude, because the worst relative gaps land where the integrand cancels. On the eta table the two maxima together bound the value carrying the worst relative gap at 1.2e-3 of the peak, so a pointwisertolwide enough to admit it would be a thousand times wider than the spectrum is — the shape Task 3.4 rejected a tolerance over. A wrong branch or dropped term still lands at O(1) against the peak;TestDroppedInteriorCellmeasures the one defect this module knows the size of at 53%.Support edges: measured at 55 ulps worst over 800 bracketed edges (547 on the identical double). 4096 leaves 74× headroom and is a relative displacement of 9.1e-13 — eleven orders below anything a wrong edge formula would produce.
Why there are three guards
On the capturing platform every comparison takes its exact branch, so neither budget nor the mode dispatch is exercised by anything else and all three could rot unnoticed. Each was verified to fail under a mutation of the constant it protects — and the first two drafts of these guards survived their mutations, because they derived their perturbations from the constants they were testing. Both now use fixed absolute figures (
BUDGET_PROBE_ERROR,EDGE_PROBE_ERROR_ULPS), and the mode guard re-derives the expected mode fromsys/platformrather than readingON_THE_CAPTURING_PLATFORMback out — reading it back would agree with the dispatcher by construction and assert nothing.Project
projects/cython-to-rust/— follow-up to Phase 03 Task 3.4 and Phase 04 Task 4.1, not a numbered task of its own.projects/cython-to-rust/task-notes/phase-03/task-3.4-interp-boost.mdcarries a dated superseding note in §Findings (the diagnosis holds; the remedy does not) plus the skip arithmetic above.projects/cython-to-rust/task-notes/phase-04/task-4.1-positron-muon.md§Findings is the reasoning this copies.Review round 1 (
fd6090b)Reviewer found durable verification counts contradicting the PR head. Documentation only — no file under
hazma/ortest/changed, so nothing in the Summary above moves.The root cause was sectioning, not arithmetic: footnoting a single row of
task-3.4-interp-boost.md's## Verificationtable as "left as taken" implicitly upgraded every unlabeled measurement beside it into a claim about the current tree, including the mutation campaign's102 passedbaseline sixty lines below. Fixed at the section level with a dated banner. The genuine miss was that same footnote's81 passed— measured correctly, then invalidated by my own later self-review edits (two tautological tests dropped, one mode guard added) and never re-derived; the head is80, now shown as a derivation from diffed collected test ids (+12 −1 = +11) rather than a bare number.Two apparent hits were deliberately left, because a stale-looking number can be right for what it claims:
phase-03/README.md:572'scargo test … (69 units)describes the foundation's units, so "fixing" it to the current 80 would fold in Task 4.1's kernel and make a true sentence false. The rule applied throughout — a measurement in a dated block is a record and stays as taken; a measurement in the present tense must match the PR head.Lessons ledger: this PR cited on
[measurement-taken-before-the-task-ended]as its third shape, plus a new[partial-historical-labeling]entry.Test plan
scripts/agents/preflight.sh --paths test/test_core_boost.py --md "<the six changed docs>"→ RESULT: PASS, every gate green:1435 passed, 14 skippedagainst1424 passed, 14 skippedat Task 4.1. The arithmetic closes: +11 passes, all of them this module (69 tests → 80), and the skip count is unchanged because this module never skipped on macOS — the whole point is that it skipped everywhere else.macOS/arm64, the capturing platform:
pytest test/test_core_boost.py -q -rs→80 passed, nothing skipped.linux/amd64, built from clean in a container:
pytest test/test_core_boost.py -q -rs→80 passed, nothing skipped — the budgets hold against a genuinely different compiled Cython, which is the thing that could not be checked before. Same run confirmstest_core_interp.pyat24 passed, 9 skipped, the follow-up's evidence.Both modes on one machine: the suite re-run with
platform.machineforced tox86_64→80 passed, so the budget branch is exercised on the capturing platform too.Guard mutations, each run from a green baseline:
OFF_PLATFORM_BUDGET → inffailstest_the_value_budget_rejects_a_real_error;OFF_PLATFORM_EDGE_ULPS → 2**60failstest_the_edge_budget_rejects_a_real_error;ON_THE_CAPTURING_PLATFORM → Falsefailstest_this_platform_gets_the_mode_it_is_supposed_to. Baseline green again after.The fused reference reproduces the port bit-for-bit on both platforms (0 misses in 1,800 tabulated points and 40,000 delta draws), while the unfused form misses 980 and 5,025 respectively. Its worst delta-function miss is 1.868e-13 — the same number, to four digits, as the Linux Cython's measured 1.8683e-13, which is the direct evidence that a compiler with no hardware FMA simply computes the unfused reference.
scripts/agents/check_doc_citations.pyover the six changed docs →out-of-range or ambiguous: NONE.Full CI matrix green (run 31638401939), and the Linux skip counts are the direct evidence the gate is no longer vacuous:
767 passed, 41 skipped797 passed, 22 skipped1424 passed, 14 skipped1435 passed, 14 skipped+30 / −19on Linux is exactly what the change predicts: the 19 previously-skipped claims now run (+19 passed, −19 skipped) plus the module's +11 net new tests. The 22 remaining Linux skips are 13 pre-existing plustest_core_interp.py's 9 — the populationdocs/followups/todo/interp-oracle-scoped-by-an-unsound-probe.mdtracks.Round-1 fixes re-gated:
scripts/agents/preflight.sh→ RESULT: PASS (1435 passed, 14 skipped in 570.32s). Every count now asserted as current re-derived on the head —test_core_{constants,special,quad,interp,dispatch}.pyat 25/65/58/33/118 unchanged,boost80,cargo test80, the Task 3.4 module pair 113.🤖 Generated with Claude Code