Skip to content

feat(utils): port the finite-interval quadpack subset to rust - #60

Merged
LoganAMorrison merged 3 commits into
masterfrom
claude/cython-to-rust/task-3.3-quadpack-port-qk15-qk21-qelg-qags-qagp
Aug 11, 2026
Merged

feat(utils): port the finite-interval quadpack subset to rust#60
LoganAMorrison merged 3 commits into
masterfrom
claude/cython-to-rust/task-3.3-quadpack-port-qk15-qk21-qelg-qags-qagp

Conversation

@LoganAMorrison

@LoganAMorrison LoganAMorrison commented Aug 10, 2026

Copy link
Copy Markdown
Owner

Summary

  • Ports the QUADPACK subset hazma integrates withqk15, qk21,
    qelg, qpsrt, qagse, qagpe — from the public-domain netlib
    Fortran into rust/src/quad.rs, plus the scipy-shaped quad driver a
    Phase 04–06 kernel will actually call. PyO3-free (rules.md rule 8), and
    deliberately literal: 1-based indexing preserved, every go to a
    labelled break carrying its Fortran statement number, so the two can
    be read side by side. Provenance header per rules.md rule 5 — nothing
    GSL-derived, exactly what ADR-0002 prescribes.
  • The break-point contract turned out to belong to scipy, not to
    QUADPACK.
    scipy.integrate.quad filters points in Python
    (np.unique, then strictly interior) before qagpe ever sees them, so
    both live degeneracies are discards: points=[-1, 1] on [-1, 1]
    leaves zero break points, and a heavy mediator's entries fall outside
    the thermal interval. Designed from the QUADPACK documentation instead,
    the five points=[-1, 1] call sites would have errored — QUADPACK
    rejects a break point equal to an endpoint. A second consequence:
    points is None selects qagse, "no break point survived" does not,
    so five of the twelve live call sites run qagpe with an empty list.
  • No public value changes. The only diff under hazma/ is a comment
    block in the non-executable hazma/_core.pyi stub (verified:
    git diff origin/master -- hazma). Nothing under hazma/ imports the
    new code; the parity corpus ran in bit-equality mode with the skip
    count unchanged at 13, which is what proves the mode.
  • Measured against scipy 1.18.0 over 11,274 random (integrand,
    tolerance, limit, points) combinations: on the 4,461 runs that
    converged, the port reproduced scipy's neval and last on all but
    5 (0.11%), agreed on the termination flag every time, and landed within
    3.6e-2 of the requested tolerance (8.2e-11 relative at worst) —
    against the 10× the exit criteria allow. Runs that exhaust limit can
    diverge, because Wynn's ε-algorithm is chaotic on a sequence that is not
    converging; no live integrand shape reaches that regime, and each
    asserts ier = 0.
  • hazma._core.quad is a test surface, joining hazma._core.special
    in cases._CORE_TEST_ONLY_MODULES under the existing importer guard —
    the same mechanism Task 3.2 built, not a widened exemption.

Project

projects/cython-to-rust/ — Phase 03, Task 3.3: QUADPACK port
(qk15/qk21/qelg/qags/qagp).

See projects/cython-to-rust/task-notes/phase-03/task-3.3-quadpack.md
for detail, including the seventeen-mutation validity campaign and the
## Stale-state sweep block.

Two canonical documents are patched in this same PR: the phase file's
Task 3.3 block gains four "criteria added during execution" bullets, and
references/numerics-replacements.md gains the measured break-point
contract — its own instruction was to pin it empirically, so the answer
belongs there rather than only in a task note.

Review round 1 (2026-08-10)

Both blocking findings were valid and are fixed in 8adcc7e.

  1. Live call-site counts were inconsistent. Re-derived by classifying
    each .pyx match as live or commented: 12 live call sites (and 11
    commented), of which 5 pass points=[-1, 1] — the sixth
    points=[-1, 1] match, hazma/spectra/_positron/_muon.pyx:134, is
    commented out. The reviewer's count was right and mine was not. Worth
    noting how the first sweep missed copies: it grepped the paired
    phrases the number usually appeared in (eleven, six of the eleven), fixed twelve occurrences, and then recorded "all twelve
    occurrences were swept" — a completeness claim the pattern could not
    support. Re-sweeping on the claim instead (any numeral or number
    word within 40 characters of call site / points=[-1, either order)
    found the copy the reviewer cited and a fourteenth in
    test/test_core_quad.py:307. Both are fixed, the sweep record now
    says what the grep actually established, and the class is appended to
    docs/agents/lessons.md under [sibling-copies-of-a-fixed-claim].

  2. The probe's error contract was inaccurate. limit = -1 died in
    PyO3's usize conversion with OverflowError where scipy raises
    ValueError, so the docstring's "raises ValueError for exactly the
    inputs scipy raises ValueError for" was false. quad_py now takes
    i64 and reproduces both of scipy's rejections: limit < 1 folds
    onto 0 and takes the existing QuadError::LimitTooSmall path, and
    limit > i32::MAX raises OverflowError exactly as scipy's own C-int
    conversion does.

    The upper guard turned out to matter beyond the contract: limit
    sizes the qagse/qagpe workspace at 16 bytes an entry, so
    limit = 10**12 was a 16 TB allocation request that this machine
    satisfied lazily — whether it survives is a property of the platform's
    overcommit policy, not of the code, against an exit criterion that
    says "never panics across FFI". Six new tests, each validity-checked
    by mutation: removing the overflow guard fails both
    ..._past_c_int_range_overflows... cases, and folding negatives to
    50 instead of 0 fails both negative ..._below_one_... cases.

On the environment difference you noted — your 1206 passed, 14 skipped under Python 3.13 / NumPy 2.5.2 against my 1207 / 13 is the
documented Task 3.1 mechanism, not a discrepancy: the corpus manifest
records NumPy 2.5.1, so 2.5.2 puts tolerances.provenance into budget
mode and drops exactly one test from passed to skipped. The counts differ
by exactly that one test in exactly that direction.

Review round 2 (2026-08-10)

All three blocking findings were valid; fixed in 9285fb0.

  1. Eager workspace allocation — the serious one. qagse allocated
    five limit-length arrays up front and qagpe six, so limit = i32::MAX (which round 1 made this port accept, matching scipy)
    reserved ~80 GiB and ~88 GiB before the first panel was evaluated.
    Rust aborts on allocation failure rather than unwinding, so it could
    never have become a Python exception. Round 1 capped limit and
    left the identical hazard directly below the cap
    , and the test it
    added asserted the opposite. The arrays now seed at 64 subintervals
    (or the initial partition) and double, capped at limit + 2; every
    index in both routines is at most last, which grows by one per
    iteration. Three cargo tests, the decisive one asking for
    usize::MAX / 4096 — a size no allocator can satisfy, so it
    discriminates on every platform rather than only where the allocator
    declines to overcommit. Reverting to eager sizing dies with
    memory allocation of 36028797018963976 bytes failed / SIGABRT.

    Worth recording why round 1's evidence was worthless: peak RSS at
    limit = INT_MAX moved 18.2 → 18.6 MB, because macOS maps zero pages
    lazily. The reservation was real and the measurement could not see it.

  2. numerics-replacements.md self-contradictory — it held three
    counts of one fact (four at :105, five at :132, six at :136). Fixed to
    five. This was a second failure of the sweep method, not a straggler:
    round 1's fix was "key the sweep on the claim, not the phrasing", but
    that was still line-oriented, and two survivors wrapped across a
    newline while a third used a synonym (those six sites against an
    anchor of call site). Re-sweeping after reflowing each file to one
    line found those and a fifteenth copy in test_core_quad.py:236,
    which was wrong twice over. The ledger entry now says to reflow before
    matching and to alternate synonyms.

  3. Stale test-count records. Re-derived from the final tree rather
    than adjusted: 58 Python tests (8 classes), 43 cargo units,
    1212 passed, 13 skipped. The class was wider than the two lines
    cited — 13 stale counts across three bookkeeping files, all swept.

Re-measured after the behavior change, as a workspace change is a
behavior change: the 11,274-combination scipy-agreement sweep reproduces
byte-identically (4,461 converged, 5 subdivision mismatches,
8.192e-11 worst relative, 3.570e-2 of the requested tolerance). The fix
changed when memory is obtained, not any value, so every agreement figure
above stands unchanged.

On the unverified full-suite count: it has now run to completion here
twice at 1212 passed, 13 skipped (round 1 and round 2 preflight, 9m03s
and 9m26s). If it stays out of reach in your window, that is the number
to challenge rather than accept.

Test plan

  • scripts/agents/preflight.sh --paths … --md …RESULT: PASS
    across all eleven rows.

    PASS   black --check           test/parity/cases.py test/parity/test_parity.py test/test_core_quad.py
    PASS   isort --check-only      test/parity/cases.py test/parity/test_parity.py test/test_core_quad.py
    PASS   ruff check              test/parity/cases.py test/parity/test_parity.py test/test_core_quad.py
    PASS   cargo fmt --check       rust/
    PASS   cargo clippy            rust/
    PASS   cargo test              rust/
    PASS   pytest                  1212 passed, 13 skipped, 5 warnings in 543.09s (0:09:03)
    PASS   import hazma            version 2.1.0
    PASS   markdownlint            <6 changed docs>
    SKIP   version bump            not a closing PR (pass --closing)
    PASS   forbidden tokens        none added
    RESULT: PASS
    
  • pytest -q1212 passed, 13 skipped (+58 on Task 3.2's
    1154 passed, 13 skipped, all of them this task's new tests). The
    skip count is unchanged, which is what proves the parity corpus
    ran in bit-equality mode; confirmed independently beforehand with
    tolerances.provenance(manifest)Provenance(exact=True, detail='').

  • pytest test/test_core_quad.py -q58 passed in 5.10s
    (8 classes; population derived with
    pytest --collect-only -q | awk -F'::' '/::/{print $2}' | sort | uniq -c).

  • cargo test --manifest-path rust/Cargo.toml --no-default-features
    43 passed (27 new; grep -c '#\[test\]' rust/src/quad.rs → 27).

  • Table provenance. The Gauss–Kronrod literals were extracted from
    the Fortran data statements by script, and re-checked as f64 bit
    patterns by a second script that parses both sides independently of
    the crate → MISMATCHES: 0.

  • Seventeen mutations against rust/src/quad.rs, each applied
    alone from a green baseline and reverted after, with the baseline
    re-asserted green at the end. All seventeen are caught; the two that
    survived the first pass (ndin, the roundoff counters) are why
    TestAdaptiveHeuristics exists. Table in the task note.

LoganAMorrison and others added 3 commits August 10, 2026 10:57
Translate netlib QUADPACK's dqk15, dqk21, dqelg, dqpsrt, dqagse and
dqagpe into rust/src/quad.rs, plus the scipy-shaped `quad` driver the
ported kernels will call. The translation is deliberately literal --
1-based indexing kept, every `go to` a labelled break carrying its
Fortran statement number -- so it can be read against the source.
Nothing under hazma/ calls it yet; the eleven live call sites move in
Phases 04-06.

The break-point contract turned out to belong to scipy rather than to
QUADPACK: `quad` filters `points` in Python (np.unique, then strictly
interior) before qagpe sees them, so both live degeneracies are
discards. Designed from the QUADPACK docs instead, the four
`points=[-1, 1]` call sites would have *errored*, because QUADPACK
rejects a break point equal to an endpoint and scipy silently drops it.

Measured against scipy 1.18.0 over 11,274 random (integrand, tolerance,
limit, points) combinations: on the 4,461 that converged the port
reproduced scipy's neval and last on all but 5 (0.11%), agreed on the
termination flag every time, and landed within 3.6e-2 of the requested
tolerance. Runs that exhaust `limit` can diverge -- the epsilon
algorithm is chaotic on a non-converging sequence -- and no live shape
reaches that regime.

No public value changes: the only diff under hazma/ is a comment block
in the non-executable _core.pyi stub.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Review round 1 on PR #60, both findings blocking and both valid.

The probe took `limit: usize`, so `limit = -1` died in PyO3's conversion
with OverflowError where scipy raises ValueError -- the docstring's
"raises ValueError for exactly the inputs scipy raises ValueError for"
was false. It now takes i64 and reproduces both of scipy's rejections:
`< 1` folds onto 0 and takes the existing QuadError::LimitTooSmall path,
and `> i32::MAX` raises OverflowError as scipy's own C-int conversion
does. That upper guard matters beyond the contract -- `limit` sizes the
qagse/qagpe workspace at 16 bytes an entry, so `limit = 10**12` was a
16 TB allocation request this machine satisfied lazily, which is a
property of the platform's overcommit policy rather than of the code.
Six new tests, each validity-checked by mutation.

Also corrects the live call-site count. Re-derived by classifying each
.pyx match as live or commented: 12 live sites, 5 of which pass
`points=[-1, 1]` (the sixth such match is commented out). The first
sweep grepped the paired phrases the number travelled in and then
claimed a completeness its pattern could not support; re-sweeping on the
claim rather than the phrasing found the cited copy and a fourteenth in
test/test_core_quad.py. The class is appended to lessons.md under
[sibling-copies-of-a-fixed-claim].

No public value changes: nothing under hazma/ is touched by this commit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Review round 2 on PR #60, three findings, all valid.

qagse allocated five limit-length arrays up front and qagpe six, so
`limit = i32::MAX` -- which scipy accepts and round 1 made this port
accept too -- reserved about 80 GiB and 88 GiB respectively before the
first panel was evaluated. Rust aborts the process when an allocation
fails rather than unwinding, so that could never have surfaced as a
Python exception, against an exit criterion of "never panics across
FFI". Round 1 capped limit at i32::MAX and left the identical hazard
directly below the cap; the test it added even asserted the opposite.

The arrays now start at 64 subintervals (or the initial partition,
whichever is larger) and double, capped at limit + 2. Every index in
both routines is at most `last`, which grows by one per iteration, so
one slot of headroom suffices. Three cargo tests cover it; the one that
matters asks for usize::MAX / 4096, a size no allocator can satisfy, so
it discriminates on every platform rather than only where the allocator
declines to overcommit. Reverting to eager sizing dies with SIGABRT.

Peak RSS at limit = INT_MAX moved 18.2 -> 18.6 MB before this change,
because macOS maps zero pages lazily -- the round-1 measurement could
not see the reservation it was meant to rule out.

Also: numerics-replacements.md held three different counts of the same
fact (four, five, six); a wrap-tolerant re-sweep fixed those and a 15th
copy in test_core_quad.py that straddled a line break. And the test-count
records are re-derived from the final tree after both review rounds --
58 python tests, 43 cargo units, 1212 passed / 13 skipped.

No public value changes: nothing under hazma/ is touched, and the
11,274-combination scipy-agreement sweep reproduces byte-identically
(4,461 converged, 5 subdivision mismatches, 8.192e-11 worst relative).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@LoganAMorrison
LoganAMorrison merged commit feae660 into master Aug 11, 2026
8 checks passed
@LoganAMorrison
LoganAMorrison deleted the claude/cython-to-rust/task-3.3-quadpack-port-qk15-qk21-qelg-qags-qagp branch August 11, 2026 00:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant