feat(utils): port the finite-interval quadpack subset to rust - #60
Merged
LoganAMorrison merged 3 commits intoAug 11, 2026
Conversation
Translate netlib QUADPACK's dqk15, dqk21, dqelg, dqpsrt, dqagse and dqagpe into rust/src/quad.rs, plus the scipy-shaped `quad` driver the ported kernels will call. The translation is deliberately literal -- 1-based indexing kept, every `go to` a labelled break carrying its Fortran statement number -- so it can be read against the source. Nothing under hazma/ calls it yet; the eleven live call sites move in Phases 04-06. The break-point contract turned out to belong to scipy rather than to QUADPACK: `quad` filters `points` in Python (np.unique, then strictly interior) before qagpe sees them, so both live degeneracies are discards. Designed from the QUADPACK docs instead, the four `points=[-1, 1]` call sites would have *errored*, because QUADPACK rejects a break point equal to an endpoint and scipy silently drops it. Measured against scipy 1.18.0 over 11,274 random (integrand, tolerance, limit, points) combinations: on the 4,461 that converged the port reproduced scipy's neval and last on all but 5 (0.11%), agreed on the termination flag every time, and landed within 3.6e-2 of the requested tolerance. Runs that exhaust `limit` can diverge -- the epsilon algorithm is chaotic on a non-converging sequence -- and no live shape reaches that regime. No public value changes: the only diff under hazma/ is a comment block in the non-executable _core.pyi stub. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Review round 1 on PR #60, both findings blocking and both valid. The probe took `limit: usize`, so `limit = -1` died in PyO3's conversion with OverflowError where scipy raises ValueError -- the docstring's "raises ValueError for exactly the inputs scipy raises ValueError for" was false. It now takes i64 and reproduces both of scipy's rejections: `< 1` folds onto 0 and takes the existing QuadError::LimitTooSmall path, and `> i32::MAX` raises OverflowError as scipy's own C-int conversion does. That upper guard matters beyond the contract -- `limit` sizes the qagse/qagpe workspace at 16 bytes an entry, so `limit = 10**12` was a 16 TB allocation request this machine satisfied lazily, which is a property of the platform's overcommit policy rather than of the code. Six new tests, each validity-checked by mutation. Also corrects the live call-site count. Re-derived by classifying each .pyx match as live or commented: 12 live sites, 5 of which pass `points=[-1, 1]` (the sixth such match is commented out). The first sweep grepped the paired phrases the number travelled in and then claimed a completeness its pattern could not support; re-sweeping on the claim rather than the phrasing found the cited copy and a fourteenth in test/test_core_quad.py. The class is appended to lessons.md under [sibling-copies-of-a-fixed-claim]. No public value changes: nothing under hazma/ is touched by this commit. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Review round 2 on PR #60, three findings, all valid. qagse allocated five limit-length arrays up front and qagpe six, so `limit = i32::MAX` -- which scipy accepts and round 1 made this port accept too -- reserved about 80 GiB and 88 GiB respectively before the first panel was evaluated. Rust aborts the process when an allocation fails rather than unwinding, so that could never have surfaced as a Python exception, against an exit criterion of "never panics across FFI". Round 1 capped limit at i32::MAX and left the identical hazard directly below the cap; the test it added even asserted the opposite. The arrays now start at 64 subintervals (or the initial partition, whichever is larger) and double, capped at limit + 2. Every index in both routines is at most `last`, which grows by one per iteration, so one slot of headroom suffices. Three cargo tests cover it; the one that matters asks for usize::MAX / 4096, a size no allocator can satisfy, so it discriminates on every platform rather than only where the allocator declines to overcommit. Reverting to eager sizing dies with SIGABRT. Peak RSS at limit = INT_MAX moved 18.2 -> 18.6 MB before this change, because macOS maps zero pages lazily -- the round-1 measurement could not see the reservation it was meant to rule out. Also: numerics-replacements.md held three different counts of the same fact (four, five, six); a wrap-tolerant re-sweep fixed those and a 15th copy in test_core_quad.py that straddled a line break. And the test-count records are re-derived from the final tree after both review rounds -- 58 python tests, 43 cargo units, 1212 passed / 13 skipped. No public value changes: nothing under hazma/ is touched, and the 11,274-combination scipy-agreement sweep reproduces byte-identically (4,461 converged, 5 subdivision mismatches, 8.192e-11 worst relative). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
LoganAMorrison
deleted the
claude/cython-to-rust/task-3.3-quadpack-port-qk15-qk21-qelg-qags-qagp
branch
August 11, 2026 00:44
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
qk15,qk21,qelg,qpsrt,qagse,qagpe— from the public-domain netlibFortran into
rust/src/quad.rs, plus the scipy-shapedquaddriver aPhase 04–06 kernel will actually call. PyO3-free (rules.md rule 8), and
deliberately literal: 1-based indexing preserved, every
go toalabelled
breakcarrying its Fortran statement number, so the two canbe read side by side. Provenance header per rules.md rule 5 — nothing
GSL-derived, exactly what ADR-0002 prescribes.
QUADPACK.
scipy.integrate.quadfilterspointsin Python(
np.unique, then strictly interior) beforeqagpeever sees them, soboth live degeneracies are discards:
points=[-1, 1]on[-1, 1]leaves zero break points, and a heavy mediator's entries fall outside
the thermal interval. Designed from the QUADPACK documentation instead,
the five
points=[-1, 1]call sites would have errored — QUADPACKrejects a break point equal to an endpoint. A second consequence:
points is Noneselectsqagse, "no break point survived" does not,so five of the twelve live call sites run
qagpewith an empty list.hazma/is a commentblock in the non-executable
hazma/_core.pyistub (verified:git diff origin/master -- hazma). Nothing underhazma/imports thenew code; the parity corpus ran in bit-equality mode with the skip
count unchanged at 13, which is what proves the mode.
tolerance, limit, points) combinations: on the 4,461 runs that
converged, the port reproduced scipy's
nevalandlaston all but5 (0.11%), agreed on the termination flag every time, and landed within
3.6e-2 of the requested tolerance (8.2e-11 relative at worst) —
against the 10× the exit criteria allow. Runs that exhaust
limitcandiverge, because Wynn's ε-algorithm is chaotic on a sequence that is not
converging; no live integrand shape reaches that regime, and each
asserts
ier = 0.hazma._core.quadis a test surface, joininghazma._core.specialin
cases._CORE_TEST_ONLY_MODULESunder the existing importer guard —the same mechanism Task 3.2 built, not a widened exemption.
Project
projects/cython-to-rust/— Phase 03, Task 3.3: QUADPACK port(qk15/qk21/qelg/qags/qagp).
See
projects/cython-to-rust/task-notes/phase-03/task-3.3-quadpack.mdfor detail, including the seventeen-mutation validity campaign and the
## Stale-state sweepblock.Two canonical documents are patched in this same PR: the phase file's
Task 3.3 block gains four "criteria added during execution" bullets, and
references/numerics-replacements.mdgains the measured break-pointcontract — its own instruction was to pin it empirically, so the answer
belongs there rather than only in a task note.
Review round 1 (2026-08-10)
Both blocking findings were valid and are fixed in
8adcc7e.Live call-site counts were inconsistent. Re-derived by classifying
each
.pyxmatch as live or commented: 12 live call sites (and 11commented), of which 5 pass
points=[-1, 1]— the sixthpoints=[-1, 1]match,hazma/spectra/_positron/_muon.pyx:134, iscommented out. The reviewer's count was right and mine was not. Worth
noting how the first sweep missed copies: it grepped the paired
phrases the number usually appeared in (
eleven,six of the eleven), fixed twelve occurrences, and then recorded "all twelveoccurrences were swept" — a completeness claim the pattern could not
support. Re-sweeping on the claim instead (any numeral or number
word within 40 characters of
call site/points=[-1, either order)found the copy the reviewer cited and a fourteenth in
test/test_core_quad.py:307. Both are fixed, the sweep record nowsays what the grep actually established, and the class is appended to
docs/agents/lessons.mdunder[sibling-copies-of-a-fixed-claim].The probe's error contract was inaccurate.
limit = -1died inPyO3's
usizeconversion withOverflowErrorwhere scipy raisesValueError, so the docstring's "raisesValueErrorfor exactly theinputs scipy raises
ValueErrorfor" was false.quad_pynow takesi64and reproduces both of scipy's rejections:limit < 1foldsonto
0and takes the existingQuadError::LimitTooSmallpath, andlimit > i32::MAXraisesOverflowErrorexactly as scipy's own C-intconversion does.
The upper guard turned out to matter beyond the contract:
limitsizes the
qagse/qagpeworkspace at 16 bytes an entry, solimit = 10**12was a 16 TB allocation request that this machinesatisfied lazily — whether it survives is a property of the platform's
overcommit policy, not of the code, against an exit criterion that
says "never panics across FFI". Six new tests, each validity-checked
by mutation: removing the overflow guard fails both
..._past_c_int_range_overflows...cases, and folding negatives to50instead of0fails both negative..._below_one_...cases.On the environment difference you noted — your
1206 passed, 14 skippedunder Python 3.13 / NumPy 2.5.2 against my1207 / 13is thedocumented Task 3.1 mechanism, not a discrepancy: the corpus manifest
records NumPy 2.5.1, so 2.5.2 puts
tolerances.provenanceinto budgetmode and drops exactly one test from passed to skipped. The counts differ
by exactly that one test in exactly that direction.
Review round 2 (2026-08-10)
All three blocking findings were valid; fixed in
9285fb0.Eager workspace allocation — the serious one.
qagseallocatedfive
limit-length arrays up front andqagpesix, solimit = i32::MAX(which round 1 made this port accept, matching scipy)reserved ~80 GiB and ~88 GiB before the first panel was evaluated.
Rust aborts on allocation failure rather than unwinding, so it could
never have become a Python exception. Round 1 capped
limitandleft the identical hazard directly below the cap, and the test it
added asserted the opposite. The arrays now seed at 64 subintervals
(or the initial partition) and double, capped at
limit + 2; everyindex in both routines is at most
last, which grows by one periteration. Three
cargotests, the decisive one asking forusize::MAX / 4096— a size no allocator can satisfy, so itdiscriminates on every platform rather than only where the allocator
declines to overcommit. Reverting to eager sizing dies with
memory allocation of 36028797018963976 bytes failed/ SIGABRT.Worth recording why round 1's evidence was worthless: peak RSS at
limit = INT_MAXmoved 18.2 → 18.6 MB, because macOS maps zero pageslazily. The reservation was real and the measurement could not see it.
numerics-replacements.mdself-contradictory — it held threecounts of one fact (four at :105, five at :132, six at :136). Fixed to
five. This was a second failure of the sweep method, not a straggler:
round 1's fix was "key the sweep on the claim, not the phrasing", but
that was still line-oriented, and two survivors wrapped across a
newline while a third used a synonym (
those six sitesagainst ananchor of
call site). Re-sweeping after reflowing each file to oneline found those and a fifteenth copy in
test_core_quad.py:236,which was wrong twice over. The ledger entry now says to reflow before
matching and to alternate synonyms.
Stale test-count records. Re-derived from the final tree rather
than adjusted: 58 Python tests (8 classes), 43
cargounits,1212 passed, 13 skipped. The class was wider than the two linescited — 13 stale counts across three bookkeeping files, all swept.
Re-measured after the behavior change, as a workspace change is a
behavior change: the 11,274-combination scipy-agreement sweep reproduces
byte-identically (4,461 converged, 5 subdivision mismatches,
8.192e-11 worst relative, 3.570e-2 of the requested tolerance). The fix
changed when memory is obtained, not any value, so every agreement figure
above stands unchanged.
On the unverified full-suite count: it has now run to completion here
twice at
1212 passed, 13 skipped(round 1 and round 2 preflight, 9m03sand 9m26s). If it stays out of reach in your window, that is the number
to challenge rather than accept.
Test plan
scripts/agents/preflight.sh --paths … --md …→ RESULT: PASSacross all eleven rows.
pytest -q→1212 passed, 13 skipped(+58 on Task 3.2's1154 passed, 13 skipped, all of them this task's new tests). Theskip count is unchanged, which is what proves the parity corpus
ran in bit-equality mode; confirmed independently beforehand with
tolerances.provenance(manifest)→Provenance(exact=True, detail='').pytest test/test_core_quad.py -q→58 passed in 5.10s(8 classes; population derived with
pytest --collect-only -q | awk -F'::' '/::/{print $2}' | sort | uniq -c).cargo test --manifest-path rust/Cargo.toml --no-default-features→
43 passed(27 new;grep -c '#\[test\]' rust/src/quad.rs→ 27).Table provenance. The Gauss–Kronrod literals were extracted from
the Fortran
datastatements by script, and re-checked as f64 bitpatterns by a second script that parses both sides independently of
the crate →
MISMATCHES: 0.Seventeen mutations against
rust/src/quad.rs, each appliedalone from a green baseline and reverted after, with the baseline
re-asserted green at the end. All seventeen are caught; the two that
survived the first pass (
ndin, the roundoff counters) are whyTestAdaptiveHeuristicsexists. Table in the task note.