Skip to content

Fix two unsatisfiable dependency floors, make the floors job real, and audit the plan's enforcement claims - #22

Merged
sgjholt merged 3 commits into
mainfrom
claude/lmfit-stub-floor-fidelity
Aug 7, 2026
Merged

Fix two unsatisfiable dependency floors, make the floors job real, and audit the plan's enforcement claims#22
sgjholt merged 3 commits into
mainfrom
claude/lmfit-stub-floor-fidelity

Conversation

@sgjholt

@sgjholt sgjholt commented Aug 7, 2026

Copy link
Copy Markdown
Owner

Completes the floors CI change already applied to main. Main's floors job is red until this merges — it now calls tools/check_floors.py, which lands here.

The job has never tested the floors

- run: uv pip install --resolution lowest-direct -e ".[dev]"
- run: uv run pytest -m "not dataset" -q

uv run re-resolves the project before running. With no committed lock file it installed the newest of everything and discarded the line above it, so the job was a duplicate of the ubuntu/3.11 matrix entry — green for that reason, for as long as it has existed.

Its own comment read "Exercise the declared minimums. CI otherwise installs the newest of everything, so the floors are only ever tested by a user", describing precisely what continued to happen.

Reproduced rather than inferred: uv run prints Uninstalled 2 packages / Installed 2 packages and comes back with lmfit 1.3.4 and numpy 2.4.6 where the previous step had put 1.2.0 and 2.0.0.

Two declared floors were wrong

Both broke real behaviour, and each was bisected rather than guessed.

declared measured what fails below it
lmfit>=1.2 >=1.3 1.2.x calls np.asfarray, removed in NumPy 2.0. lmfit>=1.2 and numpy>=2.0 were never both satisfiable; every fit raised AttributeError and 15 tests fail. 1.2.0, 1.2.1, 1.2.2 all fail; 1.3.0 works.
scipy>=1.13 >=1.15 1.15 reimplemented scipy.optimize.nnls. Below it the vendored qiinv inversion does not converge for every input scale, and the quadratic estimator's peak recovery moves between 0.53 and 1.02 for the same signal at different amplitudes. 1.13 and 1.14 fail; 1.15 passes.

The scipy one is the better illustration. test_quadratic.py already asserts scale invariance, with the message "a dimensionless ratio that moves with amplitude is a numerical-range problem, not a property of the estimator". It was right, and it had never run against the floor.

Both raised to what the suite was measured to pass on: 523 pass at the corrected floors, through the fixed job end to end.

The guard matters more than the floors

tools/check_floors.py asserts the installed versions are the declared minimums, and the job runs it before pytest. This is the point of the change rather than a nicety: a floors job quietly testing the newest versions looks exactly like one that works, so the mismatch has to be an error rather than something a reader might notice.

Verified both directions — it passes against a floor environment and fails against the dev environment — and without --no-sync uv re-syncs and the guard turns the job red.

Its first version was itself wrong: it called 2.0 and 2.0.0 different versions and failed against everything. Both sides are zero-padded to equal length now, not truncated to the shorter, which would have accepted 2.0.5 for a floor of 2.0.

uv.lock is gitignored. Not committing one is deliberate — a lock file is exactly what would make --resolution lowest-direct meaningless again.

A stub that over-promised, and the check that missed it

stubs/lmfit/model.pyi declared Model.fit(..., coerce_farray=...), which exists on lmfit 1.3 and not on 1.2. Removed — a stub has to be true across the supported range, and nothing here passes it.

The more useful half is why tests/test_stubs.py did not catch it. It skipped any function whose real signature ends in *args/**kwargs, reasoning that a stub may stop early there. That reasoning is right, and is the whole approach — declare what we use, no more. But may stop early is not may invent, and **kwargs upstream makes an invented parameter worse: the call is swallowed at runtime, so the mistake surfaces as a silently ignored argument rather than a TypeError.

The check now separates the two claims — every declared name must exist, and the ones that do must be in order — through one helper shared by the ObsPy and lmfit tests instead of two near-copies. Verified both ways: restoring coerce_farray passes against installed lmfit 1.3.4 and fails against 1.2.0 naming the parameter. The old check passed against both.

Audit of the plan's enforcement claims (new §6.6)

This is the third time a claim in REFACTOR_PLAN.md has described a mechanism that did not exist — after "CI can assert it never grows" for the mypy backlog, and the ignore_missing_imports recommendation that would have left every ObsPy and lmfit object as Any. One shape each time: an intention stated in the plan, restated as fact in a comment, with nothing behind it.

So the rest were checked rather than read. Four more were false, and are corrected in place so the document does not have to be read alongside its own errata:

Claim Reality
"pre-commit runs ruff, mypy, nbstripout, check-added-large-files" mypy is not a hook and never was. Defensible as a decision — it needs the project environment, and the typecheck job covers it — but not a hook that exists.
"Conventional Commits, commitlint-enforced" No such hook. Followed by hand.
"sphinx-build -W … an undocumented public symbol fails the build" Wrong even once the docs build exists. -W promotes warnings Sphinx already emits, and autodoc emits none for a symbol it was never asked to document. Needs sphinx.ext.coverage or nitpicky.
test.yml = one matrix doing ruff, mypy, pytest Five jobs: lint, typecheck, test, floors, plus build.yml.

One case that is neither true nor false, and is filed separately for that reason. The plan says "regression tests pin an explicit config file, never the defaults, so changing a default cannot silently move a golden test." Nothing pins a config — the golden tests read the shipped defaults. But the property was tested rather than the mechanism: bumping smoothing.n_bins from 151 to 158 makes 9 of 25 golden tests fail, loudly, because the golden values are frozen in a committed JSON file. The protection exists by a different route than claimed. The residual gap is real but narrower: a regenerated reference silently adopts whatever defaults were current, since the file records no config.

Three claims checked and holding: the Parseval contract parametrised over ESTIMATORS, PLOT_COLUMNS having exactly one definition, and the registry-wide noise-rotation property test.

One passage is simply stale: §5.2 still describes BW_METHOD/ROT_METHOD as globals in spectral, "a shell over core", with sp.ROT_METHOD = 1 as a live escape hatch. Both spectral.py and _config_legacy.py were deleted in phase 2.

The audit also surfaced something in the other direction: the pre-commit config has a local no-session-links commit-msg hook that appeared nowhere in the plan, and it is the only mechanism keeping private session URLs out of a public repository's history. Now written down.

Adding §6.6 moved branch layout to §6.7. All 21 §6.x cross-references were re-resolved against the headings programmatically rather than by eye; four pointed at the wrong section afterwards and are fixed.

Checks

  • All 10 CI jobs green on the head commit, floors included
  • Floors job replayed locally end to end: all 16 declared floors are the installed versions, then 523 passed, 5 skipped
  • Dev environment: 543 passed, 1 skipped; ruff check, ruff format --check, mypy clean

claude added 3 commits August 7, 2026 18:23
…ext one

`stubs/lmfit/model.pyi` declared `Model.fit(..., coerce_farray=...)`. That
exists on lmfit 1.3 and not on 1.2.0, which is the floor `pyproject.toml`
declares. A stub has to be true across the supported range, and nothing here
passes the argument, so it is gone.

The more useful half is why the stub test did not catch it. It skipped any
function whose real signature ends in `*args`/`**kwargs`, on the reasoning
that the stub is allowed to stop early there — which is right, and is the
whole approach: declare the parameters we use and no more.

But "may stop early" is not "may invent". `**kwargs` upstream makes an
invented parameter *worse*, not better: the call is swallowed at runtime, so
the mistake surfaces as a silently ignored argument rather than a TypeError.
The check now separates the two claims — every declared name must exist, and
the order of the ones that do must match — and applies to lmfit and ObsPy
through one helper instead of two near-copies.

Verified both ways rather than assumed: restoring `coerce_farray` passes
against the installed lmfit 1.3.4 and fails against 1.2.0 with the message
naming the parameter. The old check passed against both.

Found while confirming the floors CI job, which is a separate problem and not
addressed here: `uv run` re-syncs the project environment, so that job has
been discarding the `--resolution lowest-direct` install and testing the
newest of everything. What it would find is that `lmfit>=1.2` and
`numpy>=2.0` cannot both be satisfied — lmfit below 1.3.0 calls
`np.asfarray`, removed in NumPy 2.0.
The `floors` job has never tested the floors. It runs

    uv pip install --resolution lowest-direct -e ".[dev]"
    uv run pytest -m "not dataset" -q

and `uv run` re-resolves the project first. With no committed lock file it
installs the newest of everything and discards the line above it, so the job
is a duplicate of the ubuntu/3.11 matrix entry — green for that reason, for
as long as it has existed. Its own comment says "CI otherwise installs the
newest of everything, so the floors are only ever tested by a user",
describing exactly what continued to happen.

Reproduced rather than inferred: `uv run` prints `Uninstalled 2 packages /
Installed 2 packages` and comes back with lmfit 1.3.4 and numpy 2.4.6 where
the previous step had put 1.2.0 and 2.0.0.

Two declared floors were wrong, and both broke real behaviour:

- `lmfit>=1.2` could never be satisfied alongside `numpy>=2.0`. lmfit below
  1.3.0 calls `np.asfarray`, removed in NumPy 2.0, so every fit raised
  AttributeError — 15 tests fail on the declared floor. Bisected: 1.2.0,
  1.2.1 and 1.2.2 all fail; 1.3.0 works.

- `scipy>=1.13` silently broke the quadratic multitaper. 1.15 reimplemented
  `scipy.optimize.nnls`; below it the vendored `qiinv` inversion does not
  converge for every input scale, and peak recovery moves between 0.53 and
  1.02 for the same signal at different amplitudes. `test_quadratic.py`'s own
  scale-invariance test catches it — given a job that installs the floor.
  Bisected: 1.13 and 1.14 fail, 1.15 passes.

Both raised to what the suite was measured to pass on: 523 pass at the
corrected floors, through the fixed job end to end.

`tools/check_floors.py` asserts the installed versions *are* the declared
minimums. That guard matters more than the floors themselves, because the
failure mode is invisible by construction — a floors job testing the newest
versions looks exactly like one that works. Verified both ways: it passes
against a floor environment and fails against the dev environment.

Its first version called `2.0` and `2.0.0` different versions and failed
against everything. Floors are written at whatever precision reads well, so
both sides are zero-padded to equal length rather than truncated to the
shorter, which would accept 2.0.5 for a floor of 2.0.

The workflow half — `--no-sync` on both `uv run` lines — is not in this
commit: pushing `.github/workflows/test.yml` needs a permission this account
does not have, so it was applied directly to main and this branch is rebased
on top of it. That ordering leaves main's `floors` job red until this merges,
because it now calls a script main does not yet have.

`uv.lock` is gitignored. Not committing one is deliberate: a lock file is
what would make `--resolution lowest-direct` meaningless again.
…false

Three claims in this plan have now turned out to describe mechanisms that did
not exist — "CI can assert it never grows" for the mypy backlog, the
`ignore_missing_imports` recommendation that would have made every ObsPy and
lmfit object `Any`, and the `floors` job that discarded its own
lowest-direct install. One shape three times: an intention stated in the plan,
restated as fact in a comment, with nothing behind it.

So the rest were checked rather than read. New §6.6 records the result; the
sections it corrects are fixed in place rather than annotated, so the document
does not have to be read alongside its own errata.

False, and corrected:

- "pre-commit runs ruff, **mypy**, nbstripout, check-added-large-files" — mypy
  is not a hook and never was. Defensible (it needs the project environment
  and the `typecheck` job covers it), but the plan should not claim a hook
  that is not installed. The real set is listed now, including the local
  `no-session-links` commit-msg hook — which appeared nowhere in this document
  and is the only thing keeping private session URLs out of a public
  repository's history.

- "Conventional Commits, **`commitlint`-enforced**" — no such hook. The
  convention is followed by hand.

- "`sphinx-build -W` … an **undocumented public symbol** fails the build" —
  wrong even once the docs build exists. `-W` promotes warnings Sphinx already
  emits, and autodoc emits none for a symbol it was never asked to document.
  That needs `sphinx.ext.coverage` or `nitpicky`.

- The `test.yml` row described one matrix step doing ruff, mypy and pytest.
  It is five jobs — `lint`, `typecheck`, `test`, `floors`, plus `build.yml`.

One claim whose mechanism is absent but whose property holds, which is worth
separating rather than filing under either heading: "regression tests pin an
explicit config file, never the defaults". Nothing pins a config; the golden
tests read the shipped defaults. But the property was tested directly — bump
`smoothing.n_bins` 151 -> 158 and 9 of 25 golden tests fail, loudly — because
the golden values are frozen in a committed JSON file. The residual gap is
real but narrower: a *regenerated* reference silently adopts whatever defaults
were current, since the file records no config.

Three claims checked and holding: the Parseval contract parametrised over
`ESTIMATORS`, `PLOT_COLUMNS` having exactly one definition, and the
registry-wide noise-rotation property test.

One passage is simply stale — §5.2 still describes `BW_METHOD`/`ROT_METHOD` as
globals in `spectral`, "a shell over `core`", with `sp.ROT_METHOD = 1` as a
live escape hatch. Both `spectral.py` and `_config_legacy.py` were deleted in
phase 2.

The audit needed §6.6, so branch layout moved to §6.7. All 21 `§6.x`
cross-references were re-resolved against the headings; four pointed at the
wrong section afterwards and are fixed.
@sgjholt sgjholt changed the title Raise two dependency floors that were never installable, and make the floors job actually test them Fix two unsatisfiable dependency floors, make the floors job real, and audit the plan's enforcement claims Aug 7, 2026
@sgjholt
sgjholt merged commit 34f1c5d into main Aug 7, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants