Skip to content

Add a bracketed scalar rootfind node, kept internal for now - #5729

Draft
MarcBerliner wants to merge 12 commits into
mainfrom
claude/brent-node
Draft

Add a bracketed scalar rootfind node, kept internal for now#5729
MarcBerliner wants to merge 12 commits into
mainfrom
claude/brent-node

Conversation

@MarcBerliner

@MarcBerliner MarcBerliner commented Aug 20, 2026

Copy link
Copy Markdown
Member

Adds an expression-tree node that solves residual == 0 for its unknown over a bracket, plus the native CasADi rootfinder plugin it runs on.

The solve runs inside the CasADi graph, not in Python: the node converts to a rootfinder using a "brent" plugin registered by pybammsolvers. An expression containing one costs no extra Python frames per evaluation, and survives codegen to C.

Brent needs only a sign change over the bracket, so it converges where a Newton iteration stalls or leaves the domain, and the answer cannot leave the bracket. Derivatives come from CasADi's implicit function theorem, exactly.

Deliberately not public

The node is pybamm._Brent, with no docs page and no changelog entry. It is reachable from the pybamm namespace the way _BaseAverage already is, and nothing about it is a commitment yet.

There are no in-tree consumers, and the constructor is going to change: the residual is currently a symbol the caller names and must keep unique by hand, and that wants to become a closed sub-expression with positional arguments — a shape the Rust tape backend in #5732 independently needs, and which Brent presently reconstructs by hand via casadi.symvar and a scoped cache. Cheaper to withhold the name now than to break it later. #5730 is the first real consumer; the public name can follow once it has proven the shape.

sto = pybamm._BrentUnknown("stoichiometry")
node = pybamm._Brent(param.n.prim.U(sto, T) - voltage, sto, (0, 1))

Building the plugin

It is built out of tree, so CMake supplies four things an in-tree CasADi plugin gets for free. brent.hpp opens with this list; briefly:

  1. The internal headers. rootfinder_impl.hpp and five siblings are not installed by CasADi (INSTALL_INTERNAL_HEADERS is off by default) and are LGPL-3.0-or-later, so they are staged at configure time from the sdist rather than vendored into this BSD-3-Clause tree. -DCASADI_SOURCE_DIR=<tree> builds offline.
  2. An ABI check. Those headers are not ABI-stable, so the linked CasADi's version is read from its own installed casadi/config.h — the one source present on every discovery path, including Windows/vcpkg and conda, which have no interpreter to ask. That version is also what selects the sdist, so the pin in pyproject.toml is the only one, and -DCASADI_INTERNAL_VERSION= is an override whose only job now is to fail loudly when it disagrees with the linked library. The staging directory is keyed on the version, so a bump cannot find the previous release's headers in place and skip restaging.
  3. The compile flags CasADi itself used, from the same header. CASADI_WITH_THREADSAFE_SYMBOLICS adds a static mutex to Rootfinder, so guessing it would be an ABI break rather than a warning.
  4. The stringified iteration. brent_impl.hpp is compiled and stringified into brent_impl_str, so codegen emits the same text this file compiles and the two paths cannot drift.

Not available on Windows

The CasADi wheel for Windows is built with MinGW — it ships libstdc++-6.dll and .dll.a import libraries, and its own config.h names the compiler as x86_64-w64-mingw32.shared.posix-g++. pybammsolvers is built with MSVC against vcpkg's CasADi. Brent subclasses casadi::Rootfinder, so the two cannot be linked across that ABI, and the plugin registered inside our extension is invisible to the CasADi Python calls: the process holds two copies with two plugin registries.

Converting a node to CasADi there raises a NotImplementedError naming the cause rather than CasADi's Plugin 'brent' is not found, evaluate() still works via SciPy, and both test modules skip on win32. Closing the gap means building the plugin as a separate MinGW DLL against the wheel's libcasadi.dll.a — a second toolchain in the Windows job — or landing it upstream in CasADi, which has no bracketed rootfinder of its own.

Notes

  • The plugin caches its last solve, keyed on every input but the guess. Without it a Brent nested inside another re-solves on every enclosing iteration, which compounds per level. Covered on both the interpreted and the generated path. The cache scratch lives on the solver's memory object, which CasADi hands out per concurrent evaluation, so a threaded map over one Function does not share it.
  • The bracket is tested by sign rather than by fa * fb <= 0, which underflows to zero for two residuals near the underflow limit and reads as a sign change.
  • Exhausting max_iter is reported as a failure, not a root, on both the CasADi and the NumPy path. Likewise an empty bracket: the NumPy path used to swallow both into a quiet NaN, along with the tree probes the blanket except was meant to absorb.
  • Nothing the conversion emits is named after the node's id. Ids are a per-process hash and the names reach fn.serialize(), the AOT compile cache key, so a model holding a rootfind used to miss the on-disk cache in every process: measured over three fresh interpreters, a Brent-free function keyed to 4da87061f2ba3c82 each time while one holding a Brent keyed to 5229b804…, 0d83e675…, 277998d7…. Now 3e26c07750303134 each time. The identity was never needed, since CasADi names generated code positionally and the conversion cache — keyed on the id, which is where identity belongs — already gives two Brents differing only in tolerance their own rootfinder. A test pins the key across processes.
  • Also fixes an unrelated test bug found on the way: test_pybamm_import unloaded every pybamm module from sys.modules and never restored them, leaving a second set of classes behind so isinstance failed across the boundary for anything sharing an xdist worker afterwards.

Tests: 4207 unit, 82 pybammsolvers.

The composite electrode SOH solver built on this is #5730.

🤖 Generated with Claude Code

A `Brent` node solves `residual == 0` for its unknown over a bracket inside
the CasADi graph rather than in Python: it converts to a `rootfinder` using a
native "brent" plugin registered by pybammsolvers, so an expression containing
one costs no extra Python frames per evaluation and survives codegen to C.

Brent needs only a sign change over the bracket, so it converges where a
Newton iteration stalls or leaves the domain, and the answer cannot leave the
bracket. Derivatives come from CasADi's implicit function theorem.

The plugin is built out of tree, which needs four things from CMake that an
in-tree CasADi plugin gets for free: the internal headers staged from a pinned
sdist rather than vendored, since CasADi does not install them and they are
LGPL; an ABI check reading the linked version from CasADi's own config.h; the
compile flags CasADi itself used, one of which adds a static mutex to
Rootfinder; and the iteration stringified so codegen emits the text it
compiles. brent.hpp says so at the top.

Also fixes an unrelated test bug found on the way: test_pybamm_import unloaded
every pybamm module from sys.modules and never restored them, leaving a second
set of classes behind so `isinstance` failed across the boundary.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@MarcBerliner
MarcBerliner marked this pull request as draft August 20, 2026 14:03
MarcBerliner and others added 4 commits August 20, 2026 10:14
cibuildwheel's test env installs only pybammsolvers[dev], which has no
scipy, so the module-level `from scipy.optimize import brentq` failed
collection and took every wheel job down with exit code 2.

The residual assertion already pins the root, and which root a bracketed
solve lands on for the non-monotone case was never a contract, so the
brentq comparison bought nothing that survived being unrunnable.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Codacy runs cppcheck over each file in the diff on its own, so brent.hpp is
analysed without brent.cpp and all seven of its members come back unused.
Suppress the check for the header rather than annotating members one by one.

The variableScope finding is real: `tol`, `xm`, `step` and the interpolation
scratch each live for a single iteration, so declare them in the block that
uses them. One declaration block per scope still satisfies the C the text is
stringified into.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The staged internal headers include `options.hpp` unqualified. When that does
not resolve next to the file asking for it, MSVC widens the search to the
include stack, reaches `idaklu_source/`, and answers with our own Options.hpp
-- equal to `options.hpp` on a case-insensitive filesystem, and no declaration
of `casadi::Options` in sight. Every internal header that names `Options` then
fails to parse, starting at function_internal.hpp:117.

Staging CasADi's copy puts it in the directory the including file is in, which
is the first place either compiler looks, so the collision never comes up. The
guard is shared with the installed copy, so a translation unit that pulls in
both still sees one definition, and the config.h version check already keeps
the two identical.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The plugin registers into whichever CasADi is linked into the pybammsolvers
extension. On Linux and macOS that is the casadi wheel's own libcasadi, so the
registration is visible to the CasADi Python calls. On Windows the extension is
built with MSVC against vcpkg's CasADi while the casadi wheel is built with
MinGW -- it ships libstdc++-6.dll and .dll.a import libraries -- so the two hold
separate copies of CasADi, and separate plugin registries. Registering ours
cannot reach Python's, which falls back to loading
libcasadi_rootfinder_brent.dll and fails with WIN32 error 126.

Closing that gap needs the plugin built with MinGW against the wheel's
libcasadi.dll.a, a second toolchain in the Windows job. Until then it is
unavailable there, so skip both test modules on win32 -- 23 in pybammsolvers,
17 in pybamm -- and raise a NotImplementedError naming the cause when a Brent
node is converted to CasADi without the plugin, in place of CasADi's
"Plugin 'brent' is not found". evaluate() is unaffected: it solves with SciPy.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@codecov

codecov Bot commented Aug 20, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 91.37931% with 15 lines in your changes missing coverage. Please review.
✅ Project coverage is 98.08%. Comparing base (3e8d682) to head (fa12f8d).
⚠️ Report is 6 commits behind head on main.

Files with missing lines Patch % Lines
...ackages/pybamm/src/pybamm/expression_tree/brent.py 91.76% 14 Missing ⚠️
...ybamm/src/pybamm/discretisations/discretisation.py 66.66% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5729      +/-   ##
==========================================
- Coverage   98.20%   98.08%   -0.13%     
==========================================
  Files         340      341       +1     
  Lines       32743    32916     +173     
==========================================
+ Hits        32156    32285     +129     
- Misses        587      631      +44     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

MarcBerliner and others added 2 commits August 21, 2026 12:11
Three defects, none of which depend on the API shape:

The cache scratch vector lived on the solver object, so two threads evaluating
one Function shared it. Moved onto BrentMemory, which CasADi hands out per
concurrent evaluation.

The bracket test read `fa * fb <= 0`, and two residuals near the underflow
limit multiply to zero, which reads as a sign change. Now tested by sign, with
NaN handled explicitly, so a bracket holding no root is reported as one.

The NumPy path returned NaN where the plugin raises: a blanket
`except (ValueError, RuntimeError)` around brentq swallowed both an empty
bracket and a failure to converge along with the tree probes it was meant to
absorb. Those two now raise SolverError, and only a probe returns NaN.

`Brent` and `BrentUnknown` also lose their public names, becoming `_Brent` and
`_BrentUnknown` alongside `_BaseAverage`, and the docs page goes. The node has
no in-tree consumer yet, and lifting the residual into a closed sub-expression
-- which is where this is headed, and what the Rust tape backend in #5732 wants
too -- will change the constructor. Cheaper to withdraw the commitment now than
to break it later. The changelog bullet goes with it: nothing user-facing is
left to announce.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…nked

The version and its sdist hash were pinned here as well as in pyproject.toml,
so a CasADi bump meant editing both. The configure step already reads the
linked CasADi's version out of its own config.h to check the two agree, so use
that as the source instead: pyproject.toml stays the only pin.

CASADI_INTERNAL_VERSION survives as an override, and now disagreeing with the
linked library is the only thing that trips the mismatch error. The sdist hash
cannot be pinned in tree once the version floats, so it is optional and checked
only when passed; a truncated download still fails, at extraction.

The staging directory is keyed on the version too. It was not, so a bump would
have found the previous version's headers already in place and skipped
restaging, compiling against internals from the wrong release.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@MarcBerliner MarcBerliner changed the title Add pybamm.Brent, a bracketed scalar rootfind Add a bracketed scalar rootfind node, kept internal for now Aug 21, 2026
MarcBerliner and others added 4 commits August 21, 2026 12:43
The conversion named the unknown, the oracle and the rootfinder after
`abs(self.id)`. `Symbol.set_id` hashes a tuple containing a string, so ids are
randomised per process, and those names reach `fn.serialize()` -- the AOT
compile cache key. Any model holding a rootfind therefore missed the on-disk
cache and recompiled from scratch in every process.

Measured over three fresh interpreters before: a Brent-free function keyed to
4da87061f2ba3c82 each time, one holding a Brent to 5229b804, 0d83e675 and
277998d7. After: 3e26c07750303134 each time.

The identity was never needed. CasADi names generated code positionally --
two distinct oracles both called "brent_oracle" emit casadi_f1 and casadi_f3,
and the plugin's cache variable is already keyed on that positional name -- so
constants do the job. Two Brents that differ only in tolerance still get their
own rootfinder, because the conversion cache is keyed on the id, which is
where identity belongs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Bandit flags the subprocess call the previous version used to compare the
serialised function across interpreters, and Codacy gates on new issues.

Forcing a different id in-process tests the same invariant more directly: two
structurally identical nodes, one carrying the id another process would have
hashed, must serialise to the same bytes. It catches each of the three names
independently, and drops the three interpreter startups the old one paid for.

A substring scan for the id would not have worked here: serialize() is encoded,
so neither the id nor "brent_oracle" appears in it as text.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ached

Windows holds two copies of CasADi -- ours built with MSVC, the wheel's with
MinGW -- so the plugin registered in one is invisible to the other, and the
node refused to convert there at all. That took composite electrode SOH in
#5730 with it, which works on Windows today.

A CasADi Callback closes the gap. The residual is still compiled into the
oracle exactly as the plugin path builds it; only the bracketing iteration
runs in Python, driven by scipy.optimize.brentq. Everything around the
rootfind stays in the graph, so a rootfind nested inside another costs one
callback per enclosing iteration rather than a Python tree walk per residual
evaluation: 200 nested solves take 0.26s this way against 0.004s through the
plugin, where evaluating the tree directly could not finish one composite SOH
solve in fourteen minutes.

Derivatives come from the implicit function theorem, as the plugin's do:
dx/dp = -(dF/dp) / (dF/dx) at the root, emitted as a forward mode over the
same oracle. The tests confirm both paths agree to 1e-9 on a case with a
known answer.

Two things the driver cannot do. It re-enters Python, so the test that pins
that out of the plugin path now skips when the plugin is absent, and it holds
a callback, so an expression containing one cannot be code-generated.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An expression holding one is a CasADi Callback, so IDAKLU's deserialise step
rejects it -- observing such a variable raises "not found 'CallbackInternal'".
Same root cause as the codegen limitation already noted, and worth naming
next to it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@MarcBerliner
MarcBerliner marked this pull request as ready for review August 24, 2026 20:53

@aabills aabills left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think in general this is a good idea but let's rename it to something more general than the particular numerical method we're using

@aabills

aabills commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Claude's suggestion which I like:

sto = pybamm.Unknown(name)
return pybamm.ImplicitFunction(U(sto, T, branch) - target, sto, (lo, hi))

One simple change, maybe we call it ImplicitUnknown?

@MarcBerliner
MarcBerliner marked this pull request as draft September 3, 2026 20:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants