Add a bracketed scalar rootfind node, kept internal for now - #5729
Draft
MarcBerliner wants to merge 12 commits into
Draft
Add a bracketed scalar rootfind node, kept internal for now#5729MarcBerliner wants to merge 12 commits into
MarcBerliner wants to merge 12 commits into
Conversation
A `Brent` node solves `residual == 0` for its unknown over a bracket inside the CasADi graph rather than in Python: it converts to a `rootfinder` using a native "brent" plugin registered by pybammsolvers, so an expression containing one costs no extra Python frames per evaluation and survives codegen to C. Brent needs only a sign change over the bracket, so it converges where a Newton iteration stalls or leaves the domain, and the answer cannot leave the bracket. Derivatives come from CasADi's implicit function theorem. The plugin is built out of tree, which needs four things from CMake that an in-tree CasADi plugin gets for free: the internal headers staged from a pinned sdist rather than vendored, since CasADi does not install them and they are LGPL; an ABI check reading the linked version from CasADi's own config.h; the compile flags CasADi itself used, one of which adds a static mutex to Rootfinder; and the iteration stringified so codegen emits the text it compiles. brent.hpp says so at the top. Also fixes an unrelated test bug found on the way: test_pybamm_import unloaded every pybamm module from sys.modules and never restored them, leaving a second set of classes behind so `isinstance` failed across the boundary. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
MarcBerliner
marked this pull request as draft
August 20, 2026 14:03
cibuildwheel's test env installs only pybammsolvers[dev], which has no scipy, so the module-level `from scipy.optimize import brentq` failed collection and took every wheel job down with exit code 2. The residual assertion already pins the root, and which root a bracketed solve lands on for the non-monotone case was never a contract, so the brentq comparison bought nothing that survived being unrunnable. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Codacy runs cppcheck over each file in the diff on its own, so brent.hpp is analysed without brent.cpp and all seven of its members come back unused. Suppress the check for the header rather than annotating members one by one. The variableScope finding is real: `tol`, `xm`, `step` and the interpolation scratch each live for a single iteration, so declare them in the block that uses them. One declaration block per scope still satisfies the C the text is stringified into. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The staged internal headers include `options.hpp` unqualified. When that does not resolve next to the file asking for it, MSVC widens the search to the include stack, reaches `idaklu_source/`, and answers with our own Options.hpp -- equal to `options.hpp` on a case-insensitive filesystem, and no declaration of `casadi::Options` in sight. Every internal header that names `Options` then fails to parse, starting at function_internal.hpp:117. Staging CasADi's copy puts it in the directory the including file is in, which is the first place either compiler looks, so the collision never comes up. The guard is shared with the installed copy, so a translation unit that pulls in both still sees one definition, and the config.h version check already keeps the two identical. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The plugin registers into whichever CasADi is linked into the pybammsolvers extension. On Linux and macOS that is the casadi wheel's own libcasadi, so the registration is visible to the CasADi Python calls. On Windows the extension is built with MSVC against vcpkg's CasADi while the casadi wheel is built with MinGW -- it ships libstdc++-6.dll and .dll.a import libraries -- so the two hold separate copies of CasADi, and separate plugin registries. Registering ours cannot reach Python's, which falls back to loading libcasadi_rootfinder_brent.dll and fails with WIN32 error 126. Closing that gap needs the plugin built with MinGW against the wheel's libcasadi.dll.a, a second toolchain in the Windows job. Until then it is unavailable there, so skip both test modules on win32 -- 23 in pybammsolvers, 17 in pybamm -- and raise a NotImplementedError naming the cause when a Brent node is converted to CasADi without the plugin, in place of CasADi's "Plugin 'brent' is not found". evaluate() is unaffected: it solves with SciPy. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #5729 +/- ##
==========================================
- Coverage 98.20% 98.08% -0.13%
==========================================
Files 340 341 +1
Lines 32743 32916 +173
==========================================
+ Hits 32156 32285 +129
- Misses 587 631 +44 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Three defects, none of which depend on the API shape: The cache scratch vector lived on the solver object, so two threads evaluating one Function shared it. Moved onto BrentMemory, which CasADi hands out per concurrent evaluation. The bracket test read `fa * fb <= 0`, and two residuals near the underflow limit multiply to zero, which reads as a sign change. Now tested by sign, with NaN handled explicitly, so a bracket holding no root is reported as one. The NumPy path returned NaN where the plugin raises: a blanket `except (ValueError, RuntimeError)` around brentq swallowed both an empty bracket and a failure to converge along with the tree probes it was meant to absorb. Those two now raise SolverError, and only a probe returns NaN. `Brent` and `BrentUnknown` also lose their public names, becoming `_Brent` and `_BrentUnknown` alongside `_BaseAverage`, and the docs page goes. The node has no in-tree consumer yet, and lifting the residual into a closed sub-expression -- which is where this is headed, and what the Rust tape backend in #5732 wants too -- will change the constructor. Cheaper to withdraw the commitment now than to break it later. The changelog bullet goes with it: nothing user-facing is left to announce. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…nked The version and its sdist hash were pinned here as well as in pyproject.toml, so a CasADi bump meant editing both. The configure step already reads the linked CasADi's version out of its own config.h to check the two agree, so use that as the source instead: pyproject.toml stays the only pin. CASADI_INTERNAL_VERSION survives as an override, and now disagreeing with the linked library is the only thing that trips the mismatch error. The sdist hash cannot be pinned in tree once the version floats, so it is optional and checked only when passed; a truncated download still fails, at extraction. The staging directory is keyed on the version too. It was not, so a bump would have found the previous version's headers already in place and skipped restaging, compiling against internals from the wrong release. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The conversion named the unknown, the oracle and the rootfinder after `abs(self.id)`. `Symbol.set_id` hashes a tuple containing a string, so ids are randomised per process, and those names reach `fn.serialize()` -- the AOT compile cache key. Any model holding a rootfind therefore missed the on-disk cache and recompiled from scratch in every process. Measured over three fresh interpreters before: a Brent-free function keyed to 4da87061f2ba3c82 each time, one holding a Brent to 5229b804, 0d83e675 and 277998d7. After: 3e26c07750303134 each time. The identity was never needed. CasADi names generated code positionally -- two distinct oracles both called "brent_oracle" emit casadi_f1 and casadi_f3, and the plugin's cache variable is already keyed on that positional name -- so constants do the job. Two Brents that differ only in tolerance still get their own rootfinder, because the conversion cache is keyed on the id, which is where identity belongs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Bandit flags the subprocess call the previous version used to compare the serialised function across interpreters, and Codacy gates on new issues. Forcing a different id in-process tests the same invariant more directly: two structurally identical nodes, one carrying the id another process would have hashed, must serialise to the same bytes. It catches each of the three names independently, and drops the three interpreter startups the old one paid for. A substring scan for the id would not have worked here: serialize() is encoded, so neither the id nor "brent_oracle" appears in it as text. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ached Windows holds two copies of CasADi -- ours built with MSVC, the wheel's with MinGW -- so the plugin registered in one is invisible to the other, and the node refused to convert there at all. That took composite electrode SOH in #5730 with it, which works on Windows today. A CasADi Callback closes the gap. The residual is still compiled into the oracle exactly as the plugin path builds it; only the bracketing iteration runs in Python, driven by scipy.optimize.brentq. Everything around the rootfind stays in the graph, so a rootfind nested inside another costs one callback per enclosing iteration rather than a Python tree walk per residual evaluation: 200 nested solves take 0.26s this way against 0.004s through the plugin, where evaluating the tree directly could not finish one composite SOH solve in fourteen minutes. Derivatives come from the implicit function theorem, as the plugin's do: dx/dp = -(dF/dp) / (dF/dx) at the root, emitted as a forward mode over the same oracle. The tests confirm both paths agree to 1e-9 on a case with a known answer. Two things the driver cannot do. It re-enters Python, so the test that pins that out of the plugin path now skips when the plugin is absent, and it holds a callback, so an expression containing one cannot be code-generated. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An expression holding one is a CasADi Callback, so IDAKLU's deserialise step rejects it -- observing such a variable raises "not found 'CallbackInternal'". Same root cause as the codegen limitation already noted, and worth naming next to it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
MarcBerliner
marked this pull request as ready for review
August 24, 2026 20:53
aabills
requested changes
Aug 24, 2026
aabills
left a comment
Contributor
There was a problem hiding this comment.
I think in general this is a good idea but let's rename it to something more general than the particular numerical method we're using
Contributor
|
Claude's suggestion which I like: One simple change, maybe we call it |
MarcBerliner
marked this pull request as draft
September 3, 2026 20:07
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds an expression-tree node that solves
residual == 0for its unknown over a bracket, plus the native CasADi rootfinder plugin it runs on.The solve runs inside the CasADi graph, not in Python: the node converts to a
rootfinderusing a"brent"plugin registered bypybammsolvers. An expression containing one costs no extra Python frames per evaluation, and survives codegen to C.Brent needs only a sign change over the bracket, so it converges where a Newton iteration stalls or leaves the domain, and the answer cannot leave the bracket. Derivatives come from CasADi's implicit function theorem, exactly.
Deliberately not public
The node is
pybamm._Brent, with no docs page and no changelog entry. It is reachable from thepybammnamespace the way_BaseAveragealready is, and nothing about it is a commitment yet.There are no in-tree consumers, and the constructor is going to change: the residual is currently a symbol the caller names and must keep unique by hand, and that wants to become a closed sub-expression with positional arguments — a shape the Rust tape backend in #5732 independently needs, and which
Brentpresently reconstructs by hand viacasadi.symvarand a scoped cache. Cheaper to withhold the name now than to break it later. #5730 is the first real consumer; the public name can follow once it has proven the shape.Building the plugin
It is built out of tree, so CMake supplies four things an in-tree CasADi plugin gets for free.
brent.hppopens with this list; briefly:rootfinder_impl.hppand five siblings are not installed by CasADi (INSTALL_INTERNAL_HEADERSis off by default) and are LGPL-3.0-or-later, so they are staged at configure time from the sdist rather than vendored into this BSD-3-Clause tree.-DCASADI_SOURCE_DIR=<tree>builds offline.casadi/config.h— the one source present on every discovery path, including Windows/vcpkg and conda, which have no interpreter to ask. That version is also what selects the sdist, so the pin inpyproject.tomlis the only one, and-DCASADI_INTERNAL_VERSION=is an override whose only job now is to fail loudly when it disagrees with the linked library. The staging directory is keyed on the version, so a bump cannot find the previous release's headers in place and skip restaging.CASADI_WITH_THREADSAFE_SYMBOLICSadds a static mutex toRootfinder, so guessing it would be an ABI break rather than a warning.brent_impl.hppis compiled and stringified intobrent_impl_str, so codegen emits the same text this file compiles and the two paths cannot drift.Not available on Windows
The CasADi wheel for Windows is built with MinGW — it ships
libstdc++-6.dlland.dll.aimport libraries, and its ownconfig.hnames the compiler asx86_64-w64-mingw32.shared.posix-g++.pybammsolversis built with MSVC against vcpkg's CasADi.Brentsubclassescasadi::Rootfinder, so the two cannot be linked across that ABI, and the plugin registered inside our extension is invisible to the CasADi Python calls: the process holds two copies with two plugin registries.Converting a node to CasADi there raises a
NotImplementedErrornaming the cause rather than CasADi'sPlugin 'brent' is not found,evaluate()still works via SciPy, and both test modules skip onwin32. Closing the gap means building the plugin as a separate MinGW DLL against the wheel'slibcasadi.dll.a— a second toolchain in the Windows job — or landing it upstream in CasADi, which has no bracketed rootfinder of its own.Notes
Brentnested inside another re-solves on every enclosing iteration, which compounds per level. Covered on both the interpreted and the generated path. The cache scratch lives on the solver's memory object, which CasADi hands out per concurrent evaluation, so a threadedmapover one Function does not share it.fa * fb <= 0, which underflows to zero for two residuals near the underflow limit and reads as a sign change.max_iteris reported as a failure, not a root, on both the CasADi and the NumPy path. Likewise an empty bracket: the NumPy path used to swallow both into a quietNaN, along with the tree probes the blanketexceptwas meant to absorb.id. Ids are a per-process hash and the names reachfn.serialize(), the AOT compile cache key, so a model holding a rootfind used to miss the on-disk cache in every process: measured over three fresh interpreters, a Brent-free function keyed to4da87061f2ba3c82each time while one holding aBrentkeyed to5229b804…,0d83e675…,277998d7…. Now3e26c07750303134each time. The identity was never needed, since CasADi names generated code positionally and the conversion cache — keyed on the id, which is where identity belongs — already gives two Brents differing only in tolerance their own rootfinder. A test pins the key across processes.test_pybamm_importunloaded everypybammmodule fromsys.modulesand never restored them, leaving a second set of classes behind soisinstancefailed across the boundary for anything sharing an xdist worker afterwards.Tests: 4207 unit, 82
pybammsolvers.The composite electrode SOH solver built on this is #5730.
🤖 Generated with Claude Code