Skip to content

Support AutoDespecialize in nonlinear solvers - #1164

Merged
ChrisRackauckas merged 3 commits into
SciML:masterfrom
ChrisRackauckas-Claude:agent/nonlinear-scc-autodespecialize
Aug 14, 2026
Merged

ChrisRackauckas merged 3 commits into
SciML:masterfrom
ChrisRackauckas-Claude:agent/nonlinear-scc-autodespecialize

Conversation

@ChrisRackauckas-Claude

@ChrisRackauckas-Claude ChrisRackauckas-Claude commented Aug 14, 2026 •

Copy link
Copy Markdown
Member

Ignore this PR until reviewed by @ChrisRackauckas.

What changed

Add end-to-end AutoDespecialize support to NonlinearSolveBase and SCCNonlinearSolve. Concretized nonlinear problems now carry a stable SciMLBase.DespecializedParameters type while residuals and auxiliary callbacks receive the original parameter object through SciMLBase's public invocation barrier. Enzyme paths reconstruct both the raw parameter and raw callback before differentiation.

SCC solves now concretize nonlinear subproblems before cache construction and invoke tuple/vector explicit-transfer callbacks through the same public barrier. The change also makes NonlinearSolveBase.get_concrete_problem a documented public developer API so SCCNonlinearSolve does not depend on a sibling package's internals, adds an AutoDespecialize precompile workload, and applies the required minor version bumps to NonlinearSolve 4.27.0, NonlinearSolveBase 2.44.0, and SCCNonlinearSolve 1.15.0.

The mechanical SciMLBase and root LinearSolve floors were reviewed and merged separately in #1162 and #1163. The direct FirstOrder and QuasiNewton LinearSolve floors were then aligned in #1165 before this branch was rebased.

Failing before

With only the new tests applied to the unfixed code:

GROUP=NonlinearSolveBase_Core julia +1.12 --startup-file=no --project=. -e 'using Pkg; Pkg.test()'

The AutoDespecialize discriminator reported 9 passed / 3 failed: the parameters remained concrete, their types differed, and the concrete problem types differed.

GROUP=SCCNonlinearSolve_Core julia +1.12 --startup-file=no --project=. -e 'using Pkg; Pkg.test()'

The SCC tuple/vector discriminator reported 14 passed / 6 failed: both storage routes retained concrete parameter and problem types.

After the main implementation but before unwrapping the residual's parameter-dispatch wrapper for Enzyme, the Base discriminator reported 24 passed / 1 failed because enzyme_prob.f.f was not the original residual. Applying the one-line residual unwrap made the identical assertion pass.

Passing after

  • GROUP=NonlinearSolveBase_Core julia +1.12 --startup-file=no --project=. -e 'using Pkg; Pkg.test()' — AutoDespecialize 25/25; full NonlinearSolveBase and outer package tests passed.
  • GROUP=SCCNonlinearSolve_Core julia +1.12 --startup-file=no --project=. -e 'using Pkg; Pkg.test()' — AutoDespecialize tuple/vector 20/20; all existing SCC groups and package tests passed.
  • GROUP=Core julia +1.12 --startup-file=no --project=. -e 'using Pkg; Pkg.test()' — full root Core and package tests passed; root AutoDespecialize 7/7.
  • NONLINEARSOLVE_TEST_GROUP=QA julia +1.12 --startup-file=no --project=. -e 'using Pkg; Pkg.test()' — QA 28/28; package tests passed. The same command remained 28/28 after rebasing onto Raise LinearSolve sublibrary floor to 5.4 #1165.
  • GROUP=NonlinearSolveBase_QA julia +1.12 --startup-file=no --project=. -e 'using Pkg; Pkg.test()' — QA 20/20; package tests passed.
  • GROUP=SCCNonlinearSolve_QA julia +1.12 --startup-file=no --project=. -e 'using Pkg; Pkg.test()' — QA 20/20; package tests passed.
  • julia +1.12 --startup-file=no --project=docs docs/make.jl — exit 0; documentation rendered successfully.
  • Runic over every changed Julia file, typos over the diff, and git diff --check — passed.

I did not run GPU groups, GROUP=Everything, or the complete downstream matrix locally.

Reviewer judgment

get_concrete_problem is promoted as public developer API, documented, and accompanied by minor version bumps because SCCNonlinearSolve needs problem concretization but must not reach into NonlinearSolveBase internals.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
@ChrisRackauckas-Claude
ChrisRackauckas-Claude force-pushed the agent/nonlinear-scc-autodespecialize branch from 8e3e50f to 26467c6 Compare August 14, 2026 14:40

Copy link
Copy Markdown
Member Author

Rebased onto the merged direct-consumer LinearSolve floor fix in #1165.

Post-rebase local validation:

  • NONLINEARSOLVE_TEST_GROUP=QA julia +1.12 --startup-file=no --project=. -e 'using Pkg; Pkg.test()' — QA 28/28; package tests passed.
  • Runic over all changed Julia files, typos over all changed files, and git diff --check — passed.

Post-rebase CI checkpoint: NonlinearSolveBase Core on Julia LTS/current/pre, Base QA, SCCNonlinearSolve Core on LTS/current/pre, SCC QA, root QA, root downgrade, both Base/SCC sublibrary downgrade jobs, Runic, and spelling are green. The only concluded red job is the exact clean-master SimpleHomotopySweep allocation regression (125 errors twice), with its owner-side fix in SciML/SciMLBase.jl#1526. Documentation and broader platform/downstream jobs are still pending.

@ChrisRackauckas
ChrisRackauckas marked this pull request as ready for review August 14, 2026 21:31
@ChrisRackauckas
ChrisRackauckas merged commit e416641 into SciML:master Aug 14, 2026
90 of 97 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants