Eliminate dense continuation step allocations on Julia 1.12

[ChrisRackauckas-Claude]

Ignore this PR until reviewed by @ChrisRackauckas.

Summary

Remove the NonlinearSolve-owned dense continuation step allocations observed on Julia 1.12 x86_64 without weakening the existing allocation regression:

• keep a fixed-shape cached destination for dense AutoForwardDiff Jacobian evaluation
• return the input directly when Utils.restructure receives the same object
• run repeated sweep and arclength correctors directly to completion in their reused cache, without constructing a NonlinearSolution on every interior step
• preserve ordinary solve! results and construct full solutions for anchors, final landings, and reported failures
• add warmed allocation checks for the Jacobian destination, same-object restructure, and cache-only solve path
• bump NonlinearSolveBase to 2.35.2

The new solver helpers are private. No public API, threshold, tolerance, skip, broken-test designation, or failure behavior changes.

Root cause and platform boundary

The standalone NonlinearSolve changes remove three independently observed sources:

1. ForwardDiff reshapes its already-matrix Jacobian destination.
2. ArrayInterface.restructure creates a wrapper when the target and source are already the same buffer.
3. Each interior continuation corrector called ordinary solve!(cache), constructing a complete solution although the driver only needs cache state, residual, retcode, and stats.

Removing those sources produces the verified zero-allocation Julia 1.12 x86_64 result below. It does not, by itself, establish the Apple result.

Fresh combined-stack Apple ARM runs after all three changes still measured exactly 772.0/400.0 B per sweep/arclength step on prerelease Julia. A subsequent stable-macOS allocation profile measured 957.28/496.0 B per step and isolated the dominant remaining allocator upstream in LinearSolve's AppleAccelerateLUFactorization: repeated LU{Float64, Matrix, Vector{Int32}} construction contributed 25,984 bytes / 812 allocations in sweep and 11,648 bytes / 364 allocations in arclength, alongside tuple, Ref, range, and keyword-NamedTuple construction.

Focused draft LinearSolve #1099 caches the factor, pivot, and LAPACK-status workspaces. Its direct Apple-silicon tests pass on Julia 1.10, 1.12, and 1.13 without changing the allocation ceiling. This standalone PR still makes no downstream cross-platform allocation claim until the LinearSolve PR is merged/released and the continuation regression is rerun against that dependency.

Validation

Standalone head: 3d568afa132c4f27035622def12de95c30dec40f.

• exact root allocation regression on Julia 1.12.6: cache-only solve 0 B and returns the same cache; sweep 0.0 B/step; arclength 0.0 B/step
• standalone NonlinearSolveBase Core: passed, including allocation fast paths 6/6, workspace 50/50, routing 12/12, operator dispatch 1/1, dampening 12/12, and dense-LU allocations 14/14
• final combined tree NonlinearSolveBase Core on Julia 1.12.6 and 1.10.11: passed
• all 39 unchanged homotopy/arclength test files: passed, including allocation, Jacobian, failure, tolerance, effort, retention, and polyalgorithm paths
• NonlinearSolveBase QA: 18/18
• ForwardDiff cache reinit regression: 1/1
• Runic 1.7.0 --check --diff .: passed over both final trees
• git diff --check: passed

The canonical root Core run still stops earlier at the separately audited clean-base Brown/Broyden unexpected-pass failure; no test was skipped or altered to continue past it.

[ChrisRackauckas-Claude]

Investigation scratchpad / completed plan:

1. Reproduced clean master (579761dd2) on Julia 1.12.6: sweep 154.4 B/step, arclength 80.0 B/step.
2. Checked the history boundary: Make AutoSpecialize infer through functor residuals; zero-alloc homotopy stepping #1046 (0343930c2) has the same remaining 154.4/80.0; its parent (6a7687f54) measures 250.4/400.0. Make AutoSpecialize infer through functor residuals; zero-alloc homotopy stepping #1046 removed the intended functor boxing but added the assertion that first exposes the remaining Julia 1.12 headers.
3. Repeated Make AutoSpecialize infer through functor residuals; zero-alloc homotopy stepping #1046 with LinearSolve 4.3.0 and 5.0.0; both match, excluding the LinearSolve 5/Align downgrade floors with LinearSolve 4.3 #1059 update as the cause.
4. Used Profile.Allocs to isolate 48 B/iteration in ForwardDiff's destination reshape and 32 B/iteration in plain-vector restructure.
5. Implemented a local owned Jacobian adapter plus the plain-vector fast path, keeping cache.J a Matrix and relying only on owner/public APIs.
6. Ran the existing allocation regression across LTS/current/prerelease, both supported LinearSolve lines, nested ForwardDiff checks, native Base and FirstOrder groups, and whole-repo Runic. Current/prerelease and LinearSolve 4.3/5.0 all measure 0.0/0.0 B per step.

The #1057 prerelease Core job independently corroborates the original 154.4/80.0 signature. That branch and the separate clean-master Broyden failures were left untouched.

[ChrisRackauckas-Claude]

CI classification for the two current root failures:

• Downgrade / Downgrade Tests - Core (job 86867008813)
• tests / Core (julia lts, ubuntu-latest) (job 86867091501)

Both fail in the pre-existing 23-problem Broyden robustness set, with the same Generalized Rosenbrock | alg #4 residual failure (4.4 ≤ 0.001) tracked by #1057 and the same Brown almost linear | alg #4 unexpected pass tracked by #1056. Neither reaches or implicates the homotopy allocation regression changed by this PR. I have left #1061 unchanged.

[ChrisRackauckas-Claude]

Additional CI classification as the matrix completes:

• Root Core on Julia 1 and Julia pre stops in the same clean-main Broyden robustness test before reaching this PR's allocation test. Julia 1 also reports the separately tracked Brown unexpected pass; Julia pre reports only the Generalized Rosenbrock residual failure.
• lib/NonlinearSolveBase [QA] reports exactly the two independently audited clean-main findings: the JacobianCache{<:AbstractSciMLOperator}(::Nothing) ambiguity being handled in the stack above Raise SciMLOperators public API floors and resolve cache ambiguity #1060, and the non-public FunctionWrappersWrappers.SingleCacheStorage access whose public-constructor prerequisite is GPU Compatibility for 1D Arrays  #70.

The directly relevant NonlinearSolveBase Core lanes (LTS/current/pre) and its downgrade lane have passed on #1061. The branch remains unchanged.

[ChrisRackauckas-Claude]

Terminal CI audit for 7e96b21:

• The changed homotopy allocation path is green in the owning sublibrary: NonlinearSolveBase Core passed on Julia LTS/current/prerelease and downgrade. NonlinearSolveHomotopyContinuation Core/QA/downgrade also passed. All downstream jobs, documentation, Runic, and spell checks passed.
• All six red root Core lanes and the root downgrade lane stop earlier in the already-audited Broyden robustness cluster: generalized Rosenbrock residual 4.4 (focused fix Restore dense pinv fallback for inverse Jacobian init #1058) and, in the affected environments, the Brown unexpected pass (Brown bad-Broyden broken test is environment-sensitive #1056). They never implicate this PR's allocation assertions.
• Base QA is the clean-main JacobianCache ambiguity plus non-public FunctionWrappersWrappers.SingleCacheStorage; Raise SciMLOperators public API floors and resolve cache ambiguity #1060 handles the ambiguity and FunctionWrappersWrappers.jl#70 is the owning-package public-API prerequisite.
• CUDA has the exact clean-master pair of DimensionMismatch: array could not be broadcast to match destination errors at test/gpu/cuda_tests__item1.jl:34 and :44. Clean master evidence is job 86844917359; this PR's job is 86867091569. A separate clean-main reproduce/bisect/fix audit is queued.

The run is terminal: every failure is independent of the dense homotopy step-allocation change, and no test has been skipped, silenced, or loosened.

[ChrisRackauckas-Claude]

Rebased the branch onto current master at 41681c5e4 after #1060 merged, then rechecked the final tree locally before updating the remote branch.

Final-tree validation:

• Julia 1.12 NonlinearSolveBase native test suite passed, including linsolve_identity!! workspace 50/50, linear-solver routing 12/12, operator cache dispatch 1/1, and dense-LU allocation coverage 14/14.
• Exact root regression: homotopy allocation regression | 11 Pass; measured sweep_bytes_per_step=0.0 and arclength_bytes_per_step=0.0.
• Whole-repository Runic check passed.
• git diff --check upstream/master...HEAD passed.

The rebased head is 48d5e1fddc0589b30cec268a4f4018aa5433b5ad; CI has restarted on that commit.

[ChrisRackauckas-Claude]

The two early red downgrade jobs on rebased head 48d5e1fdd are the same shared-action resolver failure already classified on #1063, not homotopy test failures:

• Root downgrade fixes in-tree NonlinearSolveQuasiNewton 1.14.1, which requires SciMLOperators >= 1.24, but the generated downgrade manifest explicitly pins SciMLOperators = 1.22.0.
• SimpleNonlinearSolve downgrade fixes in-tree NonlinearSolveBase 2.34.3, which also requires >= 1.24, against the same explicit 1.22.0 pin.

Both jobs stop at Unsatisfiable requirements before any test starts. The shared path-source handling is addressed by stacked julia-downgrade-compat drafts #54 and #55. Remaining checks are still running.

[ChrisRackauckas-Claude]

Post-rebase functional CI classification from the exact logs:

• NonlinearSolveBase QA reports only the existing FunctionWrappersWrappers.SingleCacheStorage non-public access; owning public-API draft FunctionWrappersWrappers#70 is the prerequisite.
• Root Core on Julia current, LTS, and prerelease clears this PR's homotopy allocation section and continues to the separate generalized-Rosenbrock bad-Broyden failure (4.4 > 0.001). That clean-master solver regression is addressed by focused alternatives Preserve triangular solves for Broyden Jacobian inversion #1057/Restore dense pinv fallback for inverse Jacobian init #1058.

The allocation assertions are no longer the failing point on this branch, and no unrelated public-API allowlist or Broyden test is changed here.

[ChrisRackauckas-Claude]

Replaced the original adapter-based implementation with ab051325ba1084a597e6fe69ba38ef16bbdb5aa1, based directly on current upstream 60736b5b6.

The investigation separated the two allocation sites with Profile.Allocs, repeated endpoint measurements, a Julia 1.11.9 versus 1.12.0/1.12.5/1.12.6 microbenchmark, and a frozen-graph git bisect run. It then measured each candidate independently before combining them:

• baseline: sweep 154.4 B/step, arclength 80.0 B/step
• public full-view Jacobian destination only: 61.76 / 32.0 B/step
• same-buffer restructure shortcut only: 92.64 / 48.0 B/step
• combined: 0.0 / 0.0 B/step

This replacement removes the private _JacobianADBuffer type and extra cache field from the previous revision. It adds direct warmed zero-allocation tests, bumps NonlinearSolveBase to 2.35.2, and keeps every existing allocation threshold unchanged.

Publishing process: fetched and verified upstream and the fork at 60736b5b6; re-fetched the PR branch at exactly 48d5e1fddc0589b30cec268a4f4018aa5433b5ad; then replaced only that branch using the explicitly lease-guarded update. No other remote branch was modified.

[ChrisRackauckas-Claude]

Terminal CI audit for head ab051325ba1084a597e6fe69ba38ef16bbdb5aa1:

• Final totals: 80 successful, 3 skipped, 10 failed.
• The allocation-relevant NonlinearSolveBase Core lanes, including Julia 1/LTS/pre-release coverage, passed. The exact homotopy allocation regression also passes locally on Julia 1.11.9 and 1.12.6 at 0 B/step for both continuation methods.
• Every red check reproduces independently of this branch on clean current master:
  • root downgrade resolver conflict: fixed separately in Restore root LinearSolve 5 floor #1087;
  • duplicate BibTeX entry: fixed separately in Remove duplicate BibTeX entry #1088, whose full 11-check CI including docs is green;
  • NonlinearSolveBase QA public-API finding: owned by Use public cache construction for ForwardDiff wrappers #1073;
  • six root Core lanes: generalized-Rosenbrock/Broyden regression owned by Preserve triangular solves for Broyden Jacobian inversion #1057, with the pre/LTS Brown-almost-linear unexpected pass tracked by Brown bad-Broyden broken test is environment-sensitive #1056;
  • CUDA Broyden DimensionMismatch: clean-master regression from Quasi-Newton J⁻¹ init as a linsolve workspace: factorization + identity solve through the LinearSolve default #1039, with the focused GPU-tested fix in Restore GPU Broyden inverse initialization #1066.

None of the failing stacks enters either allocation fast path changed by this PR.

[ChrisRackauckas-Claude]

Apple ARM follow-up published at ffc0113af.

The previous full-view destination removed the ForwardDiff reshape allocation on local x86_64, but the combined stack showed that it still depended on compiler scalar replacement: macOS current measured 957.28/496.0 B per sweep/arclength step, and macOS prerelease measured 772.0/400.0 B, against the unchanged < 48 assertions.

The follow-up caches a fixed-shape Jacobian destination once. Its same-shape reshape returns the cached object, so Jacobian evaluation no longer constructs a wrapper on each call. The solver-facing cache.J remains the original Matrix; non-dense and non-ForwardDiff paths are unchanged.

Exact standalone validation on the new head:

• NonlinearSolveBase Core passed, including allocation fast paths 6/6, workspace 50/50, routing 12/12, operator 1/1, dampening 12/12, and dense LU 14/14
• the unchanged root allocation regression passed and measured sweep/arclength 0.0/0.0 B per step
• Runic 1.7.0 and git diff --check passed

The remote branch was re-fetched at ab051325ba1084a597e6fe69ba38ef16bbdb5aa1 immediately before publication and advanced by a normal fast-forward. No force push was used. New Apple ARM CI is the cross-platform verification; I am not claiming a macOS pass from the Linux measurements.

[ChrisRackauckas-Claude]

Published the cache-only corrector follow-up at 3d568afa132c4f27035622def12de95c30dec40f by normal fast-forward from the audited remote head ffc0113af.

Why this follow-up is needed: the combined Apple prerelease job still measured exactly 772.0 B/step for sweep and 400.0 B/step for arclength after the cached Jacobian destination change. Profiling isolated the remaining per-step construction of complete nonlinear solution objects. Interior correctors now finish directly in the reused cache and read state/residual/retcode/stats from it; anchors, final landings, ordinary solve!, and failure originals still construct full solutions.

Exact standalone verification on this head:

• cache-only solve: 0 B, same cache object, success retcode
• sweep/arclength slopes: 0.0/0.0 B per step
• NonlinearSolveBase Core: passed (allocation 6/6, workspace 50/50, routing 12/12, operator 1/1, dampening 12/12, dense LU 14/14)
• Runic 1.7 and git diff --check: passed

The combined-tree validation additionally passed all 39 continuation files, Base QA 18/18, Julia 1.10 Base Core, and the ForwardDiff cache reinit regression. Fresh Apple CI is still required; I am not claiming the cross-architecture result from Linux measurements.

[ChrisRackauckas-Claude]

Corrected the Apple allocation attribution in the PR description after the combined-stack diagnostic evidence:

• The standalone changes still verify the NonlinearSolve-owned Julia 1.12 x86_64 result: cache-only solve 0 B, sweep/arclength 0.0/0.0 B per step.
• Apple ARM remained at 772.0/400.0 B per step after those changes; this disproved per-corrector solution construction as the remaining Apple allocator.
• Stable-macOS profiling isolated the dominant remaining bytes to LinearSolve's AppleAccelerateLUFactorization wrapper/workspace construction.
• LinearSolve #1099 now passes direct Apple-silicon allocation coverage on Julia 1.10, 1.12, and 1.13.
• No downstream Apple success is claimed here until that dependency is merged/released and the continuation regression is rerun.

No source, test, threshold, or commit changed in this update.

[ChrisRackauckas-Claude]

Superseded — rebase onto current master produces a zero-line diff

Rebasing agent/fix-homotopy-step-allocations onto master (fdf1e69c6) resolves to exactly master. Every hunk in this PR is already upstream, so there is nothing left to merge:

$ git checkout -B rebase-1061 origin/master
$ git merge --squash agent/fix-homotopy-step-allocations
   # conflicts in Project.toml, homotopy_sweep.jl, solve.jl, homotopy_alloc_tests__item1.jl
   # (all resolved to master's side, which is strictly ahead — see below)
$ git diff origin/master rebase-1061 --stat
   # (no output — trees are identical)

All three commits were absorbed by #1100 (81450a1c8, "Add reusable homotopy solver caches"):

This PR's commit	Superseded by
ab051325b — FixedShapeJacobianDestination / J_destination, restructure same-object fast path	81450a1c8
ffc0113af — cached ForwardDiff Jacobian destinations	81450a1c8
3d568afa1 — _solve_without_solution! / _solve_result_*	81450a1c8

Confirmed with git log -S on each distinctive symbol (FixedShapeJacobianDestination, J_destination, y === x ? x : ArrayInterface.restructure, _solve_without_solution!) — every one first appears in 81450a1c8 and nowhere else.

Master is strictly ahead, not merely equivalent

Resolving the conflicts to master's side loses nothing, because master's version of each is a superset:

• _solve_without_solution! now guards on applicable(InternalAPI.step!, cache) && hasfield(…, :termination_cache) && hasfield(…, :trace) and returns a materialized solution when a BoundedWrapper is present, rather than unconditionally returning the cache. This PR's version would have returned a bare cache for bounded problems.
• _has_bounded_wrapper(cache) replaces this PR's inline hasfield(typeof(cache.prob), :f) && hasfield(typeof(cache.prob.f), :f) && cache.prob.f.f isa BoundedWrapper chain.
• homotopy_sweep.jl gained _homotopy_corrector!, contraction-rejection handling (_strict_contraction_rejection, _rejected_step_size, _accepted_step_size) and _store_sweep_state! on top of the _solve_result_* calls introduced here.
• solve.jl gained _run_cache_to_completion! with a step_observer hook and a public solve_cache! entry point.

No test assertion was lost

$ diff <(git show <pr-head>:lib/NonlinearSolveBase/test/allocation_fastpaths.jl) \
       <(git show origin/master:lib/NonlinearSolveBase/test/allocation_fastpaths.jl)
   # (identical)

$ comm -23 <(git show <pr-head>:test/Core/homotopy_alloc_tests__item1.jl | grep -o '@test.*' | sort -u) \
           <(git show origin/master:test/Core/homotopy_alloc_tests__item1.jl | grep -o '@test.*' | sort -u)
   # (empty — every assertion in this PR exists on master)

Master additionally carries a cached_sweep_allocations regression that this PR did not have.

The goal verifies on current master

The blocking dependency named in the PR description — SciML/LinearSolve.jl#1099, "Reuse direct BLAS LU workspaces during refactorization" — merged 2026-07-23 and ships in LinearSolve v5.6.0. Reran both allocation suites against unmodified master with that dependency resolved:

$ julia +1.12 --project=. -e 'include("test/Core/homotopy_alloc_tests__item1.jl")'
LinearSolve version: 5.6.0
Julia: 1.12.6 x86_64-linux-gnu
Test Summary:               | Pass  Total   Time
homotopy_alloc_tests__item1 |   17     17  14.8s

$ julia +1.12 --project=. -e 'include("lib/NonlinearSolveBase/test/allocation_fastpaths.jl")'
Test Summary:        | Pass  Total  Time
allocation_fastpaths |    6      6  2.8s

That covers the cache-only _solve_without_solution! path (0 B, returns the same cache), the fixed-shape Jacobian destination (0 B, reshape is identity), the same-object restructure fast path (0 B), the cached-sweep regression (0 B), and the end-to-end per-step slopes for both HomotopySweep and ArcLengthContinuation (< 48 B/step on the nbig = 50 problem).

Not verified

• Apple silicon. The open cross-platform question in the PR description (772.0/400.0 B per sweep/arclength step on ARM) is untested here — this run is x86_64 Linux only. LinearSolve#1099 has since merged, so it is worth re-measuring on ARM, but that is independent of this PR and needs Apple hardware.
• Full Core group. Only the two allocation files above were run, not the whole suite. master's own CI is green apart from Downgrade / Core and OrdinaryDiffEq.jl/Interface, both unrelated to this work.

The red checks visible on this PR are from its stale July base and do not reflect master.

Recommend closing as superseded — no code change is needed. Leaving open for @ChrisRackauckas to confirm.