Skip to content

Widen UJacobianWrapper.p for nested ForwardDiff through Rosenbrock (#3381)#3414

Merged
ChrisRackauckas merged 1 commit into
SciML:masterfrom
ChrisRackauckas-Claude:fix-nested-forwarddiff-tag-in-wrapfun
Apr 30, 2026
Merged

Widen UJacobianWrapper.p for nested ForwardDiff through Rosenbrock (#3381)#3414
ChrisRackauckas merged 1 commit into
SciML:masterfrom
ChrisRackauckas-Claude:fix-nested-forwarddiff-tag-in-wrapfun

Conversation

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Contributor

Fix for #3381. Revives the approach from the (closed) #3389 — widen UJacobianWrapper.p once inside jacobian! in OrdinaryDiffEqDifferentiation, before DI.jacobian! runs, so the per-chunk hot loop is allocation-free. Supersedes #3412.

Summary

  • _widen_uf_p_for_jac(f, prep) in derivative_wrappers.jl inspects the prepared ForwardDiff.JacobianConfig to recover the inner nested-Dual eltype and lifts uf.p into that type via a fresh UJacobianWrapper with convert.(inner_T, p). Runs once per jacobian! call — every chunk DI evaluates inside that call reuses the widened p.
  • Fast-path dispatch (_widen_uf_p_for_jac(f, prep) = f) is a single type-stable no-op for non-nested, AutoFiniteDiff, or FWW-wrapped cases.
  • Skips widening when the user function lives behind a FunctionWrappersWrapper (AutoSpecialize): DiffEqBase already precompiles the (nested_u, outer_p, t) FW slot, so the unwidened call matches; widening would yield (nested_u, nested_p, t) that matches no slot and trips AllowNonIsBits into NoFunctionWrapperFoundError.
  • Adds test/nested_forwarddiff_tests.jl. The hand-rolled outer-tag case deterministically reproduces the crash pre-fix (MethodError: no method matching Float64(::Dual{...nested...})) and passes post-fix.
  • Bumps OrdinaryDiffEqDifferentiation to 2.9.1.

Root cause

NonlinearSolve seeds p with Dual{NonlinearSolveTag, Float64, 2}; inside resid! the Rosenbrock solve seeds u under OrdinaryDiffEqTag. For a FullSpecialize ODEFunction, the user body multiplies p[i] * u[i] across two different Dual nesting levels. Cross-tag multiplication resolves through ForwardDiff's @generated tagcount precedence — the literal value is baked at first compile and depends on which package precompiled which tag first. That ordering can invert the nesting and produce a triple-nested Dual that cannot be stored back into du:

MethodError: no method matching Float64(::Dual{Tag{OrdinaryDiffEqTag, Dual{Tag{NonlinearSolveTag, Float64}, Float64, 2}}, Dual{...}, 2})

Lifting p on the inner solver's side normalizes both sides of p[i] * u[i] to the same nested-Dual type, so the cross-tag arithmetic never happens.

Why here, not DiffEqBase

A DiffEqBase-side change could promote the compiled FWW signature's p slot to carry the nested Dual, but it needs either an eager Vector{DualU} allocation per UJacobianWrapper call, or a lazy WidenedDualArray wrapper whose exact type parameters have to match the compiled FW slot. Neither is free, and both push the bug-class workaround into a package that otherwise does not know about OrdinaryDiffEq's Jacobian seeding. Widening once in jacobian! costs a single convert.(inner_T, p) per step and covers every chunk DI evaluates inside that step.

Test plan

  • Pkg.test(\"OrdinaryDiffEqDifferentiation\") Core group passes (25 tests, including 4 new nested-ForwardDiff tests).
  • Hand-rolled outer-tag test errors pre-fix and passes post-fix.
  • Original MRE from ForwardDiff errors for NonlinearSolve over ODE solve #3381 runs to StalledSuccess with the fix.
  • CI.

🤖 Generated with Claude Code

@ChrisRackauckas-Claude ChrisRackauckas-Claude force-pushed the fix-nested-forwarddiff-tag-in-wrapfun branch 2 times, most recently from 3d17dc0 to eca731d Compare April 13, 2026 04:54
@AdityaPandeyCN

Copy link
Copy Markdown

@ChrisRackauckas Earlier I thought the ForwardDiff < was the root cause as tagcount is @generated and bakes values at precompile time, so cross-tag multiplication produces the wrong nesting order. A fix (if a tag's value type is already Dual, it's structurally outer) would make ordering deterministic . I opened a issue for this on ForwardDiff for discussion(JuliaDiff/ForwardDiff.jl#801 (comment)).

What do you think of this?

…ciML#3381)

When a Rosenbrock integrator runs with a Vector{<:Dual} p (i.e. we are inside
an outer ForwardDiff layer - e.g. a NonlinearLeastSquares parameter fit), the
inner Jacobian widens u into a deeper nested-Dual type via its prepared
JacobianConfig, but p in UJacobianWrapper is still at the outer Dual level.
The user body then multiplies p[i] * u[i] across two different Dual nesting
levels and dispatches through ForwardDiff's @generated tagcount precedence
whose literal value is baked at first compile and varies with precompile
ordering. That ordering can invert the nesting and crash setindex!(du, ...)
with MethodError: no method matching Float64(::nested_dual).

Two-layer fix:

* OrdinaryDiffEqDifferentiation/src/derivative_wrappers.jl: `jacobian!` now
  lifts `uf.p` into the inner nested-Dual type once (via
  `_widen_uf_p_for_jac`) before delegating to `DI.jacobian!`. The widened `p`
  carries zero inner partials (correct - `p` does not depend on `u`). One
  `convert.(inner_T, p)` per `jacobian!` call is amortized across every
  chunk DI evaluates - no per-call allocation in the hot loop.

* DiffEqBase/ext/DiffEqBaseForwardDiffExt.jl: `wrapfun_iip` now compiles the
  Jacobian-case FunctionWrapper slots with the promoted `p` type, so that
  AutoSpecialize (FWW-wrapped) callers dispatch to a nested-`p` slot whose
  compiled body multiplies `u*p` within one tag hierarchy. `_dualify_eltype`
  bypasses ForwardDiff's unstable tag-precedence `promote_type` for the
  already-Dual case (which was inverting the outer tag non-deterministically
  across precompile boundaries) and uses the seeded DualT directly.

Without the DiffEqBase change the AutoSpecialize path either matched a slot
compiled with the wrong tag ordering (silently cross-tag-multiplying and
crashing in `setindex!`) or missed FWW dispatch entirely with
`NoFunctionWrapperFoundError`, depending on precompile order.

Adds test/nested_forwarddiff_tests.jl covering FullSpecialize, a hand-rolled
outer tag (the deterministic reproducer pre-fix), and the AutoSpecialize /
FunctionWrappersWrapper path.

Bumps OrdinaryDiffEqDifferentiation 2.9.0 -> 2.9.1 and DiffEqBase 6.216.0 ->
6.216.1.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
@ChrisRackauckas-Claude ChrisRackauckas-Claude force-pushed the fix-nested-forwarddiff-tag-in-wrapfun branch from eca731d to 6e61344 Compare April 30, 2026 07:25
@ChrisRackauckas ChrisRackauckas merged commit fce9d5a into SciML:master Apr 30, 2026
143 of 150 checks passed
ChrisRackauckas-Claude pushed a commit to ChrisRackauckas-Claude/OrdinaryDiffEq.jl that referenced this pull request May 13, 2026
…ciML#3381)

Re-applies the changes from PR SciML#3414 on top of current master. The
original PR was reverted by SciML#3586; this restores it now that SciML#3488
(JacReuseState dtgamma fix) has landed and the two changes coexist
without conflict (they touch disjoint files).

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
ChrisRackauckas-Claude pushed a commit to ChrisRackauckas-Claude/OrdinaryDiffEq.jl that referenced this pull request May 21, 2026
…ciML#3381)

Re-applies the changes from PR SciML#3414 on top of current master. The
original PR was reverted by SciML#3586; this restores it now that SciML#3488
(JacReuseState dtgamma fix) has landed and the two changes coexist
without conflict (they touch disjoint files).

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants