Repository navigation
ReverseDiff 1.18.7: setup_next_step! fill! throws setindex! on TrackedArray (breaks all fresh-resolve SciMLSensitivity ReverseDiff gradients) #346
Description
ReverseDiff 1.18.7 (JuliaDiff/ReverseDiff.jl#301) makes setindex!/broadcast-to-destination throw on TrackedArray. setup_next_step! in src/noise_process_interface.jl:137 does a fill! on the (possibly wrapped) state, reached from SciMLSensitivity.jl/src/concrete_solve.jl:2395 (reversediff_adjoint_forwardpass) during reverse-mode SDE gradients. Result on current resolve: ArgumentError: TrackedArrays do not support setindex! ... setindex! at ReverseDiff tracked.jl:430 → fill! → DiffEqNoiseProcess noise_process_interface.jl:137 setup_next_step! → SciMLSensitivity concrete_solve.jl:2395.
Discriminator (from the qa-hygiene fleet lane, Julia 1.13.1, JULIA_NUM_THREADS=1; program = test/SDE3/rode.jl lines 1–76 noise iip set closed with end, same manifest, only ReverseDiff version + developed SciMLSensitivity path changed):
- ReverseDiff 1.18.5:
noise iip narrow | 3 pass 3 total, exit 0 (on both masterdaeb4032and the PR head under test). - ReverseDiff 1.18.7: 1 pass / 1 error, exit 1, error above.
Versions: DiffEqNoiseProcess 5.36.5, StochasticDiffEq 7.2.0, SciMLSensitivity masterdaeb4032. CI evidence: master push CI run https://github.com/SciML/SciMLSensitivity.jl/actions/runs/39047544162 (ReverseDiff 1.18.5 — SDE3 green) vs PR run https://github.com/SciML/SciMLSensitivity.jl/actions/runs/40737715875 (1.18.7 — SDE3/Callbacks2/Core2/Core1 red on Julia 1/1.11/lts).
Blast radius: every SciMLSensitivity job that resolves fresh now pulls ReverseDiff ≥1.18.6 and fails the SDE noise-iip reverse-gradient tests — this is blocking unrelated PRs org-wide (CI gates compare against the old green master run, so these read as PR-only regressions).
Candidate fixes (owner call): make the fill! in setup_next_step! wrapped-array safe (broadcast/copyto! style assignment instead of fill!/setindex! destination), or gate/until upstream compat in the SciMLSensitivity ReverseDiff forward-pass path. No existing SciML issue found for this.
🤖 Reported by an AI agent (Claude Code, model qwen3.8-flash-next) on behalf of Chris Rackauckas, from repro work run by a Devin worker (local job 1684-grok, no session URL).
Activity
ChrisRackauckas-Claude commentedon Oct 28, 2026
First-pass evidence from the qa-hygiene lane (Devin SWE-2 measurement job, no code pushed) — the fix site is measured, not guessed:
Trace. The
TrackedArrayis the noise process's own storage (W.dW/curW/history). Trackedness enters atStochasticDiffEqCore._sde_init'srand_prototype = (u .- u) ./ sqrt(oneunit(t))overu::Vector{TrackedReal}: ReverseDiff 1.18.5 returnedVector{TrackedReal}for that broadcast; 1.18.6+ returns aTrackedArray(JuliaDiff/ReverseDiff.jl#301's newTrackedStyleclaimsAbstractArray{<:TrackedReal}).setindex!onTrackedArraythrew in both — the broadcast result type is what moved. Verified empirically.Candidate A (this package, ReverseDiff ext): convert
TrackedArray→Array{TrackedReal}atNoiseProcess{true}construction inside the existingDiffEqNoiseProcessReverseDiffExt(+18 lines). Results: discriminator 3/3 green; full SciMLSensitivityGROUP=SDE3green — 156 pass / 2 pre-existing broken / 158; vector-diagonalSDEProblemreproducer fixed, gradients match ForwardDiff to ~1e-13.Candidate B (SciMLSensitivity forward-pass): +9 lines, discriminator + SDE3 green, but
SDEProblemhas norand_prototypefield — SDE coverage needs RNG-stream replication (a re-implementation slice of_sde_init). Partial coverage only.Recommendation input: Candidate A — one place, covers RODE+SDE, restores exactly the storage shape that worked under 1.18.5, and this package already owns the ReverseDiff extension whose purpose is exactly this adaptation. A possible third home is
rand_prototypeconstruction atStochasticDiffEqCore/solve.jl:413. No existing PR/issue addresses this (searched both repos, open included); ReverseDiff's mutation ban is intentional (their #43, open since 2017) and there is no yank in General; a compat cap is possible but blunt. Note: this package's CI never loads the ReverseDiff ext today (no ReverseDiff in the test env) — the fix PR should add an ext-loading test.Draft PR implementing Candidate A (with an ext test so CI covers this path) is being opened by the same worker session.
🤖 Evidence produced by an AI agent (Devin CLI, model swe-2-high, local job 346-devin, no session URL); comment posted by the qa-hygiene fleet head (Claude Code, model qwen3.8-flash-next).