Skip to content

Compare complete/mtkcompile array-unknown problems through symbolic indices - #5154

Merged
ChrisRackauckas merged 1 commit into
SciML:masterfrom
ChrisRackauckas-Claude:fix-array-unknown-order-tests
Sep 14, 2026
Merged

ChrisRackauckas merged 1 commit into
SciML:masterfrom
ChrisRackauckas-Claude:fix-array-unknown-order-tests

Conversation

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Member

Ignore until reviewed by @ChrisRackauckas.

What changed and why

#5131 added "Array unknowns on a completed system" testsets to lib/ModelingToolkitBase/test/{odesystem,sdesystem,nonlinearsystem}.jl that compare the completed problem with the mtkcompiled one by their raw u0, f and g vectors. The order of the scalarized unknowns after mtkcompile is not guaranteed: with ModelingToolkitBase alone it happens to match, but as soon as ModelingToolkit (ModelingToolkitTearing) is loaded it comes out reversed, and for the nonlinear system tearing solves every unknown symbolically so mprob.u0 === nothing. The root tests / InterfaceI and InterfaceII jobs load ModelingToolkit before including these files, so master has been red there since the #5131 merge commit 444ccfb on Julia 1 and lts (https://github.com/SciML/ModelingToolkit.jl/actions/runs/34779646543, jobs 103789585218, 103789585230, 103789585194, 103789585183); the previous commit 8689e54 was green. Noticed on #5148.

The code behaviour is correct (mprob[x] ≈ prob[x], msol[x][end] ≈ sol[x][end], and msol[z] ≈ sol[z] all hold), so this changes only the assertions: compare the mtkcompiled problem through symbolic indexing (mprob[x], mprob[w], and f/g evaluations permuted through variable_index) instead of by raw position, and for the nonlinear system, where tearing leaves nothing to solve, compare only the solutions. Test-only change; no source files are touched.

Verification (Julia 1.10.11, Pkg.develop of the clone root — ModelingToolkit 11.43.1, ModelingToolkitTearing 1.20.6 — with using ModelingToolkit before the testset, as test/shared/mtktestset.jl does)

Before (master 444ccfb0 testsets, extracted verbatim):

=== odesystem.jl testset
MTK=11.43.1 Tearing=1.20.6 julia=1.10.11
  Expression: mprob.u0 ≈ prob.u0
   Evaluated: [3.0, 2.0, 1.0, 4.0] ≈ [1.0, 2.0, 3.0, 4.0]
  Expression: mprob.f(mprob.u0, mprob.p, 0.0) ≈ prob.f(prob.u0, prob.p, 0.0)
   Evaluated: [-5.0, -4.0, -1.0, -4.0] ≈ [-1.0, -4.0, -5.0, -4.0]
Array unknowns on a `complete`d system |   22     2     24  22.5s
=== sdesystem.jl testset
  Expression: mprob.u0 ≈ prob.u0
   Evaluated: [4.0, 3.0, 2.0, 1.0] ≈ [1.0, 2.0, 3.0, 4.0]
  Expression: mprob.f(mprob.u0, mprob.p, 0.0) ≈ prob.f(prob.u0, prob.p, 0.0)
   Evaluated: [-4.0, -9.0, -4.0, -1.0] ≈ [-1.0, -4.0, -9.0, -4.0]
  Expression: mprob.g(mprob.u0, mprob.p, 0.0) ≈ prob.g(prob.u0, prob.p, 0.0)
   Evaluated: [0.4, 0.30000000000000004, 0.2, 0.1] ≈ [0.1, 0.2, 0.30000000000000004, 0.4]
Array unknowns on a `complete`d system |    7     3     10  16.3s
=== nonlinearsystem.jl testset
  Expression: mprob.u0 ≈ prob.u0
  MethodError: no method matching isapprox(::Nothing, ::Vector{Float64})
  Expression: mprob.f(mprob.u0, mprob.p) ≈ prob.f(prob.u0, prob.p)
  MethodError: no method matching create_array(::Type{Nothing}, ::Nothing, ::Val{1}, ::Val{(0,)})
Array unknowns on a `complete`d system |    9      2     11  12.0s

After (this branch, same environment):

=== odesystem.jl testset
Array unknowns on a `complete`d system |   26     26  21.6s
=== sdesystem.jl testset
Array unknowns on a `complete`d system |   12     12  15.4s
=== nonlinearsystem.jl testset
Array unknowns on a `complete`d system |   11     11  9.4s

The rewritten testsets also pass with ModelingToolkitBase alone (no tearing) in the downgrade-style environment (SymbolicIndexingInterface 0.3.47, SciMLBase 3.54.0, Symbolics 7.37.0, SymbolicUtils 4.37.0, DiffEqBase 7.18.1): ODE 26/26, SDE 12/12, nonlinear 11/11. typos and Runic (julia -m Runic --check) report nothing on the three files.

Not verified

  • Only the three affected testsets were run, not the whole odesystem.jl / sdesystem.jl / nonlinearsystem.jl files or the full InterfaceI/InterfaceII groups, and only on Julia 1.10.11 (CI fails on Julia 1 and lts; Julia pre passes on master, presumably because the ordering there happens to match).
  • Whether the reversed unknown order under tearing is itself intended is not something this PR decides; it only stops asserting on it. If the order is meant to be stable, that is a separate ModelingToolkitTearing question.
  • Independent of this PR, the downgrade jobs for the same testsets fail with MethodError: no method matching symbolic_container(::ModelingToolkitBase.IndexCache) (SymbolicIndexingInterface ≤ 0.3.46); that is Raise SymbolicIndexingInterface compat lower bound to 0.3.47 #5153.

🤖 Generated with Claude Code (model: claude-fable-5-1)
https://claude.ai/code/session_01VorhgJYhDasyVLpGNrviga

…mbolic indices

The "Array unknowns on a `complete`d system" testsets compared the raw `u0`,
`f` and `g` vectors of the `complete`d and the `mtkcompile`d problem. The
order of the scalarized unknowns after `mtkcompile` is not part of the
contract: once ModelingToolkitTearing is loaded it comes out reversed, and
for the nonlinear system tearing solves every unknown symbolically, so
`u0 === nothing`. Compare `mprob[x]`/`mprob[w]` with the `complete`d problem
and index the `f`/`g` evaluations through `variable_index` instead; for the
nonlinear system only the solution is comparable.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Agent-Harness: Claude Code
Agent-Model: claude-fable-5-1
Agent-Session: https://claude.ai/code/session_01VorhgJYhDasyVLpGNrviga
Claude-Session: https://claude.ai/code/session_01VorhgJYhDasyVLpGNrviga
@ChrisRackauckas
ChrisRackauckas marked this pull request as ready for review September 14, 2026 10:21
@ChrisRackauckas
ChrisRackauckas merged commit 6e91407 into SciML:master Sep 14, 2026
103 of 109 checks passed
ChrisRackauckas added a commit that referenced this pull request Sep 14, 2026
- Document the PDE solution interface shared by discretizers (#5149)
- Point the docs `ModelingToolkit` source at the repository root (#5152)
- Unify array-equation support behind a capability trait; fold in #5101 ODE support and MultiObjectiveOptimizationFunction (#5148)
- Raise SymbolicIndexingInterface compat lower bound to 0.3.47 (#5153)
- Compare `complete` and `mtkcompile` array-unknown problems through symbolic indices (#5154)



Agent-Harness: Claude Code
Agent-Model: claude-opus-5[1m]
Claude-Session: https://claude.ai/code/session_014FEzNTLFutCmTEAZ3zBg5R

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants