CHECKPOINT: preserve array DEs through mtkcompile (ArrayEquationGroup / not skip-tearing)

[ChrisRackauckas-Claude]

Checkpoint (usage running out)

Draft / partial. Full status writeup also in-tree: docs/src/internals/ARRAY_EQUATIONS_CHECKPOINT.md.

Opened to save progress before agent usage hits the limit. Do not merge until StateSelection/MTKTearing companion lands and CI is green.

Related

Item	Role
This PR	MTK wiring: preserve_array_equations, docs, structural tests, codegen helpers
StateSelection mtkt-array-equation-groups (draft, same session)	ArrayEquationGroup in ModelingToolkitTearing
#5101	complete → ODEProblem for array DEs (codegen; no mtkcompile) — first useful slice
#5100	Closed — scalarize_arrays=false / skip tearing rejected as non-useful intermediate
MethodOfLines #688	Still based on #5100; needs rewrite

Goal (from Chris)

1. Array eqs + ODEProblem should build (prefer complete path first) — Build ODEProblem from array differential equations with complete only #5101.
2. Hard problem: index reduction and tearing keep array equations intact, not turn those algorithms off — this checkpoint + StateSelection.

What is done

Design (not skip-tearing)

• Scalarize array DEs into bipartite rows (algorithms still see per-element incidence).
• Track parent equation as ArrayEquationGroup; dirty when a pass breaks representability.
• Keep intact group rows out of integer-linear GE pivots/reductions.
• Reassemble intact groups when mtkcompile(...; preserve_array_equations=true).
• Default mtkcompile behavior unchanged.

Code in this branch

• preserve_array_equations keyword / reassemble wiring
• docs: docs/src/internals/mtkcompile.md + ARRAY_EQUATIONS_CHECKPOINT.md
• tests: test/structural_transformation/array_equations.jl
• codegen helpers for preserved array emission
• alias-elimination notes aligned with is_intact_array_group_row

Local: MTKTearing tests Pass; sample MTK structural/codegen/init tests Pass after env deps fixed.

What is required next

1. Product call: is scalar-graph + reassemble enough, or do we need atomic array nodes in the bipartite graph for true O(1) mtkcompile? (Structural simplify is still O(N) in graph size today even when codegen can be O(1).)
2. Merge/register StateSelection companion; bump MTKTearing compat in MTK; make CI resolve the new tearing.
3. End-to-end tests: mtkcompile(sys; preserve_array_equations=true) → ODEProblem/DAEProblem, Expr node count independent of n (compose with Build ODEProblem from array differential equations with complete only #5101).
4. Dirty-marking audit + high-index (differentiate array eq as a unit vs dirty→scalar).
5. Init blockers (DAEProblem from a system with array-form differential equations fails with MethodError: no method matching vec(::Bool) #5093) and Preserve array derivatives in implicit DAE residual code generation #5097 (expand_array_derivatives! unrolling).
6. Rewrite MethodOfLines expand_derivatives issues #688; drop scalarize_arrays=false docs.
7. Decide default-on vs opt-in for eligible array DEs.

Explicitly rejected

mtkcompile(; scalarize_arrays=false) / tearing-free compiler as the solution (#5100).

How to verify (once MTKTearing companion is developed)

using ModelingToolkit
using ModelingToolkit: t_nounits as t, D_nounits as D
n = 96
@variables u(t)[1:n]
@named heat = System([D(u[2:(n-1)]) ~ (u[1:(n-2)] .- 2 .* u[2:(n-1)] .+ u[3:n]) ./ (1/(n-1))^2,
                      u[1] ~ 0, u[n] ~ 0], t, [u], [])
sys = mtkcompile(heat; preserve_array_equations=true)
# expect: one interior array DE + observed BCs; not n-2 scalar DEs
equations(sys)

Also see #5101 for complete(sys) → ODEProblem without mtkcompile.

Made with Cursor

[ChrisRackauckas-Claude]

Fable tearing agent hit the usage limit mid-flight. Branches are pushed; treat this draft + JuliaComputing/StateSelection.jl#158 as the hard-path checkpoint.

Latest local note: checkpoint markdown was moved out of the docs tree (18af7bb) so docs CI does not need to render it.