Unify array-equation support behind a capability trait; fold in #5101 ODE support and MultiObjectiveOptimizationFunction

[ChrisRackauckas-Claude]

Ignore until reviewed by @ChrisRackauckas.

Summary

Stacked on the merged #5131 (array unknowns via flat_unknowns); folds in the closed #5049 (array equations on the nonlinear path) and the open #5101 (array differential equations on the ODE path), and, as the last commit, the multi-objective optimization work from ChrisRackauckas-Claude:multiobjective-optimization. If this lands, #5101 and that branch are subsumed.

"Accepts array equations" is a named, per-constructor capability: accepts_array_equations(constructor) gates __process_SciMLProblem, and ODEFunction, DAEFunction and NonlinearFunction opt in. All three lower an array equation to one scalar row per element through the shared array_residual_maker; full_equations scalarizes array equations, so everything downstream of it (symbolic Jacobians, sparsity, tgrad, the initialization system) sees one row per residual without further changes.

Review of the earlier revision and what changed

The earlier revision of this PR (through d834c1a) was reviewed in full. Everything in it was correct as far as tests and probes could tell; the following was simplified or extended on top:

• The residual-form rewrite is gone. explicit_array_derivative_form pattern-matched D(u[2:4]) .- f ~ 0 back into D(u[2:4]) ~ f in generate_rhs, calculate_massmatrix, full_equations and build_explicit_observed_function. The scalar path accepts no such thing: D(x) - f ~ 0 is a DAE and must go through mtkcompile or DAEProblem. Array equations now behave the same way (the ODE path rejects them with the same "nondifferentiated variables" error, and DAEProblem keeps accepting them, as array_equation_dae.jl tests). That removes ~50 lines and five call sites.
• jac = true / sparse = true work for array residuals on NonlinearFunction and DAEFunction instead of throwing. Since full_equations scalarizes, calculate_jacobian and jacobian_sparsity already produce one row per residual; the guards only hid that. Tested against a ForwardDiff Jacobian (nonlinear, dense and sparse, including a solve) and against the analytic stencil with the γ term (DAE, dense). Consequence: WIP: build implicit DAE Jacobians and sparsity from scalarized array residuals #5136 is not needed for the Jacobian itself. sparse = true on a DAE in residual form still errors inside calculate_massmatrix ("Only semi-explicit constant mass matrices"); that is pre-existing and equally true for scalar residual-form DAEs, so it is out of scope here.
• array_equation_dae.jl was included twice (InterfaceI on master, and again in InterfaceII by this PR). The duplicate is removed.
• The array-residual DAE tests solve with BrownFullBasicInit again rather than a hand-computed consistent du0 under NoInit(), so solver-driven DAE initialization is exercised through the array residual. Build ODEProblem from array differential equations with complete only #5101 had switched to NoInit because BrownFullBasicInit tripped a ParameterDespecializationWrapper assertion on the downgrade job's DiffEqBase 7.15; the DiffEqBase = "7.18.1" floor this PR already carries is what fixes that, verified by running the test against a pinned DiffEqBase 7.18.1 (see below).
• Docs: the "use mtkcompile before jac/sparse" notes are removed since they no longer apply; the nonlinear tutorial note describes what array equations look like instead.

Multi-objective optimization (last commit)

OptimizationProblem accepts f::MultiObjectiveOptimizationFunction in SciMLBase but MTK never produced one (generate_cost always consolidates to a scalar).

• costs(sys): the unconsolidated objective vector (own costs, then the consolidated cost of each subsystem, namespaced). Exported and documented next to cost.
• MultiObjectiveOptimizationFunction(sys): vector-valued objective over costs(sys); jac builds the objective Jacobian; hess is a vector of generated functions, one per objective, which is the shape OptimizationBase.instantiate_function iterates ([... for h in f.hess]). The original branch generated one function returning a vector of matrices, which that consumer cannot use.
• The six constraint fields (cons, cons_j, cons_h, their prototypes, cons_expr) are assembled by one generate_constraint_fields shared with OptimizationFunction instead of a duplicated 25-line block.
• OptimizationProblem(sys, op; multiobjective = true) routes through it; weights is rejected in combination with it.
• All new public names (costs, the generate_multiobjective_* / calculate_multiobjective_* functions, the SciMLBase constructor) have docstrings and @docs entries.

Not in scope

generate_rhs residual codegen is unchanged from #5049; OOP Dual promotion for DAEProblem{false}; array constraints in generate_cons; sparse Jacobians for residual-form DAEs (see above).

Verification

Julia 1.12.7, ModelingToolkitBase test environment on this machine.

All commands run from lib/ModelingToolkitBase unless noted; GROUP=… julia --project -e 'using Pkg; Pkg.test()' for the groups.

What	Result
GROUP=InterfaceI	InterfaceI | 1732 pass | 5 broken | 1737 | 43m52s — Testing ModelingToolkitBase tests passed. The 5 broken are pre-existing @test_brokens; this PR adds none.
GROUP=InterfaceII	every top-level testset green, tests passed: Array Equation ODEProblem Test | 51 | 51, Array-equation Nonlinear | 172 | 172, NonlinearSystem Test | 113 | 113, Code Generation Test | 65 | 65, JumpSystem Test | 4207 | 4207, … (2 pre-existing broken in Optimal Control + Constraints).
test/array_equation_dae.jl (InterfaceI member, rerun alone after the last test edit)	array_equation_dae.jl | 32 | 32 | 1m02s
test/optimization/multiobjective.jl (new, Optimization group)	optimization/multiobjective.jl | 35 | 35 | 8m38s
test/optimization/optimizationsystem.jl (shares generate_constraint_fields)	126 pass | 1 broken (pre-existing) | 127 | 3m22s
GROUP=QA	JET level tests 54 | 54; Aqua: 20 of 21 pass. The one failure is JET.report_package(...; mode = :typo) with 234 reports, all local variable ##And#…/##Call#… may be undefined from Moshi.@match expansions, none in code this PR touches. That check has never been green on master since it was enabled (#4832); a separate agent reproduced 234 on clean master with the same JET/Moshi/SymbolicUtils versions, reduced it to a Moshi+JET-only MWE, and confirmed 0 reports with the upstream fix Roger-luo/Moshi.jl#93. Tracked in #5063; no new issue opened.
BrownFullBasicInit on the array-residual DAE at the compat floor	scratch env with DiffEqBase pinned to 7.18.1 (and separately 7.21.1): retcode Success, max error 3.0e-3 against the analytic heat solution, same as the test asserts.
Root package GROUP=QA (repo root)	QA | 40 | 40 | 4m35s, Testing ModelingToolkit tests passed after adding the six new reexported names to test/qa/qa.jl's allow-list (the first CI run failed only that check).
Runic (--check --diff) and typos over every changed file	clean.
Docs (julia --project=docs docs/make.jl, repo root)	builds through Doctest, ExpandTemplates (all @example/@docs blocks), CrossReferences, CheckDocument (checkdocs = :exports) and RenderDocument; only pre-existing size_threshold_warn warnings. The external linkcheck step was disabled for the local rerun after GitHub answered 429 to the NEWS.md link on the first run.

Not verified: Julia LTS / pre; the downgrade CI job itself (only the 7.18.1 pin above); Optimization group's dynamic_optimization.jl / test_infiniteopt.jl (the Pyomo/CasADi pixi install fails on this machine's network, unrelated); downstream packages; GPU. sparse = true on residual-form DAEs is a pre-existing calculate_massmatrix limitation and is intentionally not covered.

Judgment calls to push back on: (1) dropping the residual-form ODE rewrite from #5101 (array D(u) .- f ~ 0 now needs DAEProblem/mtkcompile, exactly like scalar); (2) hess on the multi-objective function is a vector of per-objective functions rather than one function returning a vector of matrices, chosen to match OptimizationBase; (3) jac/sparse guards removed instead of extended.

CI on ee92d65 (first push) and e4f7d29 (second push)

Everything is green except the following, all of which reproduce on master's own CI and are not touched by this PR:

• sublibrary-ci / lib/ModelingToolkitBase [QA] / Julia 1 and the root tests / QA JET step: the 234 Moshi @match reports, tracked in ModelingToolkitBase QA lane red since #4832: JET typo mode reports 263 errors, 234 of them from Moshi @match codegen #5063 (the root tests / QA job additionally flagged the six new reexports; fixed by the last commit).
• tests / InterfaceI and tests / InterfaceII on Julia 1 and lts, downgrade-mtkbase (InterfaceI|InterfaceII), Downgrade Tests - InterfaceI: every failure is inside the "Array unknowns on a completed system" / "DAEProblem" testsets that Support array unknowns on a completed system for every problem type #5131 added to odesystem.jl, sdesystem.jl and nonlinearsystem.jl (raw u0/f/g order comparisons across complete vs mtkcompile, and symbolic_container(::IndexCache) missing under the downgrade dependency set). Master's run for the Support array unknowns on a completed system for every problem type #5131 merge commit https://github.com/SciML/ModelingToolkit.jl/actions/runs/34779646543 fails the same jobs; the previous master commit was green. Fixed separately: Raise SymbolicIndexingInterface compat lower bound to 0.3.47 #5153 raises the SymbolicIndexingInterface floor to 0.3.47 (the version whose getsym no longer recurses into parameter_index(::IndexCache, ::Int)), and Compare complete/mtkcompile array-unknown problems through symbolic indices #5154 makes those tests compare through symbolic indexing instead of assuming mtkcompile keeps the array unknown's element order (with ModelingToolkitTearing loaded it reverses it). This PR's own array_equation_dae.jl passes on the downgrade job, i.e. BrownFullBasicInit on the array residual works at the DiffEqBase 7.18.1 floor in CI as well.
• sublibrary-ci / lib/ModelingToolkitBase [Initialization] / Julia lts on the second push only: initializationsystem.jl "Initialization problem type is shared across models, use_scc = true", ForwardDiff.gradient(loss, [1.5])[1] evaluates to 9.898834314793689e154. The same job with identical dependency versions passed on the first push, and master's run for df57469 (https://github.com/SciML/ModelingToolkit.jl/actions/runs/34766850677) fails with the identical value, so it is a pre-existing intermittent failure, tracked in Flaky: downgrade Initialization use_scc = true gradient test returns 1e153 (Dual solve goes non-finite at t=0) #5125. Root cause is upstream: the remaked initialization problem has Dual u0 but a non-Dual-array p, so NonlinearSolve's Dual hooks do not fire and default_nlsolve's Broyden iterates in Dual arithmetic, letting the partials grow to ~1e155 while only the value residual is checked for convergence.
• Downstream SciMLSensitivity.jl/Core8/1/: Enzyme through init in SciMLSensitivity's own desauty_dae_mwe.jl raises EnzymeNoTypeError. Fails on every master Downstream.yml run since Concretize initialization problems once at construction #5062 (first bad job https://github.com/SciML/ModelingToolkit.jl/actions/runs/34695771592/job/103558875060, identical dependency set as the last good one) and on SciMLSensitivity's own CI against released MTK 11.43.1; tracked in Enzyme SCC initialization regresses at concretize_initializeprob on Julia 1.13 #5140.
• Docs build: makedocs stops at [:linkcheck] because https://docs.sciml.ai/PDEBase/stable/interface/ (a link Document the PDE solution interface shared by discretizers #5149 added on master) answers 403; master's own docs job for that commit fails the same way. Everything before linkcheck (doctests, all @example/@docs blocks, cross-references, checkdocs) passes on CI, and the full render passes locally with linkcheck off (see Verification).
• downgrade-mtkbase (Optimization) / test: cancelled after the 6-hour limit on both pushes, exactly as on master's run for 444ccfb; the group is known to be slow (see test_groups.toml) and the downgrade dependency set makes it slower still.

Links

• Subsumes Build ODEProblem from array differential equations with complete only #5101 and feat: accept array equations on the NonlinearProblem path #5049 (closed); stacked on Support array unknowns on a completed system for every problem type #5131 (merged)
• Makes WIP: build implicit DAE Jacobians and sparsity from scalarized array residuals #5136 unnecessary for dense DAE Jacobians
• Multi-objective source branch: https://github.com/ChrisRackauckas-Claude/ModelingToolkit.jl/tree/multiobjective-optimization
• Separate one-line fix found while building the docs locally: Point the docs ModelingToolkit source at the repository root #5152
• Master regressions from Support array unknowns on a completed system for every problem type #5131 found via this PR's CI, fixed separately: Raise SymbolicIndexingInterface compat lower bound to 0.3.47 #5153 and Compare complete/mtkcompile array-unknown problems through symbolic indices #5154
• Pre-existing QA JET failure: ModelingToolkitBase QA lane red since #4832: JET typo mode reports 263 errors, 234 of them from Moshi @match codegen #5063

🤖 Generated with Claude Code (model: claude-fable-5-1); the earlier revision of this PR was generated with Devin CLI (model: SWE-2 Max).

https://claude.ai/code/session_01VorhgJYhDasyVLpGNrviga

[ChrisRackauckas]

I finally like it, and there's a lot riding on it downstream so I'm going for it, but @AayushSabharwal please take a look and if any clean ups are required we can modify post.