Skip to content

test: cover multi-objective optimization acceptance cases - #5196

Draft
ChrisRackauckas-Claude wants to merge 5 commits into
SciML:masterfrom
ChrisRackauckas-Claude:agent/5150-multiobjective
Draft

ChrisRackauckas-Claude wants to merge 5 commits into
SciML:masterfrom
ChrisRackauckas-Claude:agent/5150-multiobjective

Conversation

@ChrisRackauckas-Claude

@ChrisRackauckas-Claude ChrisRackauckas-Claude commented Sep 23, 2026 •

Copy link
Copy Markdown
Member

This test-only PR covers the existing multi-objective OptimizationProblem(sys, op; multiobjective = true) API: individual costs, analytic Jacobians with whole array unknowns, weighted-sum agreement, and a discretized Poisson weighted solve checked against normal equations and the cost-vector Jacobian. A bounded NSGA-II test checks that OptimizationEvolutionary consumes the generated objectives.

Depends on JuliaSymbolics/Symbolics.jl#2013, which fixes the hash-order-dependent nested-reduction gradient failure tracked in #5199. The unchanged Poisson test now passes locally on both Julia LTS and 1.11 using that Symbolics checkout. Keep this draft blocked until the Symbolics fix is reviewed, merged, and released, then raise the relevant dependency floor and validate with the released dependency. A cross-repository dependency cannot be expressed as this Git branch's base; the base remains MTK master.

OptimizationEvolutionary/Evolutionary and StableRNGs are MIT-licensed test dependencies. No production source or public API changes in this MTK PR.

Verification

From the MTK repository root, with TMPDIR set to the job tmp directory, OPENBLAS_NUM_THREADS=4, and a job-local writable depot followed by the shared dependency depot:

julia +1.10 --project=../scratch/5199/env -e 'using Test, SafeTestsets; @safetestset "MultiObjective Optimization" begin include("lib/ModelingToolkitBase/test/optimization/multiobjective.jl") end'
julia +1.11 --project=../scratch/5199/env -e 'using Test, SafeTestsets; @safetestset "MultiObjective Optimization" begin include("lib/ModelingToolkitBase/test/optimization/multiobjective.jl") end'

The scratch environment develops MTK from this branch and Symbolics at eff957511cd194d5586972ebf377d281a1c59e60; it uses SymbolicUtils 4.47.2. These are tests of the proposed upstream fix, not a claim that registered dependencies already work.

# Julia 1.10.12 — exit 0
Test Summary:               | Pass  Total      Time
MultiObjective Optimization |   97     97  15m17.1s
# Julia 1.11.9 — exit 0
Test Summary:               | Pass  Total      Time
MultiObjective Optimization |   97     97  24m22.6s

Before the Symbolics fix, round-2 review reproduced 94 passes and 1 error twice on Julia 1.10.12:

Differentiation of `u[1]` inside lazy array expressions is not supported; scalarize the expression with `Symbolics.scalarize` first

The upstream PR includes a standalone regression that fails on clean Symbolics master on both versions (6 pass, 1 failure, 6 errors), then passes with the fix (13/13); its full differentiation test file passes 380/380 on each version.

Earlier verification retained from this PR and its round-2 review:

  • Transposing the generated multi-objective Jacobian makes the array assertion fail and the Poisson checks reject the dimensions.
  • MTK QA: JET 54/54 and Aqua 21/21, exit 0 in round-2's scratch checkout.
  • Runic and typos on this MTK diff: exit 0.

Limits and reviewer notes

  • Full GROUP=Optimization remains unverified: Pyomo/CasADi precompilation raises LookupError: no codec search functions registered in the local environment.
  • No new MTK QA run with the patched Symbolics dependency; no docs, downstream NeuralPDE, other Julia versions, or OptimizationBase AD coverage.
  • The Symbolics fix passes its full local Core suite (17548 passes plus 40 interface assertions, with 366 existing broken markers) and both CI Core and QA configurations. Its CI finished with 25 successful checks (including all four MTK downstream groups) and two failures. The docs and NeuralPDE NNPDE2 failures match exact master; the prerequisite PR records the evidence and baseline investigations.
  • The Symbolics fix's local Julia 1.11 QA run fails on both its feature branch and clean master with identical failures (18 pass, 1 persistent-task failure, 1 ExplicitImports error). Details are in the prerequisite PR.
  • NSGA-II has no Pareto-front convergence assertion: the earlier 200-seed sweep did not establish convergence at a tolerance justified by the problem. The committed test checks a nonempty returned population and analytic objective values at every returned point. Do not read it as satisfying a stronger Pareto-convergence contract.
  • OptimizationBase AD support remains outside this MTK test PR. Review the dependency release floor and those remaining acceptance items before closing the tracking issue.

Part of #5150.

Please ignore this draft until reviewed by @ChrisRackauckas.

Links

Risk assessment

  • Risk: medium
  • Blast radius: this diff changes tests and the optimization test environment; the prerequisite affects Symbolics scalar differentiation and Hessian sparsity.
  • Evidence: unchanged 97/97 MTK assertions pass on Julia 1.10.12 and 1.11.9 with the proposed Symbolics fix; exact upstream failing-before/passing-after regression evidence is linked above.
  • Independent review: Codex gpt-6-astra round 2 rated this medium, high confidence, and returned CHANGES for the LTS failure. Its review remains the last verdict; the prerequisite fix has not received independent review. Local review record: /home/crackauc/sandbox/agent-jobs/ModelingToolkit.jl/reviews/pr-5196/round2/REVIEW.md.
  • Merge: needs human review; blocked on upstream release, registered-dependency validation, and a clean independent re-review. The reproduced baseline QA failures are documented rather than treated as regressions. Never auto-merge this draft.

🤖 Updated with Codex CLI 0.157.1 (model: gpt-6-astra); local session, transcript at /home/crackauc/sandbox/agent-jobs/ModelingToolkit.jl/jobs/5150-opencode/log-cont5.txt on amdci2.julia.csail.mit.edu. Earlier branch commits retain their opencode, Cursor, and Claude Code attribution.

codex and others added 2 commits September 23, 2026 06:11
Exercise multi-objective costs and their Jacobian with array unknowns, compare weighted scalarization to the objective dot product, and solve a textbook two-objective problem on its known Pareto front.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Codex <noreply@openai.com>
Agent-Harness: Codex CLI 0.156.1
Agent-Model: gpt-6-luna
Agent-Session: local session, transcript at /home/crackauc/sandbox/agent-jobs/ModelingToolkit.jl/jobs/5150-opencode/log-cont1.txt on amdci2.julia.csail.mit.edu
Drop the seed-fragile Pareto front claim, pin NSGA-II via StableRNG on a
bounded off-front problem, assert array-unknown Jacobian values, and declare
Random/SciMLBase/StableRNGs plus OptimizationEvolutionary ≥ 0.4.13.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Cursor Agent <noreply@cursor.com>
Agent-Harness: Cursor Agent CLI 2026.09.18-9a7762b
Agent-Model: auto
Agent-Session: local session, transcript at /home/crackauc/sandbox/agent-jobs/ModelingToolkit.jl/jobs/5150-opencode/log-cont2.txt on amdci2.julia.csail.mit.edu
…problem

Array-unknown 1D Poisson least-squares with Tikhonov regularization: the
multi-objective costs and Jacobian match analytic values, the weighted
objective equals dot(weights, costs), and the LBFGS minimizer of the
weighted problem matches the normal-equations solution and is stationary
for weights' * J.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Agent-Harness: Claude Code 2.1.280
Agent-Model: claude-opus-5-5
Agent-Session: local session, transcript at /home/crackauc/sandbox/agent-jobs/ModelingToolkit.jl/jobs/5150-opencode/log-cont3.txt on amdci2.julia.csail.mit.edu
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Devin <noreply@cognition.ai>
Agent-Harness: Devin CLI 3000.11.3 (conflict resolution), Claude Code 2.1.285 (orchestration)
Agent-Model: fusion-claude-opus-5-5-high-sidekick-swe-2-medium
Agent-Session: local Claude session 3c311569-dda6-4687-bc9f-65febb212c33 (fleet master); Devin log /Users/chrisrackauckas/sandbox/master/fusion-fixes/SciML_ModelingToolkit.jl-5196/log.txt
@ChrisRackauckas

Copy link
Copy Markdown
Member

🤖 Automated comment from an AI agent running as @ChrisRackauckas — not written or reviewed by Chris.

Merged the default branch into this PR (clean merge, no conflicts; head 4e98ad5) so CI runs on current master.


🤖 Posted by an AI agent — harness: Devin CLI 3000.11.3 (merge), Claude Code 2.1.285 (posting) · model: fusion-claude-opus-5-5-high-sidekick-swe-2-medium
Conversation: local Claude session 3c311569-dda6-4687-bc9f-65febb212c33 (fleet master)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants