Skip to content

Construct SCCSteadyStateProblem from a System; drop the deferred SCC lowering - #5276

Draft
ChrisRackauckas wants to merge 11 commits into
SciML:masterfrom
ChrisRackauckas:scc-steadystate
Draft

ChrisRackauckas wants to merge 11 commits into
SciML:masterfrom
ChrisRackauckas:scc-steadystate

Conversation

@ChrisRackauckas

@ChrisRackauckas ChrisRackauckas commented Oct 14, 2026 •

Copy link
Copy Markdown
Member

🚧 UNREVIEWED — awaiting review by @ChrisRackauckas. Filed by an AI agent running as @ChrisRackauckas; Chris has not seen this.
Harness: Devin CLI 3000.11.3 (diagnosis and fix), Claude Code 2.1.285 (PR) · Model: fusion-claude-opus-5-5-high-sidekick-swe-2-medium
Conversation: local Claude session 3c311569-dda6-4687-bc9f-65febb212c33 (fleet master)

This fixes a failure on the default branch's CI. The diagnosis and the evidence below come from the investigating agent's report; "confirmed by running" and "read in logs" are kept apart there.

Part of a three-PR stack implementing Chris's direction (2026-10-14) to replace the post-solve SCC reversal with an SCC form of SteadyStateProblem stored in the compressed layout: SciMLBase (new type) → ModelingToolkit (constructs it) → SteadyStateDiffEq (solves it). Supersedes SciML/SteadyStateDiffEq.jl#175 and #177; fixes SciML/SteadyStateDiffEq.jl#171. CI on the downstream PRs cannot pass until the upstream ones are released.

ModelingToolkit.jl

Branch scc-steadystate (on master)

  • a89fdd8f2 Construct SCCSteadyStateProblem from a System; drop the deferred SCC lowering
  • Files: src/problems/sccnonlinearproblem.jl (SCCSteadyStateProblem(sys, op) with the
    {iip} and {iip, specialize} forms, docstring; deletes steady_state_sccprob and
    is_structurally_unbalanced_error), src/ModelingToolkit.jl (import),
    lib/ModelingToolkitBase/src/problems/odeproblem.jl (removes the stub, both
    SCCNonlinearProblem(::SteadyStateProblem/AbstractSciMLProblem) accessors and the
    lowered_problem kwarg), lib/ModelingToolkitBase/src/systems/problem_utils.jl
    (resid-prototype fix, below), lib/ModelingToolkitBase/test/qa/aqua.jl, test/qa/qa.jl,
    docs/src/API/problems.md, test/scc_nonlinear_problem.jl, and both Project.tomls
    (MTK 11.45.4 → 11.46.0, MTKBase 1.77.4 → 1.78.0, SciMLBase 3.59, MTKBase compat 1.78,
    test SteadyStateDiffEq 2.19).
  • The tests run every lowering regression case from
    Fix solve(::SteadyStateProblem) regressions from the deferred SCC lowering #5163 and
    SteadyStateProblem SCC lowering throws ExtraEquationsSystemException inside solve (also for DynamicSS) on structurally singular residuals #5177 (parents, initialization_eqs,
    bindings, hierarchy, multi-SCC, empty schedule, connectors, arrays, Initial/Γ) through
    both SteadyStateProblem and SCCSteadyStateProblem. For conservation systems,
    SCCSteadyStateProblem throws
    ModelingToolkit.StateSelection.ExtraEquationsSystemException.
  • mtk_scc_nonlinear_r2.log (test/scc_nonlinear_problem.jl, env-stack): all green, including
    SCCSteadyStateProblem                                   |  36  36
    SteadyStateProblem/SCCSteadyStateProblem regressions    |  69  69
    ...connectors, array unknowns and `Initial` values      |  24  24
    
  • Extra bug fix, with before/after: calculate_resid_prototype threw
    zero(Union{Nothing, Float64}) when a plain SteadyStateProblem with Initial
    placeholders was remaked (Γ case, remake(prob; p = [k2 => 4.0]) with SSRootfind()).
    On master the SCC lowering bypassed this path. The fix is a local tunable_real_eltype
    helper (Base.nonnothingtype is not public).
    • resid_prototype_before.log: solve2 ERRORED: MethodError
    • resid_prototype_after.log: solve2 ok: [2.4]
  • Runic --check and typos: clean.
  • Not verified: the full MTK suite, the QA groups (test/qa/qa.jl, MTKBase aqua.jl; only
    edited, not run) and the docs build. After the last test run I moved the 3-line helper
    above the calculate_resid_prototype docstring so the docstring stays attached; Runic was
    re-checked, but the tests were not re-run after that move.

Open design questions for Chris

  1. Plain SteadyStateProblem + nonlinear alg. With NewtonRaphson() the solution's
    sol.prob is the converted NonlinearProblem, as before the lowering existed (only
    SSRootfind/DynamicSS/SICNM give sol.prob isa SteadyStateProblem). The tests
    encode this. Should SSDE rewrap it? I left it alone because it changes the dispatch
    surface that SciMLSensitivity uses.
  2. Release order for removing the field. Should lowered_problem go now as SciMLBase
    4.0 (scc-steadystate-remove-lowered), or stay deprecated-but-unused in 3.x? The 4.0
    path needs SciMLBase = "3.59, 4" compat in MTK/SSDE and in every package that caps
    SciMLBase below 4.
  3. SemVer of the MTKBase accessor removal. The SCCNonlinearProblem(::SteadyStateProblem)
    method was a public-ish extension in MTKBase 1.77. I released its removal as a minor
    (1.78.0), on the grounds that it was only meaningful with the lowering. A strict reading
    makes it a major.
  4. Construction cost. SCCSteadyStateProblem(sys, op) builds a full
    SteadyStateProblem (including its initialization problem) only to process op, then
    compiles the residual system. build_initializeprob = false breaks hierarchical
    initialization_eqs. Is a lighter op-processing path worth adding?
  5. Monolithic DynamicSS with Tsit5. On the plain mix system, DynamicSS(Tsit5()) with
    the default tspan = Inf runs out of maxiters before the termination callback fires,
    even though the ODE is globally stable. A hand-written SteadyStateProblem behaves the
    same, so this is not caused by this change. The tests use tspan = 100. Worth an SSDE
    issue?
  6. No dedicated SCC solution type. SteadyStateSolution (= NonlinearSolution) on an
    SCCSteadyStateProblem carries the compressed layout through sol.prob.f.sys, and no
    dedicated SCC solution type was added. Is that the intended end state?

🤖 Posted by an AI agent — harness: Devin CLI 3000.11.3 (fix), Claude Code 2.1.285 (PR) · model: fusion-claude-opus-5-5-high-sidekick-swe-2-medium / claude-opus-5-5
Conversation: local Claude session 3c311569-dda6-4687-bc9f-65febb212c33 (fleet master)

🤖 Generated with Claude Code

Independent review (Codex CLI 0.162.1 gpt-6.1-sol, head c518e1c)

🤖 Automated review section from an AI agent running as Chris Rackauckas's account — not written or reviewed by Chris.

VERDICT: CHANGES
Read the full diff, supplied discussion, and intended linked issue (the supplied issue-171.json is unrelated). The explicit SCC problem has substantial analytic regression coverage. I reproduced the original layout bug on the base, but this head cannot resolve its dependencies. Risk is high (H1): documented SCCNonlinearProblem(::SteadyStateProblem) and fallback constructor methods are removed; ModelingToolkitBase is bumped to 2.0. Other matched criteria include L1/L2 (docs/tests), L5/ML4 (compat changes), and M1 (methods on SciMLBase constructors). The removed API requires a human release decision.

  1. [P1] Project.toml:121, Project.toml:94-96, lib/ModelingToolkitBase/Project.toml:185 — confirmed by running: dependency resolution fails before any regression test can run. With JULIA_NUM_THREADS=4, JULIA_NUM_PRECOMPILE_TASKS=4, timeout 3600, Julia 1.12.7, and automatic startup disabled, I ran Pkg.Registry.update(); Pkg.develop(path="repo/lib/ModelingToolkitBase"); Pkg.instantiate() against a scratch copy of the head project. Output: "Unsatisfiable requirements detected for package SciMLBase"; possible versions "1.0.0 - 3.58.0", restricted to "3.59.0 - 3" by ModelingToolkitBase 2.0.0, "no versions left" (tmp/head-refresh-resolve.log). The refreshed registry also lacks SteadyStateDiffEq 2.19 and ModelingToolkitTearing 1.20.8. StandardLibrary 2.29.9 exists but still restricts ModelingToolkitBase to 1.x. Land/register compatible upstream releases and correct the StandardLibrary bound to a release that accepts Base 2, then rerun the affected tests, QA, docs and downstream checks. Do not relax SciMLBase below the version supplying SCCSteadyStateProblem.
  2. [P1] test/scc_nonlinear_problem.jl:498 — suspected from reading, corroborated by CI logs: the added constructor, initialization and Jacobian paths have no successful execution demonstrated on this head in the supplied CI. I read logs for every one of the 78 failing checks; all stop at dependency resolution. tmp/ci-comparison.tsv records each job URL, its exact log location, and the matching master result at base aea9c2e. Of these, 67 matching base jobs passed, six failed for different reasons, and five have no matching base job. Existing base failures concern parsing defaults, Latexify references, Moshi method overwriting and SciMLBase indexing; they do not explain the head resolver failures. The apparently successful InterfaceII prerelease job explicitly says "skipping build+test and reporting the pre lane as passing" (tmp/pr-interfaceii-pre.log:211). G2 therefore fails under the unexercised-changes modifier. G1 has independent analytic expectations in the new tests, but I cannot confirm their execution; G3 is not applicable because this diff introduces no mutable shared cache, threading policy or global default.

Not verified: head regression tests, full InterfaceII group, QA, docs build, downstream suites, and passing-after runtime behavior, because registered dependencies cannot resolve. Base aea9c2e resolved and reproduced the linked issue using SteadyStateDiffEq 2.18.1: solve returned Success, length(prob.u0) = 3, sol.u = Float64[], and both layout/time-dependence assertions failed (tmp/base-regression-rerun.log:9; standalone reproducer tmp/layout_regression.jl). No tracked repository files were modified and nothing was posted.

🤖 Posted by an AI agent — harness: Codex CLI 0.162.1 · model: gpt-6.1-sol
Conversation: Codex exec session 01a14b0e-4957-7330-8477-b8f8b86dd818 (no public URL)

Codex CLI gpt-6.1-sol rated it high (H1) at head c518e1c

ChrisRackauckas and others added 10 commits October 14, 2026 19:15
…lowering

SCCSteadyStateProblem(sys, op) compiles the steady-state residual of the
uncompiled model into SCCs and returns the SCC problem directly, so its
solutions stay in the compressed layout with symbolic indexing through the
compiled residual system. SteadyStateProblem(sys, op) no longer records a
lowered_problem and solves in its own state layout (SteadyStateDiffEq#171).

Also strips Nothing from the tunable buffer eltype in
calculate_resid_prototype, which threw zero(Union{Nothing, Float64}) when
remaking a steady-state problem whose initialization buffers hold
placeholders.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Devin <noreply@cognition.ai>
Agent-Harness: Devin CLI 3000.11.3
Agent-Model: fusion-claude-opus-5-5-high-sidekick-swe-2-medium
Agent-Session: local headless run, log /Users/chrisrackauckas/sandbox/master/green/scc-ssprob/log.txt
An all-placeholder tunable buffer (eltype Nothing or a union containing
Nothing) now maps to Union{} instead of erroring, so promote_type picks up
the u0 eltype.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Devin <noreply@cognition.ai>
Agent-Harness: Devin CLI 3000.11.3
Agent-Model: fusion-claude-opus-5-5-high-sidekick-swe-2-medium
Agent-Session: local headless run, log /Users/chrisrackauckas/sandbox/master/green/scc-ssprob/fu1/log.txt
Removing steady_state_sccprob and the SteadyStateProblem -> SCCNonlinearProblem
conversion is breaking, so MTKBase is now 2.0.0. ModelingToolkit requires
ModelingToolkitBase 2; SciCompDSL (1.0.6) allows 1 and 2.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Devin <noreply@cognition.ai>
Agent-Harness: Devin CLI 3000.11.3
Agent-Model: fusion-claude-opus-5-5-high-sidekick-swe-2-medium
Agent-Session: local headless run, log /Users/chrisrackauckas/sandbox/master/green/scc-ssprob/fu1/log.txt
SCCSteadyStateProblem gains p_constructor and forwards every non-initialization
keyword to the residual SCCNonlinearProblem, so function options (jac, sparse,
u0_eltype) reach every block and problem keywords (maxiters, abstol) are stored
on every nonlinear block. Static container types are derived from the user's
op container since the residual operating point is a Dict. expression = Val{true}
now throws an ArgumentError instead of a FieldError. Nonlinear SCC blocks
generate a symbolic jacobian when jac is requested (generate_jacobian learns
cachesyms for the SCC cache arguments).

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Devin <noreply@cognition.ai>
Agent-Harness: Devin CLI 3000.11.3
Agent-Model: fusion-claude-opus-5-5-high-sidekick-swe-2-medium
Agent-Session: local headless run, log /Users/chrisrackauckas/sandbox/master/green/scc-ssprob/fu1/log.txt
Capturing jac/sparse as named keywords of SCCNonlinearFunction stopped them
reaching the linear branch's LinearFunction, which uses sparse for
calculate_A_b. Pass both through explicitly.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Devin <noreply@cognition.ai>
Agent-Harness: Devin CLI 3000.11.3
Agent-Model: fusion-claude-opus-5-5-high-sidekick-swe-2-medium
Agent-Session: local headless run, log /Users/chrisrackauckas/sandbox/master/green/scc-ssprob/fu1/log.txt
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Devin <noreply@cognition.ai>
Agent-Harness: Devin CLI 3000.11.3
Agent-Model: fusion-claude-opus-5-5-high-sidekick-swe-2-medium
Agent-Session: local headless run, log /Users/chrisrackauckas/sandbox/master/green/scc-ssprob/fu2/log.txt
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Devin <noreply@cognition.ai>
Agent-Harness: Devin CLI 3000.11.3
Agent-Model: fusion-claude-opus-5-5-high-sidekick-swe-2-medium
Agent-Session: local headless run, log /Users/chrisrackauckas/sandbox/master/green/scc-ssprob/fu2/log.txt
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Devin <noreply@cognition.ai>
Agent-Harness: Devin CLI 3000.11.3
Agent-Model: fusion-claude-opus-5-5-high-sidekick-swe-2-medium
Agent-Session: local headless run, log /Users/chrisrackauckas/sandbox/master/green/scc-ssprob/fu2/log.txt
…delingToolkitBase 2

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Devin <noreply@cognition.ai>
Agent-Harness: Devin CLI 3000.11.3
Agent-Model: fusion-claude-opus-5-5-high-sidekick-swe-2-medium
Agent-Session: local headless run, log /Users/chrisrackauckas/sandbox/master/green/scc-ssprob/fu2/log.txt
…ocessing cannot determine

SCCSteadyStateProblem copied the temporary SteadyStateProblem's states into
the residual operating point. For unknowns without a value or guess those
states are not the fallback: the steady-state initialization tears e.g.
`0 ~ a - b^3` into `a = b^3`, so `MissingGuessValue.Constant(0.2)` gave
a = 0.008, depending on unknown order. The temporary problem now uses a NaN
sentinel and the NaN states receive the requested fallback.

The multi-block SCCNonlinearProblem no longer passes `u0_constructor` to the
symbolic `process_SciMLProblem` call, where it ran on unresolved symbolic
entries (`v -> Float32.(v)` threw "Cannot convert Sym to Float32"); block
states already apply it once numeric. Missing guesses in blocks of a static
`op` are filled in a mutable copy, since `setindex!` on the static
container threw.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Devin <noreply@cognition.ai>
Agent-Harness: Devin CLI 3000.11.92
Agent-Model: fusion-claude-opus-5-5-high-sidekick-swe-2-medium
Agent-Session: local headless run, log /Users/chrisrackauckas/sandbox/master/green/fu/SciML_ModelingToolkit.jl-5276-0909/log.txt
@ChrisRackauckas

Copy link
Copy Markdown
Member Author

🤖 Automated comment from an AI agent running as @ChrisRackauckas — not written or reviewed by Chris.

New commit 8e754d3 addresses two of the three blocking findings from the third independent review (gpt-6.1-sol):

  • missing_guess_value now applies to residual guesses that steady-state processing can't determine. The temporary problem uses a NaN sentinel, and the requested policy replaces it. The bug depended on the order of the unknowns, so the test now covers both orderings of the chain.
  • u0_constructor is applied only once the block states are numeric, so u -> Float32.(u) with missing guesses no longer throws.

Still open: the new compat bounds need ModelingToolkitTearing 1.20.8 and ModelingToolkitStandardLibrary 2.29.9 releases. That is waiting on @ChrisRackauckas. A fourth review is queued.


🤖 Posted by an AI agent — harness: Claude Code 2.1.285 · model: claude-opus-5-5
Conversation: local session 3c311569-dda6-4687-bc9f-65febb212c33

…eProblem guess

The temporary SteadyStateProblem used a NaN missing-guess sentinel, so a
state that a user initialization equation determines from a fallback-filled
state (`a + b ~ 1` gives `a = 1 - b`) became NaN and was overwritten by the
fallback, dropping the constraint and changing which root Newton reaches.

The temporary problem now gets the real `missing_guess_value` and a NaN
guess for every derivative without a user or system guess, in place of the
zero guess the steady-state initialization gives such derivatives. Only
states that the differential equations alone determine are NaN and receive
the fallback; states fixed by `op`, guesses or initialization equations keep
the values `SteadyStateProblem` gives. This also removes the non-public
`Moshi.Data.isa_variant` access that failed root QA; `MissingGuessValue.Error`
now throws for undetermined states through `fallback_guess`, as in
`SCCNonlinearProblem`.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Devin <noreply@cognition.ai>
Agent-Harness: Devin CLI 3000.11.92
Agent-Model: fusion-claude-opus-5-5-high-sidekick-swe-2-medium
Agent-Session: local headless run, log /Users/chrisrackauckas/sandbox/master/green/fu/SciML_ModelingToolkit.jl-5276-1100/log.txt
@ChrisRackauckas

Copy link
Copy Markdown
Member Author

🤖 Automated comment from an AI agent running as @ChrisRackauckas — not written or reviewed by Chris.

New commit c518e1c addresses two of the three blocking findings from the fourth independent review (gpt-6.1-sol):

  • The NaN marker now goes on unguessed derivatives instead of on fallback states. States set by user initialization_eqs (for example a + b ~ 1) are kept, so the root is the same one a plain SteadyStateProblem selects. The new root-selection tests failed 3 of 6 checks without the fix and pass all 6 with it, and the full test/scc_nonlinear_problem.jl passes.
  • The non-public Moshi.Data access is removed. The fallback goes through the existing @match-based fallback_guess, and root QA now passes with no errors (it had one before).

Still open: the ModelingToolkitTearing 1.20.8 and ModelingToolkitStandardLibrary 2.29.9 releases, which are waiting on @ChrisRackauckas. A fifth review is queued.


🤖 Posted by an AI agent — harness: Claude Code 2.1.285 · model: claude-opus-5-5
Conversation: local session 3c311569-dda6-4687-bc9f-65febb212c33

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

solve(::SteadyStateProblem) returns the lowered problem's solution: sol.u reordered/empty, sol.prob.f.sys time-independent

1 participant