Skip to content

Fix discrete affects rewriting differential states (#5200) - #5245

Draft
ChrisRackauckas-Claude wants to merge 11 commits into
SciML:masterfrom
ChrisRackauckas-Claude:agent/5200-affect-algebraic-resolve
Draft

ChrisRackauckas-Claude wants to merge 11 commits into
SciML:masterfrom
ChrisRackauckas-Claude:agent/5200-affect-algebraic-resolve

Conversation

@ChrisRackauckas-Claude

@ChrisRackauckas-Claude ChrisRackauckas-Claude commented Oct 2, 2026 •

Copy link
Copy Markdown
Member

Please ignore until reviewed by @ChrisRackauckas.

Summary

Fixes #5200: discrete equational affects were tearing for parent differentials instead of re-solving algebraic variables. Pin a differential only when the parent algebraic/observed equations that contain it can be bipartite-matched to distinct free (non-user-assigned) algebraic unknowns. Never pin dummy-derivative variables from index reduction, nor other differentials that appear alongside them in differentiated algebraic constraints (those equations must stay matched to free unknowns — otherwise root InterfaceI pendulum complete throws UnsolvableCallbackError). Never pin a differential that appears in the user's affect outside Pre. Array differentials use AffectPinOrigin stand-ins so setsym never receives Pre of an indexed/array variable. OOP SVector support is split to #5246. Events.md is unchanged.

Behaviour change for Chris: in a DAE whose differentials appear in algebraic equations, a discrete-only affect now keeps those differentials fixed and re-solves the algebraics. On master the same affect could move the differential. Existing models can get different trajectories. This is what #5200 asks for, but it is a silent semantics change while Events.md rule 1 still says every unknown is modifiable unless x ~ Pre(x).

Fail before / pass after

Master callbacks only vs this PR:

Case Master This PR
#5200 scalar on~0 v 0.716→0.316 v stays 0.716; λ→−0.358
Array u[1:2] + alg + on~0 compiles; u[1]→~0 compiles; u unchanged; λ≈u[1]
Probe A: two algs share one free absorber, assign λ Success; after event x=2, λ=3, μ=−1 same (no pin; no UnsolvableCallbackError)
Residual 0~v+0.5*Pre(v) 0.5→−0.25 same
Inelastic collision v→0 same
Docs inverse y~Pre(y)+1 [1,1]→[0.5,2] same
Pendulum Symbolic affects are compiled in complete (root InterfaceI) passes passes (no longer pins x/xˍt)

CI fixes on this head (bb771781c1)

  1. Never pin dummy derivatives / index-reduced constraint variables — schedule.dummy_sub and algebraics that mention those stand-ins are banned from pinning so the affect matching stays square.
  2. Trial-pin fallback — if substituting Pre would leave an algebraic with no free unknown, skip the pin instead of throwing UnsolvableCallbackError.
  3. Symbolic sol indexing — Discrete event on a system with algebraic equations changes a differential state instead of the algebraic variable #5200 / collision / inverse regressions use sol[x], sol[λ], sol[μ] (unknown order is not stable across simplification paths).

Verification (stacked private depot; one Julia process at a time)

  • Top-level GROUP=InterfaceI +1.13.1: InterfaceI | 1588 Pass, 3 Broken (~33m) — EXIT 0.
  • Top-level GROUP=InterfaceI +1.12: InterfaceI | 1589 Pass, 3 Broken (~45m) — EXIT 0.
  • lib/ModelingToolkitBase GROUP=InterfaceI +1.13.1: InterfaceI | 1805 Pass, 5 Broken (~29m) — EXIT 0.
  • lib/ModelingToolkitBase GROUP=InterfaceI +1.12: InterfaceI | 1805 Pass, 5 Broken (~39m) — EXIT 0.

Risk assessment

Fixes #5200 (algebraic resolve). OOP: #5246.

Please ignore this draft until reviewed by @ChrisRackauckas.

Harness: Cursor Agent CLI 2026.10.01-14929f9, model auto; transcript <job>/log.txt.

Review findings addressed

Review of 80efebf (codex-cli gpt-6-astra, verdict CHANGES), fixed in the follow-up commit:

  1. All-or-nothing feasibility disabled pins in unrelated components — fixed. Pin selection now lives in affect_pin_substitutions. It builds a maximum matching of the affect equations (user, algebraic and observed) to the affect unknowns, using non-differential unknowns first and untouched differentials only where no other unknown can absorb an equation (augmenting paths over maximal_matching). Candidates left unmatched are pinned, so each independent part of the system is decided on its own. The old global can_pin_candidates/pin_ok checks are gone; the matching guarantees every equation keeps an unpinned unknown. The reviewer's composed model (#5200 plus D(z) ~ 1e-16, z*y ~ p, affect [on ~ 0, y ~ Pre(y)+1]) now keeps v at 0.7159745833 (head: → −1.2840254167) while z → 0.5, y → 2. Regression testset: "Issue#5200: differentials are pinned per independent part of the system".
  2. An indexed element update unpinned its siblings — fixed. Symbol sets are now SymbolFootprints: u[1] and u[2] with constant indices do not overlap, while a whole-array reference or a non-constant index still overlaps every element. This applies to the touched set, the dummy-derivative/index-reduced bans and the candidate filter. Array references that are not element-wise (whole array or symbolic index) in algebraic/observed equations are never pinned. Regression testset: "Issue#5200: assigning one array element keeps the other elements pinned" (head: u[2] 0.7159745833 → 0.3159745833; fixed: unchanged). It uses u[1] ~ 2.0 rather than the reviewer's u[1] ~ Pre(u[1]) + 1: under ModelingToolkitBase alone, an implicit affect containing Pre(u[1]) fails with Invalid symbol Pre((u(t))[2]) for setsym on master 2b821a5, on this PR's previous head and with this fix alike. That is a separate, pre-existing bug. Under ModelingToolkit the reviewer's exact Pre form now keeps u[2] (0.7159745833 → 0.7159745833, u[1] +1).
  3. Undocumented change of the public callback policy — docs/src/basics/Events.md now states the pinning as rule 3 and gives the #5200 example with the opt-out (λ ~ Pre(λ) holds the algebraic variable and lets v change; checked locally: v 0.716 → −1.284, λ stays 0.642). No version bump here (the repo bumps versions in separate release PRs); see Breaking change below.

Verification (Julia 1.13.0, JULIA_NUM_THREADS=4)

Fail before (previous head 80efebf) / pass after, for the #5200 testsets extracted from symbolic_events.jl and run under both packages:

Mode Previous head This commit
ModelingToolkitBase only 46 pass, 3 fail 49 pass
ModelingToolkit 46 pass, 3 fail 49 pass

The three failures before the fix were any(isequal(v), parameters(aff)) (evaluated with [p] only), sol[v] -1.2840254166877405 ≈ 0.7159745833122596, and sol[u[2]] jumping from 0.716 to 0.316.

Full suites on this commit:

  • Root GROUP=InterfaceI julia +release --project -e 'using Pkg; Pkg.test()': InterfaceI | 1640 Pass, 3 Broken, 1643 Total (26m35s), ModelingToolkit tests passed. The 3 Broken are pre-existing @test_brokens. This group also runs symbolic_events.jl under ModelingToolkit, including the index-reduced pendulum complete case.
  • ModelingToolkitBase-only run of the full lib/ModelingToolkitBase/test/symbolic_events.jl: Symbolic Event Test (ModelingToolkitBase only) | 458 Pass, 2 Broken, 460 Total (7m45s). The 2 Broken are the existing @test_brokens in the "Bump" testset.

Formatting: Runic --check is clean on the changed Julia files. typos is clean on all changed files.

Not run locally: GROUP=QA (the only import change adds construct_augmenting_path!, which BipartiteGraphs exports), the docs build (Events.md gains prose and a non-executed julia block), the full lib/ModelingToolkitBase GROUP=InterfaceI, GPU and downstream groups.

Breaking change

For an equational (symbolic) affect, a differential variable that the affect does not reference outside Pre is now held at its pre-event value whenever the algebraic equations can be re-solved through algebraic unknowns or discrete parameters instead. Before this PR, compiling the affect system could choose to move such a differential state instead (the #5200 behaviour). This applies to both discrete and continuous callbacks. Models that relied on the old choice get different post-event states. They can restore it by adding alg ~ Pre(alg) for the algebraic variable that should stay fixed. Events.md rule 1 ("all unknowns are treated as modifiable") was the documented contract and is now refined by rule 3. Under strict SemVer this is a default-behaviour change of public API and needs a major bump (ModelingToolkitBase 1.x → 2.0, and correspondingly ModelingToolkit), unless the maintainer classifies the old behaviour as the bug that #5200 describes and ships it as a fix. This PR does not bump any version.


🤖 Posted by an AI agent — harness: Claude Code 2.1.301 · model: claude-opus-5-5
Conversation: local session ID 56988e15-46f1-419c-a0f8-7f1e94a5aa67 (headless, no URL)

Addendum (after push): lib/ModelingToolkitBase GROUP=InterfaceI on e87832c gives InterfaceI | 1821 Pass, 1 Error, 5 Broken (29m48s), with Symbolic Event Test | 458 Pass, 2 Broken. The single error is Parsing Test (variable_parsing.jl:66, Variable x must have default of matching size). It comes from Symbolics v7.46.0 rejecting x = [1, 2] on a scalar variable. Master already updated that test in 06b51a13be (#5281), after this branch's merge base, so it is base-branch drift unrelated to this change. Merging master into the branch would pick up that fix.


🤖 Posted by an AI agent — harness: Claude Code 2.1.301 · model: claude-opus-5-5
Conversation: local session ID 56988e15-46f1-419c-a0f8-7f1e94a5aa67 (headless, no URL)

Risk assessment (independent review, Grok Build grok-4.7, reviewed head e87832c)


🤖 Posted by an AI agent — harness: Grok Build · model: grok-4.7 (independent reviewer; appended by Claude Code 2.1.301 · claude-opus-5-5)
Conversation: https://claude.ai/code/session_01VZjwf7Ux9YtTzwHGHMqnsq

ChrisRackauckas and others added 4 commits October 1, 2026 17:53
…updates.

Pin parent differential states that an affect does not assign so mtkcompile
re-solves algebraic/observed variables when discretes change, and apply
ExplicitAffect state/parameter updates out-of-place for immutable problems.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Cursor Agent <noreply@cursor.com>
Agent-Harness: Cursor Agent CLI 2026.10.01-14929f9
Agent-Model: auto
Agent-Session: local session, transcript at /home/crackauc/sandbox/goals/issue-backlog/orch-ibmtk/jobs/5200-cursor/log.txt on amdci2.julia.csail.mit.edu
…d OOP.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Cursor Agent <noreply@cursor.com>
Agent-Harness: Cursor Agent CLI 2026.10.01-14929f9
Agent-Model: auto
Agent-Session: local session, transcript at /home/crackauc/sandbox/goals/issue-backlog/orch-ibmtk/jobs/5200-cursor/log.txt on amdci2.julia.csail.mit.edu
…P ImplicitAffect.

Wrapping pinned differentials in Pre keeps them out of tearing unknowns with proper
parameter codegen, and ImplicitAffect now writes back through setsym_oop/setp_oop for
immutable SVector problems.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Cursor Agent <noreply@cursor.com>
Agent-Harness: Cursor Agent CLI 2026.10.01-14929f9
Agent-Model: auto
Agent-Session: local session, transcript at /home/crackauc/sandbox/goals/issue-backlog/orch-ibmtk/jobs/5200-cursor/log.txt on amdci2.julia.csail.mit.edu
Only pin differentials that appear solely in appended parent equations when
an algebraic unknown remains free; use AffectPinOrigin stand-ins for array
pins so setsym never receives Pre of indexed variables. Residual, collision,
and documented-inverse cases match master; OOP moves to a separate PR.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Cursor Agent <noreply@cursor.com>
Agent-Harness: Cursor Agent CLI 2026.10.01-14929f9
Agent-Model: auto
Agent-Session: local session, transcript at /home/crackauc/sandbox/goals/issue-backlog/orch-ibmtk/jobs/5200-cursor/log.txt on amdci2.julia.csail.mit.edu
ChrisRackauckas and others added 4 commits October 1, 2026 23:08
Observed-style algebraic definitions (e.g. O ~ f(x,y)) must not alone
authorize pinning when another algebraic constraint would become parameter-only.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Cursor Agent <noreply@cursor.com>
Agent-Harness: Cursor Agent CLI 2026.10.01-14929f9
Agent-Model: auto
Agent-Session: local session, transcript at /home/crackauc/sandbox/goals/issue-backlog/orch-ibmtk/jobs/5200-cursor/log.txt on amdci2.julia.csail.mit.edu
…gebraics.

Shared free algebraics across multiple parent alg eqs must not authorize a pin
(probe A). Also restrict Pre capture to pins, error on Pre(indexed) in setsym,
and merge duplicate helpers.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Cursor Agent <noreply@cursor.com>
Agent-Harness: Cursor Agent CLI 2026.10.01-14929f9
Agent-Model: auto
Agent-Session: local session, transcript at /home/crackauc/sandbox/goals/issue-backlog/orch-ibmtk/jobs/5200-cursor/log.txt on amdci2.julia.csail.mit.edu
Root InterfaceI compiles the pendulum with dummy derivatives; pinning
those (or other differentials in the differentiated constraints) left
the affect system non-square. Also index SciML#5200 regression sols by symbol
so unknown order does not break CI.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Devin <noreply@cognition.ai>
Agent-Harness: Devin CLI 3000.11.3 (conflict resolution), Claude Code 2.1.285 (orchestration)
Agent-Model: fusion-claude-opus-5-5-high-sidekick-swe-2-medium
Agent-Session: local Claude session 3c311569-dda6-4687-bc9f-65febb212c33 (fleet master); Devin log /Users/chrisrackauckas/sandbox/master/fusion-fixes/SciML_ModelingToolkit.jl-5245/log.txt

# Conflicts:
#	lib/ModelingToolkitBase/test/symbolic_events.jl
@ChrisRackauckas

Copy link
Copy Markdown
Member

🤖 Automated comment from an AI agent running as @ChrisRackauckas — not written or reviewed by Chris.

Merged the default branch into this PR to resolve conflicts (head 138fb1c). Re-review the resolution before merging.

Conflict resolution report

STATUS: RESOLVED

Conflicting files

  • lib/ModelingToolkitBase/test/symbolic_events.jl: both sides added new testsets at the end of the file, after the same final testset.
    • PR side: "Issue#5200: discrete affect re-solves algebraic vars, keeps differentials", "Equational affects: residuals and documented inverse keep differentials free", "Two algebraics sharing a differential stay free when only one is assigned", "Issue#5200 array differential with algebraic and discrete-only affect".
    • Master side: "Symbolic array index in affect (issue [bot-unreviewed] Indexing an array parameter with a discrete Int in a symbolic affect fails in mtkcompile (StableIndex typeassert) #5078)", the two "Issue#5077" testsets, and "User callback with discrete events".
    • Resolution: kept both. The PR's testsets come first, then master's, and each block keeps its own closing end. Nothing was edited inside either block. I formatted the file with Runic (julia +1.12 --project=@runic -m Runic -i), which changed nothing.

All other files (including lib/ModelingToolkitBase/src/systems/callbacks.jl, which both sides touched) merged automatically.

Tests (Julia 1.12.7, run one at a time)

Setup: julia --project -e 'using Pkg; Pkg.instantiate()' and julia --project=lib/ModelingToolkitBase -e 'using Pkg; Pkg.instantiate()'.

  1. ModelingToolkitBase on its own:
    JULIA_LOAD_PATH="@:@stdlib:$HOME/.julia/environments/testenv" julia --project=lib/ModelingToolkitBase -e 'using TestEnv; TestEnv.activate(); using Test; @testset "symbolic_events" begin include("lib/ModelingToolkitBase/test/symbolic_events.jl") end'
    Result: symbolic_events | 445 Pass, 2 Broken, 447 Total. 0 fail, 0 error.
  2. Through top-level ModelingToolkit, the way test/group_interfacei.jl runs it: root project, TestEnv, using ModelingToolkit; import ModelingToolkitBase; include(joinpath(pkgdir(ModelingToolkitBase), "test", "symbolic_events.jl")) inside @testset. pkgdir(ModelingToolkitBase) points at the local lib/ModelingToolkitBase.
    Result: symbolic_events | 464 Pass, 2 Broken, 466 Total. 0 fail, 0 error. The 19 extra tests are the @isdefined(ModelingToolkit) branch of master's [bot-unreviewed] Indexing an array parameter with a discrete Int in a symbolic affect fails in mtkcompile (StableIndex typeassert) #5078 testset.

The 2 Broken results were already marked @test_broken in the "Bump" testset before this merge. They are not new.

Logs: scratch/mtkbase_symbolic_events.log, scratch/mtk_symbolic_events.log.

Not run: the full GROUP=InterfaceI suites and QA. Only the test file containing the PR's new tests was run.


🤖 Posted by an AI agent — harness: Devin CLI 3000.11.3 (merge), Claude Code 2.1.285 (posting) · model: fusion-claude-opus-5-5-high-sidekick-swe-2-medium
Conversation: local Claude session 3c311569-dda6-4687-bc9f-65febb212c33 (fleet master)

ChrisRackauckas and others added 3 commits October 16, 2026 05:00
Address review findings on SciML#5245.

- Choose pinned differentials with a maximum matching of the affect
  equations (user, algebraic, observed) to the affect unknowns that uses
  untouched differentials only where no other unknown can absorb an
  equation. Feasibility is no longer all-or-nothing across the parent
  system, so an independent component that needs a differential to move
  no longer stops pins elsewhere.
- Track referenced symbols with SymbolFootprint so that an indexed
  element such as u[1] does not mark its siblings u[2] as touched or
  assigned; whole-array and non-constant-index references still overlap
  every element.
- Document the pinning rule and the `λ ~ Pre(λ)` opt-out in Events.md.
- Regression tests for a composed model and a one-element array update.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Harness: Claude Code 2.1.301
Agent-Model: claude-opus-5-5
Agent-Session: local session ID 56988e15-46f1-419c-a0f8-7f1e94a5aa67 (headless claude -p, no URL)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Discrete event on a system with algebraic equations changes a differential state instead of the algebraic variable

2 participants