Skip to content

Remove stale root test copies and fix the ModelingToolkit test group - #4637

Merged
ChrisRackauckas merged 2 commits into
SciML:masterfrom
ChrisRackauckas-Claude:test/root-layout-stale-copies
Oct 3, 2026
Merged

ChrisRackauckas merged 2 commits into
SciML:masterfrom
ChrisRackauckas-Claude:test/root-layout-stale-copies

Conversation

@ChrisRackauckas-Claude

@ChrisRackauckas-Claude ChrisRackauckas-Claude commented Sep 26, 2026 •

Copy link
Copy Markdown
Member

Summary

GROUP=ModelingToolkit is declared in test/test_groups.toml and CI schedules three lanes for it, but test/runtests.jl has no ModelingToolkit body: SciMLTesting's run_tests treats the unknown group as a fall-through to the root core = () -> nothing no-op (confirmed in SciMLTesting 2.13 source, master CI job logs where the test body finishes in ~1.3s after ~14 minutes of install/precompile, and a local SciMLTesting probe). This PR removes that empty [ModelingToolkit] entry so CI stops burning those lanes, and drops the stale Dependabot directory for the deleted test/modelingtoolkit Project.toml.

It also deletes orphaned root copies under test/modelingtoolkit/, test/multirate/, and test/multithreading/ that are never included from the root runner. Same-named tests already run in OrdinaryDiffEqNonlinearSolve, OrdinaryDiffEqDifferentiation, OrdinaryDiffEqMultirate, and OrdinaryDiffEqExtrapolation. The root copies never ran in any root lane, so their diffs vs the lib copies are not live coverage loss. Precise differences:

  • jacobian_tests: root used Rodas5; lib uses Rodas5P.
  • nlstep_tests: root had two ≈ atol=1e-3 checks where lib uses 5e-3, and root ran TRBDF2 with default smooth_est where lib deliberately uses smooth_est=false (Align NLStep TRBDF2 parity test estimator #4077). Lib is otherwise stricter: two @test_broken → @test, and the FBDF order tolerance goes from 0.3/0.35 to 0.05.
  • mreef_tests: root compared ratios to 2^order within 15%; lib uses test_convergence atol=0.3 and adds exact nf/nf2 counts and a dense-output testset.
  • preconditioners / dae_initialize_integration / ode_extrapolation_tests: differ only in imports and (for preconditioners) manual sparsity instead of MTK.

Verification

  • SciMLTesting unknown-group behavior (local probe, SciMLTesting 2.13):
    GROUP=ModelingToolkit … detect_sublibrary_group => ("ModelingToolkit", "Core")
    isdir(lib/ModelingToolkit) => false
    root would route to sublib Pkg.test? false
    core body invoked
    fell through to core? true
    PASS: GROUP=ModelingToolkit is unknown -> empty core no-op path
    
  • Master CI (gh run view 36096837419 --job 107957121978): after GROUP=ModelingToolkit and full precompile (~822s), the test phase is 1.257923 seconds then OrdinaryDiffEq tests passed with no safetestset/include of any modelingtoolkit file.
  • Diffs + git log --follow for each deleted root file vs its sublibrary counterpart (precise diffs above).
  • typos clean on touched files. No .jl content edited (only deletions / Dependabot YAML), so Runic not applicable.

Not verified

  • Full local GROUP=ModelingToolkit julia --project -e 'using Pkg; Pkg.test()' was not re-run end-to-end (would only reconfirm the empty core after a long resolve/precompile); dispatch path and master CI log were used instead.
  • Did not re-run the sublibrary ModelingToolkit / Multithreading / Multirate suites; they already run those files on master.

Please ignore this draft until reviewed by @ChrisRackauckas.

Risk assessment

  • Risk: Low. Deletes unused root files and removes a CI matrix entry that runs no tests.
  • Blast radius: Root CI only — drops three ModelingToolkit jobs; sublibrary test coverage unchanged. Also removes a Dependabot directory that no longer exists.
  • Evidence: SciMLTesting fall-through docs/source; local dispatch probe; master CI log (~1.3s empty body); git history/diffs showing root copies never ran and differ from live sublib tests as listed above.
  • Independent review: Claude Code (Opus 5.5 subagent) rated it low, high confidence: none of the deleted files ran in any root lane; each has a live sublibrary copy run by its CI (NonlinearSolve/Differentiation ModelingToolkit groups, Multirate Core, Extrapolation Multithreading); root GROUP=ModelingToolkit fell through to the no-op core; no SciML repo calls GROUP=ModelingToolkit on OrdinaryDiffEq. VERDICT MERGE.
  • Merge: needs review

Harness: Cursor Agent CLI 2026.09.26-dd393fe; model: auto; transcript: /home/crackauc/sandbox/goals/qa-hygiene/jobs/ode-layout/log.txt on amdci2

ChrisRackauckas and others added 2 commits September 26, 2026 04:01
GROUP=ModelingToolkit falls through SciMLTesting run_tests to the empty
root core body, so the three CI lanes only pay install/precompile cost.
The orphaned root modelingtoolkit/multirate/multithreading copies are
strictly stale versus the live sublibrary tests that already run them.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Cursor Agent <noreply@cursor.com>
Agent-Harness: Cursor Agent CLI 2026.09.26-dd393fe
Agent-Model: auto
Agent-Session: local session, transcript at /home/crackauc/sandbox/goals/qa-hygiene/jobs/ode-layout/log.txt on amdci2
The root ModelingToolkit test Project.toml is gone with the orphaned
copies, so Dependabot should not keep scanning that directory.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Cursor Agent <noreply@cursor.com>
Agent-Harness: Cursor Agent CLI 2026.09.26-dd393fe
Agent-Model: auto
Agent-Session: local session, transcript at /home/crackauc/sandbox/goals/qa-hygiene/jobs/ode-layout/log.txt on amdci2
@ChrisRackauckas
ChrisRackauckas marked this pull request as ready for review October 3, 2026 11:51
@ChrisRackauckas
ChrisRackauckas merged commit bd6cfb3 into SciML:master Oct 3, 2026
124 of 128 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants