Skip to content

Migrate to SciMLTesting v1.2 (off ReTestItems)#976

Merged
ChrisRackauckas merged 1 commit into
SciML:masterfrom
ChrisRackauckas-Claude:sciml-testing-rollout
Jun 15, 2026
Merged

Migrate to SciMLTesting v1.2 (off ReTestItems)#976
ChrisRackauckas merged 1 commit into
SciML:masterfrom
ChrisRackauckas-Claude:sciml-testing-rollout

Conversation

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Contributor

What

Finishes the migration of NonlinearSolve.jl's test harness off ReTestItems/TestItemRunner onto SciMLTesting v1.2.

The root and every sublibrary runtests.jl had already been moved off @testitem/ReTestItems.runtests/@run_package_tests onto a hand-written SafeTestsets group-dispatch ladder (a const GROUP = get(ENV, ...) + if GROUP == ... block + bespoke activate_*_env helpers). This PR replaces those ladders with SciMLTesting's run_tests, which owns the GROUP routing, per-group sub-environment activation, and the Julia <1.11 [sources] develop backport.

Why

ReTestItems parallelizes at the wrong level for SciML (which parallelizes per-GROUP at the CI-matrix level), breaks on new Julia releases, and causes upgrade churn. SciMLTesting centralizes the boilerplate so each runtests.jl is a single declarative run_tests(...) call.

Changes

  • test/runtests.jl and lib/*/test/runtests.jlusing SciMLTesting; run_tests(...) in explicit-args mode. The group bodies (file includes / @safetestset blocks) are preserved verbatim, so the same tests run under each GROUP — this is behavior-preserving.
    • Explicit-args (not folder-discovery) is used throughout because the Core folders mix aggregate manifests (core_tests.jl), per-item files (core_tests__itemN.jl), and shared setup_*.jl fixtures that folder-discovery would glob as standalone tests.
    • The root keeps its bespoke sublibrary dispatch (the transitive [sources] develop walk for Julia <1.11) and the Julia-1.12-gated Trim group, which don't fit folder-discovery.
  • Test deps: removed Pkg and the now-unused Hwloc (a ReTestItems/nworkers leftover) from each sublibrary test target; added SciMLTesting to every test target. The root keeps Pkg (used directly by the sublibrary/Trim dispatch block).
  • test_groups.toml files are unchanged — the group keys already match the group names (Core/NoPre/Verbosity/Downstream/Bounds/Adjoint/Wrappers/CUDA/QA/Trim), and there is no literal All key to rename.

Validation

  • Statically: every Project.toml/test_groups.toml parses; every runtests.jl parses with no error/incomplete nodes; all changed .jl files are Runic-clean.
  • Runtime (Julia 1.11): SciMLJacobianOperators Core and QA groups both report Testing SciMLJacobianOperators tests passed (Core: ~3600 passing tests; QA: Aqua 11/11 + ExplicitImports 3/3) — exercising the run_tests explicit-args core/qa paths, the sub-env activation, and resolving SciMLTesting from General. Per-PR CI validates the remaining groups across the matrix.

Ignore until reviewed by @ChrisRackauckas.

The root and every sublibrary runtests.jl had already been moved off
ReTestItems/TestItemRunner onto a hand-written SafeTestsets group-dispatch
ladder. This finishes the migration by replacing those ladders with
SciMLTesting's `run_tests` (explicit-args mode), which owns the GROUP routing,
per-group sub-environment activation, and the Julia <1.11 [sources] develop
backport.

- test/runtests.jl + lib/*/test/runtests.jl: `using SciMLTesting; run_tests(...)`
  in explicit-args mode (file/thunk group bodies preserved verbatim, so the same
  tests run under each GROUP). The root keeps its bespoke sublibrary dispatch
  (transitive [sources] develop walk) and the Julia-1.12-gated Trim group, which
  do not fit folder-discovery; explicit-args is used throughout because the Core
  folders mix aggregate manifests, per-item files, and shared `setup_*.jl`
  fixtures that folder-discovery would glob.
- Deps: removed `Pkg` and the unused `Hwloc` (a ReTestItems/nworkers leftover)
  from each sublibrary's test target; added `SciMLTesting` to every test target.
  The root keeps `Pkg` (used directly by the sublibrary/Trim dispatch).
- test_groups.toml unchanged: the group keys already match the new group names.

Validated on Julia 1.11: SciMLJacobianOperators Core and QA groups both report
`Testing SciMLJacobianOperators tests passed`. All Project.toml/test_groups.toml
parse; all runtests.jl parse; Runic-clean.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ChrisRackauckas ChrisRackauckas marked this pull request as ready for review June 15, 2026 02:14
@ChrisRackauckas ChrisRackauckas merged commit 1c22f69 into SciML:master Jun 15, 2026
8 of 15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants