Migrate to SciMLTesting v1.2 (off ReTestItems)#976
Merged
ChrisRackauckas merged 1 commit intoJun 15, 2026
Conversation
The root and every sublibrary runtests.jl had already been moved off ReTestItems/TestItemRunner onto a hand-written SafeTestsets group-dispatch ladder. This finishes the migration by replacing those ladders with SciMLTesting's `run_tests` (explicit-args mode), which owns the GROUP routing, per-group sub-environment activation, and the Julia <1.11 [sources] develop backport. - test/runtests.jl + lib/*/test/runtests.jl: `using SciMLTesting; run_tests(...)` in explicit-args mode (file/thunk group bodies preserved verbatim, so the same tests run under each GROUP). The root keeps its bespoke sublibrary dispatch (transitive [sources] develop walk) and the Julia-1.12-gated Trim group, which do not fit folder-discovery; explicit-args is used throughout because the Core folders mix aggregate manifests, per-item files, and shared `setup_*.jl` fixtures that folder-discovery would glob. - Deps: removed `Pkg` and the unused `Hwloc` (a ReTestItems/nworkers leftover) from each sublibrary's test target; added `SciMLTesting` to every test target. The root keeps `Pkg` (used directly by the sublibrary/Trim dispatch). - test_groups.toml unchanged: the group keys already match the new group names. Validated on Julia 1.11: SciMLJacobianOperators Core and QA groups both report `Testing SciMLJacobianOperators tests passed`. All Project.toml/test_groups.toml parse; all runtests.jl parse; Runic-clean. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Finishes the migration of NonlinearSolve.jl's test harness off ReTestItems/TestItemRunner onto SciMLTesting v1.2.
The root and every sublibrary
runtests.jlhad already been moved off@testitem/ReTestItems.runtests/@run_package_testsonto a hand-written SafeTestsets group-dispatch ladder (aconst GROUP = get(ENV, ...)+if GROUP == ...block + bespokeactivate_*_envhelpers). This PR replaces those ladders with SciMLTesting'srun_tests, which owns the GROUP routing, per-group sub-environment activation, and the Julia <1.11[sources]develop backport.Why
ReTestItems parallelizes at the wrong level for SciML (which parallelizes per-GROUP at the CI-matrix level), breaks on new Julia releases, and causes upgrade churn. SciMLTesting centralizes the boilerplate so each
runtests.jlis a single declarativerun_tests(...)call.Changes
test/runtests.jlandlib/*/test/runtests.jl→using SciMLTesting; run_tests(...)in explicit-args mode. The group bodies (fileincludes /@safetestsetblocks) are preserved verbatim, so the same tests run under each GROUP — this is behavior-preserving.core_tests.jl), per-item files (core_tests__itemN.jl), and sharedsetup_*.jlfixtures that folder-discovery would glob as standalone tests.[sources]develop walk for Julia <1.11) and the Julia-1.12-gatedTrimgroup, which don't fit folder-discovery.Pkgand the now-unusedHwloc(a ReTestItems/nworkersleftover) from each sublibrary test target; addedSciMLTestingto every test target. The root keepsPkg(used directly by the sublibrary/Trim dispatch block).test_groups.tomlfiles are unchanged — the group keys already match the group names (Core/NoPre/Verbosity/Downstream/Bounds/Adjoint/Wrappers/CUDA/QA/Trim), and there is no literalAllkey to rename.Validation
Project.toml/test_groups.tomlparses; everyruntests.jlparses with no error/incomplete nodes; all changed.jlfiles are Runic-clean.SciMLJacobianOperatorsCoreandQAgroups both reportTesting SciMLJacobianOperators tests passed(Core: ~3600 passing tests; QA: Aqua 11/11 + ExplicitImports 3/3) — exercising therun_testsexplicit-argscore/qapaths, the sub-env activation, and resolvingSciMLTestingfrom General. Per-PR CI validates the remaining groups across the matrix.Ignore until reviewed by @ChrisRackauckas.