Skip to content

Canonical CI: grouped-tests.yml + root test/test_groups.toml#98

Merged
ChrisRackauckas merged 4 commits into
SciML:masterfrom
ChrisRackauckas-Claude:grouped-tests-ci
Jun 10, 2026
Merged

Canonical CI: grouped-tests.yml + root test/test_groups.toml#98
ChrisRackauckas merged 4 commits into
SciML:masterfrom
ChrisRackauckas-Claude:grouped-tests-ci

Conversation

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Contributor

Summary

Converts the root Tests.yml from a hand-maintained version × os matrix (calling tests.yml@v1) into a thin caller of the canonical grouped-tests.yml@v1, with the test matrix declared once in test/test_groups.toml.

Changes

  • .github/workflows/Tests.yml — the matrix test job is replaced by the canonical thin caller:
    jobs:
      tests:
        uses: "SciML/.github/.github/workflows/grouped-tests.yml@v1"
        secrets: "inherit"
    name:, on:, and concurrency: are preserved verbatim. No with: block is needed — every input matches the canonical default and the prior behavior (group env GROUP, check-bounds: yes, coverage on, coverage-directories: src,ext, no apt packages). Other workflows (Downgrade/FormatCheck/RunicSuggestions/SpellCheck/TagBot/DependabotAutoMerge) are untouched.
  • test/test_groups.toml (new) — the matrix:
    [Core]
    versions = ["lts", "1", "pre"]
    os = ["ubuntu-latest", "windows-latest"]
    
    [QA]
    versions = ["lts", "1"]
  • QA isolation (Category B) — this repo previously ran its QA-style checks (ExplicitImports, type-stability, AllocCheck allocation tests) inline under GROUP=="all"/"QA", with AllocCheck/ExplicitImports in the root test target. They are now refactored into a gated GROUP=="QA" path with an isolated environment:
    • test/qa/Project.toml[deps] = AllocCheck + ExplicitImports + Test + LinearAlgebra + DASSL via [sources] path = "../.."; [compat] julia = "1.10".
    • test/qa/qa.jl — the moved QA testset (ExplicitImports, type-stability, alloc tests).
    • test/runtests.jlCore/All run the functional suite; QA does Pkg.activate(test/qa) + Pkg.develop(path=repo root) + Pkg.instantiate() then includes qa.jl. Group dispatch updated to the canonical capitalized names (All/Core/QA).
  • Project.toml — benign metadata fixes: dropped AllocCheck/ExplicitImports from the root [extras]/[targets] (now QA-isolated); added missing [compat] for the remaining extras (DAEProblemLibrary = "0.1", ModelingToolkit = "10, 11", Test = "1"). julia compat already at the 1.10 LTS floor.

Matrix match

Verified statically with scripts/compute_affected_sublibraries.jl --root-matrix (no tests/Aqua run locally — structural conversion only). Emitted set:

  • Core: {lts, 1, pre} × {ubuntu-latest, windows-latest} = 6 cells → exactly reproduces the OLD functional version × os coverage.
  • QA: {lts, 1} × {ubuntu-latest} = 2 cells → the QA checks (previously riding along on all 6 cells via the default GROUP=all) now run isolated on the canonical QA subset.

TOML + YAML parse and runtests.jl/qa.jl Julia-parse were verified statically.

Note

The QA group is newly wired as a separate CI job; its Aqua/JET (here: ExplicitImports + type-stability + AllocCheck) run in CI. Any failures will be triaged in a follow-up — no checks were skipped or silenced.

Ignore until reviewed by @ChrisRackauckas.

🤖 Generated with Claude Code

ChrisRackauckas and others added 4 commits June 9, 2026 17:58
Convert the root Tests.yml from a hand-maintained version x os matrix
calling tests.yml@v1 into a thin caller of the canonical
grouped-tests.yml@v1, with the matrix declared once in
test/test_groups.toml.

- Tests.yml: replace the matrix test job with the thin
  grouped-tests.yml@v1 caller (secrets: inherit); on:/concurrency:
  preserved verbatim. No `with:` needed -- all defaults match the prior
  setup (GROUP env name, check-bounds=yes, coverage on, src,ext).
- test/test_groups.toml: [Core] on [lts,1,pre] x [ubuntu-latest,
  windows-latest] (preserves the old functional os coverage); [QA] on
  [lts,1].
- Isolate the QA checks (ExplicitImports, type-stability, AllocCheck
  allocation tests) into a gated GROUP=="QA" path with its own
  test/qa/Project.toml ([sources] DASSL path=../.., julia="1.10") and
  test/qa/qa.jl; runtests.jl activates/develops/instantiates it for QA.
- Project.toml: drop AllocCheck/ExplicitImports from the root test
  target (now QA-isolated); add missing [compat] entries for the
  remaining [extras] (DAEProblemLibrary, ModelingToolkit, Test). julia
  compat already at the 1.10 LTS floor.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The grouped runtests.jl uses Pkg for the QA group's Pkg.activate, but
the Core group runs with project='.' (the root test env). Pkg was not a
declared test dependency, so the Core job died with
ArgumentError: Package Pkg not found in current path. Declare Pkg in
[extras] and add it to [targets].test.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The grouped-tests conversion removed functional test dependencies from the
root test environment while isolating QA, breaking the Core group. Re-add
the deps that the pre-conversion base test target listed (using their base
UUIDs) so Core resolves and @testset/@safetestset/domain macros are defined.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…Error

The converted runtests.jl placed `using` statements inside a top-level
`if GROUP ...` block that also used a macro (e.g. @testset) inline. Julia
macro-expands the entire if block as one unit before the in-block `using`
executes, so the macro was undefined (UndefVarError: @testset not defined
in Main). Move the functional usings to top level (the QA-subenv usings
that follow Pkg.activate stay in the QA block).

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ChrisRackauckas ChrisRackauckas marked this pull request as ready for review June 10, 2026 09:43
@ChrisRackauckas ChrisRackauckas merged commit 4182b36 into SciML:master Jun 10, 2026
2 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants