Skip to content

Rewire Brusselator, Nonlinlap_ADV, and MOL_Interface2 into CI - #710

Merged
ChrisRackauckas merged 1 commit into
SciML:masterfrom
ChrisRackauckas-Claude:ci/rewire-orphan-groups
Oct 10, 2026
Merged

ChrisRackauckas merged 1 commit into
SciML:masterfrom
ChrisRackauckas-Claude:ci/rewire-orphan-groups

Conversation

@ChrisRackauckas-Claude

@ChrisRackauckas-Claude ChrisRackauckas-Claude commented Oct 10, 2026 •

Copy link
Copy Markdown
Member

Summary

Brusselator, Nonlinlap_ADV, and MOL_Interface2 are dispatched in test/runtests.jl but were never declared in test/test_groups.toml, so SciML grouped-tests CI never scheduled them. Brusselator and MOL_Interface2 were previously removed from the hard-coded Tests.yml matrix for 6h stalls (#509, #515, #521; now closed). Nonlinlap_ADV was never in that per-group matrix: git log -S Nonlinlap_ADV shows it was only reachable via GROUP=All / individual GROUP=, and was omitted when test_groups.toml became the matrix source (#573 / #576). This PR adds all three with versions = [\"lts\", \"1\", \"pre\"] matching neighbouring groups. Test contents are unchanged.

Verification

Env: JULIA_CONDAPKG_BACKEND=Null JULIA_PYTHONCALL_EXE=/usr/bin/python3, local depot under the job scratch dir. Commands (one Julia process at a time):

GROUP=<name> julia +1 --project -e 'using Pkg; Pkg.test()'
GROUP=<name> julia +lts --project -e 'using Pkg; Pkg.test()'

(LTS needed a fresh Pkg.resolve() after removing the 1.13 Manifest; not committed.)

Group julia +1 julia +lts
MOL_Interface2 Pass 9, Broken 4; wall 685s Pass 9, Broken 4; wall 784s
Nonlinlap_ADV Broken 1; wall 144s Broken 1; wall 139s
Brusselator Pass 1154; wall 524s Pass 1154; wall 579s

Log tails:

# MOL_Interface2 +1
Test Summary:  | Pass  Broken  Total     Time
MOL_Interface2 |    9       4     13  6m02.9s
END GROUP=MOL_Interface2 julia +1 2026-10-09T17:40:15-04:00 exit=0 wall=685s

# MOL_Interface2 +lts
Test Summary:  | Pass  Broken  Total     Time
MOL_Interface2 |    9       4     13  5m23.8s
END GROUP=MOL_Interface2 julia +lts 2026-10-09T18:35:49-04:00 exit=0 wall=784s

# Nonlinlap_ADV +1
Test Summary: | Broken  Total     Time
Nonlinlap_ADV |      1      1  2m10.3s
END GROUP=Nonlinlap_ADV julia +1 2026-10-09T19:20:24-04:00 exit=0 wall=144s

# Nonlinlap_ADV +lts
Test Summary: | Broken  Total     Time
Nonlinlap_ADV |      1      1  2m03.1s
END GROUP=Nonlinlap_ADV julia +lts 2026-10-09T18:48:48-04:00 exit=0 wall=139s

# Brusselator +1
Test Summary: | Pass  Total     Time
Brusselator   | 1154   1154  8m26.4s
END GROUP=Brusselator julia +1 2026-10-09T20:02:29-04:00 exit=0 wall=524s

# Brusselator +lts
Test Summary: | Pass  Total     Time
Brusselator   | 1154   1154  9m22.7s
END GROUP=Brusselator julia +lts 2026-10-09T19:11:52-04:00 exit=0 wall=579s

Group ↔ toml check after this change: every FUNCTIONAL_GROUPS entry is a toml key. Toml keys not in FUNCTIONAL_GROUPS remain intentional: Components 32-bit (alias), NeuralNets (own env), QA (qa path).

Not verified

  • julia +pre channel (declared like neighbours; not run locally)
  • GitHub Actions runner behaviour / queueing under load
  • GROUP=All / Downgrade workflows

Please ignore this draft until reviewed by @ChrisRackauckas.

Risk assessment

Harness: Cursor Agent CLI 2026.10.01-14929f9; model: auto; transcript: /home/crackauc/sandbox/goals/qa-hygiene/jobs/mol-groups/log.txt on amdci2

Made with Cursor

Independent review: Devin Fusion (fusion-claude-opus-5-5-high-sidekick-swe-2-medium) rated it low, verdict MERGE: #710 (comment)

These groups are dispatched in runtests.jl but were omitted from
test_groups.toml, so grouped-tests CI never scheduled them. Local runs
on julia +1 and +lts now pass within minutes, so declare them with the
same versions as neighbouring groups.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Cursor Agent <noreply@cursor.com>
Agent-Harness: Cursor Agent CLI 2026.10.01-14929f9
Agent-Model: auto
Agent-Session: local session, transcript at /home/crackauc/sandbox/goals/qa-hygiene/jobs/mol-groups/log.txt on amdci2
@ChrisRackauckas

Copy link
Copy Markdown
Member

🤖 Automated comment from an AI agent running as @ChrisRackauckas — not written or reviewed by Chris.

Independent review (Devin CLI 3000.11.3, model fusion-claude-opus-5-5-high-sidekick-swe-2-medium): MERGE, risk low.

Full review

VERDICT: MERGE
RISK: low

PR: #710 (no linked issue; references closed issues 509/515/521 and merged PRs 573/576).
The diff is 9 added lines in test/test_groups.toml. It declares [MOL_Interface2], [Nonlinlap_ADV] and [Brusselator] with versions = ["lts", "1", "pre"], the same as the neighbouring groups. No test bodies, library code or workflows change.

Blocking findings

None.

Non-blocking findings

  1. "never scheduled" overstates the gap (confirmed by running). In the PR body, "never scheduled" is accurate only for the grouped-tests matrix. The Downgrade workflow runs Pkg.test() with GROUP empty, which means All = FUNCTIONAL_GROUPS (test/runtests.jl:4-30, 154). So it already ran all three groups on lts with downgraded deps. In this PR's Downgrade log (run 38008434044) I saw Brusselator | 1154 1154, Nonlinlap_ADV | Broken 1 and an MOL_Interface2 summary. What the PR actually adds is coverage on julia 1 and lts with current dependencies. That is still worth having, but the PR text should say so.
  2. The Nonlinlap_ADV job checks very little (confirmed by running). test/Nonlinlap_ADV/nonlinear_laplacian_advanced.jl:35 wraps the whole test in @test_broken begin ... end, so the job reports "Broken 1" whenever the block throws anything at all. I removed the @test_broken in a scratch copy and ran it on julia 1.12.7. discretize throws a real MOL limitation: ArgumentError: Differential w.r.t. variable (y) other than the independent variable (t) are not allowed. (from ModelingToolkitBase.check_equations via PDEBase.generate_system). So the broken marker reflects a real bug and not a typo in the test. Even so, an unrelated regression that throws a different exception would also count as "Broken". The cost is about 18 runner-minutes per CI run for this signal. A possible follow-up, separate from this PR: narrow the check to @test_throws ArgumentError discretize(...). Also, from reading line 59: Dy(H(t, x, y)^2) was probably meant to be Dy(H(t, x, y))^2. That bug predates this PR and is out of scope.
  3. The pre lane does nothing at the moment (confirmed from CI log). The shared workflow skips build and test when Julia pre is not a release candidate, and reports the job as passing. I saw Julia 1.13.1 is not a release candidate ... skipping build+test in job 114082633643. This applies to every group, not just the new ones. The PR body discloses it as not verified, which is fair.
  4. Attribution format (from reading). The commit has Agent-Model: auto, which is not an exact model ID as Chris's global CLAUDE.md requires. The PR body has no banner on its first line and no 🤖 footer in the required format. There is no AGENTS.md or CLAUDE.md at the repo root. This is a process issue, not a code issue.
  5. Sibling PR 698 also edits test/test_groups.toml. It adds [SolutionAD] before [QA], a different hunk from this PR's additions, so I do not expect a textual conflict (from reading the two diffs). I found no other open PR touching the matrix.
  6. Test-count claims (confirmed). The PR body's numbers match this PR's own CI logs:
Group julia 1 (1.13.1) lts (1.10.12)
Brusselator Pass 1154 Pass 1154
MOL_Interface2 Pass 9, Broken 4 Pass 9, Broken 4
Nonlinlap_ADV Broken 1 Broken 1

All six jobs finished in 6–18 min of wall time, well under the 120-min job timeout. That timeout is what now protects against the old 6-hour stalls from issues 509/515.

What I ran

  • gh pr view 710, git diff origin/master...HEAD. I read test/runtests.jl, test/test_groups.toml, .github/workflows/Tests.yml and Downgrade.yml, and the shared SciML/.github grouped-tests.yml@v1. That file confirms the matrix comes only from test/test_groups.toml.
  • Checked that each new toml key matches an entry in the groups Dict and in FUNCTIONAL_GROUPS. All three do.
  • gh pr checks 710: every check passes, including the 9 new jobs.
  • Pulled the logs for the 6 new 1/lts jobs and grepped the Test Summary, Julia version and timeout. Results are in the table above.
  • Pulled the log for the Brusselator (julia pre) job and confirmed the pre lane is skipped.
  • Pulled the Downgrade run log and confirmed it already runs the three groups.
  • Checked issues 509, 515 and 521 and PRs 573 and 576 for state and content, and checked open PRs that touch test_groups.toml or runtests.jl.
  • Probe: in a scratch copy of the repo (the checkout is untouched), I replaced the @test_broken begin in the Nonlinlap_ADV file with @testset, then ran GROUP=Nonlinlap_ADV julia +1.12 --project -e 'using Pkg; Pkg.test()'. I observed the ArgumentError from discretize quoted in finding 2. The Julia process has exited.

What I did not verify

  • I did not rerun Brusselator or MOL_Interface2 locally. The PR's own GitHub Actions jobs, on the real runners, reproduce the counts the PR claims on julia 1 and lts. I judged that stronger evidence than a laptop run.
  • I did not observe a pre lane run, because the shared workflow currently skips it.
  • I cannot tell whether the old 6-hour stalls (self-hosted runner queueing) could come back under load. This PR's run finished normally on GitHub-hosted ubuntu-latest.

Links


🤖 Posted by an AI agent — harness: Devin CLI 3000.11.3 (review), Claude Code 2.1.285 (posting) · model: fusion-claude-opus-5-5-high-sidekick-swe-2-medium
Conversation: local Claude session 3c311569-dda6-4687-bc9f-65febb212c33 (fleet master)

@ChrisRackauckas
ChrisRackauckas marked this pull request as ready for review October 10, 2026 17:15
@ChrisRackauckas
ChrisRackauckas merged commit 5e2f755 into SciML:master Oct 10, 2026
88 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants