Trace Base.Enum values as an enum wrapper around the traced base integer - #3232
ChrisRackauckas-Claude wants to merge 9 commits into
Conversation
96f9c6e to
0d73adf
Compare
569f7fa to
7003ec9
Compare
|
CI triage for the second push ( 🤖 Posted by an AI agent (Claude Code, model claude-fable-5) on behalf of Chris Rackauckas. Session: https://claude.ai/code/session_016LsC6pp9z6s5EABX9DnVjE |
Run every algorithm's ordinary `solve` dispatch under `Reactant.@compile`. The shared solver loop is a `ReactantCore.@trace while` over `CommonSolve.step!`, return codes are traced enums (EnzymeAD/Reactant.jl#3232, SciMLBase#1563) so the solution is built by `SciMLBase.NonlinearSolution` on every path, and the loop-carried cache state is refreshed by one reflective `dealias_traced!` instead of per-cache field lists. Trust-region, Levenberg-Marquardt, geodesic and dogleg updates are written once with `ifelse`; polyalgorithms are traced as a chain of their members. Forward-mode `AutoEnzyme` is preferred during compilation. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude <noreply@anthropic.com> Agent-Harness: Claude Code 2.1.251 Agent-Model: claude-fable-5 Agent-Session: https://claude.ai/code/session_016LsC6pp9z6s5EABX9DnVjE
Run every algorithm's ordinary `solve` dispatch under `Reactant.@compile`. The shared solver loop is a `ReactantCore.@trace while` over `CommonSolve.step!`, return codes are traced enums (EnzymeAD/Reactant.jl#3232, SciMLBase#1563) so the solution is built by `SciMLBase.NonlinearSolution` on every path, and the loop-carried cache state is refreshed by one reflective `dealias_traced!` instead of per-cache field lists. Trust-region, Levenberg-Marquardt, geodesic and dogleg updates are written once with `ifelse`; polyalgorithms are traced as a chain of their members. Forward-mode `AutoEnzyme` is preferred during compilation. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude <noreply@anthropic.com> Agent-Harness: Claude Code 2.1.251 Agent-Model: claude-fable-5 Agent-Session: https://claude.ai/code/session_016LsC6pp9z6s5EABX9DnVjE
Run every algorithm's ordinary `solve` dispatch under `Reactant.@compile`. The shared solver loop is a `ReactantCore.@trace while` over `CommonSolve.step!`, return codes are traced enums (EnzymeAD/Reactant.jl#3232, SciMLBase#1563) so the solution is built by `SciMLBase.NonlinearSolution` on every path, and the loop-carried cache state is refreshed by one reflective `dealias_traced!` instead of per-cache field lists. Trust-region, Levenberg-Marquardt, geodesic and dogleg updates are written once with `ifelse`; polyalgorithms are traced as a chain of their members. Forward-mode `AutoEnzyme` is preferred during compilation. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude <noreply@anthropic.com> Agent-Harness: Claude Code 2.1.251 Agent-Model: claude-fable-5 Agent-Session: https://claude.ai/code/session_016LsC6pp9z6s5EABX9DnVjE
Run every algorithm's ordinary `solve` dispatch under `Reactant.@compile`. The shared solver loop is a `ReactantCore.@trace while` over `CommonSolve.step!`, return codes are traced enums (EnzymeAD/Reactant.jl#3232, SciMLBase#1563) so the solution is built by `SciMLBase.NonlinearSolution` on every path, and the loop-carried cache state is refreshed by one reflective `dealias_traced!` instead of per-cache field lists. Trust-region, Levenberg-Marquardt, geodesic and dogleg updates are written once with `ifelse`; polyalgorithms are traced as a chain of their members. Forward-mode `AutoEnzyme` is preferred during compilation. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude <noreply@anthropic.com> Agent-Harness: Claude Code 2.1.251 Agent-Model: claude-fable-5 Agent-Session: https://claude.ai/code/session_016LsC6pp9z6s5EABX9DnVjE
Run every algorithm's ordinary `solve` dispatch under `Reactant.@compile`. The shared solver loop is a `ReactantCore.@trace while` over `CommonSolve.step!`, return codes are traced enums (EnzymeAD/Reactant.jl#3232, SciMLBase#1563) so the solution is built by `SciMLBase.NonlinearSolution` on every path, and the loop-carried cache state is refreshed by one reflective `dealias_traced!` instead of per-cache field lists. Trust-region, Levenberg-Marquardt, geodesic and dogleg updates are written once with `ifelse`; polyalgorithms are traced as a chain of their members. Forward-mode `AutoEnzyme` is preferred during compilation. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude <noreply@anthropic.com> Agent-Harness: Claude Code 2.1.251 Agent-Model: claude-fable-5 Agent-Session: https://claude.ai/code/session_016LsC6pp9z6s5EABX9DnVjE
Run every algorithm's ordinary `solve` dispatch under `Reactant.@compile`. The shared solver loop is a `ReactantCore.@trace while` over `CommonSolve.step!`, return codes are traced enums (EnzymeAD/Reactant.jl#3232, SciMLBase#1563) so the solution is built by `SciMLBase.NonlinearSolution` on every path, and the loop-carried cache state is refreshed by one reflective `dealias_traced!` instead of per-cache field lists. Trust-region, Levenberg-Marquardt, geodesic and dogleg updates are written once with `ifelse`; polyalgorithms are traced as a chain of their members. Forward-mode `AutoEnzyme` is preferred during compilation. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude <noreply@anthropic.com> Agent-Harness: Claude Code 2.1.251 Agent-Model: claude-fable-5 Agent-Session: https://claude.ai/code/session_016LsC6pp9z6s5EABX9DnVjE
Run every algorithm's ordinary `solve` dispatch under `Reactant.@compile`. The shared solver loop is a `ReactantCore.@trace while` over `CommonSolve.step!`, return codes are traced enums (EnzymeAD/Reactant.jl#3232, SciMLBase#1563) so the solution is built by `SciMLBase.NonlinearSolution` on every path, and the loop-carried cache state is refreshed by one reflective `dealias_traced!` instead of per-cache field lists. Trust-region, Levenberg-Marquardt, geodesic and dogleg updates are written once with `ifelse`; polyalgorithms are traced as a chain of their members. Forward-mode `AutoEnzyme` is preferred during compilation. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude <noreply@anthropic.com> Agent-Harness: Claude Code 2.1.251 Agent-Model: claude-fable-5 Agent-Session: https://claude.ai/code/session_016LsC6pp9z6s5EABX9DnVjE
Run every algorithm's ordinary `solve` dispatch under `Reactant.@compile`. The shared solver loop is a `ReactantCore.@trace while` over `CommonSolve.step!`, return codes are traced enums (EnzymeAD/Reactant.jl#3232, SciMLBase#1563) so the solution is built by `SciMLBase.NonlinearSolution` on every path, and the loop-carried cache state is refreshed by one reflective `dealias_traced!` instead of per-cache field lists. Trust-region, Levenberg-Marquardt, geodesic and dogleg updates are written once with `ifelse`; polyalgorithms are traced as a chain of their members. Forward-mode `AutoEnzyme` is preferred during compilation. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude <noreply@anthropic.com> Agent-Harness: Claude Code 2.1.251 Agent-Model: claude-fable-5 Agent-Session: https://claude.ai/code/session_016LsC6pp9z6s5EABX9DnVjE
7003ec9 to
2af7096
Compare
|
Reworked to the wrapper representation from #3231 (comment) and force-pushed as a single commit ( 🤖 Posted by an AI agent (Claude Code, model claude-fable-5) on behalf of Chris Rackauckas. Session: https://claude.ai/code/session_016LsC6pp9z6s5EABX9DnVjE |
Base.Enum values (@enum, EnumX's @enumx, ...) were silently dropped by @trace if/while instead of being traced. Represent a traced enum as TracedEnum{E}, a mutable wrapper holding the traced base integer, per the review direction in EnzymeAD#3231 — not as TracedRNumber{E} — so the payload is an ordinary traced number and the wrapper rides the generic struct tracing and result reconstruction. The wrapper carries the enum operations (comparisons, ifelse, integer conversion, conversion back to the plain enum once concrete), and traced_getfield tolerates a plain enum standing at a path that addresses the wrapper payload. Fixes EnzymeAD#3231. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude <noreply@anthropic.com> Agent-Harness: Claude Code 2.1.251 Agent-Model: claude-fable-5 Agent-Session: https://claude.ai/code/session_016LsC6pp9z6s5EABX9DnVjE
2af7096 to
3fb34c5
Compare
|
CI triage for the wrapper push ( 🤖 Posted by an AI agent (Claude Code, model claude-fable-5) on behalf of Chris Rackauckas. Session: https://claude.ai/code/session_016LsC6pp9z6s5EABX9DnVjE |
|
Codex is hunhappy
|
A wrapper with a concrete payload compares equal to other wrappers and to plain enum values, so it must hash like the enum it converts to; with the default identity hash two compiled results for the same enum ended up as distinct Set members. A traced payload keeps the payload's hash, like a bare TracedRNumber. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude <noreply@anthropic.com> Agent-Harness: Claude Code 2.1.251 Agent-Model: claude-fable-5 Agent-Session: https://claude.ai/code/session_016LsC6pp9z6s5EABX9DnVjE
`if_condition` writes branch results back into mutable arguments only when the target is already a Concrete/Traced value; any other location (a struct field holding a plain number or enum) was skipped silently, so the branch assignment was dropped without a message. Now a branch that leaves a different value at such a location errors and tells the user to trace the initial value first. Fields the branches did not touch keep working. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude <noreply@anthropic.com> Agent-Harness: Claude Code 2.1.251 Agent-Model: claude-fable-5 Agent-Session: https://claude.ai/code/session_016LsC6pp9z6s5EABX9DnVjE
|
Thanks — both addressed, in two commits so they can be reviewed separately: f6422be (hash) and d4594e2 ( P2 (hash): added P1 (one-armed
So the wrapper has reached parity with numbers; what is left is a Reactant-wide property of
The Verification (local, PJRT, Julia 1.12):
🤖 Posted by an AI agent (Claude Code, model claude-fable-5) on behalf of Chris Rackauckas. Session: https://claude.ai/code/session_016LsC6pp9z6s5EABX9DnVjE |
|
CI triage for The 8 red checks are the same set as on the previous head and are unrelated to this PR:
The 16 macOS and TPU lanes have been 🤖 Posted by an AI agent (Claude Code, model claude-fable-5) on behalf of Chris Rackauckas. Session: https://claude.ai/code/session_016LsC6pp9z6s5EABX9DnVjE |
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Codex <noreply@openai.com> Agent-Harness: Codex CLI 0.153.4 Agent-Model: gpt-6-astra Agent-Session: local session 01a070df-69eb-7e00-b6bc-a09c892048ab
Cover unpromoted local enum state and mutation of enum fields converted with to_rarray, and document the required field and loop initialization. Declare TracedEnum public and avoid the private Base.Enums.basetype helper. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Codex <noreply@openai.com> Agent-Harness: Codex CLI 0.153.4 Agent-Model: gpt-6-astra Agent-Session: local session 01a070df-69eb-7e00-b6bc-a09c892048ab
|
Updated in 71763c1 after merging current main. The new coverage exercises plain local enums without manual promotion and mutation of enum fields converted with Local verification:
The main Julia 1.12.7 core failure was independently reproduced and reduced to generic 🤖 Written by Codex CLI 0.153.4 (model: gpt-6-astra; local session: 01a070df-69eb-7e00-b6bc-a09c892048ab). |
Run every algorithm's ordinary `solve` dispatch under `Reactant.@compile`. The shared solver loop is a `ReactantCore.@trace while` over `CommonSolve.step!`, return codes are traced enums (EnzymeAD/Reactant.jl#3232, SciMLBase#1563) so the solution is built by `SciMLBase.NonlinearSolution` on every path, and the loop-carried cache state is refreshed by one reflective `dealias_traced!` instead of per-cache field lists. Trust-region, Levenberg-Marquardt, geodesic and dogleg updates are written once with `ifelse`; polyalgorithms are traced as a chain of their members. Forward-mode `AutoEnzyme` is preferred during compilation. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude <noreply@anthropic.com> Agent-Harness: Claude Code 2.1.251 Agent-Model: claude-fable-5 Agent-Session: https://claude.ai/code/session_016LsC6pp9z6s5EABX9DnVjE
|
What is left here @gbaraldi ? |
|
The changes the robot did to control flow need to be reviewed by someone else since I'm not familiar with them. @mofeing maybe? |
|
My robot agrees with me that the Ops.jl changes are odd and could be split from this PR either to another PR or to an issue |
|
• The enum support is needed; the control-flow diagnostic is independently useful and can be split out. Reviewing head
I reproduced the new field-assignment diagnostic locally. Full compiled execution was blocked by an older installed Full robot thing |
Per review on EnzymeAD#3232: the if_condition check for assignments to untraced mutable fields applies to ordinary numbers too, so it moves to its own PR. The enum-specific to_rarray_internal is dropped: the generic fallback already reaches the new make_tracer(ArrayToConcrete) method. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> Agent-Harness: Devin CLI Agent-Model: SWE-2 High Agent-Session: local CLI session (no shareable URL)
|
Addressed the review at #3232 (comment); head is now
Verification on 🤖 Generated with Devin (harness: Devin CLI, model: SWE-2 High) — local CLI session, no shareable URL. |
Adds tracing for
Base.Enumvalues usingTracedEnum{E}, a mutable wrapper around the enum's traced base integer, following #3231 (comment). Comparisons, selection, integer conversion, result reconstruction, and concrete hashing use that payload; the numeric type bounds and XLA element types stay intact. The wrapper is documented and declared public on Julia 1.11+, and its base integer type is extracted locally without relying on privateBase.Enums.basetype.Enum fields follow the same requirements as numeric fields: parameterize the field type and initialize it with
promote_to_traced, or convert the containing input withto_rarray(...; track_numbers=Number)(handled by the genericto_rarray_internalfallback viamake_tracer(..., ArrayToConcrete)). Fixedcode::Code.Tfields are still unsupported. Plain local enums work without manual promotion. The tutorial and tests cover both branch outcomes and mutation of converted input fields.The
if_conditiondiagnostic for assignments to untraced mutable fields was split out to #3271 since it applies to ordinary numbers independently of enum tracing.Verification
Latest head (
dbf10279):core/enumspasses 50/50 locally on Linux CPU under both PJRT and IFRT:Earlier verification on the pre-split head, using the juliaup launcher from the checkout's parent directory:
Final enum results (pre-split head):
The earlier IFRT enum/control-flow run passed 187/187 assertions. All 30 worker files in the full Julia 1.12.6 core run completed without failure output.
Standalone QA on Julia 1.12.7 completed with the final code:
The one broken check is the existing
Aqua.test_undocumented_names(...; broken=true); no tests were disabled or weakened. The full Julia 1.12.6 core run, including serial QA, completed:All three broken checks are pre-existing.
Several concurrent validation subprocesses were terminated by SIGKILL; the shared user cgroup records OOM kills. Those runs are not counted as passing. Final enum validation was restarted sequentially.
A standalone branch regression was run unchanged on clean main 4e164f7 and on this branch:
Before:
After:
JuliaFormatter 1 checks pass for all touched Julia files;
typosandgit diff --checkare clean. All new tutorial assertions passed independently. The initial full docs attempt lacked the eight CPU devices required by the existing sharding tutorial; rerunning with the CI setting completed the full build (exit 0,build complete in 37.52s.), and the generated HTML contains the enum tutorial. Vitepress still emits__DEPLOY_ABSPATH__ is not defined; the previous successful docs job emitted the same error 57 times: https://github.com/EnzymeAD/Reactant.jl/actions/runs/33869016459/job/101010349275. This pre-existing renderer issue is not fixed here.The earlier hash and field-diagnostic before/after evidence and NonlinearSolve 147/147 downstream run are recorded in #3232 (comment); those downstream results are from the previous author, not a new downstream run.
CI and remaining limits
The existing Julia 1.12.7 core failure was reproduced on clean main and reduced to a generic
invokedispatch bug. The same triangular-product test passes on Julia 1.12.6. The introducing LinearAlgebra backport, exact reproductions, version information, and error output are recorded in https://github.com/ChrisRackauckas/InternalJunk/issues/94. The main CI failure is visible at https://github.com/EnzymeAD/Reactant.jl/actions/runs/33772931791/job/100707462806. A smaller trigger for the underlying dispatch bug, verified on Julia 1.12.7, is:That independent compiler bug is not hidden by changing or disabling tests here.
The updated head has passed documentation, Julia style, all four Julia 1.10 Linux core lanes (ARM/x86 × PJRT/IFRT), and both dependency-downgrade core lanes. At the last check, 43 checks succeeded, three failed, and others remained running or queued. The two Windows Julia 1.12 failures match the baseline linker error below. The Linux Julia 1.12 IFRT core lane passed all 51 enum assertions and failed only the two known
core/ops.jlstack overflows at lines 1334 and 1348: https://github.com/EnzymeAD/Reactant.jl/actions/runs/33961131409/job/101293092393. CI is not fully green, and the PR remains a draft. Current checks: https://github.com/EnzymeAD/Reactant.jl/pull/3232/checks.Windows main also fails precompilation with
lld: error: undefined symbol: jl_cstr_to_string. A standalone C cross-link reproducer fails against both official Windows Julia 1.12.6 and 1.12.7 import libraries; this does not establish a 1.12.7 regression. The direct LLVM symbol reference originates in Reactant commit a5f875f. Full Windows execution was unavailable on this Linux host. Reproducer and evidence: https://github.com/ChrisRackauckas/InternalJunk/issues/96. Observed main failure: https://github.com/EnzymeAD/Reactant.jl/actions/runs/33772931791/job/100707462848.GPU/TPU, macOS, Windows, multi-node execution, and a fresh downstream NonlinearSolve run were not verified locally. Enum construction from traced integers does not validate enum membership at runtime, and the mutable wrapper deliberately has an untyped payload for result write-back.
Please ignore this draft until reviewed by @ChrisRackauckas.
Prior implementation conversation: https://claude.ai/code/session_016LsC6pp9z6s5EABX9DnVjE
🤖 Generated with Codex CLI 0.153.4 (model: gpt-6-astra; local session: 01a070df-69eb-7e00-b6bc-a09c892048ab); review split and re-verification by Devin (harness: Devin CLI, model: SWE-2 High; local session, no shareable URL).