Repository navigation
Hash embedded objects by type and content, keeping ids session-stable - #157
Merged
Merged
Conversation
`expr_to_id` hashed every non-Expr leaf by `string(leaf)`, so distinct leaves that print alike (objects with a summarizing `show`, closures with different captures, `:x` vs "x", `Int32(1)` vs `1`) got the same id and the second RGF silently ran the first one's body (issue #155). Serialize leaves into an unambiguous tagged byte stream by type and content instead. Unlike `objectid`, this stays independent of module build ids and addresses, so equal expressions still share one cached body and ids still match across sessions, pkgimages and sysimages. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UjNooJk1j2y5NpVaJ9MAp8
ChrisRackauckas
marked this pull request as ready for review
September 24, 2026 17:32
Merged
ChrisRackauckas
referenced
this pull request
Sep 26, 2026
Agent-Harness: Claude Code Agent-Model: claude-opus-5-5[1m] Agent-Session: local Claude Code session 3cd6500a-1f81-46b5-ac0b-c466e15b6a53 on Chris's Mac (session ID, no URL) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #155. This is an alternative to #156, which fixes the same bug but loses compilation caching.
What changed and why
expr_to_idhashed every non-Exprleaf bystring(leaf)(src/RuntimeGeneratedFunctions.jl:456on master). The id is a type parameter of the RGF (:73-75) and keys both the body cache (_cache_body) and thegenerated_callfuncspecialization (:401). Leaves that print alike therefore collided, and the second RGF silently ran the first one's body. This affects more than objects with a summarizingshow. On master all of these collide:Scale(2.0)vsScale(3.0)with a customshow,make_scaler(2)vsmake_scaler(3)(same closure type, different captures),x -> xvsx -> "x", and() -> Int32(1)vs() -> 1.This PR serializes the expression to an unambiguous tagged byte stream and hashes that with SHA-1. Symbols, strings, bit numbers, modules,
GlobalRefs and types (by modulefullname, type name and parameters) have dedicated encodings. Other objects are encoded by type plus fields recursively,Array/Memoryby size plus elements (primitive eltypes as a raw byte block),BigInt/BigFloatby value. Mutable objects are tracked in anIdDictso cycles become back-references. Objects with no fields and opaque storage fall back toobjectid, which is safe but uncached.Why not
objectid(PR 156)objectidof any type, function or struct instance mixes in the defining module'sbuild_id, which is new each session for non-precompiled modules and each time a package is re-precompiled or baked into a sysimage. Mutable leaves hash by address. So under PR 156, the same expression gets a different id in a new session. That reintroduces #117 (precompiled/sysimage RGF signatures must match ids computed at runtime). It also means two equal-content mutable leaves (e.g.[1.0, 2.0]twice) no longer share a compiled body. Measured with a module defined in a script, two fresh processes:The new encoding uses only names, module paths and content, so ids are identical across processes.
Tests
Added to
test/core_tests.jl(Core group):Embedded objects are identified by type and content: the collision cases above, for immutable and mutable structs.Equal expressions share one compiled body: distinct-but-equal leaves (immutable and mutable structs,BigInt,BigFloat, arrays,Vector{Any}with undef slots, tuples, types,UnionAll) yield the same RGF type and cached body. It also runstest/shared/expr_id_stability.jlin two fresh processes and asserts the printed ids match.RGFPrecompTest.hembeds a package-defined struct and anInt32at precompile time. The test rebuilds the expression at runtime and asserts the same RGF type (pkgimage cache hit).Failing before (master src, new tests), Julia 1.13.0
On master the caching testset passes on its own (13/13); master already caches correctly.
PR 156's src with the same tests
The collision testset passes (10/10). The caching testset fails 7 of 13:
typeof(f_again) === typeof(f)fails for the mutable struct, the arrays,Vector{Any},BigIntandBigFloat, and the two-process id comparison fails:Passing after
GROUP=CoreandGROUP=QAwithjulia --project -e 'using Pkg; Pkg.test()', Julia 1.13.0 (release channel here), x86_64 Linux:GROUP=Corealso passes on Julia 1.10.12 (LTS) and 1.11.9 with the same counts.runic --check .andtyposare clean.Hashing cost
expr_to_idtimings, minimum of repeated runs. The expression is a 2000-statement block, and the array is a 100,000-elementVector{Float64}leaf:Push back on
Dicthashes by its slot layout, and anyPtrfield hashes by address. That can only cause extra cache misses, never collisions.string.Not verified
julia pre, macOS, Windows and 32-bit. Those run only in CI.docs/or public API.🤖 Generated with Claude Code
https://claude.ai/code/session_01UjNooJk1j2y5NpVaJ9MAp8
Generated by Claude Code