Repository navigation
aarch64-linux: the fast stack-map walker and the unwinder disagree by 96 bytes on the same root (PERRY_STACKMAP_WALKER=verify) #7984
Description
Summary
On aarch64 Linux (ELF), PERRY_STACKMAP_WALKER=verify catches the fast
frame-pointer-chain walker and the unwinder resolving the same root to two
different addresses. The gate is working; the code is wrong.
This is the ubuntu-24.04-arm arm of gc-native-roots (#7970). It is a root
misidentification on a supported target, not a harness problem — unlike the
macos-14 arm of the same run, which was a missing PERRY_GC_DIAG=1 and is
fixed separately.
Evidence
Run 31405431962, job native-roots-rs4gc (ubuntu-24.04-arm, aarch64, ELF),
sha bf8af0e0f, probe 01_nursery_churn:
thread '<unnamed>' (20234) panicked at crates/perry-runtime/src/gc/roots/stack_maps.rs:507:5:
assertion `left == right` failed: PERRY_STACKMAP_WALKER=verify: fast walk visited 1 unique slots, unwinder visited 1
left: [281474742909688]
right: [281474742909592]
Both walkers found one slot, so this is not a missed or invented frame. They
disagree on where that slot is:
281474742909688 - 281474742909592 = 96 bytes
verify_visit (gc/roots/stack_maps.rs) runs fp_chain::visit non-mutating,
then unwind::visit for the real visitation, and asserts the address sets are
equal. A 96-byte constant offset between them is the signature of a
frame-base/CFA disagreement, not of a parse failure.
Why this matters more than a red CI arm
fp_chain is the fast walker — the one that runs when verify is off. If it
resolves a root 96 bytes away from the real slot, then on this target the
collector marks and rewrites the wrong stack word. That is precisely CLAUDE.md's
"root-store dominance" failure class: nothing is visibly wrong at collection
time, and it surfaces cycles later as TypeError: value is not a function.
verify mode exists to catch exactly this, and it did — the first time anyone
ran it end-to-end on aarch64 ELF. The workflow's own header predicted the shape:
What no arm has ever exercised is the end-to-end chain on this target — real
LLVM asm output, real ELF linking, real runtime walking — where the failure
mode is not a parse error but a wrong answer.
Reproduction
# on aarch64 Linux
export RUSTFLAGS="-C force-frame-pointers=yes -C force-unwind-tables=yes"
cargo build --profile perry-dev -p perry -p perry-runtime-static -p perry-stdlib-static
PERRY_RS4GC=1 ./target/perry-dev/perry benchmarks/gc_ratchet/probes/01_nursery_churn.ts \
-o /tmp/rs4gc-01
PERRY_GC_TRACE=1 PERRY_GC_DIAG=1 PERRY_STACKMAP_WALKER=verify \
PERRY_RS4GC=1 PERRY_GC_FORCE_EVACUATE=1 PERRY_GC_VERIFY_EVACUATION=1 \
PERRY_GC_HEAP_LIMIT=8 PERRY_GC_INCREMENTAL=0 PERRY_CONSERVATIVE_STACK_SCAN=off \
/tmp/rs4gc-01x86-64 ELF (ubuntu-latest) passes the same step, so this is aarch64-specific.
Open questions for whoever picks this up
- Which walker is right? The unwinder is the reference implementation, so
the presumption isfp_chainis 96 bytes off — but that should be confirmed
against the emitted stack map rather than assumed. - Is 96 constant? One probe, one slot is a thin sample. Worth running the
whole probe matrix underverifyon this target and checking whether the
delta is always 96 (a fixed frame-layout term) or varies (an offset-scaling
bug, e.g. the.word-vs-.longfield-width hazardword_width_forwarns
about, which is aarch64-ELF-specific). - Does the fast walker run in production on this target? If yes, this is a
live correctness bug on aarch64 Linux, not only a CI failure.
Found while repairing #7970. Sibling: the macOS arm of the same run was a gate
defect (missing PERRY_GC_DIAG=1), and the Windows arm is a separate link
failure.
Activity
ci: repair three broken gates — Windows cp1252, llvm-inprocess vacuou…
docs(gc-handoff): #7984 working notes and the changeset for #7997
test(gc): run both aarch64 stack-map walkers over a frame we chose
fix(gc): the aarch64 prologue decoder was blind to two real shapes (#…
fix(gc): decode the SVE stack adjustment instead of giving up on the …
docs(changelog): fold the #7984 fix into the changeset
docs(gc-native-roots): the ubuntu-24.04-arm arm is no longer expected…
proggeramlug commentedon Aug 12, 2026
Answered, and fixed in #7997
The fp-chain walker is the wrong one — the bad case, since it is what runs when
verifyis off.Reproduced end to end on aarch64 Linux (Ubuntu 24.04 arm64 container, LLVM 22.1.8 from apt.llvm.org, this issue's own RUSTFLAGS). The trigger is the tuning, not the distro. Perry builds a host binary with
-mcpu=native; on a Neoverse-class core that enables SVE, and SVE changes the prologue LLVM emits:The container's own
-mcpu=native(an Apple core, no SVE) passes all 14 probes under all three walkers, which is why this was never visible off that runner — and no macOS arm could ever have seen it.The frame
main, the module body, 100 stack-map records, built-mcpu=neoverse-n2, read withobjdump -d:Two independent blind spots in
fp_to_sp_offset, in one prologue:sub spends the run", and LLVM interleaves those stores with the frame-pointer setup whenever SVE is on.addvl sp, sp, #-Nwas not decoded at all. Its multiplier is in the instruction; its unit is the runtime vector length, which is not.Q1, which walker is right — established arithmetically, not by presumption
Under
qemu -cpu max(VL = 64 B), from the report #7997 adds toverify:Same function, same ip, same record, same offsets — a base disagreement:
The unwinder's base is the frame's real body SP, derived independently from the prologue the fast walker misread.
Q2, is 96 constant — no
It is that frame's missed tail and it scales with the vector length:
ubuntu-24.04-arm(VL = 16 B)0x50 + 1 x 16-cpu max(VL = 64 B)0x50 + 2 x 64A fix keyed on 96 would have been wrong on every other vector length. (Not the
.word-vs-.longhazard the issue suspected: the compact map decodes fine, and a parse error would move both walkers identically and produce no divergence at all.)Q3, does the fast walker run in production on this target — yes
rs4gc_enabled()defaults toNATIVE_ROOTS_TARGET_OK, which is true on aarch64 Linux, andWalkerMode::Fastis the default. So this was a live correctness bug on aarch64 Linux with SVE, not only a CI failure: the collector marked and rewrote a stack word 96 bytes away from the root.The fix
Stores through sp with no writeback are transparent to the accumulation run (enumerated by opcode, so an unrecognised instruction still ends it).
addvl/addplwriting sp is decoded — multiplier fromimm6, unit fromprctl(PR_SVE_GET_VL), cached; where the length cannot be read the whole decode fails and the frame goes to the platform unwinder.Failing closed only was the first cut and was not enough: on an SVE host that frame is the module body, so the fast walker would decline at every collection and
verifyturns a decline into a panic — this arm would have stayed red on its own fix.Validated at two vector lengths: 28
verifyruns (14 probes x VL 16 B and 64 B), all byte-exact, withfp_walks 12 / fallback_walks 0 / locations_visited 2678proving the fast walker is the one doing the work.fix(gc): the aarch64 fast stack-map walker misread every SVE-shaped p…