Repository navigation
gc-ratchet: 7 gating rows red on main since 0.5.1451 — and 04_dead_after_deep_stack / 11_collect_at_depth are NOT among them #7843
Description
gc-ratchet: 7 gating rows red on main — and 04_dead_after_deep_stack / 11_collect_at_depth are NOT among them
Two separate findings from measuring the pinned gc-ratchet on current main
(ab1bd464b, perry 0.5.1464) on a darwin-arm64 host.
1. The two probes everyone believes are red are green
04_dead_after_deep_stack and 11_collect_at_depth have been carried as
"pre-existing red on main, unowned" since #7799. They are not red on current
main. Measured, shared_ci profile, 7 repeats:
04_dead_after_deep_stack— all nine gating cells+0.00%, bit-identical
to the pin.11_collect_at_depth— all gating cellsok;heap_used_bytesis an
improvement (−11.99%),copied_objects6,139 → 6,141 (+0.03%, allowance
307),promoted_objects0 → 0,minor_cycles1 → 1.
Why they looked red
The six rows that were reported (04.copied_objects, and 11's
heap_used_bytes / copied_objects / copied_bytes / promoted_objects /
promoted_bytes) are exactly the six cells that #7783's re-pin already
absorbed, and every observation of them comes from a checkout that predates
that re-pin.
- The re-pin landed as
db44b31b7at 2026-08-10 18:11 UTC. - perf(gc): predict the futile full mark-sweep instead of pricing it afterwards (retain 3.99x -> 2.54x node, retain_wide 7.01x -> 2.89x) #7799's PR body names its reference build as
origin/main @ 0a2bf15bd.
git merge-base --is-ancestor 0a2bf15bd db44b31b7→ yes, it predates the
re-pin. Its baseline artifact therefore still carried the OLD values — and it
had 13 probes, which is why the PR body says "all 13 probes". Main's
baseline has 14 (14_grow_then_churncame with the re-pin). - The only two gc-ratchet CI runs that have executed and failed in the last
200 (31415780870,31408538105) both started before 18:11 UTC on 08-10,
and theirBaselinecolumn is the pre-re-pin artifact verbatim:
04.copied_objects 565,11.heap_used_bytes 5,097,776,
11.promoted_objects 6,150. TheirCurrentcolumn is exactly what main now
pins (605 / 5,307,584 / 0).
So both arms of #7799's A/B compared against a stale artifact and went red
identically. "Not my PR" was correct; "pre-existing on main" did not follow.
No code fix is needed for these two probes. The stale premise is the bug.
2. Seven different gating rows ARE red on main
Same run, check exit code 1:
| probe | metric | baseline | current | Δ |
|---|---|---|---|---|
01_nursery_churn |
heap_used_bytes | 5,229,288 | 5,640,688 | +7.87% |
06_string_retention |
heap_used_bytes | 4,960,920 | 5,373,128 | +8.31% |
09_try_catch_roots |
heap_used_bytes | 4,777,680 | 5,190,960 | +8.65% |
14_grow_then_churn |
heap_used_bytes | 4,651,912 | 5,069,000 | +8.97% |
14_grow_then_churn |
step_cycles | 26 | 24 | −7.69% |
14_grow_then_churn |
copied_objects | 594 | 561 | −5.56% |
14_grow_then_churn |
copied_bytes | 2,442,528 | 2,306,832 | −5.56% |
The four heap_used_bytes deltas are near-constant in absolute terms —
+411,400 / +412,208 / +413,280 / +417,088 bytes. That is not a fixed addition
to resident heap, though: across all 14 probes the shift is bimodal, with
five probes moving down by a similarly tight −552k…−636k and five unchanged
to the byte (full table below). Four rows fire only because the gate's
[increase] direction is the one that fails; the equally large moves the other
way pass as improvements. A retention boundary moved, not a constant.
This is code, not the host, and not noise
The baseline was pinned on the bench mini (perry-macos.fritz.box, Apple M1,
8 cores, 8 GB). I measured on a different machine (M1 Max, 10 cores, 68 GB), so
"the pin is not portable off its host" had to be excluded before any of this
could be called a regression. A binary A/B settles it — same host, same
harness, same probes, same baseline artifact, only the artifact triple
(perry + libperry_runtime.a + libperry_stdlib.a) swapped:
| arm | built at | check --profile shared_ci |
|---|---|---|
| pin | a1fd5a7e8 (perry 0.5.1451) — the commit the baseline was measured at |
gc-ratchet: OK, exit 0 |
| main | ab1bd464b (perry 0.5.1464) |
FAILED, exit 1, the 7 rows above |
So the pinned artifact reproduces green on all 14 probes on a machine that is
not the one it was pinned on — the shared_ci profile's claim that its gating
metrics "transfer across machine classes" holds, and the baseline is a valid
reference here. The 7 red rows are therefore code drift between 0.5.1451 and
0.5.1464.
Repeatability control: arm-main was measured twice, in separate processes,
and produced an identical red-row set both times. Within each run the
harness reports heap_used_bytes spread 0% across 7 repeats on every probe,
and correctness=pass on all 14 in both arms.
Per-probe heap_used_bytes, both arms
The shift is bimodal, not one constant — which is why only four rows fire
(the gate's [increase] direction) while five equally-large moves in the other
direction pass as improvements:
| probe | pin 0.5.1451 | main 0.5.1464 | Δ |
|---|---|---|---|
01_nursery_churn |
5,229,288 | 5,640,688 | +411,400 |
06_string_retention |
4,960,920 | 5,373,128 | +412,208 |
09_try_catch_roots |
4,777,680 | 5,190,960 | +413,280 |
14_grow_then_churn |
4,651,912 | 5,069,000 | +417,088 |
03_cross_gen_writes |
1,394,736 | 1,394,736 | 0 |
04_dead_after_deep_stack |
4,890,016 | 4,890,016 | 0 |
05_closure_capture |
5,329,880 | 5,329,880 | 0 |
08_map_set_sidetables |
1,512,024 | 1,512,024 | 0 |
12_large_live_set |
50,435,552 | 50,435,552 | 0 |
02_survivor_promotion |
9,414,464 | 8,862,256 | −552,208 |
07_array_grow_evacuate |
15,631,696 | 15,002,592 | −629,104 |
10_store_receiver_across_alloc |
4,664,720 | 4,029,640 | −635,080 |
11_collect_at_depth |
5,307,584 | 4,671,224 | −636,360 |
13_large_eden_survivors |
7,686,560 | 7,051,232 | −635,328 |
Both arms genuinely exercised the collector — 64 vs 63 total minor_cycles,
~684k copied_objects, no probe with zero minors — so the green arm is not a
vacuous pass.
Reproducing
The build is cheap (5m32s incremental for the artifact triple), so bisecting
this is a short job:
git checkout --detach <commit>
CARGO_TARGET_DIR=<tgt> cargo build --release -p perry -p perry-runtime-static -p perry-stdlib-static
cp <tgt>/release/{perry,libperry_runtime.a,libperry_stdlib.a} /tmp/arm/
git checkout main # harness, probes and baseline must be MAIN's
PERRY_NO_AUTO_OPTIMIZE=1 PERRY_RUNTIME_DIR=/tmp/arm \
python3 benchmarks/gc_ratchet/gc_ratchet.py measure \
--perry /tmp/arm/perry --repeats 7 --node "$(command -v node)" --output /tmp/m.json
python3 benchmarks/gc_ratchet/gc_ratchet.py check --current /tmp/m.json --profile shared_ciSwap only the artifact triple; keep the checked-out harness on main so the
probe set and baseline are constant across arms.
Prime suspects in the window, all collector- or layout-adjacent:
1d22273a1 (#7809, per-object layout tables, +296 lines in
gc/layout_tables.rs), 99705c937 (#7812, single-slot pointer masks),
d04cc3f29 (#7834, pointer-free typed layout stamped into the allocation
header). #7799's author measured its own branch as gating-identical to its
reference, so it is unlikely to be ad8240b5b.
3. The gate has not run on main since 2026-08-09
Independent of either finding, gc-ratchet is producing no signal:
- Every run on
mainsince 08-10 is stuck inqueued— ~103 of the last 200
runs across all branches, with 87cancelled, 7pending, 1in_progress
and 2failure. Zero successes in that window. - The last successful run on
mainwas 2026-08-09T06:52. gc-ratchetis not in branch protection's required contexts
(api-docs-drift,cargo-test,compile-smoke,conformance-smoke-complete,
lint,parity,security-audit) — that is GC follow-ups from #7733: one-way prototype latch (#7510 shape), missing positive-direction pacing test, unmeasured grow-then-churn transition, advisory-only ratchet #7737 item 4.
A gate that is not required, and whose runs never reach a runner, is dark in
both of the ways CLAUDE.md's "four ways a gate can be unable to fail" list
warns about. Both findings above went unnoticed for two days because of it.
Activity
proggeramlug commentedon Aug 11, 2026
Bisect narrowing: the codegen trio is excluded.
Third arm, same method as the A/B above — same host, same harness, same probes, same baseline artifact, only the artifact triple swapped:
check --profile shared_cia1fd5a7e899705c937(#7812)ab1bd464bThe split arm fails with the identical seven rows and identical numbers as main —
01/06/09/14heap_used_bytesat +7.87 / +8.31 / +8.65 / +8.97%, and14'sstep_cycles26→24,copied_objects594→561,copied_bytes2,442,528→2,306,832. Not one cell differs between the split arm and main.So the cause is at or before
99705c937, which rules out the last batch entirely:8b612e571(perf(transform): eliminate redundant property reads across diverging guard chains #7833, prop-CSE across diverging guard chains)d04cc3f29(perf(codegen): stamp a pointer-free shape's typed layout into the allocation header #7834, pointer-free typed layout stamped into the allocation header)48c5e9a43(perf(codegen,runtime): let a declaredstringpick the concat lowering #7835, declared-stringconcat lowering)Remaining window:
db44b31b7..99705c937. The collector-relevant commits in it, in reverse order:99705c937— perf(gc): don't mint per-object pointer masks for single-slot payloads #7812, don't mint per-object pointer masks for single-slot payloads (+131 ingc/layout_tables.rs)1d22273a1— perf(gc): stop the globalThis bootstrap from disabling the per-object layout fast path #7809, stop the globalThis bootstrap from disabling the per-object layout fast path (+296 ingc/layout_tables.rs, plusgc/layout.rs,gc/hot_tls.rs,gc/telemetry.rs)94fdafd50— perf(class): stop disarming every dispatch guard when a class prototype is materialized #7800, stop disarming every dispatch guard when a class prototype is materializedad8240b5b— perf(gc): predict the futile full mark-sweep instead of pricing it afterwards (retain 3.99x -> 2.54x node, retain_wide 7.01x -> 2.89x) #7799, predict the futile full mark-sweeped274e5c3— perf(runtime): gate the rare-subclass probes on the object property-miss path (asyncpipe 9.69x -> 1.76x node) #7795, gate the rare-subclass probes on the object property-miss path#7799 is the least likely of these: its author measured its own branch against its own reference and reported every gating metric identical across all probes, which is a valid relative comparison even though both sides carried the stale baseline. That leaves #7809 and #7812 as the prime suspects — they are the two that touch
gc/layout_tables.rs, and a change in which objects get a per-object layout is exactly the shape that moves a retention boundary in both directions at once, which is the bimodal pattern in the table above.The next split to run is
94fdafd50(#7800, immediately before the layout pair). Green there pins it to {#7809, #7812}; red pins it to {#7795, #7799, #7800}. I have not run it.