Repository navigation
CI cannot see crates/*/tests/*.rs regressions: per-PR runs --lib --bins, and SOURCE_SUITE_MAP is hand-maintained (2 failures beyond #7494) #7708
Description
cargo test -p perry-codegen is red on main (v0.5.1407) with 6 failures across 4 integration suites. All 6 predate the PRs merged today (verified by running the same command on main before and after: identical set).
integer_arithmetic_array_push_omits_inbounds_layout_note_and_barrier
integer_modulo::i32_counter_mod_unsafe_or_nonliteral_divisors_keep_frem
large_local_array_push_inbounds_store_emits_precise_slot_barrier
proven_buffer_and_typed_array_reads_are_numeric_operands
reassigned_typed_array_store_records_runtime_fallback
typed_f64_receiver_method_clone_raw_loads_after_composed_guards
Why this sat unseen
Per-PR cargo-test runs --lib --bins (.github/workflows/test.yml:829) — no tests/*.rs integration suite runs on a PR at all. e2e-scoped runs only suites the diff names, plus a hand-maintained SOURCE_SUITE_MAP that maps all of crates/perry-codegen/src/ to exactly three suites, none of these four.
scripts/ci_e2e_scope.py's own header already names this as the #6037 class: "a source change that regresses an existing suite the diff doesn't name… can land green and sit red for days." This is that, except it is longer than days and nobody has a number for how long, because nothing reports it.
Why it matters beyond these 6
Today two separate PRs (#7698, #7702) each broke a perry-codegen integration test, and in both cases the only way to find out was to run the full suite by hand and diff against a hand-captured main baseline. That is not a workflow — it depends on someone deciding to check. Both were caught; the next one won't be.
The two failures each PR introduced were also different in kind, which is worth recording:
- perf(typed-feedback): stop emitting recording calls into default builds (#7480) #7702 was a genuine assertion that had to change (it required a recording call the PR deliberately stops emitting).
- perf(class-fields): a conforming pointer store skips the layout note (#5094) #7698 was a test that had stopped being able to see its subject — the new conditional split a basic block in three, and the helper sliced text to the first
br. The write barrier had not moved; the test just couldn't find it, and no wider text slice would have worked because LLVM emits the join block between the other two.
Suggested
- Triage the 6 — some may be stale assertions, some may be real.
- Then either add
perry-codegen's suites to the per-PR set, or makeSOURCE_SUITE_MAPderive from something that can't silently omit a suite. A hand-maintained map of source dirs to suites has the same failure mode as theMIN_COMPILED=90floor fixed in gate(gc): make the root-dominance corpus floor a two-sided ratchet #7706: it is correct on the day it is written and nothing tells you when it stops being.
Activity
proggeramlug commentedon Aug 9, 2026
Correction: this partially duplicates #7494 — narrowing it
I filed this without finding #7494, which already documents four of the six and asks the right question about each ("did the lowering legitimately change, or did a proof regress?"):
proven_buffer_and_typed_array_reads_are_numeric_operandsreassigned_typed_array_store_records_runtime_fallbackinteger_modulo::i32_counter_mod_unsafe_or_nonliteral_divisors_keep_freminteger_arithmetic_array_push_omits_inbounds_layout_note_and_barrierThose four belong to #7494. It also establishes something I did not: they are lowering-independent (identical with and without
PERRY_RS4GC=0, so not part of the #7370 native-roots family) and they reproduce when run alone, so they are not order-dependent either.What is genuinely new here — two failures #7494 does not list
Both are in the same untested tier. Whether they are older than #7494 or arrived since, I have not established — worth checking, because "the set grew" and "the set was always this size" have different implications for how fast this tier rots.
And the structural half, which #7494 names but does not track
#7494 explains why they went unnoticed; it does not carry the fix. That is what this issue should be:
Per-PR
cargo-testruns--lib --bins(test.yml:829), so nocrates/*/tests/*.rssuite runs on a PR at all.e2e-scopedruns only suites the diff names, plus a hand-maintainedSOURCE_SUITE_MAP(ci_e2e_scope.py:86) mapping all ofcrates/perry-codegen/src/to three suites — none of them these.That map has the same failure mode as the
MIN_COMPILED=90floor fixed in #7706 (a 131-source corpus tolerating 41 silent failures) and thehandle-floorregex found in #7709 (blind to sites whose local is namedraw): a hand-maintained list of what to check is correct on the day it is written, and nothing tells you when it stops being.Two of today's PRs each broke a suite in this tier, both caught only by running the full suite by hand and diffing against a same-day baseline. That is not a workflow — it depends on someone deciding to check.
Suggested split: the four → #7494; the two new ones → triage here or move them there once dated; and the CI-coverage gap stays here as the actionable item, since it is what makes the other two issues recur.
[-]cargo test -p perry-codegen is red on main: 6 pre-existing failures no per-PR gate can see[/-][+]CI cannot see crates/*/tests/*.rs regressions: per-PR runs --lib --bins, and SOURCE_SUITE_MAP is hand-maintained (2 failures beyond #7494)[/+]fix(codegen): resolve #7494's four lowering-independent proof-test fa…
fix(codegen): resolve #7494's four lowering-independent proof-test fa…
ci: perry-codegen integration suites can no longer be silently omitte…
proggeramlug commentedon Aug 10, 2026
Triage of the six, re-measured on
mainat v0.5.1432, onecargo test -p perry-codegen --test <suite>per suite. Four are fixed, by unrelated work, with nothing recording it:integer_arithmetic_array_push_omits_inbounds_layout_note_and_barrierinteger_modulo::i32_counter_mod_unsafe_or_nonliteral_divisors_keep_fremproven_buffer_and_typed_array_reads_are_numeric_operandsreassigned_typed_array_store_records_runtime_fallbacklarge_object_barriers::large_local_array_push_inbounds_store_emits_precise_slot_barriernative_proof_regressions::typed_f64_receiver_method_clone_raw_loads_after_composed_guards— typed_f64_receiver_method_clone_raw_loads_after_composed_guards: the guard-failure edge no longer calls $generic — miscompile or intentional collapse? #7506Four entries silently stopping being true is the same failure mode #797 recorded for the parity skip-list, and it shaped the fix more than the count did.
One measurement worth having on the record: 23 of the 24 suites in
crates/perry-codegen/tests/are in-process (emit_ir_only: true— noperry compilesubprocess, no link, no runtime), and every one completes in 2.2–10.4 s wall clock, nearly all of it cargo's freshness check. The map's three entries were never a judgment about the other 20; that was what one commit had time to verify. So the "either/or" in the Suggested section turns out not to be an either/or — the suites are cheap enough to add and the map can be made unable to omit one.PR #7748: map goes 3 → 22 suites;
--self-testfails on anyperry-codegen/tests/*.rsthat is neither mapped nor in a newSUITE_EXCLUSIONS; exclusions are per-TEST (sonative_proof_regressionscontributes its other 261) and self-invalidating — the job runs exactly the held-out tests and fails if one passes. Also fixes the--capof 12, which would have silently truncated the 22-suite set and re-created this bug one layer up.ci: perry-codegen integration suites can no longer be silently omitte…
ci: perry-codegen integration suites can no longer be silently omitte…
ci: perry-codegen integration suites can no longer be silently omitte…
test(codegen): assert the push barrier ordering over the CFG region, …
test(codegen): assert the push barrier ordering over the CFG region, …
test(ci): run native_proof_buffer_views per PR again after #8302 (#8321)
ci: classify string_array_length_9160 so e2e-scoped stops failing eve…
ci: register the two unclassified perry-codegen suites
ci: register the two unclassified perry-codegen suites