llvm-inprocess reports green on PRs while skipping its only real job (hazard 4), and fails when it does run

#7966 lists llvm-inprocess.yml as the extreme outlier — "last success 194.1h ago (budget 12h)". The 194h figure is real but it understates the problem, and the cause is not (only) starvation.

1. Its PR "successes" are vacuous

Sampled three recent success runs — 31505530279, 31499833415, 31476724152. Every one shows:

changes        = success
native-backend = skipped

The path filter skips the only job that does anything, and the workflow reports green. So the gate has not merely been silent for 8 days; it has been actively reporting success while executing nothing.

This is CLAUDE.md's "★ Four ways a gate can be unable to fail" hazard 4 in its purest form — the gate runs but its subject never did — and it is the most dangerous of the four precisely because the job is genuinely green. Any confidence drawn from llvm-inprocess passing on a PR in the last week is unfounded.

2. When it does execute, it fails

Its three most recent main runs — 31461469705, 31461474406, 31461478986 (all created 2026-08-11T05:23Z) — concluded failure, with changes=success, native-backend=failure.

Run 31461478986: created=2026-08-11T05:23:28Z, updated=2026-08-12T15:06:17Z — 33.7h in queue, then it ran and failed.

3. It is also starved

Its four scheduled runs since #7856: 08-11T19:45 queued (still), 08-12T03:10 cancelled, 08-12T08:03 cancelled, 08-12T14:06 pending. Zero completed. #7969 fixes that arm; it does not touch (1) or (2).

Failure detail (partial)

The Native-mode smoke step dies with a bare exit 1 under set -euo pipefail and emits no diagnostic on that path. The log ends at Collecting modules… / Generating code… on the third compile of spike.ts — i.e. the PERRY_LLVM_INPROCESS=diff arm — after spike_text and the native compile both succeeded (in-process LLVM backend active (LLVM 22.1.8)).

I cannot determine from the artifacts whether this is a real backend regression or a toolchain/environment issue. The step needs to say something before it exits; that is arguably the first fix regardless of the underlying cause.

What needs doing

Make the vacuous-green impossible. A workflow whose only substantive job is skipped should not report success. The usual shape is an aggregator job (if: always()) that requires the real job's result to be success or a genuinely-unaffected skipped, rather than letting the path filter silently stand in for a verdict. gc-native-roots-complete is the in-repo precedent for the fan-in pattern.
Give Native-mode smoke a diagnostic before exit 1, so the next failure is triageable from the log.
Re-run and triage the actual native-backend failure once (2) is in.

Per #7966, this workflow is "the promotion prerequisite" for the in-process LLVM backend arm. It cannot serve that role in its current state: it has been green-by-skipping on PRs and red-when-executed on main.

Found while diagnosing #7966. Related: #7969, #7970.

docs: record the filed follow-ups (#7970, #7971) in the gate notes

ci: fix the #7205 relapse on the scheduled gate arm, and reap dead PR…

ci: repair three broken gates — Windows cp1252, llvm-inprocess vacuou…

Resolved and locally revalidated on exact merged main; no CI result and no mini workflow was used.

Exact revision

SHA: 0da668c95150d78ba5aa2b8eff7b78c04cc381c1
tree: 86fa39aa89fa6eac71fd993218fe69d9cfbee442
The tree exactly equals the independently tested fix(codegen): canonicalize constants before RS4GC #8068 head eb600e766e80c415e3c626f3b65883678abc793c tree.
Isolated detached worktree and isolated Cargo target; primary worktree was not touched.

Structural/non-vacuity evidence

The current workflow blob (8bf738d91ee802c966254914efab15236b69d06a) and checker blob (3e7b658193718415b41e3d2737c8fb7076041252) are byte-unchanged from the exact structural audit at fe4cd09db0f1e260fc0ff63ec2e5c0079fceac3d:

checker self-test: 18/18, exit 0
repository wiring check: 7 gates / 31 workflows, exit 0
changes=success,native=success,schedule: exit 0, EXERCISED
changes=success,native=skipped,unaffected PR: exit 0, NOT EXERCISED, explicitly says the run asserts nothing about the backend
scheduled skipped anchor: exit 1 with targeted diagnostic
backend failure: exit 1
path-filter failure: exit 1
planted echo ... | grep -q pipefail sabotage: checker exit 1 naming the SIGPIPE hazard
planted command exit 37: wrapper exit 1 naming exit 37 and dumping captured stdout/stderr

Exact merged-main build and artifacts

LLVM_SYS_221_PREFIX=/opt/homebrew/opt/llvm CARGO_TARGET_DIR=<isolated> cargo build --profile perry-dev -p perry -p perry-runtime-static -p perry-stdlib-static --features perry/llvm-inprocess: exit 0.

With PERRY_RUNTIME_DIR pinned to that target and PERRY_NO_AUTO_OPTIMIZE=1, artifact identities stayed unchanged through the complete matrix:

perry: mtime 1786674185, 144755752 bytes, SHA-256 c0d0241a3d83ccbc731f43b55d17d360d882c0002fa25b69f584f5afbc6403d9
libperry_runtime.a: mtime 1786670014, 74895032 bytes, SHA-256 c5ffa7db89cd115226b449bbd04000f68576ebae6e88be175e826cf9a87919aa
libperry_stdlib.a: mtime 1786670221, 179955112 bytes, SHA-256 5cbcfdeba357a9dff20ab833ec38c87c08b0310f68e134fbee58e6740702c5f5

Focused codegen gate

991/991 library tests passed; exit 0
corpus_spike, corpus_batch_kernel, corpus_exception_handling, and rs4gc_schedules_in_process liveness assertions: all exit 0
LLVM banner: 22.1.4

Substantive native-mode matrix

spike text compile/execute: 0/0
spike native compile/banner/execute: 0/0/0
spike text/native behavior cmp: 0; 92 bytes; SHA-256 ccfecd9c51f0cd29a7fa349f3047d527e96b2b60f2078a269f91d0e9b6a4bf9c
spike diff compile/verdict assertion: 0/0; byte-identical objects, 164671 bytes
batch diff with PERRY_CODEGEN_UNITS=3 command/verdict assertion: 0/0; byte-identical merged objects, 58160 bytes across 3 units
EH text compile/execute: 0/0
EH native compile/banner/execute: 0/0/0
EH text/native behavior cmp: 0; 116 bytes; SHA-256 1e5f575323cceebe5e2e822084f346097845a7050e885ed83de45b25e8d7a6b6
EH diff compile/verdict assertion: 0/0; byte-identical objects, 266985 bytes

The failure discovered during this audit was split into #8065 and fixed by merged #8068 without weakening the byte oracle. Relevant earlier merged work includes #7986 for the non-vacuous fan-in/diagnostics, #7998/#8000 for the reader failures tracked in #7982, #8062 for shared precise-root lowering, and #8063 for diagnostic liveness under pipefail.

All #7971 acceptance properties are now green on exact current main. Closing as completed. #7966 remains open for its separate freshness tracking.

Navigation Menu

Repository navigation

llvm-inprocess reports green on PRs while skipping its only real job (hazard 4), and fails when it does run #7971

1. Its PR "successes" are vacuous

2. When it does execute, it fails

3. It is also starved

Failure detail (partial)

What needs doing

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

llvm-inprocess reports green on PRs while skipping its only real job (hazard 4), and fails when it does run #7971

Description

1. Its PR "successes" are vacuous

2. When it does execute, it fails

3. It is also starved

Failure detail (partial)

What needs doing

Activity

proggeramlug commented on Aug 14, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions