You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Public baseline freshness gate reddens main on unrelated dependency churn #7290
benchmarks/public_baseline.py's SOURCE_PATHS includes Cargo.toml, and the artifact's source_fingerprint is a hash over those paths. #6528 made _fingerprint_bytes strip perry's own version = "x.y.z" line before hashing, so a release bump no longer trips it — but it does nothing for dependency pins.
So any routine Cargo.toml churn (dependabot/taze bumps, adding a registry entry, a new dep) changes the fingerprint and fails "Public benchmark evidence freshness" on main, even when nothing about the benchmarked TS/JS suite changed.
Why this is worth fixing rather than living with
Clearing it is not a normal CI retry. ./benchmarks/run_public_baseline.sh requires a clean tree, Node exactly v22.23.1, Bun exactly 1.3.14, AC power, and a quiet CPU (≤25% for 60 consecutive seconds) — it monopolises the machine, by design, because it is a benchmark.
Observed cost of the current instance: the gate has been red on main since 2026-07-29. Because lint is a step sequence and a failing step skips the rest, that also silently skipped the eight later gate steps (File size limit, Binding upstream pins, GC store-site inventory, Address-classification audit, Gap snapshot self-test, Platform-aware parity allowlist self-test, Moving-GC gate wiring, GC matrix liveness gate) for days — in a required job that looked red for an unrelated reason. It is currently the sole blocker on #7278.
An attempt to regenerate it today was abandoned after ~55 minutes: no 60-second quiet window was reachable on a shared machine.
The concern
A gate that predictably goes red on unrelated merges, and whose only remedy is a hardware-gated manual ritual, is one that eventually gets admin-bypassed or ignored — which is the same hazard class the repo's own gate-design notes warn about.
Possible directions (not a prescription)
Narrow SOURCE_PATHS so Cargo.toml contributes only what actually affects benchmark output (e.g. the [profile.*] sections and perry's own source deps), rather than the whole file.
Extend the _fingerprint_bytes normalisation from perry's own version line to dependency version pins generally, keeping structural changes significant.
Separate "the harness or suite changed, regenerate" from "an input moved, re-verify cheaply".
Related: #7278 (adds if: ${{ !cancelled() }} to its own lint step so it survives this skip), and separate in-flight work to guard or split the remaining lint steps.
Covered by #7821, which implements the first of the directions listed here: SOURCE_PATHS still lists Cargo.toml, but only its [profile.*] tables are fingerprinted.
That removes exactly the churn this issue is about — a dependabot/taze bump, a new dependency, or a new registry entry contributes nothing to the hash. Measured on the current manifest, extraction drops 19,466 bytes to 8,681, keeping all 45 profile tables and dropping [workspace.package] and every dependency line. It also subsumes the second direction (normalizing dependency pins): a line that is not hashed cannot need normalizing.
Not addressed there, and noted above as separate: a failing lint step silently skipping the eight gate steps after it, because lint is a step sequence. That is #7278's scope.
Activity
proggeramlug commentedon Aug 11, 2026
Covered by #7821, which implements the first of the directions listed here:
SOURCE_PATHSstill listsCargo.toml, but only its[profile.*]tables are fingerprinted.That removes exactly the churn this issue is about — a dependabot/taze bump, a new dependency, or a new registry entry contributes nothing to the hash. Measured on the current manifest, extraction drops 19,466 bytes to 8,681, keeping all 45 profile tables and dropping
[workspace.package]and every dependency line. It also subsumes the second direction (normalizing dependency pins): a line that is not hashed cannot need normalizing.Not addressed there, and noted above as separate: a failing
lintstep silently skipping the eight gate steps after it, becauselintis a step sequence. That is #7278's scope.Leaving this open until #7821 merges.