Skip to content

ci: drop the unhonoured pre-prune waiver (CACHE-034) - #3583

Merged
zackees merged 1 commit into
mainfrom
fix/cache-034-drop-unhonoured-pre-prune
Oct 10, 2026
Merged

zackees merged 1 commit into
mainfrom
fix/cache-034-drop-unhonoured-pre-prune

Conversation

@zackees

@zackees zackees commented Oct 5, 2026 •

Copy link
Copy Markdown
Owner

Closes #3678

What

ci.toml declared pre-prune = true on both writer flows (main and nightly).
Nothing in this repository has ever run ci-lint cache preprune -- verified by
grep -rn "preprune" .github/ ci/, which returns nothing.

CACHE-004's budget arithmetic honoured the declaration on its own: with every
writer flow pre-pruning, the lockfile-change peak term is dropped from the
worst-case footprint. This repository's modelled footprint was understated by
exactly that term -- 17.30GB -- for free.

This PR takes fix (B): it removes the declaration so the arithmetic counts the
peak honestly. It does not touch [cache].budget, any family size, or any live
cache entry.

CACHE-004 was already failing

The most important thing in this PR is that CACHE-004 is a violation both
before and after
. Before this change the worst case was already 19.87GB against
a 9GB budget. The pre-prune declaration was not what made it pass, and nothing
here was resized, rebudgeted, or otherwise adjusted to make it pass.

The honest statement is that this repository's declared cache shape does not fit
a 9GB budget, and the waiver was hiding 17.30GB of that. The declared shape:

  • compile alone: 2600MB x 6 platforms = 16.36GB
  • dylint-out (lockfile-keyed): 700MB = 0.73GB
  • registry (lockfile-keyed): 200MB = 0.20GB
  • [cache.pr].budget: 1.00GB on top of the above

No combination of pre-prune and the current family sizes fits 9GB. Making this
pass honestly requires a real decision about which platforms write a compile
entry and what each one actually weighs -- a measurement and cardinality
question, not a ci.toml edit. I have deliberately left that visible rather than
picking numbers to make it green. Raising [cache].budget is not an option:
9GB is already the fleet's floor near GitHub's 10GiB repository cap, and this
repository's live caches were measured at that ceiling.

Why not (A)

ci-pre.yml is the closest fit -- it already holds actions: write and runs
first -- but nothing in ci.yml needs: it. It is invoked as a job and
deliberately un-gated so that a janitor sweep never delays a build (its own
header comment says so). A pre-prune step placed there would not be ordered ahead
of this repository's setup-soldr saves in the same run, so it would not remove
the peak it is supposed to remove. Ordering it correctly means making build jobs
depend on a cache-maintenance job, which inverts the deliberate design and slows
every run. That is a larger change than this honesty fix should make, and it
would not fix the underlying over-budget shape anyway.

CACHE-004 arithmetic, before and after

Verified with
uvx --from "git+https://github.com/zackees/ci.yml@92106df315bd28de24ef1808c893c91a5859122d" ci-lint precheck --repo . --local

before after
steady total 18,800,967,680 B 18,800,967,680 B
+ lockfile-change peak 17,301,504,000 B (waived) 17,301,504,000 B (counted)
+ [cache.pr].budget 1,073,741,824 B 1,073,741,824 B
worst case 19,874,709,504 B 37,176,213,504 B
budget 9,663,676,416 B 9,663,676,416 B
CACHE-004 VIOLATION VIOLATION

Total precheck findings are unchanged at 1806 violations / 26 needs_review before
and after, confirming this edit introduces no new finding -- it only changes the
number the existing CACHE-004 finding is about.

GATE-005

No test step was added. This change touches ci.toml only -- no workflow, no
test invocation, no runner. The isolated-runner requirement for this repository's
test suite is untouched.

Unresolved

  • CACHE-004 remains a live violation and needs a sizing decision (which platforms
    write compile, and each platform's real weight). Not resolvable by editing
    ci.toml honestly.
  • Restoring pre-prune = true honestly needs a pre-prune step genuinely ordered
    ahead of the saves, which the current ci-pre.yml call structure cannot
    express.
  • No Local-Gate: trailer is attached. The precheck was run and is reported
    above; I did not fabricate a gate attestation.

@zackees

zackees commented Oct 5, 2026

Copy link
Copy Markdown
Owner Author

CACHE-034 is implemented and merged — zackees/ci.yml#352 (docs #353). It fires on soldr's main today, so this PR is the remediation for an enforced rule rather than a review suggestion.

Verified independently

Your headline claim is correct and is the important part: CACHE-004 was already a VIOLATION before this change. Confirmed against main — compile is declared max = "2600MB" with per = "platform" across 6 platforms = 15.6 GB steady against a 9 GB budget, before any peak term. The pre-prune waiver was not what made it pass.

Leaving it failing rather than resizing families is the right call, and matches how this was handled on zackees/clud#1800 — a repo that honestly fails its own check is correct; one with numbers invented to pass is not.

Why the two repos came out differently

Worth recording, since three of the six affected repos have now been checked and the answer is not uniform:

repo steady honest worst budget verdict
bosn 3.36 GB 7.27 GiB 9 GB fits
running-process 11.04 GB 19.19 GiB 9 GB already failing
soldr 18.80 GB 34.6 GiB 9 GB already failing

bosn's honest number still fits, so removing its waiver is a pure correction. soldr's shape is 2× over budget on its own — that is a cardinality/max measurement decision, and it is not solvable by any pre-prune edit.

Your point about ci-pre.yml holding actions: write but not being needs:-ordered first is the crux: ordering it correctly would invert that file's deliberate design ("a janitor never delays a build") on every run, without fixing an over-budget shape. Correct to decline.

GATE-005 untouched and correctly so — ci.toml only.

@zackees

zackees commented Oct 5, 2026

Copy link
Copy Markdown
Owner Author

One correction to my earlier framing, in the interest of not sending you after a dead end: I said adding the pre-prune step was "the smaller diff". For soldr that option is not currently expressible, and the reason is the one you correctly identified — ci-pre.yml holds actions: write but nothing needs: it, deliberately, so a janitor never delays a build. Wiring it in would invert that design on every run, and would not fix the over-budget shape regardless.

Filed the shared prerequisite as zackees/ci.yml#354, covering the three repos that hit this: soldr, bosn and running-process. It records that no repository can currently place a pre-prune ahead of its cache saves, and that this is a per-repository owner decision rather than something ci-lint should absorb.

This does not change the recommendation. soldr's real problem is not the waiver — compile is 15.6 GB steady against a 9 GB budget, so CACHE-004 fails with or without it. Removing the fiction makes an existing failure legible; it does not make it go away. The cardinality/max decision is the actual work, and this PR is the smaller half of it.

zackees added a commit that referenced this pull request Oct 5, 2026
`a07bab94f16124b5c6857b137a237a53a61e06d1` is not in zackees/ci.yml -- the
GitHub API returns 422 and the object is absent from the store, so no branch
or tag can recover it.

Introduced in this repository's first ci.toml and never valid. CT-002 did not
catch it because the SHA is well formed; a syntactically valid reference to a
nonexistent commit passes the shape check. It surfaced only when something
tried to RUN the gate: local-gate resolves the pin before executing any lane,
so the resolution failed first and no Local-Gate: trailer could be produced
(GATE-003 on every PR, which is why #3583 could not be stamped).

Repointed at 0fcff99. Verified with the rule that now enforces this --
CT-007 (zackees/ci.yml#355) reports 0 findings against this tree.

Gate isolation is untouched: [gate.isolation] in local-gate.toml keeps its
bosn runner, marker, guard, and proves-tree (GATE-005, GATE-009).

See #3584.
@zackees

zackees commented Oct 5, 2026

Copy link
Copy Markdown
Owner Author

The dead pin is fixed on this branch — e19d4b4 repoints linter at 0fcff997 (carries CACHE-034). See soldr#3584.

Verified with CT-007, the rule written for exactly this and merged as zackees/ci.yml#355: ci-lint audit --repo . now reports 0 CT-007 findings against this tree, where before it reported the 422. So the fix is confirmed by the enforcement mechanism rather than by inspection.

GATE-005 isolation is untouched — [gate.isolation] in local-gate.toml keeps its bosn runner, SOLDR_TEST_ISOLATED marker, nextest guard, and proves-tree (GATE-009).

Still not stampable — the host, not this change

local-gate run no longer fails at pin resolution, but it cannot start:

bosn: refusing to mint a new registry identity.
/tmp/bosn-soldr-pp/registry.sqlite3 is missing, but 15 Docker object(s)
already carry a Bosn registry label, so this machine is not a clean
first run.

That is the standing host-hygiene gap in zackees/ci.yml#322 — the same blocker bosn hit. bosn declines rather than mint a new identity that would orphan every labelled object, which is the correct conservative behaviour. Reclaiming them is a destructive operation on someone's machine and is the owner's call.

What this branch now contains

  1. e19d4b4 — repoint a pin that has never existed, which was making the gate unrunnable and GATE-003 fail on every PR.
  2. 04b008f — remove the unhonoured pre-prune waiver.

Worth restating the finding from the original review, since it is the substantive one and neither commit addresses it: soldr's CACHE-004 was already failing before either change. compile is declared max = "2600MB" with per = "platform" across 6 platforms = 15.6 GB steady against a 9 GB budget — over budget before any peak term. The waiver was not what made it pass. That needs a cardinality/max decision, the same wall zackees/clud#1800 hit.

Holding as a draft for the host reason alone. If you would rather land (1) on its own — it is what unblocks this repository's gate and precheck, independent of the cache question — say the word and I will split it out.

@zackees

zackees commented Oct 5, 2026

Copy link
Copy Markdown
Owner Author

Cross-reference: this PR's linter-pin repoint (a07bab94 → 0fcff997) is the core fix for #3584 (verified: the new SHA resolves). Merging this should close it — consider adding Fixes #3584 to the body. The other half of #3584 (making "must resolve" a violation rather than needs_review) lives in zackees/ci.yml's checker and stays open there. The [cache.family] sizing half of the audit is tracked separately in #3580, which will be edited to avoid conflicting with this PR's ci.toml hunks.

Local-Gate: v1 tree=eb14fb9ccdf35265dd53c7b5b584d18a1ecb4160 secs=1320 lanes=py-static:run,guards:run,ci-lint:run,rust:run,cross:run,wine:run,winvm:n/a,tests:run
Ci-Attestation: {"at":1791645117,"gate":"python/all/static","host":"linux-x86_64","key":"3ea285c960a67c2e4368e40a7b75e836df966c052ebee60f61fe1d4aacd87fb2","lane":"py-static","parents":["296c72ac78a9634d36b7e67957b06ca480169acb"],"secs":33,"stamp":"3b8aa950f06c42eb079d336598abef1b","tree":"eb14fb9ccdf35265dd53c7b5b584d18a1ecb4160","v":1,"via":"run"}
Ci-Attestation: {"at":1791645117,"gate":"python/all/guards","host":"linux-x86_64","key":"5f08ec1592089d3a0e1c31ffce7f226f726fd966d391e5b9cc01492040d65615","lane":"guards","parents":["296c72ac78a9634d36b7e67957b06ca480169acb"],"secs":56,"stamp":"819fbd9530b4db4d5b2d563d171ffeab","tree":"eb14fb9ccdf35265dd53c7b5b584d18a1ecb4160","v":1,"via":"run"}
Ci-Attestation: {"at":1791645117,"gate":"rust/all/deps","host":"linux-x86_64","key":"1b05eaf37543d0524ac8352951b72e85a12cf6df154ae4fc07cef1e8eeca8f56","lane":"ci-lint","parents":["296c72ac78a9634d36b7e67957b06ca480169acb"],"secs":3,"stamp":"a3f4e46edf6e18162ac5ed1d27cb6124","tree":"eb14fb9ccdf35265dd53c7b5b584d18a1ecb4160","v":1,"via":"run"}
Ci-Attestation: {"at":1791645117,"gate":"rust/all/fmt","host":"linux-x86_64","key":"82998775f55a1f2ce04803d843ffceb29fc1940ac02254ed628522e199c9f1ee","lane":"rust","parents":["296c72ac78a9634d36b7e67957b06ca480169acb"],"secs":252,"stamp":"caa3e66bfa575376350c51dcfdfd1c1e","tree":"eb14fb9ccdf35265dd53c7b5b584d18a1ecb4160","v":1,"via":"run"}
Ci-Attestation: {"at":1791645117,"gate":"rust/x86_64-unknown-linux-gnu/clippy","host":"linux-x86_64","key":"82998775f55a1f2ce04803d843ffceb29fc1940ac02254ed628522e199c9f1ee","lane":"rust","parents":["296c72ac78a9634d36b7e67957b06ca480169acb"],"secs":252,"stamp":"917f6c5141eb5069228eefc9f183a38d","tree":"eb14fb9ccdf35265dd53c7b5b584d18a1ecb4160","v":1,"via":"run"}
Ci-Attestation: {"at":1791645117,"gate":"rust/x86_64-unknown-linux-gnu/dylint","host":"linux-x86_64","key":"82998775f55a1f2ce04803d843ffceb29fc1940ac02254ed628522e199c9f1ee","lane":"rust","parents":["296c72ac78a9634d36b7e67957b06ca480169acb"],"secs":252,"stamp":"32f281341e9fdf28abe5876f6652bdd6","tree":"eb14fb9ccdf35265dd53c7b5b584d18a1ecb4160","v":1,"via":"run"}
Ci-Attestation: {"at":1791645117,"gate":"rust/x86_64-pc-windows-msvc/clippy","host":"linux-x86_64","key":"8d8a293b7de5f7d2b1a46ec28e8cdda412bfbf44241b26561679bf07e8674de3","lane":"cross","parents":["296c72ac78a9634d36b7e67957b06ca480169acb"],"secs":390,"stamp":"54d21a66990703c51f05f5cab34cb613","tree":"eb14fb9ccdf35265dd53c7b5b584d18a1ecb4160","v":1,"via":"run"}
Ci-Attestation: {"at":1791645117,"gate":"rust/x86_64-pc-windows-msvc/dylint","host":"linux-x86_64","key":"8d8a293b7de5f7d2b1a46ec28e8cdda412bfbf44241b26561679bf07e8674de3","lane":"cross","parents":["296c72ac78a9634d36b7e67957b06ca480169acb"],"secs":390,"stamp":"efe2433db177049d8ada95f8d9c62613","tree":"eb14fb9ccdf35265dd53c7b5b584d18a1ecb4160","v":1,"via":"run"}
Ci-Attestation: {"at":1791645117,"gate":"rust/aarch64-apple-darwin/clippy","host":"linux-x86_64","key":"8d8a293b7de5f7d2b1a46ec28e8cdda412bfbf44241b26561679bf07e8674de3","lane":"cross","parents":["296c72ac78a9634d36b7e67957b06ca480169acb"],"secs":390,"stamp":"bd4bdb559a327fbb041668d7757f1c99","tree":"eb14fb9ccdf35265dd53c7b5b584d18a1ecb4160","v":1,"via":"run"}
Ci-Attestation: {"at":1791645117,"gate":"rust/aarch64-apple-darwin/dylint","host":"linux-x86_64","key":"8d8a293b7de5f7d2b1a46ec28e8cdda412bfbf44241b26561679bf07e8674de3","lane":"cross","parents":["296c72ac78a9634d36b7e67957b06ca480169acb"],"secs":390,"stamp":"0d4dc57c8283e735d254ff455b7d5c9e","tree":"eb14fb9ccdf35265dd53c7b5b584d18a1ecb4160","v":1,"via":"run"}
Ci-Attestation: {"at":1791645117,"gate":"rust/x86_64-pc-windows-msvc/unit","host":"linux-x86_64","key":"b7b2e88792e4f78395be7409d3aaeb3dada3213f3e9f4075091e80a3a48c127f","lane":"wine","parents":["296c72ac78a9634d36b7e67957b06ca480169acb"],"secs":23,"stamp":"0de6189b30433762179a1d2f1f806e65","tree":"eb14fb9ccdf35265dd53c7b5b584d18a1ecb4160","v":1,"via":"run"}
Ci-Attestation: {"at":1791645117,"gate":"rust/x86_64-unknown-linux-gnu/test","host":"linux-x86_64","key":"7db82bbfd8c09fa11fae609ee7a9f2ca59023d73def6d2f7be2010cdf6e8c03c","lane":"tests","parents":["296c72ac78a9634d36b7e67957b06ca480169acb"],"secs":563,"stamp":"e5036d4094531da15610408acc8cea16","tree":"eb14fb9ccdf35265dd53c7b5b584d18a1ecb4160","v":1,"via":"run"}
@zackees
zackees marked this pull request as ready for review October 10, 2026 15:11
@zackees
zackees force-pushed the fix/cache-034-drop-unhonoured-pre-prune branch from e19d4b4 to 6bce4d0 Compare October 10, 2026 15:14
@zackees
zackees merged commit 15cd9e6 into main Oct 10, 2026
49 checks passed
@zackees
zackees deleted the fix/cache-034-drop-unhonoured-pre-prune branch October 10, 2026 15:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci(cache): ci.toml declares pre-prune = true but nothing runs a pre-prune (CACHE-034)

1 participant