ci: fast on branches, exhaustive on the develop → main door - #30
Closed
serialexperimentslainnnn wants to merge 4 commits into
Closed
serialexperimentslainnnn wants to merge 4 commits into
serialexperimentslainnnn wants to merge 4 commits into
Conversation
Three changes, all aimed at the same thing: the pipeline was spending its time on work nobody needed. DEVELOP NO LONGER REQUIRES BRANCHES TO BE UP TO DATE `strict_required_status_checks_policy` meant every merge into develop invalidated every other open PR, forcing each to update and re-run the entire suite. With two grouped Dependabot PRs that is annoying; with five it makes a release impractical, and the cost is paid on exactly the changes that least deserve scrutiny. What the setting guards against is real — two PRs that are green apart can break together — so it is not simply dropped. It is dropped where a second net exists: CI runs on every push to develop, so a semantic conflict is caught there, on develop, before anything reaches main. `main` KEEPS the strict policy: one merge per release, and it is the merge that publishes. THE VERIFIER RUNS WHERE THE ANSWER MATTERS ~10 minutes and 1.25 GB of IDE downloads, previously on every push to every topic branch. Now on pull requests and on the protected branches. It remains a required check, so nothing merges without it. The loss is early detection mid-branch, which is a genuine cost rather than free savings. TOPIC BRANCHES WRITE THEIR OWN CACHE Read-only was blanket-applied to everything but develop and main, so a topic branch restored the shared cache and saved nothing — every push re-downloaded what the previous one had already fetched. Read-only now applies to pull requests from forks only. Verified against GitHub's cache documentation rather than assumed: "Workflow runs cannot restore caches created for child branches or sibling branches", and a cache created on a pull request is written to the merge ref and "can only be restored by re-runs of the pull request". A topic branch cannot reach what develop reads back; the isolation is the platform's, not ours. NB the ruleset change needs ./scripts/apply-rulesets.sh to take effect.
~10 minutes and 1.25 GB of IDE downloads per run, previously paid on every iteration of a branch nobody was about to merge. It now runs only on pushes to develop and main — not on topic branches, not on pull requests. This REQUIRED dropping "Plugin verifier" and "Build plugin" from the required checks on both rulesets, and that is not a detail: a required check whose job never runs is never reported, so the pull request would wait forever. The two changes have to move together or the branch becomes unmergeable. What still covers a release: release.yml re-runs the FULL gate — verifier included — on the exact tree being published, behind the environment approval, so nothing reaches the Marketplace unverified. What is genuinely lost is catching a binary incompatibility at pull-request time rather than after the merge into develop. A real regression in feedback latency, accepted deliberately. Dependabot also moves to monthly, and majors are no longer bot-proposed in any ecosystem: they are where a bump actually breaks something and where CI is not enough on its own — the platform-plugin 2.16 -> 2.18 attempt passed CI and hung the headless suite locally, inside the platform's own test fixture. Security updates are unaffected by either change.
Corrects the previous commit, which moved the verifier in the wrong direction. It ran it only on PUSHES to develop and main and dropped it from main's required checks — so a pull request from develop into main, the merge that publishes, would not have run it at all. That is precisely the door that has to be guarded. The policy now matches the intent: topic branches and pull requests into develop run the fast checks (compile, our own JVM and frontend suites, static analysis, dependency audit) and iterate quickly. Any pull request targeting main also runs the plugin verifier and the artifact assertions, both required again on main alongside the two CodeQL analyses. The verifier is the only thing that catches a BINARY incompatibility across the declared 251 -> 262 range — compiling against 252 proves nothing about 262, which is exactly how the 4.4.1 /login regression shipped. Skipping it on a branch is a latency trade; skipping it on the way to a release would not be.
Branches now run JVM tests and frontend tests and nothing else. Static analysis
and the dependency audit join the plugin verifier behind the develop -> main
door, where the exhaustive gate belongs.
Required checks match, because they have to: a check whose job does not run is
never reported and would block the pull request forever. develop requires the two
suites; main requires all eight.
The costs, named rather than discovered later:
- A formatting, detekt, ESLint or coverage failure now lands ON develop and is
fixed by a follow-up commit, instead of being caught in the pull request. That
happened today with spotlessKotlinGradleCheck, and the PR is what caught it.
- The dependency audit no longer runs on the Dependabot PR that proposes a bump.
It runs once that bump is on develop, and again before it can reach main, so
nothing ships un-audited — the finding just arrives one merge later.
What this buys is the thing that was actually hurting: a branch iterates in about
three minutes instead of thirteen, and nothing is promoted to main without the
full gate, plus release.yml re-running all of it on the exact published tree.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Makes the pipeline fast where it iterates and exhaustive where it promotes.
The gate, after this PR
developdevelop→mainA branch iterates in ~3 minutes instead of ~13. Nothing reaches
mainwithout all eight, andrelease.ymlthen re-runs the whole gate on the exact tree it publishes, behind the environment approval.Required checks move with the jobs, because they have to: a required check whose job never runs is
never reported, and the pull request would wait forever. Both rulesets are updated and already applied.
Duplicated work removed
Measured, not assumed:
pushandpull_requestboth fire, and the concurrency group keyed ongithub.refdiffers between them, so neither cancelled the othercancel-in-progressapplied only to pull requestskoverVerifydepends on:testand lived in a separate jobverifyPluginalready produces the distributableThe last one was also subtly wrong: the artifact assertions ran against bytes that were never verified.
They now run on the verified artifact.
developno longer requires branches to be up to datestrict_required_status_checks_policymeant every merge intodevelopinvalidated every other open PRand forced a full re-run. With grouped Dependabot PRs that made releasing impractical.
It is dropped only where a second net exists: CI runs on every push to
develop, so a semantic conflictbetween two separately-green PRs is caught there, before anything reaches
main.mainkeeps thestrict policy — one merge per release, and it is the merge that publishes.
Caches
Read-only now applies to pull requests from forks only; every other branch writes its own cache.
Verified against GitHub's documentation rather than assumed: "Workflow runs cannot restore caches created
for child branches or sibling branches", and a cache created on a pull request is written to the merge ref
and "can only be restored by re-runs of the pull request". A topic branch cannot reach what
developreads back — the isolation is the platform's, not ours.
Dependabot
Monthly instead of weekly, grouped per ecosystem, with an explicit
applies-to: security-updatesgroup —security updates ignore
open-pull-requests-limitentirely and were not covered by the existing group,which is how five arrived at once under a limit of three.
Majors are no longer bot-proposed in any ecosystem. They are where a bump actually breaks something and
where CI is not sufficient on its own: the platform-plugin 2.16 → 2.18 attempt passed CI and hung the
headless suite locally, inside the platform's own test fixture.
Costs, stated rather than left to be discovered
developand is fixed by a follow-upcommit, instead of being caught in the pull request. That happened today with
spotlessKotlinGradleCheck,and the PR is what caught it. The plan is to run these locally before pushing.
on
develop, and again before it can reachmain. Nothing ships un-audited; the finding arrives onemerge later.
developrather than at PR time.Still open
develop.jsonallows["squash", "merge"]. Squash is disabled at repository level so it cannot be usedtoday, but the two layers disagree — and a squash destroys the author's hardware-backed signatures.