feat(pipeline): let a stage declare it consumes no sibling's output - #60
Merged
Conversation
The stage graph is fixed on purpose, and stays fixed. But *dependency* is
a claim about the repository, and a repository can be right that it does
not have one.
pipeline:
ci:
stages: [preflight, build, test, end2end]
independent_stages: [end2end]
A stage named here drops its sibling dependencies and depends on
preflight alone. That is all it can do — it cannot add a dependency,
invent an edge, or change the order stages render in. Removing an edge
you do not have is not rearranging the pipeline.
preflight is never dropped, because it produces no artefact: it is the
gate deciding whether a stage runs at all, and the rendered `if:` reads
its outputs, so losing it from `needs:` would leave a condition that can
never be true. `preflight` itself cannot be declared independent, having
no sibling dependency to drop; saying so is rejected rather than ignored.
Detaching a stage does NOT remove it from ci-gate. A failing end2end
still blocks the merge; only its start time moves. There is a test
pinning that, because acquiring a silent hole in branch protection as a
side effect of a performance tweak is the worst way this could go wrong.
wardnet is the case it was added for. Its end2end suite rebuilds the
daemon from source inside Docker and consumes nothing the build stage
produces — its previous hand-rolled pipeline ran the two concurrently.
Serialising them added about eleven minutes to every daemon pull request,
24 to 35, to prove an edge that does not exist.
The field is omitempty and default rendering is untouched: rendering
every managed file on v1.5.0 and on this branch and diffing leaves them
byte-identical, comments included. Unlike #58 there is no fleet-wide
comment churn, and gt's own repo stays compliant without a re-render.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The stage graph is fixed on purpose, and stays fixed here. But dependency is a claim about the repository, and a repository can be right that it doesn't have one.
wardnet is the case. Its end2end suite rebuilds the daemon from source inside Docker, so it consumes nothing the build stage produces — its hand-rolled
pr.ymlran the two concurrently (tests-e2ehadneeds: preflightonly). gt's wiring gates end2end on build, so migrating it as-is serialises them:Eleven minutes added to every daemon PR to prove an edge that does not exist. Measured from run 32688637246, not estimated.
What
Renders:
What it deliberately cannot do
This is a statement about the repository, not a reordering knob. A stage named here drops its sibling dependencies and depends on preflight alone. It cannot add a dependency, invent an edge, or change the order stages render in. Removing an edge you don't have isn't rearranging the pipeline, so
## Shape's "a central pipeline every repo can rearrange is not a central pipeline" survives intact.Two constraints fall out of that and are enforced rather than documented:
if:reads its outputs. Losing it fromneeds:would leave a condition that can never be true.preflightcannot be declared independent. It has no sibling dependency to drop, so saying so is rejected rather than silently ignored. Same for a stage that isn't enabled, a duplicate, and any declaration while the pipeline is disabled.The test I'd most want reviewed
TestIndependentStageIsStillGated. Detaching a stage must not drop it fromci-gate— a failing end2end still has to block the merge. Acquiring a silent hole in branch protection as a side effect of a performance tweak is the worst way this could go wrong, so it's pinned.Also pinned:
TestStagesKeepTheirDependenciesByDefault, so this can't quietly reshape the sixteen repos that don't use it.Backwards compatibility
omitempty, and default rendering is untouched — verified by rendering every managed file on v1.5.0 (bc668c9) and on this branch and diffing: byte-identical, comments included.Unlike #58 there's no fleet-wide comment churn, and gt's own repo stays compliant with no re-render (
gt repo check→ "All 8 managed file(s) match").go test ./...,go vet ./...,gofmtclean.Next
Needs shipping as v1.6.0 with
v1moved before wardnet's onboarding PR can land.