standard.mk: gates should accumulate, not abort — one red gate hides every gate behind it

[hunterdsp]

gates aborts at the first failing prerequisite, so one red gate hides every gate ordered behind it. The set that claims "every gate that guards a merge" reports one verdict per run instead of all of them.

The shape is already settled elsewhere in this file

docs-check solves exactly this, one level down, and says so in its own comment:

The docs gate: every check runs, every failure is reported in one pass.

It accumulates across DOCS_CHECK_PRE_CMDS, runs the build, then check_site_links, and reports the lot. gates is the same problem one level up and does the opposite.

What it cost, concretely

Measured in doppler this week, on a branch being prepared for merge.

GATES_DEPS there is:

lint changelog-check drift-check doxygen-check docs-check validate-check
test-all test-stubs test-api-docs test-snippets test-rust
abi-check link-check consumer-faces-check glibc-check
specan-check check-isotime-parity coverage coverage-gate docker-examples

glibc-check sits mid-list and cannot pass on a modern dev box by design — its own comment says so:

Only meaningful against a build made on that glibc — CI runs this in a Debian 10 container, and running it on a modern distro will fail on the local build's newer symbols, which is the check working, not a bug.

So on any dev machine make gates stops there, and the six targets behind it — specan-check, check-isotime-parity, coverage, coverage-gate, docker-examples — are structurally unreachable. Not slow, not flaky: never run.

That is worse than an ordinary abort because the failure reads as expected. A developer sees a gate whose comment says failing here is correct, shrugs, and never learns that a third of the set did not execute.

What was hiding behind it: make coverage was broken on that machine for a missing package (compiler-rt, which clang does not depend on, so -fprofile-instr-generate had no runtime and every link failed). Stepping over glibc-check with a GLIBC_MAX override is what finally surfaced it. With -k it would have been reported on the first run, next to the glibc red, weeks earlier.

Why a consumer cannot fix this locally

The contract in this file's own header:

Per-repo variation is expressed as CONFIGURATION — the variables below, set in the repo's own Makefile before include standard.mk — never as a local edit, because a local edit is the fork this exists to prevent.

and standard-check enforces it. local.mk "may only ADD targets", so it cannot redefine gates either. A consumer's only options are to remember make -k gates every time, or to add a differently-named target — which splits the SSOT that gates-check exists to protect.

So it has to change here.

Suggested change

Make gates re-invoke itself over its dependency list with -k, so the accumulate-don't-abort behaviour is the default and no one has to remember a flag:

gates: ## Run every gate that guards a merge
	@$(MAKE) -k $(GATES_DEPS)
	@echo ""
	@echo "gates: ALL PASS"

Properties worth checking against, since they are what makes the current form safe:

• a failing gate still fails the run (recursive make propagates the status), so this is not a weakening;
• gates: ALL PASS still cannot print unless everything passed;
• gates-check is unaffected — it walks the closure from gates over make's database, and $(GATES_DEPS) is still the list.

The one behaviour change is that $(GATES_DEPS) becomes a recursive invocation rather than a prerequisite list, which affects how a -j run schedules them. Worth confirming against a repo that runs gates in parallel before merging.

Related, not proposed here

glibc-check failing by design on every dev box is its own question — it could derive whether the build is old-glibc and skip with a printed reason, leaving CI's Debian 10 container to enforce it for real. That would be a genuine fix rather than a workaround. But it is separable: with -k, the gate can stay honestly red without taking six others down with it, which is the part that matters for this issue.

Filed from doppler 0.42.0.

[hunterdsp]

Fixed and published — #19, on main and served by Pages.

Reproduced as filed: with a red gate mid-list, the two behind it never ran. All three properties the issue asked to be checked against hold.

One correction, because it would have broken every adopter. gates-check is not unaffected. It builds its closure by walking prerequisites out of make's database:

for p in $(sed -n "s/^$$t:[ ]*//p" "$$db" | sed 's/|.*//'); do

Move $(GATES_DEPS) into the recipe and that edge is gone, so the closure comes back empty and every CI target reports as uncovered. gates-check hangs off lint, so the patch as proposed turns make lint red in every subscriber on the next fetch. Measured in a fixture, not reasoned from the source.

#19 therefore also seeds the closure from $(GATES_DEPS) itself, which makes the check independent of how gates invokes its list — what it was always asserting.

Also resolved from the open questions:

• -j: make -j4 gates runs every gate and still exits non-zero. The -k re-invocation keeps them schedulable in parallel; a shell loop over the list would have serialised them, which matters where gates include coverage and docker.
• Noise: --no-print-directory added, or every run gains Entering directory/Leaving directory.

A new CI step asserts all four properties, sabotage-checked both ways: reverting to prerequisites fails on behind1 never ran; applying -k without the closure seeding fails on gates-check cannot see GATES_DEPS any more.

Adopters must re-vendor

standard-check fetches canonical on every make lint with no cache, so this is live now:

• just-makeit — chore: re-vendor standard.mk — gates accumulate instead of aborting just-makeit#933, lint green, auto-merge armed.
• doppler — confirmed red as of this comment (standard-check exits 1 against the new canonical). Needs curl -fsSL -o standard.mk https://just-buildit.github.io/standard.mk and a commit; left for that repo's own lane rather than done from here.

glibc-check failing by design on a dev box is untouched, per the issue — with -k it can now stay honestly red without taking six gates with it.