Skip to content

build: add a single-binary test target - #798

Open
rustytrees wants to merge 1 commit into
indaco:mainfrom
rustytrees:chore/single-binary-test-target
Open

build: add a single-binary test target#798
rustytrees wants to merge 1 commit into
indaco:mainfrom
rustytrees:chore/single-binary-test-target

Conversation

@rustytrees

@rustytrees rustytrees commented Jul 29, 2026

Copy link
Copy Markdown

Additive build target. zig build test is untouched.

CONTRIBUTING asks for an issue first on anything non-trivial — this is a judgement call about build ergonomics rather than a bug, so I've put the proposal in the PR where the diff is visible. Happy to close it and move the discussion to an issue if you'd rather not review code for something you haven't agreed to in principle.

What and why

zig build test compiles one executable per file under tests/ — 162 of them — and runs 162 fresh processes. Two costs:

  • Link time, paid 162 times.
  • On a host with a per-binary firewall or endpoint agent, a prompt storm on every run, because each rebuild is an executable the agent has never seen. That is what prompted this; it made the suite genuinely unpleasant to run locally.

zig build test-one links and runs exactly one binary over the same files: 2591 passed, 3 skipped, 0 failed in ~2m20s.

zig build test stays authoritative and unchanged. One process per file is what actually isolates process-global state, and CI should keep using it — test-one is a local convenience and a second signal, not a replacement. tests/all.zig says so in its header.

It found something

Sharing a process surfaced one latent order dependency, fixed here rather than worked around: custom_theme_test's "valid themes file" case asserts on captured CLI output, but emitPrefixLine early-returns while the module-global quiet flag is set. In its own binary the flag happens to be clear; run after a file that leaves setQuiet(true) behind, the capture is empty and the assertion compares against nothing. boot() already pins the colour state for precisely this reason — quiet was the one input it missed.

Worth saying plainly: the per-file layout hides this class of bug, and just test-concurrent covers shared fixture paths, not shared in-process globals. So this target also guards an axis nothing currently checks. I would expect it to find more of these if it runs regularly.

The honest downside

162 files in one process means shared globals, and I only know of the one above. If a future test leaks a global, test-one will fail while zig build test stays green — confusing unless you know why. Two ways to read that: as a maintenance cost, or as the target doing its job. I lean the second, but it is your call, and it is the reason I did not touch the default.

tests/all.zig duplicates the file list from build.zig's test_modules, so a new test file needs adding in two places. I considered generating it at build time, but a generated root in the cache cannot resolve @import("foo_test.zig") relative to tests/. If drift worries you, a no_*_test-style guard comparing the two lists would fit the existing patterns in tests/ — say the word and I'll add it.

Verification

  • zig build test-one — 2591 pass, 3 skip, 0 fail, one binary
  • zig build test — still green, unchanged

Hope it helps decide the knowledge base.

`zig build test` compiles one executable per file under `tests/` — 162 of
them — and runs 162 fresh processes. That is slow to link, and on a host with
a per-binary firewall or endpoint agent it produces a prompt storm per run,
because every rebuild is a new executable the agent has not seen.

`zig build test-one` links and runs exactly one binary over the same files:
2591 passed, 3 skipped, 0 failed in ~2m20s. `zig build test` is untouched and
stays authoritative — one process per file is what actually isolates
process-global state, and CI should keep using it.

Sharing a process immediately surfaced one latent order dependency, which is
fixed here rather than papered over: `custom_theme_test`'s "valid themes file"
case asserts on captured CLI output, but `emitPrefixLine` early-returns while
the module-global quiet flag is set. Run in its own binary the flag happens to
be clear; run after a file that leaves `setQuiet(true)` behind, the capture is
empty and the assertion compares against nothing. `boot()` already pins the
colour state for exactly this reason — quiet was the one input it missed.

That is worth noting on its own: the per-file layout hides this class of bug,
and `just test-concurrent` only covers shared *fixture paths*, not shared
in-process globals. This target is a cheap way to keep the second axis honest.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant