Skip to content

perf(scripts): run the script tests beside ci and skip repeated hooks - #325

Merged
possiblyneal merged 3 commits into
mainfrom
possiblyneal/speed-up-check-scripts
Nov 19, 2026
Merged

possiblyneal merged 3 commits into
mainfrom
possiblyneal/speed-up-check-scripts

Conversation

@possiblyneal

@possiblyneal possiblyneal commented Nov 19, 2026 •

Copy link
Copy Markdown
Owner

Intent

Operator request in session: make scripts/check and its test runs faster.

scripts/check ran script-tests, then ci, then a pre-commit sweep that repeated structure and payload-parity (and, here, addon-adoption and reference-assertions, which ci's pytest already collects). The gate now runs script-tests in the background beside ci, holds the suites' output until ci finishes, fails on either status, and skips the repeated hooks in the sweep. The repo-builder pytest suite now isolates git from the operator's global config, so a global core.hooksPath attribution hook no longer runs on every fixture commit.

Read first

  • scripts/check:93 — background launch, traps, and the two-status join
  • scripts/check:143 — the pre-commit SKIP list
  • apps/repo-builder/src/scripts/tests/conftest.py — git config isolation
  • CLAUDE.md and the payload CLAUDE.md Commands line — scripts/check order now says script tests run beside ci
  • Mechanical: apps/github-repository-template/src/base-repo/scripts/{check,tests/check-test} are byte-identical mirrors (payload-parity holds them)

Decisions

  • Suites beside ci rather than inside xargs -P with it: both mostly wait on git and subprocesses, so a second job costs almost no wall time, and their outputs stay separate.
  • trap 'exit 130' INT / 'exit 143' TERM plus a process-group kill: a background job in a non-interactive shell ignores SIGINT, so without them Ctrl-C left the gate blocked in wait. set -m launches the suites in their own group, so kill -- -PGID reaches a suite started mid-shutdown with no pgrep dependency. Being its own group, it misses a TERM sent to the gate's group (timeout), so trap 'exit 141' PIPE catches the gate's write to the tee that signal killed and still runs the EXIT trap. A group SIGKILL leaves the suites to finish on their own. After wait the EXIT trap drops the kill, since the reaped PID may be reused.
  • Pre-commit hooks are skipped by id, not removed: they still run at commit time, the only other place anything runs them. Naming a hook a repository lacks is a no-op, so the payload ships the same list.
  • No fixture caching for pytest: the slowest fixtures measured 0.10–0.23 s, too little to repay the complexity.

Out of scope

  • Mirroring conftest.py into the payload: generated repositories carry no repo-builder unit.

Verified

  • scripts/check → exited 0: 0 FAIL, 0 unavailable, 5 not-applicable, 26 pass
  • scripts/tests/check-test → 10 passed, including the new "a failing script test fails the gate after ci has run" (exit 3, ci output shown, security never runs)
  • A/B under load 10–17: old gate 36–50 s, new 32–34 s
  • Per-commit cost in pytest fixtures: 51 ms with the global hook, 3 ms isolated
  • timeout 4..14 scripts/check, 6 runs: no gate process left in the worktree
  • Ctrl-C on a running gate (via setsid, signalling the process group) exits at once and leaves no script-tests children

Not verified

  • Idle-machine timing for the new gate (estimated ~28 s against a ~41 s idle baseline, not measured)
  • CI on this pull request had not run when this was written

Merge danger

Door: two-way: a revert restores everything

Blast radius: scripts/check in this repository and in every repository generated or updated after merge; scripts/tests/check-test; repo-builder's pytest run.

neal added 3 commits November 18, 2026 22:34
scripts/check ran the script suites, then ci, then a pre-commit sweep
that re-ran structure and payload-parity, which the gate had already
run over the whole tree. The suites and ci both wait mostly on git and
child processes, so running them together takes about as long as ci
alone. The suites' output is held until ci ends so the two never
interleave, both statuses must pass, and an interrupt exits at once
and stops the suites rather than waiting on them.

The pre-commit sweep now skips the hooks that repeat a gate step:
structure, payload-parity, and the template's addon-adoption and
reference-assertions, which ci's pytest already collects. They still
run at commit time.

repo-builder's pytest suite gains a conftest that points git's global
and system config at /dev/null, as the shell harness already does. A
global core.hooksPath attribution hook otherwise ran on every fixture
commit and took about a sixth of the suite's time.

Generated-By: claude-opus-5-5
Review found the recursive pgrep walk could miss a suite started between
the walk and the kill, and it leaned on pgrep, which doctor does not
require. set -m puts the suites in a process group of their own, so one
kill reaches every member, in pure bash.

The Commands contract lines in both CLAUDE.md copies still described the
old order, script tests first; they now say the tests run beside ci.

Generated-By: claude-opus-5-5
In their own process group, the background script tests miss a TERM
sent to the gate's group, as timeout and agent harnesses send it. That
signal also kills tee, so the gate's next write raised SIGPIPE, which
exited without running the EXIT trap and left the suites running.
Trapping PIPE routes that exit through the trap. A group SIGKILL still
leaves them to finish on their own, as no trap can catch it.

Generated-By: claude-opus-5-5
@possiblyneal possiblyneal added the code-reviewed Multi-axis code review passed label Nov 19, 2026
@possiblyneal

Copy link
Copy Markdown
Owner Author

Code review: standard scan, clean.

Axes: Standards+Spec, Correctness/Logic

Reviewed-Head: fb5ef63

@possiblyneal
possiblyneal merged commit 83ece89 into main Nov 19, 2026
7 checks passed
@possiblyneal
possiblyneal deleted the possiblyneal/speed-up-check-scripts branch November 19, 2026 03:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

code-reviewed Multi-axis code review passed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant