From 4924ffcdb4d7059087a625d4ff813178a6271cae Mon Sep 17 00:00:00 2001 From: neal Date: Wed, 18 Nov 2026 22:52:30 -0500 Subject: [PATCH 1/4] build(scripts): add scripts/check --fast, which leaves the build to CI Every change ran the whole gate at commit time and again in the pull request's CI, on the same self-hosted machine, so the build step ran twice for one tree. Across 2,086 gate runs in the agent transcripts from Sep 26 to Nov 18, build was the only failing step 8 times, and none of those was a defect in the change being committed: all were a known failure already on main or a worktree missing its node_modules. The step cost about 9.4 hours of waiting over those 53 days, mostly devex (61s) and glydr (40s). --fast runs everything else and passes --no-build to scripts/ci, the one place the capability list lives. The full gate stays the default, so CI and the land-time verify still build before anything reaches main. Generated-By: claude-opus-5-5 --- CLAUDE.md | 4 +- .../src/base-repo/CLAUDE.md | 2 +- .../src/base-repo/scripts/check | 43 +++++++++++---- .../src/base-repo/scripts/ci | 15 +++++- .../base-repo/scripts/tests/capabilities-test | 22 ++++++++ .../src/base-repo/scripts/tests/check-test | 53 ++++++++++++++++++- scripts/check | 43 +++++++++++---- scripts/ci | 15 +++++- scripts/tests/capabilities-test | 22 ++++++++ scripts/tests/check-test | 53 ++++++++++++++++++- 10 files changed, 245 insertions(+), 27 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 15ab1f71..d783a3cf 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -32,7 +32,7 @@ Use these instead of per-language tools; each detects the languages present and - `scripts/doctor` — verify local toolchains, dependencies, hooks, and configuration without contacting hosted services - `scripts/repo-settings check` — inspect GitHub-hosted security and branch settings; run explicitly, since it needs the network and administration visibility -- `scripts/check` — the full local gate: `doctor`, the script tests, lint, format, type check, test, build, the security audit, and pre-commit over every file, tracked and untracked. It prints only the Result table and always writes the full run to `tmp/check.log`; `--verbose` streams everything. It reads no hosted GitHub state but is not offline: `commitlint-test` and pre-commit's hook environments fetch on first use +- `scripts/check` — the full local gate: `doctor`, the script tests, lint, format, type check, test, build, the security audit, and pre-commit over every file, tracked and untracked. It prints only the Result table and always writes the full run to `tmp/check.log`; `--verbose` streams everything; `--fast` leaves out `build`, which CI runs on the pull request. It reads no hosted GitHub state but is not offline: `commitlint-test` and pre-commit's hook environments fetch on first use - `scripts/fix` — the write half of `check`: lint autofix, then formatting, for every stack, unconditionally and in that order so the formatter lays out what the linter rewrote. Go, Swift and Kotlin have no lint autofix and report `lint-fix` `not-applicable`. A `lint-fix` FAIL means findings remain that need hand work - `scripts/summarize ` — run a check and print only its Result table with findings, then a verdict line of its exit status and a count per state, exiting with that status; read `ci` and other commands through this, since `check` already does - `scripts/clean` — recursively delete build output and tool caches (`dist`, `build`, `coverage`, `__pycache__`, `.*_cache`, `*.pyc`) @@ -53,7 +53,7 @@ The only non-shell code here is the Python under `apps/repo-builder/`, declared ## Git - Pre-commit blocks commits to `main` and `master`; branch first. -- Run `scripts/check` before committing. It runs CI's checks plus pre-commit over every file, but with the machine's tool versions rather than CI's fresh installs, so a lint can pass here and fail there; `scripts/CLAUDE.md` carries the rule. +- Run `scripts/check --fast` before committing, and the whole `scripts/check` when the change could break the build. It runs CI's checks plus pre-commit over every file, `--fast` leaving the build to CI, but with the machine's tool versions rather than CI's fresh installs, so a lint can pass here and fail there; `scripts/CLAUDE.md` carries the rule. - These prompt for approval: `git reset --hard`, `git clean`, `git rebase`, `rm`, `git rm`, and the `gh` commands that merge pull requests, cut releases, or delete the repository. Edits do not: this repository's `.claude/settings.json` allows `Edit(/**)`, anchored at the session's starting directory, so a session started at the checkout root edits or writes any file without a prompt, root files and dotfiles included. Claude Code's built-in protected paths, such as `.claude/`, `.git/`, `.devcontainer/`, and `.pre-commit-config.yaml`, stay gated whatever an allow rule says, and so do the files the operator's global `ask` rules name, such as `.env`, `*.pem`, and `secrets/`, since an `ask` outranks an `allow`. - Work reaches `main` through a pull request under `.github/PULL_REQUEST_TEMPLATE.md`; a ruleset enforces it (Repository settings below). - Commit messages follow [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/#specification): `type(optional scope): subject`, a blank line, then the body. commitlint enforces it at `commit-msg`, with rules in `.commitlintrc.yaml`. The subject is lowercase after the type, with no trailing period. A body is expected: it is the only surviving record of why. diff --git a/apps/github-repository-template/src/base-repo/CLAUDE.md b/apps/github-repository-template/src/base-repo/CLAUDE.md index ad89aeb6..06adf8b5 100644 --- a/apps/github-repository-template/src/base-repo/CLAUDE.md +++ b/apps/github-repository-template/src/base-repo/CLAUDE.md @@ -6,7 +6,7 @@ These sections are identical in every repository built from the template. Do not Use these instead of per-language tools. `check` and `fix` run for every language present and report `pass`, `not-applicable`, `unavailable`, or `FAIL`. -- `scripts/check` — the full local gate, a superset of CI except the pull request's changelog check: doctor, script tests, structure, lint, format, type check, test, build, security audit, pre-commit over every file. Prints only the Result table, with the full run in `tmp/check.log`; `--verbose` streams everything +- `scripts/check` — the full local gate, a superset of CI except the pull request's changelog check: doctor, script tests, structure, lint, format, type check, test, build, security audit, pre-commit over every file. Prints only the Result table, with the full run in `tmp/check.log`; `--verbose` streams everything; `--fast`, the commit-time gate, leaves `build` to CI - `scripts/fix` — lint autofix, then formatting - `scripts/summarize ` — run a check and print only its Result table, then a verdict line of its exit status and a count per state; read `ci` and other commands through this, since `check` already does - `scripts/run [unit] [--entry ] [-- args…]` — start a unit; the unit name is required when `apps/` holds several diff --git a/apps/github-repository-template/src/base-repo/scripts/check b/apps/github-repository-template/src/base-repo/scripts/check index 867e76c5..55524945 100755 --- a/apps/github-repository-template/src/base-repo/scripts/check +++ b/apps/github-repository-template/src/base-repo/scripts/check @@ -20,23 +20,44 @@ cd "$repo_root" # the cause is gone; the log is what a failure outside the table is diagnosed # from. --verbose streams the same text live. --gate is the internal entry that # runs the steps below and nothing else, so the tee wraps exactly one run. -case "${1:-}" in +# +# --fast is the commit-time gate: every step below except build, which CI runs +# on the same commit before it can land and which costs more than the rest of +# the gate together in a repository that compiles. It rides through each entry, +# and tmp/check.log opens by saying build was left out, so a fast run's log +# never reads as the full gate's. +usage() { + echo "usage: scripts/check [--fast] [--verbose]" >&2 + exit 2 +} +mode="" +fast=() +for arg in "$@"; do + case "$arg" in + --verbose | --gate) + [[ -z "$mode" ]] || usage + mode="$arg" + ;; + --fast) fast=(--fast) ;; + *) usage ;; + esac +done +case "$mode" in "") - exec scripts/summarize scripts/check --verbose + exec scripts/summarize scripts/check --verbose "${fast[@]}" ;; --verbose) mkdir -p tmp # pipefail (set above) makes the pipeline's status the gate's, not tee's. - scripts/check --gate 2>&1 | tee tmp/check.log + scripts/check --gate "${fast[@]}" 2>&1 | tee tmp/check.log exit "${PIPESTATUS[0]}" ;; - --gate) ;; - *) - echo "usage: scripts/check [--verbose]" >&2 - exit 2 - ;; esac +if (( ${#fast[@]} )); then + echo "scripts/check --fast: build is left to CI." +fi + if [[ ! -x scripts/doctor ]]; then echo "scripts/doctor is missing or not executable." exit 1 @@ -78,7 +99,11 @@ if [[ ! -x scripts/ci ]]; then exit 1 fi -scripts/ci +if (( ${#fast[@]} )); then + scripts/ci --no-build +else + scripts/ci +fi # The security workflow blocks a pull request the same way the CI workflow does, # so leaving it out here means finding out on the pull request instead of before diff --git a/apps/github-repository-template/src/base-repo/scripts/ci b/apps/github-repository-template/src/base-repo/scripts/ci index 8cfae084..eec43ab3 100755 --- a/apps/github-repository-template/src/base-repo/scripts/ci +++ b/apps/github-repository-template/src/base-repo/scripts/ci @@ -8,6 +8,19 @@ cd "$repo_root" # shellcheck source=libs/detect.sh source "$repo_root/scripts/libs/detect.sh" +# --no-build leaves build out, for `scripts/check --fast`: the commit-time gate +# drops the slowest check CI repeats on the same commit before it can land. +# CI itself never passes it. +capabilities=(lint format-check typecheck test build) +case "${1:-}" in + "") ;; + --no-build) capabilities=(lint format-check typecheck test) ;; + *) + echo "usage: scripts/ci [--no-build]" >&2 + exit 2 + ;; +esac + # The shared policy: orphan search before the checks, which would run green over # the package none of them sees; the no-manifest early-out; the dry run. -capability_command check lint format-check typecheck test build +capability_command check "${capabilities[@]}" diff --git a/apps/github-repository-template/src/base-repo/scripts/tests/capabilities-test b/apps/github-repository-template/src/base-repo/scripts/tests/capabilities-test index 6cd66cde..286e527d 100755 --- a/apps/github-repository-template/src/base-repo/scripts/tests/capabilities-test +++ b/apps/github-repository-template/src/base-repo/scripts/tests/capabilities-test @@ -167,6 +167,28 @@ out="$(ci)" assert_dispatches python "$out" assert_absent node "$out" +fixture "--no-build leaves build out and nothing else" +printf '[project]\nname = "p"\nversion = "0.1.0"\n' > "$work/pyproject.toml" +out="$(cd "$work" && CI_DRY_RUN=1 scripts/ci --no-build 2>&1 /dev/null 2>&1 "$work/package.json" out="$(ci)" diff --git a/apps/github-repository-template/src/base-repo/scripts/tests/check-test b/apps/github-repository-template/src/base-repo/scripts/tests/check-test index ff52a920..e3be1ddd 100755 --- a/apps/github-repository-template/src/base-repo/scripts/tests/check-test +++ b/apps/github-repository-template/src/base-repo/scripts/tests/check-test @@ -4,8 +4,8 @@ set -uo pipefail # What scripts/check prints, writes, and exits with. The gate's steps are # stubbed in a scratch copy of the scripts, so the cases assert the wrapper's # contract -- table only by default, the whole run in tmp/check.log every time, -# raw output under --verbose, the gate's own exit status -- and not the output -# of any real check. +# raw output under --verbose, the gate's own exit status, --fast handing ci +# --no-build -- and not the output of any real check. repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" @@ -115,6 +115,55 @@ else fail "wanted the table and exit 0 from a subdirectory, got exit $status: $out" fi +# ci records the arguments it was given, so a case can tell the full gate from +# the fast one without running a real build. +ci_args_fixture() { + gate_fixture "$1" + printf '#!/usr/bin/env bash\necho "ci:$*" > ci-args\necho "lint pass python"\n' \ + > "$work/scripts/ci" +} + +ci_args_fixture "the default gate runs ci whole" +(cd "$work" && scripts/check >/dev/null 2>&1) +if [[ "$(cat "$work/ci-args" 2>&1)" == "ci:" ]]; then + pass +else + fail "wanted ci with no arguments, got: $(cat "$work/ci-args" 2>&1)" +fi + +ci_args_fixture "--fast leaves build to CI and says so in the log" +out="$(cd "$work" && scripts/check --fast 2>&1)" +status=$? +if (( status == 0 )) \ + && [[ "$(cat "$work/ci-args" 2>&1)" == "ci:--no-build" ]] \ + && grep -q 'chatter from security' "$work/tmp/check.log" \ + && grep -q 'build is left to CI' "$work/tmp/check.log" \ + && grep -qE '^lint +pass' <<< "$out"; then + pass +else + fail "wanted ci --no-build, every other step, and the log naming the skip, got exit $status: $out" +fi + +ci_args_fixture "--fast and --verbose combine in either order" +out="$(cd "$work" && scripts/check --fast --verbose 2>&1)" +status=$? +if (( status == 0 )) \ + && [[ "$(cat "$work/ci-args" 2>&1)" == "ci:--no-build" ]] \ + && grep -q 'chatter from doctor' <<< "$out"; then + pass +else + fail "wanted the streamed fast run, got exit $status: $out" +fi + +gate_fixture "two modes are a usage error" +(cd "$work" && scripts/check --verbose --gate >/dev/null 2>&1) +status=$? +if (( status == 2 )); then + pass +else + fail "wanted exit 2, got $status" +fi + gate_fixture "an unknown argument is a usage error" (cd "$work" && scripts/check --bogus >/dev/null 2>&1) status=$? diff --git a/scripts/check b/scripts/check index 867e76c5..55524945 100755 --- a/scripts/check +++ b/scripts/check @@ -20,23 +20,44 @@ cd "$repo_root" # the cause is gone; the log is what a failure outside the table is diagnosed # from. --verbose streams the same text live. --gate is the internal entry that # runs the steps below and nothing else, so the tee wraps exactly one run. -case "${1:-}" in +# +# --fast is the commit-time gate: every step below except build, which CI runs +# on the same commit before it can land and which costs more than the rest of +# the gate together in a repository that compiles. It rides through each entry, +# and tmp/check.log opens by saying build was left out, so a fast run's log +# never reads as the full gate's. +usage() { + echo "usage: scripts/check [--fast] [--verbose]" >&2 + exit 2 +} +mode="" +fast=() +for arg in "$@"; do + case "$arg" in + --verbose | --gate) + [[ -z "$mode" ]] || usage + mode="$arg" + ;; + --fast) fast=(--fast) ;; + *) usage ;; + esac +done +case "$mode" in "") - exec scripts/summarize scripts/check --verbose + exec scripts/summarize scripts/check --verbose "${fast[@]}" ;; --verbose) mkdir -p tmp # pipefail (set above) makes the pipeline's status the gate's, not tee's. - scripts/check --gate 2>&1 | tee tmp/check.log + scripts/check --gate "${fast[@]}" 2>&1 | tee tmp/check.log exit "${PIPESTATUS[0]}" ;; - --gate) ;; - *) - echo "usage: scripts/check [--verbose]" >&2 - exit 2 - ;; esac +if (( ${#fast[@]} )); then + echo "scripts/check --fast: build is left to CI." +fi + if [[ ! -x scripts/doctor ]]; then echo "scripts/doctor is missing or not executable." exit 1 @@ -78,7 +99,11 @@ if [[ ! -x scripts/ci ]]; then exit 1 fi -scripts/ci +if (( ${#fast[@]} )); then + scripts/ci --no-build +else + scripts/ci +fi # The security workflow blocks a pull request the same way the CI workflow does, # so leaving it out here means finding out on the pull request instead of before diff --git a/scripts/ci b/scripts/ci index 8cfae084..eec43ab3 100755 --- a/scripts/ci +++ b/scripts/ci @@ -8,6 +8,19 @@ cd "$repo_root" # shellcheck source=libs/detect.sh source "$repo_root/scripts/libs/detect.sh" +# --no-build leaves build out, for `scripts/check --fast`: the commit-time gate +# drops the slowest check CI repeats on the same commit before it can land. +# CI itself never passes it. +capabilities=(lint format-check typecheck test build) +case "${1:-}" in + "") ;; + --no-build) capabilities=(lint format-check typecheck test) ;; + *) + echo "usage: scripts/ci [--no-build]" >&2 + exit 2 + ;; +esac + # The shared policy: orphan search before the checks, which would run green over # the package none of them sees; the no-manifest early-out; the dry run. -capability_command check lint format-check typecheck test build +capability_command check "${capabilities[@]}" diff --git a/scripts/tests/capabilities-test b/scripts/tests/capabilities-test index 6cd66cde..286e527d 100755 --- a/scripts/tests/capabilities-test +++ b/scripts/tests/capabilities-test @@ -167,6 +167,28 @@ out="$(ci)" assert_dispatches python "$out" assert_absent node "$out" +fixture "--no-build leaves build out and nothing else" +printf '[project]\nname = "p"\nversion = "0.1.0"\n' > "$work/pyproject.toml" +out="$(cd "$work" && CI_DRY_RUN=1 scripts/ci --no-build 2>&1 /dev/null 2>&1 "$work/package.json" out="$(ci)" diff --git a/scripts/tests/check-test b/scripts/tests/check-test index ff52a920..e3be1ddd 100755 --- a/scripts/tests/check-test +++ b/scripts/tests/check-test @@ -4,8 +4,8 @@ set -uo pipefail # What scripts/check prints, writes, and exits with. The gate's steps are # stubbed in a scratch copy of the scripts, so the cases assert the wrapper's # contract -- table only by default, the whole run in tmp/check.log every time, -# raw output under --verbose, the gate's own exit status -- and not the output -# of any real check. +# raw output under --verbose, the gate's own exit status, --fast handing ci +# --no-build -- and not the output of any real check. repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" @@ -115,6 +115,55 @@ else fail "wanted the table and exit 0 from a subdirectory, got exit $status: $out" fi +# ci records the arguments it was given, so a case can tell the full gate from +# the fast one without running a real build. +ci_args_fixture() { + gate_fixture "$1" + printf '#!/usr/bin/env bash\necho "ci:$*" > ci-args\necho "lint pass python"\n' \ + > "$work/scripts/ci" +} + +ci_args_fixture "the default gate runs ci whole" +(cd "$work" && scripts/check >/dev/null 2>&1) +if [[ "$(cat "$work/ci-args" 2>&1)" == "ci:" ]]; then + pass +else + fail "wanted ci with no arguments, got: $(cat "$work/ci-args" 2>&1)" +fi + +ci_args_fixture "--fast leaves build to CI and says so in the log" +out="$(cd "$work" && scripts/check --fast 2>&1)" +status=$? +if (( status == 0 )) \ + && [[ "$(cat "$work/ci-args" 2>&1)" == "ci:--no-build" ]] \ + && grep -q 'chatter from security' "$work/tmp/check.log" \ + && grep -q 'build is left to CI' "$work/tmp/check.log" \ + && grep -qE '^lint +pass' <<< "$out"; then + pass +else + fail "wanted ci --no-build, every other step, and the log naming the skip, got exit $status: $out" +fi + +ci_args_fixture "--fast and --verbose combine in either order" +out="$(cd "$work" && scripts/check --fast --verbose 2>&1)" +status=$? +if (( status == 0 )) \ + && [[ "$(cat "$work/ci-args" 2>&1)" == "ci:--no-build" ]] \ + && grep -q 'chatter from doctor' <<< "$out"; then + pass +else + fail "wanted the streamed fast run, got exit $status: $out" +fi + +gate_fixture "two modes are a usage error" +(cd "$work" && scripts/check --verbose --gate >/dev/null 2>&1) +status=$? +if (( status == 2 )); then + pass +else + fail "wanted exit 2, got $status" +fi + gate_fixture "an unknown argument is a usage error" (cd "$work" && scripts/check --bogus >/dev/null 2>&1) status=$? From 6353c86356a9dc36318fa675251ecab0ea1ae0d5 Mon Sep 17 00:00:00 2001 From: neal Date: Wed, 18 Nov 2026 22:52:44 -0500 Subject: [PATCH 2/4] ci: skip the push-to-main checks when a pull request already passed that tree A clean merge ran ci.yml twice on one tree: once on the pull request and again on the push to main, spending self-hosted runner CPU for no new signal. scripts/ci-tested decides the skip with the test /land already uses for its local gate: the replaced tip is an ancestor of the landed head (the merge commit's second parent, or HEAD on a fast-forward) and HEAD's tree is that head's, then GitHub must report a passing pull_request run of this workflow on that head. The query goes by head_sha because runs created by a push carry no pull_requests field. A squash or rebase merge writes a commit no pull request run saw, a moved base fails the ancestor test, and any failed lookup answers skip=false, so every doubt runs the checks; a wrong skip leaves main unproven, a wrong run costs minutes. The job still succeeds when it skips, so a required status on main stays green. It needs actions: read for the runs query. Generated-By: claude-opus-5-5 --- .github/workflows/ci.yml | 18 +++ .../docs/github_repository_structure.md | 6 +- .../src/base-repo/.github/workflows/ci.yml | 18 +++ .../src/base-repo/scripts/ci-tested | 54 ++++++++ .../base-repo/scripts/tests/ci-tested-test | 127 ++++++++++++++++++ scripts/CLAUDE.md | 4 +- scripts/ci-tested | 54 ++++++++ scripts/tests/ci-tested-test | 127 ++++++++++++++++++ 8 files changed, 404 insertions(+), 4 deletions(-) create mode 100755 apps/github-repository-template/src/base-repo/scripts/ci-tested create mode 100755 apps/github-repository-template/src/base-repo/scripts/tests/ci-tested-test create mode 100755 scripts/ci-tested create mode 100755 scripts/tests/ci-tested-test diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 656324fd..6e834ade 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -8,6 +8,8 @@ on: permissions: contents: read + # scripts/ci-tested lists this workflow's pull request runs. + actions: read # A new push to the same branch makes the previous run's result obsolete, and # leaving it running lets a stale result report after the newer one. @@ -30,6 +32,18 @@ jobs: with: fetch-depth: 0 + # A push that lands a pull request's head unchanged -- a true merge onto + # the base that head already contains, or a fast-forward -- holds the + # tree that pull request's run already passed, so the checks below would + # prove it twice. scripts/ci-tested decides, and every doubt runs them. + - name: Find a passing run on this tree + id: tested + if: github.event_name == 'push' + env: + GH_TOKEN: ${{ github.token }} + BEFORE: ${{ github.event.before }} + run: scripts/ci-tested "$BEFORE" >> "$GITHUB_OUTPUT" + # adr-index-test and commitlint-test both skip their hook-wiring cases # when pre-commit is absent, and those cases are the ones that catch a # hook pre-commit never invokes -- the failure both suites exist for. @@ -43,11 +57,13 @@ jobs: # path of this job, which was accepted rather than shipping a # commit-message rule to every generated repository unproven. - name: Install pre-commit + if: steps.tested.outputs.skip != 'true' run: pipx install pre-commit # Before the toolchains, since these install none and verify both the # language-capabilities wiring and the local/hosted check boundary. - name: Verify repository scripts + if: steps.tested.outputs.skip != 'true' run: scripts/script-tests # The hook receives the same base-to-head file range a push does. It @@ -63,7 +79,9 @@ jobs: --from-ref "$BASE_SHA" --to-ref "$HEAD_SHA" - name: Set up toolchains + if: steps.tested.outputs.skip != 'true' uses: ./.github/actions/setup-toolchains - name: Run CI checks + if: steps.tested.outputs.skip != 'true' run: scripts/ci diff --git a/apps/github-repository-template/docs/github_repository_structure.md b/apps/github-repository-template/docs/github_repository_structure.md index ebdd7f21..1f47f601 100644 --- a/apps/github-repository-template/docs/github_repository_structure.md +++ b/apps/github-repository-template/docs/github_repository_structure.md @@ -5,7 +5,7 @@ type: Note ### GitHub Automation -`.github/workflows/ci.yml`: Runs tests, linting, formatting checks, type checks, and builds on every push and pull request. +`.github/workflows/ci.yml`: Runs tests, linting, formatting checks, type checks, and builds on every pull request and every push to `main`. A push that lands a pull request's head unchanged skips the checks and goes green, since that pull request's passing run already proved the same tree; `scripts/ci-tested` decides. `.github/workflows/release.yml`: Validates a tagged release and publishes it, both through `scripts/release`. @@ -79,7 +79,9 @@ The specific operating parameters for the AI agent. `scripts/script-tests`: Runs every `scripts/tests/*-test` concurrently, one per CPU, and prints each one's output whole once all have finished, naming every test that failed. Called by `scripts/check` and by `ci.yml`, so both run the suites the same way. The suites spend their time waiting on git and child processes, so running them side by side cuts the wall time several-fold. -`scripts/check`: The full local gate. By default it prints only the Result table and the findings under any check that failed, by running itself under `scripts/summarize`, and it writes the complete run to `tmp/check.log` every time, passing or failing, since a pre-commit fixer repairs the file and a re-run no longer reproduces the failure. `--verbose` streams the whole run live, still writing the log. Exits with the gate's status either way. +`scripts/check`: The full local gate. By default it prints only the Result table and the findings under any check that failed, by running itself under `scripts/summarize`, and it writes the complete run to `tmp/check.log` every time, passing or failing, since a pre-commit fixer repairs the file and a re-run no longer reproduces the failure. `--verbose` streams the whole run live, still writing the log. `--fast` is the commit-time gate: every step but `build`, which CI runs on the same commit before it can land. Exits with the gate's status either way. + +`scripts/ci-tested`: Called by `ci.yml` on a push to `main`, and nowhere else, since it reads hosted run state. Prints `skip=true` when the push's tree is a pull request head whose run of the same workflow passed: a true merge onto a base that head already contains, or a fast-forward. Anything else, including a failed query, prints `skip=false` and the checks run. `scripts/summarize`: Runs a check command and prints only its Result table — one line per check, the findings under any that failed — dropping every other thing the command wrote. Exits with that command's status, so it can stand in for the command rather than only report on it, and ends with a verdict line naming that status and a count per state. `scripts/check` is its first caller; it stays the way to read `scripts/ci` and any other command. It is the one consumer of the printed layout `scripts/libs/result.sh` documents, which is where a change to the column widths has to be reflected. diff --git a/apps/github-repository-template/src/base-repo/.github/workflows/ci.yml b/apps/github-repository-template/src/base-repo/.github/workflows/ci.yml index 656324fd..6e834ade 100644 --- a/apps/github-repository-template/src/base-repo/.github/workflows/ci.yml +++ b/apps/github-repository-template/src/base-repo/.github/workflows/ci.yml @@ -8,6 +8,8 @@ on: permissions: contents: read + # scripts/ci-tested lists this workflow's pull request runs. + actions: read # A new push to the same branch makes the previous run's result obsolete, and # leaving it running lets a stale result report after the newer one. @@ -30,6 +32,18 @@ jobs: with: fetch-depth: 0 + # A push that lands a pull request's head unchanged -- a true merge onto + # the base that head already contains, or a fast-forward -- holds the + # tree that pull request's run already passed, so the checks below would + # prove it twice. scripts/ci-tested decides, and every doubt runs them. + - name: Find a passing run on this tree + id: tested + if: github.event_name == 'push' + env: + GH_TOKEN: ${{ github.token }} + BEFORE: ${{ github.event.before }} + run: scripts/ci-tested "$BEFORE" >> "$GITHUB_OUTPUT" + # adr-index-test and commitlint-test both skip their hook-wiring cases # when pre-commit is absent, and those cases are the ones that catch a # hook pre-commit never invokes -- the failure both suites exist for. @@ -43,11 +57,13 @@ jobs: # path of this job, which was accepted rather than shipping a # commit-message rule to every generated repository unproven. - name: Install pre-commit + if: steps.tested.outputs.skip != 'true' run: pipx install pre-commit # Before the toolchains, since these install none and verify both the # language-capabilities wiring and the local/hosted check boundary. - name: Verify repository scripts + if: steps.tested.outputs.skip != 'true' run: scripts/script-tests # The hook receives the same base-to-head file range a push does. It @@ -63,7 +79,9 @@ jobs: --from-ref "$BASE_SHA" --to-ref "$HEAD_SHA" - name: Set up toolchains + if: steps.tested.outputs.skip != 'true' uses: ./.github/actions/setup-toolchains - name: Run CI checks + if: steps.tested.outputs.skip != 'true' run: scripts/ci diff --git a/apps/github-repository-template/src/base-repo/scripts/ci-tested b/apps/github-repository-template/src/base-repo/scripts/ci-tested new file mode 100755 index 00000000..ed0a50cc --- /dev/null +++ b/apps/github-repository-template/src/base-repo/scripts/ci-tested @@ -0,0 +1,54 @@ +#!/usr/bin/env bash +set -euo pipefail + +# Whether a push to the default branch landed a tree this workflow already +# passed on a pull request, so ci.yml can skip its checks instead of proving the +# same tree twice. Prints `skip=true` or `skip=false` for $GITHUB_OUTPUT and the +# reason on stderr, which the job log keeps. +# +# scripts/ci-tested +# +# is the branch tip the push replaced (github.event.before). The tip +# that landed is HEAD's second parent for a merge commit, else HEAD itself, a +# fast-forward. The skip holds only when is an ancestor of that tip and +# HEAD's tree is the tip's: a pull request run tests the tip merged onto the +# base it saw, every such base is an ancestor of while the branch only +# moves forward, so that merge is the tip's own tree, the tree now on the +# branch. The same test /land uses to skip its local gate. A squash or rebase +# merge writes a commit no pull request run saw, so it never skips. +# +# Called by ci.yml alone: it reads hosted run state, which no local command does. +# Every doubt answers skip=false, since running the checks again costs minutes +# and skipping them wrongly leaves the branch unproven. + +before="${1:?usage: scripts/ci-tested }" + +decide() { + echo "ci-tested: $2" >&2 + echo "skip=$1" + exit 0 +} + +[[ "$before" =~ ^0+$ ]] && decide false "the push created the branch" +git cat-file -e "$before^{commit}" 2>/dev/null || + decide false "the replaced tip $before is not in this checkout" + +read -r -a parents <<< "$(git rev-list --parents -n 1 HEAD)" +tip="${parents[0]}" +(( ${#parents[@]} == 3 )) && tip="${parents[2]}" + +git merge-base --is-ancestor "$before" "$tip" || + decide false "the branch moved past the pull request's base" +[[ "$(git rev-parse "HEAD^{tree}")" == "$(git rev-parse "$tip^{tree}")" ]] || + decide false "HEAD's tree differs from the pull request head $tip" + +workflow="${GITHUB_WORKFLOW_REF:?}" +workflow="${workflow%@*}" +workflow="${workflow##*/}" +count="$(gh api \ + "repos/${GITHUB_REPOSITORY:?}/actions/workflows/$workflow/runs?event=pull_request&status=success&head_sha=$tip&per_page=1" \ + --jq .total_count 2>/dev/null)" || decide false "could not read $workflow's runs" + +[[ "$count" =~ ^[1-9] ]] || + decide false "no passing pull request run of $workflow on $tip" +decide true "$workflow passed on pull request head $tip, the tree now on the branch" diff --git a/apps/github-repository-template/src/base-repo/scripts/tests/ci-tested-test b/apps/github-repository-template/src/base-repo/scripts/tests/ci-tested-test new file mode 100755 index 00000000..a74701b5 --- /dev/null +++ b/apps/github-repository-template/src/base-repo/scripts/tests/ci-tested-test @@ -0,0 +1,127 @@ +#!/usr/bin/env bash +set -uo pipefail + +# Whether scripts/ci-tested skips the default branch's checks only when a +# passing pull request run already proved the tree that landed. Each case builds +# the push in a scratch repository -- a merge commit, a fast-forward, a moved +# base, a merge that changed the tree -- and `gh` is a stub answering the runs +# query with a fixed count, recording what it was asked. + +repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" + +# shellcheck source-path=SCRIPTDIR +# shellcheck source=libs/harness.sh +source "$repo_root/scripts/tests/libs/harness.sh" + +echo "ci-tested-test" + +export GITHUB_REPOSITORY=owner/repo +export GITHUB_WORKFLOW_REF=owner/repo/.github/workflows/ci.yml@refs/heads/main + +# A repository with one commit on main, the tip a push would replace, and a +# branch `topic` one commit ahead of it. The gh stub answers with $2 runs, or +# fails when $2 is `error`. +fixture() { + scratch_repo + current="$1" + stub gh <> "$work/gh-calls" +[[ "$2" == error ]] && exit 1 +echo "$2" +STUB + git -C "$work" add -A + git -C "$work" commit -qm base + git -C "$work" branch -qM main + before="$(git -C "$work" rev-parse HEAD)" + git -C "$work" checkout -qb topic + echo change > "$work/file" + git -C "$work" add file + git -C "$work" commit -qm change + tip="$(git -C "$work" rev-parse HEAD)" + git -C "$work" checkout -q main +} + +decide() { + (cd "$work" && scripts/ci-tested "$1" 2>"$work/reason") +} + +fixture "a merge of an unmoved base onto a green head skips" 1 +git -C "$work" merge -q --no-ff -m merge topic +out="$(decide "$before")" +if [[ "$out" == "skip=true" ]] && grep -q "head_sha=$tip" "$work/gh-calls" \ + && grep -q 'workflows/ci.yml/runs?event=pull_request&status=success' "$work/gh-calls"; then + pass +else + fail "wanted skip=true from a runs query on $tip, got '$out': $(cat "$work/reason" "$work/gh-calls" 2>&1)" +fi + +fixture "a fast-forward onto a green head skips" 1 +git -C "$work" merge -q --ff-only topic +out="$(decide "$before")" +if [[ "$out" == "skip=true" ]] && grep -q "head_sha=$tip" "$work/gh-calls"; then + pass +else + fail "wanted skip=true, got '$out': $(cat "$work/reason" 2>&1)" +fi + +fixture "no passing pull request run runs the checks" 0 +git -C "$work" merge -q --no-ff -m merge topic +out="$(decide "$before")" +if [[ "$out" == "skip=false" ]] && grep -q 'no passing pull request run' "$work/reason"; then + pass +else + fail "wanted skip=false, got '$out': $(cat "$work/reason" 2>&1)" +fi + +fixture "a base that moved runs the checks without asking GitHub" 1 +echo other > "$work/other" +git -C "$work" add other +git -C "$work" commit -qm moved +moved="$(git -C "$work" rev-parse HEAD)" +git -C "$work" merge -q --no-ff -m merge topic +out="$(decide "$moved")" +if [[ "$out" == "skip=false" ]] && [[ ! -e "$work/gh-calls" ]]; then + pass +else + fail "wanted skip=false and no gh call, got '$out': $(cat "$work/reason" 2>&1)" +fi + +fixture "a merge that changed the tree runs the checks" 1 +git -C "$work" merge -q --no-ff --no-commit topic >/dev/null 2>&1 +echo extra > "$work/extra" +git -C "$work" add extra +git -C "$work" commit -qm merge +out="$(decide "$before")" +if [[ "$out" == "skip=false" ]] && grep -q 'tree differs' "$work/reason"; then + pass +else + fail "wanted skip=false on a changed merge tree, got '$out': $(cat "$work/reason" 2>&1)" +fi + +fixture "a failed runs query runs the checks" error +git -C "$work" merge -q --no-ff -m merge topic +out="$(decide "$before")" +if [[ "$out" == "skip=false" ]] && grep -q 'could not read' "$work/reason"; then + pass +else + fail "wanted skip=false when gh fails, got '$out': $(cat "$work/reason" 2>&1)" +fi + +fixture "a push that created the branch runs the checks" 1 +out="$(decide 0000000000000000000000000000000000000000)" +if [[ "$out" == "skip=false" ]]; then + pass +else + fail "wanted skip=false for a new branch, got '$out'" +fi + +fixture "a replaced tip missing from the checkout runs the checks" 1 +out="$(decide 1111111111111111111111111111111111111111)" +if [[ "$out" == "skip=false" ]] && grep -q 'not in this checkout' "$work/reason"; then + pass +else + fail "wanted skip=false for an unknown tip, got '$out': $(cat "$work/reason" 2>&1)" +fi + +report diff --git a/scripts/CLAUDE.md b/scripts/CLAUDE.md index 1f78747d..ba52c917 100644 --- a/scripts/CLAUDE.md +++ b/scripts/CLAUDE.md @@ -11,7 +11,7 @@ One flat directory, not a `ci/` and a `scripts/` split: `check` calls `ci` and ` Each script's header comment says what it does and why it is shaped that way, and the lists below say who calls it; each `tests/*-test` header says what that suite covers. Read the header before editing the file. This document holds only the rules that span scripts. - Run by a person: `doctor`, `check`, `fix`, `clean`, `run`, `integration`, `package`, `summarize`, `repo-settings check` -- Called by `.github/`: `ci`, `security`, `release`, `detect`, `changelog-check`, `system-packages`, `script-tests` +- Called by `.github/`: `ci`, `ci-tested`, `security`, `release`, `detect`, `changelog-check`, `system-packages`, `script-tests` - Called by pre-commit: `adr-index`, `structure`, `payload-parity`, `attribute-commit` (`prepare-commit-msg`, and `--check` at `commit-msg`), `changelog-check` and `protect-branch` (`pre-push`), `worktree-cleanup` (`post-checkout`, `post-merge`, and SessionStart with `--report`) - Sourced: `libs/detect.sh`, `libs/result.sh`, `libs/unit.sh`, `libs/precommit.sh`, `libs/dependabot.sh`, `libs/quadlet.sh`, `tests/libs/harness.sh` @@ -25,7 +25,7 @@ Each script's header comment says what it does and why it is shaped that way, an - **An adapter is honest about what it did.** Capture the tool's exit status, not only its output. No function stands in for an absent one: the probe reads `absent` off the function table and the dispatch reports `unavailable`. A runner that exits non-zero without a real failure is translated, and `libs/detect.sh`'s header lists the four. - **`libs/*.sh` and `harness.sh` are sourced, never executed**: no shebang, no executable bit, `.sh`. Every other script is extensionless and executable; pre-commit reads a shebang only on one. - **`adr-index` and `structure` are hooks, not capabilities**, and run `always_run` with no `files:` filter, since staged paths exclude deletions. -- **Local commands stay off hosted GitHub state.** The boundary is hosted state, not connectivity; only `repo-settings check` crosses it. +- **Local commands stay off hosted GitHub state.** The boundary is hosted state, not connectivity; only `repo-settings check` crosses it, and `ci-tested`, which only `ci.yml` calls. ## Work Guidance diff --git a/scripts/ci-tested b/scripts/ci-tested new file mode 100755 index 00000000..ed0a50cc --- /dev/null +++ b/scripts/ci-tested @@ -0,0 +1,54 @@ +#!/usr/bin/env bash +set -euo pipefail + +# Whether a push to the default branch landed a tree this workflow already +# passed on a pull request, so ci.yml can skip its checks instead of proving the +# same tree twice. Prints `skip=true` or `skip=false` for $GITHUB_OUTPUT and the +# reason on stderr, which the job log keeps. +# +# scripts/ci-tested +# +# is the branch tip the push replaced (github.event.before). The tip +# that landed is HEAD's second parent for a merge commit, else HEAD itself, a +# fast-forward. The skip holds only when is an ancestor of that tip and +# HEAD's tree is the tip's: a pull request run tests the tip merged onto the +# base it saw, every such base is an ancestor of while the branch only +# moves forward, so that merge is the tip's own tree, the tree now on the +# branch. The same test /land uses to skip its local gate. A squash or rebase +# merge writes a commit no pull request run saw, so it never skips. +# +# Called by ci.yml alone: it reads hosted run state, which no local command does. +# Every doubt answers skip=false, since running the checks again costs minutes +# and skipping them wrongly leaves the branch unproven. + +before="${1:?usage: scripts/ci-tested }" + +decide() { + echo "ci-tested: $2" >&2 + echo "skip=$1" + exit 0 +} + +[[ "$before" =~ ^0+$ ]] && decide false "the push created the branch" +git cat-file -e "$before^{commit}" 2>/dev/null || + decide false "the replaced tip $before is not in this checkout" + +read -r -a parents <<< "$(git rev-list --parents -n 1 HEAD)" +tip="${parents[0]}" +(( ${#parents[@]} == 3 )) && tip="${parents[2]}" + +git merge-base --is-ancestor "$before" "$tip" || + decide false "the branch moved past the pull request's base" +[[ "$(git rev-parse "HEAD^{tree}")" == "$(git rev-parse "$tip^{tree}")" ]] || + decide false "HEAD's tree differs from the pull request head $tip" + +workflow="${GITHUB_WORKFLOW_REF:?}" +workflow="${workflow%@*}" +workflow="${workflow##*/}" +count="$(gh api \ + "repos/${GITHUB_REPOSITORY:?}/actions/workflows/$workflow/runs?event=pull_request&status=success&head_sha=$tip&per_page=1" \ + --jq .total_count 2>/dev/null)" || decide false "could not read $workflow's runs" + +[[ "$count" =~ ^[1-9] ]] || + decide false "no passing pull request run of $workflow on $tip" +decide true "$workflow passed on pull request head $tip, the tree now on the branch" diff --git a/scripts/tests/ci-tested-test b/scripts/tests/ci-tested-test new file mode 100755 index 00000000..a74701b5 --- /dev/null +++ b/scripts/tests/ci-tested-test @@ -0,0 +1,127 @@ +#!/usr/bin/env bash +set -uo pipefail + +# Whether scripts/ci-tested skips the default branch's checks only when a +# passing pull request run already proved the tree that landed. Each case builds +# the push in a scratch repository -- a merge commit, a fast-forward, a moved +# base, a merge that changed the tree -- and `gh` is a stub answering the runs +# query with a fixed count, recording what it was asked. + +repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" + +# shellcheck source-path=SCRIPTDIR +# shellcheck source=libs/harness.sh +source "$repo_root/scripts/tests/libs/harness.sh" + +echo "ci-tested-test" + +export GITHUB_REPOSITORY=owner/repo +export GITHUB_WORKFLOW_REF=owner/repo/.github/workflows/ci.yml@refs/heads/main + +# A repository with one commit on main, the tip a push would replace, and a +# branch `topic` one commit ahead of it. The gh stub answers with $2 runs, or +# fails when $2 is `error`. +fixture() { + scratch_repo + current="$1" + stub gh <> "$work/gh-calls" +[[ "$2" == error ]] && exit 1 +echo "$2" +STUB + git -C "$work" add -A + git -C "$work" commit -qm base + git -C "$work" branch -qM main + before="$(git -C "$work" rev-parse HEAD)" + git -C "$work" checkout -qb topic + echo change > "$work/file" + git -C "$work" add file + git -C "$work" commit -qm change + tip="$(git -C "$work" rev-parse HEAD)" + git -C "$work" checkout -q main +} + +decide() { + (cd "$work" && scripts/ci-tested "$1" 2>"$work/reason") +} + +fixture "a merge of an unmoved base onto a green head skips" 1 +git -C "$work" merge -q --no-ff -m merge topic +out="$(decide "$before")" +if [[ "$out" == "skip=true" ]] && grep -q "head_sha=$tip" "$work/gh-calls" \ + && grep -q 'workflows/ci.yml/runs?event=pull_request&status=success' "$work/gh-calls"; then + pass +else + fail "wanted skip=true from a runs query on $tip, got '$out': $(cat "$work/reason" "$work/gh-calls" 2>&1)" +fi + +fixture "a fast-forward onto a green head skips" 1 +git -C "$work" merge -q --ff-only topic +out="$(decide "$before")" +if [[ "$out" == "skip=true" ]] && grep -q "head_sha=$tip" "$work/gh-calls"; then + pass +else + fail "wanted skip=true, got '$out': $(cat "$work/reason" 2>&1)" +fi + +fixture "no passing pull request run runs the checks" 0 +git -C "$work" merge -q --no-ff -m merge topic +out="$(decide "$before")" +if [[ "$out" == "skip=false" ]] && grep -q 'no passing pull request run' "$work/reason"; then + pass +else + fail "wanted skip=false, got '$out': $(cat "$work/reason" 2>&1)" +fi + +fixture "a base that moved runs the checks without asking GitHub" 1 +echo other > "$work/other" +git -C "$work" add other +git -C "$work" commit -qm moved +moved="$(git -C "$work" rev-parse HEAD)" +git -C "$work" merge -q --no-ff -m merge topic +out="$(decide "$moved")" +if [[ "$out" == "skip=false" ]] && [[ ! -e "$work/gh-calls" ]]; then + pass +else + fail "wanted skip=false and no gh call, got '$out': $(cat "$work/reason" 2>&1)" +fi + +fixture "a merge that changed the tree runs the checks" 1 +git -C "$work" merge -q --no-ff --no-commit topic >/dev/null 2>&1 +echo extra > "$work/extra" +git -C "$work" add extra +git -C "$work" commit -qm merge +out="$(decide "$before")" +if [[ "$out" == "skip=false" ]] && grep -q 'tree differs' "$work/reason"; then + pass +else + fail "wanted skip=false on a changed merge tree, got '$out': $(cat "$work/reason" 2>&1)" +fi + +fixture "a failed runs query runs the checks" error +git -C "$work" merge -q --no-ff -m merge topic +out="$(decide "$before")" +if [[ "$out" == "skip=false" ]] && grep -q 'could not read' "$work/reason"; then + pass +else + fail "wanted skip=false when gh fails, got '$out': $(cat "$work/reason" 2>&1)" +fi + +fixture "a push that created the branch runs the checks" 1 +out="$(decide 0000000000000000000000000000000000000000)" +if [[ "$out" == "skip=false" ]]; then + pass +else + fail "wanted skip=false for a new branch, got '$out'" +fi + +fixture "a replaced tip missing from the checkout runs the checks" 1 +out="$(decide 1111111111111111111111111111111111111111)" +if [[ "$out" == "skip=false" ]] && grep -q 'not in this checkout' "$work/reason"; then + pass +else + fail "wanted skip=false for an unknown tip, got '$out': $(cat "$work/reason" 2>&1)" +fi + +report From b38b344d65b0c9e04f400fb572e4fb3603d58e4b Mon Sep 17 00:00:00 2001 From: neal Date: Wed, 18 Nov 2026 23:03:51 -0500 Subject: [PATCH 3/4] fix(ci): skip a push only on a tree a passing run recorded The skip trusted a passing pull request run on the landed head's SHA, with the base reasoned forward from github.event.before. Review found that it trusted too much: a run against another base branch, a force-rewound main, or an older green run beside a newer red one could each skip a tree no run had proved. A passing pull request run now uploads an artifact named for the tree it tested, its merge ref's, and the push skips only when an unexpired one from a same-repository run names HEAD's tree. That drops the ancestry reasoning, and a squash or fast-forward whose tree matches now skips too. Both steps run only where vars.RUNNER is set. A hosted runner costs the operator nothing for the second run, and skipping it there would stop main saving the Go, uv and Gradle caches each pull request restores. Generated-By: claude-opus-5-5 --- .github/workflows/ci.yml | 35 ++++-- .../docs/github_repository_structure.md | 4 +- .../src/base-repo/.github/workflows/ci.yml | 35 ++++-- .../src/base-repo/CLAUDE.md | 2 +- .../src/base-repo/scripts/ci-tested | 55 +++------ .../base-repo/scripts/tests/ci-tested-test | 113 +++++++----------- scripts/ci-tested | 55 +++------ scripts/tests/ci-tested-test | 113 +++++++----------- 8 files changed, 179 insertions(+), 233 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 6e834ade..d3cfdd72 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -8,7 +8,7 @@ on: permissions: contents: read - # scripts/ci-tested lists this workflow's pull request runs. + # scripts/ci-tested lists the tested-tree artifacts. actions: read # A new push to the same branch makes the previous run's result obsolete, and @@ -32,17 +32,19 @@ jobs: with: fetch-depth: 0 - # A push that lands a pull request's head unchanged -- a true merge onto - # the base that head already contains, or a fast-forward -- holds the - # tree that pull request's run already passed, so the checks below would - # prove it twice. scripts/ci-tested decides, and every doubt runs them. + # A push whose tree a pull request run already passed -- a merge, squash + # or fast-forward onto a base that has not moved since -- would prove that + # tree twice. scripts/ci-tested looks for the tree the run recorded at the + # end of this job, and every doubt runs the checks. Only on a self-hosted + # runner: it is the operator's machine that pays for the second run, and + # it keeps toolchain caches on disk, while on a hosted runner a skipped + # push stops main saving the caches every pull request restores. - name: Find a passing run on this tree id: tested - if: github.event_name == 'push' + if: github.event_name == 'push' && vars.RUNNER != '' env: GH_TOKEN: ${{ github.token }} - BEFORE: ${{ github.event.before }} - run: scripts/ci-tested "$BEFORE" >> "$GITHUB_OUTPUT" + run: scripts/ci-tested >> "$GITHUB_OUTPUT" # adr-index-test and commitlint-test both skip their hook-wiring cases # when pre-commit is absent, and those cases are the ones that catch a @@ -85,3 +87,20 @@ jobs: - name: Run CI checks if: steps.tested.outputs.skip != 'true' run: scripts/ci + + # The tree this run just passed, the merge ref's, for scripts/ci-tested + # on the push that lands it. Reached only when every check above passed. + - name: Name the tested tree + id: tree + if: github.event_name == 'pull_request' && vars.RUNNER != '' + run: | + git rev-parse 'HEAD^{tree}' > "$RUNNER_TEMP/tested-tree" + echo "tree=$(cat "$RUNNER_TEMP/tested-tree")" >> "$GITHUB_OUTPUT" + + - name: Record the tested tree + if: steps.tree.outcome == 'success' + uses: actions/upload-artifact@v7 + with: + name: ci-tested-tree-${{ steps.tree.outputs.tree }} + path: ${{ runner.temp }}/tested-tree + retention-days: 30 diff --git a/apps/github-repository-template/docs/github_repository_structure.md b/apps/github-repository-template/docs/github_repository_structure.md index 1f47f601..3c4c423c 100644 --- a/apps/github-repository-template/docs/github_repository_structure.md +++ b/apps/github-repository-template/docs/github_repository_structure.md @@ -5,7 +5,7 @@ type: Note ### GitHub Automation -`.github/workflows/ci.yml`: Runs tests, linting, formatting checks, type checks, and builds on every pull request and every push to `main`. A push that lands a pull request's head unchanged skips the checks and goes green, since that pull request's passing run already proved the same tree; `scripts/ci-tested` decides. +`.github/workflows/ci.yml`: Runs tests, linting, formatting checks, type checks, and builds on every pull request and every push to `main`. On a self-hosted runner, a passing pull request run records the tree it tested as an artifact, and a push whose tree one of those records names skips the checks and goes green; `scripts/ci-tested` decides. `.github/workflows/release.yml`: Validates a tagged release and publishes it, both through `scripts/release`. @@ -81,7 +81,7 @@ The specific operating parameters for the AI agent. `scripts/check`: The full local gate. By default it prints only the Result table and the findings under any check that failed, by running itself under `scripts/summarize`, and it writes the complete run to `tmp/check.log` every time, passing or failing, since a pre-commit fixer repairs the file and a re-run no longer reproduces the failure. `--verbose` streams the whole run live, still writing the log. `--fast` is the commit-time gate: every step but `build`, which CI runs on the same commit before it can land. Exits with the gate's status either way. -`scripts/ci-tested`: Called by `ci.yml` on a push to `main`, and nowhere else, since it reads hosted run state. Prints `skip=true` when the push's tree is a pull request head whose run of the same workflow passed: a true merge onto a base that head already contains, or a fast-forward. Anything else, including a failed query, prints `skip=false` and the checks run. +`scripts/ci-tested`: Called by `ci.yml` on a push to `main` on a self-hosted runner, and nowhere else, since it reads hosted run state. Prints `skip=true` when an unexpired `ci-tested-tree-` artifact from a same-repository pull request run names the push's tree: a merge, squash or fast-forward onto a base that has not moved since that run. Anything else, including a failed query, prints `skip=false` and the checks run. `scripts/summarize`: Runs a check command and prints only its Result table — one line per check, the findings under any that failed — dropping every other thing the command wrote. Exits with that command's status, so it can stand in for the command rather than only report on it, and ends with a verdict line naming that status and a count per state. `scripts/check` is its first caller; it stays the way to read `scripts/ci` and any other command. It is the one consumer of the printed layout `scripts/libs/result.sh` documents, which is where a change to the column widths has to be reflected. diff --git a/apps/github-repository-template/src/base-repo/.github/workflows/ci.yml b/apps/github-repository-template/src/base-repo/.github/workflows/ci.yml index 6e834ade..d3cfdd72 100644 --- a/apps/github-repository-template/src/base-repo/.github/workflows/ci.yml +++ b/apps/github-repository-template/src/base-repo/.github/workflows/ci.yml @@ -8,7 +8,7 @@ on: permissions: contents: read - # scripts/ci-tested lists this workflow's pull request runs. + # scripts/ci-tested lists the tested-tree artifacts. actions: read # A new push to the same branch makes the previous run's result obsolete, and @@ -32,17 +32,19 @@ jobs: with: fetch-depth: 0 - # A push that lands a pull request's head unchanged -- a true merge onto - # the base that head already contains, or a fast-forward -- holds the - # tree that pull request's run already passed, so the checks below would - # prove it twice. scripts/ci-tested decides, and every doubt runs them. + # A push whose tree a pull request run already passed -- a merge, squash + # or fast-forward onto a base that has not moved since -- would prove that + # tree twice. scripts/ci-tested looks for the tree the run recorded at the + # end of this job, and every doubt runs the checks. Only on a self-hosted + # runner: it is the operator's machine that pays for the second run, and + # it keeps toolchain caches on disk, while on a hosted runner a skipped + # push stops main saving the caches every pull request restores. - name: Find a passing run on this tree id: tested - if: github.event_name == 'push' + if: github.event_name == 'push' && vars.RUNNER != '' env: GH_TOKEN: ${{ github.token }} - BEFORE: ${{ github.event.before }} - run: scripts/ci-tested "$BEFORE" >> "$GITHUB_OUTPUT" + run: scripts/ci-tested >> "$GITHUB_OUTPUT" # adr-index-test and commitlint-test both skip their hook-wiring cases # when pre-commit is absent, and those cases are the ones that catch a @@ -85,3 +87,20 @@ jobs: - name: Run CI checks if: steps.tested.outputs.skip != 'true' run: scripts/ci + + # The tree this run just passed, the merge ref's, for scripts/ci-tested + # on the push that lands it. Reached only when every check above passed. + - name: Name the tested tree + id: tree + if: github.event_name == 'pull_request' && vars.RUNNER != '' + run: | + git rev-parse 'HEAD^{tree}' > "$RUNNER_TEMP/tested-tree" + echo "tree=$(cat "$RUNNER_TEMP/tested-tree")" >> "$GITHUB_OUTPUT" + + - name: Record the tested tree + if: steps.tree.outcome == 'success' + uses: actions/upload-artifact@v7 + with: + name: ci-tested-tree-${{ steps.tree.outputs.tree }} + path: ${{ runner.temp }}/tested-tree + retention-days: 30 diff --git a/apps/github-repository-template/src/base-repo/CLAUDE.md b/apps/github-repository-template/src/base-repo/CLAUDE.md index 06adf8b5..a01fed3c 100644 --- a/apps/github-repository-template/src/base-repo/CLAUDE.md +++ b/apps/github-repository-template/src/base-repo/CLAUDE.md @@ -6,7 +6,7 @@ These sections are identical in every repository built from the template. Do not Use these instead of per-language tools. `check` and `fix` run for every language present and report `pass`, `not-applicable`, `unavailable`, or `FAIL`. -- `scripts/check` — the full local gate, a superset of CI except the pull request's changelog check: doctor, script tests, structure, lint, format, type check, test, build, security audit, pre-commit over every file. Prints only the Result table, with the full run in `tmp/check.log`; `--verbose` streams everything; `--fast`, the commit-time gate, leaves `build` to CI +- `scripts/check` — the full local gate, a superset of CI except the pull request's changelog check: doctor, script tests, structure, lint, format, type check, test, build, security audit, pre-commit over every file. Prints only the Result table, with the full run in `tmp/check.log`; `--verbose` streams everything; `--fast`, the commit-time gate, leaves `build` to CI and so is no superset of it - `scripts/fix` — lint autofix, then formatting - `scripts/summarize ` — run a check and print only its Result table, then a verdict line of its exit status and a count per state; read `ci` and other commands through this, since `check` already does - `scripts/run [unit] [--entry ] [-- args…]` — start a unit; the unit name is required when `apps/` holds several diff --git a/apps/github-repository-template/src/base-repo/scripts/ci-tested b/apps/github-repository-template/src/base-repo/scripts/ci-tested index ed0a50cc..449704c7 100755 --- a/apps/github-repository-template/src/base-repo/scripts/ci-tested +++ b/apps/github-repository-template/src/base-repo/scripts/ci-tested @@ -1,54 +1,37 @@ #!/usr/bin/env bash set -euo pipefail -# Whether a push to the default branch landed a tree this workflow already -# passed on a pull request, so ci.yml can skip its checks instead of proving the -# same tree twice. Prints `skip=true` or `skip=false` for $GITHUB_OUTPUT and the -# reason on stderr, which the job log keeps. +# Whether a passing pull request run of ci.yml already tested the tree a push to +# the default branch landed, so ci.yml can skip its checks instead of proving +# the same tree twice. Prints `skip=true` or `skip=false` for $GITHUB_OUTPUT +# and the reason on stderr, which the job log keeps. # -# scripts/ci-tested +# scripts/ci-tested # -# is the branch tip the push replaced (github.event.before). The tip -# that landed is HEAD's second parent for a merge commit, else HEAD itself, a -# fast-forward. The skip holds only when is an ancestor of that tip and -# HEAD's tree is the tip's: a pull request run tests the tip merged onto the -# base it saw, every such base is an ancestor of while the branch only -# moves forward, so that merge is the tip's own tree, the tree now on the -# branch. The same test /land uses to skip its local gate. A squash or rebase -# merge writes a commit no pull request run saw, so it never skips. +# A pull request run that passes uploads an artifact named for the tree it +# tested, its merge ref's: ci-tested-tree-. The push skips when an +# unexpired one of those names HEAD's tree, so any merge, squash or +# fast-forward whose result is a tree some run already passed skips, and one +# onto a base that moved since does not. Artifacts from a fork's run are +# ignored, since a fork's pull request can rewrite the workflow that uploads +# them. # # Called by ci.yml alone: it reads hosted run state, which no local command does. # Every doubt answers skip=false, since running the checks again costs minutes # and skipping them wrongly leaves the branch unproven. -before="${1:?usage: scripts/ci-tested }" - decide() { echo "ci-tested: $2" >&2 echo "skip=$1" exit 0 } -[[ "$before" =~ ^0+$ ]] && decide false "the push created the branch" -git cat-file -e "$before^{commit}" 2>/dev/null || - decide false "the replaced tip $before is not in this checkout" - -read -r -a parents <<< "$(git rev-list --parents -n 1 HEAD)" -tip="${parents[0]}" -(( ${#parents[@]} == 3 )) && tip="${parents[2]}" - -git merge-base --is-ancestor "$before" "$tip" || - decide false "the branch moved past the pull request's base" -[[ "$(git rev-parse "HEAD^{tree}")" == "$(git rev-parse "$tip^{tree}")" ]] || - decide false "HEAD's tree differs from the pull request head $tip" - -workflow="${GITHUB_WORKFLOW_REF:?}" -workflow="${workflow%@*}" -workflow="${workflow##*/}" -count="$(gh api \ - "repos/${GITHUB_REPOSITORY:?}/actions/workflows/$workflow/runs?event=pull_request&status=success&head_sha=$tip&per_page=1" \ - --jq .total_count 2>/dev/null)" || decide false "could not read $workflow's runs" +tree="$(git rev-parse 'HEAD^{tree}')" +count="$(gh api "repos/${GITHUB_REPOSITORY:?}/actions/artifacts?name=ci-tested-tree-$tree&per_page=100" \ + --jq '[.artifacts[] | select((.expired | not) + and .workflow_run.head_repository_id == .workflow_run.repository_id)] | length' \ + 2>/dev/null)" || decide false "could not read the tested-tree artifacts" [[ "$count" =~ ^[1-9] ]] || - decide false "no passing pull request run of $workflow on $tip" -decide true "$workflow passed on pull request head $tip, the tree now on the branch" + decide false "no passing pull request run tested tree $tree" +decide true "a passing pull request run tested tree $tree, the tree now on the branch" diff --git a/apps/github-repository-template/src/base-repo/scripts/tests/ci-tested-test b/apps/github-repository-template/src/base-repo/scripts/tests/ci-tested-test index a74701b5..88aff6bd 100755 --- a/apps/github-repository-template/src/base-repo/scripts/tests/ci-tested-test +++ b/apps/github-repository-template/src/base-repo/scripts/tests/ci-tested-test @@ -2,10 +2,9 @@ set -uo pipefail # Whether scripts/ci-tested skips the default branch's checks only when a -# passing pull request run already proved the tree that landed. Each case builds -# the push in a scratch repository -- a merge commit, a fast-forward, a moved -# base, a merge that changed the tree -- and `gh` is a stub answering the runs -# query with a fixed count, recording what it was asked. +# passing pull request run recorded the tree that landed. `gh` is a stub that +# answers the artifacts query with a fixed listing, run through the --jq filter +# it was given as gh would, and records what it was asked. repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" @@ -16,112 +15,84 @@ source "$repo_root/scripts/tests/libs/harness.sh" echo "ci-tested-test" export GITHUB_REPOSITORY=owner/repo -export GITHUB_WORKFLOW_REF=owner/repo/.github/workflows/ci.yml@refs/heads/main -# A repository with one commit on main, the tip a push would replace, and a -# branch `topic` one commit ahead of it. The gh stub answers with $2 runs, or -# fails when $2 is `error`. +# A repository with one commit, and a gh stub answering with the artifacts +# listing $2, or failing when $2 is `error`. fixture() { scratch_repo current="$1" + printf '%s\n' "$2" > "$work/listing" stub gh <> "$work/gh-calls" -[[ "$2" == error ]] && exit 1 -echo "$2" +[[ "\$(cat "$work/listing")" == error ]] && exit 1 +while (( \$# )); do + [[ "\$1" == --jq ]] && exec jq "\$2" "$work/listing" + shift +done +cat "$work/listing" STUB git -C "$work" add -A git -C "$work" commit -qm base - git -C "$work" branch -qM main - before="$(git -C "$work" rev-parse HEAD)" - git -C "$work" checkout -qb topic - echo change > "$work/file" - git -C "$work" add file - git -C "$work" commit -qm change - tip="$(git -C "$work" rev-parse HEAD)" - git -C "$work" checkout -q main + tree="$(git -C "$work" rev-parse 'HEAD^{tree}')" } decide() { - (cd "$work" && scripts/ci-tested "$1" 2>"$work/reason") + (cd "$work" && scripts/ci-tested 2>"$work/reason") } -fixture "a merge of an unmoved base onto a green head skips" 1 -git -C "$work" merge -q --no-ff -m merge topic -out="$(decide "$before")" -if [[ "$out" == "skip=true" ]] && grep -q "head_sha=$tip" "$work/gh-calls" \ - && grep -q 'workflows/ci.yml/runs?event=pull_request&status=success' "$work/gh-calls"; then - pass -else - fail "wanted skip=true from a runs query on $tip, got '$out': $(cat "$work/reason" "$work/gh-calls" 2>&1)" +# One artifact entry: expired true|false, then the head repository's id against +# the repository's 1. +artifact() { + printf '{"expired": %s, "workflow_run": {"repository_id": 1, "head_repository_id": %s}}' "$1" "$2" +} + +if ! command -v jq >/dev/null 2>&1; then + current="ci-tested" + skip "jq is not installed, and the stub runs gh's --jq filter through it" + report + exit fi -fixture "a fast-forward onto a green head skips" 1 -git -C "$work" merge -q --ff-only topic -out="$(decide "$before")" -if [[ "$out" == "skip=true" ]] && grep -q "head_sha=$tip" "$work/gh-calls"; then +fixture "a tree a same-repository run recorded skips" "{\"artifacts\": [$(artifact false 1)]}" +out="$(decide)" +if [[ "$out" == "skip=true" ]] \ + && grep -q "repos/owner/repo/actions/artifacts?name=ci-tested-tree-$tree&" "$work/gh-calls"; then pass else - fail "wanted skip=true, got '$out': $(cat "$work/reason" 2>&1)" + fail "wanted skip=true from a query for tree $tree, got '$out': $(cat "$work/reason" "$work/gh-calls" 2>&1)" fi -fixture "no passing pull request run runs the checks" 0 -git -C "$work" merge -q --no-ff -m merge topic -out="$(decide "$before")" +fixture "a tree no run recorded runs the checks" '{"artifacts": []}' +out="$(decide)" if [[ "$out" == "skip=false" ]] && grep -q 'no passing pull request run' "$work/reason"; then pass else fail "wanted skip=false, got '$out': $(cat "$work/reason" 2>&1)" fi -fixture "a base that moved runs the checks without asking GitHub" 1 -echo other > "$work/other" -git -C "$work" add other -git -C "$work" commit -qm moved -moved="$(git -C "$work" rev-parse HEAD)" -git -C "$work" merge -q --no-ff -m merge topic -out="$(decide "$moved")" -if [[ "$out" == "skip=false" ]] && [[ ! -e "$work/gh-calls" ]]; then +fixture "an expired record runs the checks" "{\"artifacts\": [$(artifact true 1)]}" +out="$(decide)" +if [[ "$out" == "skip=false" ]]; then pass else - fail "wanted skip=false and no gh call, got '$out': $(cat "$work/reason" 2>&1)" + fail "wanted skip=false for an expired artifact, got '$out': $(cat "$work/reason" 2>&1)" fi -fixture "a merge that changed the tree runs the checks" 1 -git -C "$work" merge -q --no-ff --no-commit topic >/dev/null 2>&1 -echo extra > "$work/extra" -git -C "$work" add extra -git -C "$work" commit -qm merge -out="$(decide "$before")" -if [[ "$out" == "skip=false" ]] && grep -q 'tree differs' "$work/reason"; then +fixture "a record from a fork's run runs the checks" "{\"artifacts\": [$(artifact false 2)]}" +out="$(decide)" +if [[ "$out" == "skip=false" ]]; then pass else - fail "wanted skip=false on a changed merge tree, got '$out': $(cat "$work/reason" 2>&1)" + fail "wanted skip=false for a fork's artifact, got '$out': $(cat "$work/reason" 2>&1)" fi -fixture "a failed runs query runs the checks" error -git -C "$work" merge -q --no-ff -m merge topic -out="$(decide "$before")" +fixture "a failed query runs the checks" error +out="$(decide)" if [[ "$out" == "skip=false" ]] && grep -q 'could not read' "$work/reason"; then pass else fail "wanted skip=false when gh fails, got '$out': $(cat "$work/reason" 2>&1)" fi -fixture "a push that created the branch runs the checks" 1 -out="$(decide 0000000000000000000000000000000000000000)" -if [[ "$out" == "skip=false" ]]; then - pass -else - fail "wanted skip=false for a new branch, got '$out'" -fi - -fixture "a replaced tip missing from the checkout runs the checks" 1 -out="$(decide 1111111111111111111111111111111111111111)" -if [[ "$out" == "skip=false" ]] && grep -q 'not in this checkout' "$work/reason"; then - pass -else - fail "wanted skip=false for an unknown tip, got '$out': $(cat "$work/reason" 2>&1)" -fi - report diff --git a/scripts/ci-tested b/scripts/ci-tested index ed0a50cc..449704c7 100755 --- a/scripts/ci-tested +++ b/scripts/ci-tested @@ -1,54 +1,37 @@ #!/usr/bin/env bash set -euo pipefail -# Whether a push to the default branch landed a tree this workflow already -# passed on a pull request, so ci.yml can skip its checks instead of proving the -# same tree twice. Prints `skip=true` or `skip=false` for $GITHUB_OUTPUT and the -# reason on stderr, which the job log keeps. +# Whether a passing pull request run of ci.yml already tested the tree a push to +# the default branch landed, so ci.yml can skip its checks instead of proving +# the same tree twice. Prints `skip=true` or `skip=false` for $GITHUB_OUTPUT +# and the reason on stderr, which the job log keeps. # -# scripts/ci-tested +# scripts/ci-tested # -# is the branch tip the push replaced (github.event.before). The tip -# that landed is HEAD's second parent for a merge commit, else HEAD itself, a -# fast-forward. The skip holds only when is an ancestor of that tip and -# HEAD's tree is the tip's: a pull request run tests the tip merged onto the -# base it saw, every such base is an ancestor of while the branch only -# moves forward, so that merge is the tip's own tree, the tree now on the -# branch. The same test /land uses to skip its local gate. A squash or rebase -# merge writes a commit no pull request run saw, so it never skips. +# A pull request run that passes uploads an artifact named for the tree it +# tested, its merge ref's: ci-tested-tree-. The push skips when an +# unexpired one of those names HEAD's tree, so any merge, squash or +# fast-forward whose result is a tree some run already passed skips, and one +# onto a base that moved since does not. Artifacts from a fork's run are +# ignored, since a fork's pull request can rewrite the workflow that uploads +# them. # # Called by ci.yml alone: it reads hosted run state, which no local command does. # Every doubt answers skip=false, since running the checks again costs minutes # and skipping them wrongly leaves the branch unproven. -before="${1:?usage: scripts/ci-tested }" - decide() { echo "ci-tested: $2" >&2 echo "skip=$1" exit 0 } -[[ "$before" =~ ^0+$ ]] && decide false "the push created the branch" -git cat-file -e "$before^{commit}" 2>/dev/null || - decide false "the replaced tip $before is not in this checkout" - -read -r -a parents <<< "$(git rev-list --parents -n 1 HEAD)" -tip="${parents[0]}" -(( ${#parents[@]} == 3 )) && tip="${parents[2]}" - -git merge-base --is-ancestor "$before" "$tip" || - decide false "the branch moved past the pull request's base" -[[ "$(git rev-parse "HEAD^{tree}")" == "$(git rev-parse "$tip^{tree}")" ]] || - decide false "HEAD's tree differs from the pull request head $tip" - -workflow="${GITHUB_WORKFLOW_REF:?}" -workflow="${workflow%@*}" -workflow="${workflow##*/}" -count="$(gh api \ - "repos/${GITHUB_REPOSITORY:?}/actions/workflows/$workflow/runs?event=pull_request&status=success&head_sha=$tip&per_page=1" \ - --jq .total_count 2>/dev/null)" || decide false "could not read $workflow's runs" +tree="$(git rev-parse 'HEAD^{tree}')" +count="$(gh api "repos/${GITHUB_REPOSITORY:?}/actions/artifacts?name=ci-tested-tree-$tree&per_page=100" \ + --jq '[.artifacts[] | select((.expired | not) + and .workflow_run.head_repository_id == .workflow_run.repository_id)] | length' \ + 2>/dev/null)" || decide false "could not read the tested-tree artifacts" [[ "$count" =~ ^[1-9] ]] || - decide false "no passing pull request run of $workflow on $tip" -decide true "$workflow passed on pull request head $tip, the tree now on the branch" + decide false "no passing pull request run tested tree $tree" +decide true "a passing pull request run tested tree $tree, the tree now on the branch" diff --git a/scripts/tests/ci-tested-test b/scripts/tests/ci-tested-test index a74701b5..88aff6bd 100755 --- a/scripts/tests/ci-tested-test +++ b/scripts/tests/ci-tested-test @@ -2,10 +2,9 @@ set -uo pipefail # Whether scripts/ci-tested skips the default branch's checks only when a -# passing pull request run already proved the tree that landed. Each case builds -# the push in a scratch repository -- a merge commit, a fast-forward, a moved -# base, a merge that changed the tree -- and `gh` is a stub answering the runs -# query with a fixed count, recording what it was asked. +# passing pull request run recorded the tree that landed. `gh` is a stub that +# answers the artifacts query with a fixed listing, run through the --jq filter +# it was given as gh would, and records what it was asked. repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" @@ -16,112 +15,84 @@ source "$repo_root/scripts/tests/libs/harness.sh" echo "ci-tested-test" export GITHUB_REPOSITORY=owner/repo -export GITHUB_WORKFLOW_REF=owner/repo/.github/workflows/ci.yml@refs/heads/main -# A repository with one commit on main, the tip a push would replace, and a -# branch `topic` one commit ahead of it. The gh stub answers with $2 runs, or -# fails when $2 is `error`. +# A repository with one commit, and a gh stub answering with the artifacts +# listing $2, or failing when $2 is `error`. fixture() { scratch_repo current="$1" + printf '%s\n' "$2" > "$work/listing" stub gh <> "$work/gh-calls" -[[ "$2" == error ]] && exit 1 -echo "$2" +[[ "\$(cat "$work/listing")" == error ]] && exit 1 +while (( \$# )); do + [[ "\$1" == --jq ]] && exec jq "\$2" "$work/listing" + shift +done +cat "$work/listing" STUB git -C "$work" add -A git -C "$work" commit -qm base - git -C "$work" branch -qM main - before="$(git -C "$work" rev-parse HEAD)" - git -C "$work" checkout -qb topic - echo change > "$work/file" - git -C "$work" add file - git -C "$work" commit -qm change - tip="$(git -C "$work" rev-parse HEAD)" - git -C "$work" checkout -q main + tree="$(git -C "$work" rev-parse 'HEAD^{tree}')" } decide() { - (cd "$work" && scripts/ci-tested "$1" 2>"$work/reason") + (cd "$work" && scripts/ci-tested 2>"$work/reason") } -fixture "a merge of an unmoved base onto a green head skips" 1 -git -C "$work" merge -q --no-ff -m merge topic -out="$(decide "$before")" -if [[ "$out" == "skip=true" ]] && grep -q "head_sha=$tip" "$work/gh-calls" \ - && grep -q 'workflows/ci.yml/runs?event=pull_request&status=success' "$work/gh-calls"; then - pass -else - fail "wanted skip=true from a runs query on $tip, got '$out': $(cat "$work/reason" "$work/gh-calls" 2>&1)" +# One artifact entry: expired true|false, then the head repository's id against +# the repository's 1. +artifact() { + printf '{"expired": %s, "workflow_run": {"repository_id": 1, "head_repository_id": %s}}' "$1" "$2" +} + +if ! command -v jq >/dev/null 2>&1; then + current="ci-tested" + skip "jq is not installed, and the stub runs gh's --jq filter through it" + report + exit fi -fixture "a fast-forward onto a green head skips" 1 -git -C "$work" merge -q --ff-only topic -out="$(decide "$before")" -if [[ "$out" == "skip=true" ]] && grep -q "head_sha=$tip" "$work/gh-calls"; then +fixture "a tree a same-repository run recorded skips" "{\"artifacts\": [$(artifact false 1)]}" +out="$(decide)" +if [[ "$out" == "skip=true" ]] \ + && grep -q "repos/owner/repo/actions/artifacts?name=ci-tested-tree-$tree&" "$work/gh-calls"; then pass else - fail "wanted skip=true, got '$out': $(cat "$work/reason" 2>&1)" + fail "wanted skip=true from a query for tree $tree, got '$out': $(cat "$work/reason" "$work/gh-calls" 2>&1)" fi -fixture "no passing pull request run runs the checks" 0 -git -C "$work" merge -q --no-ff -m merge topic -out="$(decide "$before")" +fixture "a tree no run recorded runs the checks" '{"artifacts": []}' +out="$(decide)" if [[ "$out" == "skip=false" ]] && grep -q 'no passing pull request run' "$work/reason"; then pass else fail "wanted skip=false, got '$out': $(cat "$work/reason" 2>&1)" fi -fixture "a base that moved runs the checks without asking GitHub" 1 -echo other > "$work/other" -git -C "$work" add other -git -C "$work" commit -qm moved -moved="$(git -C "$work" rev-parse HEAD)" -git -C "$work" merge -q --no-ff -m merge topic -out="$(decide "$moved")" -if [[ "$out" == "skip=false" ]] && [[ ! -e "$work/gh-calls" ]]; then +fixture "an expired record runs the checks" "{\"artifacts\": [$(artifact true 1)]}" +out="$(decide)" +if [[ "$out" == "skip=false" ]]; then pass else - fail "wanted skip=false and no gh call, got '$out': $(cat "$work/reason" 2>&1)" + fail "wanted skip=false for an expired artifact, got '$out': $(cat "$work/reason" 2>&1)" fi -fixture "a merge that changed the tree runs the checks" 1 -git -C "$work" merge -q --no-ff --no-commit topic >/dev/null 2>&1 -echo extra > "$work/extra" -git -C "$work" add extra -git -C "$work" commit -qm merge -out="$(decide "$before")" -if [[ "$out" == "skip=false" ]] && grep -q 'tree differs' "$work/reason"; then +fixture "a record from a fork's run runs the checks" "{\"artifacts\": [$(artifact false 2)]}" +out="$(decide)" +if [[ "$out" == "skip=false" ]]; then pass else - fail "wanted skip=false on a changed merge tree, got '$out': $(cat "$work/reason" 2>&1)" + fail "wanted skip=false for a fork's artifact, got '$out': $(cat "$work/reason" 2>&1)" fi -fixture "a failed runs query runs the checks" error -git -C "$work" merge -q --no-ff -m merge topic -out="$(decide "$before")" +fixture "a failed query runs the checks" error +out="$(decide)" if [[ "$out" == "skip=false" ]] && grep -q 'could not read' "$work/reason"; then pass else fail "wanted skip=false when gh fails, got '$out': $(cat "$work/reason" 2>&1)" fi -fixture "a push that created the branch runs the checks" 1 -out="$(decide 0000000000000000000000000000000000000000)" -if [[ "$out" == "skip=false" ]]; then - pass -else - fail "wanted skip=false for a new branch, got '$out'" -fi - -fixture "a replaced tip missing from the checkout runs the checks" 1 -out="$(decide 1111111111111111111111111111111111111111)" -if [[ "$out" == "skip=false" ]] && grep -q 'not in this checkout' "$work/reason"; then - pass -else - fail "wanted skip=false for an unknown tip, got '$out': $(cat "$work/reason" 2>&1)" -fi - report From 0d4046162fef987e336dd8644ce4c05efe03cfa9 Mon Sep 17 00:00:00 2001 From: neal Date: Wed, 18 Nov 2026 23:07:53 -0500 Subject: [PATCH 4/4] fix(ci): ignore a tested-tree artifact with no run behind it The API allows an artifact's workflow_run to be null, and then the fork filter compared null with null and counted it as a same-repository pass. It now requires a repository id, so an artifact of unknown origin answers skip=false like every other doubt. The header also says what the artifact name does not prove: a branch in this repository can upload one without running the checks. That is trusted because whoever can push such a branch can already push to the default branch. The previous commit said a hosted runner costs the operator nothing for the second run. On a private repository it uses the plan's minutes; hosted runs keep the double run to keep main's caches warm. Generated-By: claude-opus-5-5 --- .../src/base-repo/scripts/ci-tested | 10 ++++++---- .../src/base-repo/scripts/tests/ci-tested-test | 8 ++++++++ scripts/ci-tested | 10 ++++++---- scripts/tests/ci-tested-test | 8 ++++++++ 4 files changed, 28 insertions(+), 8 deletions(-) diff --git a/apps/github-repository-template/src/base-repo/scripts/ci-tested b/apps/github-repository-template/src/base-repo/scripts/ci-tested index 449704c7..e806a294 100755 --- a/apps/github-repository-template/src/base-repo/scripts/ci-tested +++ b/apps/github-repository-template/src/base-repo/scripts/ci-tested @@ -12,9 +12,11 @@ set -euo pipefail # tested, its merge ref's: ci-tested-tree-. The push skips when an # unexpired one of those names HEAD's tree, so any merge, squash or # fast-forward whose result is a tree some run already passed skips, and one -# onto a base that moved since does not. Artifacts from a fork's run are -# ignored, since a fork's pull request can rewrite the workflow that uploads -# them. +# onto a base that moved since does not. Only the name says the run passed, and +# a pull request can rewrite the workflow that uploads it, so artifacts from a +# fork's run, or with no run at all, are ignored. A branch in this repository +# is trusted the same way: whoever can push one can already push to the +# default branch. # # Called by ci.yml alone: it reads hosted run state, which no local command does. # Every doubt answers skip=false, since running the checks again costs minutes @@ -28,7 +30,7 @@ decide() { tree="$(git rev-parse 'HEAD^{tree}')" count="$(gh api "repos/${GITHUB_REPOSITORY:?}/actions/artifacts?name=ci-tested-tree-$tree&per_page=100" \ - --jq '[.artifacts[] | select((.expired | not) + --jq '[.artifacts[] | select((.expired | not) and .workflow_run.repository_id != null and .workflow_run.head_repository_id == .workflow_run.repository_id)] | length' \ 2>/dev/null)" || decide false "could not read the tested-tree artifacts" diff --git a/apps/github-repository-template/src/base-repo/scripts/tests/ci-tested-test b/apps/github-repository-template/src/base-repo/scripts/tests/ci-tested-test index 88aff6bd..43cf9d61 100755 --- a/apps/github-repository-template/src/base-repo/scripts/tests/ci-tested-test +++ b/apps/github-repository-template/src/base-repo/scripts/tests/ci-tested-test @@ -87,6 +87,14 @@ else fail "wanted skip=false for a fork's artifact, got '$out': $(cat "$work/reason" 2>&1)" fi +fixture "a record with no run runs the checks" '{"artifacts": [{"expired": false, "workflow_run": null}]}' +out="$(decide)" +if [[ "$out" == "skip=false" ]]; then + pass +else + fail "wanted skip=false for an artifact with no run, got '$out': $(cat "$work/reason" 2>&1)" +fi + fixture "a failed query runs the checks" error out="$(decide)" if [[ "$out" == "skip=false" ]] && grep -q 'could not read' "$work/reason"; then diff --git a/scripts/ci-tested b/scripts/ci-tested index 449704c7..e806a294 100755 --- a/scripts/ci-tested +++ b/scripts/ci-tested @@ -12,9 +12,11 @@ set -euo pipefail # tested, its merge ref's: ci-tested-tree-. The push skips when an # unexpired one of those names HEAD's tree, so any merge, squash or # fast-forward whose result is a tree some run already passed skips, and one -# onto a base that moved since does not. Artifacts from a fork's run are -# ignored, since a fork's pull request can rewrite the workflow that uploads -# them. +# onto a base that moved since does not. Only the name says the run passed, and +# a pull request can rewrite the workflow that uploads it, so artifacts from a +# fork's run, or with no run at all, are ignored. A branch in this repository +# is trusted the same way: whoever can push one can already push to the +# default branch. # # Called by ci.yml alone: it reads hosted run state, which no local command does. # Every doubt answers skip=false, since running the checks again costs minutes @@ -28,7 +30,7 @@ decide() { tree="$(git rev-parse 'HEAD^{tree}')" count="$(gh api "repos/${GITHUB_REPOSITORY:?}/actions/artifacts?name=ci-tested-tree-$tree&per_page=100" \ - --jq '[.artifacts[] | select((.expired | not) + --jq '[.artifacts[] | select((.expired | not) and .workflow_run.repository_id != null and .workflow_run.head_repository_id == .workflow_run.repository_id)] | length' \ 2>/dev/null)" || decide false "could not read the tested-tree artifacts" diff --git a/scripts/tests/ci-tested-test b/scripts/tests/ci-tested-test index 88aff6bd..43cf9d61 100755 --- a/scripts/tests/ci-tested-test +++ b/scripts/tests/ci-tested-test @@ -87,6 +87,14 @@ else fail "wanted skip=false for a fork's artifact, got '$out': $(cat "$work/reason" 2>&1)" fi +fixture "a record with no run runs the checks" '{"artifacts": [{"expired": false, "workflow_run": null}]}' +out="$(decide)" +if [[ "$out" == "skip=false" ]]; then + pass +else + fail "wanted skip=false for an artifact with no run, got '$out': $(cat "$work/reason" 2>&1)" +fi + fixture "a failed query runs the checks" error out="$(decide)" if [[ "$out" == "skip=false" ]] && grep -q 'could not read' "$work/reason"; then