Skip to content

fix(tools): fail closed where errexit is off - #71

Merged
VizzleTF merged 5 commits into
mainfrom
fix/errexit-contexts
Sep 14, 2026
Merged

VizzleTF merged 5 commits into
mainfrom
fix/errexit-contexts

Conversation

@VizzleTF

@VizzleTF VizzleTF commented Sep 14, 2026

Copy link
Copy Markdown
Member

POSIX turns errexit off for everything run as an operand of ||/&&, as an if condition, or after !. The same is true for functions and subshells run there. An audit listed seven places where that turns a failed read into a harmless-looking answer. Each one was checked against the code and reproduced in dash and in sh before fixing. All seven are real.

Defects

# where reproduced by consequence fix
1 land-updates.sh land (gh pr list) new test: stub gh pr list exits 1. Before the fix the branch landed on main an untrusted update waiting for a person is fast-forwarded and published if ! pr="$(gh …)" → message + return 1 (not landed)
2 land-updates.sh land (git fetch main / branch) new test: stub git fails fetch. Before the fix (tested alone) the branch landed ancestry, moved-branch and path checks run against stale refs each fetch checked → message + return 1
3 land-updates.sh superseded (git diff --name-only) new test: stub git fails diff --name-only. Before the fix the log says "adds nothing … deleting it" (tested alone) a live update branch is deleted checked; the error goes to stderr and it returns empty, which means keep
4 check-updates.sh binaries shape awk … > new && mv dash/sh snippet: a failing left side of && does not trigger errexit in the set -e subshell. The subshell carried on to commit and exited 0 VERSION/TAG rewritten, checksums stale, then committed, pushed and dispatched awk … || { message; exit 1; }, then mv
5 check-updates.sh may_automerge (git log | wc -l, git diff | grep … || true) new tests: stub git fails log / diff. Before the fix: "pushed, no pull request" (automerge) the daily ceiling is bypassed, or "only pins moved" is assumed. Either way the update merges unattended capture the git output first; on failure print a message and return 1 (pull request)
6 sources.sh find -exec jq {} + 2>/dev/null | sort -u new test: one unparsable index.json next to a readable one. Before the fix the publish passed a copyleft package in the broken index skips the source gate write to a temp file, check find's status (non-zero when an -exec {} + run fails, both GNU and BSD), then sort
7 check-origin.sh find -exec jq/awk … 2>/dev/null || true manual fixture: broken index.json. Before the fix: "every published package names its upstream", exit 0 those packages are never origin-checked masking removed; a failure prints the command and exits 1. A healthy apk-only tree (no Packages) still passes

Addition A: a failed gh release view stops the package

check-updates.sh used gh release view … 2>/dev/null || true. A 401, a 502 or a network error therefore read as "upstream has no releases", and the run stayed green without checking the package.

Measured with gh 2.99.0 — every case exits 1:

case output
repo with no releases (octocat/Hello-World) release not found
repo that does not exist release not found
bad token HTTP 401: Bad credentials (…)
unreachable proxy Get "…": proxyconnect tcp: … connection refused

Fix: only the exact text release not found counts as no releases, and only if gh api repos/<repo>/releases?per_page=1 also succeeds. That call fails for a missing repository and succeeds for an empty one. Any other failure prints the gh error and the commands, then exit 1 through the existing per-package path (check stopped for: and a non-zero run). Other packages are still checked.

Tests: test-check-updates.sh adds three packages:

  • b0-outage (HTTP 502) stops.
  • b1-missing (not found, and the repo is a 404) stops.
  • f-norelease (not found, repo exists) stays green.

Against the previous script the new assertions fail: said what it should not: b0-outage: upstream has no releases, same for b1-missing, and never said: check stopped for: a0-unfetchable b0-outage b1-missing.

Addition B: land-updates.sh goes red when a branch failed to land

Before, land … || echo left the scheduled job green when a read failed. Now the loop collects the branches whose land returned 1, keeps processing the rest, and at the end prints not landed because a step failed: … and exits 1.

  • Ordinary outcomes still return 0. Checked in the code and by tests: checks pending, pull request open, main moved ahead (not a fast-forward), superseded or deleted, branch moved, path gate refused, push refused. Only the explicit read/command failures return 1.
  • Loop mechanics. The loop now reads a here-document, so failed survives: a pipeline would run it in a subshell. land gets </dev/null, so a gh or git call inside cannot swallow the branch list.
  • update.yml. check (needs: land) and publish (needs: check) already use if: ${{ !cancelled() }}, so a red land job skips neither. The comment on the land job now explains why that makes red safe and why continue-on-error was not used: it would show green for an hour in which nothing could land.

Tests: test-land-updates.sh checks the exit status on every run.

  • Runs 1, 2, 7 (checks pending) and 8 (PR open) must exit 0.
  • Runs 3–5 (gh pr list / git diff / git fetch failing) must be non-zero.
  • Run 6: gh pr list fails for the first branch only. The run is red, names that branch, and the next branch is still read and lands.

Against the previous script: FAIL the run exited 0 although … in runs 3–6. Runs 1, 2, 7 and 8 were already exit 0, which confirms the ordinary outcomes returned 0 before this change.

Sweep of the rest of tools/

  • check-updates.sh: sha256sum inside a sed/printf argument hides its failure behind that command's status, so an empty checksum could be committed. The sum is now captured into a variable first.
  • land-updates.sh: a failed gh api …/check-runs read and the path-gate git diff both already fell the safe way ("no run" / "nothing to land"), but they logged a false reason. Both now name the failed command and return.
  • Checked and left alone because they already fail closed or are deliberate:
    • fetch.sh: download compares an empty sum against the pin, and if ! get is intended.
    • check-tree.sh.
    • intake-check.sh: a report, where a failed read lands in "Not ready".
    • git ls-remote/git show … || true in check-updates.sh: documented as "costs a rebuild".

The gh release view … || true and the green land … || echo were first left as documented intent. Additions A and B above change both.

Tests

land runs as the left side of '|| echo', so POSIX disables errexit for
every command in it. A failing 'gh pr list' left pr empty and a branch
whose pull request was waiting for a person was fast-forwarded onto main.
A failing 'git diff --name-only' in superseded read as 'adds nothing' and
deleted the branch; failing fetches continued on stale refs.

Each read now checks its own status and refuses to land (or keeps the
branch). test-land-updates.sh covers gh pr list, git diff and git fetch
failing, and lands the same branch once nothing fails.
may_automerge runs as an if condition with errexit off. 'git log | wc -l'
counted a failed git as 0 recent updates, and a failed 'git diff' piped
through '|| true' read as 'only pins moved'; both allowed automerge.
Capture git output first and return 1 on failure.

In the binaries shape 'awk ... && mv' exempted a failing awk from errexit
and committed VERSION/TAG with stale checksums; exit instead. Take
sha256sum into a variable so its failure is not hidden inside sed/printf.
sources.sh piped 'find -exec jq' into sort with stderr discarded, and
check-origin.sh appended with '2>/dev/null || true'. A jq or awk failure
on one index silently dropped its packages from the copyleft-source and
origin checks while the rest passed. Both now write to a file, check
find's status and report the failed command.
'gh release view ... 2>/dev/null || true' read a 401, a 502 or a network
error as 'upstream has no releases' and kept the run green. gh exits 1
for all of them; a repository with no releases and a missing repository
both print exactly 'release not found' (measured, gh 2.99.0). Treat only
that text as no releases, confirmed by 'gh api repos/<repo>/releases'
succeeding; anything else stops the package through the existing
per-package failure path.
A step that could not run (return 1 from land) cost that branch but left
the scheduled run green. Collect those branches and exit 1 after every
branch has been read. Ordinary outcomes (checks pending, pull request
open, not a fast-forward, superseded) still return 0. The check and
publish jobs already run on '!cancelled()', so a red land job does not
skip them.
@VizzleTF
VizzleTF merged commit 3ee8865 into main Sep 14, 2026
4 checks passed
@VizzleTF
VizzleTF deleted the fix/errexit-contexts branch September 14, 2026 10:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant