fix(tools): fail closed where errexit is off - #71
Merged
Merged
Conversation
land runs as the left side of '|| echo', so POSIX disables errexit for every command in it. A failing 'gh pr list' left pr empty and a branch whose pull request was waiting for a person was fast-forwarded onto main. A failing 'git diff --name-only' in superseded read as 'adds nothing' and deleted the branch; failing fetches continued on stale refs. Each read now checks its own status and refuses to land (or keeps the branch). test-land-updates.sh covers gh pr list, git diff and git fetch failing, and lands the same branch once nothing fails.
may_automerge runs as an if condition with errexit off. 'git log | wc -l' counted a failed git as 0 recent updates, and a failed 'git diff' piped through '|| true' read as 'only pins moved'; both allowed automerge. Capture git output first and return 1 on failure. In the binaries shape 'awk ... && mv' exempted a failing awk from errexit and committed VERSION/TAG with stale checksums; exit instead. Take sha256sum into a variable so its failure is not hidden inside sed/printf.
sources.sh piped 'find -exec jq' into sort with stderr discarded, and check-origin.sh appended with '2>/dev/null || true'. A jq or awk failure on one index silently dropped its packages from the copyleft-source and origin checks while the rest passed. Both now write to a file, check find's status and report the failed command.
'gh release view ... 2>/dev/null || true' read a 401, a 502 or a network error as 'upstream has no releases' and kept the run green. gh exits 1 for all of them; a repository with no releases and a missing repository both print exactly 'release not found' (measured, gh 2.99.0). Treat only that text as no releases, confirmed by 'gh api repos/<repo>/releases' succeeding; anything else stops the package through the existing per-package failure path.
A step that could not run (return 1 from land) cost that branch but left the scheduled run green. Collect those branches and exit 1 after every branch has been read. Ordinary outcomes (checks pending, pull request open, not a fast-forward, superseded) still return 0. The check and publish jobs already run on '!cancelled()', so a red land job does not skip them.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
POSIX turns errexit off for everything run as an operand of
||/&&, as anifcondition, or after!. The same is true for functions and subshells run there. An audit listed seven places where that turns a failed read into a harmless-looking answer. Each one was checked against the code and reproduced in dash and in sh before fixing. All seven are real.Defects
land-updates.shland(gh pr list)gh pr listexits 1. Before the fix the branch landed on mainif ! pr="$(gh …)"→ message +return 1(not landed)land-updates.shland(git fetchmain / branch)fetch. Before the fix (tested alone) the branch landedreturn 1land-updates.shsuperseded(git diff --name-only)diff --name-only. Before the fix the log says "adds nothing … deleting it" (tested alone)check-updates.shbinaries shapeawk … > new && mv&&does not trigger errexit in theset -esubshell. The subshell carried on to commit and exited 0awk … || { message; exit 1; }, thenmvcheck-updates.shmay_automerge(git log | wc -l,git diff | grep … || true)log/diff. Before the fix: "pushed, no pull request" (automerge)return 1(pull request)sources.shfind -exec jq {} + 2>/dev/null | sort -uindex.jsonnext to a readable one. Before the fix the publish passedfind's status (non-zero when an-exec {} +run fails, both GNU and BSD), then sortcheck-origin.shfind -exec jq/awk … 2>/dev/null || trueindex.json. Before the fix: "every published package names its upstream", exit 0Packages) still passesAddition A: a failed
gh release viewstops the packagecheck-updates.shusedgh release view … 2>/dev/null || true. A 401, a 502 or a network error therefore read as "upstream has no releases", and the run stayed green without checking the package.Measured with gh 2.99.0 — every case exits 1:
octocat/Hello-World)release not foundrelease not foundHTTP 401: Bad credentials (…)Get "…": proxyconnect tcp: … connection refusedFix: only the exact text
release not foundcounts as no releases, and only ifgh api repos/<repo>/releases?per_page=1also succeeds. That call fails for a missing repository and succeeds for an empty one. Any other failure prints the gh error and the commands, thenexit 1through the existing per-package path (check stopped for:and a non-zero run). Other packages are still checked.Tests:
test-check-updates.shadds three packages:b0-outage(HTTP 502) stops.b1-missing(not found, and the repo is a 404) stops.f-norelease(not found, repo exists) stays green.Against the previous script the new assertions fail:
said what it should not: b0-outage: upstream has no releases, same forb1-missing, andnever said: check stopped for: a0-unfetchable b0-outage b1-missing.Addition B:
land-updates.shgoes red when a branch failed to landBefore,
land … || echoleft the scheduled job green when a read failed. Now the loop collects the branches whoselandreturned 1, keeps processing the rest, and at the end printsnot landed because a step failed: …and exits 1.failedsurvives: a pipeline would run it in a subshell.landgets</dev/null, so aghorgitcall inside cannot swallow the branch list.update.yml.check(needs: land) andpublish(needs: check) already useif: ${{ !cancelled() }}, so a redlandjob skips neither. The comment on the land job now explains why that makes red safe and whycontinue-on-errorwas not used: it would show green for an hour in which nothing could land.Tests:
test-land-updates.shchecks the exit status on every run.gh pr listfails for the first branch only. The run is red, names that branch, and the next branch is still read and lands.Against the previous script:
FAIL the run exited 0 although …in runs 3–6. Runs 1, 2, 7 and 8 were already exit 0, which confirms the ordinary outcomes returned 0 before this change.Sweep of the rest of
tools/check-updates.sh:sha256suminside ased/printfargument hides its failure behind that command's status, so an empty checksum could be committed. The sum is now captured into a variable first.land-updates.sh: a failedgh api …/check-runsread and the path-gategit diffboth already fell the safe way ("no run" / "nothing to land"), but they logged a false reason. Both now name the failed command and return.fetch.sh:downloadcompares an empty sum against the pin, andif ! getis intended.check-tree.sh.intake-check.sh: a report, where a failed read lands in "Not ready".git ls-remote/git show … || trueincheck-updates.sh: documented as "costs a rebuild".The
gh release view … || trueand the greenland … || echowere first left as documented intent. Additions A and B above change both.Tests
dash -n tools/*.sh: clean.shellcheck -s sh tools/*.sh: only SC1090/SC2153, identical onmain.shanddash×test-sources.sh,test-land-updates.sh,test-check-updates.sh: all PASS.main(LAND_UPDATES=,CHECK_UPDATES=,SOURCES=), every new case fails. For land-updates, Add luci-app-footstrap-updater; smoke-test packages; watch upstreams hourly #2 and Verify authors' signatures, make a package a file of values, document both contribution flows #3 were also run alone, against the fixed script with only that one fix reverted:FAIL main moved/never said: could not read its pull requestsFAIL update/gamma-1.2.0 was deleted/main movedFAIL never said: cannot tell whatFAIL said what it should not: … pushed, no pull request(log and diff)FAIL unreadable index: published past an index.json it could not parsetest-land-updates.shlands the same branch once nothing fails. That shows the kept branch was landable, so the failure is what kept it.