Skip to content

release: make verify-release-version.sh refuse a silent artifact skip - #1009

Open
maxy-player wants to merge 2 commits into
MakePrisms:mainfrom
maxy-player:w-verify-release-version-hardening
Open

maxy-player wants to merge 2 commits into
MakePrisms:mainfrom
maxy-player:w-verify-release-version-hardening

Conversation

@maxy-player

@maxy-player maxy-player commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

The defect

scripts/verify-release-version.sh <version> with no binary path skipped its entire artifact arm —
the if [ -n "$BINARY" ] guard — and still printed PASS: everything states <version> and exited 0.
The three checks that vanish are the ones binding the artifact to the tree: --version and the
version subcommand both reporting maxplayer <version> (<sha>), and the build stamp resolving to a
commit that is this tree's HEAD (#818).

The evidence: the same command, old script and new

The control is the old script against the new one, run side by side with identical arguments:

invocation on main today on this branch
verify-release-version.sh 0.5.9 exit 0, 4 ok: lines, prints PASS: everything states 0.5.9 — 4 of the 7 checks absent, and nothing says so exit 1, naming the artifact arm that did not run
verify-release-version.sh 0.5.9 <binary> exit 0, 7 ok: lines exit 0, 7 ok: lines — unchanged

So the old run reports success with three of its seven checks — the whole artifact arm — never
executed, and a reader sees PASS. That is the failure this PR closes. A seven-assertion self-check
written for it is RED against the script as it stands on main (4 assertions fail) and GREEN
at this head; a green-only run would have the same shape as the bug.

The change

A binary path is required unless the caller opts out by name, with the flag this repo already uses
for exactly this decision — verify-release-surface.sh --no-artifacts:

invocation result
<version> <binary> unchanged: both arms, 7 ok: lines, exit 0
<version> --no-artifacts tree arm only, exit 0, prints ok: skipping the built-artifact check (--no-artifacts) — … where the artifact check would have run
<version> exit 1, naming the arm that did not run and how to opt out
<version> <binary> --no-artifacts exit 1, contradictory
no arguments exit 1, usage — unchanged

verify-release-surface.sh:101 prints
ok: skipping the built-artifact check (--no-artifacts) — holding the tree surfaces only; this
script now prints the same sentence with its own tail, at the point of skip, never silently. The
summary states which arms ran, so the exit code is not the only evidence:

PASS: everything states 0.5.9 — arms run: tree (crate version, npm manifests, payload pins, release notes) + built artifact (target/debug/maxplayer)
PASS (NO ARTIFACTS): the tree states 0.5.9 — arms run: tree (...); the built-artifact check did NOT run (--no-artifacts)

Arguments are parsed in a loop, so the flag, -h/--help and unknown -* options are handled rather
than silently taken as a version or a path. The refusal happens before the tree checks, so a run that
cannot be complete says so immediately. The usage comment block is rewritten, and RELEASE.md — which
carried no invocation text at all — now states the two arms, the <version> <path-to-binary> form,
the no-leading-v rule, and the --no-artifacts opt-out.

Two commits: the refusal and the arms-named summary, then the rename of the opt-out from a locally
invented --tree-only to the house --no-artifacts.

Caller sweep: every mention in the repo, and what changed

grep -rn verify-release-version over the tree, with the disposition of each hit. Only RELEASE.md
is edited; git diff covers scripts/verify-release-version.sh and RELEASE.md and nothing else.

Real callers — both read back UNEDITED, both already two-arm. Each passes a binary path, so each
keeps its current behaviour and colour under the stricter script, and each passes the version without
a leading v:

  • .github/workflows/ci.yml:32./scripts/verify-release-version.sh "$VERSION" target/debug/maxplayer, with $VERSION read from cargo metadata. Unchanged.
  • .github/workflows/release.yml:180-181./scripts/verify-release-version.sh '${{ needs.plan.outputs.version }}' \ + "target/${{ matrix.target }}/release/maxplayer", with plan.outputs.version being the tag with v already stripped. Unchanged.

Mentions that invoke nothing — all left alone, because each describes what the gate asserts rather
than how it is called, and none of them states an invocation that the new argument handling would
invalidate:

  • .github/workflows/release.yml:79 — a comment on what the crate/manifest/pin comparison covers. Unchanged.
  • .github/workflows/publish-sandbox-image.yml:118 — a comment noting this gate already holds the tag to the crate version, while that job holds something else. Unchanged.
  • scripts/package-release-asset.sh:22 — a comment on the artifact answering to the name it ships as. Unchanged.
  • RELEASE.md:193 — the existing sentence naming this script as what asserts the payload pin. Unchanged.

Changed: RELEASE.md gains one paragraph, since the file carried no invocation text at all — it
now states the two arms, the <version> <path-to-binary> form, the no-leading-v rule, and the
opt-out.

Gates at this head, exit codes captured

gate rc ok:
verify-release-version.sh 0.5.9 target/debug/maxplayer 0 7
verify-release-version.sh 0.5.9 1 0
verify-release-version.sh 0.5.9 --no-artifacts 0 5 (4 tree + the printed skip)
verify-release-surface.sh --no-artifacts 0 4
verify-release-workflow.sh 0 11
cargo build --bin maxplayer 0

Negative controls, each required to go red: artifact of the wrong version (rc 1); the real artifact
against a moved HEAD, i.e. a stale build stamp (rc 1, dies on the #818 stamp/HEAD comparison);
--no-artifacts with a path (rc 1); leading v (rc 1); wrong version under the opt-out (rc 1);
unknown option (rc 1); path to nothing (rc 1); the retired --tree-only (rc 1). bash -n clean;
shellcheck is not installed on the machine this was run on.

`verify-release-version.sh <version>` with no binary path skipped its whole
artifact arm — the `if [ -n "$BINARY" ]` guard — and still printed
`PASS: everything states <version>` and exited 0. A full run prints 7 `ok:`
lines; the skipping run printed 4, and nothing in the output or the exit code
said the teeth were absent. Both real call sites (ci.yml:32,
release.yml:180-181) pass a path, so the exposure was every manual or agent
invocation — which is how the skip was found.

The binary path is now required unless the caller opts out in writing:

  <version> <binary>     unchanged behaviour, summary names both arms
  <version> --tree-only  tree checks run, one loud ARTIFACT ARM NOT RUN line,
                         PASS (TREE ONLY) summary, exit 0
  <version>              exit 1 naming the arm that did not run
  <version> <binary> --tree-only   exit 1, contradictory

The final line always states which arms ran, so the exit code is not the only
evidence. Usage block and RELEASE.md updated; neither workflow call site is
touched.
@vercel

vercel Bot commented Sep 16, 2026

Copy link
Copy Markdown

Someone is attempting to deploy a commit to the MakePrisms Team on Vercel.

A member of the Team first needs to authorize it.

…sh does

The opt-out this script grew for its artifact arm was invented locally
(`--tree-only`) while the repo already had one: `verify-release-surface.sh`
takes `--no-artifacts` and prints

  ok: skipping the built-artifact check (--no-artifacts) — holding the tree surfaces only

Two release gates with two names for the same decision is two things to learn
and one of them to get wrong. So this script now takes `--no-artifacts`, and
prints the skip in the same shape and the same place — where the artifact check
would have run, never silently:

  ok: skipping the built-artifact check (--no-artifacts) — holding the tree-stated
  version only; nothing here binds a built artifact to <version> or to this commit

The summary closes `PASS (NO ARTIFACTS): … the built-artifact check did NOT run
(--no-artifacts)`, so the arms that ran are on the line either way. Behaviour is
otherwise unchanged: a path still runs both arms and exits 0, a bare version is
still a refusal, and no call site is touched.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant