Skip to content

Add a shared check that no workflow leaves a release as a draft - #93

Merged
imanimanyara merged 2 commits into
mainfrom
feat/verify-release-publish
Nov 8, 2026
Merged

imanimanyara merged 2 commits into
mainfrom
feat/verify-release-publish

Conversation

@imanimanyara

Copy link
Copy Markdown
Member

Summary

Deleting a tag does not delete its GitHub release — it reverts it to a draft. The release keeps its body and its assets and simply stops being visible, and the state outlives the tag: laranail/validation still holds v1.0.0 and v1.0.1 as drafts from tags withdrawn on 2026-08-24, neither of which exists on the remote any more.

gh release edit leaves that flag exactly as it finds it, so an edit after a tag is deleted and re-pushed — how this family moves a pre-stable tag wherever git push --force is unavailable — republishes nothing on a run that still goes green: correct tag, invisible release.

Nothing else reports that state. The tag resolves, composer installs, and verify-tag-currency.sh next door passes, because none of them reads the release. It was found by reading one, in laranail/validation-js, whose release had to be published by hand.

Measured 2026-11-08 across the 286 workflow files in laranail/packages: four hand-rolled call sites in three repositories, every one with green runs behind it. All four are fixed in their own repositories (password-history#21, password-tools#22, validation#42, validation-js#35); this is the shared check that keeps the next one from landing.

What this adds

scripts/verify-release-publish.py, alongside verify-action-pins.py and following it exactly:

  • It takes the caller's repository path as an argument, which is what makes it shareable — that script's own comment records that deriving the path from __file__ silently scanned the wrong repository once it stopped being a local copy.
  • No network and no token: it is a text scan, so it is the cheapest job in the hygiene workflow and reports the same thing locally as on a runner.
  • Comment lines are dropped before matching (every repository that carried the defect also carried a comment naming the command) and shell line continuations are joined, so a flag on the next line still counts.

The way this check could lie, and what stops it

Most repositories are clean by having no gh release edit at all — softprops/action-gh-release publishes an existing draft unless its draft input is true, "omit it to publish after upload" in its own words. So found nothing is the normal pass, and a matcher that silently stopped matching would report the whole estate clean while reading nothing.

So the script drives its matcher against five built-in controls on every invocation — an edit with no flag must fail, one with the flag must pass, a continuation must join, prose about the command must not match, and another gh release subcommand must not match — and refuses to report anything at all if one misbehaves, naming the matcher rather than the repository. The workflow-file floor covers the other direction, a glob that goes empty.

tests/Feature/VerifyReleasePublishScriptTest.php sabotages the pattern in a copy of the script and asserts the controls catch it, so the control mechanism is itself tested rather than assumed.

Proof

Run against each of the four repositories' unfixed main, not against synthetic samples:

Repository (at main) Workflow files Exit Findings
password-history 4 1 release.yml
password-tools 4 1 release.yml
validation 13 1 release.yml and release-benchmark.yml
validation-js 6 1 release.yml

And across all 70 package checkouts with the fix branches in place: 4 repositories hold hand-rolled edits, all clean, 0 failing. That is a second method agreeing with the original grep about which call sites exist.

Gates

composer test 1735 passed, 3 skipped (4558 assertions) · laranail-pint --test passed · phpstan analyse no errors · rector process --dry-run no changes. The new test is 10 passing (24 assertions), four of which assert exit 1 on a specific defect.

Not in this PR

The reusable hygiene.yml in laranail/.github is where the 18 callers would inherit this with no change of their own. That wiring must land after this, since the job checks laranail/package-tools@main out and runs the script from it — a hygiene job referencing a script that is not on main yet fails for every caller. It is opened as a draft against laranail/.github for that reason.

Deleting a tag does not delete its release -- it reverts it to a
draft. The release keeps its body and assets and stops being visible,
and the state outlives the tag: laranail/validation still holds
v1.0.0 and v1.0.1 as drafts from tags withdrawn on 2026-08-24, and
neither tag exists on the remote any more.

`gh release edit` leaves that flag exactly as it finds it, so an edit
after a tag is deleted and re-pushed republishes nothing on a run
that still goes green. Nothing else reports it: the tag resolves,
composer installs, and verify-tag-currency.sh next door passes,
because none of them reads the release. It was found by reading one.

The check takes the caller's path the way verify-action-pins.py does,
so the reusable hygiene workflow can point it at another repository,
and it needs no network and no token.

Most repositories pass by having no `gh release edit` at all, because
softprops/action-gh-release publishes an existing draft on its own.
So "found nothing" is the normal result, and a matcher that silently
stopped matching would report the whole estate clean. The script runs
five built-in controls on every invocation and refuses to report
anything when one misbehaves; the workflow-file floor covers a glob
that goes empty. The test sabotages the pattern in a copy of the
script and asserts the controls catch it.
Copilot AI balanced review requested due to automatic review settings November 8, 2026 04:33

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

Neither file is touched by this branch's own change. The repository
tracks no lock file, so CI resolves fresh on every run: it got Rector
2.8.4 and PHPStan 2.3.5 while the local vendor directory held 2.6.7
and 2.2.17, and AddArrowFunctionReturnTypeRector in the newer Rector
wants `fn (): never` on an arrow function whose body only throws.

So the static-analysis job went red on code that has been on main
since before this branch, and the last run on main predates the
Rector release. A `composer update -W` locally reproduces both
findings exactly; this is what Rector itself writes.

It lands here rather than in its own pull request because the branch
cannot go green without it, and the family's convention is one pull
request per repository rather than a trickle.
@imanimanyara
imanimanyara merged commit 2652919 into main Nov 8, 2026
9 checks passed
@imanimanyara
imanimanyara deleted the feat/verify-release-publish branch November 8, 2026 05:11
imanimanyara added a commit to laranail/.github that referenced this pull request Nov 8, 2026
`pull_request` with no `types` defaults to opened, synchronize and
reopened, and the job skips a draft. So a pull request opened as a
draft never gets this check: marking it ready re-triggers nothing,
the skipped run stands, and the pull request can merge with its only
job never having run.

Measured on this very pull request, which is a draft deliberately --
it must not merge before laranail/package-tools#93. Marking it ready
left `audit script tests` showing "skipping" from the draft run.

Adding `ready_for_review` is additive: every event that triggered the
workflow before still does.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants