Review: workflow SHA-pinning plan is stale — its core recommendation is already implemented, but zizmor.yml is deployed to 0 of 12 consumer repos

[twistedmelonman]

Review of docs/plans/2026-08-27-workflow-sha-pinning.md against Beacon's parallel effort, beacon-biosignals/knowledge-base#3624 ("Update recommendations for use of third-party actions"), which tightens the same guidance from the company side.

Every tooling claim below was verified by running zizmor 1.30.0 locally and querying live GitHub state, not by reading either document. Several claims in both documents turned out to be wrong.

TL;DR

The plan is stale in a way that matters: it is marked "Proposed — not started," but Phase 1 and Task 2.2 are already done, and the strategic recommendation this review would otherwise have made — a two-tier trust model via zizmor policies: — is already implemented in github-workflows/zizmor.yml.

What remains is not the 16-repo SHA migration the plan describes. It is a much smaller set of gaps, one of which is real and currently unmitigated: zizmor.yml exists in exactly one repo, and its own header says to copy it into every consumer. Zero of 12 consumer repos have it.

1. The plan's Current State is out of date

Plan says	Verified reality (2026-09-01)
"Status: Proposed — not started"	Phase 1 complete; Task 2.2 complete
Phase 1: "Add a dependabot-auto-merge.yml reusable workflow to github-workflows"	Exists. Releases dependabot-auto-merge-v1.0.0 (2026-08-07) and tag dependabot-auto-merge-v2
Task 2.2: point dev-env at the new reusable workflow	Done — dev-env/.github/workflows/dependabot-auto-merge.yml is a caller stub uses: ...@dependabot-auto-merge-v2 with trusted_namespaces: 'smartwatermelon'
Phase 1 prerequisite: "github-workflows must be added to the session"	No longer blocking; the work landed

The plan should be rewritten against actual state before any of it is executed. Executing Phase 1 as written would duplicate existing work.

2. The core recommendation is already implemented

The plan's framing — SHA-pin smartwatermelon/github-workflows, then rebuild zero-touch propagation across four phases — never asks whether first-party refs should be exempt. That question is the entire substance of the Beacon PR discussion, and github-workflows/zizmor.yml already answers it:

rules:
  unpinned-uses:
    config:
      policies:
        smartwatermelon/github-workflows/*: ref-pin
        "*": hash-pin

I confirmed this mechanism works as intended (zizmor 1.30.0, --persona=auditor): the ref-pin entry silences first-party refs while "*": hash-pin still flags third-party tags.

That config, plus its rationale comments citing GHSA-8q5r-mmjf-575q and #57, is a stronger and better-evidenced position than the Beacon PR reaches. Phases 1–3 of the plan are largely unnecessary as a security measure. Phase 1's de-duplication of 16 copied policy files was worth doing on its own maintainability merits — and it is done.

3. The real remaining gap: zizmor.yml is not deployed

github-workflows/zizmor.yml opens with:

Copy this to the root of any repo that uses the caller stubs from this repo's README, alongside your pre-commit hook. Without it, zizmor's blanket hash-pin policy reports ~9 high findings against a byte-identical standard caller stub, and the only workaround is SKIP=zizmor on every commit.

Checked all 12 consumer repos — dev-env, claude-config, dotfiles, scripts, lock-sync, homebrew-tap, crazy-larry, ralph-burndown, archive-resolver, claude-wrapper, spokane-snow, slack-mcp. None has a zizmor.yml. dev-env also has no .pre-commit-config.yaml; zizmor appears only in docs and scripts.

So the file predicts its own failure mode and that failure mode is currently live fleet-wide. Its own text names the consequence: routine linter-skipping is what let claude-code-action sit at v1.0.70 across 123 releases carrying GHSA-8q5r-mmjf-575q.

This is the highest-value action item, and it replaces Phase 3. Propagating a ~100-line config file to 12 repos is a smaller, safer change than rewriting uses: refs in 16.

4. Verified tooling corrections

Both documents make inaccurate claims. The Beacon PR states: "In the vast majority of cases, you can use the --fix=all flag to automatically convert version tags to SHA hash references." Tested:

1. --fix=all is EXPERIMENTAL per zizmor --help, and is "an alias for --persona=pedantic."
2. It does nothing offline. zizmor defaults to offline mode; the run reported No fixes available to apply. Tag→SHA rewriting requires network plus a token (--gh-token).
3. The rewrite is classified an unsafe fix, which is why plain --fix (safe, the default) will not perform it.
4. It silently changes the resolved version. actions/checkout@v4 became @11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 — a floating major resolved to a specific patch. A bulk --fix=all is a mass version bump, not a notation change.
5. The blanket hash-pin policy is persona-gated. Findings appear as "required by blanket policy" only at pedantic/auditor. The unpinned-uses rule itself is enabled by default, but a default-persona run against @v4/@main produced no pin findings.

Consequence for the plan: Task 2.3 should mean audit, not bulk-fix. --fix=all should not appear as a fleet tool.

5. Answers to the plan's two open questions

Both are now resolved.

Q: "Does zizmor's ruleset flag anything about the trailing # vX.Y.Z comment format, or only the ref itself?"
A: Only the ref. ...@<40-char-sha> # v3.0.1 passes cleanly at auditor. zizmor also treats reusable-workflow refs identically to action refs (coverage added in v1.21.0).

Q: "Should github-workflows protect its release tags (immutable releases) as defense in depth?"
A: It would not help the floating-tag strategy. GitHub immutable releases went GA 2025-10-28, but they are opt-in, protect release tags only, and are explicitly incompatible with floating major tags — GitHub's own action-release guidance tells publishers to force-push v1 forward, and a tag that must move cannot be immutable. So @v3 and @dependabot-auto-merge-v2 gain nothing. Immutable releases would only help if you pinned exact semver tags, which is the strategy #57 deliberately rejected.

6. New risk worth tracking: Dependabot can silently invalidate version comments

dependabot-core #14716 (open, filed 2026-04-14): when a pinned SHA has no tag pointing directly at it, Dependabot updates to branch HEAD (untagged) and leaves the version comment stale. It is self-perpetuating — once off-tag, it stays off-tag. The reporter's example uses a subpath ref, the same shape as owner/repo/.github/workflows/foo.yml@sha. Compounding it, #7912 (open since 2023) means an already-incorrect comment is never corrected.

Also relevant: #14787 / #14806 — multiple path-based SHA-pinned refs from one repo collided into a single dependency identity and produced invalid tag comments. Fixed 2026-06-25, but that is exactly the shape the plan's Phase 1 creates (two path-based refs into github-workflows).

Current status: not triggered. I verified v3 resolves to 688e0288, which is identical to tag v3.2.1 (ahead_by: 0, behind_by: 0). The trigger condition is absent today.

Implication: the invariant "every floating tag also carries a precise semver tag on the same commit" is load-bearing and currently holds by convention only. If it ever breaks, version comments become actively misleading while still looking like legitimate 40-char SHAs. Worth a CI assertion in github-workflows.

One correction to a claim in the plan's own framing: Dependabot does not read the trailing comment to resolve the version. Per version_commenter.rb, it resolves SHA → tags via git, then suffix-matches the existing comment against those tags. The SHA is the source of truth; the comment is output. A wrong comment cannot mislead an update — but it will not be corrected either.

7. Release-tag hygiene gap

github-workflows has tags v3.2.1, v3.2.0, v3.1.2, v3.1.1 but its most recent GitHub Release is v3.1.0 (2026-08-07). Tags exist without corresponding releases. This matters for two reasons: immutable releases only protect tags attached to a release, and release notes are how consumers assess whether a floating-tag move is safe.

8. What Beacon actually teaches (and does not)

Worth recording, because the Beacon PR reads more authoritative than its practice supports. Measured across Beacon's two largest repos:

Repo	External uses:	SHA-pinned	% pinned
infra	108	38	35%
platform-datastore	202	44	22%
Combined	310	82	26%

Three-quarters of Beacon's action references violate the policy the PR tightens, including beacon-biosignals/gha-tf-shim@main (a branch ref, already forbidden, in a repo with zero tags), and unpinned tags from orgs never on the trusted list (hashicorp, docker, slackapi, webfactory, peter-evans). Neither repo has a dependabot.yml despite the handbook recommending it.

The transferable lesson is the reverse of what the PR argues: tightening policy text without a deployment mechanism produces a document people route around. The zizmor.yml-as-enforced-config approach in github-workflows is the better pattern — which is why §3 (actually deploying it) is the top recommendation.

One caveat found in testing: ref-pin accepted beacon-biosignals/gha-tf-shim@main without complaint. ref-pin means "some symbolic ref," not "a tag" — it does not reject branches in offline mode. The github-workflows config comment says "@main or a bare repo reference is rejected"; the bare-repo half is right, the @main half did not hold in my test. Worth re-verifying online before relying on it.

Recommended actions

• Rewrite the plan doc against verified current state. Mark Phase 1 and Task 2.2 complete; drop or re-scope Phases 2–3.
• Propagate zizmor.yml to the 12 consumer repos (replaces Phase 3 — smaller, safer, addresses a live gap).
• Add zizmor to consumer pre-commit/CI, otherwise the config has nothing to configure.
• Add a CI assertion in github-workflows that every floating tag points at a commit also carrying a precise semver tag (guards dependabot#14716).
• Cut releases for v3.1.1/v3.1.2/v3.2.0/v3.2.1, or document that tags are the interface and releases are not maintained.
• Remove --fix=all as a suggested fleet tool; use zizmor for audit, make pin changes deliberately.
• Re-verify the ref-pin vs @main claim in the config comment with online mode.
• Optional: feed corrections back to beacon-biosignals/knowledge-base#3624 — the --fix=all inaccuracy and the policies: mechanism, which resolves that PR's open disagreement.

Related: #55 (closed), #57 (closed) — this issue builds on both; the two-tier model they established is confirmed correct and better-evidenced than Beacon's.