Skip to content

ci(release): run release.yml's tools from the workflow's own revision, not the caller's lock - #42

Merged
kiro-systemf[bot] merged 3 commits into
mainfrom
prm/workflow-own-tools
Oct 8, 2026
Merged

kiro-systemf[bot] merged 3 commits into
mainfrom
prm/workflow-own-tools

Conversation

@systemfsoftware-maker

@systemfsoftware-maker systemfsoftware-maker commented Oct 8, 2026 •

Copy link
Copy Markdown
Collaborator

Why

stryker-js-effect Release run 37833688902 failed in "version · open release PR". The step nix develop --command github-release-management pr --output "$GITHUB_OUTPUT" printed the CLI help, logged ##[error]Help requested and exited 1.

release.yml and the tools it runs came from two different revisions:

So any change to the flags release.yml passes broke every caller until each one bumped its lock.

Change

Each job in release.yml now starts with one step. It is written once, as the YAML anchor &release-tools in the plan job, and the version and release jobs reuse it as *release-tools. GitHub Actions has supported YAML anchors since 2025-09-18. The .#workspace-tarballs step is shared the same way.

- &release-tools
  name: Build the release tools at this workflow's own revision
  env:
    WORKFLOW_REPOSITORY: ${{ job.workflow_repository }}
    WORKFLOW_SHA: ${{ job.workflow_sha }}
  run: |
    tools=$(nix build --no-link --print-out-paths \
      "github:${WORKFLOW_REPOSITORY:?job.workflow_repository is empty}/${WORKFLOW_SHA:?job.workflow_sha is empty}#release-tools")
    echo "RELEASE_TOOLS=$tools/bin" >> "$GITHUB_ENV"

Every tool call becomes nix develop --command "$RELEASE_TOOLS/<app>" …:

  • Binary: the binary is the one from the workflow's own commit.
  • Environment: the caller's dev shell still supplies everything the tools run in: pnpm (the workspace reader spawns pnpm ls and pnpm config get), git, the sandbox and SANDBOX_PNPM_STORE.
  • Failure mode: if a runner doesn't expose the context, the job stops with job.workflow_sha is empty before any tool runs. It doesn't fall back to the caller's copy.

Why job.workflow_sha

GitHub's contexts reference, job context defines these properties:

  • job.workflow_ref: "The full ref of the workflow file that defines the current job … For jobs defined in a reusable workflow, this refers to the reusable workflow file."
  • job.workflow_sha: "The commit SHA of the workflow file that defines the current job."
  • job.workflow_repository: "The owner/repo of the repository containing the workflow file that defines the current job."

The same page shows a reusable workflow checking out its own source with repository: ${{ job.workflow_repository }} and ref: ${{ job.workflow_sha }}.

These are the only values that identify the called workflow's own revision from inside it. The alternatives don't work:

  • github.workflow_sha / github.workflow_ref: these describe the caller's top-level workflow (for example octocat/hello-world/.github/workflows/my-workflow.yml@refs/heads/my_branch), not the called one.
  • github.sha: this is the caller's commit.
  • A composite action or uses: at "our own ref": this can't be written. A uses: ref is static text and can't be an expression, and a relative ./ path in a reusable workflow resolves against the caller's checkout.
  • A tools-ref input: this moves the skew onto the caller instead of removing it.

job.workflow_sha is a commit SHA, not a branch name, so the tools are the exact commit @main (or @v1, or @<sha>) resolved to for this run, even if main moves during the run. nix build github:<repo>/<sha>#release-tools builds the same output the dev shells already take as packages.<system>.release-tools.

Runner support: github.com provides the context. The runner side landed in actions/runner#4335 (first released in v2.334.0). The docs say GitHub Enterprise Server doesn't have it; the README now says so, and such a runner fails at the guard above.

Test

The e2e (@systemfsoftware/e2e, part of pnpm test) now runs all three jobs (plan, version and release) step by step, as release.yml writes them. The old harness rewrote nix develop --command <app> to the image's binaries, which made the resolution invisible. Now:

  • nix on the runner PATH is a stub that models the two verbs release.yml uses:
    • develop --command enters a "caller dev shell" whose release-tools are an older revision's (0449f15): each one exits 1, naming itself.
    • build accepts only nix build --no-link --print-out-paths <ref>, the shape release.yml uses. It resolves github:systemfsoftware/pnpm-release-management/<workflow sha>#release-tools to the binaries built from this checkout, and .#workspace-tarballs to the tarballs the test packed.
  • job.workflow_repository and job.workflow_sha are resolved like the runner resolves them. GITHUB_ENV carries over from one step to the next.
  • Tool steps are identified by the binary's basename, so tool() still finds each step's env.
  • The release phase runs the plan job and asserts its phase=release output. It then runs the release job and checks that every step ran (build, pack, capture, tag, GitHub Releases) and that each exited 0. The existing assertions on the pushed tags and the release notes follow.

Failing before: I ran this branch's harness and test against release.yml from main 2a4722c, in a throwaway checkout (pnpm --filter @systemfsoftware/e2e test, exit 1). It fails in the version phase (pipeline.integration.test.ts:279):

Caused by: AssertionError: expected [ { …(4) } ] to deeply equal []

- Expected
+ Received

- []
+ [
+   {
+     "code": 1,
+     "stderr": "version-management at 0449f15b2db9602a1dfe58bee2598adca2a726ed, the revision the caller's flake.lock pins, ran instead of the workflow's: bump
+ ",
+     "stdout": "",
+     "step": "Bump every package and version surface, consuming the intents",
+   },
+ ]

 Test Files  1 failed (1)
      Tests  1 failed (1)

On this branch (b5700bf): pnpm --filter @systemfsoftware/e2e test exits 0 (Test Files 1 passed (1), Tests 1 passed (1), 299.12s).

Sabotage checks (throwaway edits to release.yml, reverted):

  • Dropping --print-out-paths from the build step. Before the review fix, the stub still passed this, while a real runner would set RELEASE_TOOLS=/bin. Now the suite fails:
    +     "code": 2,
    +     "stderr": "nix stub: only 'nix build --no-link --print-out-paths <ref>' is modelled, got: nix build --no-link github:systemfsoftware/pnpm-release-management/feedface…#release-tools
    +     "step": "Build the release tools at this workflow's own revision",
    
  • A bare nix develop --command github-release-management tag … in the release job. Before the review fix, no test ran that job. Now the suite fails (pipeline.integration.test.ts:419):
    +     "code": 1,
    +     "stderr": "github-release-management at 0449f15b2db9602a1dfe58bee2598adca2a726ed, the revision the caller's flake.lock pins, ran instead of the workflow's: tag --tarballs /tmp/workspace-tarballs --captured .release/captured.json
    +     "step": "Tag released versions, recording each tarball's integrity",
    

Review

A fresh-context correctness review returned VERDICT: approve-with-fixes. Its findings and how each was handled:

  • F1 (P2), fixed: the nix build stub ignored its flags, so dropping --print-out-paths passed. The stub now requires the exact build shape.
  • F2 (P2), fixed: plan and release weren't exercised by any test. The stub now models .#workspace-tarballs, and the release phase runs both jobs.
  • F3 (P3), fixed: the build step was copied three times. It is now a YAML anchor. A composite action would bring the skew back, because a reusable workflow's ./.github/actions/… resolves against the caller's checkout.
  • F4 (P3), fixed: the README quick start used WORKFLOW_REPOSITORY/WORKFLOW_SHA without introducing them. It now says where they come from and keeps the :? guards.
  • F5 (advisory), unchanged: GitHub Enterprise Server isn't supported, and the job fails at the guard. This was accepted with the mechanism (ruling 18) and is documented in the README.
  • F6 (advisory), unchanged: pnpm still comes from the caller's dev shell. This is listed under "Not covered here".
  • F7 (advisory), unchanged: nix build github: makes an unauthenticated API call. GitHub-hosted runners get a fresh IP per job, so it only matters for self-hosted pools behind one egress IP.

Consumers' dev shells

The flake outputs are unchanged, so a consumer's dev shell keeps its locked release-tools for local use. release.yml no longer reads them; the README's distribution and CI sections now say so.

Not covered here

  • .#workspace-tarballs is still built by the caller's flake. That's packing the caller's workspace, not running a tool.
  • The tools spawn the caller's pnpm (pnpm ls -r --json, pnpm config get). A future tool revision that depends on a newer pnpm CLI could still skew against an old dev shell, one layer down.
  • changeset-check.yml has the same class of skew: with devshell: true it runs changeset-management from the caller's dev shell, and otherwise it checks out tools-ref (default main). This PR leaves it alone.

No changeset: only the workflow, the e2e and the README change.

…, not the caller's lock

release.yml passed flags its own revision's CLI accepts, but ran the CLI from the caller's dev shell, built from whatever revision the caller's flake.lock pins. A caller on release.yml@main with an older lock failed in the version job: `pr --output` printed help and exited 1 (stryker-js-effect Release run 37833688902).

Each job now builds github:${job.workflow_repository}/${job.workflow_sha}#release-tools and runs those binaries by path inside the caller's dev shell, which still supplies pnpm, git and the sandbox. The e2e runs the version job's steps through a nix stub whose dev shell holds an older revision's tools
… tool build once

Review of #42 found the harness's nix build stub answered any argument
list whose last word was the tools ref, so dropping --print-out-paths
from release.yml still passed while a real runner would set
RELEASE_TOOLS=/bin. The stub now accepts only
`nix build --no-link --print-out-paths <ref>` and also models
.#workspace-tarballs.

Only the version job ran through the harness. The release phase now
runs release.yml's plan and release jobs step by step and asserts the
plan job's phase output and every release step, so a bare tool name in
either job fails the suite.

The build and pack steps are YAML anchors, defined once and aliased
in the other jobs; a composite action would resolve against the
caller's checkout and bring the skew back. The README quick start
now says where WORKFLOW_REPOSITORY and WORKFLOW_SHA come from and
keeps the empty-context guards

@kiro-systemf kiro-systemf Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified: tools built from github:${job.workflow_repository}/${job.workflow_sha} with :? guards; fleet runner 2.337.0; e2e fails on 2a4722c (caller-lock tool ran); fresh review approve-with-fixes, F1-F4 fixed with sabotage probes caught; real flake build of b5700bf lists --output; CI 5/5; hunt grep clean.

@kiro-systemf
kiro-systemf Bot merged commit bda493d into main Oct 8, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant