From a85c900debedf005f1b25d3732c7a4daeeac1f09 Mon Sep 17 00:00:00 2001 From: jamesli124 <58196853+jamesli124@users.noreply.github.com> Date: Thu, 23 Jul 2026 14:27:14 -0500 Subject: [PATCH] fix(ci): use a real token for release automation, not GITHUB_TOKEN The v1.1.0 release failed to publish to PyPI: 400 Invalid attestations supplied ... Certificate's Build Config URI (.../release-tag.yml@refs/heads/main) does not match expected Trusted Publisher (python-publish.yml @ epifluidlab/FinaleToolkit) release-tag.yml created the GitHub Release with the default GITHUB_TOKEN, which never triggers other workflow runs (anti-recursion) -- so it invoked python-publish.yml directly as a reusable workflow instead of relying on the release: published event. That routes around the recursion limit but breaks something else: PyPI's Trusted Publisher verification checks the OIDC certificate's Build Config URI, and a workflow invoked via `uses: ./...` reports the *calling* workflow's identity, not the reusable one -- so PyPI saw "release-tag.yml" instead of the registered "python-publish.yml" and rejected the upload as an unrecognized publisher. Nothing was left in a partial state; PyPI rejects unverified uploads before accepting the file. Fix: release-tag.yml and release-prepare.yml now push/create things with a RELEASE_TOKEN secret (a PAT) instead of GITHUB_TOKEN. A PAT-authored release is an ordinary release: published event with no recursion limit, so python-publish.yml runs as itself again -- back to the exact identity PyPI already has registered. This also fixes a second, previously-known gap: PRs opened by GITHUB_TOKEN don't get pull_request-triggered checks (changelog-check, the test matrix) run on them either, so the release PR was silently uncovered by CI before this. python-publish.yml also gains a workflow_dispatch trigger (existing tag -> rebuild and retry publish), so v1.1.0 -- whose tag and GitHub Release already exist, only the PyPI upload failed -- can be retried once RELEASE_TOKEN is in place, without recreating anything. RELEASING.md documents the required one-time RELEASE_TOKEN setup and the retry procedure. Verified: all four workflow YAML files parse correctly, full test suite unaffected (225 passed, 11 skipped) -- this PR touches no src/finaletoolkit/ files so changelog-check.yml correctly doesn't require a CHANGELOG entry for it. Co-Authored-By: Claude Sonnet 5 --- .github/workflows/python-publish.yml | 24 ++++------- .github/workflows/release-prepare.yml | 18 ++++++-- .github/workflows/release-tag.yml | 56 ++++++++++++++----------- RELEASING.md | 60 +++++++++++++++++++++------ 4 files changed, 102 insertions(+), 56 deletions(-) diff --git a/.github/workflows/python-publish.yml b/.github/workflows/python-publish.yml index 9eb4b95..570d397 100644 --- a/.github/workflows/python-publish.yml +++ b/.github/workflows/python-publish.yml @@ -11,21 +11,14 @@ name: Upload Python Package on: release: types: [published] - # Also invocable directly as a reusable workflow (see release-tag.yml). - # release-tag.yml creates the GitHub Release itself using the default - # GITHUB_TOKEN, and GitHub does not let a GITHUB_TOKEN-authored event - # trigger further workflow runs (to prevent recursive triggers) -- so - # `release: published` alone would never fire in that path. Calling this - # workflow directly sidesteps that limitation. - workflow_call: + # Manual escape hatch: retry publishing an existing tag if the automated + # release: published path failed (e.g. a Trusted Publisher mismatch) or + # to publish a release that predates this workflow. + workflow_dispatch: inputs: - version: - description: >- - Version being published, e.g. "1.1.0" (no leading "v"). Only - used for the PyPI project URL shown in the deployment; - publishing itself does not depend on it. - required: false - type: string + tag: + description: 'Existing git tag to build and publish, e.g. "v1.1.0"' + required: true permissions: contents: read @@ -38,6 +31,7 @@ jobs: - uses: actions/checkout@v4 with: fetch-depth: 0 # full history + tags, needed by setuptools-scm to derive the version + ref: ${{ github.event.release.tag_name || inputs.tag }} - uses: actions/setup-python@v5 with: @@ -72,7 +66,7 @@ jobs: # # ALTERNATIVE: if your GitHub Release name is the PyPI project version string # ALTERNATIVE: exactly, uncomment the following line instead: - url: https://pypi.org/project/FinaleToolkit/${{ github.event.release.name || inputs.version }} + url: https://pypi.org/project/FinaleToolkit/${{ github.event.release.name || inputs.tag }} steps: - name: Retrieve release distributions diff --git a/.github/workflows/release-prepare.yml b/.github/workflows/release-prepare.yml index f37d3b1..22efec4 100644 --- a/.github/workflows/release-prepare.yml +++ b/.github/workflows/release-prepare.yml @@ -4,6 +4,13 @@ name: Prepare release # supplies the version number. This workflow only prepares the CHANGELOG and # opens a PR for review -- nothing is tagged or published until that PR is # reviewed and merged (see release-tag.yml, triggered on merge). +# +# Pushes the branch and opens the PR with RELEASE_TOKEN rather than the +# default GITHUB_TOKEN. This isn't just for consistency with release-tag.yml +# -- GitHub does not run pull_request-triggered workflows (changelog-check, +# the test matrix, ...) on a PR opened by GITHUB_TOKEN, so without this the +# release PR would show no checks at all. See RELEASING.md for how to +# create and store RELEASE_TOKEN. on: workflow_dispatch: @@ -13,8 +20,7 @@ on: required: true permissions: - contents: write - pull-requests: write + contents: read jobs: prepare: @@ -29,9 +35,13 @@ jobs: - name: Open release PR env: - GH_TOKEN: ${{ github.token }} + GH_TOKEN: ${{ secrets.RELEASE_TOKEN }} run: | set -e + if [ -z "$GH_TOKEN" ]; then + echo "::error::The RELEASE_TOKEN secret is not set. See RELEASING.md for how to create it -- the default GITHUB_TOKEN would open the PR but with no CI checks running on it." + exit 1 + fi VERSION="${{ inputs.version }}" BRANCH="release-v${VERSION}" @@ -40,7 +50,7 @@ jobs: git checkout -b "$BRANCH" git add CHANGELOG.md git commit -m "Release v${VERSION}" - git push -u origin "$BRANCH" + git push "https://x-access-token:${GH_TOKEN}@github.com/${{ github.repository }}.git" -u "$BRANCH" # Idempotent: succeeds whether or not the label already exists. gh label create release --color "0E8A16" \ diff --git a/.github/workflows/release-tag.yml b/.github/workflows/release-tag.yml index baf7d0d..10c105a 100644 --- a/.github/workflows/release-tag.yml +++ b/.github/workflows/release-tag.yml @@ -1,15 +1,32 @@ -name: Tag and publish release +name: Tag release # Fires when a "release-vX.Y.Z" PR (opened by release-prepare.yml) is merged: -# tags the merge commit, creates a GitHub Release from the CHANGELOG section -# for that version, then directly invokes python-publish.yml to build and -# publish to PyPI. +# tags the merge commit and creates a GitHub Release from the CHANGELOG +# section for that version. That Release's `published` event is what +# triggers python-publish.yml to build and publish to PyPI. # -# Directly invoking python-publish.yml (rather than relying on the -# `release: published` event it also listens for) is deliberate: the GitHub -# Release below is created with the default GITHUB_TOKEN, and GitHub does -# not let a GITHUB_TOKEN-authored event trigger further workflow runs, to -# prevent recursive triggers. `release: published` would never fire here. +# The release must be created with a real user/PAT token (RELEASE_TOKEN), +# not the default GITHUB_TOKEN. Two independent problems rule GITHUB_TOKEN +# out here: +# +# 1. GitHub does not let a GITHUB_TOKEN-authored event trigger further +# workflow runs (anti-recursion), so `release: published` would never +# fire and python-publish.yml would simply never run. +# +# 2. Calling python-publish.yml directly as a reusable workflow (uses: +# ./.github/workflows/python-publish.yml) to route around problem 1 +# was tried and does not work either: PyPI's Trusted Publisher is +# registered for the exact file "python-publish.yml", but a workflow +# called this way has its OIDC identity (the certificate's Build +# Config URI) reported as the *calling* workflow -- this file -- not +# the reusable one, so PyPI rejects the upload as coming from an +# unrecognized publisher. +# +# A PAT/user-token-authored release event sidesteps both: it's a normal +# `release: published` event like a maintainer clicking "Draft a new +# release" by hand, so python-publish.yml runs as itself, with its own, +# already-correctly-registered identity. See RELEASING.md for how to +# create and store RELEASE_TOKEN. on: pull_request: @@ -24,10 +41,6 @@ jobs: github.event.pull_request.merged == true && startsWith(github.event.pull_request.head.ref, 'release-v') runs-on: ubuntu-latest - permissions: - contents: write - outputs: - version: ${{ steps.version.outputs.version }} steps: - uses: actions/checkout@v4 with: @@ -45,23 +58,18 @@ jobs: - name: Create tag and GitHub Release env: - GH_TOKEN: ${{ github.token }} + GH_TOKEN: ${{ secrets.RELEASE_TOKEN }} run: | set -e + if [ -z "$GH_TOKEN" ]; then + echo "::error::The RELEASE_TOKEN secret is not set. See RELEASING.md for how to create it -- the default GITHUB_TOKEN cannot be used here (it would silently fail to trigger the PyPI publish)." + exit 1 + fi VERSION="${{ steps.version.outputs.version }}" git config user.name "github-actions[bot]" git config user.email "github-actions[bot]@users.noreply.github.com" git tag "v${VERSION}" - git push origin "v${VERSION}" + git push "https://x-access-token:${GH_TOKEN}@github.com/${{ github.repository }}.git" "v${VERSION}" gh release create "v${VERSION}" \ --title "v${VERSION}" \ --notes-file release_notes.md - - publish: - needs: tag - permissions: - id-token: write - contents: read - uses: ./.github/workflows/python-publish.yml - with: - version: ${{ needs.tag.outputs.version }} diff --git a/RELEASING.md b/RELEASING.md index 2d4085a..4bc38e2 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -44,11 +44,9 @@ human picks the version number. ([release-tag.yml](.github/workflows/release-tag.yml)): - Tags the merge commit `v1.1.0`. - Creates a GitHub Release from that version's CHANGELOG section. - - Publishes to PyPI, by directly invoking - [python-publish.yml](.github/workflows/python-publish.yml) as a reusable - workflow (**not** by relying on the `release: published` event it also - listens for -- see the note in that file for why that event would never - fire here). + - That Release's `published` event is what triggers + [python-publish.yml](.github/workflows/python-publish.yml), which builds + and publishes to PyPI. No git tag is created by hand, no version string is edited anywhere (the package version is derived from the tag via `setuptools-scm`), and there's no @@ -56,14 +54,50 @@ manual "Draft a new release" click on GitHub. ## One-time repository setup -`release-prepare.yml` pushes a branch and opens a PR using the default -`GITHUB_TOKEN`. This requires, under **Settings > Actions > General > -Workflow permissions**: - -- "Read and write permissions" -- "Allow GitHub Actions to create and approve pull requests" - -Without these, step 1 above will fail to push the branch or open the PR. +Both `release-prepare.yml` and `release-tag.yml` push/create things with a +**`RELEASE_TOKEN` secret** (a Personal Access Token), not the default +`GITHUB_TOKEN`. This isn't a style choice -- it's required for the automation +to actually work, because of two GitHub Actions behaviors: + +1. **GitHub does not let a `GITHUB_TOKEN`-authored event trigger further + workflow runs** (anti-recursion). A GitHub Release created with the + default token would never fire `python-publish.yml`'s `release: published` + trigger -- the publish step would just silently never run. (We initially + worked around this by having `release-tag.yml` directly invoke + `python-publish.yml` as a reusable workflow instead of relying on that + event. That produced a *different* failure: PyPI's Trusted Publisher is + registered for the exact file `python-publish.yml`, but a workflow invoked + this way reports the *calling* workflow's identity to PyPI's OIDC + verification, not the reusable one -- so PyPI rejected the upload as an + unrecognized publisher. A real token sidesteps both problems at once: the + release is a normal, non-recursion-limited event, and `python-publish.yml` + runs as itself.) +2. **GitHub does not run `pull_request`-triggered workflows** (the test + matrix, `changelog-check.yml`) **on a PR opened by `GITHUB_TOKEN`**. Using + a real token for `release-prepare.yml` means the release PR actually gets + CI checks, like any other PR. + +To set this up: + +1. Create a [Personal Access Token](https://github.com/settings/tokens) -- + either a classic token with the `repo` scope, or a fine-grained token + scoped to this repository with **Contents: Read and write** and **Pull + requests: Read and write** permissions. +2. Add it as a repository secret named `RELEASE_TOKEN`: **Settings > Secrets + and variables > Actions > New repository secret**. + +Without this secret, both workflows fail fast with a clear error rather than +silently doing the wrong thing. + +## Retrying a failed PyPI publish + +If the tag and GitHub Release already exist but the PyPI upload failed (check +the failed run's logs under the repo's Actions tab), nothing needs to be +re-tagged or re-released -- **Actions -> "Upload Python Package" -> Run +workflow**, entering the existing tag (e.g. `v1.1.0`), rebuilds from that +exact tag and retries the publish. +([python-publish.yml](.github/workflows/python-publish.yml)'s +`workflow_dispatch` trigger.) ## Bioconda