Repository navigation
ci(release): dry-run the publish path on pull requests; attest provenance and publish an SBOM - #269
Conversation
oci/bluefin-server-sbom.bst runs FSDK's collect_manifest plugin (output-type: spdx) over the runtime closure of the release payload: the /usr and initrd stacks, the kernel, and the k0s, KubeStellar, kubeadm and OpenZFS sysext payloads. The image element stamps creationInfo.created, drops the identical duplicate packages FSDK bootstrap elements produce, and lists bluefin-server_<ver>.spdx.json in SHA256SUMS so the GPG signature covers it. It also exports the keyring the build trusts as sysupdate-keys/import-pubring.pgp (not published, like efi-keys/) so a pull request's release dry run can check SHA256SUMS.gpg against the throwaway key. Assisted-by: Claude Opus 5.5 via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
One publish path for main and for the pull-request rehearsal: verify (gpgv against a keyring, every SHA256SUMS entry present and matching, every published file listed, file names match the version, SBOM is valid SPDX 2.3), release (gh release create at the built commit, or print it with --dry-run), and oci (oras push, then compare the registry manifest layer by layer with the local files; optionally pull it back). Assisted-by: Claude Opus 5.5 via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ance and SBOM The release job now calls scripts/publish-release.sh for every publish step and verifies the set against the committed release keyring first. It attests SLSA build provenance and the SPDX SBOM for every published file and for the OCI artifact digest (push-to-registry, so the Sigstore bundles live next to the artifact as referrers). release-dry-run runs the same verify, release --dry-run and oci commands on every pull request, pushing to a digest-pinned registry service container and pulling the artifact back. Read-only token, no secrets. Assisted-by: Claude Opus 5.5 via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
systemd-sysupdate-verification.md owns the trust story and the gh attestation / cosign verify commands, replacing the dangling signing-and-sbom reference; ci-tooling.md lists the release-dry-run job and the release job's new permissions. Assisted-by: Claude Opus 5.5 via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
castrojo
left a comment
There was a problem hiding this comment.
Reviewed at e31529b. I read the full diff, ran the local checks and read CI.
What I verified:
- Token scope.
release-dry-runruns only onpull_requestwithpermissions: contents: read, no secrets, apersist-credentials: falsecheckout, and a digest-pinnedregistry:3.1.2service on localhost.id-token,attestations,artifact-metadataand the write scopes are only onrelease, which is still gated ongithub.ref == 'refs/heads/main'. On main,verifychecks against the committedfiles/os/sysupdate-keys/import-pubring.gpg, not the build's own copy. scripts/publish-release.sh.verifyrunsgpgvoverSHA256SUMS, thensha256sum --check --strict. It rejects duplicate and unlisted top-level files, requires exactly one file for each expected release pattern, and checks that the SBOM is SPDX-2.3 with unique SPDXIDs before emitting attestation subjects.releasekeeps the publish-once behaviour (gh release createrefuses an existing tag) and now pins--targetto the built SHA.ocichecks that:<ver>and:latestresolve to the same digest and that the manifest layers match the local files byte for byte.- CI on this head. build, boot-test, release-dry-run, unit and docs all pass (runs 36514803017, 36514802994, 36514802942). The dry-run log shows
release set 0.680: 12 files match SHA256SUMS, signature verified, the renderedgh release createcommand, andpulled localhost:5000/bluefin-server@sha256:370fdde… back: 14 files match. - Local.
pytest tests/unit: 388 passed.bats tests/unit: pass.shellcheck -S warning scripts/publish-release.sh: clean.actionlint .github/workflows/*.yml: clean apart from my local actionlint not knowing theubuntu-26.04label. - SBOM dependency walk. I checked against the pinned
buildstream-plugins-community2.3.1collect_manifest.pyand BuildStream'sElementProxy.dependencies. The plugin walks the listed build-depends, then only runtime dependencies below them, sozfs/openzfs.bst's autotools build stack does not leak into the SBOM.
Non-blocking follow-ups:
elements/oci/bluefin-server-sbom.bst:27-30listsk0s/k0s-bin.bst,kubeadm/kubeadm-bin.bstandzfs/openzfs.bstrather than the publishedoci/{k0s,kubeadm,zfs}-sysext.bst. So the local sources those sysexts ship (files/k0s/sysext,files/kubeadm/sysext,files/zfs/sysext) are not listed, whileoci/kubestellar-sysext.bstis listed directly. The upstream binaries themselves are covered. Adding the sysext elements next to the-binones would make the "sysext payloads" line in the description exact.release-dry-runchecks outpull_request.head.ref(the branch name), the same asbuild, so a push landing mid-run could pair the scripts from one commit with the artifacts from another. Usinghead.shain both jobs would close that gap, but the existingbuildjob already behaves this way.- This PR and #270 both edit
build.yml; whichever lands second will need a rebase.
A wall-clock creation time made every rebuild of the release set differ and defeats the reproducibility check proposed in projectbluefin#281. Assisted-by: Claude Opus 5.5 via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The SBOM listed k0s-bin, kubeadm-bin and openzfs but not the sysexts
that ship them, so their local sources (files/{k0s,kubeadm,zfs}/sysext)
were missing, while the KubeStellar sysext was listed directly. A sysext
only build-depends on its payload, which collect_manifest does not
follow, so both stay listed: 183 -> 186 SPDX packages.
Assisted-by: Claude Opus 5.5 via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Both checked out pull_request.head.ref, the branch, so a push landing mid-run could pair release-dry-run's scripts with artifacts built from another commit. Check out head.sha on pull requests and github.sha otherwise, the commit release passes to gh release --target. boot-test keeps head.ref: projectbluefin#270 rewrites that job. Assisted-by: Claude Opus 5.5 via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Thanks @castrojo. Your review was dismissed by my later push, so here is what changed since e31529b. Head is now 941edb7. ffa818c: SBOM timestamp from 8c2ff71: the SBOM lists the sysext elements.
941edb7:
main. Checks.
|
Takes projectbluefin#270's changes/build/boot-test rows in the ci-tooling.md job table and keeps this branch's release and release-dry-run rows. build.yml merged cleanly; the update sets go to the separate image-update artifact, so the release set that publish-release.sh verifies is unchanged. Assisted-by: Claude Opus 5.5 via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Reviewed at head 7a42b26.
correctness
elements/oci/bluefin-server-image.bst:167stampscreationInfo.createdfrom${SOURCE_DATE_EPOCH}, but inside the BuildStream sandbox that variable is the fixed default1321009871(BuildStreamprojectconfig.yaml;project.confsets noenvironment:override). Every SBOM will therefore carrycreated: 2011-11-11T11:11:11Z. The PR body anddocs/skills/systemd-sysupdate-verification.md:116describe this as stamping a creation time, andscripts/publish-release.sh:121only checks that the field is a string, so nothing catches it. Either feed a real timestamp (e.g. viaspdx-info.created, which the plugin already supports, or derive it from the version) or document that the date is intentionally constant for reproducibility.
docs-currency
docs/skills/systemd-sysupdate-verification.md:119saysscripts/publish-release.sh"is the only publish path", butJustfile:138(publish-oci) still runs its ownoras pushwithout the newrevisionannotation or post-push checks, anddocs/skills/ddi-installer.md/AGENTS.mdstill point at it. Either route the recipe through the script or narrow the claim to CI.
No findings from: security, intent-alignment, style.
Confidence: 3/5 (needs attention) — 1 perspective requested changes
— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88
The first release from main (run 36482792704) failed at the oras step, because that code had never run before merge. This PR runs the publish path on every pull request, adds provenance attestations, and ships an SBOM.
One publish path:
scripts/publish-release.shverify DIR VER KEYRING: runsgpgvagainst the keyring and checks that everySHA256SUMSentry is present and matches (sha256sum --check --strict). It also checks that every top-level file a release would publish is listed, that each file name belongs toVER(exactly one file per expected kind; k0s has its own version axis), and that the SBOM is SPDX 2.3 with unique SPDXIDs and a creation time. It writes the attestation subjects file.release DIR VER [--dry-run]:gh release create v<ver> --target <sha>(the--targetis new: before, the tag landed on whatever main's HEAD was at the time). With--dry-runit prints the command instead of running it.oci DIR VER REF [--plain-http] [--pull-back]:oras push(same artifact type and annotations, plusrevision), then checks that:latestresolves to the same digest and compares the registry manifest's layer titles and digests with the local files.--pull-backalso pulls the artifact and re-checks it.CI (
build.yml; only thereleasejob and a new job changed)release(main): checks out the repo, verifies against the committedfiles/os/sysupdate-keys/import-pubring.gpg, attests, creates the release, pushes to OCI, attests the digest. Addsid-token: write,attestations: writeandartifact-metadata: write.release-dry-run(new,pull_request,needs: build): sameverify/release --dry-run/ocicommands against aregistry:3.1.2@sha256:…service container with--plain-http --pull-back. Onlycontents: read, no secrets. Uses the samesetup-oraspin;tests/unit/test_release_workflow.pyenforces that the pins match and that both jobs run the same commands.actions/attest@v4.2.2, pinned by SHA.attest-build-provenancev4 is now a thin wrapper around it, and upstream recommends it for new code. It runs four times: provenance and SBOM over every published file (subject-checksums), and provenance and SBOM over the OCI digest withpush-to-registry: true.push-to-registry, the Sigstore bundle is already stored in ghcr.io as an OCI referrer and is signed by the same keyless workflow identity.cosign verify-attestation --new-bundle-formatverifies it, so acosign signwould only repeat that identity.SBOM
collect_manifestplugin (buildstream-plugins-community, already our junction) emits SPDX 2.3 directly (output-type: spdx), so no conversion script is needed.oci/bluefin-server-sbom.bstcovers the runtime closure of the payload: the os and initrd stacks, the kernel, the k0s and kubeadm binaries, OpenZFS, and the KubeStellar sysext.creationInfo.createdand dedupes packages that appear twice with identical content: FSDK bootstrap elements are reached through two junction paths.bluefin-server_<ver>.spdx.json, insideSHA256SUMS, so the GPG signature covers it.sysupdate-keys/import-pubring.pgp. This is how the dry run checks a PR's throwaway key. Likeefi-keys/, the directory is not published, because releases carry top-level files only.Docs
systemd-sysupdate-verification.mdnow covers provenance, the SBOM and the verify commands (gh attestation verify …,cosign verify-attestation …), and fixes the danglingsigning-and-sbomreference.ci-tooling.mdgets the job table, permissions and checklist updates.Verification (local)
just export-image) plus the exact CI commands againstregistry:3.1.2(pinned digest) in podman:just validatepasses; pytest 388 passed in a detached worktree without boot-keys; bats has no failures;docs-checks.pypasses; shellcheck is clean; actionlint shows only the existingubuntu-26.04label warning.