This document is the step-by-step procedure for releasing
big-code-analysis. It describes what to do, in what order, and what
to check when something looks wrong.
Status. The release pipeline described here is being built up in stages (
S1–S8of the public-release roadmap). The repository currently ships with a Cargo workspace, the MSRV declaration, the CHANGELOG, and the contributor docs. The signed-artefact pipeline, minisign key, packaging matrix, and external taps/buckets land in the remaining stages. Sections below that describe in-flight pieces say so explicitly.
The pipeline, once landed, is defined in
.github/workflows/release.yml. Everything downstream of git push --tags is automated.
The workspace pins MSRV at Rust 1.94 via
[workspace.package] rust-version = "1.94". Every member crate
inherits this with rust-version.workspace = true.
Rationale:
- Edition 2024 is the active edition for every crate;
let-else, let-chains, and the relaxed lifetime-elision rules used acrosssrc/languages/require Rust 1.85+, but several individual improvements rely on later releases (e.g. const slice indexing stabilizations, refined drop-order semantics). - Treating 1.94 as the floor avoids "works on my machine" reports where a contributor on a slightly older toolchain hits an edition-2024 surprise that the CI image silently papers over.
- 1.94 is the toolchain the
msrvjob in.github/workflows/ci.ymlexercises (the rest of the CI matrix usesstable). Lowering MSRV without updating that job is meaningless; raising MSRV without updating it is a foot-gun. (A repo-rootrust-toolchain.tomlpin is on the roadmap but not yet committed; once it lands, treat it as the third point of truth that must move in lockstep.)
Bumping MSRV is a deliberate workspace-wide change: update
[workspace.package] rust-version, the CI matrix, and any clippy
msrv directives in lockstep (plus rust-toolchain.toml once it
lands). Note the bump in the CHANGELOG under ### Changed.
One push of a v* tag will run this end-to-end:
- preflight — validates the tag, checks
Cargo.tomlversion parity against[workspace.package] version, confirmsminisign.pubis not a placeholder, and extracts the matchingCHANGELOG.mdsection as release notes. - build — cross-compiles
bcaandbca-webfor the target matrix: Linux gnu/musl × x86_64/aarch64, macOS aarch64, Windows x86_64/aarch64.x86_64-unknown-freebsdis tracked separately (see #346 under Known pipeline issues). Strips binaries, captures debug symbols, and produces per-target.tar.gz/.ziparchives. - package-* — builds
.deb,.rpm,.apk, and any other OS packages from the staged binaries. - smoke-* — installs each package inside the appropriate
container/VM and asserts
bca --versionandbca-web --versionmatch the tag. - sign-attest — flattens every artefact into
release/, generates CycloneDX SBOMs, computesSHA256SUMS, signs it with minisign, and attaches SLSA build provenance. - publish — creates/updates the GitHub Release, attaches every
artefact +
SHA256SUMS+SHA256SUMS.minisig, and (for non pre-releases, subject to the gating variables below) pushes the Homebrew formula and Scoop manifest. - publish-crates — for non pre-releases, subject to the gating
variables below, runs
cargo publishfor each publishable workspace crate in dependency order: the fivebca-tree-sitter-*grammar leaves first, thenbig-code-analysis(library), thenbig-code-analysis-cliandbig-code-analysis-web. Skips idempotently if the version is already on crates.io. - verify — downloads the published musl tarball back out of the release, verifies the minisign signature, checksum, and SLSA provenance.
If any stage fails, nothing downstream runs. publish and
publish-crates are the only jobs that mutate anything outside this
repo; they run in parallel so a crates.io failure does not block the
GitHub Release's verify step (and vice versa).
The same v* tag push also triggers two independent PyPI wheel
workflows that are not part of release.yml and run fully in
parallel with it (a failure in one does not block the others):
python-wheels.ymlpublishes the importable library bindings (big-code-analysis) — an abi3 extension wheel. See Python wheels (PyPI).python-cli-wheels.ymlpublishes thebcacommand-line tool (big-code-analysis-cli) — a-b binwheel that dropsbcaontoPATH. See CLI wheels (PyPI).
Both read the workspace version (dynamic = ["version"]), so they
publish in lockstep with the crates above on every bump — no separate
version step. Their one-time Trusted-Publisher setup is in the
Post-public-release checklist; after
that they fire automatically on each tag.
The repository is staging for a future public release. Until the maintainer flips the dials, the workflow must not push to crates.io, Homebrew, or Scoop, even on a stable tag. This is enforced by three repo-level GitHub Actions variables (Settings → Secrets and variables → Actions → Variables), each defaulting to unset:
| Variable | Gates |
|---|---|
ENABLE_CRATES_PUBLISH |
The publish-crates job. |
ENABLE_HOMEBREW_TAP |
The Homebrew formula push inside publish. |
ENABLE_SCOOP_BUCKET |
The Scoop manifest push inside publish. |
Each variable is skipped when unset or set to anything other than the
literal string true. Each gated step uses an if: guard of the
shape:
if: vars.ENABLE_CRATES_PUBLISH == 'true'
&& needs.preflight.outputs.prerelease != 'true'So:
- Pre-release tags (
-rc1,-beta2,-alpha3) never publish externally, regardless of the variable. - A stable tag with the variable unset still produces signed artefacts on the GitHub Release; it just does not push to crates.io or downstream package managers.
To turn on publication for the public-release cutover, set the
relevant variable(s) to the literal string true. Leave them unset
to keep the dry-run posture.
The workspace vendors five tree-sitter grammar crates under path dependencies. As of issue #149, they publish to crates.io under project-namespaced names so they don't collide with the Mozilla-published originals (which sit at older versions and a different owner):
| Path-dep directory | Published crate name | Rust import path |
|---|---|---|
tree-sitter-ccomment |
bca-tree-sitter-ccomment |
tree_sitter_ccomment |
tree-sitter-mozcpp |
bca-tree-sitter-mozcpp |
tree_sitter_mozcpp |
tree-sitter-mozjs |
bca-tree-sitter-mozjs |
tree_sitter_mozjs |
tree-sitter-preproc |
bca-tree-sitter-preproc |
tree_sitter_preproc |
tree-sitter-tcl |
bca-tree-sitter-tcl |
tree_sitter_tcl |
Each leaf manifest sets [lib] name = "tree_sitter_<lang>" so the
produced Rust crate keeps its original import path even though the
published package name is bca-tree-sitter-<lang>. The workspace
alias in the root Cargo.toml (and enums/Cargo.toml) uses Cargo's
package = ... aliasing so every consumer site reads
tree-sitter-<lang> = { workspace = true } as before — call sites
under src/, enums/, and feature flags did not change.
Publish order is leaf-first. The publish-crates job in
release.yml publishes the five bca-tree-sitter-* crates ahead of
big-code-analysis, because the parent's =<leaf-version> pin can
only resolve once each leaf is on crates.io. The sparse-index
existence check in each step makes the job idempotent across re-runs
of the same tag.
Bootstrap on the very first release. The parent's
cargo publish --dry-run -p big-code-analysis cannot resolve until
the five leaves are on crates.io. The preflight job in release.yml
handles this automatically: it queries the sparse index for
bca-tree-sitter-ccomment at the workspace-pinned version, and only
runs the parent dry-run if that leaf is already published. On the
first tag with ENABLE_CRATES_PUBLISH=true, the parent dry-run is
skipped with a ::notice:: and the publish-crates job uploads the
five leaves first, then big-code-analysis, then the binaries —
in one workflow run, no manual intervention. From the second tag
onwards the parent dry-run becomes a hard gate.
make release-check VERSION=… mirrors the same logic: it
unconditionally dry-runs the five leaves, then wraps the parent
dry-run in a warning that points back to this section if the
bootstrap state is detected.
Lockstep version policy. Every crate in this repository — the
library, the CLI, the web crate, the Python crate, the enums /
xtask helpers, and the five bca-tree-sitter-* vendored grammar
leaves — shares one version number. There is no per-crate version
drift. A version bump touches:
[workspace.package] versionin the rootCargo.toml— this covers every workspace member that declaresversion.workspace = true.[package] versioninenums/Cargo.toml(excluded from the workspace; cannot inherit).[package] versionin each of the fivetree-sitter-<lang>/Cargo.tomlfiles (also excluded).- The
version = "=<new>"pin on everybca-tree-sitter-*entry in[workspace.dependencies](rootCargo.toml) and the matching block inenums/Cargo.toml. - The
version = "=<new>"pin on thebig-code-analysispath-dep inbig-code-analysis-cli/Cargo.tomlandbig-code-analysis-web/Cargo.toml. - The hard-coded version references in user-facing docs
(
README.md,STABILITY.md, the book'squick-start.mdandcargo-features.md) and the install snippet in every leaf'sbindings/rust/README.md(5 files), since those ship inside the publishedbca-tree-sitter-*tarballs and render as the crates.io landing page. - The man pages (re-run
cargo run -p xtask). - The SARIF tool-version snapshots (re-run
cargo insta testand accept). - A new
## [<new>]section inCHANGELOG.md(the unreleased block is collapsed into it at release time).
Run ./check-versions.py (also wired into make pre-commit and
the lint job in .github/workflows/ci.yml) after editing to
catch any item the human eye missed.
A grammar refresh (recreate-grammars.sh regenerates the parsers)
is a normal change under the current version — bumping the
grammars does not bump the version on its own. The next workspace
release picks up the regenerated grammars at whatever leaf version
already matches the workspace version.
You only need to do this once per project, but verify each item before the first real release.
Configure these under Settings → Secrets and variables → Actions → Secrets:
| Secret | Purpose |
|---|---|
MINISIGN_SECRET_KEY |
minisign secret key, signs SHA256SUMS. |
MINISIGN_PASSWORD |
Password for the minisign key. |
ALPINE_ABUILD_KEY_PRIV |
abuild RSA private key (Alpine .apk signing). |
ALPINE_ABUILD_KEY_PUB |
Matching abuild public key. |
HOMEBREW_TAP_TOKEN |
Fine-grained PAT for the Homebrew tap repo. |
SCOOP_BUCKET_TOKEN |
Fine-grained PAT for the Scoop bucket repo. |
The two PATs need write access to
dekobon/homebrew-tap (shared tap; the workflow only touches
Formula/big-code-analysis.rb) and dekobon/scoop-bucket (shared
bucket; the workflow only touches bucket/big-code-analysis.json)
respectively. Both are minted at
https://github.com/settings/personal-access-tokens/new as
fine-grained PATs with Repository access: Only select repositories
(scoped to the single tap or bucket repo) and Repository permissions
→ Contents: Read and write — leave every other permission at No
access. Store each token under Settings → Secrets and variables →
Actions → Secrets on dekobon/big-code-analysis.
crates.io authentication uses
Trusted Publishing — no
long-lived CARGO_REGISTRY_TOKEN is stored as a secret. The
publish-crates job mints a GitHub OIDC ID token and exchanges it for
a short-lived registry token scoped to that run.
If HOMEBREW_TAP_TOKEN or SCOOP_BUCKET_TOKEN is missing — or if the
target tap/bucket repo is unreachable (deleted, renamed, or the PAT
cannot see it) — the corresponding step emits a GitHub Actions
warning and skips without failing the release.
minisign.pub at the repo root must be a real public key, not a
committed placeholder. The preflight job greps for the placeholder
comment and aborts if it is still present.
To create a fresh key:
minisign -G -p minisign.pub -s minisign.keyCommit minisign.pub. Store minisign.key as the
MINISIGN_SECRET_KEY repo secret via stdin redirection — do not
paste the contents into the web UI:
gh secret set MINISIGN_SECRET_KEY -R dekobon/big-code-analysis < minisign.key
# The second command opens an interactive prompt on stdin; type the
# password, press Enter, then Ctrl-D to signal EOF.
gh secret set MINISIGN_PASSWORD -R dekobon/big-code-analysisA minisign secret key file is two lines and ends with \n. Paste-via-
UI silently strips the trailing newline (and can introduce other
whitespace artefacts) so that minisign -S later fails with Error while loading the secret key file — masquerading as a wrong-key /
wrong-password failure when the bytes are actually one newline short.
Stdin redirection from the file preserves the exact file bytes —
including the trailing newline that the web UI eats. Keep
minisign.key itself out of the repo.
Stable releases push to (subject to the gating variables above):
dekobon/homebrew-tap— shared Homebrew tap; the release workflow commits onlyFormula/big-code-analysis.rband leaves the other formulae in the tap untouched.dekobon/scoop-bucket— shared Scoop bucket; the release workflow commits onlybucket/big-code-analysis.jsonand leaves the other manifests in the bucket untouched.- crates.io — leaf-first: the five
bca-tree-sitter-*grammar crates, thenbig-code-analysis(library), thenbig-code-analysis-cliandbig-code-analysis-web. See crates.io ownership for the publish loop and rate-limit details.
Both tap and bucket repos must exist and accept the configured PAT.
Before the first automated publish you must manually claim all eight
crate names — the five bca-tree-sitter-* leaves plus the three
top-level crates. The publish-crates job in release.yml uses
Trusted Publishing which requires the crate to exist before TP can be
registered, so the very first publish has to be a hand-rolled
cargo publish from your workstation.
-
Check name availability. Open each of the following on
https://crates.io/crates/<name>:bca-tree-sitter-ccomment,…-mozcpp,…-mozjs,…-preproc,…-tclbig-code-analysisbig-code-analysis-clibig-code-analysis-web
If any name is taken by someone else, pick a different name and update the matching
[package].name(and the workspace alias for leaves) before tagging —cargo owner --addonly works on crates you already own. -
Verify the parent's
includewhitelist is present. The[package].include = […]block in the rootCargo.tomlrestricts the published.cratetosrc/**,Cargo.toml,README.md,LICENSE, andCHANGELOG.md. Without it,cargo publishpackages the entire repo — notablytests/repositories/(~130 MiB compressed of snapshot fixtures) — and the upload fails against crates.io's size limit with a Varnish503 backend write errorrather than a useful error message. Verify before the first publish:cargo package -p big-code-analysis --allow-dirty --no-verify ls -lh target/package/big-code-analysis-*.crate # expect ≲ 1 MiB
If the
.crateis larger than a few MiB, fix theincludeblock before continuing. -
Publish leaf-first, with rate-limit pacing. crates.io rate-limits new crates at roughly one per ten minutes after a short burst. Publishing all eight in a single pass will trip the limit; the second-half publishes return
429 Too Many Requestswith an explicittry again after <timestamp>hint. The simplest workaround is to retry on a loop:cargo login <your-token> # Leaf-first — the parent's `=<leaf-version>` pin cannot resolve # until each leaf is on the sparse index. cargo publish waits # for the index to catch up, so the next publish can resolve the # previous one without an explicit sleep. for d in tree-sitter-{ccomment,mozcpp,mozjs,preproc,tcl}; do until cargo publish --locked --manifest-path "$d/Cargo.toml"; do sleep 60; done done # Parent + binaries. These will hit the new-crate rate limit on # the first try; the until-loop retries every 60s until cargo # exits 0. until cargo publish -p big-code-analysis --locked; do sleep 60; done until cargo publish -p big-code-analysis-cli --locked; do sleep 60; done until cargo publish -p big-code-analysis-web --locked; do sleep 60; done
After all eight crates are on the registry, the
publish-cratesjob's idempotency check makes it a no-op for any tag at the same version. -
Add additional owners.
cargo owner --add <github-handle> <crate>for each of the eight crates. A single-owner crate is one forgotten password away from being orphaned. If you have a GitHub team, usegithub:<org>:<team>. -
Register a Trusted Publisher for each crate (see below). This replaces any long-lived API token a future contributor might otherwise wire into the workflow.
Trusted Publishing lets the release workflow authenticate to crates.io via a short-lived OIDC token instead of a static API token. Two one-time setup steps are required on top of the crates.io ownership checklist above:
-
Create a
releaseGitHub Environment. Go to Settings → Environments → New environment and name it exactlyrelease. Thepublish-cratesjob references this environment and the crates.io trusted publisher matches theenvironmentOIDC claim against it. Optional protection rules (required reviewers, deployment branch filters) act as a manual gate on every publish — the environment is the right place to add them, not the workflow. The name must match the TP registration exactly; a typo here is the most common self-inflicted failure mode. -
Register a Trusted Publisher for each of the eight crates. On crates.io, open the settings page for each of the five
bca-tree-sitter-*leaves,big-code-analysis,big-code-analysis-cli, andbig-code-analysis-web. In the Trusted Publishing section, add a GitHub publisher with:- Repository owner:
dekobon. - Repository name:
big-code-analysis. - Workflow filename:
release.yml(basename only, not a path). - Environment:
release.
Every publishable crate needs its own trusted-publisher entry — a TP registered on
big-code-analysisdoes not cover the CLI, the web crate, or any of the leaves. The workflow still performs a singleauthexchange for all publishes because crates.io issues one token covering every crate whose TP config matches the JWT claims. - Repository owner:
-
First stable release after cutover validates the path. The prerelease gate (
if: needs.preflight.outputs.prerelease != 'true') skipspublish-cratesfor-rctags, so TP cannot be rehearsed viaworkflow_dispatch. The first non-prerelease tag after the cutover, withENABLE_CRATES_PUBLISH=true, is the real end-to-end test. Watch theauthstep logs.
The release pipeline is strict about version parity: the preflight job
rejects the tag if it does not match the workspace version, and the
smoke jobs reject the build if bca --version does not contain the
tag string. Bump the version deliberately, in one commit, before
tagging.
Member crates inherit their version from [workspace.package], so
edit these in lockstep:
- Root
Cargo.toml,[workspace.package] version = "x.y.z"— the canonical version that every member crate picks up viaversion.workspace = true. - Any
[workspace.dependencies]entries that pin an internal crate (e.g.big-code-analysis = { path = "...", version = "x.y.z", ... }). Must match the workspace version, otherwisecargo publishon the dependent crate will reject the dependency. - The
enums/helper crate (excluded from the root workspace). Its own[package] versioncarries the same value — bump it alongside the workspace bump, never on its own. - Each
tree-sitter-<lang>/Cargo.toml(also excluded). Same discipline asenums/: bump in lockstep with the workspace.
After editing, regenerate the lockfile and sanity-check the bump:
cargo update --workspace
cargo metadata --format-version 1 --no-deps \
| python3 -c "import json,sys; d=json.load(sys.stdin); \
print({p['name']: p['version'] for p in d['packages']})"
# Expect big-code-analysis, big-code-analysis-cli, and
# big-code-analysis-web at the target version.The cargo update --workspace step is mandatory, not
nice-to-have: publish-crates runs cargo publish --locked, which
fails late in the release pipeline if Cargo.lock drifts from what
the workspace resolves to. Commit the refreshed lockfile alongside
the Cargo.toml edits.
Regenerate the committed man pages in the same release-prep commit:
cargo xtaskman/*.1 embeds both the binary version (big-code-analysis x.y.z
in the .TH line and vX.Y.Z in .SH VERSION) and the live clap
schema, so any version bump — workspace-wide or CLI-only (e.g. the
big-code-analysis-cli version override at #235) — leaves the
committed pages stale. The per-PR man pages up to date CI job
gates against drift; release.yml regenerates the pages again per
build leg so the shipped artefacts cannot ship with a stale schema,
but committing the regenerated pages keeps the gate green between
release-prep and tag push. Same rule applies any time a CLI flag is
added or renamed — not just at release time.
Pick the version using semver. The workspace is on the 1.x line
and ships under the STABILITY.md contract: the public
Rust API surface (big-code-analysis library re-exports, the bca
CLI argument grammar, and the bca-web REST schema) is held stable
across patch and minor bumps. Additive shape changes (new items,
new LANG / MetricsError variants, new language features) belong
under a minor bump. Breaking shape changes are reserved for the
next major bump and must be called out under (breaking) in the
CHANGELOG entry for that release; do not slip a SemVer break into a
patch or minor bump. Metric value drift from a grammar pin move or
a metric-definition fix remains allowed under patch and minor bumps
and must be listed in the entry that introduces it.
Commit the version bump together with the changelog move (see below) so the release-prep commit is a single, self-contained change:
chore(release): prepare v1.2.0
Before tagging, on main:
- All intended changes are merged and CI is green.
- Workspace version is bumped per
Bumping the version — all
Cargo.tomlsites, plus a refreshedCargo.lock. -
cargo xtaskhas been run and the resultingman/*.1edits are committed in the release-prep commit.git diff man/after a freshcargo xtaskmust be empty. -
CHANGELOG.mdhas a## [x.y.z]section with the release notes. The header must match the tag exactly, minus the leadingv. Move entries out of## [Unreleased]into the new section. -
cargo test --workspace --all-featurespasses locally (including integration snapshots — initialize submodules first). -
minisign.pubis a real key (rungrep '^untrusted comment: placeholder' minisign.pub— it should print nothing). - Parent crate packages to a sane size —
cargo package -p big-code-analysis --allow-dirty --no-verifyfollowed byls -lh target/package/big-code-analysis-*.crateshould show well under 10 MiB (the crates.io upload ceiling). If it balloons, the[package].includeblock has regressed or a newly-added directory needs to be excluded; see crates.io ownership. - The defer-and-gate variables (
ENABLE_CRATES_PUBLISH,ENABLE_HOMEBREW_TAP,ENABLE_SCOOP_BUCKET) are set to the intended state for this release.
Commit and push these changes. The final commit on main before
tagging should be the release-prep commit.
Pick a semver version (e.g. 1.2.0). The tag is the version prefixed
with v.
# From a clean main checkout at the release-prep commit:
git tag -a v1.2.0 -m "v1.2.0"
git push origin v1.2.0That's it — the push of the tag triggers release.yml and the two
PyPI wheel workflows (python-wheels.yml for the library bindings,
python-cli-wheels.yml for the bca CLI), all in parallel. Watch all
three in the Actions tab:
gh run watch
# or, per workflow:
gh run list --workflow=Release
gh run list --workflow="Python wheels"
gh run list --workflow="Python CLI wheels"The wheel workflows publish to PyPI automatically once their one-time Trusted Publishers are registered (see the Post-public-release checklist); no per-release action beyond the tag is needed. Confirm both wheels landed in Post-release verification.
Pre-release tags match vX.Y.Z-<suffix> where <suffix> is
[A-Za-z][0-9]* — e.g. v1.2.0-rc1, v1.2.0-beta2,
v1.2.0-alpha3. Do not use dotted forms like v1.2.0-rc.1:
Alpine's abuild grammar rejects dots in the pre-release suffix.
The preflight classifier sets prerelease=true for any suffix, which:
- Marks the GitHub Release as a pre-release.
- Skips the Homebrew tap, Scoop bucket, and crates.io publish steps
regardless of the defer-and-gate variables. crates.io uploads are
irrevocable, so rehearsal tags like
v0.0.0-test1must not reach the registry.
Use this for any version that should not reach package managers. Signed artefacts, SBOMs, and SLSA provenance still publish normally, so a pre-release is a full test of everything except the external pushes.
Both PyPI wheel workflows still build and smoke-test every wheel on
a pre-release tag but skip the PyPI publish step, so a pre-release
never lands a wheel on PyPI. The CLI wheel (python-cli-wheels.yml)
skips publish for any hyphenated suffix — !contains(github.ref, '-'), matching release.yml's *-* prerelease rule exactly; the
library wheel (python-wheels.yml) skips the recognised -rc /
-beta / -alpha suffixes. For the suffixes this project actually
uses (above), all three pipelines stay aligned — one tag cannot publish
a prerelease to one registry while skipping another.
The pipeline's own verify job downloads the musl tarball from the
published Release and re-runs minisign + SLSA verification. That
covers the critical path automatically.
Verify manually if you want extra assurance:
# From a fresh directory:
TAG=v0.1.0
VERSION=0.1.0
TARBALL="big-code-analysis-${VERSION}-x86_64-unknown-linux-musl.tar.gz"
gh release download "$TAG" -R dekobon/big-code-analysis \
-p "$TARBALL" -p SHA256SUMS -p SHA256SUMS.minisig
# Fetch minisign.pub from the tag, not main — if the key was rotated
# after this release, main has a different key and verification fails.
RAW_BASE="https://raw.githubusercontent.com/dekobon/big-code-analysis"
curl -fsSLO "${RAW_BASE}/${TAG}/minisign.pub"
minisign -Vm SHA256SUMS -p minisign.pub
grep "${TARBALL}" SHA256SUMS | sha256sum -c
gh attestation verify "${TARBALL}" -R dekobon/big-code-analysisCheck that the downstream package managers updated (only applicable once the corresponding gating variable is on):
- Homebrew tap: new commit on
dekobon/homebrew-taptouchingFormula/big-code-analysis.rb. - Scoop bucket: new commit on
dekobon/scoop-buckettouchingbucket/big-code-analysis.json.
Confirm both PyPI wheels published at the new version (these ship on every tag once their Trusted Publishers are registered). Either check the project pages — https://pypi.org/project/big-code-analysis/ and https://pypi.org/project/big-code-analysis-cli/ — or verify the CLI end-to-end from a clean environment:
VERSION=0.1.0
python -m venv /tmp/bca-rel && . /tmp/bca-rel/bin/activate
# Library bindings (importable module):
pip install "big-code-analysis==${VERSION}"
python -c "import big_code_analysis as bca; print(bca.__version__)"
# CLI tool (drops `bca` on PATH):
pip install "big-code-analysis-cli==${VERSION}"
bca --version # must print the tagged version
deactivateThe first time the repository goes public and a stable release is cut, complete the items below in order. None of them belongs in the per-release flow, but skipping any of them on the cutover release turns into a foot-gun on the next release.
- crates.io ownership and Trusted Publisher. For each of
the eight publishable crates (the five
bca-tree-sitter-*leaves,big-code-analysis,big-code-analysis-cli,big-code-analysis-web): claim the name with a manualcargo publish(leaf-first, retry on the new-crate rate limit — see crates.io ownership for the loop), add at least one co-owner viacargo owner --add, and register a Trusted Publisher (repo ownerdekobon, repobig-code-analysis, workflowrelease.yml, environmentrelease). - PyPI Trusted Publisher and
pypiGH environment. Claimbig-code-analysison PyPI via the pending-publisher flow at https://pypi.org/manage/account/publishing/ (registers the TP and reserves the name in one step), and create thepypiGitHub environment so protection rules can attach before the first wheel publish. See Python wheels (PyPI). -
python-wheelsPR label. Create the label (see the Python wheels section) so contributors can opt PRs into the wheel matrix. - PyPI Trusted Publisher and
pypi-cliGH environment (CLI wheel). Claimbig-code-analysis-clion PyPI via the pending-publisher flow, registering a TP for workflowpython-cli-wheels.yml+ environmentpypi-cli, and create thepypi-cliGitHub environment. See CLI wheels (PyPI). -
python-cli-wheelsPR label. Create the label (see the CLI wheels section) so contributors can opt PRs into the CLI wheel matrix. - Shared Homebrew tap reachable. Confirm
dekobon/homebrew-tapexists and the configured PAT can push to it. The release workflow appendsFormula/big-code-analysis.rbto that tap alongside the other formulae; no dedicated tap repo is required. - Shared Scoop bucket reachable. Confirm
dekobon/scoop-bucketexists and the configured PAT can push to it. The release workflow appendsbucket/big-code-analysis.jsonalongside the other manifests; no dedicated bucket repo is required. - Fine-grained PATs minted and stored. Generate
HOMEBREW_TAP_TOKENandSCOOP_BUCKET_TOKENas fine-grained PATs scoped to the tap and bucket repos respectively, with write access only. Store under Settings → Secrets and variables → Actions. - Repo secrets and variables wired. Confirm
MINISIGN_SECRET_KEY,MINISIGN_PASSWORD, the Alpine abuild pair (if Alpine ships),HOMEBREW_TAP_TOKEN, andSCOOP_BUCKET_TOKENare all present. Confirm the defer-and-gate variables (ENABLE_CRATES_PUBLISH,ENABLE_HOMEBREW_TAP,ENABLE_SCOOP_BUCKET) are set totruefor the cutover release. - First release tag. Cut the first stable tag with all gates
on. Watch the
publish-crates,homebrew-tap-push, andscoop-bucket-pushjobs end-to-end. Theverifyjob's success on the published tarball is the canonical "release is done" signal. - Delete any stray
CARGO_REGISTRY_TOKENsecret after the first successful TP-authenticated release. Leaving it around is not actively harmful (nothing references it), but deleting it removes a tempting footgun for a future contributor.
Python bindings ship via .github/workflows/python-wheels.yml, not
release.yml. The two workflows trigger on the same v* tag push
but run in parallel — a crates.io publish failure does not block the
PyPI upload, and vice versa.
What the python-wheels pipeline does:
- build —
PyO3/maturin-action@v1.51.0builds a manylinux_2_28 abi3 wheel onubuntu-latest(x86_64) andubuntu-24.04-arm(aarch64).[tool.maturin].featuresinbig-code-analysis-py/pyproject.tomlpinspyo3/extension-module+pyo3/abi3-py312so the wheel uses the limited (stable) Python C API and targets CPython 3.12+ forward-compatibly. One wheel per architecture covers every future 3.12+ minor release. - sdist —
maturin sdistproduces a source distribution as the PyPI fallback for niche architectures and a reproducibility anchor for the wheels. - smoke-test — pulls each wheel onto a clean runner of the
matching architecture, installs it with
pip install --no-index --find-links=dist big-code-analysis, and asserts that the public API surface (analyze_source,flatten_spaces,to_sarif,language_for_file) loads and produces the documented dict shape under both Python 3.12 and 3.13. An abi3 wheel that loaded on 3.12 but failed on 3.13 (the most plausible silent forward-compat regression) trips here. - publish — gated on a
v*tag and thepypideployment environment. Authentication is via PyPI Trusted Publishing (OIDC); the workflow has noPYPI_API_TOKENsecret to leak. PEP 740 Sigstore attestations are generated automatically bypypa/gh-action-pypi-publish@v1.14.0.
Before the first v* tag is cut after the cutover, complete these
on PyPI as the maintainer:
-
Claim the project name. Open
https://pypi.org/project/big-code-analysis/. If the name is taken by another project, pick a different name and bump[project] nameinbig-code-analysis-py/pyproject.tomlbefore tagging. -
Register a Trusted Publisher. Under
https://pypi.org/manage/account/publishing/(for a brand new project, the pending publisher flow at the same URL works the same way), add a GitHub publisher with:- PyPI Project Name:
big-code-analysis. - Owner:
dekobon. - Repository name:
big-code-analysis. - Workflow filename:
python-wheels.yml(basename only). - Environment name:
pypi.
The environment name is intentionally distinct from the
releaseenvironment used by the crates.io trusted publisher inrelease.yml— keeping them separate prevents the OIDCenvironmentclaim from accidentally satisfying the wrong registry's TP entry. - PyPI Project Name:
-
Create the
pypiGitHub Environment. Settings → Environments → New environment →pypi. The publish job references this environment; protection rules (required reviewers, branch / tag filters) attached here are the right place to add a manual approval gate on every wheel publish.⚠️ GitHub will auto-create a referenced-but-undefined environment with no protection rules the first time the workflow runs. Create the environment manually before the firstv*tag if you want the approval gate to apply on the first publish — otherwise the cutover release goes through immediately with no manual checkpoint. -
Create the
python-wheelsPR label. The wheel build / sdist / smoke-test jobs are gated by apython-wheelslabel on PRs so Rust-only PRs that happen to share a path-filter neighbour (e.g.Cargo.lock) do not pay the wheel-matrix cost. GitHub does not auto-create custom labels — until the label exists, contributors cannot opt PRs into wheel verification. One-off via theghCLI:gh label create python-wheels \ --color 1d76db \ --description "PR opts in to the manylinux wheel CI matrix"Tag pushes and
workflow_dispatchruns ignore the label — they always build the full matrix. -
First tagged release validates the path. Trusted Publishing cannot be rehearsed via
workflow_dispatch(the environment claim mismatches). The first non-prereleasev*tag after registration is the canonical end-to-end test — watch thepublishjob's log for the OIDC exchange and the attestation upload.
big-code-analysis-py inherits its version from
[workspace.package] version via version.workspace = true in its
Cargo.toml, and pyproject.toml reads the same value at build
time (dynamic = ["version"]). The "Bumping the version" steps
above are therefore sufficient — there is no separate
big-code-analysis-py/pyproject.toml version field to keep in sync.
workflow_dispatch from the Actions tab runs the full build +
smoke-test matrix without invoking the publish job (the if:
guard requires a v* tag push). Use this to validate a
release-prep branch before tagging.
To exercise the PyPI side end-to-end against
https://test.pypi.org/, temporarily change the
pypa/gh-action-pypi-publish step's repository-url input to
https://test.pypi.org/legacy/ and register a matching TP entry
on TestPyPI — keep this off main to avoid leaking a real upload
into a production-shaped flow.
The wheel pipeline ships Linux only (x86_64 + aarch64). macOS and
Windows wheels are tracked separately under
#103's
"Out of scope" section. (This Linux-only scope is for the library
bindings wheel above; the CLI bca wheel — see below — does ship
macOS and Windows.)
The bca command-line tool ships as its own pip-installable
distribution via .github/workflows/python-cli-wheels.yml, separate
from both release.yml (crates.io / native packages) and
python-wheels.yml (the library bindings). All three trigger on the
same v[0-9]* tag push and run in parallel; a failure in one does not
block the others.
This is a maturin -b bin wheel: the compiled bca binary is packaged
as a console script, so pip install big-code-analysis-cli drops bca
onto the user's PATH. Key differences from the library wheel:
- No abi3 / no per-Python matrix. A bin wheel is tagged
py3-none-<platform>; one wheel per (OS, arch) covers every CPython 3.x and PyPy. The matrix is per-platform, not per-Python-version. - Distribution name
big-code-analysis-cli, commandbca. The installed command intentionally differs from the dist name (thebcaname on PyPI is taken;big-code-analysisis the library bindings). - Full grammar set is inherited from the crate (
all-languages, via #252) — no[tool.maturin] featureswiring. - Wider platform matrix: Linux
manylinux_2_28(x86_64/aarch64), macOS (x86_64/arm64), Windows (x86_64). - Compliance artefacts ride in the wheel: the per-binary
THIRD-PARTY-LICENSES-bca.md(cargo-about) andLICENSEland in.dist-info/licenses/; thebcaman pages are bundled; maturin emits a CycloneDX SBOM into.dist-info/sboms/.
Mirror the library-wheel setup above, with CLI-specific values:
-
Claim the project name at
https://pypi.org/project/big-code-analysis-cli/(the pending-publisher flow reserves the name in the same step as the TP registration). The name was confirmed available when #408 was filed. -
Register a Trusted Publisher at
https://pypi.org/manage/account/publishing/with:- PyPI Project Name:
big-code-analysis-cli. - Owner:
dekobon. - Repository name:
big-code-analysis. - Workflow filename:
python-cli-wheels.yml(basename only). - Environment name:
pypi-cli.
pypi-cliis intentionally distinct from the library'spypienvironment and the crates.ioreleaseenvironment so each registry/project's OIDCenvironmentclaim is unambiguous. - PyPI Project Name:
-
Create the
pypi-cliGitHub Environment (Settings → Environments → New environment →pypi-cli) before the firstv*tag if you want a manual approval gate on the first publish — GitHub auto-creates a referenced-but-undefined environment with no protection rules otherwise. -
Create the
python-cli-wheelsPR label so contributors can opt a PR into the wheel matrix (Rust-only PRs that merely brush a path-filter neighbour skip it):gh label create python-cli-wheels \ --color 1d76db \ --description "PR opts in to the bca CLI wheel CI matrix"Tag pushes and
workflow_dispatchruns ignore the label. -
First tagged release validates the path, exactly as for the library wheel — Trusted Publishing cannot be rehearsed via
workflow_dispatch.
big-code-analysis-cli inherits its version from
[workspace.package] version (version.workspace = true), and its
pyproject.toml reads the same value at build time (dynamic = ["version"]). No separate version field to maintain.
- Generate a new keypair:
minisign -G -p minisign.pub.new -s minisign.key.new. - Replace
minisign.pubwith the new public key and commit it. - Update
MINISIGN_SECRET_KEYandMINISIGN_PASSWORDsecrets with the new values. Use stdin redirection —gh secret set MINISIGN_SECRET_KEY -R dekobon/big-code-analysis < minisign.key.new— to preserve the trailing newline of the key file; see Minisign key for why paste-via-UI bites. - Cut a new release — its
SHA256SUMS.minisigwill be signed with the new key, self-documenting the rotation.
Users verifying an older release still need the old minisign.pub
from that release's tagged commit.
The pipeline fails before publish on any preflight, build,
package, smoke, or sign-attest error, so a broken release almost
never reaches users. sign-attest is the latest hard-gate before
external state changes; it is the right place to expect a noisy red
if MINISIGN_SECRET_KEY is missing, corrupted, or doesn't pair with
MINISIGN_PASSWORD.
post-publish verify runs after publish and is an internal
sanity check — its failure does not invalidate the published
artefacts and does not roll back any external state. Treat a verify
red as a CI bug to triage, not as a botched release.
If publish itself partially succeeds (e.g. GitHub Release created but tap push failed), the fix is usually to re-run the workflow against the same tag — Actions tab → open the failed run → Re-run failed jobs (top-right of the run page). The pipeline is designed to be idempotent on re-run, and re-runs pick up freshly-set repo secrets without needing a force-retag.
If you need to pull a release entirely:
gh release delete vX.Y.Z --cleanup-tag --yesThen fix the underlying issue, bump to vX.Y.(Z+1), and re-tag.
Do not re-use a published version number — Homebrew/Scoop and
crates.io users may have already cached the old artefacts.
The recovery rule above (bump to the next patch version) is correct
for any release that already produced external state. On the very
first tag for a brand-new repo, before publish has touched
crates.io / Homebrew / Scoop, no downstream state exists yet to
poison — and bumping the version mid-cutover adds churn (workspace
version, man pages, SARIF snapshots, CHANGELOG section). In that
narrow window, force-moving the tag is the cheaper recovery:
# Fix and push the underlying issue first
git push origin main
# Move the tag to point at the fix
git tag -d vX.Y.Z
git push origin :refs/tags/vX.Y.Z
git tag -a vX.Y.Z -m "vX.Y.Z"
git push origin vX.Y.ZThis is only safe while:
- The GitHub Release object does not yet exist (or contains nothing irrevocable).
python-wheels.ymlhas not yet uploaded to PyPI (PyPI versions are immutable; a re-fire of the tag will trip the publish step but won't roll back). Accept that single noisy red if the wheels are already correctly on PyPI.- crates.io has not yet been told about this version. ANY publish
for the version — workflow-driven or manual
cargo publishfrom the maintainer's workstation — makes a force-retag inappropriate, because the published version is irrevocable (yank-able, not delete-able).
Outside that window, never force-move — use vX.Y.(Z+1).
Tracked as GitHub issues; a maintainer triaging a red run should check these first before deeper debugging:
- #346 —
x86_64-unknown-freebsddropped from the binary matrix; cross v0.2.5 + the vendored grammars' C++ scanners cannot link againstlibcxxrtwithout a deeper toolchain change. Restoration viavmactions/freebsd-vmis the queued remediation. While the target is absent, FreeBSD users install from source. - #351 —
post-publish verifyfails on a brand-new release becauseSHA256SUMSis emitted with./-prefixed filenames and the verify-step awk filter compares against the unprefixed basename. The artefacts themselves verify correctly with a manualsha256sum -c SHA256SUMS(sha256sum canonicalises./XtoX). Will be fixed alongside the producer in v1.0.1.