Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions .github/RELEASE_PR_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,17 @@

This PR was opened by `make release-cut VERSION=X.Y.Z` from `develop`.
It cuts the release branch, bumps the version sources, generates the
CHANGELOG entry from the `origin/main` patch delta, drops maintainer paths
CHANGELOG entry from the `origin/release` patch delta, drops maintainer paths
(`_project/`, `_blog/`, agent configs, dev-tooling root files), and queues the
change for a squash-merge into `main`.
change for a squash-merge into `release`.

### Reviewer checklist

- [ ] CHANGELOG.md has an accurate `## [X.Y.Z] - YYYY-MM-DD` entry
- [ ] `pyproject.toml` and `benchbox/__init__.py` show the new version
- [ ] Documentation landing pages and `landing/` show the new version
- [ ] `_project/`, `_blog/`, agent config dirs, and dev-tooling root files
were removed from this branch (verify with `git diff main...HEAD --stat`
were removed from this branch (verify with `git diff release...HEAD --stat`
— the diff should look like a curated subtree, not arbitrary edits)
- [ ] No surprise file additions (the curation only *removes*)
- [ ] `validate-base` is green for this release branch
Expand Down Expand Up @@ -54,21 +54,21 @@ context is missing, pending, skipped, failed, or canceled, stop and fix the
release PR or ruleset/workflow contract before rerunning the command.

After that pre-merge check passes, `release-finalize` squash-merges this PR,
fast-forwards `main`, tags `vX.Y.Z`, and pushes the tag — which fires
fast-forwards `release`, tags `vX.Y.Z`, and pushes the tag — which fires
`.github/workflows/release.yml`: `dependency-bounds` → `build` (with
`SOURCE_DATE_EPOCH` from the tag commit) → `publish` (PyPI trusted
publisher) → `github-release` → `test-installation` (cross-platform pip
install verification).

Push-to-main jobs are post-merge signals. They can fail after the tag has
Push-to-release jobs are post-merge signals. They can fail after the tag has
already started publication; recovery is a patch release or incident process,
not treating the public tag as if it had been blocked.

`develop` is intentionally NOT modified by `release-finalize`. Dev-only
paths (`_project/`, `_blog/`, agent configs, etc.) live only on develop
by design (per A3 in `_project/decisions/single-repo-migration.md`).

If anything fails downstream, fix on a new branch, PR to `main`, squash-merge,
If anything fails downstream, fix on a new branch, PR to `release`, squash-merge,
and bump to the next patch version (PyPI rejects re-uploads of an existing
version).

Expand Down
12 changes: 6 additions & 6 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: Documentation

on:
push:
branches: [main]
branches: [release]
paths:
- "benchbox/**"
- "docs/**"
Expand All @@ -19,7 +19,7 @@ on:
- "README.md"
- "pyproject.toml"
pull_request:
branches: [main, develop]
branches: [release, develop]
paths:
- "benchbox/**"
- "docs/**"
Expand Down Expand Up @@ -79,8 +79,8 @@ jobs:
# Steps gate on package.json as a sentinel so the workflow degrades
# gracefully if the explorer directory is ever removed or renamed.
# This is also the intended dormancy on the release branch: results-explorer/
# and _project/scripts/explorer_publish.py ship on develop, not main, so on
# a main build hashFiles(...) is empty and every explorer step below is a
# and _project/scripts/explorer_publish.py ship on develop, not release, so on
# a release build hashFiles(...) is empty and every explorer step below is a
# deliberate no-op rather than accidental dead code (see
# tests/uat/phases/explorer_smoke.py:explorer_present).
- name: Validate results-explorer lockfile
Expand Down Expand Up @@ -199,11 +199,11 @@ jobs:
with:
path: site

# Deploy to GitHub Pages (main pushes only)
# Deploy to GitHub Pages (release pushes only)
# Isolated as its own job so the github-pages environment protection
# rules don't gate the build job on PRs.
deploy:
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
if: github.event_name == 'push' && github.ref == 'refs/heads/release'
needs: build
runs-on: ubuntu-latest
environment:
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,10 @@ name: Lint
on:
push:
branches:
- main
- release
pull_request:
branches:
- main
- release

concurrency:
group: lint-${{ github.ref }}
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/perf-smoke.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: Performance Smoke

on:
pull_request:
branches: [main]
branches: [release]

concurrency:
group: perf-smoke-${{ github.ref }}
Expand Down
34 changes: 17 additions & 17 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,37 +12,37 @@ on:
type: boolean

jobs:
# Note: ruleset main-release-only gates the release-cut PR on lint + test
# before squash-merge to main, so the tag fired here is expected to have
# already been CI-validated. verify-tag-on-main below enforces that
# Note: ruleset release-only gates the release-cut PR on lint + test
# before squash-merge to release, so the tag fired here is expected to have
# already been CI-validated. verify-tag-on-release below enforces that
# invariant in the workflow itself (rather than relying solely on the
# out-of-band process of "nobody tags/dispatches from anywhere but main")
# out-of-band process of "nobody tags/dispatches from anywhere but release")
# before any build/publish work starts. The previous check-ci-passed JS
# poller was redundant with the ruleset and has been removed.
verify-tag-on-main:
verify-tag-on-release:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Assert release ref is main or an ancestor of main
- name: Assert release ref is release or an ancestor of release
run: |
git fetch origin main --quiet
git fetch origin release --quiet
target_sha="${{ github.sha }}"
if [[ "${{ github.ref }}" == refs/tags/v* ]]; then
echo "Tag push ${{ github.ref_name }}: verifying $target_sha is an ancestor of origin/main"
echo "Tag push ${{ github.ref_name }}: verifying $target_sha is an ancestor of origin/release"
else
echo "workflow_dispatch from ${{ github.ref }}: verifying $target_sha is origin/main or an ancestor of it"
echo "workflow_dispatch from ${{ github.ref }}: verifying $target_sha is origin/release or an ancestor of it"
fi
if ! git merge-base --is-ancestor "$target_sha" origin/main; then
echo "::error::Ref $target_sha (${{ github.ref }}) is not origin/main or an ancestor of it. Refusing to build/publish."
if ! git merge-base --is-ancestor "$target_sha" origin/release; then
echo "::error::Ref $target_sha (${{ github.ref }}) is not origin/release or an ancestor of it. Refusing to build/publish."
exit 1
fi
echo "OK: $target_sha is origin/main or an ancestor of it."
echo "OK: $target_sha is origin/release or an ancestor of it."

dependency-bounds:
needs: verify-tag-on-main
needs: verify-tag-on-release
runs-on: ubuntu-latest
outputs:
report_path: ${{ steps.gate.outputs.report_path }}
Expand Down Expand Up @@ -79,7 +79,7 @@ jobs:
retention-days: 90

build:
needs: [verify-tag-on-main, dependency-bounds]
needs: [verify-tag-on-release, dependency-bounds]
runs-on: ubuntu-latest
outputs:
version: ${{ steps.version.outputs.version }}
Expand Down Expand Up @@ -139,8 +139,8 @@ jobs:
# below). Test PyPI publishing via workflow_dispatch from any ref is
# the intended pre-release validation path and is lower risk by
# design (test.pypi.org is not the real distribution channel);
# verify-tag-on-main above still requires the dispatched ref be
# main or an ancestor of it, which is sufficient here.
# verify-tag-on-release above still requires the dispatched ref be
# release or an ancestor of it, which is sufficient here.
if: github.event.inputs.test_pypi == 'true'
uses: pypa/gh-action-pypi-publish@release/v1
with:
Expand All @@ -150,7 +150,7 @@ jobs:
verbose: true

- name: Publish to PyPI
# Defense in depth alongside verify-tag-on-main: even if that job is
# Defense in depth alongside verify-tag-on-release: even if that job is
# ever bypassed or miswired, this step alone refuses to publish to
# real PyPI unless the triggering ref is an actual v* tag. A
# workflow_dispatch from a branch (with the test_pypi input left at
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/results-explorer-browser.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,15 @@ name: Results Explorer browser tests

on:
push:
branches: [ main ]
branches: [ release ]
paths:
- 'results-explorer/**'
- '_project/scripts/explorer_pipeline/**'
- '_project/scripts/explorer_publish.py'
- 'results-data/**'
- '.github/workflows/results-explorer-browser.yml'
pull_request:
branches: [ main, develop ]
branches: [ release, develop ]
paths:
- 'results-explorer/**'
- '_project/scripts/explorer_pipeline/**'
Expand Down
24 changes: 12 additions & 12 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,10 @@ name: Tests
on:
push:
branches:
- main
- release
pull_request:
branches:
- main
- release

concurrency:
group: test-${{ github.ref }}
Expand Down Expand Up @@ -59,12 +59,12 @@ jobs:

compat-test:
name: compat-test (${{ matrix.os }}, ${{ matrix.python-version }})
if: ${{ (github.event_name != 'pull_request' || github.base_ref != 'develop') && (github.event_name != 'pull_request' || github.base_ref != 'main' || !startsWith(github.head_ref, 'v')) }}
if: ${{ (github.event_name != 'pull_request' || github.base_ref != 'develop') && (github.event_name != 'pull_request' || github.base_ref != 'release' || !startsWith(github.head_ref, 'v')) }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
# Additional compatibility coverage for main/push surfaces.
# Additional compatibility coverage for release/push surfaces.
# Develop PRs keep only the required Ubuntu 3.12 test job; broad
# validation is covered by nightly/manual workflows.
include:
Expand Down Expand Up @@ -170,10 +170,10 @@ jobs:
uvx pip-audit
fi

# Quick integration smoke tests - non-required on main/push; develop broad
# Quick integration smoke tests - non-required on release/push; develop broad
# validation runs through nightly/manual workflows.
integration-smoke:
if: ${{ (github.event_name != 'pull_request' || github.base_ref != 'develop') && (github.event_name != 'pull_request' || github.base_ref != 'main' || !startsWith(github.head_ref, 'v')) }}
if: ${{ (github.event_name != 'pull_request' || github.base_ref != 'develop') && (github.event_name != 'pull_request' || github.base_ref != 'release' || !startsWith(github.head_ref, 'v')) }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
Expand Down Expand Up @@ -205,7 +205,7 @@ jobs:
# These don't require external credentials and catch common issues
uv run -- python -m pytest tests/integration -m "platform_smoke or (integration and fast)" --tb=short

# Table-format dependency lane for main/push validation.
# Table-format dependency lane for release/push validation.
# Nightly/manual workflows cover this lane for develop after Step 3.
integration-table-formats:
if: ${{ github.event_name != 'pull_request' || github.base_ref != 'develop' }}
Expand All @@ -231,11 +231,11 @@ jobs:
run: |
uv run -- python -m pytest -q tests/integration/core/data_organization/test_presorted_generation.py -m "requires_table_formats" --tb=short

# Credential-free integration suite - release-required for main PRs and runs on main pushes.
# Credential-free integration suite - release-required for release PRs and runs on release pushes.
integration:
runs-on: ubuntu-latest
timeout-minutes: 20
if: github.event_name == 'push' || (github.event_name == 'pull_request' && github.base_ref == 'main')
if: github.event_name == 'push' || (github.event_name == 'pull_request' && github.base_ref == 'release')
steps:
- name: Checkout code
uses: actions/checkout@v4
Expand All @@ -260,7 +260,7 @@ jobs:
name: correctness-gate
runs-on: ubuntu-latest
timeout-minutes: 20
if: github.event_name == 'push' || (github.event_name == 'pull_request' && github.base_ref == 'main')
if: github.event_name == 'push' || (github.event_name == 'pull_request' && github.base_ref == 'release')
steps:
- name: Checkout code
uses: actions/checkout@v4
Expand Down Expand Up @@ -341,7 +341,7 @@ jobs:

release-readiness:
name: release-readiness
if: ${{ github.event_name == 'pull_request' && github.base_ref == 'main' }}
if: ${{ github.event_name == 'pull_request' && github.base_ref == 'release' }}
runs-on: ubuntu-latest
steps:
- name: Checkout code
Expand Down Expand Up @@ -410,7 +410,7 @@ jobs:
release-required-result:
name: release-required-result
needs: [test, integration, correctness-gate, test-package, release-readiness]
if: ${{ always() && github.event_name == 'pull_request' && github.base_ref == 'main' }}
if: ${{ always() && github.event_name == 'pull_request' && github.base_ref == 'release' }}
runs-on: ubuntu-latest
steps:
- name: Aggregate release-required result
Expand Down
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
name: Validate main PR base
name: Validate release PR base

on:
pull_request:
branches:
- main
- release

concurrency:
group: validate-main-pr-${{ github.ref }}
group: validate-release-pr-${{ github.ref }}
cancel-in-progress: true

permissions:
Expand All @@ -18,16 +18,16 @@ jobs:
name: validate-base
runs-on: ubuntu-latest
steps:
- name: Reject non-release-branch PRs to main
- name: Reject non-release-branch PRs to release
env:
HEAD_REF: ${{ github.head_ref }}
run: |
set -euo pipefail
if [[ "${HEAD_REF}" =~ ^v[0-9]+\.[0-9]+\.[0-9]+(-[A-Za-z0-9.+-]+)?$ ]]; then
echo "Release branch '${HEAD_REF}' approved for merge into main."
echo "Release branch '${HEAD_REF}' approved for merge into release."
exit 0
fi
echo "::error::Only release branches matching 'vX.Y.Z' (optionally with a '-suffix') may merge into main. Got '${HEAD_REF}'."
echo "::error::Only release branches matching 'vX.Y.Z' (optionally with a '-suffix') may merge into release. Got '${HEAD_REF}'."
echo "::error::Open this PR against 'develop' instead, or cut a release branch via 'make release-cut VERSION=X.Y.Z'."
exit 1

Expand Down Expand Up @@ -57,7 +57,16 @@ jobs:
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
RELEASE_CANARY_WORKFLOW: release-canary.yml
RELEASE_CANARY_BRANCH: main
# release-canary.yml runs scheduled from the repository's DEFAULT
# branch (develop, per the branch-rename migration this PR is part
# of) even though it checks out develop's content either way via
# RELEASE_CANARY_REF. A workflow run's `branch` field reflects
# where the run was TRIGGERED from, not RELEASE_CANARY_REF's
# checkout target -- so the branch= filter here must track the
# default branch, not the release branch, or release_readiness_check.py
# queries a branch that never runs the canary and every release PR
# fails with "No completed release canary runs found."
RELEASE_CANARY_BRANCH: develop
RELEASE_CANARY_CHECKED_REF: develop
RELEASE_CANARY_MAX_AGE_HOURS: "48"
RELEASE_READINESS_OVERRIDE_SHA: ${{ vars.RELEASE_READINESS_OVERRIDE_SHA }}
Expand Down
10 changes: 5 additions & 5 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ This document provides guidelines and instructions for contributing.

## Branches & PR gate

`develop` is the long-lived development branch; **all changes land via PR**. `main` is release-only (handled by the version-branch flow — see `docs/operations/release-guide.md`). PRs target `develop` and squash-merge with linear history.
`develop` is the long-lived development branch; **all changes land via PR**. `release` is release-only (handled by the version-branch flow — see `docs/operations/release-guide.md`). PRs target `develop` and squash-merge with linear history.

Required CI on `develop` reports through `ci-required-result`. The umbrella uses `.github/path-filters.yml` to classify each PR: content-only PRs run content validation and skip Python fast tests, while code, infra, workflow, tooling, and unknown paths run the post-Step-3 lint/type + Ubuntu 3.12 fast-test gate. Reviews are not required for solo-dev work; auto-merge handles landing.

Expand Down Expand Up @@ -88,7 +88,7 @@ The canonical loop is **branch → edit → preflight → `make pr-open`**. Auto
make pr-open # push + gh pr create --base develop + gh pr merge --auto --squash
```

`make pr-open` refuses to run from `develop` or `main`. The PR will squash-merge the moment the required checks turn green. Don't poll for CI — auto-merge handles it.
`make pr-open` refuses to run from `develop` or `release`. The PR will squash-merge the moment the required checks turn green. Don't poll for CI — auto-merge handles it.

5. **After merge**, the remote branch auto-deletes (repo setting `delete_branch_on_merge`). Sweep any stale local branches and worktrees with:

Expand Down Expand Up @@ -218,10 +218,10 @@ are cut by maintainers via the version-branch flow documented in

1. `make bump VERSION=X.Y.Z` and `make changelog-draft VERSION=X.Y.Z` on `develop`.
2. `make release-prepare VERSION=X.Y.Z` cuts `vX.Y.Z` from `develop` with a
release-curated tree and opens a PR against `main`.
3. Squash-merge the PR; tag `main`; `release.yml` publishes to PyPI.
release-curated tree and opens a PR against `release`.
3. Squash-merge the PR; tag `release`; `release.yml` publishes to PyPI.
4. `make release-rebase-develop VERSION=X.Y.Z` rebases `develop` onto the
release-shaped `main`.
release-shaped `release`.

We follow semantic versioning.

Expand Down
Loading
Loading