Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,14 @@ Closes #
- [ ] `tests/configs/assets/<name>.json` added
- [ ] Asset appears in [ASSETS.md](../ASSETS.md) catalog

## Release (if this PR cuts a release)

<!-- Leave unchecked / delete this section if the PR does not bump the version. See RELEASING.md. -->

- [ ] `CHANGELOG.md` finalized: `[Unreleased]` renamed to `[X.Y.Z] - <date>`, fresh empty `[Unreleased]` added above
- [ ] Both version markers bumped to `X.Y.Z` (`pyproject.toml` and `template/{{.project_name}}/bundle_init_config.json.tmpl`)
- [ ] Version guard test passes (`tests/test_release_metadata.py`)

## Checklist

- [ ] Go template syntax is valid (no unclosed `{{ }}` blocks)
Expand Down
9 changes: 8 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,14 +17,21 @@ When implementing features, fixes, or changes, follow this workflow:
- `docs/<name>` for documentation-only changes
2. **Implement**: Make changes, add/update tests. Run `pytest tests/ -V` and ensure all tests pass.
3. **Update metadata** (as applicable):
- `CHANGELOG.md` — add entry under `[Unreleased]` following [Keep a Changelog](https://keepachangelog.com/) format
- `CHANGELOG.md` — add entry under `[Unreleased]` following [Keep a Changelog](https://keepachangelog.com/) format; entries stay under `[Unreleased]` until a release finalizes them (the release renames the heading to `[X.Y.Z] - <date>` and adds a fresh `[Unreleased]`; see [RELEASING.md](RELEASING.md))
- `ROADMAP.md` — update feature status if the change relates to a tracked item
- `DEVELOPMENT.md` — add/update design decisions if architectural choices were made
- `ARCHITECTURE.md` — update if the change affects project structure or architecture
- `tests/configs/` — add new test configurations if new configuration axes are introduced
4. **Propose commit message** — output a suggested commit message (imperative mood, free-form, no conventional commit prefixes). Do **not** stage or commit; the maintainer reviews and commits manually.
5. **PR** (on request) — create a pull request via `gh pr create` following `.github/PULL_REQUEST_TEMPLATE.md`.

### Release Process

Cutting a release follows the dedicated checklist in [RELEASING.md](RELEASING.md). When asked to prepare or cut a release, follow that document exactly rather than reconstructing the steps. Two points it enforces that are easy to miss:

- The version lives in two markers that must agree: `pyproject.toml` (`version`) and `template/{{.project_name}}/bundle_init_config.json.tmpl` (`_template_version`). Bump both in the same PR, before merge.
- Finalize the changelog in that PR too (rename `[Unreleased]` to `[X.Y.Z] - <date>`, add a fresh `[Unreleased]`), so `main` is release-ready at merge. The annotated tag and the GitHub release come after merge. A guard test (`tests/test_release_metadata.py`) fails CI if the markers and the changelog drift.

See [CONTRIBUTING.md](CONTRIBUTING.md) for full contributor guidelines.

## Commands
Expand Down
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/).

## [Unreleased]

### Added
- **`RELEASING.md`**: canonical release checklist. Documents the two version markers (`pyproject.toml` and the generated bundle's `_template_version`), the changelog finalization step (rename `[Unreleased]` to `[X.Y.Z] - <date>` in the PR before merge), the annotated-tag and GitHub-release steps after merge, and a fill-in release-notes template.
- **`tests/test_release_metadata.py`**: version guard test. Asserts the two version markers and the latest finalized `CHANGELOG.md` version all agree, and that an `[Unreleased]` section is always present. Runs in CI on every pull request, so a partial or forgotten version bump fails before merge.

### Changed
- **`.github/PULL_REQUEST_TEMPLATE.md`**: added an optional "Release" block (changelog finalized, both version markers bumped, guard test passes) so release facts surface at review time instead of being buried in free-text.
- **`CONTRIBUTING.md` and `AGENTS.md`**: clarified that changelog entries stay under `[Unreleased]` until a release finalizes them, and added pointers to `RELEASING.md`. `AGENTS.md` gained a Release Process subsection.

## [1.9.0] - 2026-06-06

### Added
Expand Down
10 changes: 7 additions & 3 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,10 @@ All contributions must:
- If your change affects generated output, include a sample of the before/after
- Update documentation if behavior changes

## Releasing

Cutting a release (bumping the version, finalizing the changelog, tagging, and publishing a GitHub release) follows a dedicated checklist in [RELEASING.md](RELEASING.md). The short version: the version bump and the `[Unreleased]` to `[X.Y.Z] - <date>` rename happen inside the PR before merge, so `main` is always release-ready; the annotated tag and the GitHub release happen after merge. A guard test (`tests/test_release_metadata.py`) keeps the two version markers and the changelog in sync.

## Adding an Asset

The repository also ships an **asset library**: standalone sub-templates under `assets/<asset-name>/` that install individual Databricks artifacts (pipelines, jobs, dashboards) via `databricks bundle init --template-dir`. See [ASSETS.md](ASSETS.md) for the end-user catalog and install pattern, and [ARCHITECTURE.md §8](ARCHITECTURE.md#8-asset-library--plugins-layer) for the design rationale.
Expand All @@ -167,7 +171,7 @@ Every asset must follow these seven rules:
--template-dir assets/<asset-name>
```
6. **Tests are per-asset.** Framework-level smoke checks (`tests/assets/test_framework.py`) run automatically for every `assets/*/`. Asset-specific deep tests live at `tests/assets/test_<asset_name>.py` and are added only when the asset has non-trivial structure to verify (exact file lists, YAML validity, Python parses, etc.). Both reuse the `install_asset` fixture from `tests/assets/conftest.py`.
7. **Single repo-wide CHANGELOG.** No per-asset versioning for now. Changes are grouped under `[Unreleased]` → `### Added / ### Changed`.
7. **Single repo-wide CHANGELOG.** No per-asset versioning for now. Changes are grouped under `[Unreleased]` → `### Added / ### Changed`. At release time the `[Unreleased]` heading is renamed to `[X.Y.Z] - <date>` and a fresh empty `[Unreleased]` is added above it; see [RELEASING.md](RELEASING.md).

### Authoring walkthrough

Expand Down Expand Up @@ -200,7 +204,7 @@ Every asset must follow these seven rules:

4. Add a row to the catalog table in [ASSETS.md](ASSETS.md).

5. Add an `[Unreleased]` entry in [CHANGELOG.md](CHANGELOG.md) under `### Added`.
5. Add an entry under `## [Unreleased]` in [CHANGELOG.md](CHANGELOG.md) under `### Added`. The entry stays under `[Unreleased]` until a release finalizes it (see [RELEASING.md](RELEASING.md)); do not invent a version heading in your PR unless the PR itself cuts the release.

6. Verify locally:

Expand All @@ -222,7 +226,7 @@ When opening a PR that introduces or modifies an asset:
- [ ] Framework smoke tests pass (`pytest tests/assets/test_framework.py -v`)
- [ ] Asset-specific tests pass (if added)
- [ ] `ASSETS.md` catalog is updated
- [ ] `CHANGELOG.md` has an `[Unreleased]` entry
- [ ] `CHANGELOG.md` has an entry under `[Unreleased]` (or, if this PR cuts the release, a finalized `[X.Y.Z] - <date>` entry per [RELEASING.md](RELEASING.md))
- [ ] No changes outside `assets/<asset-name>/`, `tests/assets/`, `tests/configs/assets/`, and the four cross-reference docs (`ASSETS.md`, `CHANGELOG.md`, `README.md`, and `ROADMAP.md` if status changes)

## Reporting Issues
Expand Down
133 changes: 133 additions & 0 deletions RELEASING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
# Releasing

This is the canonical checklist for cutting a release of `databricks-bundle-template`. It exists so the steps are the same every time and nothing is rediscovered from memory. If you are an AI assistant helping with a release, follow this document exactly.

The guiding rule: **`main` is always release-ready the moment a PR merges.** The version bump and the changelog finalization happen inside the PR, before merge, so the commit that gets tagged is exactly what was reviewed.

## Versioning

This project follows [Semantic Versioning](https://semver.org/): `MAJOR.MINOR.PATCH`.

- **MINOR** (`1.8.0` to `1.9.0`): a new asset, a new template feature, or a new configuration axis. This is the common case.
- **PATCH** (`1.7.0` to `1.7.1`): a fix or a small refinement to an existing asset or feature, no new capability.
- **MAJOR**: a breaking change to the generated output or the install contract. Rare; discuss first.

### The two version markers

The version lives in two places and they must always agree:

1. `pyproject.toml`: the `version = "X.Y.Z"` field.
2. `template/{{.project_name}}/bundle_init_config.json.tmpl`: the `"_template_version": "X.Y.Z"` field, which stamps provenance into every generated bundle.

A guard test (`tests/test_release_metadata.py`) asserts these two markers and the most recent finalized `CHANGELOG.md` version all match, and it runs in CI on every PR. If you bump one and forget the other, or finalize the changelog without bumping the markers, the test fails before merge.

## Phase 1: in the PR (before merge)

Do all of this on the feature branch, in the PR that ships the change. By the time the PR merges, `main` is release-ready.

1. **Implement and test.** Make the change, add or update tests, and confirm the full suite is green:

```bash
pytest tests/ -V
```

2. **Finalize the changelog.** In `CHANGELOG.md`, rename the `## [Unreleased]` heading to the new version with today's date, then add a fresh empty `## [Unreleased]` above it:

```markdown
## [Unreleased]

## [X.Y.Z] - YYYY-MM-DD

### Added
- ...
```

Use the real release date (the day you cut the release), not the day you started the work.

3. **Bump both version markers** to `X.Y.Z`: `pyproject.toml` and `template/{{.project_name}}/bundle_init_config.json.tmpl` (see "The two version markers" above).

4. **Update the catalog and roadmap as applicable.** `ASSETS.md` (new or changed asset row), `ROADMAP.md` (move the item to "Shipped" or update its status).

5. **Open the PR.** Fill out the PR template, including the optional release block confirming steps 2 and 3 are done. Run the suite once more so the version guard test passes in CI.

6. **Merge** once review and CI are green.

## Phase 2: tag and release (after merge)

Do this on a synced `main`.

1. **Sync and verify.**

```bash
git checkout main && git pull
```

Confirm `pyproject.toml`, `_template_version`, and the top `CHANGELOG.md` entry all read `X.Y.Z` and the changelog date is correct. (The guard test already enforces this, but eyeball it.)

2. **Create the annotated tag** on the merge commit. The tag name is `vX.Y.Z` and the message follows the established format `vX.Y.Z - <asset-or-area>: <Short Title>`:

```bash
git tag -a vX.Y.Z -m "vX.Y.Z - <asset-or-area>: <Short Title>"
git push origin vX.Y.Z
```

Use an annotated tag (`-a`), not a lightweight tag; every prior release tag is annotated.

3. **Create the GitHub release** from the tag, with a notes file written from the template in the appendix below:

```bash
gh release create vX.Y.Z --verify-tag \
--title "vX.Y.Z - <asset-or-area>: <Short Title>" \
--notes-file <path-to-filled-notes>
```

Not a draft, not a prerelease. Pin every link in the notes to the `vX.Y.Z` ref so they keep working as the repo evolves.

4. **Verify.**

```bash
gh release view vX.Y.Z
```

### If the change already merged without finalizing

If a PR merged with entries still under `## [Unreleased]` and the version markers not bumped, do not tag yet. Open a small dedicated "Release vX.Y.Z" PR that does Phase 1 steps 2 and 3 (finalize the changelog, bump both markers), merge it, then run Phase 2. Keep the tag pointing at a commit where the markers and changelog already read `X.Y.Z`.

## Release notes template

Fill this in for the `gh release create --notes-file`. It mirrors the structure of the v1.7.0 and v1.8.0 releases. Drop any section that does not apply (for a patch release, "Highlights" and "What's new" may collapse into one short section).

```markdown
## Highlights

<!-- One or two paragraphs: what this release is and why it matters. Lead with the headline change. -->

## What's new

### <Asset or feature name>

<!-- The install command (for an asset), the prompts, what ships, the key behavior. -->

### Documentation and metadata

<!-- Catalog/roadmap updates, version bump note, any doc changes. -->

## Upgrade

<!-- What existing users need to do. Usually "No action required for existing installs." plus the install command for a new asset. -->

## Breaking changes

<!-- "None." in the common case. -->

## Pointers

- [Full changelog](https://github.com/vmariiechko/databricks-bundle-template/blob/vX.Y.Z/CHANGELOG.md#xyz---yyyy-mm-dd)
- [Asset README](https://github.com/vmariiechko/databricks-bundle-template/blob/vX.Y.Z/assets/<name>/README.md)
- [Asset catalog](https://github.com/vmariiechko/databricks-bundle-template/blob/vX.Y.Z/ASSETS.md)
```

## See also

- [CONTRIBUTING.md](CONTRIBUTING.md): how to make changes and the asset framework rules.
- [CHANGELOG.md](CHANGELOG.md): the running history this release pipeline finalizes.
85 changes: 85 additions & 0 deletions tests/test_release_metadata.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
"""Release metadata consistency guards.

These tests enforce the release invariant documented in RELEASING.md: the two
version markers (`pyproject.toml` and the generated bundle's `_template_version`)
and the most recent finalized `CHANGELOG.md` version must always agree. They run
in CI on every pull request, so a forgotten or partial version bump fails before
merge rather than surfacing during the release itself.
"""

import re
import tomllib
from pathlib import Path

REPO_ROOT = Path(__file__).parent.parent
PYPROJECT = REPO_ROOT / "pyproject.toml"
TEMPLATE_VERSION_FILE = (
REPO_ROOT / "template" / "{{.project_name}}" / "bundle_init_config.json.tmpl"
)
CHANGELOG = REPO_ROOT / "CHANGELOG.md"

_SEMVER = re.compile(r"^\d+\.\d+\.\d+$")


def _pyproject_version() -> str:
"""Return the `version` field from `pyproject.toml`."""
data = tomllib.loads(PYPROJECT.read_text(encoding="utf-8"))
return data["project"]["version"]


def _template_version() -> str:
"""Return the `_template_version` stamped into generated bundles."""
text = TEMPLATE_VERSION_FILE.read_text(encoding="utf-8")
match = re.search(r'"_template_version":\s*"([^"]+)"', text)
assert match is not None, "_template_version marker not found"
return match.group(1)


def _latest_changelog_version() -> str:
"""Return the most recent finalized version heading in the changelog.

Skips the `[Unreleased]` heading and returns the first `## [X.Y.Z] - DATE`
entry below it.
"""
text = CHANGELOG.read_text(encoding="utf-8")
match = re.search(r"^## \[(\d+\.\d+\.\d+)\]", text, re.MULTILINE)
assert match is not None, "no finalized version heading found in CHANGELOG.md"
return match.group(1)


def test_version_markers_are_semver():
"""Both version markers are well-formed semantic versions."""
assert _SEMVER.match(_pyproject_version())
assert _SEMVER.match(_template_version())


def test_version_markers_agree():
"""The two version markers never drift apart."""
assert _pyproject_version() == _template_version(), (
"Version drift: pyproject.toml "
f"({_pyproject_version()}) != _template_version "
f"({_template_version()}). Bump both (see RELEASING.md)."
)


def test_version_matches_latest_changelog():
"""The version markers match the latest finalized changelog entry.

This catches the case where the changelog was finalized to a new version but
the markers were not bumped (or the reverse), and the case where neither was
touched for a release.
"""
assert _pyproject_version() == _latest_changelog_version(), (
f"Version markers ({_pyproject_version()}) do not match the latest "
f"CHANGELOG.md entry ({_latest_changelog_version()}). Finalize the "
"changelog and bump both markers together (see RELEASING.md)."
)


def test_unreleased_section_present():
"""The changelog keeps an `[Unreleased]` section for in-flight work."""
text = CHANGELOG.read_text(encoding="utf-8")
assert re.search(r"^## \[Unreleased\]", text, re.MULTILINE), (
"CHANGELOG.md must keep a `## [Unreleased]` section at the top "
"(re-add it when finalizing a release; see RELEASING.md)."
)
Loading