Skip to content

Verify theme delivery in CI and define release policy (#25) - #29

Merged
sehkone merged 1 commit into
mainfrom
sehkone/issue-25
Aug 7, 2026
Merged

Verify theme delivery in CI and define release policy (#25)#29
sehkone merged 1 commit into
mainfrom
sehkone/issue-25

Conversation

@sehkone

@sehkone sehkone commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Summary

Closes the two remaining delivery/release-safety gaps around the vendored theme and documents the release policy.

  • Assert theme delivery in CI. A new Verify theme delivery in built sites step in the build-html job runs scripts/verify-theme-delivery.py against each sample's built site-ci/<name> directory. It reads extra_css and theme.logo from the resolved config (so it tracks mkdocs-base.yml), asserts every extra_css entry, the logo, and the six shipped woff2 faces exist in the output, and asserts the deliberately excluded paths (theme/pdf/, theme/mkdocs-base.yml, theme/build-docs-pdf.sh, theme.toml) are absent. The check runs against the real built site/, not a re-run --strict build.
  • Guard the release surface. A new Guard release surface step in release.yml runs scripts/check-release-surface.sh, which byte-diffs the release surface (everything fetch-theme.sh installs plus fetch-theme.sh itself) between the new tag and its MAJOR.MINOR.PATCH-ordered predecessor and fails with an explanatory message when identical. It passes without comparing when no predecessor resolves. The checkout gains fetch-depth: 0 and fetch-tags: true so the previous tag's tree is readable. This is an added gate; the release-notes extraction and gh release create step are untouched.
  • Document the versioning policy. README.md gains a ## Versioning section defining the no-v-prefix MAJOR.MINOR.PATCH tags, the MAJOR/MINOR/PATCH meanings, and that templates are not versioned separately; CHANGELOG.md links to it.
  • Re-scope the CHANGELOG by area. Unreleased entries are regrouped under ### manual, ### api-reference, ### shared, and ### scripts with all content preserved, and the intro note is updated. Release-notes extraction keys off ## <version> headings only, so this is unaffected.
  • Firm up the consumer procedure. README.md now states explicitly that a consumer commits docs/theme/ (vendored, not git-ignored) and verifies the installed tree against docs/theme/.meta.
  • Sample-site provenance. docs.yml stamps the built commit (short SHA + link, build date) onto the landing page footer, so the Pages deploy is not mistaken for installable released state.

Test plan

  • Install the theme into the samples with scripts/install-samples.sh, build each sample, and confirm the delivery-assertion step passes against a clean build.
  • CI fails when any resolved extra_css entry is missing from a sample's built site output.
  • CI fails when the theme.logo file is missing from a sample's built site output.
  • CI fails when a shipped woff2 face is missing (e.g. deleting a file under shared/fonts/).
  • CI fails when any excluded path (theme/pdf/, theme/mkdocs-base.yml, theme/build-docs-pdf.sh, theme.toml) appears in a built site.
  • The delivery assertions run against the built site/ directory, not a re-run --strict build.
  • Pushing a tag whose release surface is byte-identical to the previous tag's is rejected with the explanatory message; a tag whose surface differs proceeds to create the release.
  • Pushing a tag with no resolvable predecessor proceeds (guard passes rather than crashing or rejecting).
  • README.md documents the MAJOR.MINOR.PATCH (no v prefix) policy with the MAJOR/MINOR/PATCH definitions and states templates are not versioned separately.
  • README.md states a consumer commits docs/theme/ and verifies its digest against docs/theme/.meta.
  • CHANGELOG.md groups entries under ### manual, ### api-reference, ### shared, and ### scripts, with the previous Unreleased content preserved.
  • The published sample site shows the revision it was built from (footer provenance).

Closes #25

CI built the samples with --strict but never asserted the theme's assets
reached the output, so a site linking to CSS or woff2 that were never
emitted could still pass green. Releasing had neither a guard nor a
stated policy: a tag whose installed surface matched the previous tag's
still cut an empty-diff bump in every consumer, MAJOR/MINOR/PATCH had no
documented meaning, and the published sample site showed unreleased
state with no provenance.

- Add scripts/verify-theme-delivery.py and run it in ci.yml after each
  sample builds. It reads extra_css and theme.logo from the resolved
  config and asserts, against the built site/ directory, that every
  stylesheet, the logo, and the six shipped woff2 faces landed and that
  the deliberately excluded paths did not.
- Add scripts/check-release-surface.sh and gate release.yml on it. It
  resolves the previous tag by MAJOR.MINOR.PATCH order and fails when the
  release surface is byte-identical, passing when no predecessor exists.
  The release checkout now fetches full history and tags so the previous
  tree resolves.
- Document the versioning policy in README and reference it from
  CHANGELOG; re-scope the CHANGELOG Unreleased section by consumer area
  (manual, api-reference, shared, scripts).
- State in README that a consumer commits docs/theme/ and verifies its
  digest against docs/theme/.meta.
- Stamp the built revision onto the sample landing page in docs.yml so
  the published preview is not mistaken for a released state.

Closes #25
@sehkone

sehkone commented Aug 7, 2026

Copy link
Copy Markdown
Contributor Author

[Reviewer Round 1]

Solid work — this closes all six scopes and the design choices match the issue's intent closely. A few observations, mostly minor; nothing I'd consider blocking.

What holds up well

  • Delivery verification is meaningful, not decorative. scripts/verify-theme-delivery.py reads extra_css/theme.logo from the resolved config via load_config, so it tracks mkdocs-base.yml as required, and it runs against the real site-ci/<name> tree rather than a re-run --strict build. The excluded-path assertions genuinely test exclude_docs: install-samples.sh installs the full theme tree (theme/pdf/, theme/mkdocs-base.yml, theme/build-docs-pdf.sh), so their absence in site/ is a real signal that the exclusion works, not a vacuous pass. Deriving the fonts dir from base.css's position in extra_css (dirname(dirname(base.css))/fontstheme/fonts) correctly matches how fetch-theme.sh stages styles/ and fonts/ as siblings.
  • The release-surface guard's edge handling is careful. The predecessor resolution (sort -V + grep -B1 -Fx) correctly yields "no predecessor" for the first/lowest tag and passes without comparing, satisfying the acceptance criterion. || true on both the git tag and the grep -B1 | head pipelines is the right defensive move under set -o pipefail (SIGPIPE from head). SURFACE_PATHS matches the issue's surface definition exactly, and comparing templates whole covers all four templates by byte-diff as intended. The fetch-depth: 0 + fetch-tags: true addition with its explanatory comment is exactly what's needed to avoid silently diffing against nothing.

Minor points

  1. docs.yml provenance sed fails open (docs.yml:47-51). The substitution is anchored on the exact string <span id="build-revision">Preview build from the local working tree</span>. If that markup in samples/index.html ever drifts, sed silently no-ops and the deployed site ships the literal "Preview build from the local working tree" text — the very "misleading unreleased state" scope 6 exists to prevent — with nothing failing. Since the whole point is to not mislead an evaluator, consider making it self-checking (e.g. assert the built site/index.html no longer contains the placeholder, or that it now contains the short SHA) so a drifted anchor breaks the deploy loudly instead of quietly.

  2. Excluded-path check only inspects the site root (verify-theme-delivery.py:112-118). os.path.exists(site/theme/pdf) etc. looks at the top level only. This is correct today because exclude_docs applies globally and mkdocs-static-i18n's ko subtree inherits it, so nothing excluded lands under site/ko/. Just flagging that the assertion is root-only, so it wouldn't catch a future config/plugin change that re-emitted machinery under a language subdir. Optional; not worth complicating now.

  3. git diff --quiet conflates "differs" with "errored" (check-release-surface.sh:60). A git failure (exit 128, e.g. an unreadable ref) is indistinguishable from "surface differs" and would let the release proceed. Benign in practice — the predecessor comes from git tag --list and the tree is present after the deeper checkout, and erring toward proceeding is the safe direction for a release gate — but noting it for completeness.

Hygiene

Closes #25 with all scopes addressed (no partial ## Not addressed needed) and a complete ## Test plan — correct. CHANGELOG re-scoping preserves the Unreleased content (including the VERSION-removal note, now under ### scripts) and updates the intro note as required. README versioning/commit/digest statements are all present and consistent.

Nice PR.

@sehkone

sehkone commented Aug 7, 2026

Copy link
Copy Markdown
Contributor Author

[Review Verdict Round 1: APPROVED]

@sehkone
sehkone merged commit c8b7f55 into main Aug 7, 2026
6 checks passed
@sehkone
sehkone deleted the sehkone/issue-25 branch August 7, 2026 22:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Verify theme delivery in CI and define the release policy

1 participant