From c27d4c46b5533099ca444dc4588221e3348f9671 Mon Sep 17 00:00:00 2001 From: wshallwshall Date: Wed, 29 Jul 2026 11:08:45 -0500 Subject: [PATCH] feat(release): a publish path for the console, and a guard for the key inventory MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two ASVS threads, both about a claim nothing was checking. ## ASVS 15.2.4 — the console has no publish path, so its PyPI name stays free release.yml had zero mentions of the web console: jobs for the engine and the harness, none for messagefoundry-webconsole. So the name could never be claimed by CI, and PyPI confirms it is UNCLAIMED today while messagefoundry and messagefoundry-harness are both taken. A pending Trusted Publisher is now registered against this workflow, which grants permission to publish but -- PyPI's own words -- "does not reserve that name". release-webconsole is a close sibling of release-harness with one deliberate difference: it is NOT lockstep. The console is separately versioned (0.2.15 vs the engine's 0.3.2, its own __version__ root, changelog and cadence per docs/WEBCONSOLE-PACKAGE.md), so it fires on its own `webconsole-v*` tag and carries no `needs: release`. An engine release must not drag it along and a console release must not wait on one. It lives in release.yml rather than its own file because the Trusted Publisher is registered against `release.yml`; a separate workflow would not match it. The engine job is now guarded with `!startsWith(github.ref_name, 'webconsole-')`. Without that a console tag would have run the engine release AT THE CONSOLE'S VERSION -- and neither job's version check could catch it, since each compares its own wheel against the same tag. Publishing stays gated on PUBLISH_WEBCONSOLE, like the harness: the build and version-check run on every console tag so the path is exercised before it is armed. Flipping the variable is what actually creates the project and claims the name. ## Two existing release guards were brittle, and this proved it Adding the job reds two tests that had nothing to do with it. Both were pinning an incidental shape rather than the property, so both are de-hardcoded rather than renumbered: - test_release_pypi_publish_is_last_step: isolated "the release job" by splitting on the literal string "\n release-harness:". Inserting a job between the two silently widened that slice, so the assertion measured the wrong job's last step. The boundary is now derived from the next top-level job. - test_both_wheel_smokes_compare_versions: asserted exactly 2 PEP 440 comparisons. That number needs bumping every time a wheel job is added, which is how it gets bumped without thought. It is now derived from the count of jobs that actually build a wheel, so a NEW wheel job carrying a string compare fails there instead of lowering the bar. ## ASVS 11.1.2 — the usage-scope clauses were scored Pass and nothing pinned them `grep -rn 'Usage scope' tests/` returned nothing. The cell was scored Pass on the **Usage scope:** clauses in ASVS-L2-PHASE0-CHANGES.md section 4, and a new key-material row with no clause -- or a deleted clause -- would have shipped green. Writing the guard found a live gap: the Audit chain row keys its HMAC on an HKDF-derived subkey of the store DEK (and a named Transit audit key under vault_transit) and documented no usage scope, despite being exactly 11.1.2's case -- a key derived from a confidentiality key but used for a different property, where the separation is the thing worth recording. Clause added. Classification is explicit rather than a heuristic. A "does this look like key material" regex mis-sorted 2 of 20 rows on first contact, and a guard with a non-zero false-positive rate gets suppressed. So every row is classified and an UNCLASSIFIED row fails -- a new row cannot be added without someone deciding which side it is on. Same shape as CRITICAL_SECRETS in the rotation inventory. It also caught two of my own exclusion reasons being too thin to be reasons. Three mutations red: delete the audit-chain clause, delete the store-DEK clause, add an unclassified row. Unlike the threat-model guard this document is TRACKED, so this one runs where CI runs. --- .github/workflows/release.yml | 97 ++++++++++++++- docs/ASVS-L2-PHASE0-CHANGES.md | 2 +- tests/test_key_usage_scope_inventory.py | 151 ++++++++++++++++++++++++ tests/test_release_pipeline.py | 99 +++++++++++++++- 4 files changed, 343 insertions(+), 6 deletions(-) create mode 100644 tests/test_key_usage_scope_inventory.py diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 0d03908b..80fe4091 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -32,6 +32,12 @@ on: tags: - "v[0-9]+.[0-9]+.[0-9]+" - "v[0-9]+.[0-9]+.[0-9]+-*" # pre-releases: v0.1.0-rc1, … + # The web console is SEPARATELY VERSIONED (its own __version__, changelog and PyPI cadence — + # docs/WEBCONSOLE-PACKAGE.md), so it gets its own tag namespace rather than riding an engine tag. + # It lives in THIS workflow rather than its own file because the PyPI Trusted Publisher for + # messagefoundry-webconsole is registered against `release.yml`; a separate file would not match it. + - "webconsole-v[0-9]+.[0-9]+.[0-9]+" + - "webconsole-v[0-9]+.[0-9]+.[0-9]+-*" workflow_dispatch: # Deny-all at the workflow level; each job below requests only the scopes it uses (least privilege). @@ -55,7 +61,9 @@ jobs: # was already flipped to `==` at the cutover; release.yml was missed. # # `==` is also the safe form now: a fork, or the retired private vault, still cannot release. - if: github.repository == 'MEFORORG/MessageFoundry' + # NOT on a console tag: the console has its own job below and its own version root, so an + # engine release built from `webconsole-v*` would ship the engine at the console's version. + if: github.repository == 'MEFORORG/MessageFoundry' && !startsWith(github.ref_name, 'webconsole-') runs-on: ubuntu-latest permissions: contents: write # create the release + upload its assets @@ -309,6 +317,93 @@ jobs: skip-existing: true attestations: true + release-webconsole: + # Build + publish the SEPARATE `messagefoundry-webconsole` distribution (the browser ops console, + # packaging/messagefoundry-webconsole/). Unlike release-harness this is NOT lockstep with the engine: + # the console has its own __version__ root, changelog and PyPI cadence (docs/WEBCONSOLE-PACKAGE.md), + # so it fires on its OWN `webconsole-v*` tag and has NO `needs: release` — an engine release must not + # drag the console along, and a console release must not wait on one. + # + # ASVS 15.2.4 (dependency confusion). `messagefoundry-webconsole` is registered on PyPI as a PENDING + # Trusted Publisher against this workflow, which grants permission to publish but does NOT reserve the + # name: until a distribution is actually uploaded the name is claimable by anyone, and our own docs + # reference it. The FIRST successful run of this job creates the project and closes that exposure. + # Until then docs must point at the source-tree install (pinned by tests/test_install_instruction_provenance.py). + if: github.repository == 'MEFORORG/MessageFoundry' && startsWith(github.ref_name, 'webconsole-') + runs-on: ubuntu-latest + permissions: + contents: write # gh release upload (attach the console wheel) + id-token: write # PyPI Trusted Publishing (OIDC) — no API token anywhere + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + + - name: Set up Python + uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 + with: + python-version: "3.14" + + - name: Build the console wheel (wheel-only — the package tree is force-included from the repo root) + run: | + python -m pip install --upgrade pip build + # Wheel-only for the same reason as the harness: messagefoundry_webconsole/ lives OUTSIDE this + # project dir and is pulled in via force-include, so an sdist would not be self-contained. + python -m build --wheel ./packaging/messagefoundry-webconsole --outdir webconsole-dist + ls -l webconsole-dist/ + + - name: Smoke-check the console wheel (version == the console's OWN __version__ == tag) + run: | + built=$(python -c "import glob,re; print(re.search(r'messagefoundry_webconsole-([^-]+)-', glob.glob('webconsole-dist/*.whl')[0]).group(1))") + echo "console wheel version: $built" + # Compared against the CONSOLE's version root, never the engine's — that is the whole point of + # the separate tag namespace. PEP 440 comparison because hatchling normalises the filename + # (tag webconsole-v0.3.0-rc1 -> 0.3.0rc1), so a string compare could never match a pre-release. + if [ "${GITHUB_REF_TYPE:-}" = "tag" ]; then + want="${GITHUB_REF_NAME#webconsole-v}" + python -m pip install --quiet packaging + python - "$built" "$want" <<'PYVER' + import sys + from packaging.version import InvalidVersion, Version + + built, want = sys.argv[1], sys.argv[2] + try: + b, w = Version(built), Version(want) + except InvalidVersion as exc: + raise SystemExit(f"::error::unparseable version ({exc}) — built={built!r} tag={want!r}") + if b != w: + raise SystemExit( + f"::error::console wheel version {built} != tag {want} (normalised {b} != {w})" + ) + print(f"console version matches tag: {b}") + PYVER + fi + + - name: Attach the console wheel to the GitHub release + if: startsWith(github.ref, 'refs/tags/') + env: + GH_TOKEN: ${{ github.token }} + run: gh release upload "${GITHUB_REF_NAME}" webconsole-dist/*.whl --clobber + + - name: Upload console artifact (dry-run / always) + if: always() + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7 + with: + name: webconsole-artifacts + path: webconsole-dist/ + if-no-files-found: ignore + + # GATED like the harness: no-op until the owner sets repo variable PUBLISH_WEBCONSOLE=true. The + # build + version-check above still run on every console tag, so the publish path is exercised + # before it is armed. Flipping the variable is what actually claims the PyPI name. + - name: Publish messagefoundry-webconsole to PyPI (Trusted Publishing / OIDC) + if: ${{ startsWith(github.ref, 'refs/tags/') && vars.PUBLISH_WEBCONSOLE == 'true' }} + uses: pypa/gh-action-pypi-publish@ba38be9e461d3875417946c167d0b5f3d385a247 # v1.14.1 + with: + packages-dir: webconsole-dist/ + skip-existing: true + attestations: true + release-harness: # Build + publish the SEPARATE `messagefoundry-harness` distribution (the synthetic test/load/failover # harness, packaging/messagefoundry-harness/) in lockstep with the engine — same version, same tag. diff --git a/docs/ASVS-L2-PHASE0-CHANGES.md b/docs/ASVS-L2-PHASE0-CHANGES.md index 43b78ddc..1ddc19c7 100644 --- a/docs/ASVS-L2-PHASE0-CHANGES.md +++ b/docs/ASVS-L2-PHASE0-CHANGES.md @@ -94,7 +94,7 @@ Update it whenever a crypto dependency, algorithm, or key source changes. | Store-at-rest cipher — in-process (`[store].cipher_provider = aesgcm`, default) | AES-256-GCM (`mfenc:v1::…`, additive AAD-bound `mfenc:v2`) in-process keyring cipher. **The shipped default is key-required, NOT keyless:** every built-in environment (`dev`/`staging`/`prod`) derives `data_class=phi` ([ADR 0148](../adr/0148-phi-default-posture-and-an-explicit-security-enforcement-level.md) GIVEN 1), and a PHI instance **refuses to start keyless** in *any* environment (the `serve` gate in [`__main__.py`](../../messagefoundry/__main__.py); `[store].require_encryption` forces the refusal even for a synthetic instance). The identity/plaintext cipher runs only on a **synthetic / non-PHI** instance (`data_class ≠ phi`, e.g. CI) or via the loud, audited `[store].allow_unencrypted_phi` opt-out (a second `[security].allow_unencrypted_phi_under_strict_enforcement` ack under strict enforcement). Covers the `_CIPHER_COLUMNS` set — `messages.raw`/`.summary`/`.metadata`/`.error`, `queue.payload`/`.last_error`, `message_events.detail`, `users.totp_secret`, `connection_event.reason`, `alert_instance.reason` (`summary`/`metadata` carry ingest-derived MRN/patient-name PHI); WebAuthn COSE public keys are deliberately excluded (verification material, not a secret). **Usage scope:** this DEK is the confidentiality key for the at-rest store columns above **only** — message bodies + ingest-derived MRN/patient-name PHI; it is **not** a transport, signing, or token key, is never used to verify a signature, and never leaves the engine host. | **Keyring:** active key `MEFOR_STORE_ENCRYPTION_KEY` (or DPAPI-protected `MEFOR_STORE_ENCRYPTION_KEY_FILE`) + decrypt-only `MEFOR_STORE_ENCRYPTION_KEYS_RETIRED`; `key_id` = SHA-256 fingerprint | **See key-management policy below** | | Vault Transit KeyProvider (opt-in, [ADR 0019](../adr/0019-pluggable-keyprovider-hsm-kms-vault.md)) | Envelope-unwraps the store DEK: Vault Transit `decrypt_data` returns the base64 32-byte DEK, decrypting the wrapped DEK (`vault:v1:…`) against a **non-extractable KEK held inside Vault**; only the wrapped DEK sits at rest. **Fail-closed** — any missing config or Transit/transport failure raises `KeyProviderError` and `serve` refuses to start (never degrades to the identity/plaintext cipher); key material is never logged (only the exception type). `hvac` in [`store/keyprovider_vault.py`](../../messagefoundry/store/keyprovider_vault.py). **Usage scope:** the Vault KEK **only** unwraps the store DEK inside Vault — it never encrypts store data directly and never leaves Vault; the DEK it yields protects exactly the at-rest store columns above, and no transport / signing / token material. | Env-sourced: `MEFOR_STORE_VAULT_ADDR` / `MEFOR_STORE_VAULT_TOKEN` / `MEFOR_STORE_VAULT_TRANSIT_KEY` / `MEFOR_STORE_VAULT_WRAPPED_DEK`; `hvac` behind the optional `[vault]` extra | **OFF by default** — `[store].key_provider` defaults to `auto` (env-then-DPAPI key sourcing for the in-process `aesgcm` cipher), not `vault` | | Store-at-rest cipher — Transit mode (`[store].cipher_provider = vault_transit`, opt-in, [ADR 0138](../adr/0138-transit-bulk-crypto-provider-dek-out-of-engine-heap-for-asvs-13-3-3-demand-gated.md)) | Bulk at-rest AEAD encrypt/decrypt runs **inside** Vault/OpenBao Transit — each value is stored as the `mfenc:v3:` marker + Transit's own `vault:v1:…` ciphertext, and the audit-chain MAC is computed **inside** Transit via `generate_hmac` (forgery-**resistant**, not keyless SHA-256). `TransitCipher` in [`store/crypto_transit.py`](../../messagefoundry/store/crypto_transit.py) imports **none** of the six stdlib crypto modules — every primitive is delegated over the Vault HTTP seam, so the plaintext DEK **never enters engine heap** (ASVS 13.3.3 isolated security module; 13.3.1's L3 hardware clause still wants the vault HSM-sealed). ASVS 11.3.3 cell-binding rides for free — `cell_aad(table, column, *pk)` is forwarded as Transit `associated_data`. **Fail-closed** — missing config or an unreachable/unknown Transit key raises `KeyProviderError` and `serve` refuses to start (never degrades to plaintext); a per-op Transit failure raises `CipherError`, surfacing only the exception **type**, never key material or PHI. **Usage scope:** the named Transit keys encrypt/decrypt **only** the at-rest store columns above (data key) and compute **only** the audit-chain MAC (audit key); they are Vault-resident, non-exportable, and protect no transport / signing / token material. | Env-named Transit keys `MEFOR_STORE_TRANSIT_KEY` (data) + optional `MEFOR_STORE_TRANSIT_AUDIT_KEY` (audit MAC; unset ⇒ reuses the data key), reached via `MEFOR_STORE_VAULT_ADDR` / `MEFOR_STORE_VAULT_TOKEN`; `hvac` behind the optional `[vault]` extra | **OFF by default** — `[store].cipher_provider` defaults to `aesgcm`; the key material stays inside Vault, so **roll the Transit key version in Vault** (prior versions still decrypt existing `mfenc:v3` rows) | -| Audit chain | Row-hash chain (tamper-evident): **keyless SHA-256** in the default keyless posture, upgraded to **HMAC-SHA256** — keyed on an HKDF-SHA256-derived (`mefor/audit-chain/v1`) subkey of the store DEK — only when a store key is configured (#190), or on an isolated-module Transit MAC under `cipher_provider=vault_transit` (ADR 0138); both modes hash identical canonical bytes, so keyless deployments and legacy rows still verify. The digest primitive is `audit_row_hash` in `store/store.py` (`hashlib` + `hmac`), shared verbatim by all three backends. **Verification is constant-time and full-walk (ASVS 11.2.4):** `hmac.compare_digest` over `audit_mac_bytes` on every row MAC *and* on the external-anchor head, in `store/store.py`, `store/postgres.py` and `store/sqlserver.py` — the walk never returns early, so verify duration is a function of chain length, not of where a forgery sits (the first divergent row id is still named in the operator-facing result) | `audit_log.row_hash` | Append-only; verified by `messagefoundry audit-verify` | +| Audit chain | Row-hash chain (tamper-evident): **keyless SHA-256** in the default keyless posture, upgraded to **HMAC-SHA256** — keyed on an HKDF-SHA256-derived (`mefor/audit-chain/v1`) subkey of the store DEK — only when a store key is configured (#190), or on an isolated-module Transit MAC under `cipher_provider=vault_transit` (ADR 0138); both modes hash identical canonical bytes, so keyless deployments and legacy rows still verify. The digest primitive is `audit_row_hash` in `store/store.py` (`hashlib` + `hmac`), shared verbatim by all three backends. **Usage scope:** the HKDF-derived subkey (and, under `vault_transit`, the named Transit audit key) authenticates the **audit row chain only** — an integrity/tamper-evidence key, never a confidentiality key. It is domain-separated from the store DEK it is derived from (`mefor/audit-chain/v1`), decrypts nothing, protects no message body, and is not transport, signing, or token material. **Verification is constant-time and full-walk (ASVS 11.2.4):** `hmac.compare_digest` over `audit_mac_bytes` on every row MAC *and* on the external-anchor head, in `store/store.py`, `store/postgres.py` and `store/sqlserver.py` — the walk never returns early, so verify duration is a function of chain length, not of where a forgery sits (the first divergent row id is still named in the operator-facing result) | `audit_log.row_hash` | Append-only; verified by `messagefoundry audit-verify` | | Config fingerprint ([ADR 0041](../adr/0041-load-path-attestation-and-change-attribution.md)) | SHA-256 content digest of a loaded config bundle — path-relative Merkle fold over every loaded file (`*.py` incl `_*.py`, `connections.toml`, `codesets/*`, `environments/*.toml`); `hashlib` in `config/fingerprint.py` | Recorded in the `config_reload` audit detail (not stored as a secret) | Recomputed per reload/startup; binds reviewed-commit → loaded-bytes (integrity/attribution, not confidentiality) | | Engine wheel attestation ([ADR 0041](../adr/0041-load-path-attestation-and-change-attribution.md) D3) | SHA-256 over each **loaded** first-party `messagefoundry` module file, compared to the installed wheel's `*.dist-info/RECORD` baseline (a base64 `sha256=` manifest already in the wheel); `hashlib` in `integrity.py` | Drift recorded in the hash-chained `startup_integrity` audit row (not a secret); RECORD baseline read from site-packages metadata | Recomputed at startup + on demand; in-place-tamper tripwire (integrity, not confidentiality). Alert-only by default; `[integrity].fail_closed_on_drift` refuses to start on drift; no-op on an editable install | | Outbound message signing (opt-in) | Detached JWS (RFC 7515) — RS256/PS256 (RSA) or ES256 (ECDSA P-256), SHA-256; `cryptography` in `transports/signing.py` (ASVS 4.1.5, [ADR 0018](../adr/0018-per-message-signatures-accepted-risk.md)) | Operator-supplied PEM **private** signing key per connection (inline via `env()` or a PEM file path; encrypted-key passphrase via `env()`); the **public** key is shared with the partner out-of-band. **Usage scope:** this private key **only** signs this connection's outbound per-message JWS — a message-**authenticity/integrity** key in transit; it is never used for at-rest encryption or session/token material, and the partner holds only the matching **public** verification half | **OFF by default**; per-connection opt-in. `kid` carried in the JWS header so key rotation / a managed provider ([ADR 0019](../adr/0019-pluggable-keyprovider-hsm-kms-vault.md)) slots in without a wire change | diff --git a/tests/test_key_usage_scope_inventory.py b/tests/test_key_usage_scope_inventory.py new file mode 100644 index 00000000..910049f7 --- /dev/null +++ b/tests/test_key_usage_scope_inventory.py @@ -0,0 +1,151 @@ +# SPDX-License-Identifier: AGPL-3.0-or-later +# Copyright (C) 2026 MessageFoundry Organization and contributors +"""ASVS 11.1.2: every key-material row in the §4 inventory must document its **usage scope**. + +11.1.2 asks that cryptographic keys be inventoried *with the scope of their use* — which key protects +which data, for what property. The 2026-07 assessment scored this cell **Pass** on the strength of the +``**Usage scope:**`` clauses in ``docs/ASVS-L2-PHASE0-CHANGES.md`` §4. + +**Nothing pinned them.** ``grep -rn 'Usage scope' tests/`` returned nothing before this module: a new +key-material row could ship with no scope clause, or an existing clause could be deleted, and the cell +would silently regress from a Pass that had been claimed. That is the same defect class as the +threat-model absence claim (``test_threat_model_doc_drift``) and the un-fingerprinted rotation class +(``test_secret_rotation_inventory``) — a *documentation* control with no guard over the document. + +Writing this guard found one live gap: the **Audit chain** row keys its HMAC on an HKDF-derived subkey +of the store DEK (and, under ``vault_transit``, a named Transit audit key) and carried no scope clause, +despite being exactly the case 11.1.2 is about — a key derived from a confidentiality key but used for +a different property, whose separation is the thing worth writing down. + +**Why an explicit classification rather than a regex.** A "does this row look like key material" +heuristic mis-sorted two of twenty rows on first contact (it missed a plural, and it flagged the audit +chain — which turned out to be right, but for the wrong reason). A guard whose false-positive rate is +non-zero gets suppressed. So every row is classified here, and an UNCLASSIFIED row fails: a new row +cannot be added without someone deciding which side it is on. This is the ``CRITICAL_SECRETS`` pattern +from ``test_secret_rotation_inventory`` — a curated registry plus a completeness check against +discovery, so the curation cannot silently fall behind. + +Unlike the threat-model guard, this document is **tracked**, so this module runs where CI runs. +""" + +from __future__ import annotations + +import re +from pathlib import Path + +_ROOT = Path(__file__).resolve().parent.parent +_DOC = _ROOT / "docs" / "ASVS-L2-PHASE0-CHANGES.md" +_SECTION = "## 4. Key-management" + +#: Rows that describe KEY MATERIAL — a key the engine holds, derives, or consumes to protect the +#: confidentiality or authenticity of something. Each MUST carry a ``**Usage scope:**`` clause naming +#: what it protects and, as importantly, what it does NOT. +_KEY_MATERIAL = frozenset( + { + "Store-at-rest cipher", # both rows: in-process aesgcm and vault_transit + "Vault Transit KeyProvider", + "Audit chain", + "Outbound message signing", + "DIRECT S/MIME", + "OIDC IdP JWKS verification keys", + "OIDC IdP TLS trust anchor", + "Cert tooling", + } +) + +#: Rows that are NOT key material, each with the reason. These are inventoried in §4 because the +#: section covers the whole cryptographic surface, but 11.1.2's usage-scope clause does not apply: +#: there is no key whose scope could be stated. +_NOT_KEY_MATERIAL: dict[str, str] = { + "Local password hashes": "a one-way hash with no key — argon2id output, nothing to scope", + "Session tokens": "CSPRNG bearer values stored as SHA-256; a token is not a key", + "WebAuthn ceremony challenges": "single-use CSPRNG nonces — no key material", + "WebAuthn credentials": "COSE PUBLIC keys supplied by the authenticator; the engine holds no " + "private half and the row already says they are verification material, not a secret", + "Config fingerprint": "a keyless content hash for change attribution", + "Engine wheel attestation": "a keyless digest over the installed distribution, verified against " + "a recorded value; no key is involved on either side", + "AD transport": "a TLS hop whose key material is the OS/directory trust store, not engine-held", + "SQL Server transport": "a TLS hop trusted via the OS certificate store and the ODBC driver; the " + "engine holds no key for it", + "Console → engine TLS": "a TLS hop configured from the engine's own listener cert (scoped in " + "the Cert tooling row)", + "Tray → engine TLS": "a tokenless local TLS probe; no engine-held key", + "Engine-shard lane ownership": "a coordination record, not cryptographic material", +} + + +def _rows() -> list[tuple[str, str]]: + """``(label, full row text)`` for every body row of the §4 inventory table.""" + lines = _DOC.read_text(encoding="utf-8").splitlines() + start = next(i for i, line in enumerate(lines) if line.startswith(_SECTION)) + end = next(i for i, line in enumerate(lines[start + 1 :], start + 1) if line.startswith("### ")) + out: list[tuple[str, str]] = [] + for line in lines[start:end]: + if not line.startswith("| ") or re.match(r"^\|\s*-+", line) or "| Asset " in line: + continue + asset = line.strip().strip("|").split("|")[0].strip() + # The label is the asset name before its first qualifier — an ADR link, a parenthetical, an + # em-dash variant, or a config token. Two rows share the "Store-at-rest cipher" label (the + # aesgcm and vault_transit modes); both are key material, so collapsing them is correct. + out.append((re.split(r"\s*\(|\s*\[|\s*—|\s*`", asset)[0].strip(), line)) + return out + + +def test_the_inventory_table_was_actually_parsed() -> None: + """Liveness receipt. Every assertion below iterates the parsed rows, so a heading rename or a + table restructure would turn this module into a wall of green over an empty list.""" + rows = _rows() + assert len(rows) >= 18, f"parsed only {len(rows)} inventory rows — the §4 table shape moved" + labels = {label for label, _ in rows} + assert "Store-at-rest cipher" in labels and "Audit chain" in labels, ( + f"the anchor rows are missing from the parse; got {sorted(labels)}" + ) + + +def test_every_row_is_classified() -> None: + """Completeness: a NEW inventory row must be classified as key material or not. + + This is what stops the curated set falling behind the document. Mutation: add a row to §4 without + touching this file. Red: named below. + """ + unclassified = sorted({label for label, _ in _rows()} - _KEY_MATERIAL - set(_NOT_KEY_MATERIAL)) + assert not unclassified, ( + f"unclassified §4 inventory row(s): {unclassified}. Add each to _KEY_MATERIAL (and give the " + f"row a **Usage scope:** clause) or to _NOT_KEY_MATERIAL with the reason it holds no key." + ) + + +def test_every_key_material_row_documents_its_usage_scope() -> None: + """The property ASVS 11.1.2 was scored Pass on. + + Mutation: delete the ``**Usage scope:**`` clause from any key-material row. Red: that row is named. + """ + missing = sorted( + label for label, row in _rows() if label in _KEY_MATERIAL and "**Usage scope:**" not in row + ) + assert not missing, ( + f"key-material row(s) with no **Usage scope:** clause: {missing}. ASVS 11.1.2 requires the " + f"inventory to record what each key protects — and what it does not. This cell is currently " + f"scored Pass on exactly these clauses." + ) + + +def test_the_classification_does_not_rot() -> None: + """Both sets must name rows that still exist, and must not overlap. + + A stale entry silently stops guarding (the row it names is gone); an overlapping one is + self-contradictory. Mutation: rename a row in the doc without updating this file. Red below. + """ + labels = {label for label, _ in _rows()} + stale = sorted((_KEY_MATERIAL | set(_NOT_KEY_MATERIAL)) - labels) + assert not stale, f"classification names §4 rows that no longer exist: {stale}" + overlap = sorted(_KEY_MATERIAL & set(_NOT_KEY_MATERIAL)) + assert not overlap, f"rows classified as BOTH key material and not: {overlap}" + + +def test_a_non_key_row_carries_a_reason_not_an_empty_excuse() -> None: + """An exclusion list whose entries carry no reason is where real key material gets parked. Every + reason must be a sentence, not a placeholder.""" + thin = sorted(k for k, v in _NOT_KEY_MATERIAL.items() if len(v.split()) < 5) + assert not thin, f"_NOT_KEY_MATERIAL entries with no real reason: {thin}" diff --git a/tests/test_release_pipeline.py b/tests/test_release_pipeline.py index ab890ffe..c411dadc 100644 --- a/tests/test_release_pipeline.py +++ b/tests/test_release_pipeline.py @@ -180,7 +180,16 @@ def test_release_load_bearing_canaries_present() -> None: def test_release_pypi_publish_is_last_step_and_tag_gated() -> None: rel = _release() # Isolate the `release` job (up to the next top-level job) so ordering is measured within it. - release_job = rel.split("\n release-harness:", 1)[0] + # The boundary is DERIVED rather than the name of whichever job happened to follow: this read + # `rel.split("\n release-harness:")`, so inserting release-webconsole between the two silently + # widened the slice to include it and the assertion then measured the WRONG job's last step. + _start = rel.index("\n release:") + 1 + _next = re.search(r"^ [a-z][\w-]*:$", rel[_start:], re.M | re.I) + _after = re.search( + r"^ [a-z][\w-]*:$", rel[_start + (_next.end() if _next else 0) :], re.M | re.I + ) + assert _after, "could not find the job after `release` — the workflow shape moved" + release_job = rel[_start:][: (_next.end() if _next else 0) + _after.start()] # Step boundaries are ` - name:` / ` - uses:` at the job's step indent. steps = list(re.finditer(r"^ - (?:name|uses): (.+)$", release_job, re.M)) @@ -331,7 +340,89 @@ def test_both_wheel_smokes_compare_versions_not_strings() -> None: "a raw string compare of tag vs built version is back; it rejects canonical pre-release " "versions (0.3.0rc1 != 0.3.0-rc1) and blocks every rc tag" ) - assert rel.count("from packaging.version import") == 2, ( - "both the engine and harness wheel smokes must compare PEP 440 versions — fixing only one " - "moves the failure rather than removing it" + # DERIVED, not hardcoded. This read `== 2` and broke the day a third wheel job (the separately + # versioned console) was added — a guard that must be edited whenever the thing it guards grows is + # a guard that gets its number bumped without thought. The property worth pinning is "EVERY wheel + # smoke normalises", so count the jobs that actually build a wheel and require one comparison each, + # plus the engine's own. A new wheel job carrying a string compare now fails HERE. + wheel_builds = rel.count("python -m build --wheel") + assert wheel_builds >= 2, f"expected the harness + console wheel builds, found {wheel_builds}" + assert rel.count("from packaging.version import") == wheel_builds + 1, ( + f"every wheel smoke must compare PEP 440 versions — {wheel_builds} wheel-building job(s) plus " + f"the engine smoke require {wheel_builds + 1} comparisons, found " + f"{rel.count('from packaging.version import')}. Fixing only one moves the failure rather than " + f"removing it." + ) + + +# --- the separately-versioned web console (ASVS 15.2.4) -------------------------------------------- + + +def _jobs() -> dict: + import yaml + + return yaml.safe_load(RELEASE_YML.read_text(encoding="utf-8"))["jobs"] + + +def test_the_console_and_engine_tag_namespaces_are_mutually_exclusive() -> None: + """The console is SEPARATELY VERSIONED (its own ``__version__`` root, changelog and PyPI cadence — + docs/WEBCONSOLE-PACKAGE.md), so it fires on ``webconsole-v*`` while the engine fires on ``v*``. + + If the two guards ever overlap the damage is silent and asymmetric: an engine tag would publish the + console at a version nobody chose, and a console tag would publish the ENGINE at the console's + version. Both are wrong in a way the version-check steps cannot catch, because each checks its own + wheel against the same tag. + + Mutation: drop either ``startsWith(github.ref_name, 'webconsole-')`` clause. Red here. + """ + jobs = _jobs() + engine, console = jobs["release"]["if"], jobs["release-webconsole"]["if"] + assert "!startsWith(github.ref_name, 'webconsole-')" in engine, ( + "the engine release job would fire on a console tag and ship the engine at the console's version" + ) + assert ( + "startsWith(github.ref_name, 'webconsole-')" in console and "!startsWith" not in console + ), "the console release job is not gated to its own tag namespace" + + +def test_the_console_release_does_not_depend_on_the_engine_release() -> None: + """``release-harness`` is deliberately lockstep and so carries ``needs: release``. The console is + deliberately NOT: an engine release must not drag it along, and a console release must not wait on + one. A ``needs`` here would silently couple two cadences the design separates.""" + assert "needs" not in _jobs()["release-webconsole"], ( + "release-webconsole must not depend on the engine release — the console has its own cadence" + ) + + +def test_the_console_version_check_reads_the_console_tag_not_the_engine_tag() -> None: + """The strip must be ``webconsole-v``, not ``v``. With the wrong prefix ``want`` keeps the + ``webconsole-`` text, no PEP 440 parse succeeds, and the job fails on EVERY console tag by + construction — the exact shape of the bug the harness job carried until it was fixed.""" + body = RELEASE_YML.read_text(encoding="utf-8") + console = body[body.index("release-webconsole:") : body.index("release-harness:")] + assert '"${GITHUB_REF_NAME#webconsole-v}"' in console, ( + "the console version check must strip the console tag prefix, not the engine's" + ) + assert "messagefoundry_webconsole-" in console, "it must read the CONSOLE wheel's version" + + +def test_the_console_publish_uses_trusted_publishing_and_is_tag_gated() -> None: + """Same bar as the engine and harness: OIDC, never an API token, and never on a branch push. + + The ``PUBLISH_WEBCONSOLE`` variable gate is deliberate — the build and version-check run on every + console tag so the path is exercised before it is armed. Flipping the variable is what actually + creates the PyPI project and CLAIMS the name (ASVS 15.2.4): a registered *pending* publisher grants + permission to publish but reserves nothing. + """ + body = RELEASE_YML.read_text(encoding="utf-8") + console = body[body.index("release-webconsole:") : body.index("release-harness:")] + assert "pypa/gh-action-pypi-publish@" in console, ( + "the console must publish via the pinned action" + ) + assert "id-token: write" in console, "Trusted Publishing needs the OIDC identity" + assert not re.search(r"password:|PYPI_.*TOKEN|api-token", console), ( + "the console publish must not use an API token — Trusted Publishing only" ) + assert ( + "startsWith(github.ref, 'refs/tags/')" in console and "vars.PUBLISH_WEBCONSOLE" in console + ), "the console publish must be tag-gated AND variable-gated"