Skip to content

ci: check the pinned threat-model baseline hash against its tag - #370

Merged
ChelseaKR merged 2 commits into
mainfrom
ci/366-threat-model-baseline-check
Oct 17, 2026
Merged

ChelseaKR merged 2 commits into
mainfrom
ci/366-threat-model-baseline-check

Conversation

@ChelseaKR

Copy link
Copy Markdown
Owner

Closes #366

What and why

Nothing verified the hash pinned in docs/audits/threat-model-baseline.md; the only check was a manual shasum snippet, which is how docs/threat-model.md drifted from B1 unnoticed.

scripts/check_threat_model_baseline.py (standard library only):

  1. Fails if docs/threat-model.md read from a baseline's git tag no longer hashes to the value pinned in the audit file (the immutability B1 claims), or if the pinned tag cannot be read. A tag that cannot be read is a failure, not a pass, so the CI job checks out full history with tags.
  2. Reports, never fails, when the current document differs from the latest baseline, naming commits and lines changed since the freeze. In Actions this is a ::warning:: annotation. Today it prints: 2 commit(s) since the freeze, 80 lines added, 28 removed.

It parses every ## Baseline Bn section, so a future B2 is appended without editing the script: all baselines are checked for immutability, and drift is measured against the latest.

Wiring: new threat-model baseline job in ci.yml (own job, fetch-depth: 0 and fetch-tags: true), plus make threat-model-baseline. It is deliberately not in make verify, because the shallow gate job and tagless clones do not have the tags. Tests in tests/test_threat_model_baseline_check.py build throwaway git repos: untouched passes, edited pin fails, moved tag fails, missing tag fails, drift reports but exits 0, uncommitted edits count, B2 is measured separately from B1.

Not touched

The pinned hash, docs/audits/threat-model-baseline.md, and docs/threat-model.md are unchanged. Cutting B2 still needs a human re-review.

Follow-up for the owner

I did not edit .github/rulesets/main-branch.json. If you want the new job to be a required check, add its context (threat-model baseline (frozen tags must match their pins; drift is report-only)) there.

Checklist

  • make lint type i18n doc-links markers green; new tests pass. make cov run locally as root shows one failure, test_a_failed_container_write_leaves_the_sealed_original_alone, which is the root-permission test fixed separately in test: make the failed container write test hold when the suite runs as root #369 and is unrelated to this change.
  • Tests added; nothing here touches evidence, so no tamper-detection test applies (the immutability check is itself tested against tampering)
  • No real tenant data
  • Respects the README hard rules; verifier untouched
  • Conventional Commit title; commit signed off
  • No change to the threat model, packet format, or verification protocol; CHANGELOG.md entry added
  • Observability: N/A, CI-only, no telemetry
  • Rollback/migration: N/A; delete the job to revert
  • ISO 25010: maintainability and security (integrity of the audit trail)
  • No new dependency

Generated by Claude Code

Fail when docs/threat-model.md at a baseline tag no longer matches the hash
pinned in docs/audits/threat-model-baseline.md (or the tag is unreadable).
Report, without failing, how many commits and lines the current document has
drifted since the latest baseline. The pin and the baseline are untouched.

Closes #366

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GqtiAsmkmDmpynCN35XTzg
Signed-off-by: Claude <noreply@anthropic.com>
On the CI runner the script emitted a ::warning:: annotation instead of the
plain DRIFT line the test expected. Set the variable explicitly per run.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GqtiAsmkmDmpynCN35XTzg
Signed-off-by: Claude <noreply@anthropic.com>
@ChelseaKR
ChelseaKR merged commit 01ad6ee into main Oct 17, 2026
15 checks passed
@ChelseaKR
ChelseaKR deleted the ci/366-threat-model-baseline-check branch October 17, 2026 14:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a CI check for the pinned threat-model baseline hash

2 participants