Skip to content

[P1][ci][governance][adr] ADR governance enforcement — deterministic anti-regression test suite + ADR-0074 smoke-test #1162

Description

@FabioLeitao

Context

The ADR-0080 incident (born Accepted, pt-BR body, ADR-NNNN hyphen in H1, dangling refs, a delete-attempt) exposed that no LLM auditor — across three environments/models — caught violations of ADR-0045 (the ADR that governs ADRs). Root cause: rules-in-place ≠ rules-followed. The fix is mechanical — deterministic anti-regression tests that guarantee ADR governance regardless of which agent (or tired human) is at the keyboard. The ADR-0074 phantom (PR #997 merged but only touched INVENTORY.txt; the .md never existed; 10 orphan refs across CI/Dockerfile/other ADRs) is the live proof and the smoke-test.

Golden design rules (from the incident — non-negotiable)

  1. Self-validate against the corpus. Each test runs GREEN against all 78 existing ADRs before it becomes a hard gate. A naive metadata regex false-positives on Datadog/Database; a naive Status parse false-fails on a CRLF working-tree file → normalize EOL (\r\n\n) and anchor exact tokens. (The ADR-0080 working tree was 69/69 CRLF; git normalized to LF on commit, but a naive parser reading the working tree false-fails.)
  2. Never an irrecoverable false-block. A deterministic gate must not block a legitimate operation with no escape. Uncertain validation → a smarter test (e.g. rename-detection) OR an operator override (trailer/label) — never a dead-end.

Phase 1 — deterministic (pre-commit + CI-required backstop)

  • T1 — lifecycle: a newly-added ADR (git diff --diff-filter=A -- 'docs/adr/ADR-[0-9]*.md') must have ## Status ∈ {Proposed, Reserved}; else fail. ADR-0045 §5. Mirror scripts/gate_change_tripwire.py. Normalize EOL before parsing.
  • T2 — locale/structure: H1 matches ^# ADR \d{4} — (space, not the filename hyphen); ##/### headings free of pt-BR words (denylist: Contexto, Decisão, Consequências, Governança, Autoridade, Justificativa, Alternativas Consideradas, Decisões Relacionadas, Referências, …); metadata labels EXACT (- **Date (UTC):**, - **Authors:**, - **Deciders:** — anchored, NOT a Data* prefix). ADR-0045 §7/§3.
  • T5 — anti-deletion: an ADR file removed → fail (ADR-0045 §1 / §Decision item 1: ADRs are never deleted). rename-aware (git diff -M --find-renames): a rename/reformat (ADR-nnnADR-nnnn, e.g. an imported ADR) is R (allowed); only a true erasure (D with no corresponding new ADR) blocks.
  • T6 — date immutability: - **Date (UTC):** equals the file's git first-appearance date, and never changes on later edits. ADR-0045 §3.

Phase 2 — CI (heuristic / repo-wide)

  • T3 — prose locale (deeper): for each ADR, strip fenced/inline code, markdown links [..](..), blockquotes, and the ## References + ## Related Decisions sections; then pt-BR word density > threshold → fail. Heuristic → validate vs the 78-corpus, ship as warning then harden once tuned. Extends tests/test_docs_pt_br_locale.py.
  • T4 — anti-dangling reference: every ADR-\d{4} token anywhere in the repo (code/CI/Dockerfile/docs/PLANs/other ADRs) resolves to an existing docs/adr/ADR-NNNN-*.md OR a declared Reserved number; else fail. Catches the docs(adr): ADR-0074 supply-chain posture Proposed — refs #987 #997/ADR-0074 phantom class. Complements test_adr_inventory_sync (covers file↔inventory but NOT code-refs↔file).

Candidates (confirm mechanizability later)

Status ∈ valid enum (§4) · filename ADR-NNNN-kebab (§6) · minimum canonical sections (§3) · Superseded/Duplicate of ADR-NNNN cites an existing ADR (§4) · Status history append-only (complex — historical diff).

Governance amend (ADR-0045, in-place per §8, dated Status history)

Codify: "Every mechanizable ADR-0045 governance rule SHALL have a deterministic anti-regression test (pre-commit and/or CI). An unmechanized rule is recorded with its reason and a revisit path — never 'unenforceable by default.'" Host = ADR-0045 (NOT ADR-0048, which is operator-facing naming-contract preservation).

Validation — ADR-0074 smoke-test

Materialize the phantom ADR-0074-supply-chain-layer1-digest-pins-and-rust-sca.md (decision content in #987 + PLAN_IMAGE_HARDENING + #998) with 3 planted violations to confirm each test catches them: born AcceptedT1; a pt-BR heading → T2; a pt-BR paragraph → T3. The existing 10 orphan refs validate T4 (RED now → GREEN once the file exists). Run blind (don't coach the reviewers) to also measure whether the AI auditors catch them unaided.

Acceptance criteria

  • Tests in tests/ (extend where noted; do not duplicate test_adr_inventory_sync / test_adr_readme_index_sync).
  • Each test proven GREEN against the current 78-ADR corpus (paste the run) before wiring as a hard gate.
  • EOL-normalized parsers (\r\n\n) — the ADR-0080 working tree was CRLF; a naive parser false-fails.
  • Phase-1 wired into pre-commit + CI-required; Phase-2 into CI.
  • docs/plans/PLAN_ADR_GOVERNANCE_ENFORCEMENT.md + python scripts/plans_hub_sync.py --write + a PLANS_TODO.md entry.
  • Materialize ADR-0074 (born Proposed, en_US) resolving the 10 orphan refs.
  • The full local gate (./scripts/check-all.sh --enforced) green before push/PR (ADR-0080). Do not merge — operator-gated.

Ties

ADR-0045 §1/§3/§4/§5/§7/§8 · ADR-0056 (cryptographic ADR inventory) · ADR-0080 (the incident) · #1151 (versioned pre-push hook) · #987/#997/#998 (ADR-0074 lineage) · #1160/#1161 (ADR-0080 saga).


Spec by Claude Code (Linux, read-only auditor) — implementation + PR by Cursor. Claude Code stays OUT of the ADR-0074 audit chain for the smoke-test (planted the errors + built the backstop; cannot be a fair auditor of them).

Metadata

Metadata

Assignees

Labels

adrArchitecture Decision Record

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions