From 1a14453cf9874ad45a23ec2737bc45c19ce17b8e Mon Sep 17 00:00:00 2001 From: wshallwshall Date: Wed, 12 Aug 2026 16:38:32 -0500 Subject: [PATCH 1/2] docs(claude): list the four missing subpackages in the section 3 tree The repository-layout tree listed 10 subpackages while messagefoundry/ has 14. security/, support/, verify/ and tray/ were absent, so CLAUDE.md contained ZERO occurrences of "tray" despite ADR 0113 and a 17-module package shipping at HEAD. Cause: the tray, verify and support packages landed in the private lineage between 2026-07-13 and 07-25, and that content never reached the repo of record at publication -- five commits dropped by omission, absent from HEAD. The tree has been stale since. Descriptions taken from each package's own __init__ docstring. The tray line records what it wraps, verified rather than assumed: tray/ imports only messagefoundry.service, messagefoundry.service_status and its own submodules -- no api/, no engine internals -- so section 4's dependency rule needs no carve-out for it. Insertion-only, no section renumbered. That is deliberate: 281 tracked files cite this file by section number, nothing validates a section number (link_check.py:21 validates the path and skips the fragment), and the same rot has already landed once -- test_sds_rule_ids_are_stable.py:7-11 records four security citations that still resolve to the wrong section today. Verified: tests/test_link_resolution.py and tests/test_sds_rule_ids_are_stable.py, 34 passed. The second was confirmed to read this worktree's file by injecting a bogus SDS-9.9 citation and watching it go red, then removing it. --- CLAUDE.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/CLAUDE.md b/CLAUDE.md index a627b8af..c5e75076 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -189,6 +189,10 @@ messagefoundry/ api/ # FastAPI app.py + models.py + security.py (auth deps) + auth_routes.py (the engine's only external surface) apiclient/ # Qt-free / FastAPI-free engine-client library (ADR 0088) — the shared HTTP client (httpx) generators/ # conformant synthetic HL7 generators (adt.py, …) — `messagefoundry generate`; corpus git-ignored + security/ # security assets shipped in the wheel (ADR 0144) + support/ # support-bundle assembly + redaction (bundle.py, redact.py) + verify/ # deployment verifier — `messagefoundry verify` (checks.py, smoke.py, federation.py) + tray/ # Windows tray service-manager (ADR 0113) — stdlib ctypes, no PySide6; wraps service/service_status only checks.py # `messagefoundry check` commit/CI gate (validate + dryrun + advisory lint) ide/ # VS Code extension (TypeScript): setup, promote, test bench, AI commands environments/ # per-environment .toml value files for env() lookups (dev/staging/prod) From e7e825ed9b6e69d5715c6d0cc7868be2824dbc18 Mon Sep 17 00:00:00 2001 From: wshallwshall Date: Wed, 12 Aug 2026 17:13:42 -0500 Subject: [PATCH 2/2] feat(gate): resolve every cited CLAUDE.md section number, and stop a nested anchor escaping the SDS gate CLAUDE.md's numbered sections are a de facto API and NOTHING validated a section number. Measured: 281 tracked files name the file, 646 citations name one of its sections, and tests/test_dependency_boundaries.py cites section 4 in its own docstring, so this reaches code and not only prose. Every existing gate stays green through a renumber. link_check.py:21 says so in its own header -- it resolves the PATH and skips the #fragment -- and the two doc guards that already read CLAUDE.md check other things: one checks SDS-N.N identifiers, one checks link paths. Path resolves, identifiers resolve, only the meaning moves. The checkable half stays green. The rot is not hypothetical. The sibling guard tests/test_sds_rule_ids_are_stable.py:5-11 records it landing on the Secure Development Standards -- a new section 5 pushed 5-9 to 6-10, and citations still resolve to the wrong section today. The instance was fixed and the class left open. This closes it for the anchor. MEASURED BLAST RADIUS: renumbering section 11 alone breaks 61 citations. scripts/docs/claude_section_check.py, stdlib only, beside its siblings: - parses the anchor's own `## N.` headings as the source of truth - scans tracked .md/.py/.ps1/.yml via git ls-files - requires the anchor's NAME on the same line within a bounded window, so a section number belonging to another document is never attributed to it -- false positives are how a gate gets disabled - ALWAYS prints what it scanned; a finding count alone cannot distinguish a clean scan from one that examined nothing - REFUSES to run when zero headings parse or the anchor is missing, rather than reporting every citation broken or the corpus clean ADDED TO DOC_GUARDS in ci.yml. Without that it would be born unable to fire: a markdown-only PR is classified non-code and runs only that list, so a check landed outside it goes green by absence on exactly the PR shape it exists to catch. That enumeration hole has already been fixed twice for other paths (BACKLOG #327, #1200) and left open both times; ci.yml:1050 documents its own recurrence. Also widens _citation_sources() in the SDS guard to include nested CLAUDE.md files. Root is a literal path and the globs are rooted at docs/, messagefoundry/ and scripts/, so a subpackage anchor -- which CLAUDE.md section 3 invites -- would take its SDS citations out of coverage silently. Resolved through git ls-files, not rglob, because this repository nests worktrees under .claude/worktrees/ and an rglob would scan sibling checkouts. There are no nested anchors today; this is the prerequisite, not a response. Verified. Both guards were made to FAIL ON PURPOSE and the injected defect confirmed to land: - renumber section 11 to 13 -> rc=1, 61 citations; revert -> rc=0 - nested CLAUDE.md citing SDS-9.9 -> SDS guard red; remove -> green - 8 self-bite tests, including both refuse-to-run guards and a false-positive case (another document's section 9 is not claimed) - ruff check, ruff format, mypy strict clean - full DOC_GUARDS set, 15 modules: 219 passed, 89 skipped --- .github/workflows/ci.yml | 3 +- scripts/docs/claude_section_check.py | 163 +++++++++++++++++++++++ tests/test_claude_section_citations.py | 174 +++++++++++++++++++++++++ tests/test_sds_rule_ids_are_stable.py | 24 +++- 4 files changed, 362 insertions(+), 2 deletions(-) create mode 100644 scripts/docs/claude_section_check.py create mode 100644 tests/test_claude_section_citations.py diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index b854d13f..92d07ed7 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -214,7 +214,8 @@ jobs: tests/test_docs_runbooks.py tests/test_docs_security_pathways.py tests/test_security_doc_drift.py tests/test_security_doc_rate_limits.py tests/test_threat_model_doc_drift.py tests/test_backlog_status_check.py tests/test_sds_rule_ids_are_stable.py - tests/test_link_resolution.py tests/test_dast_claims.py" + tests/test_link_resolution.py tests/test_dast_claims.py + tests/test_claude_section_citations.py" # Every named module must EXIST. A path typo would otherwise make pytest error on an unknown # file, or — worse under a future -k/--ignore form — silently scan nothing and read as a pass. for m in $DOC_GUARDS; do diff --git a/scripts/docs/claude_section_check.py b/scripts/docs/claude_section_check.py new file mode 100644 index 00000000..3ef1a030 --- /dev/null +++ b/scripts/docs/claude_section_check.py @@ -0,0 +1,163 @@ +"""Resolve every citation of a root ``CLAUDE.md`` section number to a real section. + +WHY THIS EXISTS. ``CLAUDE.md``'s numbered sections are a de facto API: measured 2026-08-12, 281 +tracked files cite them and ``tests/test_dependency_boundaries.py`` names section 4 in its own +docstring. **Nothing validated a section number.** ``scripts/docs/link_check.py`` says so in its own +header -- it resolves the PATH and skips the ``#fragment`` -- so renumbering lands entirely green: the +link still resolves, the SDS identifiers still resolve, and only the meaning moves. That is the +half-rot shape, where the checkable half stays green. + +It is not hypothetical. ``tests/test_sds_rule_ids_are_stable.py`` records the same rot landing on a +different document: inserting a new section 5 pushed 5-9 to 6-10, and four security citations still +resolve to the wrong section today. The lesson was learned for one document and the class left open +for this one. + +WHAT IT DOES NOT DO, stated because a completeness claim is a liability (SDS-3.6). It validates *at +least* the citations that name ``CLAUDE.md`` and a section on the same line, within +``_WINDOW`` characters. A citation that names the file on one line and the section on the next is not +seen; neither is a bare ``section 4`` whose subject is established a paragraph earlier. This is a +deliberate floor, not a ceiling: the alternative is guessing which document a bare section number +belongs to, and a checker that guesses produces false positives, which is how a gate gets disabled. + +USAGE + python scripts/docs/claude_section_check.py # scan, print counts, exit non-zero on a miss + python scripts/docs/claude_section_check.py --list # print every citation found and its verdict + +The scan volume is always printed. A gate that reports only what it FOUND cannot be told apart from +one that scanned nothing. +""" + +from __future__ import annotations + +import argparse +import re +import subprocess +import sys +from dataclasses import dataclass +from pathlib import Path + +_REPO = Path(__file__).resolve().parents[2] +_ANCHOR = "CLAUDE.md" + +# Section headings in the anchor: "## 4. Modularity & Extension Points" -> 4. +_HEADING = re.compile(r"^##\s+(\d+)\.\s") + +# A citation is the anchor's name followed, within _WINDOW characters on the SAME line, by a section +# reference. Both spellings the repo actually uses are accepted. +_WINDOW = 60 +_CITATION = re.compile( + rf"CLAUDE\.md(?P.{{0,{_WINDOW}}}?)(?:§|(?\d+)" +) + +# Files that may carry a citation. Matches the scan the finding was measured over. +_SUFFIXES = (".md", ".py", ".ps1", ".yml") + + +@dataclass(frozen=True) +class Citation: + path: str + line: int + section: int + text: str + + +def anchor_sections(anchor: Path) -> set[int]: + """Section numbers defined by the anchor's own ``## N.`` headings.""" + found: set[int] = set() + for line in anchor.read_text(encoding="utf-8").splitlines(): + m = _HEADING.match(line) + if m: + found.add(int(m.group(1))) + return found + + +def tracked_files(root: Path) -> list[Path]: + """Tracked files that may carry a citation. + + Uses ``git ls-files`` rather than a filesystem walk so the scan matches what is committed -- + an untracked scratch copy of a doc is not part of the repo's citation surface. + """ + out = subprocess.run( # nosec B603 B607 - fixed argv, no shell, no caller-supplied executable + ["git", "-C", str(root), "ls-files", "-z"], + capture_output=True, + text=True, + encoding="utf-8", + check=True, + ).stdout + paths = [] + for rel in out.split("\0"): + if rel and rel.endswith(_SUFFIXES): + p = root / rel + if p.is_file(): + paths.append(p) + return paths + + +def citations_in(path: Path, root: Path) -> list[Citation]: + """Every anchor-section citation on a single line of ``path``.""" + try: + text = path.read_text(encoding="utf-8") + except UnicodeDecodeError: + return [] + rel = path.relative_to(root).as_posix() + out: list[Citation] = [] + for lineno, line in enumerate(text.splitlines(), start=1): + if _ANCHOR not in line: + continue + for m in _CITATION.finditer(line): + out.append(Citation(rel, lineno, int(m.group("num")), m.group(0).strip())) + return out + + +def scan(root: Path) -> tuple[set[int], list[Citation], list[Citation]]: + """Return (defined sections, every citation found, the unresolvable ones).""" + anchor = root / _ANCHOR + if not anchor.is_file(): + raise SystemExit(f"{_ANCHOR} not found at {anchor} -- refusing to report a clean scan") + sections = anchor_sections(anchor) + if not sections: + raise SystemExit( + f"parsed ZERO section headings from {_ANCHOR}. The heading format changed, or the file " + f"is empty. Refusing to report every citation as broken, and refusing to pass." + ) + every: list[Citation] = [] + for path in tracked_files(root): + every.extend(citations_in(path, root)) + return sections, every, [c for c in every if c.section not in sections] + + +def main(argv: list[str] | None = None) -> int: + ap = argparse.ArgumentParser(description=__doc__) + ap.add_argument("--list", action="store_true", help="print every citation found") + ap.add_argument("--root", type=Path, default=_REPO) + args = ap.parse_args(argv) + + sections, every, broken = scan(args.root) + + # Print WHAT WAS SCANNED, always. A count of findings alone cannot distinguish a clean scan from + # one that examined nothing. + files = len({c.path for c in every}) + print( + f"{_ANCHOR} defines sections {sorted(sections)}; " + f"scanned {len(tracked_files(args.root))} tracked {'/'.join(_SUFFIXES)} files; " + f"found {len(every)} section citations across {files} files." + ) + if args.list: + for c in sorted(every, key=lambda c: (c.path, c.line)): + mark = "OK " if c.section in sections else "BAD" + print(f" {mark} {c.path}:{c.line} section {c.section} {c.text!r}") + + if broken: + print(f"\n{len(broken)} citation(s) name a section {_ANCHOR} does not define:") + for c in sorted(broken, key=lambda c: (c.path, c.line)): + print(f" {c.path}:{c.line} cites section {c.section}: {c.text!r}") + print( + f"\n{_ANCHOR} defines only {sorted(sections)}. Either the citation is wrong, or a " + f"section was renumbered and its citers were not updated." + ) + return 1 + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tests/test_claude_section_citations.py b/tests/test_claude_section_citations.py new file mode 100644 index 00000000..e5f89bf4 --- /dev/null +++ b/tests/test_claude_section_citations.py @@ -0,0 +1,174 @@ +"""Every cited ``CLAUDE.md`` section number must resolve to a section that exists. + +``CLAUDE.md``'s numbered sections are a de facto API. Measured 2026-08-12: 281 tracked files name the +file and 646 citations name one of its sections; ``tests/test_dependency_boundaries.py`` cites section +4 in its own docstring, so this reaches code and not only prose. + +Before this guard, nothing validated a section NUMBER. ``scripts/docs/link_check.py`` states in its own +header that it resolves the path and skips the ``#fragment``, and the two doc guards that already read +``CLAUDE.md`` check other things -- ``test_sds_rule_ids_are_stable`` checks ``SDS-N.N`` identifiers and +``test_link_resolution`` checks link paths. So a renumber landed entirely GREEN: path resolves, +identifiers resolve, only the meaning moved. + +That rot is not hypothetical. ``test_sds_rule_ids_are_stable``'s own module docstring records it +happening to a different document -- a new section 5 pushed 5-9 to 6-10, and four security citations +still resolve to the wrong section today. The instance was fixed and the class left open. This closes +it for the anchor. + +MEASURED BLAST RADIUS, which is why the guard is worth its maintenance: renumbering section 11 alone +breaks 61 citations, and every existing gate stays green through it. +""" + +from __future__ import annotations + +import importlib.util +import sys +from pathlib import Path +from typing import Any + +import pytest + +_REPO = Path(__file__).resolve().parents[1] +_CHECKER = _REPO / "scripts" / "docs" / "claude_section_check.py" + + +def _load() -> Any: + """Import the checker by FILE path -- ``scripts/`` is not a package. + + Same loading pattern as the other scripts/docs guards. Importing the shipped module rather than + re-implementing its rule is the point: a test that re-states the logic passes however the real + code behaves. + """ + spec = importlib.util.spec_from_file_location("claude_section_check", _CHECKER) + assert spec and spec.loader + mod = importlib.util.module_from_spec(spec) + # Register BEFORE exec: ``@dataclass`` resolves its class's module through ``sys.modules`` at + # decoration time, and an unregistered module makes that lookup return None. The failure is an + # AttributeError raised from inside dataclasses.py, which reads like a bug in the checker. + sys.modules[spec.name] = mod + spec.loader.exec_module(mod) + return mod + + +@pytest.fixture(scope="module") +def checker() -> Any: + return _load() + + +# --- the live corpus --------------------------------------------------------------------------- + + +def test_every_cited_section_resolves(checker: Any) -> None: + """The whole repository's citations resolve against the anchor as it stands.""" + sections, every, broken = checker.scan(_REPO) + assert not broken, "\n".join( + [ + "Citations name a CLAUDE.md section that does not exist:", + *(f" {c.path}:{c.line} cites section {c.section}: {c.text!r}" for c in broken), + f"CLAUDE.md defines only {sorted(sections)}.", + ] + ) + + +def test_the_scan_actually_scanned_something(checker: Any) -> None: + """A clean result is evidence only if the scan had a corpus to be clean ABOUT. + + Without this, deleting the anchor's headings or breaking the citation regex would produce zero + findings and a passing test -- the failure mode this whole guard exists to prevent, reproduced + inside the guard itself. + """ + sections, every, _ = checker.scan(_REPO) + assert len(sections) >= 10, f"parsed only {len(sections)} section headings from CLAUDE.md" + assert len(every) >= 400, f"found only {len(every)} citations; expected hundreds" + assert len({c.path for c in every}) >= 100, "citations found in suspiciously few files" + + +# --- prove the instrument ---------------------------------------------------------------------- +# +# Each assertion above is fired at input built to break it. A guard that has never been observed to +# fail is not evidence. + + +def _fixture_repo(tmp_path: Path, anchor_body: str, citer_body: str) -> Path: + """A throwaway git repo with an anchor and one citing file. + + Filesystem-real and git-real because the checker resolves its corpus through ``git ls-files``; + a mock would test a different program. + """ + import subprocess + + tmp_path.mkdir(parents=True, exist_ok=True) + (tmp_path / "CLAUDE.md").write_text(anchor_body, encoding="utf-8") + (tmp_path / "doc.md").write_text(citer_body, encoding="utf-8") + for cmd in ( + ["git", "init", "-q"], + ["git", "add", "-A"], + ["git", "-c", "user.email=t@t", "-c", "user.name=t", "commit", "-qm", "x"], + ): + subprocess.run(cmd, cwd=tmp_path, check=True, capture_output=True) + return tmp_path + + +_ANCHOR = "# T\n\n## 1. One\n\ntext\n\n## 2. Two\n\ntext\n" + + +def test_selfbite_a_dangling_citation_is_detected(tmp_path: Path, checker: Any) -> None: + root = _fixture_repo(tmp_path, _ANCHOR, "See CLAUDE.md section 9 for this.\n") + _, every, broken = checker.scan(root) + assert len(every) == 1 and len(broken) == 1 + assert broken[0].section == 9 + + +def test_selfbite_a_renumber_breaks_its_citers(tmp_path: Path, checker: Any) -> None: + """The actual threat: the citation never changes, the anchor's numbering does.""" + citer = "CLAUDE.md section 2 is the rule.\n" + ok = _fixture_repo(tmp_path / "before", _ANCHOR, citer) + assert not checker.scan(ok)[2], "citation should resolve before the renumber" + + renumbered = _ANCHOR.replace("## 2. Two", "## 3. Two") + bad = _fixture_repo(tmp_path / "after", renumbered, citer) + assert checker.scan(bad)[2], "a renumber must break its citers" + + +def test_selfbite_both_citation_spellings_are_seen(tmp_path: Path, checker: Any) -> None: + """The repo uses a section sign and the word. Missing either would silently halve coverage.""" + root = _fixture_repo(tmp_path, _ANCHOR, "CLAUDE.md §9 and CLAUDE.md section 8.\n") + _, every, broken = checker.scan(root) + assert {c.section for c in every} == {8, 9} + assert len(broken) == 2 + + +def test_selfbite_an_unrelated_section_reference_is_not_claimed( + tmp_path: Path, checker: Any +) -> None: + """A section number belonging to ANOTHER document must not be attributed to the anchor. + + False positives are how a gate gets disabled, so the checker deliberately requires the anchor's + name on the same line and within a bounded window. + """ + root = _fixture_repo(tmp_path, _ANCHOR, "See docs/PHI.md section 9 for the retention rule.\n") + _, every, broken = checker.scan(root) + assert every == [] and broken == [] + + +def test_selfbite_it_refuses_to_pass_when_the_anchor_has_no_headings( + tmp_path: Path, checker: Any +) -> None: + """Zero parsed headings must ABORT, never report every citation broken or the corpus clean. + + This is the shape that makes a gate lie: a heading-format change would otherwise turn the guard + into a generator of hundreds of false failures, and the cheapest way to silence that is to delete + the guard. + """ + root = _fixture_repo(tmp_path, "# T\n\nno numbered headings\n", "CLAUDE.md section 1.\n") + with pytest.raises(SystemExit): + checker.scan(root) + + +def test_selfbite_it_refuses_to_pass_when_the_anchor_is_missing( + tmp_path: Path, checker: Any +) -> None: + root = _fixture_repo(tmp_path, _ANCHOR, "CLAUDE.md section 1.\n") + (root / "CLAUDE.md").unlink() + with pytest.raises(SystemExit): + checker.scan(root) diff --git a/tests/test_sds_rule_ids_are_stable.py b/tests/test_sds_rule_ids_are_stable.py index 5a74582a..df82c027 100644 --- a/tests/test_sds_rule_ids_are_stable.py +++ b/tests/test_sds_rule_ids_are_stable.py @@ -39,6 +39,7 @@ from __future__ import annotations import re +import subprocess from dataclasses import dataclass from pathlib import Path @@ -369,14 +370,35 @@ def _citation_sources() -> list[Path]: ``tests/`` is deliberately excluded: the self-bite fixtures below define synthetic identifiers (``SDS-9.1`` and friends) that name no real rule by design, and scanning them would make this guard fail on its own evidence. + + NESTED ``CLAUDE.md`` FILES ARE INCLUDED, and the enumeration above is why they have to be named + explicitly. Root ``CLAUDE.md`` is a literal path and the globs are rooted at ``docs/``, + ``messagefoundry/`` and ``scripts/`` -- so a subpackage ``CLAUDE.md`` (which + ``CLAUDE.md`` section 3 invites) would be scanned only if it happened to land under one of those + three, and a relocated ``SDS-N.N`` citation would otherwise leave this guard's coverage SILENTLY. + There are none today; this is the prerequisite, not a response to one. + + They are resolved through ``git ls-files`` rather than ``_REPO.rglob("CLAUDE.md")`` because this + repository nests worktrees under ``.claude/worktrees/`` -- an rglob would walk into sibling + checkouts and scan a different tree's file. """ + tracked = subprocess.run( # nosec B603 B607 - fixed argv, no shell, no caller-supplied executable + ["git", "-C", str(_REPO), "ls-files", "-z", "CLAUDE.md", "*/CLAUDE.md"], + encoding="utf-8", + capture_output=True, + text=True, + check=True, + ).stdout + nested = sorted(_REPO / rel for rel in tracked.split("\0") if rel) paths = [ _REPO / "CLAUDE.md", + *nested, *sorted((_REPO / "docs").rglob("*.md")), *sorted((_REPO / "messagefoundry").rglob("*.py")), *sorted((_REPO / "scripts").rglob("*.py")), ] - return [p for p in paths if p.is_file()] + seen: set[Path] = set() + return [p for p in paths if p.is_file() and not (p in seen or seen.add(p))] def test_every_cited_id_resolves(rules: list[Rule], retired: list[Retired]) -> None: