diff --git a/docs/ADRs/014_claims_and_the_guards_that_carry_them.md b/docs/ADRs/014_claims_and_the_guards_that_carry_them.md index afddd36..102b66e 100644 --- a/docs/ADRs/014_claims_and_the_guards_that_carry_them.md +++ b/docs/ADRs/014_claims_and_the_guards_that_carry_them.md @@ -79,6 +79,40 @@ Where a guard's inputs are declared (a path list, a module list, a set of names) that the inputs are real**. `rglob` on a nonexistent directory yields nothing rather than raising; a list of paths that no longer exist does not fail a scan, it empties it. +**Amended 2026-08-12.** Mutation-proving is necessary and it is not sufficient, and the +gap has a shape worth naming. + +A mutant list written by the author of the guard covers the cases the author already had +in mind — which are, by construction, the cases the guard already handles. Measured on the +guard that keeps coordinate values out of this public repository: **thirteen input forms +proposed by its author, thirteen caught; twenty-nine proposed independently, fifteen +missed.** Every case in the first list genuinely passed. *"Proven against N mutations"* +reads as a statement about the guard when it is a statement about N. + +The useful rule is a **diagnostic**, not a process: + +> If a guard can only be proven against inputs you invented, that is the signal the guard +> is on the wrong side of a boundary — it is verifying a fact you do not own. + +Prefer moving the check to where the fact lives, or anchoring the mutant list in something +real: this repository's own corpus, the registry's actual rows, an outcome you can observe. +The no-copy scan's stopping rule is the worked example — it derives its list of accepted +forms from this repository's tracked markdown, so a form no document here uses is not a gap. + +Where that is impossible, two cheap obligations: + +- the mutant list is a **declared artifact in the test file**, not a paragraph in a pull + request (`tests/test_ci_sibling_coverage.py::_MUTANTS` is the pattern); +- **a proof must be able to fail** — mutate the function, watch the proof go red, record + it. An assertion whose inputs are derived from the thing under test proves nothing, and + three of those shipped here in two days. + +Deliberately **not** adopted: a requirement that someone other than the author write the +mutants for every guard. The friction would exceed the disease for a single maintainer. +The diagnostic above is the load-bearing part; independent mutants are worth buying only +for the silent-failure class — a leak, an invisible delivery — where the cost of missing +is not a red build but a thing nobody ever notices. + ### §3 Prefer a false negative to a false alarm A guard that cries wolf gets deleted, and then the rule it carried is unguarded — which is diff --git a/docs/ADRs/017_facts_across_a_private_boundary.md b/docs/ADRs/017_facts_across_a_private_boundary.md index 2710c15..b580595 100644 --- a/docs/ADRs/017_facts_across_a_private_boundary.md +++ b/docs/ADRs/017_facts_across_a_private_boundary.md @@ -250,6 +250,52 @@ because a download that has quietly stopped earning its place is how the previou of all this went wrong, and because ADR-016 cannot know it: the decision that obsoletes it is this one. +### §7a What a check in this repository may rest on + +The rule §7 arrives at for one case generalises, and it is worth stating on its own, +because every defect this document's arc produced sat on the wrong side of it. + +> **A check here may rest on a fact we own, on a fact another repository has *declared* in +> the public registry, or on an *outcome* we can observe. It may never rest on another +> repository's implementation.** +> +> Where a needed fact is none of those three, the check is not built here. It is requested +> as a declaration, requested as a consumer obligation, or registered as an accepted gap +> with a named owner and trigger. + +[ADR-003](003_authority_of_declarations_over_inference.md) already says declarations over +inference. It was applied rigorously to the product and not at all across the repository +boundary, and that asymmetry is where the cost landed. + +**The evidence, because this is a rule bought with two days.** The source-reading check +broke twice in twenty-four hours — views-faoapi on 11 August, views-crafdapi on the 12th +— each time because that repository refactored a literal argument into a named constant. +Verified: commits `8615574` and `0c493ae` each introduce that constant *and* add that +repository's registry-binding test. **The improvement and the breakage were one edit.** +A mechanism a consumer breaks by improving itself will keep breaking. + +The corollary is the one that costs something to accept: when a fact is none of the three, +the honest move is to say so and register it, not to build a proxy. The delivery-label +composition is exactly that — see §8 and register C-92. + +### §7b An ADR records a decision, and never the current state of the code + +This document and [ADR-016](016_ci_read_access_to_private_siblings.md) both described what +the code did at the moment of writing — *"this is the shape the registry check now has"*, +*"today the check still reads the consumer's source"*. Every such sentence needed an +erratum the first time the code moved, and between them they produced nine stale claims in +two days, one of which repealed a rule in this document on a premise that was false by six +hours. + +> **An ADR states a decision, its reasoning, and what would reverse it. It does not +> describe the current implementation.** Where a reader needs to know what the code does +> now, the ADR names the test or module that answers, and the answer lives there. + +A decision record that doubles as a description of the code is a second copy of the code, +and it rots on a schedule nobody is watching. This clause is written into ADR-017 rather +than ADR-000 because it was learned here; if it survives contact with a second document it +belongs in the ADR conventions. + ### §8 What this does **not** verify, stated plainly This checks **our copy against the declaration**. It does **not** check the consumer's code diff --git a/reports/technical_risk_register.md b/reports/technical_risk_register.md index f2ebe2d..948621e 100644 --- a/reports/technical_risk_register.md +++ b/reports/technical_risk_register.md @@ -6,8 +6,8 @@ | Owner | Dylan Pinheiro / PRIO MD&D Team | | Last Updated | 2026-08-12 | | Total Concerns | 97 | -| Open Concerns | 24 | -| Resolved Concerns | 73 | +| Open Concerns | 23 | +| Resolved Concerns | 74 | --- @@ -247,29 +247,6 @@ Cross-refs: **C-92** (the check we deleted rather than replaced), **C-87** (the --- -### C-93: A mutation proof written by whoever wrote the guard tests that author's imagination, not the guard - -| Field | Value | -|-------|-------| -| ID | C-93 | -| Tier | 3 — no defect of its own. It is the reason several of the defects below survived three reviews, and it changes what "mutation-proven" is allowed to mean in this codebase. | -| Source | `/code-review max` on PR #239 post-merge, 2026-08-11, corroborated by measurement | -| Trigger | The next time a guard is defended in a pull-request description as "mutation-proven" against a list of cases the same change authored. | -| Owner | Whoever writes the next guard; the standard belongs in ADR-014 §2. | -| Location | ADR-014 §2; `tests/test_env_declaration.py::test_no_coordinate_value_is_copied_into_this_repo`; every `_MUTANTS`-style proof in `tests/`. | - -ADR-014 §2 says a guard is mutation-proven or it is decoration. That is right, and it is not sufficient. **A mutation proof is only as good as the mutant list, and a mutant list written by the author of the guard covers the cases the author already had in mind — which are, by construction, the cases the guard already handles.** - -Measured, on the guard that protects a public repository from publishing a coordinate value. Its author (this assistant) proposed thirteen input forms, proved all thirteen caught, and wrote that result into C-57 and into a commit message. An independent review then proposed twenty-nine forms. **Fifteen missed.** The thirteen were not a sample of how people write markdown; they were a sample of what the author had thought of, and every one of them happened to share the property the guard depended on. - -This is not the same as ADR-014 §2 failing. The proof was real, it was executed, and every case in it genuinely passed. The gap is that "proven against N mutations" reads as a statement about the guard when it is a statement about N. - -**What would change.** Two candidates, and they are not exclusive: (a) for a guard whose failure is silent — a leak, an invisible delivery — the mutant list is written or extended by something other than the change that wrote the guard; (b) the mutant list is a *declared, named* artifact in the test file rather than a paragraph in a commit message, so the next contributor can see what was and was not tried. `tests/test_ci_sibling_coverage.py::_MUTANTS` is already (b) and is the pattern to copy. - -Cross-refs: **C-57** and **C-89** (the guard this was measured on), **C-90** (a proof that proved nothing at all), ADR-014 §2. - ---- - ### C-89: The guard against publishing a coordinate value publishes it — in one branch of three, proven on one side of two | Field | Value | @@ -1080,6 +1057,38 @@ See also C-40 (the inheritance/representation coupling this migration unwinds), ## Resolved Concerns +### C-93: A mutation proof written by whoever wrote the guard tests that author's imagination, not the guard — RESOLVED + +| Field | Value | +|-------|-------| +| ID | C-93 | +| Tier | 3 — no defect of its own. It is the reason several of the defects below survived three reviews, and it changes what "mutation-proven" is allowed to mean in this codebase. | +| Source | `/code-review max` on PR #239 post-merge, 2026-08-11, corroborated by measurement | +| Trigger | The next time a guard is defended in a pull-request description as "mutation-proven" against a list of cases the same change authored. | +| Owner | Whoever writes the next guard; the standard belongs in ADR-014 §2. | +| Location | ADR-014 §2; `tests/test_env_declaration.py::test_no_coordinate_value_is_copied_into_this_repo`; every `_MUTANTS`-style proof in `tests/`. | + +ADR-014 §2 says a guard is mutation-proven or it is decoration. That is right, and it is not sufficient. **A mutation proof is only as good as the mutant list, and a mutant list written by the author of the guard covers the cases the author already had in mind — which are, by construction, the cases the guard already handles.** + +Measured, on the guard that protects a public repository from publishing a coordinate value. Its author (this assistant) proposed thirteen input forms, proved all thirteen caught, and wrote that result into C-57 and into a commit message. An independent review then proposed twenty-nine forms. **Fifteen missed.** The thirteen were not a sample of how people write markdown; they were a sample of what the author had thought of, and every one of them happened to share the property the guard depended on. + +This is not the same as ADR-014 §2 failing. The proof was real, it was executed, and every case in it genuinely passed. The gap is that "proven against N mutations" reads as a statement about the guard when it is a statement about N. + +**RESOLVED 2026-08-12 (#250) — ADR-014 §2 amended.** + +The rule landed is a **diagnostic**, not a process: *if a guard can only be proven against inputs you invented, that is the signal the guard is on the wrong side of a boundary — it is verifying a fact you do not own.* Prefer moving the check to where the fact lives, or anchoring the mutant list in something real — this repository's corpus, the registry's rows, an observable outcome. The no-copy scan's stopping rule is the worked example and shipped in #243. + +Two cheap obligations where that is impossible: the mutant list is a declared artifact in the test file, not a paragraph in a pull request; and a proof must be able to fail. + +**Deliberately not adopted:** requiring an independent mutant author for every guard. The friction would exceed the disease for a single maintainer. Independent mutants are worth buying only for the silent-failure class — a leak, an invisible delivery — and the arc that produced this entry is the evidence for both halves: five parallel reviewers found what four rounds of self-review had not, and that cost was proportionate exactly once. + +Cross-refs: **C-57** and **C-89** (the guard this was measured on), **C-90** (a proof that proved nothing at all), ADR-014 §2. + +--- + +--- + + ### C-82: Governance-artifact prose carries numbers and statuses that nothing checks — RESOLVED 2026-08-05 | Field | Value |