From 605ba7d92ba903b1129667440be70452e6f4944c Mon Sep 17 00:00:00 2001 From: wshallwshall Date: Thu, 13 Aug 2026 09:00:38 -0500 Subject: [PATCH] backlog: file #1244 -- a broken vault anchor changes no colour, so nobody sees it The scorecard lives in the vault and cites engine code: 1,998 evidence anchors name engine paths and 192 of them cite engine prose under docs/, the kind of line an ordinary documentation edit rewrites without anyone thinking about security evidence. Nothing engine-side references the scorecard and nothing can, because the data is what must stay private. Filed narrower than proposed, and the narrowing is the finding. The premise "nothing checks that the two repos agree" is FALSE: the vault gate's own trigger block documents this hazard, records a live instance where an engine PR moved eleven anchors while the gate stayed green, and a daily run was added to close it. The detector exists and works. What remains is that it cannot block pre-merge, cannot attribute a break to an engine change, and -- the real defect -- lands in a gate that is ALREADY RED at eleven failures and is not a required context. A twelfth failure moves no colour, no mergeability and no notification, so it is indistinguishable from yesterday's run. Detection that cannot change state is not detection anybody sees, which is why the live instance was caught by a person reading a diff. The fix has a constraint that makes the obvious build wrong: an engine-side check runs in a PUBLIC repository with public logs, so it must report the file and line that stopped matching and must not name the cell or the requirement id. A failing public log that enumerates those hands out coverage by subtraction. Report the line, never the cell. The item records its own framing error rather than presenting the narrow version as the original intent. It was first proposed as one item covering a whole class, generalised from three sightings in one evening. A class has no prove-a-fix so it could never be closed, and bundling limbs with different fixes is what splitting #1236 out of #1131 existed to prevent. Two of the three sightings did not survive: one had resolved itself, and the third was carried second-hand, never opened, and on re-reading was a different mechanism entirely. Counts asserted either side of the edit: 274 -> 275 items, open 200 -> 201, closed 74 -> 74. Exactly one new OPEN item and no status moved, which is the check that catches a glyph being read as a banner. --- docs/BACKLOG.md | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/docs/BACKLOG.md b/docs/BACKLOG.md index 9f6a79f6..25330d7c 100644 --- a/docs/BACKLOG.md +++ b/docs/BACKLOG.md @@ -8479,3 +8479,34 @@ _FHIR_ID_RE.fullmatch("abc\n") -> False the fix > **BOUND ON WHAT WAS VERIFIED.** The control flow above was read statically at `origin/main` and each cited line confirmed individually; it has **not** been reproduced by executing a login sequence. A test that resets the password of a claimed `admin` account on a two-administrator system and then asserts the temporary credential works is the confirming experiment, and it does not exist today. **Cluster:** Authentication / account lifecycle. **Priority:** P2. **Verdict:** build. **Severity:** no deployment axis -- zero instances; on first deployment an administrator following the documented reset **would** render the `admin` account permanently unusable without either party being told. + +## 1244. an engine change that breaks a vault ASVS anchor produces no state change and no attribution: the signal is real, late, and lands in a gate that is already red + +> 🔢 **Filed 2026-08-13. NOT "nothing detects this" -- that framing was wrong and is recorded below so nobody re-derives it.** Value **6/10** -- Difficulty **4/10**. The detection already exists and works; what is missing is that it cannot **block**, cannot **attribute**, and currently cannot **change colour**. + +> **THE COUPLING.** The scorecard lives in the vault (private); the code it cites lives in the engine (public). Every one of the **1,998** evidence anchors names an engine path, and **192** of them cite engine *prose* under `docs/` -- the kind of line an ordinary documentation edit rewrites without anyone thinking about security evidence. Nothing in the engine repo references the scorecard, and nothing can: **the engine tree contains the verifier (`scripts/asvs/scorecard.py`) but zero scorecard data files**, by deliberate design, because the data is what must stay private. + +> **WHAT ALREADY WORKS, and it must not be rebuilt.** The vault gate's own trigger block documents this exact hazard and already fixed the worst of it: *"the anchors point into the ENGINE, but the triggers above all fire on VAULT events... Verified live: #119 landed in the engine and moved eleven anchors; this gate stayed green because nothing here had changed."* A daily `schedule:` run was added precisely to close that, and it does. **Any fix here starts from a working daily detector.** + +> **THE THREE GAPS THAT REMAIN, in ascending order of how badly they bite:** +> ``` +> 1. NOT PRE-MERGE an engine PR merges; the break is found on the next daily run. +> The engine PR is where the change was still reviewable. +> 2. NOT ATTRIBUTED the run reports a broken anchor, not which engine change broke +> it. Whoever reads it bisects a day of engine history by hand. +> 3. NO STATE CHANGE the gate is ALREADY RED -- 11 FAIL rows on the current run -- +> and `verify` is NOT a required context. So a 12th failure moves +> nothing anyone watches: not the check's colour, not +> mergeability, not a notification. +> ``` +> **Gap 3 is the real defect.** A detector whose output is a count inside an already-failing, non-blocking job is indistinguishable from the same job yesterday. That is why the instance below was caught by **a person reading a diff**, not by the instrument that exists to catch it. + +> **THE INSTANCE THAT PROMPTED THIS.** Engine PR #361 deletes a sentence from `docs/SECURITY.md` that an anchor pins verbatim. The break was found by a session reading the PR's diff and reasoning about the vault; the retirement was prepared by hand as a separate vault change. Had nobody looked, #361 would have merged and the count would have gone 11 to 12 on a schedule, hours later, attributed to nothing. + +> **A CONSTRAINT ON THE FIX THAT IS EASY TO MISS, and it is why the obvious build is wrong.** An engine-side check runs in a **public** repository with public logs. It must report **the engine file and line that stopped matching** and must **not** name the cell, the requirement id, or the count of affected cells -- those are the vaulted content, and a failing public CI log enumerating them hands out coverage by subtraction (CLAUDE.md section 12). "Report the line, never the cell" is a hard requirement of any design here, not a preference. This is what makes the item non-trivial: the natural implementation leaks. + +> **How to prove a fix:** in an engine PR, delete a line that an anchor pins verbatim, and assert the PR goes red **before merge**, naming the engine file and line and **naming no cell id**. Then assert the negative control: a PR touching an unanchored line stays green, so the check is discriminating rather than always-red. A fix that only shortens the schedule addresses gap 1 and neither of the others; a fix that makes `verify` required **without first closing the standing 11 failures** converts a silent problem into a permanently blocked repository. + +> ⚠️ **Recording the framing error, because it was mine and it was nearly filed.** This was first proposed as one item covering a whole class -- *"two artifacts that must agree with nothing noticing when they stop"* -- generalised from three sightings in one evening. That was wrong twice over. **A class has no prove-a-fix**, so it could never be closed, and bundling limbs with different fixes is exactly what splitting #1236 out of #1131 existed to prevent. And the premise did not survive contact: the detector exists, a daily run already covers the case, and only the blocking, attribution and visibility limbs are real. Two of the three original sightings did not survive at all -- one had resolved itself correctly, and the third was carried second-hand and never verified. On re-reading it was a **different mechanism** anyway (one instrument answering a narrower question than its declared scope, rather than two artifacts drifting apart); they share a smell, not a cause. + +**Cluster:** Security tooling / CI. **Priority:** P2. **Verdict:** build. **Severity:** no deployment axis -- this is evidence-integrity plumbing across two repositories, not shipped engine behaviour. The exposure is that the ASVS record can quietly describe code that no longer exists, which is a **stale anchor** (the evidence went stale), never an engine weakness.