ci(asvs): port the --prove-absences wiring onto current main, and make docs/CI.md honest about the vault mirror (supersedes #304) - #350
Merged
Conversation
… semantic re-read) PR #304 was 145+ commits behind main, with both sides having rewritten prove_absences and _run_prove_absences and main having moved the shared types underneath them. This is a SEMANTIC port, written against main's current code -- not a merge, rebase or cherry-pick, because git conflicts on concurrent edits and is SILENT on invalidated claims, so a textual resolve would have produced working code with false documentation beside it. THE INVALIDATION THAT MADE IT UN-MECHANICAL. main's Findings carries a documented accounting identity (checked_absences - proved_absences - static_screened - skipped_absences == claims that ended in a problem-only branch) and asserted "FIVE outcomes raise a problem and increment no counter", enumerated. #304 adds more problem-only branches. Nothing anywhere ASSERTED the identity -- zero references to static_screened outside scorecard.py -- so the prose would have gone stale silently, ~500 lines from the hunks that broke it. Resolved by making a refusal PROBLEM-ONLY: it appends a PROVE-ERROR and increments no counter, so the identity still closes. That was chosen over two alternatives and the reasoning is in the code -- folding a refusal into static_screened would be a lie about kind, since that counter means "this claim's evidence is a screen rather than a proof" and a refused claim carries no evidence at all; giving refusals their own counter would be arbitrary, since every problem-only branch has an equal claim to one and the PROBLEM is the record. The brittle enumeration is GONE, replaced by test_prove_absences_counters_close_against_the_population, which computes BOTH sides from a real run and DERIVES the right side from `problems` rather than asserting a hand-chosen constant. The test it replaces asserted remainder == 1 against a fixture: true of that fixture, silent about the identity -- which is exactly how three problem-only branches could be added while the prose enumerating them went stale and nothing moved. Ported additively: prove_report.py, the scheduled advisory workflow, the wiring test, and the mutation arity screens (main has no counterpart -- _toplevel/_signature/ast.arguments all return zero on main). The one-pristine-scratch-tree rework is kept with its VERIFIED restore, because a restore that silently did not happen makes every later claim a fact about the previous claim's mutation. NOT YET VERIFIED beyond ruff and a parse/run check -- the adversarial and quartet passes were stopped early to bank the work under a usage ceiling. See the handoff for what remains.
…ero file overlap The port touches 11 files, main's 11 new commits touch 21, and the two sets are DISJOINT, so the merge is textually clean. That is exactly the state in which a clean merge is least informative: the port's changes include repo-wide static scans (test_security_static, test_key_usage_scope_inventory, test_asvs_verifier_vault_contract) and main added new modules (api/_ui_seam.py, uploads changes) plus BACKLOG.md rows -- so the verification quartet must run on the MERGED tree, never on the pre-merge tip.
…stop a comment anchor rotting Verification pass over the --prove-absences semantic port (PR #304, ported at af1b1d9, merged with current main at 1121539). Twenty findings were raised across six dimensions and fifteen were refuted; five survived, reducing to five distinct issues. This commit closes three of them. The two it leaves are recorded at the end, with why. BASELINE CACHE KEYING -- the false proof. A claim whose observable is already red on the pristine tree must be refused, because a red that was already there cannot be attributed to the mutation, and that refusal is reached ONLY through the claim's own baseline. So coarsening the cache key does not merely slow the prover down: it manufactures a proof. Measured, not argued -- re-keying `baselines` to a constant makes prove_absences report proved_absences=2 with problems=[], an already-failing test booked as proof of a control that never fired, and NOTHING printed. Under that same injection 159 other tests across the two ASVS suites stayed GREEN, which is what the new test is worth. Its fixture is one cell carrying two claims that share a mutation_path and differ only in observable, so it also reddens a cell_id key (_live_claim hardcodes 1.1.1, so cell id separates no live claims at all) and a mutation_path key. Only the node id separates them, which is the property being pinned. THE INNER LOOP OVER A CELL'S CLAIMS. #304's test_the_prove_summary_reports_the_total_it_saw was the only test driving prove_absences over a cell carrying more than one absence claim, and the port dropped it without mentioning it. Two of its three properties were genuinely superseded, one of them by a stronger guard main already had; this third was not. Every other `absence=` tuple in the file has length 1, so `checked_absences` counting CLAIMS was indistinguishable from it counting CELLS. Cell 1.1.2 in the identity test now carries two. Measured: degenerating the loop to `c.absence[:1]` reddens it at 6 != 7 while 158 other tests stay green. Both of 1.1.2's claims land in a COUNTED branch deliberately -- the identity test derives its right-hand side by set-dedupe on cell id, so a cell carrying a second PROBLEM-only claim would break that derivation silently, which is the exact class of failure this test exists to catch. A DECAYED ANCHOR. prove_report.py cited "the loader at scorecard.py:339-340". That resolved exactly in #304 and resolves to unrelated docstring prose here, because the port moved the loader to :544-545 -- so it was already stale at af1b1d9 and the merge did not cause it. Replaced with the function NAME rather than corrected line numbers: nothing in this repo checks a line citation inside a Python comment, so a corrected anchor decays again on the next move. NOT FIXED HERE, deliberately. (1) _prove_one probes `target_in_root.is_file()` against root but reads `target_in_scratch.read_bytes()` from the scratch copy, and _scratch_ignore is what makes those trees differ, so a claim naming docs/security, .env*, or *.db would pass the probe and then die on an unhandled FileNotFoundError. It is inherited from BOTH parents rather than introduced here, and the shipped workflow roots the prover at engine/ where none of those paths exist, so it would surface only in an ad-hoc local run -- and it fails loudly, which prove_report already classifies as an instrument failure. It belongs in its own change against main, not folded into a port. (2) The tree-rebuild guard has no test. Both are in the handoff. scripts/asvs/scorecard.py is deliberately UNCHANGED by this commit. Four injections were applied to it while measuring the above; each was reverted and confirmed byte-identical to HEAD with `git status` before the next measurement, because an unverified restore turns every later measurement into a fact about the previous one's mutation.
…mail TTL Two commits, disjoint from the port's eleven files again. Merged rather than left behind because the port edits repo-wide static scans and one of the two touches docs/BACKLOG.md, which two tests read from disk -- so a docs-only base move is not test-neutral here and the verification has to be a statement about the tree that will actually land.
… redaction 5f7eb55 touches three files (docs/BACKLOG.md, messagefoundry/config/wiring.py, tests/test_connection_factory_redaction_domain.py), none of them among the port's eleven, so the merge is textually clean. Re-verified rather than assumed on that basis: the port edits REPO-WIDE static scans (test_security_static, test_key_usage_scope_inventory, crypto_inventory_check), and wiring.py is exactly the kind of file those scans read -- a clean merge says nothing about whether a scan now has something new to say.
…chanism delivers The row this port added said prove_report.py "is mirrored into the vault on the same footing as scorecard.py, which MIRRORED_TOOLS pins". Measured against the vault: its mirror automation is scoped to the literal path scripts/asvs/scorecard.py, repeated in four job envs, and copies exactly that one file; the vault holds no copy of prove_report.py and no workflow referencing it. So the sentence would have been false on the day it landed. Worse than false, it cited a test as its evidence. MIRRORED_TOOLS asserts the listed tool imports only the standard library -- it says nothing about whether a vault copy exists. A reader would have taken the citation as proof of a mirroring relationship that nothing checks and nothing performs. That is the compensating-control-on-a-false-premise shape SDS-3.7 names, and CLAUDE.md section 11's test for security prose is what a reader would DO with it, not whether it parses as accurate. The replacement states the contract that IS in force (stdlib-only, so a vault copy would run on the bare interpreter there), says plainly that no vault copy exists, and blocks the misreading in the sentence itself rather than leaving it to inference. Widening the vault's automation is named as the open half instead of being implied to be done. The claim was inherited from PR #304 rather than invented here; this port made it slightly worse by adding the MIRRORED_TOOLS citation, so it is fixed here. The repo's own ratchet caught two drafts of this text. test_cutover_slug_rot's _PROSE pattern matches the literal "the mirror" with no trailing word boundary, so "the mirrored-tool contract" and then "never the mirroring" each tripped it -- and because a table row is one physical line, the row carries no retrospective marker to exempt it. Reworded both times; the ceiling was NOT raised and that file is untouched. This is the guard the MIRRORED_TOOLS comment in test_asvs_verifier_vault_contract.py already warned would ratchet on this exact phrasing. Verified: 201 tests across the nine files that read docs/CI.md or scan repo prose, plus a direct scan of the file with the ratchet's own two patterns -- zero hits, with a positive control confirming the detector still fires on a known-bad string.
236a1ea is docs/BACKLOG.md only, disjoint from the port's files. Not treated as test-neutral on that basis: several tests read the live BACKLOG.md from disk, so a ledger-only base move can still move a test result.
…w reorder Neither commit touches the port's eleven files, but e5bd07d edits .github/workflows/ci.yml and this port ADDS a workflow -- and tests/_workflow_contexts.py, test_required_contexts.py and test_merge_gate_controls.py all read the workflow inventory. That is the interaction a clean merge cannot speak to, so it is checked rather than assumed.
wshallwshall
enabled auto-merge (squash)
August 12, 2026 15:49
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Supersedes #304. A semantic re-read onto current
main, not a merge of it --mainrewrote theprove_absencesfamily under #303, so merging #304 would have reapplied a stale implementation of code that no longer exists in that form, and git would have raised nothing, because it conflicts on concurrent edits and never on obsolete intent.DELIBERATELY NOT ARMED. See the flag below before arming.
Verification (the authoring session's, labelled as theirs)
Full suite 11944 passed / 855 skipped / 0 failed; ruff format/check clean; actionlint clean on the added workflow. A 28-agent adversarial pass: 20 findings, 15 refuted, verdict safe to land. Two silent failure modes were found and fixed -- a coarsened baseline cache key that would have manufactured a false proof, and a dropped #304 test that left the prover's multi-claim loop uncovered. Each hole was proved real by running the rest of the suite under the injection (159 and 158 other tests stayed green), which is what distinguishes a fixed hole from an asserted one.
Verified here before pushing
Tip
2f9aabd8, genuinely unpushed, 0 behindorigin/main,merge-treerc=0 read with no pipe, 11 files. Leak scan: 1734 added lines scanned, and the onlydocs/securityhits are paths, not content -- confirmed no vault document content is added.THE FLAG -- why this is not armed
This PR introduces a pattern that does not exist on
main: a public-repo workflow that sparse-checks-out the private vault. Measured --mainhas zero workflows referencing vaultdocs/securitypaths and zero vault sparse-checkouts. So this is a new cross-repo coupling in the public repo, not an extension of an existing one.Mitigating, and stated so the flag is not overstated: no vault content is committed; the
provejob isworkflow_dispatchonly; and it fails closed with exit 2 when no input is configured, so it is inert without deliberate setup. But a coupled engine/vault pair is the case that wants the owner present for both halves, so the arming decision is his.Also in here, and worth reading on its own
bb825e9cmakesdocs/CI.mdhonest: it previously asserted thatprove_report.py"is mirrored into the vault on the same footing asscorecard.py". Independently confirmed false -- the vault'sasvs-verifier-drift.ymlhard-wiresFILE: scripts/asvs/scorecard.pyat four sites and mentionsprove_reportnowhere. The test it cited as pinning that claim is honest and pins something else (stdlib-only imports); the doc was the defect. Widening the vault's mirror to actually carryprove_report.pyis named as the open half rather than assumed.#1203 is deliberately absent
#304 filed a ledger row for #1203; this port does not carry it, and its
docs/CI.mddeliberately does not cite it. That is correct:#1203is allocated to another worktree so the ledger gate refuses it from anywhere else, and the vault cites#1203in prose written 2026-07-24 -- sixteen days before the number was allocated -- describing unrelated content. Today that citation resolves to nothing; the day #1203 lands it would resolve to a real item saying something else, which is worse than dangling because it looks correct.