probe: TEMPORARY diff-coverage probe — do not merge - #28
Closed
wshallwshall wants to merge 1 commit into
Closed
Conversation
Four merged PRs surfaced the advisory quality gates, but the headline claim of that work — that diff-cover emits INLINE `::notice` annotations on the Files changed tab — has never been observed on a real diff. None of those PRs touched `messagefoundry/`, and `--cov` only measures that package, so every run correctly reported "no lines with coverage information in this diff". Reporting correctly that there was nothing to report is not the same as rendering. This supplies real changed lines under coverage: `probe_covered` is exercised by its test, `probe_uncovered` deliberately is not. A correct run annotates only the latter, which proves both that the surface renders and that it is scoped to genuinely uncovered changed lines rather than to the whole diff. Verified locally with the exact CI invocation before spending CI time: ::notice file=messagefoundry/_diffcov_probe.py,line=27,endLine=30,title=Missing Coverage ::notice file=messagefoundry/_diffcov_probe.py,line=32,endLine=34,title=Missing Coverage messagefoundry/_diffcov_probe.py (36.4%): Missing lines 27-30,32-34 Total: 11 lines / Missing: 7 Worth noting what that also settles: coverage.xml names files RELATIVE TO ITS `<source>` element (`filename="_diffcov_probe.py"`, source = .../messagefoundry), not repo-relative. The adversarial review of the liveness work flagged exactly this shape as a way the gate could go permanently "not applicable" — a source-path resolution mismatch. diff-cover resolves it correctly, and the annotation paths come out repo-relative, which is what GitHub needs to place them on the diff. DELETE BEFORE MERGE. This branch exists to observe the annotation and is not intended to land.
Collaborator
Author
|
Probe served its purpose — closing without merging, as intended. The open question from the quality-gate work is now settled by observation rather than inference. What that proves, beyond "it renders":
The probe module and its test are deleted with this branch. They were dead code in the shipped |
wshallwshall
added a commit
that referenced
this pull request
Aug 4, 2026
…ones (#163) * docs(backlog): close BACKLOG #226 — the estate Hybrid-layout sweep is done, off-repo The per-feed Hybrid split (connections.toml / <INBOUND>_router.py / <INBOUND>_handler.py / _<feed>_transforms.py) landed across the ported estate in the maintainer-internal migration repository. Owner-attested; nothing in this repository changes, which is also why leaving the item open could never have closed it. Both "Also" clauses are recorded as NOT delivered, with the reason each is not a residual of this item: - "align the IDE Corepoint-import / scaffold path to emit the Hybrid layout" — there is no Corepoint-import path in ide/ to align. That tooling is #105, still open, so the clause is a constraint on #105's design rather than work #226 can perform. The scaffold half is misaddressed too: Insert Element (#48) drops per-file idioms into the current buffer (ide/src/insertElement.ts:1-5) and emits no multi-file feed layout. - "consider a recursive-glob / folder-per-feed loader enhancement" — filed as a consider, and not taken: load_config still globs *.py non-recursively (config/wiring.py:4162), the flat-merge behaviour the Hybrid layout is built around. Follows the #227 precedent: close the primary, state the off-repo/misaddressed residuals explicitly so the item is not re-opened for them. backlog_status_check.py: OK — 277 items, each declaring exactly one status. * fix(ledger): teach the number-space gates to span an archive, and fix two holes found proving it Prerequisite for moving the 185 closed BACKLOG items into docs/archive/backlog/. No item has moved yet; this only makes the guards able to see one when it does. The item namespace will span two paths, so every guard now reads their UNION: - backlog_status_check.py: scan() takes (label, text) pairs and parses them as ONE namespace. A number re-used across BACKLOG.md and the archive was structurally undetectable before -- `seen` was per-parse -- which is the erratum's own shape. - ledger_check.py: triggers on any backlog-bearing path, not the one literal, and builds head/base as the union. Reading the union on both sides also removes a false positive: the move relocates 185 items, so head-union == base-union and `head - base` stays empty, where a per-file view would report 185 vanished numbers with a remedy that renumbers cited items. - alloc.ps1: sweeps both paths in the all-refs term and the working-tree term. - backlog-hygiene.yml: accepts a banner updated in either location. Two pre-existing defects surfaced only because the gates were made to fail on purpose first, neither of which is about the archive: 1. alloc.ps1's working-tree term has NEVER worked. `[regex]'^...'` anchors at the start of the STRING; the term feeds it `Get-Content -Raw`, one string starting "# Backlog". Measured: 0 of 277 headings matched without Multiline, 277 with. The all-refs term hid it by covering every number committed somewhere -- i.e. every case except the uncommitted one this term exists for. 2. backlog-hygiene.yml diffed BASE_SHA..HEAD_SHA (two-dot), which credits a PR for main-side changes to paths it never touched. One main-side edit to BACKLOG.md -- the move being a large one -- would let every PR with an older base pass the "must update BACKLOG.md" required check while enforcing nothing. Now three-dot, matching ci.yml's form for the same question. Anti-narrowing, because a green gate over a shrunken corpus is the failure mode: - `--min-items N` fails when fewer items are found than required, and CI pins 277. Without it, 277 -> 92 fails nothing. - The scanned files are always printed with the count; a bare integer cannot distinguish "items closed" from "a file stopped being read". - A liveness receipt in the test suite asserts the same floor. - An explicitly-named --backlog path that does not exist is an error, not a skip. alloc.ps1 gains `-ShowFloor`: print the floor and the swept paths, allocate nothing. Allocation is a one-way door, so before this the only way to ask what the floor could see was to spend a number on the question -- which is how it ran a whole release reading two refs while its header promised all of them. Get-Floor takes -Peek so the inspection cannot advance the high-water ratchet; the first -ShowFloor run against a planted number moved this clone's watermark 316 -> 990 before that was fixed. Proofs run, each observed failing BEFORE the fix: - archive-only unallocated #1007 staged: old gate rc=0, new gate BLOCKED. - #990 planted in the archive: old sweep floor 353 (blind), new sweep 990. - cross-file duplicate #118: detected, naming the other file. - banner violations inside the archive only: detected. - --min-items over a narrowed corpus: rc=1 with the scanned-file list. - -ShowFloor twice against a plant: watermark unchanged at 316. ruff + mypy --strict clean; 43 gate tests pass. * docs(backlog): move the 185 closed items into docs/archive/backlog/BACKLOG-CLOSED.md docs/BACKLOG.md becomes the ~92 items someone can act on: 8,742 -> 3,648 lines. The closed items are not deleted, summarised, or rewritten -- they are relocated verbatim, so the file that gets opened, grepped and edited daily is the open set. MOVED, NOT REWRITTEN. Every relocated block is byte-identical to the one that left BACKLOG.md, headings included. Verified mechanically against a pre-move copy: - 277 items before = 92 after + 185 archived, no overlap, union identical - every OPEN block byte-identical to its source - every ARCHIVED block byte-identical to its source - all non-item prose in BACKLOG.md preserved verbatim Byte-identical headings are load-bearing, not tidiness: GitHub derives anchor slugs from heading text, so all 64 archived->archived cross-references keep resolving with no edit at all. That is the whole argument for one archive file rather than a split by status, year, or cluster -- #52 alone receives 99 of the 110 in-file anchors, and its citers span #65 to #184, so no cut isolates them. Cutting item blocks at the next '## ' heading of EITHER kind, not the next numbered item: 4 blocks in this file are followed by a section header, which a naive cut would have dragged into the archive along with the prose beneath it. Anchors, all 127 re-resolved against real headings after the edit: - 44 rewritten in BACKLOG.md -> archive/backlog/BACKLOG-CLOSED.md#<same-slug> - 1 rewritten in the archive -> ../../BACKLOG.md#<same-slug> (#226 -> #105) - 3 cross-file links repointed: AOAG-DEPLOYMENT.md (#100, #101), ADR 0026 (#30) - 64 archived->archived untouched, by design 13 anchors still do not resolve, and ALL 13 WERE ALREADY DEAD BEFORE THIS COMMIT -- confirmed by running the same check over the pre-move file, which returns the identical multiset (11 bare-number self-anchors: #40 x4, #323 x3, #28, #29, #329, #333; plus 2 links to #13 in COUNSEL-ENGAGEMENT-BRIEF.md, a number this sequence never had). They are left dead and documented in the archive header rather than repointed at a plausible neighbour: a citation resolving to the WRONG item is the erratum's failure mode, and unlike a dead link it looks like success. The archive carries its retirement banner inline rather than in a sibling README -- docs/archive/throughput/ needs a README because it indexes five documents; one file does not, and two documents that must agree is a drift surface. It states the rules that keep the namespace honest: never renumber, re-open by moving the block back (never by copying, which creates the cross-file duplicate the status check now fails), and add any future archive file to alloc.ps1's $backlogPaths AND backlog_status_check.py's DEFAULT_SOURCES in the same commit -- a file named in neither is policed by nothing. Gates verified post-move: - backlog_status_check.py --min-items 277: OK, 277 items, and it now PRINTS "scanned: docs/BACKLOG.md (92), docs/archive/backlog/BACKLOG-CLOSED.md (185)" - ledger_check.py on the staged move: rc=0 (relocation adds no numbers, because head-union == base-union -- the exact false positive the union view removes) - alloc.ps1 -ShowFloor: floor 353 across both paths, next 1000 - 43 gate tests pass Note the floor is unchanged at 353 because the highest item (#353) is open and stays in BACKLOG.md. The archive-sweep fix is therefore PROSPECTIVE, not a save: it starts mattering the first time a top-of-range item closes and moves. * docs(backlog): re-score all 92 open items on the ten-level scale (2026-08-03) Every open item now carries a current value x difficulty score. Before this, 23 had none at all and the other 69 were from the frozen 2026-07-10 pass, which predates the 2026-07-28 reconcile that closed 31 items -- and a stale score reads exactly like a fresh one. Method, unchanged from the pass it supersedes: scored from each item's own Scope / Why / Trigger / Nearest-existing-mechanism text rather than rescaled from the old number, then adversarially verified against the code -- a second reader per batch attacking build state first, then verdict/tier, then value and difficulty. 26 of 92 scores were overturned by that pass and carry the refuter's number. The banner is the live record and the table is a view of it; both are written here and a mechanical check confirms 92 banners and 92 rows agree on every triple. THE RATIONALE IS REPLACED, NOT JUST THE NUMBERS. Carrying an old justification under a new score is how a banner comes to argue against itself: - #114's surviving "clean workaround via the on-demand test probe" is a claim PR #162 explicitly retracted -- both destinations' test_connection CREATE the target dir, so the probe cannot answer the question the toggle asks. That is what lifts it off the parity-with-a-workaround band to 6/3. Its replacement rationale was ALSO stale (it described the silent-ignore #162 had just fixed) and is hand-corrected. - #105's "large greenfield 71-action mapper needing its own ADR" describes an importer that has since shipped under ADR 0086. Scheduling barely moved, which is the reassuring result: only TWO tiers changed -- #64 DEMAND-GATE -> P3 (an index over levers that live in #62/#63/#47/#34, so it ships nothing runnable of its own) and #105 P3 -> DEMAND-GATE. Neither contradicts an explicit demand-gate/on-trigger ruling in its own body; that was checked for all 51 items carrying a prior tier. Distribution is RECOMPUTED with the table rather than carried forward, and all four lines sum to 92. The superseded table keeps its own frozen lines and now says so. Tiers: P1 4, P2 19, P3 17, DEMAND-GATE 52 Quadrants: quick win 22, big bet 5, fill-in 56, money pit 9 The four P1s: #341 (9/3, a handler returning a tuple/set of Sends delivers nothing silently -- an accept-and-drop CLAUDE.md §12 forbids), #324 (7/2), #325 (6/2), #327 (6/2). NOT in this commit: 24 items were found to misdescribe their own build state -- prose asserting a gap that has since shipped, or citing messagefoundry/console/, a package retired with #103. Those are banner corrections and land separately; the scores here already price the remainder rather than the original scope. Two mechanical faults were caught by reading the output rather than trusting the run: the quadrant regex omitted the hyphen in "fill-in", so 57 of 69 items took the fallback branch and got a SECOND score inserted beside the first; and the synthesizer's own distribution lines did not follow from its own table (11 quadrant mismatches, 8 ordering violations, difficulty summing to 95 of 92). The script now refuses to write when any line carries two score spans or the scored count is not 92. backlog_status_check.py --min-items 277: OK, 277 items across both files. * docs(backlog): correct 10 items whose own prose misdescribed build state The 2026-08-03 re-score flagged 24 open items as misdescribing what the code does. Re-verified each against the tree as it stands -- after the archive move and after PR #162, both of which post-date the findings -- and 10 survived. The other 14 did not, and are recorded here rather than silently dropped: #84 #95 #99 #105 #114 #124 #125 #127 #133 #137 #167 #169 #214 #228 Most of those already carry an amendment that covers the stale sentence (#95, #99, #105, #114, #124, #125, #127, #133, #228), and stacking a second ruling saying the same thing is noise. The rest did not survive verification: the finding was itself wrong or overstated, and a wrong correction in a ledger is worse than a stale one. CORRECTIONS ARE ADDED AS DATED AMENDMENTS, NOT PROSE REWRITES. This file's convention is to leave the original claim standing and rule against it, so the record shows what was believed and what replaced it. Silently editing the stale sentence would destroy the evidence that makes the correction checkable. Applied to #62 #64 #131 #166 #179 #182 #237 #321 #329 #336. Representative: - #329 "Five MEFOR_ALLOW_INSECURE_TLS cells": the census is FOUR. #323 landed and routed transports/direct.py through the clamp; it now holds no call to the raw predicate at all (:63, :197, :215). - #321 "no test asserts the detectors can see a site code": false -- tests/test_scan_forbidden.py has per-class hit tests for at least the site code (:126), a customer name (:83), a case-sensitive code (:91) and a routable IP (:107). The detector-coverage half of its Proposed 2 is already in the tree. - #62 plans a dual-read over "existing mfenc:v1 rows", but cell-bound mfenc:v2 is the default writer (settings.py:383 -> base.py:1841; crypto.py:36), and v2 folds (table, column, pk) into the GCM tag -- so a body landing under a different column must be RE-ENCRYPTED, not merely re-encoded. That tightens the catch. - #64's ordered plan still reads live ("Nothing builds before it"), but the measure-first phase completed 2026-07-12 (ADR 0051) and its step-2 lever is refused outright (ADR 0055 withdrawn; ADR 0107 "Do not build F2 or F3"). The refuters removed two overclaims before they landed: #62's draft asserted a live store holds both mfenc markers (a fresh store under the shipped default holds only v2 -- the defensible claim is that a MIGRATION must expect both), and #64's asserted the multi-DB log split still remains, which could not be verified against ADR 0098 and would have been a fresh false claim. No item closes here: in every case the correction narrows the remainder rather than discharging it, and the 2026-08-03 scores already price the remainder. backlog_status_check.py --min-items 277: OK, 277 items, one status banner each. * docs(backlog): file BACKLOG #1000 — prove each required merge context can fail Escalated by the coordinator on the ground that it outlives the PR that fixed it. Deliberately NOT filed as "fix the two-dot diff": that instance already landed in 39b62bf, and filing shipped work is the rot the hygiene gate exists to prevent. The item is the CLASS. `.github/required-contexts.txt` names 13 contexts that block merge, and not one of them is proven able to go red. The deliverable is a negative control per context -- a fixture carrying the exact violation that context exists to catch -- plus a CI job that fails when a required context has none, so the coverage cannot silently decay as contexts are added. Scoped narrower than "test the gates" on purpose: it does not re-test what each gate checks, since the gates' own suites do that. It asserts one property per context -- this gate is capable of failing. The argument is that the class has now fired at least four times here, each found by hand and none by CI: #334 semgrep, required and blocking, scans a two-directory allow-list #327 six .gitignore rules are the sole control over maintainer-internal docs, and nothing asserts they still match anything #321 the forbidden-content gate exited 0 on a real site code and partner product #325 the same gate's home-path detector misses 1 of 4 spellings of a Windows path Each is correctly filed as its own defect. None of them establishes the property that would have caught all four before they shipped, and that property is a different artifact from any of the individual fixes. Value 7 / Difficulty 3, quick win, P1 -- not demand-gated; the trigger fired four times. Ranked table and all four distribution lines recomputed to 93 open items; a mechanical check confirms 93 banners and 93 rows agree on every triple. Number allocated atomically via scripts/coord/alloc.ps1 (#1000 -- the first in the post-partition public sequence, clamped to >= PUBLIC_BACKLOG_FLOOR), never grepped. backlog_status_check.py --min-items 277: OK, 278 items across both files. The floor is a floor, so growth passes it; it is there to catch shrinkage.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Four merged PRs (#18,
#19,
#25,
#27) built and hardened the advisory quality
gates. The headline claim of that work — that diff-cover emits inline
::noticeannotations onthe Files changed tab — has never actually been observed. None of those PRs touched
messagefoundry/, and--covonly measures that package, so every run correctly reported "no lineswith coverage information in this diff".
Reporting correctly that there was nothing to report is not the same as rendering.
What this probe does
probe_coveredis exercised by its test;probe_uncovereddeliberately is not. A correct runannotates only the latter — proving both that the surface renders, and that it is scoped to
genuinely uncovered changed lines rather than to the whole diff.
Verified locally first, with the exact CI invocation
Two coalesced ranges, repo-relative paths, and
probe_covered's lines correctly excluded.A side finding worth recording
coverage.xmlnames files relative to its<source>element —filename="_diffcov_probe.py"with
<source>…/messagefoundry, not repo-relative. The adversarial review of #25 flagged exactlythis shape as a way the coverage gate could go permanently "not applicable": a source-path
resolution mismatch would leave
src_statsempty for every diff. diff-cover resolves it correctly andemits repo-relative paths, which is what GitHub needs to place an annotation. That is now observed
rather than assumed.
After observation
The probe module and its test are deleted and this PR closed. Nothing here is intended for
main.