fix(tests): the spreadsheet-writer gate scanned a sibling worktree's entire checkout - #29
Merged
Merged
Conversation
…entire checkout
Running the full suite on main failed with writers "outside the ASVS 1.2.10 gate" at paths under
`.claude/worktrees/mefor-jdbc-support-c275bd/...` -- another session's working copy.
docs/WORKTREES.md places sibling worktrees at `.claude/worktrees/<name>/`, NESTED INSIDE this checkout.
This gate walks `_REPO.rglob("*.py")` and its skip set covered `.venv`, `.git`, `node_modules`,
`__pycache__` and the two caches -- but not `.claude`. So with a parallel session active it scanned a
SECOND FULL COPY of the repo, 5,712 extra .py files, AT A DIFFERENT COMMIT.
CI never sees it (no nested worktrees there), so this reds only the local run -- which is where people
iterate, and the natural response is to learn that this test "just fails sometimes". Then it guards
nothing, which is the actual cost.
Both directions are wrong, and the one that did NOT happen is the worse one: a gate searching for a
REQUIRED registration could find it in the other worktree and pass while this checkout lacks it. This
gate only fails-loud, but any future repo-root scan inherits the hazard.
Added a guard so the next repo-root walker cannot reintroduce it, mutation-verified: remove `.claude`
from the skip set and it fails.
SIZING, CORRECTED MID-INVESTIGATION. I first measured "13 test files use rglob" and was about to treat
all 13 as affected. They mostly walk NARROW roots -- `messagefoundry/`, `messagefoundry/auth/`,
`samples/messages/` -- which cannot reach `.claude/`. Exactly ONE walks from the repo root.
test_ldap_timeouts.py looked like a second (`root = Path(__file__).resolve().parent.parent`) but uses
that root to read one specific file, not to walk. The question is not "who calls rglob" but "who walks
from the repo root", and those give very different answers.
The new test's two assertions differ in strength and the docstring says so: the `_SKIP_DIRS` membership
check is always-on and is what the mutation confirms; the "no scanned path under .claude/" check only
has teeth in a tree that HAS a nested worktree, and passes trivially in CI or a sibling worktree. A
belt-and-braces assertion is fine; quietly presenting it as evidence would not be.
Also seen in the same run and NOT a repo defect: test_version::test_installed_metadata_matches_dunder_version
fails locally because this venv's editable install still registers 0.3.0 while __init__ is 0.3.2 after
today's releases. `pip install -e .` refreshes it. Left alone.
Collaborator
Author
|
Heads-up: this gate's The filter tested The docstring here reasons from the main checkout and concludes "in a sibling worktree (or in CI) #33 judges the exclusion relative to the repo root, which holds in both directions: from the main Credit: the No action needed unless you disagree with the fix; #33 is armed to merge. |
wshallwshall
added a commit
that referenced
this pull request
Jul 29, 2026
… not the absolute path (#33) The spreadsheet-writer gate excludes `.claude/` so a repo-root walk cannot wander into a sibling worktree's checkout (#29). It tested `_SKIP_DIRS & set(path.parts)` on the ABSOLUTE path -- and docs/WORKTREES.md puts sibling worktrees at `.claude/worktrees/<name>/`, so when the suite runs FROM one of them the checkout ITSELF sits under `.claude/`. Every absolute path in the repo then contains `.claude`, the filter matches everything, and the walk collapses. Measured in a worktree: 5712 .py files found, 0 kept. Both tests red -- every recorded spreadsheet writer read as "no longer exists", because nothing was scanned to find them. The inversion is the subtle part. #29's docstring reasons about the main checkout and concludes that "in a sibling worktree (or in CI) there is nothing under `.claude/` to find, so it passes trivially". From a worktree the opposite is true: the filter matches EVERYTHING. A guard written to stop a scan leaking INTO `.claude/` blinded itself completely when run FROM there, and CI never sees it because CI has no nested worktrees -- which is exactly the local-only redness that docstring warns turns a gate into something people learn to ignore. Judged relative to the repo root, both directions hold: from the main checkout a sibling worktree's file is `.claude/worktrees/<x>/foo.py` and is still excluded (#29's actual purpose, preserved); from inside a worktree the same file is `harness/foo.py` and is kept. Verified both. Three call sites shared the defect and are now one helper (`_is_skipped`), including the `leaked` assertion -- fixing only the scan would have inverted that one instead, since it also matched on absolute parts and would have flagged every correctly-kept file as leaked. Credit where it is due: #29 shipped a non-vacuity assertion on its own gate (`len(scanned) > 500`, "the walk collapsed, so a pass proves nothing") and that is what named the failure precisely. The gate caught its own blindness. Added a regression test asserting BOTH directions, because neither is observable from the other and CI only ever exercises one of them.
wshallwshall
added a commit
that referenced
this pull request
Aug 4, 2026
…ones (#163) * docs(backlog): close BACKLOG #226 — the estate Hybrid-layout sweep is done, off-repo The per-feed Hybrid split (connections.toml / <INBOUND>_router.py / <INBOUND>_handler.py / _<feed>_transforms.py) landed across the ported estate in the maintainer-internal migration repository. Owner-attested; nothing in this repository changes, which is also why leaving the item open could never have closed it. Both "Also" clauses are recorded as NOT delivered, with the reason each is not a residual of this item: - "align the IDE Corepoint-import / scaffold path to emit the Hybrid layout" — there is no Corepoint-import path in ide/ to align. That tooling is #105, still open, so the clause is a constraint on #105's design rather than work #226 can perform. The scaffold half is misaddressed too: Insert Element (#48) drops per-file idioms into the current buffer (ide/src/insertElement.ts:1-5) and emits no multi-file feed layout. - "consider a recursive-glob / folder-per-feed loader enhancement" — filed as a consider, and not taken: load_config still globs *.py non-recursively (config/wiring.py:4162), the flat-merge behaviour the Hybrid layout is built around. Follows the #227 precedent: close the primary, state the off-repo/misaddressed residuals explicitly so the item is not re-opened for them. backlog_status_check.py: OK — 277 items, each declaring exactly one status. * fix(ledger): teach the number-space gates to span an archive, and fix two holes found proving it Prerequisite for moving the 185 closed BACKLOG items into docs/archive/backlog/. No item has moved yet; this only makes the guards able to see one when it does. The item namespace will span two paths, so every guard now reads their UNION: - backlog_status_check.py: scan() takes (label, text) pairs and parses them as ONE namespace. A number re-used across BACKLOG.md and the archive was structurally undetectable before -- `seen` was per-parse -- which is the erratum's own shape. - ledger_check.py: triggers on any backlog-bearing path, not the one literal, and builds head/base as the union. Reading the union on both sides also removes a false positive: the move relocates 185 items, so head-union == base-union and `head - base` stays empty, where a per-file view would report 185 vanished numbers with a remedy that renumbers cited items. - alloc.ps1: sweeps both paths in the all-refs term and the working-tree term. - backlog-hygiene.yml: accepts a banner updated in either location. Two pre-existing defects surfaced only because the gates were made to fail on purpose first, neither of which is about the archive: 1. alloc.ps1's working-tree term has NEVER worked. `[regex]'^...'` anchors at the start of the STRING; the term feeds it `Get-Content -Raw`, one string starting "# Backlog". Measured: 0 of 277 headings matched without Multiline, 277 with. The all-refs term hid it by covering every number committed somewhere -- i.e. every case except the uncommitted one this term exists for. 2. backlog-hygiene.yml diffed BASE_SHA..HEAD_SHA (two-dot), which credits a PR for main-side changes to paths it never touched. One main-side edit to BACKLOG.md -- the move being a large one -- would let every PR with an older base pass the "must update BACKLOG.md" required check while enforcing nothing. Now three-dot, matching ci.yml's form for the same question. Anti-narrowing, because a green gate over a shrunken corpus is the failure mode: - `--min-items N` fails when fewer items are found than required, and CI pins 277. Without it, 277 -> 92 fails nothing. - The scanned files are always printed with the count; a bare integer cannot distinguish "items closed" from "a file stopped being read". - A liveness receipt in the test suite asserts the same floor. - An explicitly-named --backlog path that does not exist is an error, not a skip. alloc.ps1 gains `-ShowFloor`: print the floor and the swept paths, allocate nothing. Allocation is a one-way door, so before this the only way to ask what the floor could see was to spend a number on the question -- which is how it ran a whole release reading two refs while its header promised all of them. Get-Floor takes -Peek so the inspection cannot advance the high-water ratchet; the first -ShowFloor run against a planted number moved this clone's watermark 316 -> 990 before that was fixed. Proofs run, each observed failing BEFORE the fix: - archive-only unallocated #1007 staged: old gate rc=0, new gate BLOCKED. - #990 planted in the archive: old sweep floor 353 (blind), new sweep 990. - cross-file duplicate #118: detected, naming the other file. - banner violations inside the archive only: detected. - --min-items over a narrowed corpus: rc=1 with the scanned-file list. - -ShowFloor twice against a plant: watermark unchanged at 316. ruff + mypy --strict clean; 43 gate tests pass. * docs(backlog): move the 185 closed items into docs/archive/backlog/BACKLOG-CLOSED.md docs/BACKLOG.md becomes the ~92 items someone can act on: 8,742 -> 3,648 lines. The closed items are not deleted, summarised, or rewritten -- they are relocated verbatim, so the file that gets opened, grepped and edited daily is the open set. MOVED, NOT REWRITTEN. Every relocated block is byte-identical to the one that left BACKLOG.md, headings included. Verified mechanically against a pre-move copy: - 277 items before = 92 after + 185 archived, no overlap, union identical - every OPEN block byte-identical to its source - every ARCHIVED block byte-identical to its source - all non-item prose in BACKLOG.md preserved verbatim Byte-identical headings are load-bearing, not tidiness: GitHub derives anchor slugs from heading text, so all 64 archived->archived cross-references keep resolving with no edit at all. That is the whole argument for one archive file rather than a split by status, year, or cluster -- #52 alone receives 99 of the 110 in-file anchors, and its citers span #65 to #184, so no cut isolates them. Cutting item blocks at the next '## ' heading of EITHER kind, not the next numbered item: 4 blocks in this file are followed by a section header, which a naive cut would have dragged into the archive along with the prose beneath it. Anchors, all 127 re-resolved against real headings after the edit: - 44 rewritten in BACKLOG.md -> archive/backlog/BACKLOG-CLOSED.md#<same-slug> - 1 rewritten in the archive -> ../../BACKLOG.md#<same-slug> (#226 -> #105) - 3 cross-file links repointed: AOAG-DEPLOYMENT.md (#100, #101), ADR 0026 (#30) - 64 archived->archived untouched, by design 13 anchors still do not resolve, and ALL 13 WERE ALREADY DEAD BEFORE THIS COMMIT -- confirmed by running the same check over the pre-move file, which returns the identical multiset (11 bare-number self-anchors: #40 x4, #323 x3, #28, #29, #329, #333; plus 2 links to #13 in COUNSEL-ENGAGEMENT-BRIEF.md, a number this sequence never had). They are left dead and documented in the archive header rather than repointed at a plausible neighbour: a citation resolving to the WRONG item is the erratum's failure mode, and unlike a dead link it looks like success. The archive carries its retirement banner inline rather than in a sibling README -- docs/archive/throughput/ needs a README because it indexes five documents; one file does not, and two documents that must agree is a drift surface. It states the rules that keep the namespace honest: never renumber, re-open by moving the block back (never by copying, which creates the cross-file duplicate the status check now fails), and add any future archive file to alloc.ps1's $backlogPaths AND backlog_status_check.py's DEFAULT_SOURCES in the same commit -- a file named in neither is policed by nothing. Gates verified post-move: - backlog_status_check.py --min-items 277: OK, 277 items, and it now PRINTS "scanned: docs/BACKLOG.md (92), docs/archive/backlog/BACKLOG-CLOSED.md (185)" - ledger_check.py on the staged move: rc=0 (relocation adds no numbers, because head-union == base-union -- the exact false positive the union view removes) - alloc.ps1 -ShowFloor: floor 353 across both paths, next 1000 - 43 gate tests pass Note the floor is unchanged at 353 because the highest item (#353) is open and stays in BACKLOG.md. The archive-sweep fix is therefore PROSPECTIVE, not a save: it starts mattering the first time a top-of-range item closes and moves. * docs(backlog): re-score all 92 open items on the ten-level scale (2026-08-03) Every open item now carries a current value x difficulty score. Before this, 23 had none at all and the other 69 were from the frozen 2026-07-10 pass, which predates the 2026-07-28 reconcile that closed 31 items -- and a stale score reads exactly like a fresh one. Method, unchanged from the pass it supersedes: scored from each item's own Scope / Why / Trigger / Nearest-existing-mechanism text rather than rescaled from the old number, then adversarially verified against the code -- a second reader per batch attacking build state first, then verdict/tier, then value and difficulty. 26 of 92 scores were overturned by that pass and carry the refuter's number. The banner is the live record and the table is a view of it; both are written here and a mechanical check confirms 92 banners and 92 rows agree on every triple. THE RATIONALE IS REPLACED, NOT JUST THE NUMBERS. Carrying an old justification under a new score is how a banner comes to argue against itself: - #114's surviving "clean workaround via the on-demand test probe" is a claim PR #162 explicitly retracted -- both destinations' test_connection CREATE the target dir, so the probe cannot answer the question the toggle asks. That is what lifts it off the parity-with-a-workaround band to 6/3. Its replacement rationale was ALSO stale (it described the silent-ignore #162 had just fixed) and is hand-corrected. - #105's "large greenfield 71-action mapper needing its own ADR" describes an importer that has since shipped under ADR 0086. Scheduling barely moved, which is the reassuring result: only TWO tiers changed -- #64 DEMAND-GATE -> P3 (an index over levers that live in #62/#63/#47/#34, so it ships nothing runnable of its own) and #105 P3 -> DEMAND-GATE. Neither contradicts an explicit demand-gate/on-trigger ruling in its own body; that was checked for all 51 items carrying a prior tier. Distribution is RECOMPUTED with the table rather than carried forward, and all four lines sum to 92. The superseded table keeps its own frozen lines and now says so. Tiers: P1 4, P2 19, P3 17, DEMAND-GATE 52 Quadrants: quick win 22, big bet 5, fill-in 56, money pit 9 The four P1s: #341 (9/3, a handler returning a tuple/set of Sends delivers nothing silently -- an accept-and-drop CLAUDE.md §12 forbids), #324 (7/2), #325 (6/2), #327 (6/2). NOT in this commit: 24 items were found to misdescribe their own build state -- prose asserting a gap that has since shipped, or citing messagefoundry/console/, a package retired with #103. Those are banner corrections and land separately; the scores here already price the remainder rather than the original scope. Two mechanical faults were caught by reading the output rather than trusting the run: the quadrant regex omitted the hyphen in "fill-in", so 57 of 69 items took the fallback branch and got a SECOND score inserted beside the first; and the synthesizer's own distribution lines did not follow from its own table (11 quadrant mismatches, 8 ordering violations, difficulty summing to 95 of 92). The script now refuses to write when any line carries two score spans or the scored count is not 92. backlog_status_check.py --min-items 277: OK, 277 items across both files. * docs(backlog): correct 10 items whose own prose misdescribed build state The 2026-08-03 re-score flagged 24 open items as misdescribing what the code does. Re-verified each against the tree as it stands -- after the archive move and after PR #162, both of which post-date the findings -- and 10 survived. The other 14 did not, and are recorded here rather than silently dropped: #84 #95 #99 #105 #114 #124 #125 #127 #133 #137 #167 #169 #214 #228 Most of those already carry an amendment that covers the stale sentence (#95, #99, #105, #114, #124, #125, #127, #133, #228), and stacking a second ruling saying the same thing is noise. The rest did not survive verification: the finding was itself wrong or overstated, and a wrong correction in a ledger is worse than a stale one. CORRECTIONS ARE ADDED AS DATED AMENDMENTS, NOT PROSE REWRITES. This file's convention is to leave the original claim standing and rule against it, so the record shows what was believed and what replaced it. Silently editing the stale sentence would destroy the evidence that makes the correction checkable. Applied to #62 #64 #131 #166 #179 #182 #237 #321 #329 #336. Representative: - #329 "Five MEFOR_ALLOW_INSECURE_TLS cells": the census is FOUR. #323 landed and routed transports/direct.py through the clamp; it now holds no call to the raw predicate at all (:63, :197, :215). - #321 "no test asserts the detectors can see a site code": false -- tests/test_scan_forbidden.py has per-class hit tests for at least the site code (:126), a customer name (:83), a case-sensitive code (:91) and a routable IP (:107). The detector-coverage half of its Proposed 2 is already in the tree. - #62 plans a dual-read over "existing mfenc:v1 rows", but cell-bound mfenc:v2 is the default writer (settings.py:383 -> base.py:1841; crypto.py:36), and v2 folds (table, column, pk) into the GCM tag -- so a body landing under a different column must be RE-ENCRYPTED, not merely re-encoded. That tightens the catch. - #64's ordered plan still reads live ("Nothing builds before it"), but the measure-first phase completed 2026-07-12 (ADR 0051) and its step-2 lever is refused outright (ADR 0055 withdrawn; ADR 0107 "Do not build F2 or F3"). The refuters removed two overclaims before they landed: #62's draft asserted a live store holds both mfenc markers (a fresh store under the shipped default holds only v2 -- the defensible claim is that a MIGRATION must expect both), and #64's asserted the multi-DB log split still remains, which could not be verified against ADR 0098 and would have been a fresh false claim. No item closes here: in every case the correction narrows the remainder rather than discharging it, and the 2026-08-03 scores already price the remainder. backlog_status_check.py --min-items 277: OK, 277 items, one status banner each. * docs(backlog): file BACKLOG #1000 — prove each required merge context can fail Escalated by the coordinator on the ground that it outlives the PR that fixed it. Deliberately NOT filed as "fix the two-dot diff": that instance already landed in 39b62bf, and filing shipped work is the rot the hygiene gate exists to prevent. The item is the CLASS. `.github/required-contexts.txt` names 13 contexts that block merge, and not one of them is proven able to go red. The deliverable is a negative control per context -- a fixture carrying the exact violation that context exists to catch -- plus a CI job that fails when a required context has none, so the coverage cannot silently decay as contexts are added. Scoped narrower than "test the gates" on purpose: it does not re-test what each gate checks, since the gates' own suites do that. It asserts one property per context -- this gate is capable of failing. The argument is that the class has now fired at least four times here, each found by hand and none by CI: #334 semgrep, required and blocking, scans a two-directory allow-list #327 six .gitignore rules are the sole control over maintainer-internal docs, and nothing asserts they still match anything #321 the forbidden-content gate exited 0 on a real site code and partner product #325 the same gate's home-path detector misses 1 of 4 spellings of a Windows path Each is correctly filed as its own defect. None of them establishes the property that would have caught all four before they shipped, and that property is a different artifact from any of the individual fixes. Value 7 / Difficulty 3, quick win, P1 -- not demand-gated; the trigger fired four times. Ranked table and all four distribution lines recomputed to 93 open items; a mechanical check confirms 93 banners and 93 rows agree on every triple. Number allocated atomically via scripts/coord/alloc.ps1 (#1000 -- the first in the post-partition public sequence, clamped to >= PUBLIC_BACKLOG_FLOOR), never grepped. backlog_status_check.py --min-items 277: OK, 278 items across both files. The floor is a floor, so growth passes it; it is there to catch shrinkage.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Running the full suite on
mainfailed with spreadsheet writers "outside the ASVS 1.2.10 gate" — at paths under.claude/worktrees/mefor-jdbc-support-c275bd/…. Another session's working copy.docs/WORKTREES.mdplaces sibling worktrees at.claude/worktrees/<name>/— nested inside this checkout. This gate walks_REPO.rglob("*.py"), and its skip set covered.venv,.git,node_modules,__pycache__and the two caches — but not.claude. So with a parallel session active it scanned a second full copy of the repo, 5,712 extra.pyfiles, at a different commit.CI never sees this (no nested worktrees), so it reds only the local run — which is where people iterate, and the natural response is to learn that this test "just fails sometimes." Then it guards nothing. That's the real cost.
Both directions are wrong, and the one that didn't happen is worse
This gate only fails loud. But a gate searching for a required registration could find it in the other worktree and pass while this checkout lacks it — invisible in CI, because CI has no nested worktrees. Any future repo-root scan inherits that.
Added a guard so the next one can't reintroduce it. Mutation-verified: remove
.claudefrom the skip set and it fails.Sizing, corrected mid-investigation
I first measured "13 test files use rglob" and was about to treat all 13 as affected. They mostly walk narrow roots —
messagefoundry/,messagefoundry/auth/,samples/messages/— which cannot reach.claude/. Exactly one walks from the repo root.test_ldap_timeouts.pylooked like a second (root = Path(__file__).resolve().parent.parent) but uses that root to read one specific file, not to walk.The question isn't "who calls rglob" but "who walks from the repo root", and those give very different answers.
The two assertions differ in strength, and the docstring says so
_SKIP_DIRSmembership — always-on, and what the mutation confirms..claude/" — only has teeth in a tree that has a nested worktree; passes trivially in CI or a sibling worktree.Belt-and-braces is fine; quietly presenting the second as evidence would not be.
Also seen, and not a repo defect
test_version::test_installed_metadata_matches_dunder_versionfails locally because this venv's editable install still registers0.3.0while__init__is0.3.2after today's releases.pip install -e .refreshes it. Left alone.🤖 Generated with Claude Code