feat(#114): the outbound validate_startup hook, with its ledger banner (train) - #327
Merged
Conversation
…d directory toggle The remainder of #114. DestinationConnector had no startup-validation hook and FileDestination mkdir'd on write, so an outbound target directory was never validated: a typo'd directory / remote_dir would not fail, it would be CREATED, and every message delivered into it counted and logged as delivered -- because it was. On a first deployment that is a feed landing in a path nobody is watching with no error anywhere. (Nothing is misdelivering today; there are zero deployments.) The item's 6/10 rested on "a clean workaround via the on-demand test probe". Measured against the shipped code before building: FileDestination.test_connection creates the missing directory (exists False -> True) and RemoteFileDestination.test_connection calls ensure_dir, which creates. The probe cannot answer the question the toggle asks, because asking changes the answer. - DestinationConnector.validate_startup(), defaulting to a no-op -- the exact shape of the SourceConnector hook, so the other eleven destination connectors are untouched and this is not a protocol change that ripples. FILE and REMOTEFILE override it; DestinationStartupError mirrors SourceStartupError. - The runner awaits it in _start_outbound right after the build, INSIDE the existing ADR-0031 isolation try, so a refusal takes the same path as a build failure: lane recorded failed with no connector, its delivery worker still spawned, routed rows retried and never dropped. On an outbound that degraded-lane state is what "invalid means not-started" means. Same call on the operator start path (_ensure_destination_built); deliberately NOT on the reload path, whose stated invariant is that a connector build there cannot fail. - validate_directory becomes a both-directions option and the 2026-08-03 outbound WiringError is removed: it existed only because no destination read the setting. - Under the toggle nothing is ever created -- not at start, not on write, and not by POST /connections/{name}/test, which would otherwise silently repair the typo the toggle exists to catch. The REMOTEFILE arm pre-checks with a listing to RECLASSIFY the failure as transient: an SFTP/FTP no-such-dir is permanent, so letting the upload fail on its own would dead-letter live traffic over a merely-unmounted share. - Default unchanged, with one deliberate addition: a target directory the engine actually had to CREATE now logs a WARNING, so that delivery is no longer indistinguishable from a normal one. The FILE path costs no extra syscall (mkdir(exist_ok=True) already probed is_dir on its FileExistsError branch); _RemoteClient.ensure_dir now reports whether it created. The lenient arm is the default because the item's own trigger is the opposite case: an intermittently-available directory must NOT fail startup. Tests were written red first (14 failing across both directions), and both directions were then re-sabotaged to prove they have teeth: dropping the runner call reds the refusal test, and making the toggle non-optional reds the deferral tests.
…NS rows, test-plan row The 2026-07-17 amendment deferred the outbound hook on two grounds: the destination "already mkdirs on write", and it "has the on-demand test probe". The 2026-08-03 follow-on withdrew the second; this amendment withdraws the first and marks that follow-on superseded, since the WiringError it added existed only while the hook did not. CONNECTIONS.md moves validate_directory from "in" to "both" on File and Sftp/Ftp and states the outbound semantics: the target must already exist and accept a write, is then never created at start, on write or by the test probe, and a delivery into a vanished directory fails retryably. It also records the default-arm change that applies to every existing outbound -- a directory the engine had to create is now logged. The master-test-plan row moves 7 -> 9 and names what the two new runner-level cases assert.
The banner half of #114's train. Pairs with w3-outbound-validate (0871320); backlog-hygiene demands a ledger change from a PR that cites an item and touches engine code, and a lane is forbidden to make one. CLAIMS VERIFIED AGAINST THE BRANCH RATHER THAN TRANSCRIBED FROM ITS REPORT: validate_startup exists on the DestinationConnector contract with an override, ADR 0031 carries the amendment with its 2026-08-03 follow-on marked superseded, and 5 WiringError lines are removed. The defect is written in the conditional, as it must be: FileDestination previously mkdir-ed its target on write, so a deploying operator who typo'd remote_dir WOULD get a directory silently created and messages delivered into it -- the feed reading healthy while writing to the wrong place, with nothing reporting it. Recorded because it is the more interesting half: THIS ITEM'S OWN 6/10 RATIONALE WAS RE-MEASURED AND IS FALSE. It claimed a 'clean workaround via the on-demand test probe', but both destinations' test_connection CREATE the directory -- so the act of asking changes the answer. An item's scoring rationale falsified by execution, the same class as #1011's refuted premise. A workaround that alters the state it reports on is not a workaround. INHERITED FAILURE, STATED SO IT IS NOT READ AS MINE. This branch is cut from origin/main, which is still red on test_dast_claims::test_no_file_claims_dast_closes_the_independent_gap. Confirmed INHERITED BY IDENTITY, not by count: the failing line is docs/BACKLOG.md:3738, the #1008 'independent of the gate ... INCOMPLETE' text, while this commit's banner is at line 1686 and contains zero occurrences of 'independen'. It clears when w3-docs-guard-gap (c2b85ee) lands, which fixes both the guard and that prose. Doc guards run locally before handover, which is the practice this session's incident established: 46 passed, 1 failed, that one being the inherited case above. Ledger gates clean; 484 items each declaring exactly one status. Open 181 -> 180.
Two blockquotes, two lines below the SHIPPED banner, asserted in the PRESENT TENSE that DestinationConnector 'still has no validate_startup hook' and FileDestination 'still mkdirs on write'. Both false once the code lands, in an item declaring itself complete. Rewritten to past tense. WHY NO GATE CAUGHT IT, and this is the part worth keeping: the offending quote leads with U+26A0, which is in NEITHER _CLOSED nor _OPEN. So parse_items is structurally silent about it. THE GREEN BANNER GATE PROVES ONE STATUS BANNER PER ITEM; IT CANNOT SEE A PROSE CONTRADICTION TWO LINES AWAY, and reading its green as 'the item is truthful' is reading a different sentence than the one it asserts. Also recorded, because the close REVERSED it: the 2026-08-03 outbound WiringError that quote describes as BUILT was subsequently REMOVED -- it existed only to reject a setting no destination read, and a destination now reads it. A record that lists something as built, when a later change deleted it, is worse than silence. And the REMAINDER quote closed with a BUILD GATE -- 'if the hook is built, build it together with suppressing the mkdir-on-write, because a start-time-only check leaves the run-time fabrication intact under a setting name that promises otherwise.' That gate was HONOURED, and the banner now says so rather than leaving the instruction reading as outstanding work. MY OWN VERIFICATION SCAN THEN PRODUCED A FALSE POSITIVE, WHICH IS ITSELF THE LESSON. It flagged 'only the outbound validation HOOK remains' as a surviving present-tense claim. It is not: it is that phrase QUOTED inside my correction, immediately followed by 'which was true then and is not now'. A substring scan cannot distinguish a CLAIM from a QUOTED AND RETRACTED claim -- the same presence-equals-meaning failure CLAUDE.md section 11 describes for glyphs, and the same shape as the DAST guard reading INCOMPLETE as COMPLETE. I checked the context rather than trusting my own scanner, and the scanner was wrong. Verified: #114 reads closed via parse_items, 484 items each declaring exactly one status, doc guards 30 passed locally before handover.
The outbound validate_startup hook (BACKLOG #114). Rides with its ledger banner because a code lane cannot edit the ledger and a banner alone would assert SHIPPED for code not on main.
The close banner for BACKLOG #114, with the two stale blockquotes rewritten to past tense. parse_items cannot see that class of contradiction: the offending quote leads with a glyph in neither _CLOSED nor _OPEN, so a green banner gate proves one status banner per item, not that the item is truthful.
wshallwshall
added a commit
that referenced
this pull request
Aug 13, 2026
…e four missing subpackages (#356) * docs(claude): list the four missing subpackages in the section 3 tree The repository-layout tree listed 10 subpackages while messagefoundry/ has 14. security/, support/, verify/ and tray/ were absent, so CLAUDE.md contained ZERO occurrences of "tray" despite ADR 0113 and a 17-module package shipping at HEAD. Cause: the tray, verify and support packages landed in the private lineage between 2026-07-13 and 07-25, and that content never reached the repo of record at publication -- five commits dropped by omission, absent from HEAD. The tree has been stale since. Descriptions taken from each package's own __init__ docstring. The tray line records what it wraps, verified rather than assumed: tray/ imports only messagefoundry.service, messagefoundry.service_status and its own submodules -- no api/, no engine internals -- so section 4's dependency rule needs no carve-out for it. Insertion-only, no section renumbered. That is deliberate: 281 tracked files cite this file by section number, nothing validates a section number (link_check.py:21 validates the path and skips the fragment), and the same rot has already landed once -- test_sds_rule_ids_are_stable.py:7-11 records four security citations that still resolve to the wrong section today. Verified: tests/test_link_resolution.py and tests/test_sds_rule_ids_are_stable.py, 34 passed. The second was confirmed to read this worktree's file by injecting a bogus SDS-9.9 citation and watching it go red, then removing it. * feat(gate): resolve every cited CLAUDE.md section number, and stop a nested anchor escaping the SDS gate CLAUDE.md's numbered sections are a de facto API and NOTHING validated a section number. Measured: 281 tracked files name the file, 646 citations name one of its sections, and tests/test_dependency_boundaries.py cites section 4 in its own docstring, so this reaches code and not only prose. Every existing gate stays green through a renumber. link_check.py:21 says so in its own header -- it resolves the PATH and skips the #fragment -- and the two doc guards that already read CLAUDE.md check other things: one checks SDS-N.N identifiers, one checks link paths. Path resolves, identifiers resolve, only the meaning moves. The checkable half stays green. The rot is not hypothetical. The sibling guard tests/test_sds_rule_ids_are_stable.py:5-11 records it landing on the Secure Development Standards -- a new section 5 pushed 5-9 to 6-10, and citations still resolve to the wrong section today. The instance was fixed and the class left open. This closes it for the anchor. MEASURED BLAST RADIUS: renumbering section 11 alone breaks 61 citations. scripts/docs/claude_section_check.py, stdlib only, beside its siblings: - parses the anchor's own `## N.` headings as the source of truth - scans tracked .md/.py/.ps1/.yml via git ls-files - requires the anchor's NAME on the same line within a bounded window, so a section number belonging to another document is never attributed to it -- false positives are how a gate gets disabled - ALWAYS prints what it scanned; a finding count alone cannot distinguish a clean scan from one that examined nothing - REFUSES to run when zero headings parse or the anchor is missing, rather than reporting every citation broken or the corpus clean ADDED TO DOC_GUARDS in ci.yml. Without that it would be born unable to fire: a markdown-only PR is classified non-code and runs only that list, so a check landed outside it goes green by absence on exactly the PR shape it exists to catch. That enumeration hole has already been fixed twice for other paths (BACKLOG #327, #1200) and left open both times; ci.yml:1050 documents its own recurrence. Also widens _citation_sources() in the SDS guard to include nested CLAUDE.md files. Root is a literal path and the globs are rooted at docs/, messagefoundry/ and scripts/, so a subpackage anchor -- which CLAUDE.md section 3 invites -- would take its SDS citations out of coverage silently. Resolved through git ls-files, not rglob, because this repository nests worktrees under .claude/worktrees/ and an rglob would scan sibling checkouts. There are no nested anchors today; this is the prerequisite, not a response. Verified. Both guards were made to FAIL ON PURPOSE and the injected defect confirmed to land: - renumber section 11 to 13 -> rc=1, 61 citations; revert -> rc=0 - nested CLAUDE.md citing SDS-9.9 -> SDS guard red; remove -> green - 8 self-bite tests, including both refuse-to-run guards and a false-positive case (another document's section 9 is not claimed) - ruff check, ruff format, mypy strict clean - full DOC_GUARDS set, 15 modules: 219 passed, 89 skipped
wshallwshall
added a commit
that referenced
this pull request
Aug 14, 2026
…ot enforcing (#373) * config: track .claude/settings.json, and fix what it silently was not enforcing `/.claude/` ignored the directory, so `.claude/settings.json` was never tracked and `git worktree add` could not deliver it -- the same failure the neighbouring comment already records for CLAUDE.md. Measured 2026-08-13: of 62 local checkouts carrying CLAUDE.md, only 12 had `.claude/settings.json`. The other 50 ran with no deny-list over `.env` / `secrets/**` / keys / the local `*.db` store, and no `block-blanket-git-stage` PreToolUse guard. Those are client-ENFORCED controls; CLAUDE.md section 5's prose is context, not enforcement, so it does not substitute. BACKLOG #327 recorded the same gap from the other side -- "it is wired through .claude/settings.json, which is itself inside the now-gitignored /.claude/ tree and untracked ... Do not count it as coverage" -- and carried the stale `.gitignore:84` comment to the owner rather than fixing it. Both are resolved here. Ignore by contents (`/.claude/*`) rather than by directory, then re-include the one file. The directory form would have made the negation a silent no-op, because git cannot re-include a file whose parent directory is excluded. `/.claude/worktrees/` is now named explicitly instead of relying on that wildcard plus a per-clone `.git/info/exclude` line that reaches nobody else. Publishing the file made two of its defects load-bearing, so both are fixed here rather than shipped: * Both hook commands were bare `pwsh -NoProfile -File scripts/hooks/...`, which resolves against the session's working directory. A hook that cannot start is NON-BLOCKING -- the action proceeds and the only trace is a notice -- so the staging guard read as enforced in the file and was absent in any session started elsewhere. Now `${CLAUDE_PROJECT_DIR}` in exec form. * All 16 file deny rules used the `./` anchor, which matches one directory. Bare patterns follow gitignore semantics and match at any depth, so `Read(.env)` is strictly broader than `Read(./.env)` and reads identically in review. Prefix dropped. Three `Get-Content` denies added for the PowerShell path, which the documented Read/Edit deny coverage does not reach. The allow list went from 11 hyper-literal command strings to 5 wildcards. Allow rules are the only permission rules gated on the workspace trust dialog, so they applied in none of the untrusted checkouts anyway; the literals also could not survive an argument change, which is how the list grew three near-duplicate pytest invocations. Deny rules are not trust-gated and are evaluated before the auto-mode classifier, so they are the half worth getting right. tests/test_private_paths_stay_ignored.py FAILED on the previous commit and that was correct: it asserts nothing under a private rule is tracked, and tracking settings.json violated that. Pre-commit does not run pytest, so nothing caught it at commit time. Updated deliberately, per that file's own doctrine -- the tracked set under `.claude/` is now pinned as an exact SET, not a floor, so a second negation fails the build instead of publishing. Its new companion test asserts the asymmetry directly: settings.json un-ignored, and `rules/`, `skills/`, `agents/`, `worktrees/` and `settings.local.json` still ignored. tests/test_claude_settings_contract.py is new and covers the payload: the PHI and secret denies are present, no rule regresses to `./`, every hook anchors to the project root, and every referenced script exists. Both absence checks carry a planted-omission self-test, because an absence assertion over a currently-correct file passes just as well when the check itself is broken. `.claude/` also leaves link_check.py's WITHHELD tuple. It was exempt because 7 links pointed at a file no clone had; all 7 name settings.json, which is now tracked, so they resolve honestly and are COUNTED -- the exemption `continue`d before `checked += 1`, so those links were never in the total. Repo-wide link count moves 5359 to 5405. CONTRIBUTING.md discloses what cloning now configures: two PowerShell scripts wired to SessionStart and PreToolUse, that they need pwsh and fail open without it, and that the deny rules anchor at the directory the agent was started in and do not cover writes into sibling worktrees by absolute path. * config: hold the hooks block, land only the deny-list (owner: "do a") The owner chose option (a) on PR #373: land the 27-rule deny-list, HOLD the three hook registrations. Held is NOT rejected -- no verdict was given on the hooks, and no later change may cite this as one. The two halves have different risk profiles, which is why they separate: DENY-LIST 27 rules mirroring CLAUDE.md section 5 almost literally -- .env, .env.*, secrets/**, *.key, *.pem, *.pfx, *.db, *.db-wal, *.db-shm, bootstrap-admin.txt. The repo's own written policy as enforcement. HOOKS PreToolUse (Bash, PowerShell) -> block-blanket-git-stage.ps1, and SessionStart -> session-context.ps1. Changes what EXECUTES on every matching tool call in every checkout. A deny-list can only refuse; a PreToolUse hook runs code. KNOWN INTERIM CONDITION, measured rather than assumed: with no `hooks` key, test_every_hook_resolves_through_the_project_dir_placeholder and test_every_hook_script_actually_exists iterate an EMPTY set -- 0 handlers -- so they pass VACUOUSLY. Nothing fails; two guards quietly stop guarding until the hooks decision is made, at which point both become live again automatically. This file's own test_the_checks_can_actually_fail exists because absence assertions over a correct file are "the shape that passes just as well when the check is broken" -- but it runs against PLANTED documents, so it passes and does NOT catch this vacuity. The negative control does not cover an empty input set. Verified: 7 passed in tests/test_claude_settings_contract.py; deny 27, allow 5, dot-anchored denies 0, hook handlers 0.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes BACKLOG #114 -- the outbound
validate_startuphook, landing as a two-lane train: the codeand its ledger banner together, in one PR.
Why a train, and why this one is not optional
A code lane cannot edit
docs/BACKLOG.md(single-writer), so the banner is authored separately. Landingeither half alone is wrong, and the banner half is wrong in a way nothing would catch:
git merge-tree origin/main w3-banner-114merges CLEAN. The resulting tree declares"SHIPPED -- #114 is complete" while
transports/base.pyhas noDestinationConnector.validate_startupand ADR 0031 still reads "remains out of scope and deferred". No conflict, no marker, no gate. Git
conflicts on concurrent edits, never on a claim its own merge falsifies.
Verified on the combined tree instead of on either branch:
transports/base.py:validate_startupat line 498, insideDestinationConnector(class opens at470) -- plus the source-side hook at 447 in
SourceConnector, andDestinationStartupErrorpresent.docs/BACKLOG.md: the#114SHIPPED banner, one status banner for the item.The two lanes touch zero files in common (13 code/doc files vs one ledger file), which is what makes
the train safe rather than merely convenient.
What shipped
DestinationConnector.validate_startup()is a default no-op, so the other eleven destinationconnectors are untouched and this is not a protocol change that ripples.
FileDestinationandRemoteFileDestinationoverride it. The runner awaits it in_start_outboundand_ensure_destination_builtinside the existing ADR 0031 isolationtry, so a refusal is afailedlane with no connector whose delivery worker still spawns -- routed rows are retained, retried and
buildup-alerted rather than dropped, and the count-and-log invariant holds.
The defect, in the conditional (this is a beta with zero deployments):
FileDestinationpreviouslymkdir-ed its target on write, so a deploying operator who typo'dremote_dirwould get a directorysilently created and messages delivered into it -- the feed reading healthy while writing to the wrong
place.
validate_directorybecomes a both-directions option, and the 2026-08-03 outboundWiringErroris removed, because it existed only to reject a setting no destination read.One deliberate change reaches every existing outbound, toggled or not: a target directory the engine
actually had to create now logs a
WARNING, so a delivery into an invented path is no longerindistinguishable from a normal one.
The ledger half fixes a self-contradiction the banner gate cannot see
#114carried two older blockquotes asserting "DestinationConnectorstill has novalidate_startuphook and
FileDestinationstillmkdirs on write" -- false once this lands, and sitting two lines belowthe new SHIPPED banner. Both are rewritten to past tense.
parse_itemsis structurally silent about this. The offending quote leads with a glyph in neither_CLOSEDnor_OPEN, so the gate proves one status banner per item and nothing more. Reading a greenbanner gate as "the item is truthful" is reading a different sentence than the one it asserts.
A second reversal was folded in: the 2026-08-03 blockquote listed the outbound
WiringErroras BUILT,and this change removes it. A record listing something as built when a later change deleted it is
worse than silence. The
REMAINDERquote also closed with a build gate -- build the hook togetherwith suppressing the mkdir-on-write, since a start-time-only check would leave the run-time fabrication
intact under a setting name that promises otherwise. That gate was honoured, and the item now says so
rather than leaving the instruction reading as outstanding work.
Verification
Peer-verified, then independently re-verified by a separate agent against current
origin/mainrather than the branch's base. Three measurement corrections came out of that and are recorded here so
the numbers in the banner are right:
validate_startupoverrides are two (FileDestinationatfile.py:243,RemoteFileDestinationatremotefile.py:667), not one -- which makes the banner's "other eleven destination connectors" exact:13 subclasses minus 2.
WiringErrorlines removed" is exact forconfig/wiring.py; whole-branch is 10 across 5 files,and only one of the five is an actual
raise.--stattotal churn asinsertions.
Drift checked against current
main, which advanced 50 commits past the code lane's base and touched 2of its 13 files -- neither invalidates: the
CONNECTIONS.mdedit is retry-cap prose with novalidate_directorymentions, and thewiring_runner.pyedits sit at lines 4573+ against the lane's1815-2480.
Not measured by the verifier (read-only, no checkout):
ruff,mypy, and the full suite against thecombined tree. CI is the instrument for those, and they are unmeasured rather than assumed green. The
sync-to-async conversion of three runner methods is the change most worth watching in the test legs.