Make every cf-gate say what it examined, not just that it passed - #28
Merged
Conversation
The board printed eleven bare labels and one measurement. complexipy was the
only honest line because it alone runs in-process and builds its GateVerdict
directly; every other gate crossed a subprocess boundary and the measurement
did not survive the crossing.
Three breaks, not one:
1. print_verdict built GateVerdict(gate, violations, error) and never passed
notices or evidence. `notices` was a print-only parameter, so the wire
shipped "notices": [], "evidence": {} for every gate — empty at the source.
2. _structured_verdict never read either field back off the parsed JSON.
3. No cf-gate computed a denominator at all. None counted what it examined,
so closing (1) and (2) alone would still have printed a bare label.
print_verdict now takes `measured` (one line, rides GateVerdict.notices to the
wire) and `evidence` (the same measurement in machine form). The pre-existing
`notices` parameter stays the gate's multi-line human prose and deliberately
does NOT reach the wire, so a board line stays one line. _structured_verdict
reads both back. Each of the seven cf-gates now threads a count out of the walk
it actually performed — never a second walk, never a number the gate did not
use. cf-file-budget additionally stops hand-rolling its own JSON and routes
through the shared print_verdict, so it finally honours CF_QUALITY_JSON.
A PASS with no denominator is indistinguishable from a PASS that examined
nothing, so zero is the case that had to work: every gate renders a visible 0
on an empty tree, proven by control-rod tests rather than asserted on
hand-built verdicts.
gate_runner.py stays at exactly 500 lines, the file ceiling.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The first pass gave every cf-gate a denominator. Three of them counted the
wrong thing, so the line was confident and false — worse than a bare label,
because an operator can act on it.
cf-no-bon-ref counted files the walk offered, including the binaries it
skips unread. A tree of one PNG reported "swept 1 file(s)" having opened
nothing. scan_file now returns whether it READ the file, and the sweep
counts only reads.
cf-exemptions counted surface BASES. A declared source_root, or a present
src/, is always exactly one base — so three Python files and zero Python
files produced byte-identical lines. That is a gate selecting by the value
it guards, reproduced inside the rung meant to cure it. The surface is now
resolved once into a file list, threaded into the scanner, and the count is
that very walk.
cf-import-contract claimed one surface covered another ("linted N clause(s)
over M module(s)"), fusing PASS 3's AST walk into a sentence about PASS 1,
and shipped two different evidence schemas from its two branches. Both
branches now speak one schema and the line names each surface separately.
cf-file-budget reported only its frozen FILE entries and stayed silent on
the package ceilings it also enforces; both baselines now ride the evidence.
Tests assert each zero against the SHAPE that hides the defect — one
unreadable binary, a src/ holding no Python — because an empty directory
drives every counting bug to zero and proves nothing.
A refuter lens claimed three denominators were vacuous. I verified it by mutation rather than by argument: hardcoding cf-recursion-check, cf-sticky-check and cf-mirror-check to report 0 left the whole suite GREEN, and the board then printed "examined 0 CLAUDE.md" on a repo that has one and "walked 0 Python file(s)" over a 19-file tree. Confident and false — the exact defect this branch exists to cure, reproduced inside it. Each of those three had a zero assertion and no non-zero counterpart, so the zero was proving nothing. Each now has the other half of the rod: a known count over real content. Re-running the same four mutants after the change kills all four. The previous commit message overstated its fixtures: it claimed every zero used "the SHAPE that hides the defect" when three were bare empty directories, which drive every counting bug to 0. cf-recursion-check and cf-file-budget now use a present tree holding no Python. cf-file-budget's denominator was also untrue: it called the whole baseline "frozen file entry(ies)", but a declared-not-banned entry carries a purpose and no line count, and _check_file treats it exactly like an undeclared file. It claimed a shrink-only ratchet over files that have none. The two kinds are now counted and named separately. Finally, the aggregated board itself had no guard: nothing failed if a cf-* gate reached it carrying no measurement, because every other denominator test drives one gate in-process. The full-battery test now asserts every cf-* verdict crosses the real subprocess wire with exactly one notice and a non-empty evidence dict, which also makes the runner's silent unparseable-output fallback fatal for a kit gate.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CONTRACT
Every
cf-*gate line on the aggregated board must say what it examined. A line readingPASS cf-no-bon-refis a name, not an event: it reads identically whether the gate swept fourhundred files or none at all.
WORLD (measured, before)
Eleven board lines, exactly one of them a measurement —
complexipy, the only gate that runsin-process:
The break was not the single parse-boundary bug it looked like. Measured on the raw wire, it was
a three-link chain:
reporting.print_verdictbuiltGateVerdict(gate, violations, error)and neverpassed
notices/evidence.noticeswas a print parameter routed only to the human emitter, sothe wire was empty at the source.
gate_runner._structured_verdictrebuilt the verdict without readingnotices/evidenceback. Real, but downstream: fixing it alone changes nothing.cf-*gate computed a denominator at all. With links 1 and 2 closed, a clean runwould still have printed a bare label.
cf-file-budgetadditionally hand-rolled its own JSON and printedcf-file-budget: cleaneven underCF_QUALITY_JSON=1, bypassing the shared wire entirely.FAILURE (what this closes)
All three links, plus a denominator in each of the seven
cf-*gates.GateVerdict.noticesnowcarries the one denominator line and
evidencethe same measurement in machine form; both ride thewire and are parsed back onto the board.
cf-mirror-checkis skip-if-absent and emits no line in this repo (noMIRRORS.md); driven directlyit reports
— checked N declared mirror row(s) in MIRRORS.md.Three denominators that lied, caught by the cadre
The first pass gave all seven gates a number. Three of them counted the wrong thing, which is worse
than a bare label because an operator can act on it.
cf-no-bon-refcounted files it never opened. A tree holding one binary — which the gate skipsunread — reported
swept 1 file(s).scan_filenow returns whether it read the file.cf-exemptionswas vacuous. It counted surface bases, and a declaredsource_rootor apresent
src/is always exactly one base, so three Python files and zero produced byte-identicallines. The surface is now resolved once into a file list and the count is that very walk.
cf-import-contractclaimed one surface covered another and shipped two differentevidenceschemas from its two branches. Both branches now speak one schema.
The denominators are now falsifiable
A refuter lens argued three more denominators were vacuous. Verified by mutation, not argument:
hardcoding
cf-recursion-check,cf-sticky-checkandcf-mirror-checkto0left the suite fullygreen, and the board printed
examined 0 CLAUDE.mdon a repo that has one andwalked 0 Python file(s)over a 19-file tree. Each had a zero assertion and no non-zero counterpart, so the zeroproved nothing. All three now have one, and re-running the same mutants kills every one.
cf-file-budgetwas also mislabelling: it called the whole baselinefrozen file entry(ies), but adeclared-not-banned entry has a purpose and no line count and
_check_filetreats it exactly like anundeclared file. It claimed a shrink-only ratchet over files that have none. The two kinds are now
counted and named separately.
The board itself had no guard — every other denominator test drives one gate in-process. The
full-battery integration test now asserts every
cf-*verdict crosses the real subprocess wire withexactly one notice and non-empty evidence, which also makes the runner's silent
unparseable-output fallback fatal for a kit gate.
Control rod
Zero is asserted in the rendered output for all seven gates, each against a shape that would hide a
counting bug rather than a bare empty directory (an empty directory drives every counting bug to 0):
Scope and budget
External tools (
ruff,ruff-format,mypy,pytest) are deliberately untouched — they emit nocf-JSON and giving them denominators means parsing four foreign output formats. That is a separate
change. No consumer pin is bumped, no workflow file is touched, and no
conftest.pywas created.Src churn is 261 changed lines against a brief that budgeted 120. The budget was mis-set, not
overspent: it was sized for a single parse-boundary fix, but the measured world required each of
nine modules to emit its own denominator (10–61 lines each). Recorded openly rather than hidden by
compressing docstrings.
gate_runner.pystays at exactly 500/500 lines, the file ceiling, paid for in place by inliningtwo named locals rather than displacing logic elsewhere. It now has zero headroom — the next
change to it forces a real split.
Regression surface
No repo outside this kit imports
cf_quality, parses the cf-gate JSON, or greps the board; consumersinvoke the reusable workflow, which runs
cf-gateand asserts only its exit code. Every violationpredicate is unchanged, so no pinned consumer's verdict can flip. The wire keys already existed —
only their values change, which is additive.
Note for the reviewer: live consumers pin
adaa4ebf(bonfire, bonfire-lexicon, bonfire-public,code-forge, ishtar, sweetcrm) and
86c9e0b5(mexxa) — six of seven are ~60 commits behind thisbranch's base, so any future pin bump absorbs that whole delta, not just this change.
Known residue
tests/test_gate_denominator.pyimports two private helpers from a sibling test module becausetests/conftest.pydoes not exist. Known and accepted; creating it is a separate change.cf-exemptions' fold-in-failure pathbypasses
print_verdictentirely, so it can still print a bare label on that one red path.cf-exemptions/cf-no-bon-refreport a denominator but not thefrozen_countratchet they alsoenforce; that ratchet prose rides human
notices, which by design never reaches the wire.GateVerdict.notices' docstring inerrors.pystill describes the field generically and was notupdated to say it now holds the denominator.
Verification
ruff,ruff-format,mypy,complexipyand all sevencf-*gates are green.pytestis574 passed, 3 errors; the 3 errors are
tests/test_packaging.pyonly and are pre-existing onorigin/main, reproduced there at 553 passed, 3 errors in an independent virtualenv — theauthoring machine lacks
python3.12-venv/ensurepipand those tests build a clean venv. CI hasensurepipand is the authority.No Constable has run on this change.