Skip to content

Make every cf-gate say what it examined, not just that it passed - #28

Merged
Antawari merged 3 commits into
mainfrom
catrina/kit-notices-denominator
Jul 30, 2026
Merged

Make every cf-gate say what it examined, not just that it passed#28
Antawari merged 3 commits into
mainfrom
catrina/kit-notices-denominator

Conversation

@Antawari

Copy link
Copy Markdown
Contributor

CONTRACT

Every cf-* gate line on the aggregated board must say what it examined. A line reading
PASS cf-no-bon-ref is a name, not an event: it reads identically whether the gate swept four
hundred files or none at all.

WORLD (measured, before)

Eleven board lines, exactly one of them a measurement — complexipy, the only gate that runs
in-process:

PASS  ruff-check
PASS  ruff-format
PASS  cf-sticky-check
PASS  cf-file-budget
PASS  cf-recursion-check
PASS  cf-exemptions
PASS  cf-no-bon-ref
PASS  cf-import-contract
PASS  mypy
PASS  complexipy — measured 252 function(s) in 18 file(s) against a 0-function committed floor
FAIL  pytest

The break was not the single parse-boundary bug it looked like. Measured on the raw wire, it was
a three-link chain:

  1. Constructorreporting.print_verdict built GateVerdict(gate, violations, error) and never
    passed notices/evidence. notices was a print parameter routed only to the human emitter, so
    the wire was empty at the source.
  2. Parsergate_runner._structured_verdict rebuilt the verdict without reading notices/
    evidence back. Real, but downstream: fixing it alone changes nothing.
  3. Source — no cf-* gate computed a denominator at all. With links 1 and 2 closed, a clean run
    would still have printed a bare label.

cf-file-budget additionally hand-rolled its own JSON and printed cf-file-budget: clean even under
CF_QUALITY_JSON=1, bypassing the shared wire entirely.

FAILURE (what this closes)

All three links, plus a denominator in each of the seven cf-* gates. GateVerdict.notices now
carries the one denominator line and evidence the same measurement in machine form; both ride the
wire and are parsed back onto the board.

PASS  ruff-check
PASS  ruff-format
PASS  cf-sticky-check — examined 1 CLAUDE.md against the canonical sticky block
PASS  cf-file-budget — measured 47 file(s) against 0 frozen file entry(ies), 0 declared-not-banned entry(ies) and 0 frozen package budget(s)
PASS  cf-recursion-check — walked 19 Python file(s) for undeclared self-recursion
PASS  cf-exemptions — read 19 Python file(s), found 4 suppression(s) against 4 registered entry(ies)
PASS  cf-no-bon-ref — read 120 text file(s) of the code/config tree for ticket refs
PASS  cf-import-contract — linted 1 contract clause(s); scanned 19 module(s) for dynamic imports
PASS  mypy
PASS  complexipy — measured 255 function(s) in 18 file(s) against a 0-function committed floor
FAIL  pytest

cf-mirror-check is skip-if-absent and emits no line in this repo (no MIRRORS.md); driven directly
it reports — checked N declared mirror row(s) in MIRRORS.md.

Three denominators that lied, caught by the cadre

The first pass gave all seven gates a number. Three of them counted the wrong thing, which is worse
than a bare label because an operator can act on it.

  • cf-no-bon-ref counted files it never opened. A tree holding one binary — which the gate skips
    unread — reported swept 1 file(s). scan_file now returns whether it read the file.
  • cf-exemptions was vacuous. It counted surface bases, and a declared source_root or a
    present src/ is always exactly one base, so three Python files and zero produced byte-identical
    lines. The surface is now resolved once into a file list and the count is that very walk.
  • cf-import-contract claimed one surface covered another and shipped two different evidence
    schemas from its two branches. Both branches now speak one schema.

The denominators are now falsifiable

A refuter lens argued three more denominators were vacuous. Verified by mutation, not argument:
hardcoding cf-recursion-check, cf-sticky-check and cf-mirror-check to 0 left the suite fully
green, and the board printed examined 0 CLAUDE.md on a repo that has one and walked 0 Python file(s) over a 19-file tree. Each had a zero assertion and no non-zero counterpart, so the zero
proved nothing. All three now have one, and re-running the same mutants kills every one.

cf-file-budget was also mislabelling: it called the whole baseline frozen file entry(ies), but a
declared-not-banned entry has a purpose and no line count and _check_file treats it exactly like an
undeclared file. It claimed a shrink-only ratchet over files that have none. The two kinds are now
counted and named separately.

The board itself had no guard — every other denominator test drives one gate in-process. The
full-battery integration test now asserts every cf-* verdict crosses the real subprocess wire with
exactly one notice and non-empty evidence, which also makes the runner's silent
unparseable-output fallback fatal for a kit gate.

Control rod

Zero is asserted in the rendered output for all seven gates, each against a shape that would hide a
counting bug rather than a bare empty directory (an empty directory drives every counting bug to 0):

$ cf-no-bon-ref --root <tree holding one binary>
— read 0 text file(s) of the code/config tree for ticket refs
cf-no-bon-ref: OK (no ticket references in the code/config tree)

$ cf-exemptions --root <src/ present, holds no Python>
— read 0 Python file(s), found 0 suppression(s) against 0 registered entry(ies)

$ cf-exemptions --root <same shape, 3 .py files>     # the non-vacuity half
— read 3 Python file(s), found 0 suppression(s) against 0 registered entry(ies)

$ cf-mirror-check check --repo <MIRRORS.md, header, no rows>
— checked 0 declared mirror row(s) in MIRRORS.md

Scope and budget

External tools (ruff, ruff-format, mypy, pytest) are deliberately untouched — they emit no
cf-JSON and giving them denominators means parsing four foreign output formats. That is a separate
change. No consumer pin is bumped, no workflow file is touched, and no conftest.py was created.

Src churn is 261 changed lines against a brief that budgeted 120. The budget was mis-set, not
overspent
: it was sized for a single parse-boundary fix, but the measured world required each of
nine modules to emit its own denominator (10–61 lines each). Recorded openly rather than hidden by
compressing docstrings.

gate_runner.py stays at exactly 500/500 lines, the file ceiling, paid for in place by inlining
two named locals rather than displacing logic elsewhere. It now has zero headroom — the next
change to it forces a real split.

Regression surface

No repo outside this kit imports cf_quality, parses the cf-gate JSON, or greps the board; consumers
invoke the reusable workflow, which runs cf-gate and asserts only its exit code. Every violation
predicate is unchanged, so no pinned consumer's verdict can flip. The wire keys already existed —
only their values change, which is additive.

Note for the reviewer: live consumers pin adaa4ebf (bonfire, bonfire-lexicon, bonfire-public,
code-forge, ishtar, sweetcrm) and 86c9e0b5 (mexxa) — six of seven are ~60 commits behind this
branch's base, so any future pin bump absorbs that whole delta, not just this change.

Known residue

  • tests/test_gate_denominator.py imports two private helpers from a sibling test module because
    tests/conftest.py does not exist. Known and accepted; creating it is a separate change.
  • Gate error paths (exit 2) carry no denominator, and cf-exemptions' fold-in-failure path
    bypasses print_verdict entirely, so it can still print a bare label on that one red path.
  • cf-exemptions / cf-no-bon-ref report a denominator but not the frozen_count ratchet they also
    enforce; that ratchet prose rides human notices, which by design never reaches the wire.
  • GateVerdict.notices' docstring in errors.py still describes the field generically and was not
    updated to say it now holds the denominator.

Verification

ruff, ruff-format, mypy, complexipy and all seven cf-* gates are green. pytest is
574 passed, 3 errors; the 3 errors are tests/test_packaging.py only and are pre-existing on
origin/main
, reproduced there at 553 passed, 3 errors in an independent virtualenv — the
authoring machine lacks python3.12-venv/ensurepip and those tests build a clean venv. CI has
ensurepip and is the authority.

No Constable has run on this change.

Antawari and others added 3 commits July 29, 2026 13:33
The board printed eleven bare labels and one measurement. complexipy was the
only honest line because it alone runs in-process and builds its GateVerdict
directly; every other gate crossed a subprocess boundary and the measurement
did not survive the crossing.

Three breaks, not one:

1. print_verdict built GateVerdict(gate, violations, error) and never passed
   notices or evidence. `notices` was a print-only parameter, so the wire
   shipped "notices": [], "evidence": {} for every gate — empty at the source.
2. _structured_verdict never read either field back off the parsed JSON.
3. No cf-gate computed a denominator at all. None counted what it examined,
   so closing (1) and (2) alone would still have printed a bare label.

print_verdict now takes `measured` (one line, rides GateVerdict.notices to the
wire) and `evidence` (the same measurement in machine form). The pre-existing
`notices` parameter stays the gate's multi-line human prose and deliberately
does NOT reach the wire, so a board line stays one line. _structured_verdict
reads both back. Each of the seven cf-gates now threads a count out of the walk
it actually performed — never a second walk, never a number the gate did not
use. cf-file-budget additionally stops hand-rolling its own JSON and routes
through the shared print_verdict, so it finally honours CF_QUALITY_JSON.

A PASS with no denominator is indistinguishable from a PASS that examined
nothing, so zero is the case that had to work: every gate renders a visible 0
on an empty tree, proven by control-rod tests rather than asserted on
hand-built verdicts.

gate_runner.py stays at exactly 500 lines, the file ceiling.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The first pass gave every cf-gate a denominator. Three of them counted the
wrong thing, so the line was confident and false — worse than a bare label,
because an operator can act on it.

cf-no-bon-ref counted files the walk offered, including the binaries it
skips unread. A tree of one PNG reported "swept 1 file(s)" having opened
nothing. scan_file now returns whether it READ the file, and the sweep
counts only reads.

cf-exemptions counted surface BASES. A declared source_root, or a present
src/, is always exactly one base — so three Python files and zero Python
files produced byte-identical lines. That is a gate selecting by the value
it guards, reproduced inside the rung meant to cure it. The surface is now
resolved once into a file list, threaded into the scanner, and the count is
that very walk.

cf-import-contract claimed one surface covered another ("linted N clause(s)
over M module(s)"), fusing PASS 3's AST walk into a sentence about PASS 1,
and shipped two different evidence schemas from its two branches. Both
branches now speak one schema and the line names each surface separately.

cf-file-budget reported only its frozen FILE entries and stayed silent on
the package ceilings it also enforces; both baselines now ride the evidence.

Tests assert each zero against the SHAPE that hides the defect — one
unreadable binary, a src/ holding no Python — because an empty directory
drives every counting bug to zero and proves nothing.
A refuter lens claimed three denominators were vacuous. I verified it by
mutation rather than by argument: hardcoding cf-recursion-check,
cf-sticky-check and cf-mirror-check to report 0 left the whole suite GREEN,
and the board then printed "examined 0 CLAUDE.md" on a repo that has one and
"walked 0 Python file(s)" over a 19-file tree. Confident and false — the
exact defect this branch exists to cure, reproduced inside it. Each of those
three had a zero assertion and no non-zero counterpart, so the zero was
proving nothing.

Each now has the other half of the rod: a known count over real content.
Re-running the same four mutants after the change kills all four.

The previous commit message overstated its fixtures: it claimed every zero
used "the SHAPE that hides the defect" when three were bare empty
directories, which drive every counting bug to 0. cf-recursion-check and
cf-file-budget now use a present tree holding no Python.

cf-file-budget's denominator was also untrue: it called the whole baseline
"frozen file entry(ies)", but a declared-not-banned entry carries a purpose
and no line count, and _check_file treats it exactly like an undeclared
file. It claimed a shrink-only ratchet over files that have none. The two
kinds are now counted and named separately.

Finally, the aggregated board itself had no guard: nothing failed if a cf-*
gate reached it carrying no measurement, because every other denominator
test drives one gate in-process. The full-battery test now asserts every
cf-* verdict crosses the real subprocess wire with exactly one notice and a
non-empty evidence dict, which also makes the runner's silent
unparseable-output fallback fatal for a kit gate.
@Antawari
Antawari merged commit 74dc393 into main Jul 30, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant