Skip to content

Put the complexity floor out of the gate's reach, including via consumer config - #27

Merged
Antawari merged 2 commits into
mainfrom
catrina/gates-r2-complexipy-write-free
Jul 29, 2026
Merged

Put the complexity floor out of the gate's reach, including via consumer config#27
Antawari merged 2 commits into
mainfrom
catrina/gates-r2-complexipy-write-free

Conversation

@Antawari

Copy link
Copy Markdown
Contributor

CONTRACT

The complexity ratchet compares the tree against a committed complexipy-snapshot.json and fails on regression. The floor is the whole value of the gate.

WORLD — a PASSING run silently emptied the floor and exited 0

Root cause, read in the pinned tool's own source: complexipy/utils/snapshot.py::handle_snapshot_watermark calls create_snapshot_file on its no-violation branch. The tool's green path is its destructive path. Reproduced two ways, both exit 0:

### A: snapshot has 1 entry; run ONLY the low-complexity file
before 1
exit=0
after 0

### B: restore, then raise -mx above everything
before 1
exit=0
after 0

It is not hypothetical. Two working trees on the maintainer's machine already hold a destroyed floor:

########## .worktrees/bonfire__04 ##########
  COMMITTED at HEAD: entries=12 functions=13
  ON DISK NOW:       entries=0 functions=0  raw='[]'
########## .worktrees/bonfire__05 ##########
  COMMITTED at HEAD: entries=12 functions=13
  ON DISK NOW:       entries=0 functions=0  raw='[]'

One git add from permanent, on two branches.

The fix, and the two ways configuration walked around the first attempt

Measurement runs write-free (--plain --color no --snapshot-ignore) and the ratchet is graded here as a pure function of (committed floor, measured census). Detection no longer depends on the tool destroying the artifact first, and there is no window where the write happened and was undone. Verified against the real tool on a 142-file consumer: 681 stdout rows, every one a census row, 0 bytes of stderr, committed snapshot byte-unchanged.

Then two refuters found configuration defeats it.

Blocker 1 — a consumer's own complexipy config re-opens the destructive branch

snapshot-create resolves CLI-first, then TOML (utils/toml.py:235-240). The kit passes no CLI value and --snapshot-create is declared with no negating secondary name (main.py:97-102), so a committed snapshot-create = true wins and handle_snapshot_file_creation (main.py:323) rewrites the floor — a branch --snapshot-ignore never touches (main.py:332).

Neutralising it is provably impossible without breaking the design: the tool derives the snapshot path (main.py:321) and every reported path (main.py:308-310) from the same os.getcwd(), so moving the cwd re-keys the entire census and drops the legitimately-honoured max-complexity-allowed.

So the gate reads the consumer's config and REFUSES. Sixteen keys, including three the refuters did not name (ratchet, failed/details, report-ignored). The search order mirrors the tool's own (utils/toml.py:135-148), including that an empty complexipy.toml shadows a populated pyproject.toml.

Blocker 2 — a raised threshold emptied the offender set and the ratchet graded nothing, green

The rule only ever iterated offenders, and the vacuity guard watched the census, which a raised bar leaves full. So with max-complexity-allowed = 100, a function at 33 could reach 90 and the stage reported clean — the exact defect this rung exists to fix, re-entered inside the fix.

Closed without a second threshold authority, from data already in hand: a floor function whose measured value is at or above its committed watermark yet absent from the offender set proves the bar moved since the floor was booted. The tool keeps sole say over what an offender is, so a consumer who booted at their own budget is still graded at it. measurement_argv still carries no -mx.

FAILURE — the rods, driven against the REAL pinned tool

===== ROD 1  [tool.complexipy] snapshot-create = true  -> MUST REFUSE =====
  RAISED GateError -> the battery records it and exits 2
  error code : GATE_COMPLEXIPY_CONFIG_DEFEATS_MEASUREMENT
  message    : pyproject.toml sets complexipy option(s) this gate cannot measure through:
               snapshot-create (would make the run WRITE complexipy-snapshot.json from what
               it just measured (main.py:323, a branch --snapshot-ignore does not disarm))
               — remove them from pyproject.toml; max-complexity-allowed and exclude ARE
               honoured, so a real budget or a real exclusion needs no workaround here
  FLOOR UNTOUCHED: YES

===== ROD 2  max-complexity-allowed = 100, watermark 16, real 22 -> MUST NOT BE GREEN =====
  error code : GATE_COMPLEXIPY_THRESHOLD_RAISED
  message    : 1 committed watermark(s) are at-or-below what complexipy now calls acceptable,
               so its offender set no longer covers the floor — the ratchet would grade
               nothing and report clean.
  passed     : False
  FLOOR UNTOUCHED: YES

===== ROD 3  genuine shrink (real 0) below watermark 16 -> MUST STAY GREEN =====
  violations : []
  passed     : True
  FLOOR UNTOUCHED: YES

===== ROD 4  watermark 16, real 22, DEFAULT bar -> ordinary regression, MUST BE RED =====
  violations : ['COMPLEXIPY_WATERMARK_REGRESSION']
    - hairy rose above its committed watermark: 16 -> 22 — bring it back to 16 or below;
      raising the floor instead is a deliberate, reviewed act: re-boot the floor from the
      repo root: complexipy s...
  passed     : False
  FLOOR UNTOUCHED: YES

Rods 3 and 4 are the control rods on the rods: rod 2's red is attributable to the raised bar, not to a blanket "everything is red now", and a genuine improvement still passes. In all four, the floor is untouched.

Two structural rods: reverting to the pre-fix stage turns 12 tests red; removing --snapshot-ignore from the argv turns 2 red by name ("the committed floor must be out of reach").

A note on my own first attempt at rod 2: I set the watermark to 33 while the fixture measured 22, so the gate correctly read a shrink and passed. The rod was mis-specified, not the gate. Corrected to watermark 16 / real 22, which is the only shape in which "the bar moved" is provable.

Also closed (each with rods)

  • The instrument's exit code and stderr were discarded, so every tool-side failure was re-attributed to the repo with the reason deleted.
  • A non-UTF-8 ledger raised UnicodeDecodeError past the stage and killed the whole 12-gate board instead of the typed refusal it promised. Closed for the snapshot, the complexipy config, and the parent-side subprocess decode.
  • A Python-free source_root beside real code reported PASS having measured nothing — both vacuity legs consulted the same possibly-wrong surface. The repo-wide answer is threaded in, and the third tree-pruning rule that caused it is deleted.
  • A # complexipy: ignore comment voided a watermark silently — now caught per function, not per file.
  • A symlinked module inside the tree was accused of lying outside it.
  • Duplicate function keys collapsed last-wins — confirmed live on the real consumer: 681 census rows → 680 keys (FrontDoorServer::on_message twice). Under-reported the count and could fake a skew. Now max()-aggregated with rows counted separately.
  • A cross-repo mirror of the tool's normalize_path was undeclared; a test now pins it against the installed function, which beats a declaration file nobody gates.
  • The measured count is evidence, so it stops being a print to stderr the workflows never tee and rides the verdict and the board: PASS complexipy — measured 233 function(s) in 16 file(s) against a 0-function committed floor.

Documents that asserted the opposite

Three files said "the snapshot does NOT auto-shrink" and "only --snapshot-create rewrites it." Both halves are false for the pinned version, and tests/test_configs.py pinned the false sentence in place. Corrected, with the version the behaviour was re-measured on, and the test now pins the true rule and asserts the false phrasing cannot return.

Worse, and fixed: README.md's day-one boot command was complexipy src — no --snapshot-create, so it creates nothing, and once a floor exists it runs the rewriting compare. A live destructive instruction.

An integration fixture that made a test pass for the wrong reason

test_injected_type_error_flips_the_battery_red wrote over the consumer module, deleting its only function. The stricter vacuity leg then correctly refused for having measured nothing, and the battery went red at exit 2 rather than 1. It appends now, so mypy is genuinely the sole failure. That also exposed the injected literal as formatter-dirty — ruff-format had been a silent second red gate in a test whose comment claims one.

The gate at harvest

PASS  ruff-check          PASS  cf-exemptions        PASS  mypy
PASS  ruff-format         PASS  cf-no-bon-ref        PASS  complexipy — measured 233 function(s)
PASS  cf-sticky-check     PASS  cf-import-contract   FAIL  pytest
PASS  cf-file-budget      PASS  cf-recursion-check

The single red is environmental, isolated: tests/test_packaging.py calls venv.create(with_pip=True) and ensurepip is absent system-wide on this machine. Excluding that one file: 532 passed. Those tests touch no file in this diff and CI's setup-python provides ensurepipbut I have not seen them pass, and that is stated rather than assumed.

Split by what each part answers: complexipy_floor.py 86 (the artifact, read-only), complexipy_measure.py 466 (the instrument), complexipy_ratchet.py 446 (the rule). gate_runner.py unchanged at exactly 500 — the rule takes a structural read-only Protocol so it cannot import the runner. Every file under the cap; the kit ships no file-budget.json, so nothing to declare.

The divergent candidate is preserved on catrina/gates-r2-complexipy-floor with its reasoning, rather than discarded. It wrapped-and-restored; this branch was preferred because it removes the failure mode instead of compensating for it.

Findings for a maintainer

  1. 🔴 Paired --no-ignore question, deliberately unresolved. Adding it to the gate argv requires the boot command in three documents to carry it too, or every ignored function floods as a new offender. I closed the hole structurally instead, which needs no flag on either side. Your ruling.
  2. _exec's UnicodeDecodeError hole is only closed for complexipy. text=True decodes in the parent, so ruff/mypy/pytest/cf-* keep the same battery-killing hole. Fixing it needs a line in a 500/500 file — a separate rung.
  3. The new modules are absent from the import-linter layers list, so they are unconstrained by the kit's own layering contract. pyproject.toml was fenced; one line each in a follow-up.
  4. Consumer impact to expect: any consumer whose config carries one of the sixteen keys now goes red at exit 2 with the key named — intended. A consumer whose exclude covers a floor file will now see SNAPSHOT_FILE_UNMEASURED.
  5. The LawGiver pre-gate could not run on this repo (pregate_ledger: file-budget.json is missing) — it requires a ledger the kit legitimately does not have, since the kit has no over-500 offenders to baseline. Same family as its grading markdown by the .py size law.

Co-Authored-By: Claude Opus 5 (1M context) noreply@anthropic.com

Antawari and others added 2 commits July 28, 2026 21:41
The complexity ratchet compares the tree against a committed watermark file and
fails on regression. A PASSING run silently emptied that file and exited 0, so
the floor vanished and every later run compared against nothing.

Root cause, read in the pinned tool's own source: handle_snapshot_watermark
calls create_snapshot_file the moment it finds no violation. The tool's green
path IS its destructive path. Reproduced two ways, both exit 0: measuring a
narrower surface than the floor describes, and raising the threshold above every
function.

It is not hypothetical. Two working trees on the maintainer's machine already
hold a floor of 12 entries / 13 functions committed and a literal [] on disk,
one stage away from permanent.

So the gate no longer lets the tool near the file. Measurement runs write-free
(--snapshot-ignore switches the compare, and its rewrite, off), and the ratchet
is graded here as a pure function of the committed floor and the measured
census. Detection no longer depends on the tool destroying the artifact first,
and there is no window where the write happened and was undone.

The census also makes the run answerable about what it graded. Every verdict
carries measured_functions, measured_files and the floor size, and a run that
measured nothing while there was something to grade REFUSES rather than
reporting clean. A floor file that still exists but went unmeasured is a
finding, not a shrug: that is the narrowed-surface case, caught without
reference to any threshold.

Threshold authority stays with the tool, so a consumer who booted at a
non-default value is graded at that value and not at ours.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…loor

The first pass put the floor out of the gate's reach by measuring write-free.
Two refuters then showed configuration walks around it.

A consumer's own complexipy config re-opens the destructive branch. snapshot-create
resolves CLI-first then TOML, the kit passes no CLI value, and --snapshot-create has
no negating secondary name, so a committed snapshot-create = true wins and
handle_snapshot_file_creation rewrites the floor from what the run just measured --
a branch --snapshot-ignore never touches. Neutralising it is impossible without
breaking the design, because the tool derives the snapshot path AND every reported
path from the same working directory, so moving the cwd re-keys the whole census.
So the gate reads the consumer's config and REFUSES, naming the key, the call site
that would fire, and the remedy -- and it names the keys that ARE honoured, so a
real budget or a real exclusion never needs a workaround. Sixteen keys, including
three the refuters did not name, and the search order mirrors the tool's own down
to an empty config file shadowing a populated one.

A raised threshold emptied the offender set and the ratchet graded nothing, green.
The rule only ever iterated offenders, and the vacuity guard watched the census,
which a raised bar leaves full. A function at 33 could reach 90 unremarked. Closed
without a second threshold authority, from data already in hand: a floor function
whose measured value is at or above its committed watermark, yet absent from the
offender set, proves the bar moved since the floor was booted. The tool keeps sole
say over what an offender is, so a consumer who booted at their own budget is still
graded at it.

Also closed: the instrument's exit code and stderr were discarded, so every
tool-side failure was re-attributed to the repo with the reason deleted. A non-UTF-8
ledger raised past the stage and killed the whole board instead of the typed refusal
it promised. A Python-free source root beside real code reported PASS having measured
nothing, because both vacuity legs consulted the same possibly-wrong surface. An
ignore comment voided a watermark silently, now caught per function rather than per
file. A symlinked module inside the tree was accused of lying outside it. Duplicate
function keys collapsed last-wins, under-reporting the measured count and able to
fake a skew.

The count is evidence, so it stops being a print to stderr the workflows never tee
and rides the verdict and the board instead. Every violation and refusal now names a
remedy.

An integration fixture wrote OVER the consumer module to inject a type error,
deleting its only function; the stricter vacuity leg then refused for having measured
nothing and the battery went red for the wrong reason. It appends now. That also
exposed the injected literal as formatter-dirty, so ruff-format had been a silent
second red gate in a test that claims one.

Three documents asserted the snapshot does not shrink itself and only
--snapshot-create rewrites it. Both halves are false for the pinned version, and one
test pinned the false sentence in place. The README's day-one boot command was worse
than stale: it named the rewriting compare rather than the creating flag.

Split by what each part answers -- the artifact, the instrument, the rule -- because
the runner is at its line limit and the rule may not import the runner.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@Antawari
Antawari merged commit d8d575a into main Jul 29, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant