Skip to content

fix(falsify): two falsifications that proved nothing, and the budget the first finished night measured - #788

Merged
stephrobert merged 2 commits into
mainfrom
fix/781-falsifications-that-depend-on-timing
Sep 21, 2026
Merged

stephrobert merged 2 commits into
mainfrom
fix/781-falsifications-that-depend-on-timing

Conversation

@stephrobert

Copy link
Copy Markdown
Owner

Summary

The suite has been red ever since #737 stopped it being cancelled — which is the mechanism working: it now finishes and says something. It named two guards as no longer biting, and neither guard had stopped working. Both verdicts came from the falsification rather than from the subject, each in a different way.

touch-attribution was non-deterministic

barrierPack's comment claims the two handlers "hold each other in flight for the entire window in which they touch the store". Its single barrier only synchronised them before the first touch. A handler that finished its touches and returned left the flight before the other touched anything, so the other attributed correctly even with the goroutine check removed.

mutation detected
before 3 runs out of 5 missed it
after 5 out of 5

A second barrier holds both requests in flight until both have touched — what the sentence always claimed, and what the guard actually has to survive.

probe-side-zeros aimed at an axis the record can no longer contradict

The mutation declared shape client-borne, and the test looks for undriven operations that earned it. Measured on the committed record:

axis undriven operations that earn it
contract 21
probed 21
shape 0

There was no discrepancy to find, so the mutation proved nothing. It now aims at contract.

The non-vacuity guard that should have caught this is global — one witnessed axis satisfies it — so flipping a different axis passed underneath it. It now also reports, per axis, what this record cannot judge:

this record cannot contradict a declaration on 1 axis/axes, because no
undriven operation earns them: shape. A falsification aimed at one of
those proves nothing.

Logged rather than failed, deliberately: an axis no undriven operation earns is an ordinary state of the record, not a defect, and failing on it would be a red nobody can clear. What is not ordinary is nobody noticing.

The budget moves to 210 minutes

Each raise is measured, not chosen:

30 cancelled this job for twenty-four consecutive nights (#737)
150 a local projection of 97 minutes
145 what the first night that actually finished took (04:30 → 06:55, 2026-09-20)

Five minutes of margin, on a suite that grew from 199 to 207 specs in that same week, turns the next dozen specs into a cancelled night.

a-silent-night-is-refused was retargeted at the new budget rather than left pointing at a line that no longer exists — which falsify:lint refused, correctly.

Type of change

  • Bug fix
  • Chore / tooling

Checklist

Always

  • mise run check passes, and mise run prepush in full — including falsify:lint, which caught the stale fragment above
  • No new external Go dependency
  • internal/core still knows no provider — the change in internal/core/emulator is in a test, and adds a second sync.WaitGroup to a stub pack
  • Nothing in the diff could be written identically for another provider
  • mise run testplan was run. conformance is not played: no route, no handler and no response shape changed. Two test files, one spec, one workflow budget. The falsifications themselves are the proof this change is about, and each was replayed five times.

When a model wrote a substantive part of this

  • An Assisted-by: trailer names the tool and the model version
  • N/A on mise run conformance, for the reason stated just above
  • No field name is involved; the axis names come from evidence_axes.go and the witness counts from coverage/evidence.json, both read directly

When a route is added or changed

N/A.

When behaviour a client can observe changes

N/A — nothing a client sees changes.

When .github/workflows/ is touched

  • Every action pinned to a full 40-character SHA — unchanged by this diff
  • step-security/harden-runner is the job's first step — unchanged
  • permissions: least-privilege — unchanged
  • No job renamed
  • The exit-code contract holds — only timeout-minutes moves

When a machine runtime is involved

N/A.

Related issues

Should close #781 once a night finishes green. The issue is a scheduled-red one, so the first green night closes it automatically.

🤖 Generated with Claude Code

…the first finished night measured

The suite has been red since it stopped being cancelled. It named two guards as
no longer biting, and **neither guard had stopped working**. Both verdicts came
from the falsification, not from the subject, and each failed in a different way.

**touch-attribution was non-deterministic.** `barrierPack`'s comment claims the
two handlers "hold each other in flight for the entire window in which they
touch the store". Its single barrier only synchronised them BEFORE the first
touch: a handler that finished its touches and returned left the flight before
the other touched anything, so the other attributed correctly even with the
goroutine check removed. Measured on 2026-09-21, the mutation went undetected in
three runs out of five. A second barrier now holds both requests in flight until
both have touched, which is what the sentence always claimed. Five runs out of
five bite.

**probe-side-zeros aimed at an axis the record can no longer contradict.** The
mutation declared `shape` client-borne, and the test looks for undriven
operations that earned it. The committed record holds 21 undriven operations and
**not one** carries a recorded answer, so there was no discrepancy to find. The
mutation now aims at `contract`, which has 21 witnesses.

The non-vacuity guard that should have caught this is global — one witnessed
axis satisfies it — so flipping a different axis passed underneath it. It now
also reports, per axis, which ones this record cannot contradict:

    this record cannot contradict a declaration on 1 axis/axes, because no
    undriven operation earns them: shape. A falsification aimed at one of
    those proves nothing.

Logged rather than failed, on purpose: an axis no undriven operation earns is an
ordinary state of the record, not a defect, and failing on it would be a red
nobody can clear. What is not ordinary is nobody noticing.

**The budget moves to 210 minutes.** 30 cancelled this job for twenty-four
nights (#737); 150 came from a local projection of 97 minutes; the first night
that actually finished took **145** (04:30 to 06:55 on 2026-09-20), on a suite
that grew from 199 to 207 specs in the same week. Five minutes of margin turns
the next dozen specs into a cancelled night.

Both specs bite again, and `a-silent-night-is-refused` was retargeted at the new
budget rather than left pointing at a line that no longer exists — which
`falsify:lint` refused, correctly.

Assisted-by: Claude Code (claude-opus-5)
@stephrobert
stephrobert merged commit 0034187 into main Sep 21, 2026
39 of 40 checks passed
@stephrobert
stephrobert deleted the fix/781-falsifications-that-depend-on-timing branch September 21, 2026 08:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Red scheduled night: Falsify

1 participant