Skip to content

Protocol shapes: the branch-as-step check reads flat numbered protocols - #592

Open
m2ux wants to merge 2 commits into
mainfrom
guard-branch-as-step-reach
Open

Protocol shapes: the branch-as-step check reads flat numbered protocols#592
m2ux wants to merge 2 commits into
mainfrom
guard-branch-as-step-reach

Conversation

@m2ux

@m2ux m2ux commented Sep 3, 2026

Copy link
Copy Markdown
Owner

Summary

We have a mechanical check for caveats written as protocol steps, and it reports a clean run. It
could only ever have parsed some of the corpus. A technique's protocol is written one of two ways —
either with heading-separated phases, or as a flat numbered list directly under the Protocol heading
— and the check only recognised the first. 221 of the corpus's 569 technique files use the flat form,
so it never looked at them.

That is 39% of the technique corpus outside a check whose passing message claims no protocol encodes
a conditional branch as a step. 27 real defects were sitting in the gap.

This teaches it the second shape and pins both with fixtures.

What happens today

The check builds its list of instruction-bearing blocks by opening a new one each time it sees a
phase heading. In a protocol with no phase headings it never opens one, so every line is discarded
and the file contributes nothing. Nothing distinguishes this from a genuinely clean file: the run
prints the same message either way.

That is the failure our own guidance on adding a check warns about — a guard that lands green proves
nothing, because the corpus fix that motivated it merges first and a passing run then cannot be told
apart from a pattern that matches nothing. This one appears to have been calibrated against the
heading-separated shape and its silence on the flat shape was never measured.

The test suite pinned the corpus at a ceiling of eleven known sites, which held the count wherever it
happened to land rather than driving it to zero.

The fix

Open a block on entering the Protocol heading. A flat numbered sequence is then covered, and each
phase heading closes that block and opens the named one, so both shapes are reached.

Keep two carve-outs, deliberately. A complete selection ladder — an "if / else if / otherwise"
chain over one choice — is the branch table the governing catalogue entry exempts, and stands
whichever way it is indented. Indented numbered sub-items also stay out of scope: the numbered form
carries ordered sub-actions and precedence ladders, which is the same exemption. The variable-binding
technique resolves a bind through four numbered branches under a step that declares them a
precedence, and flagging it would contradict the entry.

Prove the reach before the corpus changes. Run against the corpus as it stands, the widened check
names 28 sites — 27 real, plus one selection ladder the carve-out then excludes. The same check
reported a clean run on all 28 beforehand. That measurement is what bounds what it detects, and it
cannot be taken once the corpus is converted.

Pin both shapes and both carve-outs. New fixtures cover a caveat under a flat numbered protocol,
the same caveat correctly written as a note, a selection ladder that must not fire, and two
independent caveats under one step that must both fire.

Assert a clean corpus rather than a ceiling. The corpus assertion now expects zero, so the count
cannot settle where it landed.

Why now is cheap

The reach measurement above is only available while the defects are present. After the paired corpus
change merges, a passing run proves nothing again and establishing what this check detects would mean
checking out an older corpus commit to do it.

Scope

Two files: the check and its test.

Acceptance criteria

  • A caveat under a flat numbered protocol is named
  • The same caveat written as a note is not named
  • A complete selection ladder is not named
  • Two independent caveats under one step are both named
  • The corpus assertion expects a clean corpus

Non-goals

Mutually exclusive branches written as top-level peer bullets remain uncovered. Separating them from
a step that legitimately handles several cases in sequence needs judgement a line pattern cannot
supply, and a check needing a large triage list on the day it lands is bookkeeping rather than
measurement.

The inline spelling of this defect — a condition buried in the step's own sentence — is not
mechanically detectable and is not attempted here. The paired corpus change extends the catalogue
entry to name it so a human audit can.

Paired change

#591 converts the 27 sites. The corpus assertion in this pull request stays red until that merges and
the submodule pointer moves; verified locally against that branch, where the check runs clean and all
11 fixtures pass. The full suite is 33 of 34, with one pre-existing binding-fidelity failure
confirmed identical before and after.

Investigation detail

The blind-spot measurement and the site list:
.engineering/artifacts/planning/2026-09-03-manual-technique-review/

A Protocol takes `### N. Title` phase headings or a flat numbered sequence, and
entering `## Protocol` opens a block so a caveat is reached under both. The flat
shape carries the majority of the corpus — every atlassian-operations,
cargo-operations, gitnexus-operations and knowledge-base-search op is written
that way.

A complete `If` / `Else if` selection ladder stands as the branch table AP-59's
Do-not-flag keeps, and an indented numbered sub-item stays out of scope: the
numbered form carries ordered sub-actions and precedence ladders, which is that
same carve-out.

Fixtures pin both directions of the flat shape, the ladder carve-out, and
independent caveats that are not ladder arms. The corpus assertion is clean
rather than a ceiling, so the count cannot settle where it landed.

Measured against the corpus before its conversion: 28 sites named, one of them a
selection ladder, leaving 27 real. The same guard reported OK on all of them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant