Skip to content

guard: suite count ignores unmerged-stage duplicates (was inflating mid-merge) - #759

Open
matt82198 wants to merge 1 commit into
mainfrom
guard/count-ignores-unmerged-stages
Open

matt82198 wants to merge 1 commit into
mainfrom
guard/count-ignores-unmerged-stages

Conversation

@matt82198

Copy link
Copy Markdown
Owner

The finding

git ls-files <pattern> lists an unmerged path once per index stage (1=base, 2=ours, 3=theirs). tools/verify_test_suite_count.py counted git's output lines, so during an in-progress merge a single conflicted tests/test_*.py was counted two or three times. --regenerate then wrote that inflated number into tests/CLAUDE.md, and only the pre-push gate caught it — the conflict sweep had to correct it twice (PRs #710, #711).

Repro (red first)

Hermetic temp repo, one conflicted + one clean Python suite, merge left in progress:

$ git ls-files 'tests/test_*.py'
tests/test_conflict.py
tests/test_conflict.py
tests/test_conflict.py
tests/test_stable.py            <- 4 lines, 2 files

$ git ls-files --unmerged
100644 df967b96... 1  tests/test_conflict.py
100644 ba2906d0... 2  tests/test_conflict.py
100644 2299c379... 3  tests/test_conflict.py

Against the unfixed tool: --regenerate wrote **Python (4 suites)**: and --check reported actual is 4.

TestUnmergedStageDeduplication before the fix: 4 failed / 4 passed. After: 48/48 pass across both suite-count test files.

Fix

list_git_files() collects paths into a set; count_git_files() returns len() of it.

Dedupe mechanism chosen: Python-side set, not git ls-files --deduplicate. --deduplicate was verified working on the git here (2.54.0) but it needs git >= 2.31, and — more importantly — it only fixes the stage case. The set form also fixes a second latent double-count: the shell family is derived from three globs and one path can match two of them (tests/test_x.test.sh matches both tests/*.test.sh and tests/test_*.sh). That is covered by its own test.

Line parsing moved from stdout.strip().split("\n") to splitlines() so a path is never mangled by stripping.

Should --regenerate refuse while a merge is in progress?

Decision: WARN loudly, do not refuse.

Refusal was the tempting answer — a derived count from a half-merged tree is arguably meaningless. But conflict resolution is precisely when suite counts drift and precisely when the sweep must run --regenerate; the merge automation calls the tool in exactly that window, so a refusal would break the workflow that needs this tool most, and would push operators toward editing the count by hand.

The actual failure mode was not "ran during a merge" — it was a silently wrong number. Per-unique-path derivation removes the wrongness, and a loud stderr [WARN] naming MERGE_HEAD removes the silence, telling the operator the tree is only half resolved and to re-run after the merge concludes. Both --check and --regenerate warn; exit codes are unchanged.

Guards preserved

No weakening. The exactly-one count-line assertion, the homoglyph/fence/HTML-comment masking, the not-a-git-work-tree check, the git-failure fail-closed, and the per-family vacuous-zero guard all still fail closed — the last has an explicit regression test in the new class (test_vacuous_zero_guard_still_fails_closed). The merge probe itself can never break the gate: it is check=False and swallows to False, since ensure_git_repo() already fails closed on an unusable git.

Sibling tools checked (no action needed)

tools/verify_test_coverage.py and tools/ci_shard_runner.py both already collect git ls-files output into a set(), so neither is exposed to this. verify_test_suite_count.py was the only one counting raw lines.

Also in this PR

Regenerates a pre-existing Python count drift on main (238 -> 239) so the gate is clean, plus the INDEX: docstring line and the resulting tools/INDEX.md regeneration.

Verification

  • pytest tests/test_verify_test_suite_count.py tests/test_verify_test_suite_count_prepush.py -> 48 passed
  • Adjacent guard suites (test_ci_workflow_lint, test_test_hygiene, test_no_bare_test_functions, test_cli_help_hygiene, test_subprocess_guard, test_traps) -> 131 passed
  • Full Python suite via tools/ci_shard_runner.py 0 1
  • secret_scan.py --staged -> exit 0 (CLEAN: 4 files scanned)
  • encoding_lint.py --check, import_resolution_check.py, claudemd_lint.py, claudemd_sync_gate.py --check, verify_test_coverage.py --check -> all exit 0
  • bash hooks/pre-push-policy.sh --test -> 18/18 passed
  • verify_test_suite_count.py --check -> [OK] Test suite counts match

🤖 Generated with Claude Code

…id-merge)

`git ls-files <pattern>` lists an UNMERGED path once per index stage
(1=base, 2=ours, 3=theirs). verify_test_suite_count.py counted output
LINES, so during an in-progress merge a single conflicted
`tests/test_*.py` was counted two or three times, `--regenerate` wrote
that inflated number into tests/CLAUDE.md, and only the pre-push gate
caught it. The conflict sweep had to correct it twice (PRs #710, #711).

Repro (hermetic temp repo, one conflicted + one clean python suite):
raw `git ls-files 'tests/test_*.py'` emits 4 lines for 2 files, and the
unfixed `--regenerate` wrote `**Python (4 suites)**:`.

Fix: list_git_files() collects paths into a set. Deduplicating in Python
rather than via `git ls-files --deduplicate` (git >= 2.31; verified
supported on the local git 2.54 but not portable to every builder) keeps
this correct on any git AND additionally fixes cross-pattern double
counting -- the shell family is derived from three globs, and
`tests/test_x.test.sh` matches both `tests/*.test.sh` and
`tests/test_*.sh`.

Merge-in-progress decision: WARN, do not refuse. A refusal was
considered, but conflict resolution is exactly when counts drift and
exactly when the sweep needs `--regenerate` (the merge automation calls
it there), so refusing would break the workflow that needs the tool
most. The actual hazard was a SILENT wrong number; per-unique-path
derivation removes the wrongness, and a loud stderr `[WARN]` naming
MERGE_HEAD removes the silence. Fail-closed guards are untouched: the
exactly-one count-line assertion and the per-family vacuous-zero guard
both still fail closed (covered by a regression test).

Also regenerates a pre-existing Python count drift on main (238 -> 239)
so the gate is clean.

Tests: 8 new cases in TestUnmergedStageDeduplication. Red first --
4 failed / 4 passed before the fix; 48/48 pass after.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@matt82198 matt82198 added the merge-queue Queued for the merge-queue advancer daemon label Aug 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-queue Queued for the merge-queue advancer daemon

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant