Skip to content

guard: move suite counts to generated artifact - #776

Open
matt82198 wants to merge 2 commits into
mainfrom
guard/generated-suite-counts
Open

matt82198 wants to merge 2 commits into
mainfrom
guard/generated-suite-counts

Conversation

@matt82198

Copy link
Copy Markdown
Owner

Summary

Structural fix: Moves hand-maintained test suite counts from tests/CLAUDE.md into generated tests/SUITE-COUNTS.json artifact. Eliminates merge-conflict serialization that struck the PR board 2026-08-03 when 4+ concurrent PRs all edited the same count lines.

Problem

  • Hand-maintained count lines (**Python (240 suites):**) in tests/CLAUDE.md
  • Every PR adding/removing tests must edit same lines
  • When 4+ PRs in flight → ALL editing same line → merge conflicts
  • Incident 2026-08-03: Turned 4 green PRs red simultaneously, serialized board

Solution

  • New tool: tools/gen_suite_counts.py generates JSON from git ls-files (deterministic)
  • Artifact: tests/SUITE-COUNTS.json (registered in generated_paths.py)
  • tests/CLAUDE.md now references artifact, no volatile counts
  • verify_test_suite_count.py updated to wrapper (delegates to gen_suite_counts.py)
  • Registered in merge_queue.py REGENERATORS
  • All existing gates continue to work (backward compatible)

Fail-Closed Proofs

  • test_gen_suite_counts.py: 8 tests prove drift detection + regeneration + idempotency
  • test_list_test_suites.py::test_counts_match_verify_gate: validates against generated artifact
  • Exit codes: 0=clean, 1=drift, 2=cannot-evaluate (non-git-repo, git failure)

Result

Adding/removing tests no longer conflicts with sibling PRs. Merge serialization eliminated.

🤖 Generated with Claude Code

Problem: Hand-maintained count lines in tests/CLAUDE.md serialize PR merges.
- Every PR that adds/removes tests must edit the same lines
- When 4+ PRs in flight, all editing count lines → merge conflicts
- 2026-08-03: Turned 4 green PRs red simultaneously

Solution: Generate suite counts in tests/SUITE-COUNTS.json (GEN artifact)
- New tool: tools/gen_suite_counts.py generates JSON from git ls-files (deterministic)
- tests/CLAUDE.md now references the artifact, contains no volatile counts
- Merge conflict class ELIMINATED: generated files bypass serialization
- Fail-closed: counts must match actual files or gate fails (exit 1/2)
- verify_test_suite_count.py updated to delegate to gen_suite_counts.py (wrapper)
- Registered in generated_paths.py + merge_queue.py REGENERATORS

Tests prove fail-closed behavior:
- Drift detection works (exit 1 on mismatch)
- Regeneration is idempotent
- Non-git-repo fails-closed (exit 2)
- Wrapper delegates correctly

Result: Adding/removing tests no longer conflicts with sibling PRs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@matt82198
matt82198 enabled auto-merge August 4, 2026 04:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant