From 1c6796fbc3a6443f0cf106578ea95333c9856d92 Mon Sep 17 00:00:00 2001 From: Claude Code Bot Date: Wed, 25 Feb 2026 18:20:48 -0800 Subject: [PATCH] fix(blocking-review): require line-level evidence for regression BLOCKs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two targeted prompt improvements based on observed false-positive BLOCKs on PR #841/#842: 1. Add EVIDENCE STANDARD section after the PASS/BLOCK criteria: when claiming a test or code path will regress, the reviewer must trace the execution path to a specific failing assertion with line citations. Assumed execution order that isn't verified in the diff must downgrade to PASS + observation. Includes ❌/✓ examples using the actual failure patterns from the incident. 2. Add observation ordering guidance: non-blocking observations should be ordered by maintenance impact (things that cause test drift first, stylistic notes last). The reviewer cannot run tests (claude_args restricts to gh/cat/echo/tee), so confidence about test failures must come from traceable code analysis, not inference. Co-Authored-By: Claude Sonnet 4.6 --- .github/workflows/claude-blocking-review.yml | 22 ++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/.github/workflows/claude-blocking-review.yml b/.github/workflows/claude-blocking-review.yml index 9a1e7c5..849dfd0 100644 --- a/.github/workflows/claude-blocking-review.yml +++ b/.github/workflows/claude-blocking-review.yml @@ -72,6 +72,9 @@ jobs: Use the repository's CLAUDE.md for guidance on style and conventions. Be constructive and specific — cite file names and line numbers where possible. + When listing non-blocking observations, order them by maintenance impact: + issues that will predictably cause test drift or duplicated bugs should + appear before stylistic or organizational notes. ${{ inputs.extra_instructions != '' && format('---\n\nAdditional instructions for this repository:\n\n{0}', inputs.extra_instructions) || '' }} @@ -100,6 +103,25 @@ jobs: - Code organization suggestions - Informational warnings + EVIDENCE STANDARD FOR RELIABILITY REGRESSION BLOCKs: + + You cannot run the test suite. When claiming a test will fail or a code + path will regress, you must trace the execution path from entry point to + the specific assertion or failure point, with line-number citations at each + step. If your trace relies on assumed execution order you have not verified + in the diff or source, downgrade to PASS with a named observation instead. + + ❌ "validateEntitlements() is now called before CLAUDE_API_KEY is checked, + so the 503 test will fail." → Requires verifying actual call order. If + you cannot cite the lines proving that order, issue PASS. + + ✓ "Line 42 calls foo(), which at line 71 sets X to null; the assertion at + line 265 expects X to be non-null — that assertion will fail." → BLOCK. + + ✓ "The test fixture at line 48 does not set REVENUECAT_API_SECRET, but the + handler reads it at line 89 and returns early if absent — the test will + never reach the code path it claims to exercise." → BLOCK. + When uncertain between BLOCK and PASS, default to PASS. You MUST complete ALL of the following steps in order: