Summary
The claude-pr-review quality gate systematically false-BLOCKs release PRs (and any PR whose diff is a CHANGELOG that documents already-merged commits). The review model can't see the commits the changelog describes — they're in the PR's base branch, not in the PR diff — so it concludes the entries are "fabricated" and emits a blocking finding.
Concrete incident
glitchwerks/claude-configs PR #932 (chore(release): v0.18.0). Diff = CHANGELOG.md + pyproject.toml only. The changelog correctly documented 6 commits merged to main since v0.17.0. The review returned:
- 🔴 Critical — "fabricated CHANGELOG entries" (only #927 is "the single actual change")
- 🟡 High-Priority — "version should be PATCH not MINOR"
- Verdict: BLOCK
Both findings were provably wrong: git log v0.17.0..main showed all 6 commits, git tag --contains was empty for each (all unreleased), and four touched agents/**/skills/** (MINOR-triggering). The release was correct; it had to be admin-override-merged.
Root cause
This is a context-scoping gap, not a gate-logic bug. The gate scripts (pr-review/lib/severity-regex.sh, lib/parse-marker.sh) correctly counted a marker the model produced. The defect is upstream in pr-review/action.yml:
- For
opened/reopened PRs the diff instruction is "Use gh pr diff <PR_NUMBER> to retrieve the full PR diff."
- For a release PR,
gh pr diff returns only the changelog + version bump — by definition, because the commits it documents are already in the base branch.
- The model is handed a changelog claiming N entries but a 2-file diff, with no instruction to validate entries against
git log <last-tag>..HEAD. It defaults to "fabricated."
This recurs on every multi-commit release PR.
Chosen fix (option 1): skip / make-advisory the gate for release PRs
Detect a release-shaped PR and do not let the quality gate block it:
- Detection — PR title matches
^chore\(release\): (the repo's release-commit convention), OR the PR diff touches only changelog/version files (CHANGELOG.md, pyproject.toml, */VERSION, package.json version line).
- Behavior — skip the
claude-pr-review/quality-gate (and shadow) status, or post it as a neutral/success advisory state, so a release PR is never blocked by a finding the review can't substantiate from the diff.
This is the cleanest stop. (Alternatives considered but not chosen: feeding git log $(git describe --tags --abbrev=0)..HEAD into the review context; or a prompt instruction to validate changelog entries against the tag range before flagging fabrication. Either would also work but is more invasive than skipping a gate that has no meaningful diff to review.)
Implementation pointers
pr-review/action.yml — the "Resolve diff base" step (~L48–89) and the diff-instruction rendering (~L118–137); add the release-PR short-circuit before the gate steps (~L335–393 authoritative gate, ~L395–534 shadow gate).
Release/propagation note
claude-configs consumes this via uses: glitchwerks/github-actions/.github/workflows/claude-pr-review.yml@v2. The fix won't reach consumers until a v2.x.y is cut and the floating v2 tag is bumped (see #145 / #181 for that process).
Related (distinct root causes)
This issue is specifically the release/changelog-PR diff-context gap.
🤖 Generated by Claude Code on behalf of @cbeaulieu-gt
Summary
The
claude-pr-reviewquality gate systematically false-BLOCKs release PRs (and any PR whose diff is a CHANGELOG that documents already-merged commits). The review model can't see the commits the changelog describes — they're in the PR's base branch, not in the PR diff — so it concludes the entries are "fabricated" and emits a blocking finding.Concrete incident
glitchwerks/claude-configsPR #932 (chore(release): v0.18.0). Diff =CHANGELOG.md+pyproject.tomlonly. The changelog correctly documented 6 commits merged tomainsincev0.17.0. The review returned:Both findings were provably wrong:
git log v0.17.0..mainshowed all 6 commits,git tag --containswas empty for each (all unreleased), and four touchedagents/**/skills/**(MINOR-triggering). The release was correct; it had to be admin-override-merged.Root cause
This is a context-scoping gap, not a gate-logic bug. The gate scripts (
pr-review/lib/severity-regex.sh,lib/parse-marker.sh) correctly counted a marker the model produced. The defect is upstream inpr-review/action.yml:opened/reopenedPRs the diff instruction is "Usegh pr diff <PR_NUMBER>to retrieve the full PR diff."gh pr diffreturns only the changelog + version bump — by definition, because the commits it documents are already in the base branch.git log <last-tag>..HEAD. It defaults to "fabricated."This recurs on every multi-commit release PR.
Chosen fix (option 1): skip / make-advisory the gate for release PRs
Detect a release-shaped PR and do not let the quality gate block it:
^chore\(release\):(the repo's release-commit convention), OR the PR diff touches only changelog/version files (CHANGELOG.md,pyproject.toml,*/VERSION,package.jsonversion line).claude-pr-review/quality-gate(and shadow) status, or post it as a neutral/success advisory state, so a release PR is never blocked by a finding the review can't substantiate from the diff.This is the cleanest stop. (Alternatives considered but not chosen: feeding
git log $(git describe --tags --abbrev=0)..HEADinto the review context; or a prompt instruction to validate changelog entries against the tag range before flagging fabrication. Either would also work but is more invasive than skipping a gate that has no meaningful diff to review.)Implementation pointers
pr-review/action.yml— the "Resolve diff base" step (~L48–89) and the diff-instruction rendering (~L118–137); add the release-PR short-circuit before the gate steps (~L335–393 authoritative gate, ~L395–534 shadow gate).Release/propagation note
claude-configsconsumes this viauses: glitchwerks/github-actions/.github/workflows/claude-pr-review.yml@v2. The fix won't reach consumers until av2.x.yis cut and the floatingv2tag is bumped (see #145 / #181 for that process).Related (distinct root causes)
github-actions[bot]workflows post on a PR #184 — gate selects wrong comment when multiple bot workflows post. Different cause.This issue is specifically the release/changelog-PR diff-context gap.
🤖 Generated by Claude Code on behalf of @cbeaulieu-gt