Repository navigation
check-pr-fully-clean: skip agent-disclosed replies in the verdict scan - #4922
Conversation
An agent replying to a review under the maintainer's login often quotes the reviewer's "Blocking" label. The verdict scan admitted every non-bot not-clean comment, so the quote became a standing maintainer veto that no later clean review could clear, and the only way out was editing the agent's own comment (measured on Morrison-Lab/mds#599, where two clean Claude reviews of the head could not pass the gate). Skip a comment only on positive evidence that it is a reply: it ends with the Claude Code disclosure footer, carries no review shape the scan admits from a human login (report structure, a named review head) and no review-data payload, and once each disposition label is removed (a bold finding label followed at once by rebutted, deferred, done, ..., or "Addressing the blocking finding") the rest of the body no longer reads as not-clean. The rest is classified as a whole, so a relayed "Needs more work", a second finding after a disposition, or a verdict spread over several lines still blocks. What may follow a disposition is an allowlist, so the skip fails closed: nothing, punctuation or a commit reference after any disposition, or, after rebutted, declined or deferred only, an explanation holding no negation, contrast, modal, open-work word, number or file path. Anything else keeps the whole line, label included. The residual risk is an explanation that states an open defect in none of those words, which is then dropped; that is accepted as the price of not editing the agent's own comments, and is stated here so a reviewer can refuse it. Closes #4826. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019xfc9pMBq15Hq3DgNBx1ox
…lined/deferred
An explanation after a weighed disposition now drops its line only when
it reads as a rebuttal of the reviewer's reading ("The line already
breaks ...", "The reviewer misread ...") and holds no risky word, so
"Persists on the head." and "Agree with the reviewer." keep blocking.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019xfc9pMBq15Hq3DgNBx1ox
Each rebuttal opener is now a closed phrase followed by at most six plain words, and confirmations (valid, true, confirmed, persists, ...) are risky words, so "The finding is already valid." keeps blocking. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019xfc9pMBq15Hq3DgNBx1ox
Six free words after a closed phrase still admitted "The check handles nothing."; only an optional "after/before/at the X" may follow now. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019xfc9pMBq15Hq3DgNBx1ox
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019xfc9pMBq15Hq3DgNBx1ox
|
Claude finished review — View run That matches the saved diff file's length (no additional content I'm missing). I've completed a thorough review of this PR. Review: #4922 — check-pr-fully-clean: skip agent-disclosed replies in the verdict scanSummary of change. Verification performed.
Findings. None. I found no bugs, no CLAUDE.md violations, and no hallucinated APIs/references. The function-length and tidyverse-style sections of the lab manual target R code; this is infrastructure Python in ai-config, where the applicable standard is this repo's own extensive regex-test-driven convention, which the PR follows (comments justify every regex choice, and the new logic is tested from both the clean and not-clean directions). VerdictReady for merge Structured Review Data (JSON){
"schema_version": "1.1",
"reviewer": "claude",
"commit_sha": "56ec81b809d008b9791ce4fd5f92a97dcdafa4a4",
"verdict": "CLEAN",
"findings": [],
"detailed_assessment": "Traced check-pr-fully-clean.py's new _is_agent_disclosed_reply and its disposition-line regexes against the full 1271-test suite (0 failures) and found no logic errors in the allowlist, tail-risk, or rebuttal-phrase matching paths.",
"holistic_assessment": "The change is narrowly scoped to issue 4826, placed correctly relative to the existing bot-author and verdict-supersession branches, more conservative than the issue's own fix sketch, and verified live on mds#599 per the PR description."
}Reviewed commit: 56ec81b |
|
Merging under ai-config's standing An independent adversarial review (a separate Claude subagent) ran nine rounds and returned CLEAN on d862ed5; the only later commit escapes one em dash for the ASCII check. Posted by Claude Code (AI agent) --- not written by a human. Generated by Claude Code |
Closes #4826.
Why
An agent replying to a review under the maintainer's login often quotes the reviewer's "Blocking" label.
check-pr-fully-clean.pyadmitted every non-bot not-clean comment, so that quote became a standing maintainer veto that no later clean review could clear.Measured on Morrison-Lab/mds#599: two clean Claude reviews of the head could not pass the gate, and the only way out was editing the agent's own comment, which edits evidence.
What
check_review_commentsnow skips a comment only on positive evidence that it is a reply:A disposition line is removed only when its tail is on an allowlist, so the skip fails closed:
Anything else keeps the whole line, label included.
Residual risk
The allowlist is closed, so the remaining way to drop a real veto is an agent reply that says "rebutted" or "declined" with a closed rebuttal phrase that is false.
That is accepted as the price of not editing the agent's own comments; say so if you would rather not accept it.
Verification
scripts/test_check_pr_fully_clean.py: 1271 passed, 0 failed, including about 40 negative controls built from nine rounds of adversarial probes.per-reviewer: Claude=clean).Generated by Claude Code