feat(ship-check): port output-honesty contracts from plan-review - #9
Conversation
Every phase summary now names the PR head SHA it actually reviewed (a review that doesn't say what it checked is indistinguishable from one that checked nothing — and pr-monitor's delta review keys off this SHA), and closes with proof-of-dismissal one-liners for suspicions considered and dropped, so a clean bill is distinguishable from an unexamined diff. Applied to all four phase skills (report sections + comment-mode review bodies) and their agents' output templates. Verification-state pinning was not ported — a PR head is already a pinned state. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
umm-actually re-reviewed at 1 new finding(s) posted (4 tracked finding(s) across all runs). Context notes
umm-actually · deepseek/deepseek-v4-flash-0731 |
…views The dismissal list only reached the orchestrator summary — in comment mode the PR-visible review body had no dismissal line, and a 0-findings phase skipped posting entirely, so the artifact meant to distinguish a clean diff from an unexamined one never landed on the PR. Added the dismissal line to all four review-body templates and narrowed the skip rule: skip only when there are neither findings nor dismissals. (umm-actually finding on PR #9.) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Add Reviewed-at and Dismissed to the pipeline's own status report
Pre-existing analog gap: the PR establishes a reviewed-SHA plus proof-of-dismissal contract for the phase skills, but the pipeline's final status snapshots (both default and comment mode) do not emit Failure scenario: A user reads only the final Suggested fixAdd `- Reviewed at: <SHA>` and `- Dismissed: …` to the ship-check output templates' Reporting section, mirroring the phase reports in this PR, so the pipeline conclusion carries the same two-line contract as the phases it summarizes.umm-actually · deepseek/deepseek-v4-flash-0731 |
…nism The honesty block claimed the pre-merge delta review keys off the phase-reported SHA; the orchestrator actually records the branch head itself at Phase 4 close and nothing reads the phase's Reviewed-at field. Reworded to what the reported SHA really provides: a cross-check of what the phase saw against the orchestrator's own baseline. (umm-actually finding on PR #9.) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Remove spliced-in monitoring-loop content from the Phase 2 code-quality reviewer
The Phase 2 code-quality reviewer agent ( Failure scenario: A user dispatches the code-quality reviewer for ship-check Phase 2 (or standalone). The agent reads its own body, reaches the “Continuous monitoring” block after the orientation, and follows the pr-monitor directives — scheduling 240-second wakeups, re-fetching the PR, scrubbing threads, and refusing to terminate “for any other reason” than an explicit user “stop” or merge/close. Instead of completing a finite code-quality pass (fix findings, run tests, commit, report), it enters an indefinite monitoring loop on a PR it was only asked to review, posting or re-evaluating comments. Milder case: the contradictory instructions just garble the agent's behavior at the boundary between “summarize” and “keep monitoring,” so the phase either hangs until the user notices or produces PR-side activity the caller never asked for. Suggested fixMove the pr-monitor block (thread-resolution prerequisites, issue-coverage checks, ScheduleWakeup / never-auto-terminate loop, “pipeline context” notes about the delta review) out of the code-quality reviewer into the actual monitoring agent where it belongs, and restore the code-quality reviewer's body to end with its own monitor-free instruction — i.e., after the orientation section, jump straight to the Phase-2 procedure and Output Format, with no reference to ScheduleWakeup or the “never auto-terminate” rule.umm-actually · deepseek/deepseek-v4-flash-0731 |
|
Re umm-actually finding on 🔍 ship-check · pr-monitor · claude-fable-5 |
|
Re umm-actually finding 5547367668 (Reviewed-at/Dismissed missing from the pipeline's own status snapshots): valid pre-existing analog gap — it landed just before this PR merged, so it's resolved in follow-up #10: both orchestrator reporting templates now carry the two-line contract (default mode reports the delta-review high-water SHA, comment mode the reviewed head, plus a Dismissed roll-up of the phases' dismissal lines). 🔍 ship-check · pr-monitor · claude-fable-5 |
…napshot (#10) * feat(ship-check): carry Reviewed-at and Dismissed into the pipeline snapshot The phase skills report their reviewed SHA and proof-of-dismissal (PR \#9), but the orchestrator's Ship-check-complete snapshots — the report a consumer actually reads — carried neither, so the concluding verdict still couldn't distinguish a reviewed PR from an unexamined one. Both reporting templates gain the two lines: Reviewed-at (the delta-review high-water mark in default mode, the reviewed head in comment mode) and a Dismissed roll-up of the phases' dismissal lines. (umm-actually finding on PR \#9, post-merge.) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ship-check): comment-mode Reviewed-at must not assume one head SHA Phases run sequentially in comment mode and the head can move between dispatches, so a single reported SHA overstates what earlier phases saw. Reworded to the Phase 4 completion head with per-phase SHAs when the head moved — each phase already reports its own. (umm-actually finding on PR \#10.) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
What
Ports two output-honesty contracts from the plan-check plugin's
plan-reviewskill back into all four ship-check phase skills (pr-review, code-quality, test-audit, bug-check) and their agents:What was deliberately NOT ported
plan-review's verification-state pinning (review against the repo as of the plan's date, not HEAD) doesn't transfer: ship-check reviews a PR head, which is already a pinned state. Only the output half — stating the SHA — carries over.
Changes
Reviewed at <HEAD_SHA>added to comment-mode review body templates; bug-check's inline summary template gains both linesReviewed atandDismissedlines added to default- and comment-mode output templates🤖 Generated with Claude Code