From ca79a232a23b0a59293f8bbe6aa1042e60b46192 Mon Sep 17 00:00:00 2001 From: Andy Anderson Date: Mon, 16 Nov 2026 21:50:58 -0500 Subject: [PATCH 1/3] =?UTF-8?q?=F0=9F=8C=B1=20ci:=20no=20mutation=20commen?= =?UTF-8?q?t=20for=20an=20error=20verdict;=20no=20reopen=20hint=20for=20bo?= =?UTF-8?q?t=20or=20maintainer=20reporters?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit mutation-report.sh post no longer creates a comment for an 'error' verdict (no report, nothing to act on): the report stays in the job summary and the failing verdict step, and an earlier finding comment is edited so it is not stale. issue-closed-reopen-hint.yml skips issues whose reporter is a bot or OWNER/MEMBER/COLLABORATOR. pr-overlap.sh already posts only when a sibling PR overlaps (#17870); unchanged. Fixes #18607 Signed-off-by: Andy Anderson --- .github/workflows/issue-closed-reopen-hint.yml | 12 ++++++++++-- src/docs/notifications.md | 4 ++-- src/scripts/mutation-report.sh | 12 ++++++++++-- src/scripts/test-mutation-report.sh | 13 ++++++++++++- 4 files changed, 34 insertions(+), 7 deletions(-) diff --git a/.github/workflows/issue-closed-reopen-hint.yml b/.github/workflows/issue-closed-reopen-hint.yml index d0eea0c7c6..4e5142acf1 100644 --- a/.github/workflows/issue-closed-reopen-hint.yml +++ b/.github/workflows/issue-closed-reopen-hint.yml @@ -11,7 +11,11 @@ name: Issue Closed Hint # instead of depending on whoever closed the issue remembering to say so. # # Skipped when the reporter closed the issue themselves (nothing to tell them) -# and for pull requests (the `issues` event never fires for those). +# and for pull requests (the `issues` event never fires for those). Also +# skipped when the reporter is a bot (the hive's own issues: nobody reads the +# hint) or a maintainer (OWNER / MEMBER / COLLABORATOR, who can reopen the +# issue themselves): those were most of the hint's comments and none of them +# told anyone something they could not already do (#18607). on: issues: @@ -26,7 +30,11 @@ jobs: name: Point the reporter at /reopen runs-on: ubuntu-latest timeout-minutes: 5 - if: github.event.issue.user.login != github.event.sender.login + if: >- + github.event.issue.user.login != github.event.sender.login && + github.event.issue.user.type != 'Bot' && + !endsWith(github.event.issue.user.login, '[bot]') && + !contains(fromJSON('["OWNER","MEMBER","COLLABORATOR"]'), github.event.issue.author_association) steps: - name: Comment env: diff --git a/src/docs/notifications.md b/src/docs/notifications.md index 722270d30c..5b783193ff 100644 --- a/src/docs/notifications.md +++ b/src/docs/notifications.md @@ -721,7 +721,7 @@ third party's comment that happens to contain the marker: | Workflow / script | Comment it posts | Behaviour | | --- | --- | --- | | Refs sweep (`.github/workflows/refs-sweep.yml`, `src/scripts/comment-merged-refs.sh`) | "PR #N has merged and referenced this issue with a non-closing `Refs`" (``) | **Skips** when the card carries `issue-partial:` for that PR; otherwise unchanged (still once per issue). | -| Issue Closed Hint (`.github/workflows/issue-closed-reopen-hint.yml`) | "@reporter this issue was closed by … comment `/reopen`" | **Pending** (tracked in #16363; the workflow edit must be landed by a maintainer). Once applied: **skips** when the card carries `issue-close`, which already says "Reply /reopen"; otherwise **edits** its earlier comment (``) instead of adding one per closure. Until then it posts a new comment on every closure as before. | +| Issue Closed Hint (`.github/workflows/issue-closed-reopen-hint.yml`) | "@reporter this issue was closed by … comment `/reopen`" | **Pending** (tracked in #16363; the workflow edit must be landed by a maintainer). Once applied: **skips** when the card carries `issue-close`, which already says "Reply /reopen"; otherwise **edits** its earlier comment (``) instead of adding one per closure. Until then it posts a new comment on every closure as before. **Skips** when the reporter is a bot or a maintainer (`author_association` OWNER / MEMBER / COLLABORATOR), who can reopen the issue themselves (#18607). | Editing a comment notifies nobody, so a workflow comment that is edited in place stops re-mentioning the reporter on every close/reopen cycle. Both @@ -781,7 +781,7 @@ clean case, still goes to the job's step summary: | Workflow / script | Comment | Behaviour | | --- | --- | --- | | Open PR Overlap (`.github/workflows/pr-overlap.yml`, `.github/scripts/pr-overlap.sh`) | `` table of conflicting / same-file open PRs | **Posts** only when at least one sibling PR conflicts or shares a file; **edits** the one marker comment in place on later pushes; when a later run finds nothing and a comment exists, **edits it** to a one-line "✅ Resolved" note; a PR that never had an overlap gets no comment. The `conflicts-with-open-pr` label is unchanged. | -| Mutation (advisory) (`.github/workflows/mutation-advisory.yml`, `src/scripts/mutation-report.sh post`) | `` surviving / uncovered mutant report | **Posts** only for a non-`pass` verdict (`lived`, `uncovered`, `misrouted`, `error`); **edits** the one marker comment on later pushes; a later `pass` **edits** the existing comment to a one-line "✅ Resolved" note; a PR whose every run passes gets no comment. | +| Mutation (advisory) (`.github/workflows/mutation-advisory.yml`, `src/scripts/mutation-report.sh post`) | `` surviving / uncovered mutant report | **Posts** only for a finding verdict (`lived`, `uncovered`, `misrouted`); an `error` verdict (gremlins wrote no report, so nothing for the author to act on) goes to the job summary and the failing verdict step only and never creates a comment (#18607); **edits** the one marker comment on later pushes (an `error` run edits an earlier finding comment so it is not stale); a later `pass` **edits** the existing comment to a one-line "✅ Resolved" note; a PR whose every run passes or errors gets no comment. | | Feedback Wanted (`.github/workflows/feedback.yml` → `hivecommons/infra` `reusable-feedback.yml`, gated by `src/scripts/feedback-gate.sh`) | "Thank you for your contribution! Your PR has been merged." | **Posts** only for a person's first merged PR in the repo. Skipped for `[bot]` / App authors, hive-managed head branches (`scanner/*`, `hive/*`), PR bodies carrying a ` comment)" + echo "::error::mutation: gremlins failed or wrote no report for a changed package, so no verdict could be derived (see the run log and the job summary)" return 1 ;; misrouted) echo "::error::mutation: a mutant on a line this PR added was SKIPPED; gremlins did not match the diff for that file (see the comment)" @@ -101,7 +101,7 @@ post_comment() { return 0 fi # The report always lands in the step summary; the PR comment is reserved - # for findings (any verdict but `pass`). A clean run never creates a + # for findings (`lived`, `uncovered`, `misrouted`; not `pass` or `error`). A clean run never creates a # comment, and one that follows an earlier finding edits that comment down # to a one-line resolved note instead of posting again (#17863). if [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then @@ -118,6 +118,14 @@ post_comment() { local id id=$(gh api --paginate "repos/${repo}/issues/${pr_number}/comments" \ --jq ".[] | select(.user.login == \"github-actions[bot]\" and (.body | startswith(\"${marker}\"))) | .id" 2>/dev/null | head -n 1 || true) + if [ "$verdict" = error ] && [ -z "$id" ]; then + # An `error` verdict means gremlins produced no report, so there is no + # finding for the author to act on: it lives in the step summary and the + # failing verdict step only, never a fresh notifying comment (#18607). + # An earlier finding comment is still edited below so it is not stale. + echo "mutation: verdict error and no earlier comment; not posting (report is in the step summary)." + return 0 + fi if [ -n "$resolved" ]; then if [ -z "$id" ]; then echo "mutation: verdict pass and no earlier comment; not posting (report is in the step summary)." diff --git a/src/scripts/test-mutation-report.sh b/src/scripts/test-mutation-report.sh index f41b6a9bdf..9d394d9ca0 100755 --- a/src/scripts/test-mutation-report.sh +++ b/src/scripts/test-mutation-report.sh @@ -218,7 +218,9 @@ run_post() { # run_post [existing-id] MUTATION_OUT_DIR="$1" PR_NUMBER=42 GITHUB_REPOSITORY=example/hive bash "$SCRIPT" post 2>&1) writes=$(cat "$GH_WRITE_LOG") } -mkdir -p "$TMP/out-pass" "$TMP/out-lived" +mkdir -p "$TMP/out-pass" "$TMP/out-lived" "$TMP/out-error" +printf '\n### Mutation testing\n\ngremlins did not produce a report for 1 package(s)\n' > "$TMP/out-error/comment.md" +echo error > "$TMP/out-error/verdict" printf '\n### Mutation testing\n\nAll 3 mutant(s) in changed files were killed.\n' > "$TMP/out-pass/comment.md" echo pass > "$TMP/out-pass/verdict" printf '\n### Mutation testing\n\ngremlins found **1 surviving mutant(s)**\n' > "$TMP/out-lived/comment.md" @@ -237,6 +239,15 @@ grep -q "^api -X PATCH repos/example/hive/issues/comments/99 -f body=