Skip to content

🐛 PR Verifier: record PR Type and PR Issue with one completed create, and fail visibly when it can't - #18796

Merged
hivecommons-hive[bot] merged 4 commits into
hivecommons:v6from
Danathar:fix/pr-verifier-single-post
Nov 17, 2026
Merged

hivecommons-hive[bot] merged 4 commits into
hivecommons:v6from
Danathar:fix/pr-verifier-single-post

Conversation

@Danathar

@Danathar Danathar commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

The required PR Type and PR Issue checks could hang in_progress for ever while verify PR contents went green. That is what blocks #18667. This PR makes the verifier record each check in one step, already finished, so the checks always reach a result. If the verifier can't record a result, the job now goes red and says why.

Refs #18764. This is the v6 copy. The v5 copy is #18795. Note: GitHub runs pull_request_target workflows from the default branch (v5). The v5 copy, opened alongside, is the copy that matters for #18667 once merged. This v6 copy keeps v6 the same and covers workflow_dispatch runs on v6 branches, which use the branch's own copy.

Why it happened

The pinned verifier image (upstream kubebuilder-release-tools v0.4.3, deprecated upstream) worked in two steps. First it created each check as "in progress". Then it updated the check with the verdict. Sometimes GitHub answers that update with 404 Not Found (run 44069761061 on #18667). When that happens the check stays "in progress" for ever. Edits, pushes and reopens all start by updating the existing check, so the stuck check never recovers. The image step was continue-on-error, so the job reported success anyway.

What this changes for operators

  • No image pull. No GHCR login and no image. The workflow works out both verdicts in the repo, reusing the port of the upstream title rules that already exists in integration-governance.cjs. Check names, output titles and summaries are the same as the image's. The job runs on the normal runner pool instead of a native ARM64 runner, and drops packages: read.
  • Checks always conclude. Each check is recorded with one create call that already carries status: completed, the conclusion and the output. There's no "in progress" step and no update that could fail.
  • Red with a reason when a result can't be written. The verifier retries once. If it still can't record a check, it posts a completed failure check that gives the reason (best effort). Then verify PR contents fails with an error saying the verifier could not record its result.
  • A bad title still doesn't turn the job red. A title with no type prefix, or with an issue number in it, fails PR Type / PR Issue itself. The wrapper job stays green, as before.
  • A label flip now also records a fresh pair. The image skipped labeled, unlabeled and ready_for_review events. These checks only read the title, so the extra runs agree with each other.

Example: an edited event on #18667 would record PR Type: success (Found 🐛 PR (bugfix)) and PR Issue: success, one call each.

After merge: un-sticking a PR

The fix only applies to events after it merges. Re-running an old run re-runs the old workflow. For a PR whose checks are already stuck, do one of these on its current head:

  • add or remove any label (a person or App doing it; a github.token flip fires nothing), or
  • for a same-repo PR, run gh workflow run pr-verifier.yml --ref <branch> -f pr_number=<N>.

Either one records a fresh, completed pair.

Open question (unverified): the old stuck in_progress runs stay on the commit. I expect GitHub and Tide to use the newest run of each name, but I have not verified it. If they still count the stuck ones, the PR stays blocked. I'll check this on #18667 after merge.

Files

  • .github/workflows/pr-verifier.yml: replaces the login, pull and docker run steps with a sparse checkout of .github/scripts (pinned actions/checkout, no credentials kept) and a pinned actions/github-script step. Keeps the dispatch payload rebuild, the stale-head bailout, the concurrency block and the check names. Updates the comments.
  • .github/scripts/pr-verifier.cjs (new): record.
  • .github/scripts/integration-governance.cjs: prTypeVerdict and prIssueVerdict (the upstream check text). verifyContents now uses them.
  • src/deploy/test_pr_verifier_advisory.sh: checks the new setup. Nothing touches a registry, no step is continue-on-error, and a stale dispatch records nothing.
  • .github/scripts/test-native-queue-workflows.py: contract tests for the new step.
  • .github/workflows/pr-auto-update-test.yml: runs the new node test.
  • src/docs/hivecommons-migration.md, changelog.d/fixed-18764-pr-verifier-single-post.md.

Tests

  • New pr-verifier.test.cjs, mocked GitHub. One completed create per check, and no update or lookup calls. A bad title makes the check fail but not the job. A transient error is retried. When a result can't be recorded, the job fails and a fallback failure check is posted. A response that isn't completed counts as not recorded.
  • integration-governance.test.cjs: exact upstream check text for both checks.
  • CI: this PR's own checks.
  • Live run of the new workflow in the fork (Danathar/hive): a workflow_dispatch run on the v6 copy (run 44137409710) recorded PR Type completed/failure ("No matching PR type indicator found in title.", correct for that TEST ONLY … title) and PR Issue completed/success, one create each, with nothing left in_progress. The pull_request_target path could not be exercised in the fork, because GitHub ran the fork's default-branch copy of the workflow there.

🤖 Generated with Claude Code

https://claude.ai/code/session_01WcWm7qFmFJp26r9LPdmi3t

Danathar and others added 2 commits November 17, 2026 12:28
The pinned kubebuilder-release-tools v0.4.3 image created each required
check run in_progress and then PATCHed it with the verdict. When GitHub
answered the PATCH with 404 (hivecommons#18667, run 44069761061) the runs stayed
in_progress for ever, Tide blocked the PR, and the continue-on-error
wrapper job still went green.

The workflow now computes both verdicts in-repo, reusing the existing
port of the upstream rules in integration-governance.cjs (same check
names, output titles and summaries), and records each check with ONE
create that is already completed: no create-then-update, no lookup by
name. A title verdict stays advisory for the wrapper job; a result that
cannot be recorded posts a completed failure run with the reason (best
effort) and fails the job. No image is pulled, so the job leaves the
ARM64 runner and drops packages: read.

Refs hivecommons#18764

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WcWm7qFmFJp26r9LPdmi3t
Signed-off-by: Danathar <6772335+Danathar@users.noreply.github.com>
GitHub runs the default branch's copy of a pull_request_target workflow
(verified on the fork: a PR based on a branch carrying this change ran
the default branch's old workflow). Correct the checkout comment.

Refs hivecommons#18764

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WcWm7qFmFJp26r9LPdmi3t
Signed-off-by: Danathar <6772335+Danathar@users.noreply.github.com>
@kubestellar-prow kubestellar-prow Bot added dco-signoff: yes Indicates the PR's author has signed the DCO. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Nov 17, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign nicholasjackson for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@hivecommons-hive

hivecommons-hive Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor.

ℹ️ Sentinel: informational, author is trusted (owner), merge is not blocked. Head d12c9ecc8cd2 matched:

  • sensitive_path: changes 3 sensitive path(s) (.github/workflows/pr-auto-update-test.yml, .github/workflows/pr-verifier.yml, src/deploy/test_pr_verifier_advisory.sh)

No sentinel-alert label was added. The finding is recorded in the dashboard Security tab and /api/sentinel; set sentinel.trusted_authors_block: true to block trusted authors too.

Evidence bundle `#1879619f411c` sha256:a29a5809a881d3ab9773fc564d3875327020ff992190f94103bf9bbe4f555942 — download from the dashboard or `hivectl review evidence #18796`.

merged by Hive trusted-author auto-merge: author Danathar holds role owner and GitHub permission maintain on repo; CI green at 19f411c.

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Needs a decision, not a merge:

Head: verdict requires_human @d12c9ec, 75/75 checks green, DCO yes, soak 0s of 10m0s.

How to release it:

  • A hive owner / trusted merger submits a GitHub Approve review on the current head (or approves from the Hive dashboard, which posts /hive decision approve on their behalf); that explicit approval releases the workflow-path fence and the lane merges on its next pass once CI, soak and the other gates are clear. Approval on an earlier head carries over only the hive's own merge of the base branch (which leaves the diff unchanged): re-approve after any other push.
  • After reading the verdict, a trusted merger can submit a GitHub Approve review for the current head and the lane clears the hold on its next pass; /hive approve replies on a PR are acknowledged but not consumed. An ordinary contributor cannot release their own PR.

Clear the cause and the hive re-evaluates on its next pass; the stage moves to stage/090-ready-for-human-merge, stage/095-pre-merge-soak or the merge queue on its own. Approval does not bypass CI, hold labels or other merge gates.

@hivecommons-hive
hivecommons-hive Bot merged commit f632d9e into hivecommons:v6 Nov 17, 2026
79 of 80 checks passed
Danathar added a commit to Danathar/hive that referenced this pull request Nov 18, 2026
No conflicts. A new head so the PR Verifier from hivecommons#18796 records PR Type
and PR Issue: the runs on ac459c3 came from the earlier verifier and
have sat in_progress since 11:23 AM.

Signed-off-by: Danathar <6772335+Danathar@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BQYnPGFJx81c8JXR3o1sg1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dco-signoff: yes Indicates the PR's author has signed the DCO. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant