Skip to content

🐛 PR Verifier (v5): record PR Type and PR Issue with one completed create, and fail visibly when it can't - #18795

Merged
hivecommons-hive[bot] merged 9 commits into
hivecommons:v5from
Danathar:fix/pr-verifier-single-post-v5
Nov 17, 2026
Merged

hivecommons-hive[bot] merged 9 commits into
hivecommons:v5from
Danathar:fix/pr-verifier-single-post-v5

Conversation

@Danathar

@Danathar Danathar commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

The required PR Type and PR Issue checks could hang in_progress for ever while verify PR contents went green. That is what blocks #18667. This PR makes the verifier record each check in one step, already finished, so the checks always reach a result. If the verifier can't record a result, the job now goes red and says why.

Refs #18764. This is the v5 copy. The v6 copy is #18796. This is the copy that fixes the live problem. GitHub runs pull_request_target workflows from the repository's default branch, which is v5. So this copy is the one that runs for v6 PRs like #18667 too.

Why it happened

The pinned verifier image (upstream kubebuilder-release-tools v0.4.3, deprecated upstream) worked in two steps. First it created each check as "in progress". Then it updated the check with the verdict. Sometimes GitHub answers that update with 404 Not Found (run 44069761061 on #18667). When that happens the check stays "in progress" for ever. Edits, pushes and reopens all start by updating the existing check, so the stuck check never recovers. The image step was continue-on-error, so the job reported success anyway.

What this changes for operators

  • No image pull. No GHCR login and no image. The workflow works out both verdicts in the repo, reusing the port of the upstream title rules that already exists in integration-governance.cjs. Check names, output titles and summaries are the same as the image's. The job runs on the normal runner pool instead of a native ARM64 runner, and drops packages: read.
  • Checks always conclude. Each check is recorded with one create call that already carries status: completed, the conclusion and the output. There's no "in progress" step and no update that could fail.
  • Red with a reason when a result can't be written. The verifier retries once. If it still can't record a check, it posts a completed failure check that gives the reason (best effort). Then verify PR contents fails with an error saying the verifier could not record its result.
  • A bad title still doesn't turn the job red. A title with no type prefix, or with an issue number in it, fails PR Type / PR Issue itself. The wrapper job stays green, as before.
  • A label flip now also records a fresh pair. The image skipped labeled, unlabeled and ready_for_review events. These checks only read the title, so the extra runs agree with each other.

Example: an edited event on #18667 would record PR Type: success (Found 🐛 PR (bugfix)) and PR Issue: success, one call each.

After merge: un-sticking a PR

The fix only applies to events after it merges. Re-running an old run re-runs the old workflow. For a PR whose checks are already stuck, do one of these on its current head:

  • add or remove any label (a person or App doing it; a github.token flip fires nothing), or
  • for a same-repo PR, run gh workflow run pr-verifier.yml --ref <branch> -f pr_number=<N>.

Either one records a fresh, completed pair.

Open question (unverified): the old stuck in_progress runs stay on the commit. I expect GitHub and Tide to use the newest run of each name, but I have not verified it. If they still count the stuck ones, the PR stays blocked. I'll check this on #18667 after merge.

Files

  • .github/workflows/pr-verifier.yml: replaces the login, pull and docker run steps with a sparse checkout of .github/scripts (pinned actions/checkout, no credentials kept) and a pinned actions/github-script step. Keeps the dispatch payload rebuild, the stale-head bailout, the concurrency block and the check names. Updates the comments.
  • .github/scripts/pr-verifier.cjs (new): record.
  • .github/scripts/integration-governance.cjs: prTypeVerdict and prIssueVerdict (the upstream check text). verifyContents now uses them.
  • src/deploy/test_pr_verifier_advisory.sh: checks the new setup. Nothing touches a registry, no step is continue-on-error, and a stale dispatch records nothing.
  • .github/scripts/test-native-queue-workflows.py: contract tests for the new step.
  • .github/workflows/pr-auto-update-test.yml: runs the new node test.
  • src/docs/hivecommons-migration.md, changelog.d/fixed-18764-pr-verifier-single-post.md.

Tests

  • New pr-verifier.test.cjs, mocked GitHub. One completed create per check, and no update or lookup calls. A bad title makes the check fail but not the job. A transient error is retried. When a result can't be recorded, the job fails and a fallback failure check is posted. A response that isn't completed counts as not recorded.
  • integration-governance.test.cjs: exact upstream check text for both checks.
  • CI: this PR's own checks.
  • Live run of the new workflow in the fork (Danathar/hive): a workflow_dispatch run on the v6 copy (run 44137409710) recorded PR Type completed/failure ("No matching PR type indicator found in title.", correct for that TEST ONLY … title) and PR Issue completed/success, one create each, with nothing left in_progress. The pull_request_target path could not be exercised in the fork, because GitHub ran the fork's default-branch copy of the workflow there.

🤖 Generated with Claude Code

https://claude.ai/code/session_01WcWm7qFmFJp26r9LPdmi3t

Danathar and others added 2 commits November 17, 2026 12:28
The pinned kubebuilder-release-tools v0.4.3 image created each required
check run in_progress and then PATCHed it with the verdict. When GitHub
answered the PATCH with 404 (hivecommons#18667, run 44069761061) the runs stayed
in_progress for ever, Tide blocked the PR, and the continue-on-error
wrapper job still went green.

The workflow now computes both verdicts in-repo, reusing the existing
port of the upstream rules in integration-governance.cjs (same check
names, output titles and summaries), and records each check with ONE
create that is already completed: no create-then-update, no lookup by
name. A title verdict stays advisory for the wrapper job; a result that
cannot be recorded posts a completed failure run with the reason (best
effort) and fails the job. No image is pulled, so the job leaves the
ARM64 runner and drops packages: read.

Refs hivecommons#18764

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WcWm7qFmFJp26r9LPdmi3t
Signed-off-by: Danathar <6772335+Danathar@users.noreply.github.com>
(cherry picked from commit 5750c0f)
GitHub runs the default branch's copy of a pull_request_target workflow
(verified on the fork: a PR based on a branch carrying this change ran
the default branch's old workflow). Correct the checkout comment.

Refs hivecommons#18764

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WcWm7qFmFJp26r9LPdmi3t
Signed-off-by: Danathar <6772335+Danathar@users.noreply.github.com>
(cherry picked from commit d12c9ec)
@kubestellar-prow kubestellar-prow Bot added the dco-signoff: yes Indicates the PR's author has signed the DCO. label Nov 17, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign danathar for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubestellar-prow kubestellar-prow Bot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Nov 17, 2026
@github-actions

github-actions Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Open PR overlap check

Base branch: v5

Conflicts are checked with every PR merged onto the current tip of v5, so a PR that only needs a rebase is not reported as conflicting with its siblings.

PR Title Relation Files
#18788 ✨ Weekly PR adds new Claude models to the lists and price table once the pinned Claude Code accepts them same files .github/scripts/integration-governance.cjs
#18792 🐛 PR Verifier: never leave required checks in progress when the update fails CONFLICT .github/workflows/pr-auto-update-test.yml
.github/workflows/pr-verifier.yml (conflict)
#18738 🐛 images: align image-bump commit author with sign-off same files .github/scripts/integration-governance.cjs

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Decision needed: @clubanderson should decide whether #18795 (replace the pinned verifier image with an in-repo script) or #18792 (keep the image and settle stuck check runs afterwards) is the fix for #18764, since both rewrite .github/workflows/pr-verifier.yml and cannot both merge.

Two open PRs fix #18764 in incompatible ways; a maintainer has to pick one. This is a routing note, not a defect report.

  • #18795 (this PR) removes the GHCR image, login and docker run. It records PR Type / PR Issue with one already-completed checks.create each, in .github/scripts/pr-verifier.cjs. It also drops packages: read, adds contents: read, and changes the runner selection (pr-verifier.yml runs-on, now HIVE_RUNNER_LABELS).
  • #18792 keeps the image and adds a "settle" step that closes stuck runs in the same check suite. The hive overlap comment already flags pr-verifier.yml as a CONFLICT between the two.
  • #18796 is the v6 twin of this PR. I diffed the two and only the blob index lines differ, so this is intentional paired copies, not a duplicate to close. The PR body says the v5 copy is the one that matters, because pull_request_target runs the default branch's workflow.

Read at the PR head: the workflow checks out github.sha (default-branch tip) with persist-credentials: false and a sparse .github/scripts checkout, so no PR code runs under pull_request_target. The verdicts are computed from event.pull_request.title only, as plain data. record() treats a response that isn't a completed verdict on the same head as unrecorded. I found no defect in this path.

The question is a trust-surface one: it changes workflow permissions and the required-check producer, and it changes what the required checks mean. The author's own open question (do the old stuck in_progress runs still count for Tide?) is unverified, so neither PR is proven to unblock #18667.

— hive: agent=reviewer backend=claude model=claude-sonnet-5-5 claude=2.1.326

@hivecommons-hive

hivecommons-hive Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor.

Review findings from reviewer-2 (folded here because this head already carries its hive review):

Needs a maintainer choice between two fixes for #18764, and it edits a CI trust surface.

Competing PR (not a duplicate). #18792 (v5, Closes #18764) keeps the pinned image and adds a "Settle PR Type and PR Issue check runs" step to the same pr-verifier.yml; this PR (also v5, Refs #18764) deletes the image, login, pull and docker run steps (.github/workflows/pr-verifier.yml, the Verifier action step) and records both checks from .github/scripts/pr-verifier.cjs. Per #18792's body, its settle step reads the image's log and PATCHes stuck runs, so it has nothing to act on once this PR removes the image. They solve the same problem differently; one should survive.

#18796 is not a duplicate of this PR. I diffed the two: they are byte-identical apart from index hashes, but target v5 and v6 respectively, which the body states is intentional (pull_request_target runs from the default branch). Treat them as one decision.

What I read. pr-verifier.cjs:44-93: each check is one checks.create with status: completed, so no later update can 404; the response is validated, there is one retry, a best-effort fallback failure run is posted, and core.setFailed fires only when a result could not be recorded. That matches the body. The workflow checks out github.sha (default-branch tip on pull_request_target) with persist-credentials: false and a sparse .github/scripts, so no PR code runs with checks: write. The title is read from the event JSON, not interpolated into shell. I found no defect in this logic.

Why a person should decide. It changes workflow permissions (drops packages: read, adds contents: read), the runner selection (vars.HIVE_RUNNER_LABELS), and makes a previously always-green required wrapper job able to go red. The body's own open question (do GitHub/Tide ignore the old stuck in_progress runs?) is unverified, so whether this unblocks #18667 is not established.

ℹ️ Sentinel: informational, author is trusted (owner), merge is not blocked. Head 506535b452ca matched:

  • sensitive_path: changes 3 sensitive path(s) (.github/workflows/pr-auto-update-test.yml, .github/workflows/pr-verifier.yml, src/deploy/test_pr_verifier_advisory.sh)

No sentinel-alert label was added. The finding is recorded in the dashboard Security tab and /api/sentinel; set sentinel.trusted_authors_block: true to block trusted authors too.

Evidence bundle `#187950243a9c` sha256:e84519bc9fd8104b00304a5156c7bdaf12f12a9fed3b7b5fbe1acfaac7cc58ce — download from the dashboard or `hivectl review evidence #18795`.

merged by Hive trusted-author auto-merge: author Danathar holds role owner and GitHub permission maintain on repo; CI green at 0243a9c.

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Decision needed: @clubanderson should decide which fix for #18764 lands on v5 — this PR (replace the verifier image with an in-repo one-create pr-verifier.cjs) or #18792 (keep the pinned image and add a settle step)?

Two open v5 PRs rewrite .github/workflows/pr-verifier.yml for the same issue (#18764) with different designs, so they conflict rather than duplicate.

  • #18795 (this PR) removes the pinned verifier image and records PR Type/PR Issue with one already-completed checks.create each (.github/scripts/pr-verifier.cjs:19-33), so no in_progress run can be left behind.
  • #18792 keeps the image and adds a "Settle PR Type and PR Issue check runs" step that PATCHes stuck runs to failure (per its body).

Both edit the same workflow, so only one can merge cleanly. They also differ in trust terms: this PR drops the packages: read permission and the native ARM64 runner, and reimplements the upstream title rules in-repo (integration-governance.cjs, prTypeFromTitle/prTypeVerdict/prIssueVerdict); #18792 leaves the image-based rules in place. Both touch a required-check workflow, which is a trust surface, so I am routing this rather than judging it.

What I read in this diff looks internally consistent: create() verifies the response's name, head SHA, status and conclusion before counting a result as recorded, verifyContents now delegates to the same verdict functions, and the tests assert update/get/listForRef are never called. I did not run anything. #18796 is the v6 twin of this PR (same file set, different base, stated in both bodies), so it is not a duplicate; I am not commenting on it separately.

— hive: agent=reviewer backend=claude model=claude-sonnet-5-5 claude=2.1.326

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Decision needed: @clubanderson should the PR Verifier hang fix land as this PR (replace the GHCR image with an in-repo verifier) or as #18792 (keep the image and settle its checks afterwards)?

This PR and #18792 both fix hung PR Type / PR Issue checks by editing .github/workflows/pr-verifier.yml and .github/workflows/pr-auto-update-test.yml, with different designs. This one drops the image and records one completed check run per verdict (pr-verifier.cjs). #18792 keeps the image and adds a settle step (test-pr-verifier-settle.py). They will conflict, and only one design should merge.

The diff looks sound to me. I did not read pr-verifier.yml past the diff's first hunk (the diff is truncated in my view). Read:

  • pr-verifier.cjs creates each check with status: 'completed' in a single call and verifies GitHub's echoed response.
  • prTypeVerdict / prIssueVerdict are covered by tests.
  • The workflow test pins checkout to github.sha with persist-credentials: false, which is the right choice under pull_request_target.

The decision is a human call because the change rewires a required check under pull_request_target, with checks: write, and removes the pinned upstream image. Trust-surface files are never mine to approve.

— hive: agent=reviewer backend=claude model=claude-sonnet-5-5 claude=2.1.326

@hivecommons-hive

hivecommons-hive Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Needs a decision, not a merge:

  • the reviewer swarm returned requires_human @0243a9c
  • the review found: the reviewer marked this change high-risk
  • reviewer question: @clubanderson should decide whether the required PR Type / PR Issue checks may move from the pinned upstream verifier image to in-repo github-script code, since this rewrites a required-check workflow that runs under pull_request_target with checks: write.
  • the PR edits CI workflow / action files: awaiting owner approval — a hive owner / trusted merger's GitHub Approve review on the current head (or a dashboard /hive decision approve) releases the fence and the hive merges it; approval on an earlier head does not count unless the hive's own merge of the base branch is all that moved the head, so re-approve after a push

Head: verdict requires_human @0243a9c, 44/57 checks green, 13 pending, DCO yes, soak 0s of 10m0s.

How to release it:

  • A hive owner / trusted merger submits a GitHub Approve review on the current head (or approves from the Hive dashboard, which posts /hive decision approve on their behalf); that explicit approval releases the workflow-path fence and the lane merges on its next pass once CI, soak and the other gates are clear. Approval on an earlier head carries over only the hive's own merge of the base branch (which leaves the diff unchanged): re-approve after any other push.
  • After reading the verdict, a trusted merger can submit a GitHub Approve review for the current head and the lane clears the hold on its next pass; /hive approve replies on a PR are acknowledged but not consumed. An ordinary contributor cannot release their own PR.

Clear the cause and the hive re-evaluates on its next pass; the stage moves to stage/090-ready-for-human-merge, stage/095-pre-merge-soak or the merge queue on its own. Approval does not bypass CI, hold labels or other merge gates.

Updated 2026-11-17 21:20 UTC.

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Decision needed: @clubanderson should decide whether the required PR Type / PR Issue checks may move from the pinned upstream verifier image to an in-repo github-script port that runs under checks: write.

I found no code defect in the logic, but this changes a required-check producer in .github/workflows/pr-verifier.yml, which is a CI-permissions surface, so I'm routing it rather than judging it alone.

What I read (pr-verifier.yml, pr-verifier.cjs, integration-governance.cjs):

  • The pull_request_target job now checks out ${{ github.sha }} (the default branch tip) with persist-credentials: false and a sparse .github/scripts checkout, then require()s pr-verifier.cjs from it. PR code is never executed, so the standard pwn-request path is closed for fork PRs.
  • Permissions go from checks: write, pull-requests: read, packages: read to checks: write, contents: read, pull-requests: read. The reduction is fine.
  • A workflow_dispatch run checks out the dispatching same-repo branch's own scripts, and that run holds checks: write. Whoever can push a branch and dispatch can post a green PR Type / PR Issue on any head. The workflow comment says this is limited to same-repo branches, which I believe is right, but it is new, because the previous image was pinned by digest.
  • The verifier no longer comes from an immutable digest-pinned upstream image. The rules are re-implemented in JS (prTypeFromTitle, prTypeVerdict), so the required checks can now drift from the upstream rules. The new tests do pin the output text.

The decision for a maintainer: is that trade acceptable? It removes the GHCR dependency and the create-then-update stuck-in_progress failure (#18764), at the cost of owning the verifier logic in-repo.

No changelog.d text issue seen; the fragment is present.

— hive: agent=reviewer backend=claude model=claude-sonnet-5-5 claude=2.1.326

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Decision needed: @clubanderson should decide whether the required PR Type / PR Issue checks may move from the pinned upstream verifier image to in-repo github-script code, since this rewrites a required-check workflow that runs under pull_request_target with checks: write.

Routing to a human, not a code defect. This PR replaces the pinned verifier image with in-repo code in .github/workflows/pr-verifier.yml, which is the workflow that posts the required PR Type / PR Issue contexts. That is a CI-permissions / required-check surface, so a maintainer needs to sign off.

What I verified in the diff:

  • permissions: goes to checks: write, contents: read, pull-requests: read (packages: read is dropped). No broader grant.
  • The new checkout uses ref: ${{ github.sha }} with persist-credentials: false and a sparse .github/scripts checkout. On pull_request_target that is the default-branch tip, so PR code is not executed. The script reads only pr.title and pr.head.sha.
  • runs-on now allows vars.HIVE_RUNNER_LABELS for same-repo PRs and workflow_dispatch, and ubuntu-latest for forks. Forks stay off the self-hosted pool.
  • record() in pr-verifier.cjs creates each check already completed and fails the job when it cannot record. That matches the stated goal of fixing the in_progress hang.

Open question from the PR body, still unverified: whether the old stuck in_progress runs on a head are ignored in favour of the newer completed run. That decides whether #18667 actually unblocks. It is worth confirming on #18667 after merge, as the author says.

I read the diff and the workflow hunks only; I did not run anything.

— hive: agent=reviewer-2 backend=claude model=claude-sonnet-5-5 claude=2.1.326

@hivecommons-hive
hivecommons-hive Bot merged commit 5da6f72 into hivecommons:v5 Nov 17, 2026
78 of 79 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

conflicts-with-open-pr Open PR has a merge conflict with another open PR dco-signoff: yes Indicates the PR's author has signed the DCO. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant