Repository navigation
🐛 fix(ci): native-queue-policy names a red provider as a per-PR verdict - #18281
Conversation
native-queue-policy went red on #17512, #18273, #18275 and #18278 and the scanner filed it as one shared-CI incident (#18280). The gate was right every time: #17512 is a draft carrying do-not-merge/work-in-progress, #18273 and #18275 have probot DCO action_required, and #18278 has changelog-fragment-guard failure. The defect was the surface. A red provider was thrown as a bare `Error('DCO: action_required')` and printed as "Unhandled error" with a stack trace, which reads like a broken workflow. evidenceForHead and assertProwDCO now throw a ProviderVerdict naming the provider, its conclusion, the head, and that this is a per-PR verdict and not a shared CI incident. The evidence-wait step reports it with a job summary and core.setFailed instead of rethrowing. The conclusion stays failure: this is a required check. Hive-Run: #18280 Hive-Plan: shared-ci incident triage Hive-Spec: native-queue-policy#provider-verdict Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Danathar <Danathar@users.noreply.github.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
1d30d13
ℹ️ Sentinel notice — informational; author is trusted (owner), merge is not blockedHive flagged this PR (author @Danathar, head
Hive did not add the |
There was a problem hiding this comment.
Decision needed: @clubanderson should decide whether a change to the required native-queue-policy workflow (.github/workflows/native-queue-policy.yml) may land, since it alters how a red admission provider is reported on a required check.
I read the changed workflow step at the PR head (native-queue-policy.yml:71-107) and the two throw sites in native-queue-policy.cjs (evidenceForHead, assertProwDCO). I found no defect.
- The new
ProviderVerdictcatch sits before thePendingEvidencebranch and returns aftercore.setFailed, so the job still fails on a red provider. It cannot turn a failure into a pass. ProviderVerdictis not aPendingEvidence, so a completed red verdict does not enter the 90×15s wait. That matches the old plainError, which was rethrown at once.- The
<provider>: <conclusion>prefix is preserved, and the forward-merge[DCO path: …]suffix is still appended, as the updated test regex expects. - The tests cover the subclass, the non-pending property and the path-scoped
skippedcase.
I'm routing it only because it edits a workflow on a required check, which is a trust surface. As the PR says, its own run exercises the old base-branch script, so the new behaviour is not self-verified.
— hive: agent=reviewer-2 backend=claude model=claude-sonnet-5-5 claude=2.1.325
|
Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor. Review findings from The change looks correct to me; it routes to a human only because it edits the admission workflow, which is a trust surface. What I verified (read at the PR head):
Suggestion, not a defect: the existing assertions on |
There was a problem hiding this comment.
Decision needed: @clubanderson @Danathar (CODEOWNERS for .github/workflows/**) should confirm this change to the native-queue-policy admission gate is acceptable to land, since it edits a trust-surface workflow.
I read the change in the tree at the PR head. The code looks correct to me; it is routed only because it edits .github/workflows/native-queue-policy.yml, which CODEOWNERS assigns to a human.
What I checked:
.github/workflows/native-queue-policy.yml:93: the newProviderVerdictbranch sits inside the samecatchasProviderUnavailable(line 85) and returns aftercore.setFailed. It precedes thePendingEvidence/rethrow check at line 104, so a completed red provider still fails at once and does not wait out the 90 × 15 s loop. The conclusion staysfailure, so the required check does not turn green..github/scripts/native-queue-policy.cjs:267appends[DCO path: …]toerror.messageafter construction. That still works on aProviderVerdict, and the updated test regex (prow dco: failure on .*per-PR provider verdict.*DCO path: prow dco) matches that ordering.- The earlier
catchat workflow line 62 belongs to the cosmetic-label replay step, which never callsevidenceForHead, so it is unaffected. - The PR body says the new tests do not exercise the new workflow code on this run, because the step checks out base-branch scripts. I agree; the new Python test (
test-native-queue-workflows.py) asserts only the ordering of thesetFailedcall against thePendingEvidencecheck, as text.
No defects found. Open question for the owner: the wording not a shared CI incident in a failure message is a policy statement shown to every contributor, so the owners may want to confirm they are happy with it.
— hive: agent=reviewer-2 backend=claude model=claude-sonnet-5-5 claude=2.1.325
Summary
native-queue-policywas red on #17512, #18273, #18275 and #18278, and the scanner filed that as one shared-CI incident (#18280). It isn't one. The gate gave the right verdict on every PR, and each failure traces back to that PR's own provider. The real problem is how the failure was reported: a red provider was thrown as a bareError('DCO: action_required'), which prints asUnhandled errorwith a stack trace and looks like a broken workflow. #18128 / #18134 fixed the same kind of misfiling for the ineligibility path; this covers the provider-verdict path.Root cause per PR
#17512: head/base moved, draft, closed, or held(run 42751483260, 2026-11-10, before #18134)do-not-merge/work-in-progressError: DCO: action_required(run 43730603419)DCOcheck isfailon the headError: DCO: action_required(run 43730709304)DCOisfail. The head istopup/v5-to-v6-20261115, opened by a human, so the narrowsync/v5-to-v6forward-merge path (Decision B on #17030) correctly doesn't applyError: changelog-fragment-guard: failure(run 43730895376)changelog-fragment-guardisfailon the headNo base-branch run exists because the workflow triggers only on
pull_request_targetandmerge_group, so that's expected. The four PRs don't share a cause.Change
.github/scripts/native-queue-policy.cjs:evidenceForHeadandassertProwDCOnow throw a newProviderVerdictinstead of a plainError. The message keeps the<provider>: <conclusion>prefix, so existing greps and the forward-merge[DCO path: ...]suffix still work. It then adds the head andper-PR provider verdict, not a shared CI incident: fix it on this PR; the next push re-evaluates..github/workflows/native-queue-policy.yml: the evidence-wait step handlesProviderVerdictthe same way it already handlesProviderUnavailable. It writes a job summary (Admission blocked: DCO is action_required on this PR) and callscore.setFailed, instead of rethrowing into an unhandled stack trace.PendingEvidence, prefix, note) and that path-scopedskippedbuild checks still pass. The three exact-message assertions now expect the new wording. A new Python workflow test asserts the step reportsProviderVerdictthroughsetFailedbefore the pending/rethrow branch.Deliberately unchanged: the conclusion stays
failure. This is a required check, andneutral/skippedwould count as passing.ProviderVerdictis notPendingEvidence, so a completed red verdict still fails at once and doesn't hold the job for 22.5 min.Testing: per AGENTS.md, CI is the verdict. Locally I only ran
node --checkon both.cjsfiles, a parse of the workflow YAML and Python test, and a smoke check of the new error class. The workflow's ownTest policy behaviorstep runs the node suite, but it checks out base-branch scripts, so this PR's run still uses the old code. The updated node tests show up in the unit-test CI job.No changelog fragment: everything changed is under
.github/, which the fragment guard exempts automatically.Fixes #18280
🤖 Generated with Claude Code
— hive: backend=claude model=claude-opus-5-5
🐝 Hive Agent:
contributor| SHA:1d30d135c