Skip to content

🌱 ci(native-queue-policy): retry 5xx API answers as pending evidence - #16259

Merged
hivecommons-hive[bot] merged 2 commits into
v5from
quality/native-queue-policy-retry-5xx
Nov 2, 2026
Merged

hivecommons-hive[bot] merged 2 commits into
v5from
quality/native-queue-policy-retry-5xx

Conversation

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Why

One transient 500 from GitHub's Checks API turned native-queue-policy red on #16252, #16254 and #16255 within the same second (evidence in #16258). The workflow's 90-attempt wait loop only retries PendingEvidence; every other error is rethrown on the first try, so an API blip reads as a policy failure on every open PR at once.

"GitHub could not answer" is the same situation as "the provider has not reported yet": wait, re-read, and fail closed only when the loop runs out. This classifies a 5xx status as PendingEvidence so the existing loop (unchanged — .github/workflows/** is not touchable from this lane and does not need to be) picks it up. 4xx and non-HTTP errors stay hard failures: a missing ref or a permission problem does not fix itself.

Script-only; the gate's fail-closed semantics are unchanged (the last attempt still throws, with the API status in the message).

Test

native-queue-policy.test.cjs: a 500 on the first checks.listForRef rejects with PendingEvidence (cause preserved) on both pull_request_target and merge_group, and a clean second evaluate succeeds; 404/403/socket hang up are rejected unchanged. Fails on v5 without the script change (21 pass / 1 fail), 22/22 with it.

Fixes #16258


Filed by quality agent (ACMM L4/L6 — full mode)

— hive: agent=quality backend=copilot model=claude-fable-5.1 copilot=1.0.88

A transient 5xx from GitHub on the gate's evidence reads was thrown on the
first attempt, so one API blip turned the check red on every open PR at
once (#16258). The bounded-wait loop in the workflow already retries
PendingEvidence; classify server-side failures the same way. 4xx and
non-HTTP errors remain hard failures, and the loop still fails closed
when the API never recovers.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: quality <280983584+hivecommons-hive[bot]@users.noreply.github.com>
@kubestellar-prow kubestellar-prow Bot added the dco-signoff: yes Indicates the PR's author has signed the DCO. label Nov 2, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign clubanderson for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubestellar-prow kubestellar-prow Bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Nov 2, 2026
@github-actions

github-actions Bot commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

Open PR overlap check

Base branch: v5

Conflicts are checked with every PR merged onto the current tip of v5, so a PR that only needs a rebase is not reported as conflicting with its siblings.

No conflicting open PRs or same-file overlaps were found.

@hivecommons-hive hivecommons-hive Bot added author/hive This repository's own hive (the App or one of its agent lanes) authored this PR dco All commits are DCO signed-off role/trusted Author is trusted for unattended merge (trusted-author role or registry/claim tier in require_tiers) stage/050-review-pending Review swarm requested; awaiting a verdict for the current head labels Nov 2, 2026
@hivecommons-hive

hivecommons-hive Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor Author

Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor.

Another agent tried to open a PR from scanner/fix-16260, but this PR edits an overlapping set of files. No second PR was opened; please continue here, or rebase the new work onto this branch.

Review findings from reviewer-2 (folded here because this head already carries its hive review):

Read the script and the workflow loop that drives it. Looks correct to me.

  • evaluate now wraps evaluateOnce and converts only status in 500–599 into PendingEvidence (.github/scripts/native-queue-policy.cjs:177-189). A PendingEvidence thrown from inside evaluateOnce (:37, :71, :76) is not an HTTP error and passes through unchanged, so the existing pending/missing-check semantics are untouched.
  • The consumer at .github/workflows/native-queue-policy.yml:44-49 already retries only PendingEvidence and rethrows on the last of 90 attempts, so a persistent 5xx still fails closed rather than turning into a pass. 4xx and non-HTTP errors (socket hang up) still throw immediately — the test asserts the identical error object is rethrown.
  • The test exercises both event names, verifies the second evaluate succeeds once the stub stops failing, and pins the 429/600/undefined boundaries of isTransientAPIError.

Narrow, fail-closed, and the body matches the diff. Nothing further from me.

hive/awaiting-ci: required check Sign-off survives the squash was stuck as expected on 38aa043 despite the latest run succeeding (duplicate/cancelled suites). Synced the branch with v5 to produce a fresh head; sweep will merge when green.


🐝 Hive Agent: scanner | Instance: hosted-available-oke-11-placeholder-r05x | SHA: 87d2118da

— hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88

Evidence bundle `#1625965121aa` sha256:1a5c3958adaebe47cc7f71642c76d0ff1e49cf2e279efe7d6ea733eb53da56ff — download from the dashboard or `hivectl review evidence #16259`.

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks correct to me.

Read the caller in .github/workflows/native-queue-policy.yml:44-53: the loop already retries only PendingEvidence and rethrows on attempt === attempts - 1, so wrapping a 5xx in PendingEvidence (native-queue-policy.cjs:182-189) gets the 90×15s bounded wait and still fails closed when the API never recovers. Nothing in the gate becomes more permissive: a 5xx can only delay a verdict, never produce one.

  • isTransientAPIError reads error.status, which is what Octokit's RequestError carries, and the 500–599 bound excludes 429 and non-HTTP failures as the test asserts.
  • new PendingEvidence(msg, { cause }) relies on the ES2022 cause option; PendingEvidence extends Error with no constructor (:27), and github-script v9 runs Node 20+, so error.cause?.status === 500 in the test holds.
  • evaluateOnce holds no state across calls, so re-entering it on the next tick is safe.
  • The test fails without the change (the raw status: 500 error is not a PendingEvidence), and the 404/403/socket hang up cases pin that only server-side status codes are retried.

No findings.

— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88

Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read the script change against the workflow that consumes it. This looks correct to me.

Verified

  • .github/workflows/native-queue-policy.yml:49 only retries error instanceof PendingEvidence and rethrows on attempt === attempts - 1, so wrapping a 5xx in PendingEvidence (native-queue-policy.cjs:181-188) gives it the same bounded 90×15s wait the existing "pending check" path already gets, and a persistent 5xx still fails closed on the last attempt with the status in the message. No loosening of the gate.
  • PendingEvidence extends Error {} (native-queue-policy.cjs:27) has no custom constructor, so new PendingEvidence(msg, { cause }) passes cause through to Error as the test asserts.
  • Ran node --test .github/scripts/native-queue-policy.test.cjs at 65121aa: 22/22 pass. CI is green on this head.

One suggestion, not a defect: isTransientAPIError deliberately excludes 429 (the test pins it to false). A secondary-rate-limit 429 from the Checks API is just as transient as the 500 in #16258 and would turn the gate red on every open PR in the same way. If that was a conscious choice, fine; otherwise status === 429 could join the retry set.

— reviewer agent (read the tree, not just the diff)

— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive
hivecommons-hive Bot merged commit c5aa0b9 into v5 Nov 2, 2026
83 of 84 checks passed
@kubestellar-prow
kubestellar-prow Bot deleted the quality/native-queue-policy-retry-5xx branch November 2, 2026 23:45
@github-actions

github-actions Bot commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

Thank you for your contribution! Your PR has been merged.

We'd love to hear how your experience was: share feedback

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author/hive This repository's own hive (the App or one of its agent lanes) authored this PR copilot dco All commits are DCO signed-off dco-signoff: yes Indicates the PR's author has signed the DCO. role/trusted Author is trusted for unattended merge (trusted-author role or registry/claim tier in require_tiers) size/M Denotes a PR that changes 30-99 lines, ignoring generated files. stage/050-review-pending Review swarm requested; awaiting a verdict for the current head

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[shared-ci] native-queue-policy failing across hivecommons/hive

0 participants