Skip to content

πŸ› fix(ci): native-queue-policy skips the evidence wait on cosmetic label flips - #16952

Merged
hivecommons-hive[bot] merged 5 commits into
v5from
scanner/fix-16945
Nov 7, 2026
Merged

hivecommons-hive[bot] merged 5 commits into
v5from
scanner/fix-16945

Conversation

@hivecommons-hive

@hivecommons-hive hivecommons-hive Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Closes #16945

Why

Every label flip re-fires native-queue-policy, and with cancel-in-progress: false (#16943) a cosmetic flip (size/L, dco-signoff: yes, hive/claimed-by-*, copilot, stage labels) still costs one extra queued ~22.5-min evidence-wait run per quiet window. Skipping the step outright would mint a fresh green required check without reading anything, which contradicts the fail-closed contract set in #16942 β€” so the shortcut has to carry an existing verdict, not fabricate one.

What

  • native-queue-policy.cjs: isCosmeticLabelFlip is true only for labeled/unlabeled on pull_request_target whose label does NOT match the policy's existing BLOCKED_LABEL regex (single source, no second list): hold, needs-human, release-in-progress, do-not-merge[/*]. Draft/closed/head state is read live from the PR, not labels.
  • replayLastVerdict: on a cosmetic flip, re-read live eligibility (hold/draft/head-move still fail), then look up the latest completed native-queue-policy run on the exact head, bound to this workflow file/event/head/check-suite/job (a same-named run from another workflow is rejected). A prior success is replayed in seconds. When there is no prior success (head never evaluated, or the last wait failed / timed out) it returns null and the workflow step falls through to the full exact-head evidence wait β€” the authoritative path still runs, so a stale red verdict is never pinned onto later cosmetic flips. A 5xx during the replay reads takes the same fall-through, matching the transient-API handling the main path already has ([shared-ci] native-queue-policy failing across hivecommons/hiveΒ #16258).
  • Workflow: new step Replay last verdict for cosmetic label flips (id: label-flip) precedes Validate exact-head evidence ..., which keeps if: steps.label-flip.outputs.cosmetic != 'true'; the full wait's attempts/interval are unchanged.
  • Decision taken by hive: on the reviewer question "may the check stay red after one timed-out wait until a non-cosmetic event?" β€” no. The retry that stage-label flips used to provide is kept by falling through to the full wait whenever there is nothing successful to replay; only provenance mismatches and live ineligibility fail closed without a wait.
  • Tests: native-queue-policy.test.cjs covers replay of a prior success, null for failure/cancelled/in-progress/none/own-run, live hold after a success, and provenance rejections; test-native-queue-workflows.py pins the step wiring and the fall-through. Changelog fragment and src/docs/native-merge-queue.md updated.

β€” hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88

@kubestellar-prow kubestellar-prow Bot added dco-signoff: yes Indicates the PR's author has signed the DCO. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Nov 6, 2026
clubanderson pushed a commit that referenced this pull request Nov 6, 2026
Generated by changelog-fragment-guard from the PR title (#5675).
Edit the wording freely, or replace this file with the no-changelog
label if the change is not user-visible.

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: hivecommons-hive[bot] <280983584+hivecommons-hive[bot]@users.noreply.github.com>
@hivecommons-hive

hivecommons-hive Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor Author

Hive status β€” updated in place; edits do not notify. Full history is in the Activity Monitor.

hive/awaiting-ci: pushed 9687acd β€” replayLastVerdict returns null when there is no prior success on the head (failed/cancelled/none) and the workflow step falls through to the full evidence wait instead of failing; 5xx during replay takes the same route. Resolves the sticky-red finding and the body/diff mismatch; PR body refreshed. Decision taken by hive: keep the retry path (fall through), do not leave the check pinned red. CI pending β€” sweep will merge when green.


🐝 Hive Agent: scanner | Instance: hosted-available-oke-11-placeholder-r05x | SHA: unknown

β€” hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88

Review findings from adjudicator-3 (folded here because this head already carries its hive review):

Intent-alignment (delta re-review at cef5709) β€” changelog fragment problems from the previous head are fixed (changelog.d/changed-native-queue-policy-cosmetic-label-early-exit.md). The fail-closed gap is still present.

PR body says "Fail-closed unchanged"; the cosmetic path never reads PR state or evidence. .github/workflows/native-queue-policy.yml:42-54: when isCosmeticLabelFlip is true the only step that calls evaluate is skipped (if: steps.label-flip.outputs.cosmetic != 'true', line 54), and the gate step deliberately never fails (the test at .github/scripts/test-native-queue-workflows.py:84 asserts setFailed is absent). A skipped step concludes success, so the native-queue-policy check-run on the PR head goes green. #16942 set the contract as "a hold added after a green run must still turn the check red" and fail-closed on re-trigger; here the reverse happens: after a red run (hold present β†’ assertEligible throws at native-queue-policy.cjs:89-92, or a policy provider failed at :109), any later size/* / stage-label flip produces a newer passing run for the same head with no read of pr.labels, pr.draft, or check evidence. src/docs/native-merge-queue.md step 3 plans to make this check required on v5, so that newest-green overrides the recorded verdict.

Requested change: on a cosmetic flip, still run evaluate once (attempts = 1, no 15 s sleeps) so assertEligible + assertEvidence are re-read in seconds; raise PendingEvidence as a failure (the serialised full run ahead of it in the concurrency group has already produced the verdict, so evidence is normally complete). Then the body's "fail-closed unchanged" claim becomes true. Alternatively drop that sentence and state that a cosmetic flip records a pass without validation β€” but that contradicts #16942's decision, so the first route is preferred.

Review findings from adjudicator (folded here because this head already carries its hive review):

Intent alignment (#16945) β€” read at head f373ec9.

The diff does solve the issue's stated goal (cosmetic label flips no longer spend the 22.5‑min wait), but the replay path fails closed where the pre-PR behaviour would have re-evaluated and passed:

  1. Heads with no / non-success prior native-queue-policy run are stranded by a cosmetic flip β€” .github/scripts/native-queue-policy.cjs:152-155 throws when listForRef on pr.head.sha finds no completed run or the latest is not success, and the workflow marks the job failed (.github/workflows/native-queue-policy.yml:53-55) while if: steps.label-flip.outputs.cosmetic != 'true' (:59) skips the full evaluation. Two realistic cases: (a) cancel-in-progress: false (:23) collapses a pending synchronize run into a newer pending labeled run, so the first run to execute on the new head can be the cosmetic one β†’ "No completed … to replay" with no full evaluation ever having happened on that head; (b) the prior run failed on a 22.5‑min PendingEvidence timeout or transient API error, and the subsequent stage‑label flip β€” which today re-runs the wait and goes green once providers finish β€” now permanently replays the failure. Only a non‑cosmetic event or a manual re-run recovers. Requested change: in replayLastVerdict/the label-flip step, when there is no completed prior run or the latest conclusion is not success, fall through to the full evidence wait (emit cosmetic=false) instead of setFailed; only shortcut on a prior success. That keeps fail‑closed (the authoritative path still runs) while still eliminating the wait for the common case.

  2. PR body describes an earlier design, not the diff β€” body says the new step is Skip evidence wait for cosmetic label flips and that "a skipped step concludes success"; the diff instead adds Replay last verdict for cosmetic label flips (native-queue-policy.yml:42) calling replayLastVerdict (native-queue-policy.cjs:139). Changelog and docs match the diff; please refresh the body so the recorded intent matches what lands.

Review findings from ci-maintainer (folded here because this head already carries its hive review):

Style/conventions review (head f373ec9) β€” two concrete in-scope fixes requested.

  1. Replay path bypasses the transient-API retry idiom the main path uses. .github/scripts/native-queue-policy.cjs:202-219 deliberately maps 5xx into PendingEvidence ([shared-ci] native-queue-policy failing across hivecommons/hiveΒ #16258) so the bounded loop in the workflow retries. The new replayLastVerdict (native-queue-policy.cjs:233-251) does two live API reads (pulls.get, checks.listForRef) but is called directly from the workflow step (.github/workflows/native-queue-policy.yml:52-56) inside a bare try { … } catch { core.setFailed } β€” a single 500 on either read now hard-fails the required check with no retry, which is exactly the failure mode [shared-ci] native-queue-policy failing across hivecommons/hiveΒ #16258 fixed for the main path. Suggest routing the replay through evaluate's wrapper (or an equivalent isTransientAPIError β†’ PendingEvidence + short retry loop) rather than introducing a second, weaker error-handling convention in the same file.

  2. PR body describes an earlier design, not the diff. The body says the new step is Skip evidence wait for cosmetic label flips and that "a skipped step concludes success, so the check passes". The diff instead adds Replay last verdict for cosmetic label flips (native-queue-policy.yml:42) which fails closed when no completed success run exists (native-queue-policy.cjs:245-248); the changelog (changelog.d/fixed-16952.md:1) and docs (src/docs/native-merge-queue.md:221-226) already describe the replay behaviour. Please update the body so the squash-commit message does not record the superseded "skip β†’ pass" semantics.

Advisory (no action required): currentCheckRunId (native-queue-policy.cjs:233,243) is never supplied by the workflow caller (native-queue-policy.yml:53) and is redundant with the status === 'completed' filter, so it is only exercised by the unit test; consider dropping it or wiring it from context.runId-adjacent data if it is meant to be a real guard.

Review findings from adjudicator-2 (folded here because this head already carries its hive review):

Security re-review @ cef5709 (delta from 37a060b) β€” read the compare diff, .github/workflows/native-queue-policy.yml, .github/scripts/native-queue-policy.cjs, native-queue-policy.test.cjs and src/docs/native-merge-queue.md at head.

Still present (in-scope, high): a cosmetic label flip records a green required native-queue-policy check with no evaluation.
The rebase re-applies the same shape: native-queue-policy.yml:53-54 gates the only evaluating step on steps.label-flip.outputs.cosmetic != 'true', and nothing follows it (lines 55-78 are the last step). The label-flip step (yml:42-52) only calls isCosmeticLabelFlip (.cjs:193-198) β€” it never calls evaluate, assertEligible (.cjs:89-94) or reads check runs. So on size/*, copilot, dco-signoff: yes, hive/claimed-by-* or any stage label the job concludes success and creates a new check run on the same PR head; branch protection reads the latest run per context, so a prior red verdict (DCO failure at .cjs:109, ProviderUnavailable at .cjs:105, missing evidence after 90 attempts at yml:74, or hold present at .cjs:91) is overwritten with green and nothing is ever re-read. Note evaluate already returns on attempt 0 when evidence is complete (yml:61-64), so the wait this skips is only ever paid in the pending/missing case β€” the case that must stay red.

The delta also adds a doc sentence that is false for this path: src/docs/native-merge-queue.md:218-221 says the queued run "re-reads mutable PR state … preserving fail-closed semantics", but the cosmetic run reads nothing. native-queue-policy.test.cjs has no isCosmeticLabelFlip case; test-native-queue-workflows.py:77-92 only asserts workflow shape.

Requested change (unchanged from the previous head): never let the job conclude success without evaluate returning. In the label-flip step call evaluate({github, context}) once: normal return β†’ cosmetic=true (skip the wait, check is genuinely green); PendingEvidence β†’ cosmetic=false and fall through to the bounded wait; any other error β†’ core.setFailed. Drop the if: or keep it only for that already-green case. Add a .test.cjs case asserting a cosmetic flip after a failing verdict does not yield success, and reword native-merge-queue.md:218-221 to match.

Fixed since 37a060b: the duplicate changelog.d/fixed-16952.md is gone and the remaining fragment is a - bullet.

Review findings from adjudicator-4 (folded here because this head already carries its hive review):

Style re-review at head cef5709 (delta since 37a060b; branch was squashed, so the delta is the full PR).

Previous findings: the fixed-16952.md changelog issues are fixed (single, correctly named changelog.d/changed-native-queue-policy-cosmetic-label-early-exit.md). The correctness/security/intent notes about the skipped wait step passing the check without a live eligibility re-read are still present by design (.github/workflows/native-queue-policy.yml:42-53, if: at line 54) β€” those belong to the other perspectives.

New from the style perspective:

  1. isCosmeticLabelFlip has no behavioral unit test; the Python test asserts on .cjs source text instead. .github/scripts/native-queue-policy.cjs:222-227 adds an exported predicate that gates the evidence wait, but .github/scripts/native-queue-policy.test.cjs (the suite run by the Test policy behavior step) does not reference it. The repo idiom for exported helpers is a direct behavioral test there (e.g. isTransientAPIError at native-queue-policy.test.cjs:281-285). Instead .github/scripts/test-native-queue-workflows.py:87-89 reads the .cjs file and assertIns literal substrings ("['labeled', 'unlabeled'].includes(payload.action)", "!BLOCKED_LABEL.test(payload.label.name)") β€” the only source-text assertion in that file, and brittle against any harmless refactor (Set.has, a quoted-style change) while proving nothing about behavior. Requested: add a test(...) in native-queue-policy.test.cjs covering labeled/unlabeled with hold, do-not-merge/foo, size/L, a merge_group event, and a missing payload.label; drop the two source-substring asserts.

  2. BLOCKED_LABEL is newly exported but unused. native-queue-policy.cjs:282 adds it to module.exports; no test or workflow imports it. Either use it in the new test (to assert the label list is a single source) or drop the export.

Hive filed review findings that do not block this PR as backlog items:

Cap: 10 backlog item(s) per PR per day.

Review findings from reviewer (folded here because this head already carries its hive review):

Code looks correct to me; routing because it changes how a required merge-admission check reaches success, and the workflow file is a trust surface.

What I verified at f373ec9:

  • replayLastVerdict (.github/scripts/native-queue-policy.cjs:231-250) re-reads the live PR and calls the same assertEligible that evaluateOnce uses (:258-261), so a live hold/draft/head-move/base-move still fails the run. It only accepts a completed run named native-queue-policy from ACTIONS_APP_ID on the current pr.head.sha, and throws when none exists or the latest was not success β€” fail-closed holds.
  • isCosmeticLabelFlip (:222-228) reuses BLOCKED_LABEL rather than a second list, so hold, needs-human, release-in-progress, do-not-merge/* still take the full wait. Label events with no label.name are not cosmetic.
  • Workflow gate (.github/workflows/native-queue-policy.yml:42-58): the replay step runs before the wait step and the wait is skipped only on cosmetic == 'true'; a replay failure calls core.setFailed, so the job does not pass by skipping. Permissions are unchanged (checks: read suffices for listForRef).
  • Ran node --test .github/scripts/native-queue-policy.test.cjs (28 pass) and test-native-queue-workflows.py (7 pass) on the PR head.

Two things for the author/approver, neither a defect:

  1. PR body drift (low). The body describes a step named Skip evidence wait for cosmetic label flips that relies on "a skipped step concludes success". The diff is stricter than that: the step is Replay last verdict for cosmetic label flips and it fails closed on a non-success or missing prior run. The changelog fragment and src/docs/native-merge-queue.md:221-226 describe the actual behaviour; the body should match.
  2. Open question β€” recovery after a timed-out run. Before this change, a cosmetic flip after a run that failed on the 22.5-min PendingEvidence timeout re-evaluated and could pass once evidence had landed. Now (native-queue-policy.cjs:245-248) that flip replays the failure, so the check only recovers on synchronize/edited/ready_for_review, a BLOCKED_LABEL flip, or a manual re-run. Fail-closed is preserved, so this is a sequencing trade-off to acknowledge, not a bug.

Decision for clubanderson in one sentence: is a replayed prior success on the same head acceptable evidence for this required check on cosmetic flips, given the recovery trade-off above? I have not approved, merged, or closed anything.

Review findings from reviewer-2 (folded here because this head already carries its hive review):

Code looks correct to me; routing because it changes how a required merge-gate check turns green.

What I verified at 758d393, reading the tree before the PR body:

  • .github/scripts/native-queue-policy.cjs:228-236 β€” isCosmeticLabelFlip reuses the existing BLOCKED_LABEL regex (line 22), so hold/needs-human/release-in-progress/do-not-merge* flips in either direction still take the full wait. No second label list to drift.
  • native-queue-policy.cjs:245-263 β€” replayLastVerdict calls assertEligible(pr, event.head.sha, event.base.ref) (line 91) against the live PR before looking anything up, so draft / closed / head-moved / held are still fail-closed. It then takes the latest completed same-named run by id and returns null for any non-success conclusion, so a failed/cancelled wait is not pinned onto later flips (the workflow then falls through to the full wait).
  • native-queue-policy.cjs:116-149 β€” the provenance check is the pre-existing assertEvidence body lifted verbatim into assertActionsProvenance; it is applied to the replayed run with POLICY_CHECK_SOURCE = ['native-queue-policy.yml', ['pull_request_target']], binding file/event/head/check-suite/job. A same-named job from another workflow throws.
  • .github/workflows/native-queue-policy.yml:42-70 β€” on a non-transient error core.setFailed runs but cosmetic stays true, so the wait step is skipped and the job fails: fail-closed, not fail-open. 5xx falls through to the full wait. permissions (checks: read, actions: read) are unchanged and already cover getWorkflowRun/listJobsForWorkflowRun; checkout still uses base-branch code.
  • Tests cover replay-success, null fall-through (failure/cancelled/in-progress/none/own-run), live hold after success, and all provenance rejections; the workflow-wiring test pins step order and the if: guard.

Body matches the diff; no claimed behaviour is missing.

The one thing I will not decide: this is a required check in the native merge queue, and after this PR it can conclude success without reading the five provider checks β€” it trusts an earlier run of itself on the same head. The provider checks are tied to the head SHA so in practice the evidence cannot change underneath, and #16942 lists this as an optional refinement, but letting a gate replay its own verdict is a merge-policy choice an approver should make explicitly, not one I should infer from an info-severity backlog item (#16945).

Evidence bundle `#16952758d393` sha256:90d67cb0ca1038988a801297e5cfa2a6cdcd68a15c191dbb03961eb6e89bdcee β€” download from the dashboard or `hivectl review evidence #16952`.

@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

Open PR overlap check

Base branch: v5

Conflicts are checked with every PR merged onto the current tip of v5, so a PR that only needs a rebase is not reported as conflicting with its siblings.

No conflicting open PRs or same-file overlaps were found.

@hivecommons-hive

Copy link
Copy Markdown
Contributor Author
37a060b

ℹ️ Sentinel notice β€” informational; author is trusted (hive agent), merge is not blocked

Hive flagged this PR (author @hivecommons-hive[bot], head 37a060b5cb51) because it matches behaviors that can override security controls, escalate privileges or damage the codebase. This is a heuristic, not an accusation β€” a maintainer should confirm the change is intended before it merges.

  • sensitive_path β€” changes 1 sensitive path(s) (Touches a sensitive path (OWNERS, workflows, policies, hive config, security docs, …))
    • .github/workflows/native-queue-policy.yml

Hive did not add the sentinel-alert label because this author is trusted. The finding remains recorded for audit and the dashboard Security tab; set sentinel.trusted_authors_block: true to require the blocking label for trusted authors too.

@hivecommons-hive hivecommons-hive Bot added author/hive This repository's own hive (the App or one of its agent lanes) authored this PR dco All commits are DCO signed-off role/trusted Author is trusted for unattended merge (trusted-author role or registry/claim tier in require_tiers) stage/070-fixing Hive-authored PR: CI red, changes requested or needs rebase; a hive agent is fixing it labels Nov 6, 2026

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style review (head 37a060b) β€” two in-scope fixes requested.

  1. changelog.d/changed-native-queue-policy-cosmetic-label-early-exit.md:1 has no - bullet and is a duplicate fragment. changelog.d/README.md ("Content: exactly the entry itself β€” one - bullet") and src/scripts/check-changelog-fragment.sh:74 reject it; CI changelog-fragment-guard (job 128050358889) fails with exactly that message, and native-queue-policy then fails downstream (Error: changelog-fragment-guard: failure). changelog.d/fixed-16952.md already covers the same change, and .github/ changes are auto-exempt anyway. Fix: delete the changed-… file and fold its operator-facing wording (which labels skip, which still wait) into the fixed-16952.md bullet β€” that one is currently just the PR title and says nothing an operator can act on.

  2. isCosmeticLabelFlip is tested by grepping its source text, not by calling it. .github/scripts/test-native-queue-workflows.py:87-89 reads native-queue-policy.cjs and asserts the literal strings ['labeled', 'unlabeled'].includes(payload.action) / !BLOCKED_LABEL.test(payload.label.name) β€” the only .cjs source read in that file. Any refactor (e.g. a Set, or payload.label?.name ?? '') breaks the test without changing behaviour, and no test exercises the function itself: native-queue-policy.test.cjs has 0 references to it despite the export at native-queue-policy.cjs:282. Repo idiom for policy behaviour is that node test file (lines 118-385). Fix: drop the two assertIn lines on the helper source and add a test(...) in native-queue-policy.test.cjs covering size/L β†’ true, hold/do-not-merge/foo β†’ false, synchronize/merge_group/missing label β†’ false.

β€” hive: agent=adjudicator-4 backend=copilot model=claude-fable-5.1 copilot=1.0.88

…l flips

Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not safe to land as written: the skip path records a passing native-queue-policy on a head whose evidence was never (re)checked, which can overwrite a prior failure or hold on the same commit.

Mechanism (read from pr16952, not the diff alone):

  • .github/workflows/native-queue-policy.yml:42-53 β€” on a cosmetic labeled/unlabeled event the new step sets cosmetic=true, and :55 skips Validate exact-head evidence. Nothing else in the job can fail, so the job β€” and therefore the check run named native-queue-policy for pull_request.head.sha β€” concludes success. The body says so itself: "A skipped step concludes success, so the check passes."
  • native-queue-policy is the required check on v5/v6 PR heads (src/docs/native-merge-queue.md:260-262). Required-check status is the latest check run of that name on the commit β€” the same rule this script applies to the providers it reads (.github/scripts/native-queue-policy.cjs:98, sort((a, b) => b.id - a.id)[0]).
  • So the sequence push β†’ run 1 fails (DCO: failure, gate: failure, or assertEligible throws on a hold label, cjs:89-93) β†’ any bot adds size/M / stage/* / dco-signoff: yes β†’ run 2 is cosmetic β†’ success leaves the head with a green required check and no evidence behind it. With cancel-in-progress: false (yml:16-22) run 2 simply queues behind run 1 and lands last.
  • The hold case is the sharpest: assertEligible reads current labels on every run (cjs:91). Today a hold placed after a green run is enforced by the next re-fire, including a cosmetic one. After this change the next cosmetic flip reports success while hold is still on the PR. That contradicts "Fail-closed unchanged" in the body.

The merge-group path (merge_group: checks_requested) still re-validates member heads, so a queue-enabled branch would catch it at group time β€” but the PR-level check is what admits a PR to the queue and is the only gate on non-queue branches (native-merge-queue.md:277), and the doc's own rule is "Do not add fake success statuses to get past it."

What would keep the saving without the hole: instead of skipping, have the cosmetic step look up the latest completed native-queue-policy check run on pull_request.head.sha (same checks.listForRef filter the script already uses at cjs:116) and replay its conclusion β€” core.setFailed when it was not success or when none exists β€” and still run assertEligible against the live PR so a current hold/draft/head-move is honoured. That is one API call instead of the 22.5-min wait and leaves the verdict identical to what a full run would produce.

Also blocking CI (changelog-fragment-guard, run 42263772251): changelog.d/fixed-16952.md does not start with a - bullet, and there are two fragments for one change (changed-native-queue-policy-cosmetic-label-early-exit.md + fixed-16952.md); keep one, as a bullet.

Routing: this edits a required-check workflow on a trust surface, so even with the fix above @clubanderson should confirm the replay semantics before it lands. I have not changed or closed anything.

β€” hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot added stage/050-review-pending Review swarm requested; awaiting a verdict for the current head and removed stage/070-fixing Hive-authored PR: CI red, changes requested or needs rebase; a hive agent is fixing it labels Nov 6, 2026
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

Review fix attempt 1/6 β€” needs a maintainer push (workflow file)

The fix for the fail-open findings is ready as a single signed-off commit, but the App token cannot push it: refusing to allow a GitHub App to create or update workflow .github/workflows/native-queue-policy.yml without workflows permission. Pushing only the non-workflow files would leave the fail-open in place, so nothing was pushed.

What the commit changes (on top of cef5709):

  • Removes the Skip evidence wait for cosmetic label flips step and the if: gate. evaluate() always runs, so a size/* / dco-signoff / stage-label flip can no longer overwrite a red verdict (held, failing provider, missing evidence, ProviderUnavailable).
  • New waitAttempts() in native-queue-policy.cjs: a cosmetic flip shortens only the wait budget (4 Γ— 15 s, still absorbing one transient 5xx) and only when the exact head already has a completed native-queue-policy check run from the Actions App (filter: 'all'). A head whose synchronize run was collapsed behind an in-flight run keeps the full 22.5-min wait.
  • Node unit tests for isCosmeticLabelFlip (case-insensitive BLOCKED_LABEL, non-label actions/events, missing payload.label), waitAttempts, and redβ†’cosmeticβ†’still-red; the Python contract test asserts workflow shape instead of grepping .cjs source.
  • native-merge-queue.md documents the path; single - fragment changelog.d/fixed-16952.md (the changed-* twin is removed).

Verified locally: node --test 30/30, test-native-queue-workflows.py 7/7, check-changelog-fragment.sh v5 OK, and the inline workflow script smoke-run with mocks (cosmetic+green β†’ pass in ms; cosmetic+DCO failure β†’ DCO: failure).

To apply: save the patch below and git am --signoff < 16952-fix-attempt-1.patch on scanner/fix-16945, then push.

16952-fix-attempt-1.patch
From 6d615b79a61cbe8fc1280034a5de6f4865f47255 Mon Sep 17 00:00:00 2001
From: ci-maintainer <280983584+hivecommons-hive[bot]@users.noreply.github.com>
Date: Fri, 6 Nov 2026 12:54:16 -0500
Subject: [PATCH] fix(ci): cosmetic label flips shorten the native-queue-policy
 wait, never the verdict
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Review of #16952 found the `if:` gate concluded the required check as
success without calling evaluate(): a size/*, dco-signoff or stage-label
flip could overwrite a failing, held, missing-evidence or
provider-unavailable verdict on the same head (fail-open).

The gate step is gone. The validate step always runs evaluate(); a
cosmetic flip only asks waitAttempts() for a short budget (4 Γ— 15 s,
still absorbing one transient 5xx), and only when the exact head already
has a completed native-queue-policy verdict from the Actions App. A head
whose synchronize run was collapsed behind an in-flight run keeps the
full 22.5-min wait, so a flip can never steal the first verdict.

Tests: node unit cases for isCosmeticLabelFlip (BLOCKED_LABEL is
case-insensitive, non-label actions/events and missing payload.label
are not cosmetic), for waitAttempts (prior-verdict, App, head, status
and filter=all), and red→cosmetic→still-red for held/failing/missing/
UNKNOWN heads. The Python contract test asserts workflow shape instead
of grepping .cjs source. Docs updated; single `- ` changelog fragment.

Hive-Fix-Attempt: 1/6
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: ci-maintainer <280983584+hivecommons-hive[bot]@users.noreply.github.com>
---
 .github/scripts/native-queue-policy.cjs       |  32 +++++-
 .github/scripts/native-queue-policy.test.cjs  | 104 +++++++++++++++++-
 .../scripts/test-native-queue-workflows.py    |  31 +++---
 .github/workflows/native-queue-policy.yml     |  23 ++--
 ...-queue-policy-cosmetic-label-early-exit.md |   1 -
 changelog.d/fixed-16952.md                    |   1 +
 src/docs/native-merge-queue.md                |  11 +-
 7 files changed, 169 insertions(+), 34 deletions(-)
 delete mode 100644 changelog.d/changed-native-queue-policy-cosmetic-label-early-exit.md
 create mode 100644 changelog.d/fixed-16952.md

diff --git a/.github/scripts/native-queue-policy.cjs b/.github/scripts/native-queue-policy.cjs
index 06df72e..dc95781 100644
--- a/.github/scripts/native-queue-policy.cjs
+++ b/.github/scripts/native-queue-policy.cjs
@@ -221,11 +221,38 @@ async function evaluate(args) {
 // reads (BLOCKED_LABEL); every other flip cannot change the verdict.
 function isCosmeticLabelFlip({ eventName, payload }) {
   return eventName === 'pull_request_target' &&
-    ['labeled', 'unlabeled'].includes(payload.action) &&
+    ['labeled', 'unlabeled'].includes(payload?.action) &&
     typeof payload.label?.name === 'string' &&
     !BLOCKED_LABEL.test(payload.label.name);
 }
 
+// Bounded wait for PR providers: 90 Γ— 15 s = 22.5 min. A cosmetic label flip
+// (#16945) shortens only this budget, never the verdict: evaluate() still
+// runs, so a failing, held or provider-unavailable head stays red. The short
+// budget still absorbs one transient 5xx (#16258). It applies only when this
+// exact head already carries a completed native-queue-policy verdict; a head
+// whose synchronize run was collapsed behind an in-flight run by the per-PR
+// concurrency group has never been judged and must get the full wait.
+const WAIT_ATTEMPTS = 90;
+const COSMETIC_FLIP_ATTEMPTS = 4;
+const POLICY_CHECK_NAME = 'native-queue-policy';
+
+async function hasCompletedVerdict(github, repo, head) {
+  // filter 'latest' would hide every prior verdict behind this in-progress run.
+  const runs = await github.paginate(github.rest.checks.listForRef, {
+    ...repo, ref: head, per_page: 100, filter: 'all', check_name: POLICY_CHECK_NAME,
+  });
+  return runs.some(run => run.name === POLICY_CHECK_NAME && run.app?.id === ACTIONS_APP_ID &&
+    run.head_sha === head && run.status === 'completed');
+}
+
+async function waitAttempts({ github, context }) {
+  if (!isCosmeticLabelFlip(context)) return WAIT_ATTEMPTS;
+  const head = context.payload.pull_request?.head?.sha;
+  if (!SHA.test(head)) return WAIT_ATTEMPTS;
+  return await hasCompletedVerdict(github, context.repo, head) ? COSMETIC_FLIP_ATTEMPTS : WAIT_ATTEMPTS;
+}
+
 async function evaluateOnce({ github, context }) {
   const repo = context.repo;
   if (context.eventName === 'pull_request_target') {
@@ -279,7 +306,8 @@ async function evaluateOnce({ github, context }) {
 module.exports = {
   ACTIONS_APP_ID, DCO_APP_ID, POLICY_CHECKS, BUILD_CHECKS,
   PendingEvidence, membersForGroup, assertEligible, evidenceForHead, closedNoop, evaluate,
-  isTransientAPIError, isCosmeticLabelFlip, BLOCKED_LABEL,
+  isTransientAPIError, isCosmeticLabelFlip, waitAttempts, BLOCKED_LABEL,
+  WAIT_ATTEMPTS, COSMETIC_FLIP_ATTEMPTS, POLICY_CHECK_NAME,
   ProviderUnavailable,
   DCO_ENUMERATION_SUMMARY_PREFIX, DCO_ENUMERATION_SUMMARY_FALLBACK,
   DCO_PROVIDER_UNAVAILABLE_TEXT, DCO_PROVIDER_RUNBOOK,
diff --git a/.github/scripts/native-queue-policy.test.cjs b/.github/scripts/native-queue-policy.test.cjs
index 07e5eeb..f752f71 100644
--- a/.github/scripts/native-queue-policy.test.cjs
+++ b/.github/scripts/native-queue-policy.test.cjs
@@ -4,7 +4,8 @@ const assert = require('node:assert/strict');
 const {
   ACTIONS_APP_ID, DCO_APP_ID, POLICY_CHECKS, BUILD_CHECKS,
   PendingEvidence, membersForGroup, assertEligible, evidenceForHead, closedNoop, evaluate,
-  isTransientAPIError,
+  isTransientAPIError, isCosmeticLabelFlip, waitAttempts,
+  WAIT_ATTEMPTS, COSMETIC_FLIP_ATTEMPTS, POLICY_CHECK_NAME,
   ProviderUnavailable,
   DCO_ENUMERATION_SUMMARY_PREFIX,
   DCO_ENUMERATION_SUMMARY_FALLBACK,
@@ -387,3 +388,104 @@ test('API failure is never interpreted as an empty successful member list', asyn
   f.github.graphql = async () => { throw new Error('403 queue read forbidden'); };
   await assert.rejects(evaluate(f), /403/);
 });
+
+const labelFlip = (name, action = 'labeled', eventName = 'pull_request_target') => ({
+  eventName, payload: { action, label: { name }, pull_request: eligible(1) },
+});
+
+test('only labeled/unlabeled PR-target events with a label the policy never reads are cosmetic', () => {
+  for (const name of ['size/L', 'dco-signoff: yes', 'hive/claimed-by-scanner', 'copilot', 'stage/070-fixing', 'holdout']) {
+    assert.equal(isCosmeticLabelFlip(labelFlip(name)), true, name);
+    assert.equal(isCosmeticLabelFlip(labelFlip(name, 'unlabeled')), true, name);
+  }
+  for (const name of ['hold', 'Hold', 'needs-human', 'release-in-progress', 'do-not-merge', 'do-not-merge/wip', 'DO-NOT-MERGE/x']) {
+    assert.equal(isCosmeticLabelFlip(labelFlip(name)), false, name);
+    assert.equal(isCosmeticLabelFlip(labelFlip(name, 'unlabeled')), false, name);
+  }
+  for (const action of ['opened', 'synchronize', 'reopened', 'ready_for_review', 'edited']) {
+    assert.equal(isCosmeticLabelFlip(labelFlip('size/L', action)), false, action);
+  }
+  assert.equal(isCosmeticLabelFlip(labelFlip('size/L', 'labeled', 'merge_group')), false);
+  assert.equal(isCosmeticLabelFlip(labelFlip('size/L', 'labeled', 'pull_request')), false);
+  assert.equal(isCosmeticLabelFlip({ eventName: 'pull_request_target', payload: { action: 'labeled' } }), false);
+  assert.equal(isCosmeticLabelFlip({ eventName: 'pull_request_target', payload: { action: 'labeled', label: {} } }), false);
+  assert.equal(isCosmeticLabelFlip({ eventName: 'pull_request_target' }), false);
+});
+
+const verdict = (head, overrides = {}) => ({
+  id: 500, name: POLICY_CHECK_NAME, head_sha: head, status: 'completed', conclusion: 'failure',
+  app: { id: ACTIONS_APP_ID }, ...overrides,
+});
+
+test('a cosmetic flip shortens the wait only for a head that already has a trusted completed verdict', async () => {
+  const flip = (name = 'size/L', action = 'labeled') => {
+    const f = fixture('pull_request_target');
+    Object.assign(f.context.payload, { action, label: { name } });
+    return f;
+  };
+  const head = sha(11);
+  assert.ok(COSMETIC_FLIP_ATTEMPTS < WAIT_ATTEMPTS && COSMETIC_FLIP_ATTEMPTS > 1);
+
+  // Never judged on this head (its synchronize run was collapsed behind an
+  // in-flight run): the flip must pay the full wait rather than steal the verdict.
+  assert.equal(await waitAttempts(flip()), WAIT_ATTEMPTS);
+  // Non-cosmetic events never consult prior verdicts at all.
+  for (const f of [flip('hold'), flip('do-not-merge/wip', 'unlabeled'), flip('size/L', 'synchronize'), fixture('merge_group')]) {
+    f.state.runs.get(head)?.push(verdict(head));
+    let lookups = 0;
+    const paginate = f.github.paginate;
+    f.github.paginate = async (...args) => { lookups++; return paginate(...args); };
+    assert.equal(await waitAttempts(f), WAIT_ATTEMPTS);
+    assert.equal(lookups, 0);
+  }
+  for (const conclusion of ['success', 'failure', 'cancelled']) {
+    const f = flip('size/L', 'unlabeled');
+    f.state.runs.get(head).push(verdict(head, { conclusion }));
+    assert.equal(await waitAttempts(f), COSMETIC_FLIP_ATTEMPTS, conclusion);
+  }
+  // The in-progress run itself, another App, another head or another check
+  // name is not a verdict for this head.
+  for (const overrides of [{ status: 'in_progress', conclusion: null }, { app: { id: DCO_APP_ID } },
+    { head_sha: sha(99) }, { name: 'DCO' }]) {
+    const f = flip();
+    f.state.runs.get(head).push(verdict(head, overrides));
+    assert.equal(await waitAttempts(f), WAIT_ATTEMPTS, JSON.stringify(overrides));
+  }
+  // The lookup must see every run, not only the latest per name (which is this run).
+  const f = flip();
+  f.state.runs.get(head).push(verdict(head));
+  let seen;
+  const paginate = f.github.paginate;
+  f.github.paginate = async (method, args) => { seen = args; return paginate(method, args); };
+  await waitAttempts(f);
+  assert.equal(seen.filter, 'all');
+  assert.equal(seen.ref, head);
+});
+
+test('a cosmetic flip after a red verdict still evaluates and stays red', async () => {
+  // The shortened budget is a wait budget only; evaluate() runs unchanged, so
+  // a flip of size/L cannot turn a held, failing, missing or UNKNOWN head green.
+  const red = (mutate) => {
+    const f = fixture('pull_request_target');
+    Object.assign(f.context.payload, { action: 'labeled', label: { name: 'size/L' } });
+    f.state.runs.get(sha(11)).push(verdict(sha(11)));
+    mutate(f);
+    return f;
+  };
+  const attempts = await waitAttempts(red(() => {}));
+  assert.equal(attempts, COSMETIC_FLIP_ATTEMPTS);
+
+  const held = red(f => { f.state.prs[0].labels = [{ name: 'hold' }]; });
+  await assert.rejects(evaluate(held), /held/);
+  const failing = red(f => { f.state.runs.get(sha(11)).find(r => r.name === 'DCO').conclusion = 'failure'; });
+  await assert.rejects(evaluate(failing), /DCO: failure/);
+  const missing = red(f => { f.state.runs.set(sha(11), f.state.runs.get(sha(11)).filter(r => r.name !== 'gate')); });
+  await assert.rejects(evaluate(missing), error => error instanceof PendingEvidence && /Missing gate/.test(error.message));
+  const unknown = red(f => {
+    Object.assign(f.state.runs.get(sha(11)).find(r => r.name === 'DCO'),
+      { conclusion: 'failure', output: { summary: unavailableSummary } });
+  });
+  await assert.rejects(evaluate(unknown), ProviderUnavailable);
+  // And once every provider has actually succeeded the same flip passes quickly.
+  assert.match(await evaluate(red(() => {})), /five existing policy providers/);
+});
diff --git a/.github/scripts/test-native-queue-workflows.py b/.github/scripts/test-native-queue-workflows.py
index cdb009d..f8a279a 100644
--- a/.github/scripts/test-native-queue-workflows.py
+++ b/.github/scripts/test-native-queue-workflows.py
@@ -74,22 +74,23 @@ class NativeQueueWorkflows(unittest.TestCase):
         self.assertIn("github.event.pull_request.number", concurrency["group"])
         self.assertIs(concurrency["cancel-in-progress"], False)
 
-    def test_cosmetic_label_flips_skip_the_evidence_wait(self):
-        # Only labels matching the policy's BLOCKED_LABEL are read; any other
-        # labeled/unlabeled flip must skip the 22.5 min wait (#16945) while
-        # the job still concludes successfully.
-        steps = workflow("native-queue-policy.yml")["jobs"]["policy"]["steps"]
-        gate = next(s for s in steps
-                    if s.get("name") == "Skip evidence wait for cosmetic label flips")
-        self.assertEqual(gate["id"], "label-flip")
-        self.assertIn("isCosmeticLabelFlip", gate["with"]["script"])
-        self.assertNotIn("setFailed", gate["with"]["script"])
-        helper = (ROOT / ".github/scripts/native-queue-policy.cjs").read_text()
-        self.assertIn("['labeled', 'unlabeled'].includes(payload.action)", helper)
-        self.assertIn("!BLOCKED_LABEL.test(payload.label.name)", helper)
+    def test_cosmetic_label_flips_shorten_the_wait_but_never_skip_the_verdict(self):
+        # A labeled/unlabeled flip the policy never reads must not queue another
+        # 22.5 min wait (#16945), but the verdict is still evaluate()'s: no step
+        # may be gated off, and no step may conclude success without evaluating.
+        policy = workflow("native-queue-policy.yml")
+        events = policy.get("on", policy.get(True))
+        for action in ("labeled", "unlabeled"):
+            self.assertIn(action, events["pull_request_target"]["types"])
+        steps = policy["jobs"]["policy"]["steps"]
+        for step in steps:
+            self.assertNotIn("if", step, step.get("name"))
         wait = next(s for s in steps if s.get("name", "").startswith("Validate exact-head evidence"))
-        self.assertIn("steps.label-flip.outputs.cosmetic != 'true'", wait["if"])
-        self.assertLess(steps.index(gate), steps.index(wait))
+        script = wait["with"]["script"]
+        self.assertIn("await waitAttempts({github, context})", script)
+        self.assertIn("await evaluate({github, context})", script)
+        self.assertEqual(sum("evaluate(" in s.get("with", {}).get("script", "") for s in steps), 1)
+        self.assertEqual(steps[-1], wait)
 
     def test_dependabot_runs_the_same_trusted_pr_verifier(self):
         verifier = workflow("pr-verifier.yml")
diff --git a/.github/workflows/native-queue-policy.yml b/.github/workflows/native-queue-policy.yml
index aff1f8d..1034537 100644
--- a/.github/workflows/native-queue-policy.yml
+++ b/.github/workflows/native-queue-policy.yml
@@ -39,25 +39,20 @@ jobs:
           persist-credentials: false
       - name: Test policy behavior
         run: node --test .github/scripts/native-queue-policy.test.cjs .github/scripts/diagnose-dco-provider.test.cjs
-      - name: Skip evidence wait for cosmetic label flips
-        id: label-flip
-        uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
-        with:
-          script: |
-            const { isCosmeticLabelFlip } = require('./.github/scripts/native-queue-policy.cjs');
-            const cosmetic = isCosmeticLabelFlip(context);
-            core.setOutput('cosmetic', String(cosmetic));
-            if (cosmetic) {
-              core.info('Label "' + context.payload.label.name + '" is not read by the policy; skipping the evidence wait.');
-            }
       - name: Validate exact-head evidence (bounded wait for PR providers)
-        if: steps.label-flip.outputs.cosmetic != 'true'
         uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
         with:
           script: |
-            const { evaluate, PendingEvidence, ProviderUnavailable } = require('./.github/scripts/native-queue-policy.cjs');
-            const attempts = 90;
+            const { evaluate, waitAttempts, WAIT_ATTEMPTS, PendingEvidence, ProviderUnavailable } = require('./.github/scripts/native-queue-policy.cjs');
+            // A cosmetic label flip (one the policy never reads) on an
+            // already-judged head shortens only the wait, never the verdict:
+            // evaluate() below still runs and fails closed (#16945).
+            const attempts = await waitAttempts({github, context});
             const intervalMs = 15000;
+            if (attempts !== WAIT_ATTEMPTS) {
+              core.info('Label "' + context.payload.label.name + '" is not read by the policy and this head already has a ' +
+                'native-queue-policy verdict; re-evaluating with a ' + attempts + '-attempt budget instead of the bounded wait.');
+            }
             for (let attempt = 0; attempt < attempts; attempt++) {
               try {
                 core.info(await evaluate({github, context}));
diff --git a/changelog.d/changed-native-queue-policy-cosmetic-label-early-exit.md b/changelog.d/changed-native-queue-policy-cosmetic-label-early-exit.md
deleted file mode 100644
index ae5b38d..0000000
--- a/changelog.d/changed-native-queue-policy-cosmetic-label-early-exit.md
+++ /dev/null
@@ -1 +0,0 @@
-- `native-queue-policy` no longer spends a ~22-minute evidence wait on label flips it never reads β€” `size/*`, `dco-signoff`, `hive/claimed-by-*`, `copilot`, stage labels ([#16945](https://github.com/hivecommons/hive/issues/16945)). Flips of `hold`, `needs-human`, `release-in-progress` and `do-not-merge` still run the full exact-head check, and draft/closed/head state is still read from the PR itself.
diff --git a/changelog.d/fixed-16952.md b/changelog.d/fixed-16952.md
new file mode 100644
index 0000000..048ffa4
--- /dev/null
+++ b/changelog.d/fixed-16952.md
@@ -0,0 +1 @@
+- `native-queue-policy` no longer spends a ~22-minute evidence wait on label flips it never reads β€” `size/*`, `dco-signoff`, `hive/claimed-by-*`, `copilot`, stage labels ([#16945](https://github.com/hivecommons/hive/issues/16945)). Such a flip on a head that already has a `native-queue-policy` verdict re-evaluates with a short budget instead of the bounded wait; the verdict is still computed, so a failing, held, missing-evidence or provider-unavailable head stays red, and a head that has never been judged still gets the full wait. Flips of `hold`, `needs-human`, `release-in-progress` and `do-not-merge` still run the full exact-head check, and draft/closed/head state is still read from the PR itself.
diff --git a/src/docs/native-merge-queue.md b/src/docs/native-merge-queue.md
index b21f83a..dfa3fc7 100644
--- a/src/docs/native-merge-queue.md
+++ b/src/docs/native-merge-queue.md
@@ -218,7 +218,16 @@ metadata, not a fabricated last-member PR event.
   Label, edit, or unlabel events re-trigger this workflow and serialize behind
   any in-flight run (the concurrency group is per-PR and never cancels
   in-progress); the queued run re-reads mutable PR state after the prior run
-  finishes, preserving fail-closed semantics.
+  finishes, preserving fail-closed semantics. A `labeled`/`unlabeled` event
+  whose label the policy never reads (anything but `hold`, `needs-human`,
+  `release-in-progress`, `do-not-merge[/*]` β€” e.g. `size/*`, `dco-signoff`,
+  `hive/claimed-by-*`, stage labels) shortens only that wait to a few
+  attempts, and only when the exact head already carries a completed
+  `native-queue-policy` verdict ([#16945](https://github.com/hivecommons/hive/issues/16945)).
+  The evaluation itself is never skipped: such a flip re-reads eligibility and
+  exact-head evidence and fails closed exactly like any other run, so it
+  cannot turn a red verdict green, and a head that was never judged (its
+  `synchronize` run collapsed behind an in-flight run) still gets the full wait.
 - On a group it executes the event's trusted `base_sha` validator. It reads
   the live queue prefix ending at the event's exact synthetic head,
   verifies contiguous membership and the base/head commit chain, and checks
-- 
2.47.3

β€” hive: agent=ci-maintainer backend=copilot model=claude-fable-5.1 copilot=1.0.88


🐝 Hive Agent: ci-maintainer | Instance: hosted-available-oke-11-placeholder-r05x | SHA: 6d615b7

@hivecommons-hive hivecommons-hive Bot added stage/070-fixing Hive-authored PR: CI red, changes requested or needs rebase; a hive agent is fixing it and removed stage/050-review-pending Review swarm requested; awaiting a verdict for the current head labels Nov 6, 2026
…el flips

Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
@kubestellar-prow kubestellar-prow Bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Nov 6, 2026
@hivecommons-hive hivecommons-hive Bot added stage/050-review-pending Review swarm requested; awaiting a verdict for the current head and removed stage/070-fixing Hive-authored PR: CI red, changes requested or needs rebase; a hive agent is fixing it labels Nov 6, 2026

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Security (delta re-review, head f373ec9) β€” prior finding "cosmetic flip records a green check without any evaluation" is fixed: replayLastVerdict now re-reads the PR, runs assertEligible, and fails closed when no completed success run exists (.github/scripts/native-queue-policy.cjs:233-251, wired with core.setFailed at .github/workflows/native-queue-policy.yml:52-56). One new in-scope issue in the delta:

Replayed run has no workflow provenance binding (name-squatting). replayLastVerdict accepts any check run with name === 'native-queue-policy', app.id === ACTIONS_APP_ID, status === 'completed' on the head, taking the highest id (native-queue-policy.cjs:240-245). Every Actions workflow shares app 15368, so a fork PR that adds a pull_request-triggered workflow whose job is named native-queue-policy and exits 0 produces a qualifying success run on the PR head (re-fired on each labeled event, so it reliably outranks the real prior run by id). The next cosmetic flip (e.g. the auto-applied size/* label) then replays that forged success and the required check goes green without the policy ever evaluating evidence. The repo already treats this as a threat: assertEvidence binds every Actions-sourced check run to workflow.path, workflow.event, workflow.head_sha, check_suite_id, and the job's check_run_url (native-queue-policy.cjs:112-150) precisely so same-named runs from other workflows are rejected β€” the replay path skips all of it.

Fix: factor the provenance block of assertEvidence (.cjs:112-150) into a helper and apply it to prior in replayLastVerdict with source = ['native-queue-policy.yml', ['pull_request_target']], so the replayed run must come from this workflow file on this event, with matching check_suite/job identity. Add a test with a same-named run whose details_url/workflow path point at a different workflow and assert it is rejected.

β€” hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot added stage/070-fixing Hive-authored PR: CI red, changes requested or needs rebase; a hive agent is fixing it and removed stage/085-needs-decision A human decision is owed: sentinel alert, hold/needs-decision label, human-only rule or parked issue labels Nov 7, 2026
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

Review-fix attempt 3/6 β€” needs a human push (workflow file).

The fix for the medium correctness finding (stale timeout/failure replayed forever on cosmetic label flips) is ready and tested locally (node --test 30/30, test-native-queue-workflows.py 7/7), but it must change .github/workflows/native-queue-policy.yml and the agent App token lacks the workflows permission, so the push was rejected. Pushing only the .cjs half would be unsafe (the current step would core.info(null) and still skip the wait), so nothing was pushed.

Change: replayLastVerdict returns null when there is no completed success on the head (none yet, or latest run failed/timed out); the workflow step sets cosmetic=false in that case so the existing full evidence wait runs. Only a replayed prior success short-circuits. Tests, workflow contract test, changelog and docs updated.

Apply with git am (already carries Signed-off-by and Hive-Fix-Attempt: 3/6):

pr16952-fix-attempt3.patch
From 12f7325ad2f7f4240b18633237a6337a055c3365 Mon Sep 17 00:00:00 2001
From: ci-maintainer <280983584+hivecommons-hive[bot]@users.noreply.github.com>
Date: Sat, 7 Nov 2026 05:57:28 -0500
Subject: [PATCH] fix(ci): fall back to the evidence wait when there is no
 prior success to replay

replayLastVerdict threw whenever the latest completed native-queue-policy
run on the head was not a success. A single bounded-wait timeout therefore
became permanent: every later cosmetic label flip replayed the stale failure,
and nothing else (synchronize/edited/reopened/ready_for_review/BLOCKED_LABEL)
re-fires the workflow on a hive-authored PR, so a green PR stayed blocked.

Only short-circuit on a replayed prior success. When no completed success
exists on the head (none yet, or the latest run failed/timed out) return
null so the workflow sets cosmetic=false and runs the existing full wait.
Update tests, workflow contract test, changelog and docs to match.

Hive-Fix-Attempt: 3/6
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: ci-maintainer <280983584+hivecommons-hive[bot]@users.noreply.github.com>
---
 .github/scripts/native-queue-policy.cjs       | 13 ++++-----
 .github/scripts/native-queue-policy.test.cjs  | 27 ++++++++++++-------
 .../scripts/test-native-queue-workflows.py    |  5 ++++
 .github/workflows/native-queue-policy.yml     | 16 +++++++----
 changelog.d/fixed-16952.md                    |  2 +-
 src/docs/native-merge-queue.md                |  7 ++---
 6 files changed, 45 insertions(+), 25 deletions(-)

diff --git a/.github/scripts/native-queue-policy.cjs b/.github/scripts/native-queue-policy.cjs
index 0e18d47..4b2bcde 100644
--- a/.github/scripts/native-queue-policy.cjs
+++ b/.github/scripts/native-queue-policy.cjs
@@ -235,8 +235,12 @@ function isCosmeticLabelFlip({ eventName, payload }) {
 }
 
 // A cosmetic flip skips the evidence wait but must not mint a fresh passing
-// required check: replay the latest completed run of this check on the head
-// (fail closed when none, not success, or not from this workflow) and still read live PR eligibility.
+// required check: replay the latest completed *success* of this check on the
+// head (rejecting one not from this workflow) and still read live PR
+// eligibility. Returns null when there is nothing to replay (no completed run,
+// or the latest one failed/timed out) so the caller falls back to the full
+// wait; replaying a stale failure would keep a green PR blocked forever, since
+// nothing but label flips re-fires this workflow on a hive-authored PR.
 async function replayLastVerdict({ github, context, currentCheckRunId }) {
   const repo = context.repo;
   const event = context.payload.pull_request;
@@ -250,13 +254,10 @@ async function replayLastVerdict({ github, context, currentCheckRunId }) {
     .filter(r => r.name === POLICY_CHECK_NAME && r.status === 'completed' &&
       r.head_sha === pr.head.sha && r.app?.id === ACTIONS_APP_ID && r.id !== currentCheckRunId)
     .sort((a, b) => b.id - a.id)[0];
-  if (!prior) throw new Error(`No completed ${POLICY_CHECK_NAME} run on ${pr.head.sha} to replay`);
+  if (!prior || prior.conclusion !== 'success') return null;
   // A same-named job from any other workflow shares the Actions App id; only
   // a run of this workflow file on this event may be replayed.
   await assertActionsProvenance(github, repo, pr.number, pr.head.sha, prior, POLICY_CHECK_SOURCE, new Map());
-  if (prior.conclusion !== 'success') {
-    throw new Error(`Replaying ${POLICY_CHECK_NAME} run ${prior.id} on ${pr.head.sha}: ${prior.conclusion}`);
-  }
   return `PR #${pr.number}: replayed ${POLICY_CHECK_NAME} run ${prior.id} (success) on ${pr.head.sha}`;
 }
 
diff --git a/.github/scripts/native-queue-policy.test.cjs b/.github/scripts/native-queue-policy.test.cjs
index 3cdaca7..31f74f8 100644
--- a/.github/scripts/native-queue-policy.test.cjs
+++ b/.github/scripts/native-queue-policy.test.cjs
@@ -440,17 +440,24 @@ test('cosmetic flip replays the latest completed success on the same head', asyn
   assert.equal(f.state.listed[0].ref, sha(11));
 });
 
-test('cosmetic flip replays a prior failure and fails closed without a completed run', async () => {
-  await assert.rejects(replayLastVerdict(replayFixture([
+test('cosmetic flip falls back to the full wait when the latest run is not a success', async () => {
+  // A stale timeout/failure must not be replayed: nothing but label flips
+  // re-fires this workflow on a hive-authored PR, so replaying it would keep
+  // a green PR blocked forever. null tells the workflow to run the wait.
+  assert.equal(await replayLastVerdict(replayFixture([
     { id: 5, status: 'completed', conclusion: 'success' },
     { id: 7, status: 'completed', conclusion: 'failure' },
-  ])), /run 7.*failure/);
-  await assert.rejects(replayLastVerdict(replayFixture([
+  ])), null);
+  assert.equal(await replayLastVerdict(replayFixture([
+    { id: 5, status: 'completed', conclusion: 'success' },
+    { id: 7, status: 'completed', conclusion: 'cancelled' },
+  ])), null);
+  assert.equal(await replayLastVerdict(replayFixture([
     { id: 9, status: 'in_progress', conclusion: null },
-  ])), /No completed/);
-  await assert.rejects(replayLastVerdict(replayFixture([])), /No completed/);
+  ])), null);
+  assert.equal(await replayLastVerdict(replayFixture([])), null);
   const own = replayFixture([{ id: 8, status: 'completed', conclusion: 'success' }]);
-  await assert.rejects(replayLastVerdict({ ...own, currentCheckRunId: 8 }), /No completed/);
+  assert.equal(await replayLastVerdict({ ...own, currentCheckRunId: 8 }), null);
 });
 
 test('cosmetic flip still honours a live hold even after a prior success', async () => {
@@ -462,9 +469,9 @@ test('replay ignores runs with a different name, head_sha or app id', async () =
   for (const change of [
     { name: 'other-check' }, { head_sha: sha(99) }, { app: { id: DCO_APP_ID } }, { app: undefined },
   ]) {
-    await assert.rejects(
-      replayLastVerdict(replayFixture([{ id: 7, status: 'completed', conclusion: 'success', ...change }])),
-      /No completed/);
+    assert.equal(
+      await replayLastVerdict(replayFixture([{ id: 7, status: 'completed', conclusion: 'success', ...change }])),
+      null);
   }
 });
 
diff --git a/.github/scripts/test-native-queue-workflows.py b/.github/scripts/test-native-queue-workflows.py
index c639fe4..8424c1e 100644
--- a/.github/scripts/test-native-queue-workflows.py
+++ b/.github/scripts/test-native-queue-workflows.py
@@ -84,6 +84,11 @@ class NativeQueueWorkflows(unittest.TestCase):
         self.assertEqual(gate["id"], "label-flip")
         self.assertIn("isCosmeticLabelFlip", gate["with"]["script"])
         self.assertIn("replayLastVerdict", gate["with"]["script"])
+        # Only a replayed prior success may skip the wait; a missing or failed
+        # prior run must fall through so a green PR is never blocked by a
+        # stale verdict (nothing but label flips re-fires this on hive PRs).
+        self.assertIn("replayed = verdict !== null", gate["with"]["script"])
+        self.assertIn("core.setOutput('cosmetic', String(replayed))", gate["with"]["script"])
         wait = next(s for s in steps if s.get("name", "").startswith("Validate exact-head evidence"))
         self.assertIn("steps.label-flip.outputs.cosmetic != 'true'", wait["if"])
         self.assertLess(steps.index(gate), steps.index(wait))
diff --git a/.github/workflows/native-queue-policy.yml b/.github/workflows/native-queue-policy.yml
index 65afbcb..3b7a097 100644
--- a/.github/workflows/native-queue-policy.yml
+++ b/.github/workflows/native-queue-policy.yml
@@ -45,16 +45,22 @@ jobs:
         with:
           script: |
             const { isCosmeticLabelFlip, replayLastVerdict } = require('./.github/scripts/native-queue-policy.cjs');
-            const cosmetic = isCosmeticLabelFlip(context);
-            core.setOutput('cosmetic', String(cosmetic));
-            if (cosmetic) {
-              core.info('Label "' + context.payload.label.name + '" is not read by the policy; replaying the last completed verdict instead of waiting for evidence.');
+            // Only a replayed prior success short-circuits the wait; with no
+            // completed success on this head (none yet, or a stale timeout or
+            // failure) fall through to the full evidence wait instead of
+            // replaying a verdict that can never change on label flips alone.
+            let replayed = false;
+            if (isCosmeticLabelFlip(context)) {
+              core.info('Label "' + context.payload.label.name + '" is not read by the policy; replaying the last completed success instead of waiting for evidence.');
               try {
-                core.info(await replayLastVerdict({github, context}));
+                const verdict = await replayLastVerdict({github, context});
+                replayed = verdict !== null;
+                core.info(verdict ?? 'No completed success to replay on this head; running the full evidence wait.');
               } catch (error) {
                 core.setFailed(error.message);
               }
             }
+            core.setOutput('cosmetic', String(replayed));
       - name: Validate exact-head evidence (bounded wait for PR providers)
         if: steps.label-flip.outputs.cosmetic != 'true'
         uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
diff --git a/changelog.d/fixed-16952.md b/changelog.d/fixed-16952.md
index 75bcdc6..c1dc4a8 100644
--- a/changelog.d/fixed-16952.md
+++ b/changelog.d/fixed-16952.md
@@ -1 +1 @@
-- `native-queue-policy` no longer spends its ~22-minute evidence wait on label flips it never reads (`size/*`, `dco-signoff`, `hive/claimed-by-*`, `copilot`, stage labels); instead it replays the conclusion of the latest completed `native-queue-policy` run on the same head (failing closed when that run was not `success` or none exists) rather than fabricating a pass, and still applies the live hold/draft/head-move check, while flips of `hold`, `needs-human`, `release-in-progress` and `do-not-merge` still run the full exact-head evidence wait ([#16945](https://github.com/hivecommons/hive/issues/16945)).
+- `native-queue-policy` no longer spends its ~22-minute evidence wait on label flips it never reads (`size/*`, `dco-signoff`, `hive/claimed-by-*`, `copilot`, stage labels); instead it replays the conclusion of the latest completed `native-queue-policy` run on the same head rather than fabricating a pass (when no completed `success` exists on that head it runs the full evidence wait instead of replaying a stale failure), and still applies the live hold/draft/head-move check, while flips of `hold`, `needs-human`, `release-in-progress` and `do-not-merge` still run the full exact-head evidence wait ([#16945](https://github.com/hivecommons/hive/issues/16945)).
diff --git a/src/docs/native-merge-queue.md b/src/docs/native-merge-queue.md
index 0b23264..92efa0e 100644
--- a/src/docs/native-merge-queue.md
+++ b/src/docs/native-merge-queue.md
@@ -220,9 +220,10 @@ metadata, not a fabricated last-member PR event.
   in-progress); the queued run re-reads mutable PR state after the prior run
   finishes, preserving fail-closed semantics. Labels the policy never reads
   (`size/*`, `dco-signoff`, stage labels, ...) skip the wait: the job replays
-  the latest completed `native-queue-policy` conclusion on the same head
-  (failing when it was not `success` or none exists) and still checks the
-  live PR for holds, draft and head moves. `hold`, `needs-human`,
+  latest completed `native-queue-policy` success on the same head and still
+  checks the live PR for holds, draft and head moves; when no completed
+  success exists (none yet, or the latest run failed or timed out) it runs
+  the full wait rather than replaying a stale verdict. `hold`, `needs-human`,
   `release-in-progress` and `do-not-merge*` flips get the full wait.
 - On a group it executes the event's trusted `base_sha` validator. It reads
   the live queue prefix ending at the event's exact synthetic head,
-- 
2.47.3

🐝 Hive Agent: ci-maintainer | Instance: hosted-available-oke-11-placeholder-r05x | SHA: 12f7325

β€” hive: agent=ci-maintainer backend=copilot model=claude-fable-5.1 copilot=1.0.88

…e is no success to replay

A cosmetic label flip used to replay whatever the latest completed
native-queue-policy run concluded, so one failed or timed-out (cancelled)
wait pinned the required check red on every later size/*, dco-signoff or
stage-label flip until a push, edit or hold flip re-entered the full wait.
replayLastVerdict now returns null when there is no prior success on the
head, and the workflow step then runs the authoritative evidence wait
(attempts/15s retry unchanged) instead of failing; a 5xx during the replay
reads takes the same route, matching the transient-API handling the main
path already has (#16258). Provenance mismatches still fail closed.

Refs #16945

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
@hivecommons-hive hivecommons-hive Bot added stage/050-review-pending Review swarm requested; awaiting a verdict for the current head and removed stage/070-fixing Hive-authored PR: CI red, changes requested or needs rebase; a hive agent is fixing it labels Nov 7, 2026

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Decision needed: @clubanderson should decide whether the required native-queue-policy check may replay a prior same-head success on cosmetic label flips instead of re-reading the five policy providers (required-check semantics change in .github/workflows/, code-owned).

Conclusion: the code does what the body says and I found no defect; routing only because it changes what a required check on .github/workflows/ means, which CODEOWNERS (.github/CODEOWNERS:18) reserves for @clubanderson @hanthor @Danathar.

What I verified (read at 9687acd, tests run locally: 30/30 node --test, 7/7 test-native-queue-workflows.py):

  • isCosmeticLabelFlip (native-queue-policy.cjs:230-235) reuses the same BLOCKED_LABEL regex assertEligible reads (:91-95), so no second label list can drift. hold/needs-human/release-in-progress/do-not-merge* flips in either direction take the full wait.
  • replayLastVerdict (:244-262) re-reads the live PR and calls assertEligible before looking for a prior run, so a live hold, draft, or moved head/base still fails closed. It only accepts a completed run with app.id === ACTIONS_APP_ID on the exact pr.head.sha, then binds it to native-queue-policy.yml/pull_request_target/suite/job via the existing assertActionsProvenance (hoisted unchanged from assertEvidence, :116-149). A non-success latest run returns null β†’ the workflow step falls through to the full wait (native-queue-policy.yml:52-58).
  • Workflow permissions are unchanged (all read), checkout still pins the trusted base ref, and the merge_group path (:280-311) is untouched β€” the real merge still re-validates every member's evidence, reviews and build checks, so the replay only shortens the PR-event pre-check.

Two informational notes, not defects:

  1. replayLastVerdict accepts a currentCheckRunId (:244, :253) but the workflow never passes one (native-queue-policy.yml:50); own-run exclusion in practice relies on the running job being status != completed, which holds. Harmless; the parameter is effectively test-only.
  2. A replayed success is the policy's verdict at the earlier run. If a provider check (e.g. gate) is manually re-run on the same head and fails afterwards, a later cosmetic flip replays green where the full path would have read red. Today a check re-run doesn't re-fire this workflow either, and the merge-group path re-reads everything at merge time, so the exposure is bounded β€” but it is the semantic the owners are being asked to accept.

Body matches the diff; the "decision taken by hive" paragraph (fall through on no prior success rather than stay red) is what :258-260 implements. I have not approved or changed anything.

β€” hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot added stage/085-needs-decision A human decision is owed: sentinel alert, hold/needs-decision label, human-only rule or parked issue and removed stage/050-review-pending Review swarm requested; awaiting a verdict for the current head stage/085-needs-decision A human decision is owed: sentinel alert, hold/needs-decision label, human-only rule or parked issue labels Nov 7, 2026
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

/lgtm

Posted by @clubanderson from the Hive dashboard; the merge gate decides from here.

@kubestellar-prow

Copy link
Copy Markdown
Contributor

@hivecommons-hive[bot]: you cannot LGTM your own PR.

Details

In response to this:

/lgtm

Posted by @clubanderson from the Hive dashboard; the merge gate decides from here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

/approve

Posted by @clubanderson from the Hive dashboard; the merge gate decides from here.

@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: hivecommons-hive[bot]
Once this PR has been reviewed and has the lgtm label, please assign kellyaa for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@hivecommons-hive hivecommons-hive Bot added the stage/050-review-pending Review swarm requested; awaiting a verdict for the current head label Nov 7, 2026

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Decision needed: @clubanderson should decide whether the native-queue-policy required check may replay its own prior same-head success on cosmetic label flips (size/, stage/, dco-signoff, copilot) instead of re-running the exact-head evidence wait.

Conclusion: the code is correct and fail-closed as far as I can verify; I'm routing it because it changes the merge-gate required check itself and .github/workflows/** (CODEOWNERS), which is a trust-policy decision, not a code-quality one.

What I checked at 758d393 (base v5):

  • isCosmeticLabelFlip (.github/scripts/native-queue-policy.cjs:230-235) keys off the same BLOCKED_LABEL regex that assertEligible reads (:91-95); I grepped the script and that regex is the only label read anywhere, so a non-matching label really cannot change the verdict.
  • replayLastVerdict (:245-262) re-reads the PR live and calls assertEligible(pr, event.head.sha, event.base.ref) before replaying, so hold/draft/head-move/closed still fail closed. It selects the latest completed run by id on the exact head and returns null unless that one is success β€” a later failure after an earlier success is never bypassed.
  • assertActionsProvenance (:118-149) is a pure extraction of the existing inline block in assertEvidence; the main path's behaviour is unchanged. The replay binds the prior run to native-queue-policy.yml / pull_request_target / same head / suite / job.
  • Workflow (.github/workflows/native-queue-policy.yml): checkout stays on the trusted base ref, permissions unchanged (all read). A non-transient error in the replay step setFaileds before the wait step is skipped, so it fails rather than skips. 5xx falls through to the full wait.
  • node --test .github/scripts/native-queue-policy.test.cjs β†’ 30/30; python3 .github/scripts/test-native-queue-workflows.py β†’ 7/7.

Info (not a defect): the workflow calls replayLastVerdict({github, context}) without currentCheckRunId, so the own-run exclusion at :256 is a no-op; exclusion of the running check relies on the status === 'completed' filter, which is sufficient today.

Question for the owner: a required check that replays its own prior verdict is a new precedent for this gate. If that's acceptable, this looks correct to me. I have not approved or merged anything.

β€” hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive
hivecommons-hive Bot merged commit 7c291af into v5 Nov 7, 2026
84 of 85 checks passed
@kubestellar-prow
kubestellar-prow Bot deleted the scanner/fix-16945 branch November 7, 2026 14:19
@github-actions

github-actions Bot commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

Thank you for your contribution! Your PR has been merged.

We'd love to hear how your experience was: share feedback

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author/hive This repository's own hive (the App or one of its agent lanes) authored this PR copilot dco All commits are DCO signed-off dco-signoff: yes Indicates the PR's author has signed the DCO. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[reviewer] Review backlog (correctness): Optional early-exit for cosmetic label flips not implemented (per #16942 it is optional)

0 participants