Repository navigation
Description
CI Issue
The stage reconciler is flapping PR #16607 (scanner/fix-16265, head 687f4cb) between stage/070-fixing and stage/085-needs-decision every 1–10 minutes, and has been since 2026-11-05 ~15:00Z — about 19 hours at the time of filing, with no push, review, comment or other external event in between. Each flip is a labeled + unlabeled pair, and each of those fires three workflows (changelog-fragment-guard, DCO squash attribution, Native queue policy), so one idle PR is generating ~120 workflow runs per hour on an unchanged head.
The visible CI damage is on Native queue policy: it is pull_request_target with cancel-in-progress: true keyed on the PR number and waits up to 22.5 min for evidence, so almost every run is cancelled by the next flip before it can report. The native-queue-policy check on #16607 shows pending and almost never settles.
Evidence
GET repos/hivecommons/hive/issues/16607/timeline(2026-11-06 ~10:00Z, truncated at 300 events by the API): 133labeled+ 126unlabeledevents, all byhivecommons-hive[bot]; 70×stage/070-fixingon/off, 64×stage/085-needs-decisionon, 63× off, 13×stage/050-review-pendingon/off. Per-hour label-event counts climb from 12 (15Z) to 44 (20Z, 21Z) and are still 30–40/hour at 09Z today.- Non-label events after the head was pushed (2026-11-05T15:00:48Z): one
reviewed(bot adjudicator, 15:16:46Z, "HUMAN DECISION NEEDED — confirm the hive-merge relay may App-merge trusted-author pin-only PRs that touch auto_merge.human_merge_paths") and one bot comment (18:44:54Z). Nothing else. The flips are not reacting to new facts. gh run list --branch scanner/fix-16265 --limit 1000: 1000 runs fit in 8.2 hours (01:44Z → 09:57Z); breakdownNative queue policy332 cancelled / 1 running,changelog-fragment-guard331 success,DCO squash attribution167 success / 167 cancelled. All on head687f4cb.gh run list --workflow native-queue-policy.yml --limit 600(00:12Z → 09:57Z): 423 runs on this branch — 402 cancelled, 20 success. The next-busiest branch in the same window has 17 runs. The flap is a single-PR outlier, not a fleet pattern.gh run list --limit 300: the 300 most recent runs in the repository span only 90 minutes, and 144 of them are this one branch.- Where the two opinions come from (v5 @ 8b5a160):
src/pkg/stageengine/derive_pr_classify.go—RequiresHumanStagereturns 085 for arequires_humanverdict with no trusted-human approval on the head (line ~577);PreserveNormalizedFixingStage(~585) converts that same verdict back to070-fixing/LaneReasonChangesRequestedwhen the previous record was already 070 on the sameActivityHead;PreserveStageOnStaleDecisionAnswer(~614) does the reverse for 085. The observed sequence — 070 → 085 → 070 with the same head and the same verdict — looks like the two preserve-paths taking turns depending on which record the reconciler saw last, rather than a stable fixed point.
Recommendation
Fix in the stage engine, not in the workflows — the three workflows run on labeled/unlabeled on purpose (changelog-fragment-guard.yml header, #14755/#14629: skipping or cancelling label runs left a required check scored by a stale sibling) and must keep doing so.
- Make the 070↔085 derivation for a
requires_humanverdict idempotent on the sameHeadSHA: whichever ofRequiresHumanStage/PreserveNormalizedFixingStagewins the first time for a given (head, verdict) must keep winning until the head, the verdict, a trusted-human review or a decision reply changes. A regression test that feeds the reconciler the same observation twice (prev = result of the first pass) and asserts no transition is the minimal fail-without-fix test. - Add a generic anti-flap guard in the label writer (
src/pkg/github/stage_writer.go/src/pkg/stageengine/writer.go): refuse to apply a stage transition that exactly reverses the previous transition on the same head within a short window (e.g. 15 min) and log it asstage flap suppressedinstead. This bounds the blast radius of any future derivation bug: a flap would cost one label pair per window instead of one per sweep. - Operationally, for ✨ feat(github): enforce pin-only exception in merge-request relay #16607 itself: the adjudicator's question is a merge/trust-policy decision for @hanthor ([scanner] human-merge pin-only exception: enforce in merge-request relay #16265 / feat: bounded trusted pin-only exception to per-repository human-merge paths #16242). Answering it stops the churn today; it does not fix the engine.
Decision taken by hive: filed as one issue for the single root cause (the reconciler oscillation); the per-workflow symptoms are not filed separately because the workflows' label-event behaviour is intentional and documented in-tree.
Done when: #16607 (or a reproduction PR) holds a single stage label across ≥3 consecutive reconciler sweeps with no external event, and gh run list --branch <head> shows no new Native queue policy runs between pushes.
Refs #16607 (affected PR; left open — it is the scanner's feature PR, not this bug).
Filed by ci-maintainer agent (ACMM L6 — full mode)
🐝 Hive Agent: ci-maintainer | Instance: hosted-available-oke-11-placeholder-r05x | SHA: unknown
— hive: agent=ci-maintainer backend=copilot model=claude-fable-5.1 copilot=1.0.88
Activity
hivecommons-hive commentedon Nov 6, 2026
🔓 Claim by @mendezr released (lease revoked).
Claim history
mendezr(contributor on hive hosted-available-oke-11-placeholder-r05x)mendezr(contributor; lease revoked)🐛 fix: stabilize same-head stage verdicts and suppress reversals (#16881