Skip to content

🐛 fix(ci): surface and recover shared CI permission blocks - #16277

Merged
clubanderson merged 2 commits into
v5from
fix/shared-ci-recovery-permission-20261102
Nov 2, 2026
Merged

clubanderson merged 2 commits into
v5from
fix/shared-ci-recovery-permission-20261102

Conversation

@clubanderson

Copy link
Copy Markdown
Member

Summary

  • Classify observed shared-CI rerun permission refusal as an exact-head operator handoff through the existing unauthorized label/comment and stage mechanism, rather than source repair work.
  • Preserve the repository cooldown and head-scoped notification dedupe, while retrying unchanged PRs after permission recovery and reconciling existing blocks beyond incident discovery lookback. Do not mark a partially accepted recovery complete.
  • Align bot-PR squash attribution with the existing required checker, including stale same-head cache entries; unsigned commits and human attribution remain enforced.
  • Document optional installation-scoped Actions write without changing baseline App grants or selecting alternate credentials.

Validation

  • Independent read-only code review completed; corrected its scope and aged-incident findings.
  • gofmt and git diff --check pass. Local Go/tests/build/vet/docs guards intentionally not run per AGENTS; CI is the verdict.
  • Added cooldown, unchanged-head recovery, manual recovery after discovery expiry, partial permission refusal, stale-head isolation, bot/human/unsigned attribution and pure stage regression coverage.

Operational evidence (separate from this code)

Related #16188 and #16048. One owner-authorized expected-head base refresh as clubanderson produced 9b93938. Required gosec job125454843482/run41497095558 succeeded at2026-11-02T18:07:00Z in132seconds. No App permission grant, deployment, merge approval or check bypass was performed. The obsolete optional PR Type and normal OWNERS/Prow approvals are separate gates.

Signed-off-by: Andrew Anderson <andy@clubanderson.com>
@kubestellar-prow kubestellar-prow Bot added the dco-signoff: yes Indicates the PR's author has signed the DCO. label Nov 2, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign danathar for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubestellar-prow kubestellar-prow Bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Nov 2, 2026
Signed-off-by: Andrew Anderson <andy@clubanderson.com>
@github-actions

github-actions Bot commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

Open PR overlap check

Base branch: v5

Conflicts are checked with every PR merged onto the current tip of v5, so a PR that only needs a rebase is not reported as conflicting with its siblings.

No conflicting open PRs or same-file overlaps were found.

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Decision needed: @hanthor should decide whether the DCO squash gate may stop counting hive-authored commits without the PR author's sign-off as squash-unsafe on bot-authored PRs (automerge_stage_dco.go:67,101-104), mirroring the shell check's existing skip.

Read the diff against v5 plus src/scripts/check-squash-signoff-attribution.sh, automerge_stage_dco.go, shared_ci_rerun.go, and the passDecision path in stageengine/derive_pr.go. The code does what the body says; I'm routing it because it relaxes a merge gate and the author is one of the approvers, so a second approver should sign.

What I verified

  • Bot-PR squash attribution. botSquashAuthor (automerge_stage_dco.go:125) suppresses squashUnsafeHive when the PR author login ends in [bot], both on the fresh read (:67) and the cached same-head path (:99-103). The shell gate already does this: check-squash-signoff-attribution.sh:104-105 exits early with "PR author … is a bot; squash will stay bot-authored". The Go predicate is narrower than the script's is_bot_identity (:93-102, which also matches github-actions, copilot noreply addresses, etc.), so this does not exempt anything the script would still check. unsigned/unsignedOther are untouched, so unsigned commits still fail. The alignment claim holds — this is a policy relaxation of the Go gate, not a bug fix, which is why I'm asking an approver to confirm it.

  • Permission block → NeedsDecision. trySharedCIRerun now records SharedCIRerunBlock{HeadSHA, RunID} on 403/405 (shared_ci_rerun.go:338-349) and on the repo cooldown short-circuit (:331-334); the markSeen memo is skipped when blocked (:358-361), so the unchanged-PR dedupe at :267 no longer swallows the retry. hasSharedCIRerunBlocks (:95-103) keeps the repo in the sweep when discovery finds no closed incidents, and :298-310 re-seeds the previously seen incident set for the blocked head only. In ApplyStageLabels (automerge_stage_labels.go:1141-1148) and passDecision (derive_pr.go:305-310) the block parks the PR in NeedsDecision with the unauthorized:rerun-shared-ci reason, and sharedCIRecoveryUnauthorized clears the label once the block is gone. Consistent with the doc added in github-app-setup.md.

Open question (not a finding): at shared_ci_rerun.go:363-366, when some runs were re-run and a later one was refused, the function returns handled=true without stamping the comment or markSeen. The next sweep will call RerunFailedJobsByID again on the already-accepted runs. I could not verify from the tree what GitHub returns for a rerun already in progress; if it is a non-403 error the loop just logs and continues (:352), so this is at worst noise, but worth a glance.

— hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor.

@clubanderson this PR is held at stage/071-needs-author and the hive will not merge it yet.

Why:

  • the review swarm returned requires_human: it has a question for you, not a fix for the hive
  • the reviewer asked you: @hanthor should decide whether the DCO squash gate may stop counting hive-authored commits without the PR author's sign-off as squash-unsafe on bot-authored PRs (automerge_stage_dco.go:67,101-104), mirroring the shell check's existing skip.

Next: the hive does not push to your branch. Push a new head (or reply here); CI and the review swarm re-run automatically and the PR returns to the normal path.

@clubanderson

Copy link
Copy Markdown
Member Author

/hive decision

Owner-directed disposition for head 6bac75c: accept the narrow bot-authored squash-attribution alignment and complete this PR under my instruction to finish these without another person-review wait. This is a PR-specific owner decision, not a second reviewer’s approval or a repository-wide rule change.

The Go gate only suppresses the extra PR-author sign-off attribution counter when the authenticated PR-author login ends in [bot]. That is narrower than the already-required shell checker’s existing bot-author exemption at src/scripts/check-squash-signoff-attribution.sh:93-105. Unsigned commits remain rejected, human PRs retain attribution enforcement, and both fresh and cached paths have regression coverage. No App permissions or alternate credentials are granted by this change.

The partial-rerun advisory is not a merge blocker: latestFailedWorkflowRuns selects the newest attempt per workflow and excludes queued/in-progress attempts before any retry. Partial permission refusal preserves the outstanding block and does not stamp the incident complete. The existing bounded permission cooldown is retained; there is no new retry or permission bypass.

All 12 protected entries, native-queue-policy, all Go shards, coverage, patch coverage, lint and the build/runtime/contract checks currently pass at this exact head. Preserve those gates and use an exact-head guard for completion. Source merge does not claim deployment or an App grant.

@clubanderson
clubanderson merged commit 263c201 into v5 Nov 2, 2026
78 of 79 checks passed
@kubestellar-prow
kubestellar-prow Bot deleted the fix/shared-ci-recovery-permission-20261102 branch November 2, 2026 20:24
@github-actions

github-actions Bot commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

Thank you for your contribution! Your PR has been merged.

We'd love to hear how your experience was: share feedback

hivecommons-hive Bot pushed a commit that referenced this pull request Nov 18, 2026
…blocker (#19024)

The rerun 403 (`unauthorized:rerun-shared-ci`, #16277) replaced the stage,
reason and route of every PR whose exact head it was refused for, whatever
held the PR: on projectbluefin it showed "re-run CI" over a reviewer's
changes request (utah#1313), a withheld security approval
(utah-packages#1013) and high-risk decision requests. It was meant to keep an
infrastructure failure out of source fixing, not to outrank review gates.

RecoveryUnauthorizedOwnsStage lets it take the stage only when CI is what
holds the PR (a red/pending check, a shared-CI or base-branch-red deferral,
a red optional check) and no review gate stands; it is used by both the
stage-label pass and DerivePR. The hive:unauthorized label and comment are
still written either way.

Signed-off-by: Danathar <6772335+Danathar@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BQYnPGFJx81c8JXR3o1sg1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dco-signoff: yes Indicates the PR's author has signed the DCO. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant