Repository navigation
π fix(ci): attribute check runs by PR, not by SHA alone - #16830
Conversation
projectbluefin/utah-packages#890 (in-repo, danathar/supersede-878) and the fork PR hivecommons#878 shared head 95b9d22. hivecommons#878's fork Canary run (run 42111306103) had failed; 41 seconds after hivecommons#890 opened, before its own Canary started, the contributor lane sent hivecommons#890 to 072 for "Failing checks: Canary". EvaluateCommitCI, when given a PR number, drops a red check run another PR produced on the head: one whose pull_requests names other PRs but not this one, or one naming none (fork runs) whose check suite ran for a different head branch. Lookups are per suite, only for red unattributed runs, and a failed lookup keeps the run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BQYnPGFJx81c8JXR3o1sg1 Signed-off-by: Danathar <Danathar@users.noreply.github.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Open PR overlap checkBase branch: Conflicts are checked with every PR merged onto the current tip of
|
There was a problem hiding this comment.
Correctness review (head ccf0052) β the attribution logic and its callers (automerge_sweep_queued.go:405, merge_ci_gate.go:207) check out; failure modes fail closed as the body says. One advisory, no blocker.
Same branch name on both heads defeats the suite-branch fallback β src/pkg/github/commit_ci.go:1161-1163. For a fork run with empty pull_requests, the only discriminator is suite.GetHeadBranch() vs the PR's head.ref. An in-repo copy made with gh pr checkout 878 && git push origin HEAD keeps the fork's branch name, so both check suites report the same head_branch, branch != ref is false, and the fork's red run still counts against the in-repo PR β the exact #890 shape, just with the default branch naming. It over-blocks rather than under-blocks, so it's safe, but the PR body/stage-labels.md:385 present the rule as general. Suggest either noting the limitation in the doc paragraph, or (stronger) disambiguating by repository when branches match: the check suite object carries no head repository, but GET /repos/{o}/{r}/actions/runs?check_suite_id={id} returns head_repository.full_name, which can be compared with pr.GetHead().GetRepo().GetFullName() (one extra read, only on the branch-equal path).
Minor, same function (commit_ci.go:1161): prHeadBranch() is evaluated before branch != "", so a failed suite lookup still spends the pulls/{n} read it can't use.
if branch != "" {
if ref := prHeadBranch(); ref != "" && branch != ref {
continue
}
}
β hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88
Reviewer advisory on hivecommons#16830: an in-repo copy pushed under the fork's branch name (gh pr checkout 878 && git push origin HEAD) shares the branch, so the suite head_branch alone still charged the fork's red run to it. The check-suite API names no head repository; the Actions workflow run behind the suite does (actions/runs?check_suite_id=). A red unattributed run now counts only when that run's head repository and branch match the PR's head; a suite with no workflow run (another app) falls back to its head_branch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BQYnPGFJx81c8JXR3o1sg1 Signed-off-by: Danathar <Danathar@users.noreply.github.com>
|
Thanks, you were right about the shared branch name. Fixed in caefb93.
|
There was a problem hiding this comment.
Read against v6 with the callers. Looks correct to me.
- Plumbing is real, not aspirational: both production callers already set
PRNumber(src/pkg/github/merge_ci_gate.go:207,src/pkg/github/automerge/automerge_sweep_queued.go:405), so the new filter atcommit_ci.go:550-552is live for the contributor lane that sent utah-packages#890 to 072;issue_release_approve.go:184passes no number and is untouched, as the body says. ListWorkflowRunsOptions.CheckSuiteIDandChecks.GetCheckSuiteexist in the pinned go-github v72, and every lookup failure keeps the run.- Only
completed+checkConclusionIsRedruns are judged, so pending/green runs and the "latest per identity" grouping downstream are unaffected except that a dropped red run can leave a required identity empty βrequired-check-pending, which is the right side to err on.
One open question, not a finding: checkRunNamesPR drops a red run whose pull_requests lists other PRs but not this one. That is correct when GitHub computes pull_requests at read time from open same-repo PRs at the head (my understanding), but if it were snapshotted at run creation, a same-repo PR opened after a push-triggered run would lose a red run that genuinely tested its SHA β an under-block, not the over-block the comment at commit_ci.go:1130 promises. If someone has seen a stale pull_requests list in the wild, that row is the one to re-check; otherwise fine.
β hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88
There was a problem hiding this comment.
Correctness review (head d36d570)
Attribution is applied to red runs only, so another PR's green run still masks this PR's own red run β src/pkg/github/commit_ci.go:1165 (if cr == nil || cr.GetStatus() != "completed" || !checkConclusionIsRed(...) { out = append(out, cr); continue }).
The mirror of the #890/#878 incident: evaluate fork PR #878 whose own Canary (suite A, pull_requests empty) failed, while in-repo #890's Canary (newer ID, pull_requests: [890]) passed on the same SHA. checkRunsForPR keeps #878's red run and also keeps #890's green run untouched; latestCheckRunsByNameAndApp then picks the green one because checkRunIsNewer (commit_ci.go:1296) orders by ID alone. #878 reads green on a Canary it never passed β and for Canary specifically (fork runs lack secrets, in-repo runs have them) that is the common shape, not an exotic one. A false green feeds the automerge lanes, which is worse than the false red this PR fixes. The PR table's "green or still running: as is" row documents the choice, but the title ("attribute check runs by PR, not by SHA alone") is not met for the half that can wrongly merge.
Suggested fix β judge every completed run, not only red ones. Runs with a non-empty pull_requests cost no lookup; unattributed ones are already cached per suite, so the cost is one actions/runs call per suite, not per run:
for _, cr := range checks {
- if cr == nil || cr.GetStatus() != "completed" || !checkConclusionIsRed(cr.GetConclusion()) {
+ if cr == nil || cr.GetStatus() != "completed" {
out = append(out, cr)
continue
}TestCheckRunAttributionLookupsAreBounded's second half ("green runs cost no lookup") would need to flip to assert one suite read, and a new row should cover: pr 878, runs {own Canary failure suite 878 (no PRs), Canary success id 9 suite 890 (pr 890)} β wantRed: true. If you'd rather keep lookups red-only, the minimum is to also filter green runs that carry a pull_requests list naming only other PRs (free), since that is exactly the in-repo-copy case above.
Everything else checked out: checkRunNamesPR precedence (a || b || c && d) is as intended; failed lookups keep the run; prHeadOf/suiteHeadOf are read once; commit-only path (PRNumber == 0) is untouched at commit_ci.go:550.
β hive: agent=adjudicator backend=copilot model=claude-fable-5.1 copilot=1.0.88
|
Hive status β updated in place; edits do not notify. Full history is in the Activity Monitor. Review findings from Read The filter is one-directional, and the open direction is the unsafe one (
Suggested shape: apply the Everything else I checked ( Review findings from docs-currency (read at head d36d570) The new behaviour is documented where operators read it (
Review findings from Correctness review (head d36d570) Attribution is applied to red runs only, so another PR's green run still masks this PR's own red run β The mirror of the #890/#878 incident: evaluate fork PR #878 whose own Canary (suite A, Suggested fix β judge every completed run, not only red ones. Runs with a non-empty for _, cr := range checks {
- if cr == nil || cr.GetStatus() != "completed" || !checkConclusionIsRed(cr.GetConclusion()) {
+ if cr == nil || cr.GetStatus() != "completed" {
out = append(out, cr)
continue
}
Everything else checked out: Review findings from Read the filter against the tree at merge-base, not just the diff. Looks correct to me. What I verified:
One non-blocking note: |
There was a problem hiding this comment.
Read against the v6 tree, not just the diff. Looks correct to me.
What I verified:
checkRunsForPR(src/pkg/github/commit_ci.go:550) runs right aftercheckRunsOnHeadand only whenopts.PRNumber > 0. Both lane callers pass it:automerge_sweep_queued.go:405andmerge_ci_gate.go:207;issue_release_approve.go:184(commit-only) does not, so release-target reads are unchanged as the body says.- The 072 nudge's "Failing checks" list is derived from the lane reason (
automerge_contributor_nudge.go:96-99,check-failing:/required-check-failing:), which comes out ofEvaluateCommitCIβ so filtering at this point is upstream of everything that returned utah-packages#890. foreignHeadtreats unknown as not-foreign, and the PR GET / suite lookups are cached (prHeadRead,suiteHeads), so a failed lookup can only over-block, never drop a run.go build ./pkg/github/passes locally; thetest (*)shards are green on this head.
One note, not a defect: a red run from another app whose suite has no workflow run is judged on head_branch alone, so two in-repo PRs sharing both SHA and branch name would still share that app's red run. That is the documented fallback and is the conservative direction.
β hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88
Executive summary
projectbluefin/utah-packages#890 was sent back to its author as "Failing checks: Canary" for a Canary run that belonged to a different PR. #890 is an in-repo branch, and the fork PR #878 sat on the same commit, 95b9d22. #878's fork Canary run (run 42111306103) had failed. At 21:36:42Z, 41 seconds after #890 opened and before its own Canary had started, the contributor lane moved #890 to
stage/072-returned-to-contributorand nudged its author. #890's own Canary later passed.GitHub lists check runs per commit, so the hive took the newest Canary run on the SHA, whichever PR it came from. This PR attributes check runs to the PR being evaluated.
What
pull_requestsnames this PRpull_requestsnames other PRs onlypull_requestsempty, workflow run on this PR's head repo and branchpull_requestsempty, workflow run from another head repo or branch (#878's fork run; also an in-repo copy under the fork's branch name)Why
EvaluateCommitCIlists check runs for the SHA, keeps those on the head, then takes the latest per name and app. Two PRs at one head share that list. GitHub leavespull_requestsempty on a fork PR's runs, and the check-suite API names no head repository. The Actions workflow run behind the suite (GET actions/runs?check_suite_id=) does, so it is compared with the PR's head repository and branch. A suite from another app, with no workflow run, falls back to itshead_branch.How
checkRunsForPRinsrc/pkg/github/commit_ci.goruns right aftercheckRunsOnHead, only when the caller passesPRNumber. That covers the automerge lanes (commitCIForPRSkippingWith) and the merge-request CI gate.GET check-suites/{id}for another app) is read once per suite. The PR's head ref (GET pulls/{n}) is read once, and only when an unattributed red run exists.Tests
TestCheckRunsAttributedByPR(9 rows: + same-branch-name in-repo copy, other-app fallback)TestCheckRunAttributionLookupsAreBoundedTestCheckRunAttributionCachesSuitego test ./pkg/github/... ./pkg/writeback/ ./pkg/dashboard/check-docs-citations.py src/docsNotes
π€ Generated with Claude Code
https://claude.ai/code/session_01BQYnPGFJx81c8JXR3o1sg1