Repository navigation
π fix(contribute): treat an instantly failed relay fix-pr task as undelivered and retry - #14690
Conversation
β¦ivered and retry A relay that failed a stage-072 fix-pr task 116 ms after dispatch was still reported as reached (mode=spawn, "your relay session received this as a fix-pr task"), and handleTaskFailed settled the dispatch exactly like task_complete, so the PR silently left the fix queue and was never retried. Separately, a conflicting contributor PR kept its hive:unauthorized mark because the lane returned "conflicting" before the step that clears it, so the row named "approve runs" as the blocker when only a rebase can move the PR. - Nudge waits briefly for the relay to refuse the dispatch; a refusal is reported as queued with the relay's reason, and the comment says so. - task_failed on a fix requeues it with backoff (served at the relay's next task boundary); after three failures the PR leaves the queue for the stage engine's re-nudge / adoption path. - The contributor lane clears the approve-workflow-runs unauthorized mark when the head conflicts; the next head re-marks if its runs are held. Closes #14668 Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com> Co-authored-by: clubanderson <407614+clubanderson@users.noreply.github.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Open PR overlap checkBase branch: Conflicts are checked with every PR merged onto the current tip of No conflicting open PRs or same-file overlaps were found. |
contributorLaneConflictReason returns "conflicting" before the later dirty/conflicting branch, so the clearUnauthorized added there never ran and the label stayed on a conflicting head. Clear it at the early return instead. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
|
hive/awaiting-ci: CI pending β sweep will merge when green. π Hive Agent: β hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88 |
There was a problem hiding this comment.
Read the diff against the v5 tree and the callers; this looks correct to me.
What I verified:
- Refusal wait has no lost-wakeup window.
beginFixWait(contribute_fixqueue.go:272) takes the signal channel beforedispatchFixes,pulseFixFailedrecords the reason only while a waiter exists and closes/replaces the channel, andawaitFixTaskRefusalre-checkstakeFixFailurebefore each select β so atask_failedracing the dispatch is caught either way. No lock is held across the 1 s wait, andfailFixTasktakesc.muthenfixQueueMuin disjoint sections, so no deadlock against the WS session goroutine. - Requeue keeps backoff across re-nudges.
Scheduler.Enqueuenow carriesAttempts/NotBeforeover an existing key (fixqueue.go:232,239), so a stage-engine re-nudge of the same PR does not reset the counter;DispatchhonoursNotBeforeandDispatchAtBoundarydeliberately does not (fixqueue.go:363), which matches the body's "the relay's nextreadystill picks it up". - Conflict clears the mark at the right place.
contributorLaneConflictReasonreturns before the later dirty branch, so the newclearUnauthorizedatautomerge_contributor_lane.go:1180is the only point that runs on a conflicting head;clearUnauthorizedandunauthorizedStepApproveWorkflowRunsexist at base (automerge_unauthorized.go:51,150). - Body matches the diff; each of the four named tests is present.
One design note, not a defect: Dispatch checks NotBefore on queue[0] only (fixqueue.go:392), so while a failed head backs off, other queued fixes for that contributor are also withheld from the interrupt path for the same 5 min. That reads as intended β a relay that just refused one fix-pr task is likely to refuse the next β but if per-item backoff was the intent, the guard would need to skip past backing-off items instead of returning nil.
β hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88
|
Thank you for your contribution! Your PR has been merged. We'd love to hear how your experience was: share feedback |
Closes #14668
Why
The issue's rationguard#397 trace shows three separate problems:
fix-pr task failedarrived 116 ms afterfix-pr task dispatched.Nudgestill returnedmode=spawnbecause it reported the mode at send time. That's why the stage log saidcontributor nudged β¦ mode=spawnand the comment told the contributor their relay had received the task. NowNudgewaits up to 1s for the relay to refuse the dispatch. If the relay does refuse it, the outcome isqueuedwithRelayFailed, and the comment says the relay could not take the task and quotes the relay's reason. The stage log also getsrelay_failed=true.handleTaskFailedcalledcompleteFixTask, so a failure was settled like a success. The PR was dequeued, its owner was forgotten, and nothing re-enqueued it. That matches the 40 silent minutes in the issue. Now a failure requeues the PR. It keeps its place by age and gets 5 minutes of backoff, so a relay that may be mid-turn isn't interrupted again straight away. The relay's nextreadystill picks it up, because the existing boundary dispatch path ignores the backoff. After 3 failed dispatches the PR leaves the fix queue and falls back to the stage engine's existing re-nudge / adoption path. The failure reason is logged.hive:unauthorizedmark stayed after a conflict.evaluateContributorLanePRreturnsconflictingbefore reaching the step that callsclearUnauthorized, so the approve-workflow-runs mark could never clear on a dirty head. As the issue says, approving those held runs is wasted work because the rebase push creates new ones. The lane now clears the mark when the head conflicts, and the next head is re-marked if its runs are held. The row's blocker is then the conflict, which only the contributor can fix.Nothing in this issue was already fixed on v5.
Tests (each fails without its change):
fixqueue.TestRequeueFailedDispatchBacksOffdashboard.TestFixQueueInstantRelayFailureIsNotDeliveredautomerge.TestContributorNudgeCommentRelayFailedautomerge.TestContributorLaneConflictClearsUnauthorizedβ hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88 requested_by=@clubanderson