Skip to content

πŸ› [v6] relay: a fix for your own PR is taken from any hive you subscribe to, and a polling-slot refusal no longer counts as a failed attempt - #17405

Merged
hivecommons-hive[bot] merged 1 commit into
hivecommons:v6from
Danathar:fix/v6-fix-pr-any-hub
Nov 9, 2026

Conversation

@Danathar

@Danathar Danathar commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

Summary

A relay subscribed to several hives refused a fix-pr task for its contributor's own PR because it happened to be polling another hive, and the hub counted that refusal as a failed fix attempt. On projectbluefin/utah-packages#972 the relay had already accepted the fix and deferred it behind a headless task; when that task ended it refused the fix with Hub is not the active polling slot. With this PR the relay takes a fix-pr from any subscribed hive (making that hive active and telling the others it is busy elsewhere), and the hub puts a fix refused for the polling slot back in its queue without booking an attempt. Ordinary work from a hive the relay is not polling is still refused.

What went wrong

From the projectbluefin hive's log (build 467557e; nothing in these files changed between it and the v6 tip apart from #17402), checked against the PR on GitHub:

  • 14:41:50Z Karibhou (relay, backend omp) opened fix: require human review for fuse and thin-provisioning bumpsΒ projectbluefin/utah-packages#972 from the fork Karibhou/utah-packages.
  • 14:42:58Z the hub dispatched fix-karibhou-1 to the relay, mode=preempt priority=interrupt. The relay was running a task, so the fix was handled as an interrupt. With a headless primary the relay defers it and sends task_accepted.
  • 14:45:39Z the relay's local pr-review-1794235343201 task ended (the hub logged its task_complete as an unassigned task). In the same second the hub logged fix-pr task failed; requeued ... attempts=1 retry_after=14:50:39Z reason "Hub is not the active polling slot".

That is startDeferredFix re-entering the task_assign handler for the hub that owns the fix. With no task running, the handler refuses any assignment from a hub that is not hubs[activeHubIndex], and the relay's polling slot was on another hive. So a fix the relay had already accepted was thrown away at the moment it should have started. The same gate refuses a fix-pr interrupt pushed to an idle relay that is polling another hive.

On the hub, failFixTask treats every task_failed for a fix as a failed attempt. Three of them remove the PR from the fix queue and hand it to the stage engine's re-nudge and adoption path, although the relay never tried the fix.

What changes

Out of scope: Canary itself and merge policy.

Files

  • bin/contributor-relay.js: the fix-pr exemption in the polling-slot gate
  • src/pkg/fixqueue/fixqueue.go: Return (a requeue that keeps Attempts)
  • src/pkg/dashboard/contribute_fixqueue.go: failFixTask uses Return for a polling-slot refusal
  • src/docs/contributor-relay.md (multi-hive section), changelog.d/fixed-fix-pr-any-hub.md
  • Tests: bin/contributor-relay.test.js, src/pkg/fixqueue/fixqueue_test.go, src/pkg/dashboard/contribute_fixqueue_test.go

Tests

No tests were run locally (repo policy); CI is the check. Why each fails on the v6 tip:

  • Relay, #972 a fix-pr pushed by a non-active hub to an idle relay is taken, and the other hive hears busy_elsewhere: two hubs, the slot on hive A, a fix-pr from hive B. On the tip hive B gets task_failed: Hub is not the active polling slot and the relay stays idle. Now the fix becomes the current task owned by hive B, hive A gets one busy_elsewhere busy frame, and finishing the fix clears it.
  • Relay, #972 a fix-pr deferred behind a headless primary runs when the primary ends, even with the polling slot on another hive: the live sequence. The fix is accepted and deferred behind a local pr-review task, then requestWork(hive A) starts it. On the tip that start is refused with the polling-slot reason.
  • Relay, #972 the polling-slot refusal still applies to ordinary work from a non-active hub: an issue task from hive B is still refused (passes on the tip too; it guards the narrow scope).
  • TestReturnKeepsAttempts (fixqueue): Return requeues with the backoff and Attempts 0, and a later real failure counts 1. Return does not exist on the tip.
  • TestFixQueuePollingSlotRefusalIsNotAnAttempt (dashboard): four polling-slot refusals in a row leave the PR queued with 0 attempts, and a real failure afterwards counts 1. On the tip the first refusal books Attempts=1, and the third removes the PR from the queue.

This PR and the open #17403 touch nearby lines in contributor-relay.js and contribute_fixqueue.go; a trial merge of the two heads is clean.

Refs projectbluefin/utah-packages#972

πŸ€– Generated with Claude Code

https://claude.ai/code/session_01BQYnPGFJx81c8JXR3o1sg1

…fusal is not a failed attempt

On projectbluefin/utah-packages#972 (2026-11-09) the projectbluefin hive
sent Karibhou's relay a fix-pr preempt at 14:42:58Z. The relay accepted
it and deferred it behind its headless primary, then refused it with
"Hub is not the active polling slot" at 14:45:39Z, the moment the primary
ended, because its polling slot was on another hive. The hub booked
attempts=1 and a five-minute retry.

- Relay: the task_assign polling-slot gate lets a fix-pr through from any
  subscribed hub (as it already does for an acknowledged standby push).
  The ordinary path then makes that hub active and the busy_elsewhere_v1
  report tells the other hives.
- Hub: a fix-pr task_failed with that reason is requeued with the same
  backoff but no attempt counted (new fixqueue.Scheduler.Return), so
  refusals from older relays never hand the PR to adoption.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BQYnPGFJx81c8JXR3o1sg1
Signed-off-by: Danathar <Danathar@users.noreply.github.com>
@kubestellar-prow kubestellar-prow Bot added the dco-signoff: yes Indicates the PR's author has signed the DCO. label Nov 9, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign danathar for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubestellar-prow kubestellar-prow Bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Nov 9, 2026
@hivecommons-hive
hivecommons-hive Bot merged commit 7423d91 into hivecommons:v6 Nov 9, 2026
72 of 77 checks passed
@hivecommons-hive

hivecommons-hive Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Hive status β€” updated in place; edits do not notify. Full history is in the Activity Monitor.

Evidence bundle `#17405a1a126a` sha256:550adc76474fd86b9b584ef0b5dd173889fbeb1816dbb22ed12f3239e5bb87ca β€” download from the dashboard or `hivectl review evidence #17405`.

Review findings from reviewer-2 (folded here because this head already carries its hive review):

Read the diff against the v6 tree and the callers it lands in. Looks correct to me.

Relay (bin/contributor-relay.js): the new fixPush exemption at the polling-slot gate (~L10004) only skips that one break; the fix-pr then takes the ordinary assignment path, which already does activeHubIndex = hubs.indexOf(hub) (L10109) and sets currentTask._hub = hub (L10106), so busyElsewhereFor() (L1972) reports busy to every other hub that advertises busy_elsewhere_v1. No new code path; the mechanism the body describes is the existing one. Ordinary tasks from a non-active hub are still refused β€” the gate is otherwise unchanged. The currentTask && FIX_PR_KIND interrupt branch below it is unaffected.

Hub (src/pkg/fixqueue/fixqueue.go:362-400): Requeue/Return share one requeue() that only differs on Attempts++; in-flight removal, preempted reset, NotBefore, sort and log are identical. contribute_fixqueue.go:542-554 matches the relay's literal reason string (EqualFold+TrimSpace) and still calls pulseFixFailed, so a Nudge waiting on beginFixWait is released the same way.

Overlap with #17403: confirmed a trial merge-tree of the two heads is clean β€” different hunks in contribute_fixqueue.go and fixqueue.go.

One open question, not a defect: there is no TTL on a queued fix (fixqueue.go has no expiry path), so a pre-#972 relay that keeps polling another hive will have its fix re-dispatched and refused every fixTaskRetryBackoff (5 min) indefinitely, never reaching adoption. The body says that is intended ("their refusals never move the PR toward adoption"); flagging only so the maintainer knows it is unbounded, not capped.

@hivecommons-hive

Copy link
Copy Markdown
Contributor

merged by Hive trusted-author auto-merge: author @Danathar holds role owner and GitHub permission maintain on repo; CI green at a1a126a.

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style/maintainability pass at head a1a126a. No blocker; two advisory notes, worst first.

1. The hub now routes on a free-text reason string that the relay owns, with no marker on the relay side β€” src/pkg/dashboard/contribute_fixqueue.go:577 (relayRefusedNotActiveSlot) / :584 (strings.EqualFold(strings.TrimSpace(msg.Reason), …)) vs bin/contributor-relay.js:10009, where the same sentence is an inline literal. The protocol already has a typed discriminator for exactly this (contribute_protocol.go:518-531, whose comment names "routing on a value the client controls" as the hazard), and the relay already sends failure_kind on its other refusals (contributor-relay.js:9984, :9992). The hub-side constant documents the coupling; the relay side does not, so a wording tweak at :10009 (punctuation, "hive" for "hub") silently reinstates the attempt-counting the PR fixes. The string match has to stay for older relays, but the relay literal should be a named constant with the cross-reference, so the next editor sees the contract:

      const fixPush = msg.kind === FIX_PR_KIND;
      if (!currentTask && !standbyPush && !fixPush && hub !== hubs[activeHubIndex]) {
        console.log(`Rejecting task ${taskDisplay(msg)} from ${hub.url} β€” hub is not the active polling slot`);
        // Matched verbatim by the hub (src/pkg/dashboard/contribute_fixqueue.go,
        // relayRefusedNotActiveSlot) to requeue a fix-pr without counting an
        // attempt; change both sides together.
        sendTo(hub, { type: 'task_failed', seq: nextSeq(), task_id: msg.task_id, reason: NOT_ACTIVE_SLOT_REASON });
        break;
      }

with const NOT_ACTIVE_SLOT_REASON = 'Hub is not the active polling slot'; next to FIX_PR_KIND at bin/contributor-relay.js:1363.

2. failFixTask duplicates its tail in the new branch β€” contribute_fixqueue.go:542-553 repeats the len(...) == 0 "fix-pr task failed" log (:547-549 vs :555-557) and h.pulseFixFailed (:551 vs :569). The only real differences are which scheduler method runs and whether the max-attempts check applies. One shape keeps them in step:

-	if relayDidNotTakeFix(msg) {
-		// ...
-		returned := sched.Return(user, msg.TaskID, fixTaskRetryBackoff)
-		for _, it := range returned { h.logger.Info(...) }
-		if len(returned) == 0 { h.logger.Info("[contribute-ws] fix-pr task failed", ...) }
-		h.pulseFixFailed(msg.TaskID, reason)
-		return true
-	}
-	requeued := sched.Requeue(user, msg.TaskID, fixTaskRetryBackoff)
+	// A polling-slot refusal means the relay never started the fix: it goes
+	// back without an attempt and can never reach fixTaskMaxAttempts.
+	notTaken := relayDidNotTakeFix(msg)
+	requeue := sched.Requeue
+	if notTaken {
+		requeue = sched.Return
+	}
+	requeued := requeue(user, msg.TaskID, fixTaskRetryBackoff)
 	if len(requeued) == 0 {
 		h.logger.Info("[contribute-ws] fix-pr task failed", ...)
 	}
 	for _, it := range requeued {
-		if it.Attempts >= fixTaskMaxAttempts {
+		if !notTaken && it.Attempts >= fixTaskMaxAttempts {

Also worth a clause in the failFixTask doc comment (:517-522), which still says every failure counts toward fixTaskMaxAttempts.

Scope note: I did not run the Go or relay tests; they read as covering the stated cases.

β€” hive: agent=ci-maintainer backend=copilot model=claude-fable-5.1 copilot=1.0.88

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dco-signoff: yes Indicates the PR's author has signed the DCO. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant