Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/fixed-army-rework-queue.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
- Contribute "Your army" card: a remote agent's "PRs awaiting rework" now lists only its open, not-yet-dispatched PRs — merged, closed and hive-adopted PRs are dropped (and a queued fix for a PR that is gone leaves the fix queue instead of reaching the relay); the line says why a fix is not dispatched (offline, no fix-pr capability, scope, orphaned dispatch, backoff, focus window, waiting for `ready`) instead of "not yet dispatched"; a fix dispatched to a socket that dropped is released at the relay's next `ready` and a completion reported on a reconnected socket settles it; links collapse to three inline plus a "show N more" disclosure.
5 changes: 3 additions & 2 deletions src/docs/contributor-relay.md
Original file line number Diff line number Diff line change
Expand Up @@ -1775,9 +1775,10 @@ Each row of the "Connected contributor agents (ClankeR)" list on `/contribute/op
| `fix_mode` | `spawn` or `preempt`, the interrupt mode that capacity implies. |
| `in_flight[]` | One entry per work item on the connection — the primary task plus every `fix-pr` sub-session — each with `kind` (`new` or `rework`), `task_kind`, `repo`, `number`, `title`, `url`, `stage` (the item's `stage/NNN-*` label as the hive last observed it), `mode` and `started_at`. The row shows one line per item with a **NEW** / **REWORK** badge. A relay running only a boundary-dispatched fix therefore shows as *working*, not *idle*. |
| `fix_queue_depth` | Red PRs waiting in the contributor's server-side fix queue (not dispatched). |
| `pending_rework`, `pending_rework_prs[]` | The contributor's open PRs in `stage/072-returned-to-contributor` that have **not** been dispatched to them. The row prints a red **N PRs awaiting rework (not yet dispatched)** line, and an idle relay reads **idle · N PRs awaiting rework** instead of `idle: no matching work`. |
| `pending_rework`, `pending_rework_prs[]` | The contributor's **open** PRs in `stage/072-returned-to-contributor` that have **not** been dispatched to them. Merged and closed PRs, PRs the hive adopted (074) and stage-store records no stage pass has seen within the `072` dwell are never listed or counted. The row prints a red **N PRs awaiting rework (…)** line with at most three PRs linked inline and the rest behind a *show N more* disclosure, and an idle relay reads **idle · N PRs awaiting rework** instead of `idle: no matching work`. |
| `pending_rework_blocked` | Why the pending rework is not on its way to the relay right now, in the dispatcher's own order: relay offline, no `fix_pr_interrupt_v1` capability, repository scope not acknowledged, not in this hub's fix queue, an earlier fix dispatch still booked on a dropped connection, retry backoff, focus window, slots busy, or simply waiting for the relay's next `ready`. Printed in place of the old *not yet dispatched*. Empty when nothing pends. |

`kind: rework` means the item repairs one of the contributor's own PRs: any `fix-pr` task, or a task whose target the stage store has in a repair stage (`070-fixing`, `071-needs-author`, `072-returned-to-contributor`, `074-adopted-by-hive`). Everything else is `new`. `pending_rework` is derived without a GitHub call: the stage-label store (which remembers each PR's author) ∪ the fix queue, minus the fix tasks in flight on any of the contributor's connections, minus any PR the status snapshot shows as merged or closed. A stage record can outlive its PR when the stage sweep never saw it close, so a PR is dropped when its repo's open-PR list is complete (enumerated, not paused, not incomplete, stale or carrying unlisted PRs) and does not include it. Until that listing is complete (for example in the first minutes after a hub restart) a stage record counts only if a stage pass saw its PR within the last 24 hours (the `072` stalled dwell). The stage pass sees every open PR on each sweep, so a freshly returned PR still shows, while a PR a person closed or merged days ago does not. A record the store marks superseded (the hive's copy merged, or the original was closed in favour of its copy) never counts. A non-zero `pending_rework` on an idle row is the signal to look at — typically a relay that did not advertise `fix_pr_interrupt_v1`, a preempt-mode relay inside its `contribute.min_focus_window`, or a contributor who is offline.
`kind: rework` means the item repairs one of the contributor's own PRs: any `fix-pr` task, or a task whose target the stage store has in a repair stage (`070-fixing`, `071-needs-author`, `072-returned-to-contributor`, `074-adopted-by-hive`). Everything else is `new`. `pending_rework` is derived without a GitHub call: the stage-label store (which remembers each PR's author) ∪ the fix queue, minus the fix tasks in flight on any of the contributor's connections, minus any PR the status snapshot shows as merged or closed. A stage record can outlive its PR when the stage sweep never saw it close, so a PR is dropped when its repo's open-PR list is complete (enumerated, not paused, not incomplete, stale or carrying unlisted PRs) and does not include it. Until that listing is complete (for example in the first minutes after a hub restart) a stage record counts only if a stage pass saw its PR within the last 24 hours (the `072` stalled dwell). The stage pass sees every open PR on each sweep, so a freshly returned PR still shows, while a PR a person closed or merged days ago does not. A record the store marks superseded (the hive's copy merged, or the original was closed in favour of its copy) never counts. Where the snapshot cannot vouch for a repo, the forge cache the hive already holds decides when it knows the PR. A PR in `074-adopted-by-hive` is the hive's and never counts. A non-zero `pending_rework` on an idle row is the signal to look at — `pending_rework_blocked` names the cause (typically a relay that did not advertise `fix_pr_interrupt_v1`, a preempt-mode relay inside its `contribute.min_focus_window`, a fix dispatched to a socket that dropped before the relay reported on it, or a contributor who is offline). A queued fix whose PR turns out merged or closed leaves the fix queue instead of being handed to the relay, and a fix booked in flight that no live connection holds is released at the relay's next `ready`.

## Reconnecting without losing in-flight work

Expand Down
3 changes: 2 additions & 1 deletion src/docs/labels-and-control-signals/stage-labels.md
Original file line number Diff line number Diff line change
Expand Up @@ -1388,7 +1388,8 @@ When a contributor-agent PR goes red the hive does not just relabel it (a human-
in-place adoption their push makes the hive yield (above).
The contribute page's "PRs awaiting rework" list for a relay shows
`adopting (contributor offline)` on a nudged PR whose relay connection is
stale, and `adopted by hive` once the PR is in 074.
stale; once the PR is in 074 it is the hive's and leaves that relay's
list.
5. Backpressure — the per-contributor fix budget. Capacity = the relay's
advertised `max_parallel_fixes` (fix sub-sessions beside the primary),
clamped by `contribute.max_parallel_fixes` (default 2); an unadvertised
Expand Down
67 changes: 65 additions & 2 deletions src/pkg/dashboard/contribute_fixqueue.go
Original file line number Diff line number Diff line change
Expand Up @@ -623,6 +623,12 @@ func (h *ContributeWSHub) selectFixTask(c *ContributorConnection) *WSMessage {
}
user := c.profile.GitHubUsername
sched := h.fixQueue()
// A `ready` says nothing is running on this relay. A fix dispatch the
// scheduler still books for this contributor that no live connection
// holds (its socket dropped before the relay reported on it) would
// otherwise occupy the only slot forever: nothing dispatched, new work
// refused with fix_pr_in_flight. Put it back at the head of the queue.
h.releaseOrphanedFixes(user, sched)
if !sched.Pending(user) {
return nil
}
Expand Down Expand Up @@ -656,13 +662,34 @@ func (h *ContributeWSHub) completeFixTask(c *ContributorConnection, taskID strin
delete(c.fixTasksAt, taskID)
}
c.mu.Unlock()
if !ok || c.profile == nil {
if c.profile == nil {
return ok
}
user := c.profile.GitHubUsername
sched := h.fixQueue()
var prs []WSFixPR
if ok {
prs = assign.FixPRs
} else {
// The relay finished a fix it was handed on an earlier socket (the
// 1006 flap): this connection never saw the task_assign, but the
// scheduler still books the dispatch under this contributor. Settle
// it there rather than leaving the slot occupied for good.
for _, d := range sched.InFlight(user) {
if d.ID != taskID {
continue
}
ok = true
for _, it := range d.Items {
prs = append(prs, WSFixPR{Repo: it.Repo, Number: it.Number})
}
}
if !ok {
return false
}
}
if sched.CompleteDispatch(user, taskID) {
for _, pr := range assign.FixPRs {
for _, pr := range prs {
h.forgetFixOwner(user, fixqueue.Key(pr.Repo, pr.Number))
}
// Refill the freed slot with the next queued fix.
Expand All @@ -673,6 +700,42 @@ func (h *ContributeWSHub) completeFixTask(c *ContributorConnection, taskID strin
return true
}

// heldFixTaskIDs returns the fix-pr task IDs every live connection of user
// currently holds.
func (h *ContributeWSHub) heldFixTaskIDs(user string) map[string]bool {
held := map[string]bool{}
h.mu.RLock()
defer h.mu.RUnlock()
for _, c := range h.connections {
c.mu.Lock()
if c.profile != nil && strings.EqualFold(c.profile.GitHubUsername, user) {
for id := range c.fixTasks {
held[id] = true
}
}
c.mu.Unlock()
}
return held
}

// releaseOrphanedFixes returns to user's queue every dispatch the scheduler
// books in flight that no live connection holds. Called at a task boundary,
// when the relay has just said it is running nothing.
func (h *ContributeWSHub) releaseOrphanedFixes(user string, sched *fixqueue.Scheduler) {
var held map[string]bool
for _, d := range sched.InFlight(user) {
if held == nil {
held = h.heldFixTaskIDs(user)
}
if held[d.ID] {
continue
}
if items := sched.Release(user, d.ID); len(items) > 0 {
h.logger.Warn("[contribute-ws] fix-pr dispatch orphaned by a dropped connection; requeued", "username", user, "task", d.ID, "prs", len(items))
}
}
}

// failFixTask settles a fix-pr task the relay reported failed: its PRs go
// back in the queue with backoff, and a PR that has failed
// fixTaskMaxAttempts times leaves the queue for the stage engine's re-nudge
Expand Down
24 changes: 23 additions & 1 deletion src/pkg/dashboard/contribute_fleet.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import (
"encoding/json"
"fmt"
"sort"
"strings"
"time"
)

Expand Down Expand Up @@ -260,6 +261,23 @@ func (h *ContributeWSHub) FleetSnapshot() FleetSnapshot {
Clankers: make([]FleetClanker, 0, len(h.connections)),
Work: make([]FleetWorkItem, 0),
}
// Fix-pr task IDs held per contributor across all their live sockets,
// gathered before any row lock is taken so no two connections are ever
// locked at once (fleetReworkFor names a dispatch nobody holds).
heldFix := map[string]map[string]bool{}
for _, c := range h.connections {
c.mu.Lock()
if c.profile != nil && len(c.fixTasks) > 0 {
user := strings.ToLower(c.profile.GitHubUsername)
if heldFix[user] == nil {
heldFix[user] = map[string]bool{}
}
for id := range c.fixTasks {
heldFix[user][id] = true
}
}
c.mu.Unlock()
}
for _, c := range h.connections {
c.mu.Lock()
fc := FleetClanker{
Expand Down Expand Up @@ -353,7 +371,11 @@ func (h *ContributeWSHub) FleetSnapshot() FleetSnapshot {
if task == nil {
fc.IdleReason = c.lastIdleReason
}
fc.FleetRework = h.fleetReworkFor(c, fc.Stale, openPRs)
var held map[string]bool
if c.profile != nil {
held = heldFix[strings.ToLower(c.profile.GitHubUsername)]
}
fc.FleetRework = h.fleetReworkFor(c, fc.Stale, openPRs, held)
c.mu.Unlock()
fc.CurrentTask = task
fc.PromptPreview = promptPreview
Expand Down
Loading
Loading