Repository navigation
π fix: pause the 074 stall clock while an adopted PR waits in the fixer queue - #15131
Conversation
β¦er queue Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com> Co-authored-by: Danathar <6772335+Danathar@users.noreply.github.com> Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
hive/awaiting-ci: CI pending β sweep will merge when green. π Hive Agent: β hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88 |
Open PR overlap checkBase branch: Conflicts are checked with every PR merged onto the current tip of No conflicting open PRs or same-file overlaps were found. |
Generated by changelog-fragment-guard from the PR title (#5675). Edit the wording freely, or replace this file with the no-changelog label if the change is not user-visible. Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: hivecommons-hive[bot] <280983584+hivecommons-hive[bot]@users.noreply.github.com>
There was a problem hiding this comment.
The design is sound and the wiring is complete end to end; two things in the diff itself make CI red.
1. TestFixerQueueRecordResetsOnNewHead fails on its own (job 123128926316, fixer_attempt_test.go:18). The test's first call is s.MarkAdoptedInPlace(key, "aaa", β¦) on a fresh NewStore, but MarkAdoptedInPlace is a no-op when no record exists (store.go:1067-1069, if !ok { return }), and MarkFixerQueued has the same guard (store.go:963). So Get(key) returns a zero record and FixerQueuedUnattempted is false. Seed the record first the way stagelabels_test.go:1184 does (s.Observe(key, Fixing, β¦) or equivalent) before the adopt/queue marks. The production code is not at fault here β those guards are the intended "only a live adoption gets a queue entry" behaviour.
2. changelog-fragment-guard fails: changelog.d/fixed-adoption-stall-fixer-queue.md:1 is a bare sentence without the - bullet the guard requires. fixed-15131.md already carries the entry; drop the second file or bullet it.
Verified correct
ci-failing.jsonrows do carryhead_sha(merge_eligibility.go:1005tag, populated at:1196), soformatRedPRFixData's newHeadSHAfield (policy_overlays.go:248) is non-empty andmarkAdoptedFixerAttemptfires.- The attempt mark sits after the
owner != agentfilter (policy_overlays.go:288), so only the fixer that was actually handed the row records the attempt; andformatRedPRFixDatais reached only viaassembleKickβbuildAgentMessageFor(kickmessage.go:312,476), i.e. real kick assembly. adoptionStale(automerge_contributor_adopt.go:1509-1518): attempt resetssince; queued-unattempted pauses only within4Γdwell; no record β unchanged behaviour.FixerQueueFreshness(30m) stops the pause whenwriteMergeEligiblestops listing the row (fixer paused/unavailable perfixerAvailable,merge_eligibility.go:920-927). TheTestAdoptionStaleWaitsForFixerQueuetable covers each branch.
Coverage failure should resolve once (1) passes.
β hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88
There was a problem hiding this comment.
Read adoptionStale (automerge_contributor_adopt.go:1506-1522), the new Record/Store methods in stagelabels/store.go, and the two producers (merge_eligibility.go markAdoptedFixerQueued, scheduler/policy_overlays.go markAdoptedFixerAttempt). The mechanism is sound: both marks are keyed to AdoptedHead, a new head matches neither and MarkFixerQueued clears the attempt on a head change, fixerAvailable keeps a paused/cadence-less fixer from pausing the clock, and the 4-dwell cap bounds a fixer that lists but never kicks. The body matches the diff.
Two defects, both the reason CI is red:
-
src/pkg/stagelabels/fixer_attempt_test.go:16-20βTestFixerQueueRecordResetsOnNewHeadfails (test,test (rest 3/4),coveragejobs). The test callss.MarkAdoptedInPlace(key, "aaa", β¦)on a fresh store, butMarkAdoptedInPlaceis a no-op when the record does not exist (store.go:1067-1070:r, ok := s.items[key]; if !ok { return }). SoAdoptedHeadstays empty,MarkFixerQueuedbails at itsr.AdoptedHead != headguard, and the first assertion sees an all-zero record β exactly what the CI log prints. The existing adoption tests create the record first (stagelabels_test.go:955s.Observe(key, ReturnedToContributor, "check-failing:dco")beforeMarkAdoptedInPlaceat:971); add the sameObservebefore line 20. -
changelog.d/fixed-adoption-stall-fixer-queue.md:1starts with prose, not a-bullet, sochangelog-fragment-guardfails (changelog.d/README.md: "a fragment must start with a '- ' entry bullet"). The bot-generatedchangelog.d/fixed-15131.mdalready carries a valid bullet, so delete the slug file or make it the single bulleted fragment. (Same issue on #15129 β worth checking the scanner's fragment template.)
Sign-off survives the squash is CANCELLED, not failed β infra.
β hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88
MarkAdoptedInPlace is a no-op on a key the store has never observed, so the test asserted against an empty Record. Also give the changelog fragment the '- ' bullet the guard requires. Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
|
hive/awaiting-ci: CI pending β sweep will merge when green. π Hive Agent: β hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88 |
There was a problem hiding this comment.
Looks correct to me, with one low-severity note on where the "attempt" is stamped.
Read the three new paths against v6:
src/cmd/hive/merge_eligibility.go:1240,1258βmarkAdoptedFixerQueuedruns only on the twoadoptedbranches afterrouteConflictFixAgentresolvedagent, andfixerAvailablemirrors the paused/no-cadence checks already used there, so an unavailable fixer still escalates at the dwell as the body says.src/pkg/stagelabels/store.goβMarkFixerQueued/MarkFixerAttemptedboth refuse whenr.AdoptedHead != head, andFixerQueuedUnattemptedrequiresFixerQueuedSeenAtwithin 30m, so a new head or a queue that stops listing the row self-resets. No import cycle:go list -deps ./pkg/stagelabelspulls onlypkg/authorkind.src/pkg/github/automerge/automerge_contributor_adopt.go:1509-1517β attempted-after-since moves the clock; queued-unattempted pauses only whilenow-since < 4Γdwell, so a dead fixer still escalates.
Note (low): markAdoptedFixerAttempt fires inside formatRedPRFixData (src/pkg/scheduler/policy_overlays.go:355), i.e. when the kick is rendered, not when it is delivered. src/cmd/hive/main.go:8086-8092 renders all due kicks via BuildKickMessagesWithSkips and then continues past agents in provider-inference backoff. For a fixer stuck in backoff the row is marked attempted on a kick it never received, which cuts the queue pause short and starts the dwell from that render. The outcome is still "escalate at one dwell" β the same as before this PR β so it only weakens the fix during provider outages rather than regressing anything. If you want the pause to hold through backoff, stamping the attempt where the message is actually handed to the agent (or after the filtered list is built) would do it; fine to leave as a follow-up.
Tests: fixer_attempt_test.go and the new cases in automerge_adoption_escape_test.go cover queued-pause, cap, attempted-shift and head reset; CI build-and-test is still running at this head.
β hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88
There was a problem hiding this comment.
Read the three new seams against the v6 tree (stagelabels/store.go, automerge_contributor_adopt.go:adoptionStale, scheduler/policy_overlays.go, cmd/hive/merge_eligibility.go). The mechanism holds together; one timing nuance below, not a blocker.
Verified
failingPRalready serialiseshead_sha(src/cmd/hive/merge_eligibility.go:985), soformatRedPRFixData's newHeadSHAfield is populated andmarkAdoptedFixerAttemptis not a silent no-op.MarkFixerQueued/MarkFixerAttemptedboth requirer.AdoptedHead == head, and a differingFixerQueuedHeadclears the attempt β so a new head resets both without any extra plumbing, asTestFixerQueueRecordResetsOnNewHeadasserts.adoptionStale: theattempted.After(since)branch moves the clock forward only; the queued branch is bounded byadoptionQueuedDwellCapfrom the head clock, so a fixer that is listed but never kicks still escalates at 4 dwells. The "no record β today's behaviour" fallback is intact.fixerAvailablemirrors the paused/cadence checks already used for routing in this file, so an unavailable fixer never pauses the dwell.
Low β attempt is recorded at render time, not delivery (src/pkg/scheduler/policy_overlays.go:355)
formatRedPRFixData marks the attempt while the kick body is being assembled. Fixer kicks are built in BuildKickMessagesWithSkips and can still be withheld afterwards: provider-error backoff (src/cmd/hive/main.go:8080-8092), the spending-limit gate (:8115-8125), and the self-upgrade drain that follows. In those cycles the head is marked attempted although no fixer session ever saw it, so the queue pause ends early and the 074 clock runs from the render time. Consequence is bounded β that is still no earlier than today's head clock β so this degrades to the pre-PR behaviour rather than below it. If you want the pause to be exact, moving the mark to where the kick is actually dispatched (or clearing it for withheld messages) would do it; fine as a follow-up.
Open question I did not settle: whether a kick for a fixer whose previous session is still running is rendered and queued (which would also mark the attempt early) or not built at all. partitionKickableAgents filters paused agents, not running ones, so it may be worth a look.
Body matches the diff; Closes #15036 is right.
β hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88
Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
|
hive/awaiting-ci: π Hive Agent: β hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88 |
There was a problem hiding this comment.
Looks correct to me, with two low-severity points worth a maintainer glance. Read the diff plus MarkAdoptedHead/MarkAdoptedInPlace on v6 (store.go:1000-1045) and the overlay call chain kick_layout.go:117 β addRedPRFixFirst β formatRedPRFixData.
Verified:
store.goβMarkFixerQueued/MarkFixerAttemptedonly act whenr.AdoptedHead == head, andMarkFixerQueuedclears the attempt on a head change, so both records are per head as the body states;FixerQueuedUnattemptedreturns false once attempted or afterFixerQueueFreshness.automerge_contributor_adopt.go:1509-1518βsinceonly moves to the attempt time whenattempted.After(since); the pause branch is bounded bydwell*adoptionQueuedDwellCapfrom the head clock, so a never-attempted row still escalates at 4 dwells. The no-record path (FixerAttemptedSincezero and not queued) falls through to the existingnow.Sub(since) >= dwell, unchanged.merge_eligibility.goβmarkAdoptedFixerQueuedis gated byfixerAvailable, which reuses the existingallConfiguredCadencesPausedplusHasAnyCadence(config/expected_active.go:56), so an unavailable fixer leaves rows escalating as today.
Two points, both erring toward earlier escalation (i.e. today's behaviour), so neither is a regression:
policy_overlays.go:355β the attempt is recorded while the kick message is composed (formatRedPRFixData), not when it is delivered. If a composed kick is later dropped (pacing, budget), the row counts as attempted and the dwell restarts from that moment. Acceptable per the body's stated decision; noting it because aformat*function now has a store side effect, which a future reader may not expect.FixerQueueFreshness = 30m(store.go:936) is refreshed only by the routing pass inwriteMergeEligible. If that pass runs less often than every 30 minutes on a quiet hive, the pause lapses between passes and this fix silently does nothing. Open question for whoever knows the sweep cadence β I could not confirm it from the tree.
β hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88
There was a problem hiding this comment.
Looks correct to me. Read adoptionStale on v6 and traced both record paths end to end.
automerge_contributor_adopt.go:1509-1518β the new branch sits after the existing head-match / CI-pending / merge-commit guards (:1497-1508), so a moved head or running CI still short-circuits before any queue state is consulted. Attempt restarts the clock only whenattempted.After(since); the queue pause is bounded byadoptionQueuedDwellCap(4 dwells from the head clock), so a dead fixer still escalates.stagelabels/store.goβMarkFixerQueued/MarkFixerAttemptedboth refuse to write unlessr.AdoptedHead == head, and a new head clears the attempt (FixerQueuedHead != headpath), so the per-head reset the body describes is real, not implied.FixerQueuedUnattemptedrequiresFixerQueuedSeenAtwithin 30m, so a row the routing pass stopped listing (CI running, fixer paused) stops pausing on its own.merge_eligibility.goβmarkAdoptedFixerQueuedis gated byfixerAvailable(paused agent / no cadence / all cadences paused), matching the stated "only while the fixer is available". ThefailingPRstruct already serialiseshead_sha(merge_eligibility.go:983), so the newHeadSHAfield informatRedPRFixData's decoder reads a value that is actually emitted.- All checks green except tide, so no import cycle from
scheduler β stagelabels.
One low note: formatRedPRFixData (policy_overlays.go:355) now has a persistence side effect inside a formatting function. It's idempotent per head and the only caller is addRedPRFixFirst, so it's harmless today, but a future caller that formats for preview/dry-run would stamp a fixer attempt that never happened. Worth a comment at the call site, or lifting the mark into addRedPRFixFirst; not blocking.
Body matches the diff; Closes #15036.
β hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88
|
Thank you for your contribution! Your PR has been merged. We'd love to hear how your experience was: share feedback |
Closes #15036
An adopted PR's 074 stall clock ran while the PR was still unattempted in the busy single fixer's queue, so queued rebases parked at 085
adoption-stalled. The owner approved the dwell-pause direction.adoptionStale: queued and unattempted (listed within 30m) pauses the dwell, capped at 4Γ the 074 dwell from the head clock; once attempted, the dwell counts from the attempt; no queue record (unavailable fixer, same-repo or no queued row) behaves as today.Decision taken by hive: the fixer-kick "attempt" is the row being listed in detail in a scanner FIX-BEFORE-NEW kick.
β hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88 requested_by=@Danathar