Skip to content

Continuous re-kick wake never fires on a busy run loop; busy-agent kick bounces escalate continuous backoff #14820

Closed

Description

@clubanderson

Observed

On hive's own hosted spoke (image rev 1997bc9, ACMM 6, evalMode=webhook, governor mode quiet), the continuous re-kick wake from #12561 never ran during a 3h window (00:30–03:30Z, 2026-10-28):

  • grep -c "continuous re-kick wake" over --since=3h logs: 0 (the string is present in the binary).
  • 17 eval cycles in 3h, 13 of them from review/verdict/github webhook wakes, the rest from the 900s cadence ticker.
  • Continuous agents (scanner, reviewer, reviewer-2, adjudicators, guide) sat idle an average of 4.4 min (max 17.7 min) after their NextKick was due; kicks only landed when a webhook or cadence eval happened to run. Scanner idle 03:05–03:11Z with 21 actionable issues.
  • /api/status showed continuous.nextKickAt in the past for several agents while governor reported no wake.

Secondary: kicks that bounce off an agent that is mid-turn are recorded as delivery failures:

Analysis

  1. cmd/hive/main.go run loop: armContinuousWake() runs after every tick and re-arms the timer with a minimum delay of 1s (continuousWakeMinDelay) even when the re-kick is already due. Each tick takes 6–13 min live (eval → kick dispatch with 120s prompt waits per busy lane → sweeps, especially automerge 2–6 min). A re-kick that comes due during the tick is "due now" at tick end, but at that instant another select channel is already ready (the cadence ticker's pending fire — e.g. the 23:04:40 fire was consumed at 23:17:55 — or a coalesced review/verdict/webhook wake), so wakeCh never wins the select. The loop goroutine was never idle in the window ("state persisted" followed within ~1s by the next tick every time), so the timer path was unreachable in practice.
  2. onFailed in the kick dispatcher unconditionally calls RecordKickFailure; the dispatcher also sends every message in an agent's lane even after the first one has been delivered and started a turn.

Expected

  • A due continuous re-kick is served at tick end regardless of which channel the loop wakes on, as a lean cycle (no sweeps) so the kick lands in eval time rather than eval+sweeps time.
  • A kick that bounces because the agent is mid-turn is a benign deferral: no backoff escalation, no further sends into that agent's lane in the same cycle.

Activity

  1. hivecommons-hive commented on Oct 28, 2026

    @hivecommons-hive
    Contributor

    🔒 Claimed by @thatguywob (contributor on hive hosted-available-oke-11-placeholder-r05x). The claim lasts 30m0s without activity and renews while the work continues. Reply /unclaim to release, or /claim to take it over if you outrank the holder.

  2. added
    help wantedDenotes an issue that needs help from a contributor. Must meet "help wanted" guidelines.
    on Oct 28, 2026
  3. hivecommons-hive commented on Oct 28, 2026

    @hivecommons-hive
    Contributor

    🔓 Claim by @thatguywob released (lease revoked).

  4. github-actions commented on Oct 28, 2026

    @github-actions
    Contributor

    @clubanderson this issue was closed by @hivecommons-hive[bot] as completed.

    If this was closed by a merged PR, the fix only takes effect once it ships in a tagged release and your deployment updates to it — check your running version against the fix before concluding the fix failed.

    If you are still seeing the problem, comment /reopen and it will be reopened for you — you do not need write access. Please add what you are still observing (running version, logs, steps) so it is not closed the same way again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    help wantedDenotes an issue that needs help from a contributor. Must meet "help wanted" guidelines.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions