Skip to content

πŸ› Explain why a scheduled hive upgrade window was held - #12061

Merged
hivecommons-hive[bot] merged 1 commit into
v5from
scanner/fix-11196
Oct 15, 2026
Merged

hivecommons-hive[bot] merged 1 commit into
v5from
scanner/fix-11196

Conversation

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Gap

#11236 fixed stable promotion (stable now advances every ~15–60 min on v5 β€” verified in the Oct 14 promote-stable runs). The reopen is about the other half: the FMA hive (:stable, hub-managed, daily window 13:00 ET) did not upgrade at 13:00 EDT Oct 14, and its dashboard still showed the 2026-10-14T03:30Z manual upgrade with no explanation.

In triggerAutoUpgrades, every gate that can turn away a daily/weekly hive after its window has opened β€” earlier upgrade latch still pending, digest pin, claim in flight, provisioning/error status, no reported commit, unresolved channel, merge debounce, no cluster config, undeliverable hive, full upgrade wave β€” only logged at Debug (or a hub-side Warn). Nothing reached the hive record, the heartbeat, or the spoke dashboard, so a skipped window could not be told apart from "the schedule never ran", and there was nothing for the reporter or a maintainer to act on.

Fix

  • Hub: while a hive's daily/weekly window is open, each of those gates records its reason on the hive record (auto_upgrade_hold_reason / auto_upgrade_hold_at, written only when the reason changes, so a persistent hold costs one save). The hold clears when the hive is armed, reaches or passes its target, or its auto-upgrade setting is changed (single and bulk toggle).
  • Heartbeat: HeartbeatUpgradePolicy relays it as scheduled_hold_reason / scheduled_hold_at for hub-managed daily/weekly hives (mirrored in the spoke struct).
  • Spoke dashboard: /api/version.autoUpdate gains heldReason / heldAt, appended to the behind/unknown detail ("The scheduled update window opened but the hub held the upgrade at …: …"), and shown as a Held row in the version menu plus a line in the Hub-tab auto-update status (uses the existing text-warn class, no inline styles).
  • Docs: release-channels.md waiting section; changelog fragment.

Tests

  • pkg/hub/scheduled_upgrade_hold_test.go: window-open predicate, the real sweep records a held claim at 14:00 ET but not before the window or for a manual hive, persist/repeat/clear semantics, and heartbeat relay (daily/weekly yes; instant, disabled, nothing-held no).
  • pkg/dashboard/autoupdate_status_test.go: held detail/JSON in behind state; cleared when up to date, paused or disabled; index.html renders the hold.

Refs #11196 (needs-human: reporter must confirm the FMA hive upgrades at its next window after this ships)

β€” hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88 requested_by=@MikeSpreitzer

Every gate in triggerAutoUpgrades that turned away a daily/weekly hive while
its window was open logged at Debug only, so a stable-channel hive that missed
its 13:00 ET window kept showing the previous manual upgrade with no reason.
Persist the hold reason on the hive record, relay it in the heartbeat upgrade
policy, and render it in the spoke auto-update status and version menu.

Refs #11196

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: scanner <scanner@hive.kubestellar.io>
Co-authored-by: MikeSpreitzer <14296719+MikeSpreitzer@users.noreply.github.com>
@kubestellar-prow kubestellar-prow Bot added the dco-signoff: yes Indicates the PR's author has signed the DCO. label Oct 15, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign hanthor for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

hive/awaiting-ci: CI pending β€” sweep will merge when green.


🐝 Hive Agent: scanner | Instance: hosted-available-oke-11-placeholder-r05x | SHA: unknown

β€” hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88

@kubestellar-prow kubestellar-prow Bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Oct 15, 2026
@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Open PR overlap check

Base branch: v5

Conflicts are checked with every PR merged onto the current tip of v5, so a PR that only needs a rebase is not reported as conflicting with its siblings.

PR Title Relation Files
#12073 πŸ› dashboard: show upgrade animation when upgrade is hub-initiated same files src/pkg/dashboard/static/index.html
#12059 πŸ› hub: deliver cluster_id via heartbeat so pre-#11432 push-reported spokes send node health same files src/pkg/hub/heartbeat.go
src/pkg/hub/spoke/heartbeat.go
#12058 🌱 dashboard: unique, section-specific sidebar nav icons same files src/pkg/dashboard/static/index.html
#12051 πŸ› fix(automerge): gate contributor-lane unattended merge on vetted identity same files src/pkg/dashboard/static/index.html

@github-actions

Copy link
Copy Markdown
Contributor

Thank you for your contribution! Your PR has been merged.

We'd love to hear how your experience was: share feedback

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

copilot dco-signoff: yes Indicates the PR's author has signed the DCO. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants