Skip to content

🐛 fix: clear resolved upgrade delay warnings - #15747

Merged
hivecommons-hive[bot] merged 2 commits into
v5from
fix/upgrade-delay-20261030
Oct 30, 2026
Merged

hivecommons-hive[bot] merged 2 commits into
v5from
fix/upgrade-delay-20261030

Conversation

@clubanderson

Copy link
Copy Markdown
Member

Summary

  • Show temporary scheduled-upgrade holds as Delayed instead of missed windows on hub and spoke dashboards.
  • Clear resolved hold messages after successful upgrades, including legacy history entries, without rewriting the audit log.
  • Keep genuine unresolved failures visible and prevent stale holds or older successful outcomes from masking newer attempts.

Coverage

Added shared history normalization, hub/spoke state, heartbeat policy, version API, and shipped JavaScript rendering regressions. Regenerated the spoke dashboard from source fragments. Pre-commit code review found no significant issues. Test/build/lint verdicts are delegated to CI per AGENTS.md.

Signed-off-by: Andrew Anderson <andy@clubanderson.com>
@kubestellar-prow kubestellar-prow Bot added the dco-signoff: yes Indicates the PR's author has signed the DCO. label Oct 30, 2026
@clubanderson

Copy link
Copy Markdown
Member Author

/lgtm
/approve

@kubestellar-prow kubestellar-prow Bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Oct 30, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

@clubanderson: you cannot LGTM your own PR.

Details

In response to this:

/lgtm
/approve

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: clubanderson

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubestellar-prow kubestellar-prow Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 30, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Open PR overlap check

Base branch: v5

Conflicts are checked with every PR merged onto the current tip of v5, so a PR that only needs a rebase is not reported as conflicting with its siblings.

No conflicting open PRs or same-file overlaps were found.

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks correct to me; one operator-doc gap below.

What I verified (read at head aa824df):

  • src/pkg/hub/spoke/upgrade_history.go:8-30 only relabels rows with Outcome == delayed or Trigger == window && Outcome == missed. UpgradeTriggerWindow is written solely by windowMissed (src/pkg/hub/upgrade_history.go:190-209) and is documented as "not an attempt at all" (spoke/upgrade_progress.go:58-60), so no genuinely failed attempt gets relabelled as a hold. Persisted history is untouched (recent() returns the filtered copy, byID stays raw).
  • src/pkg/hub/upgrade_progress.go:141-142: the live-hold branch now also requires in.HoldAt.After(last.EndedAt). AutoUpgradeHoldAt is always set alongside AutoUpgradeHoldReason (upgrade_schedule.go:290-291) and cleared with it (saas_upgrade.go:929-930, saas_bulk.go:396-397), so the zero-HoldAt case cannot suppress a real hold.
  • heartbeat_upgrade_policy.go:66 ships the filtered recent() to the spoke, which runs UpgradeHistoryForDisplay again in dashboard/upgrade_progress.go:66-70 and api_version.go:98-104 — the function is idempotent so the double pass is harmless.
  • Both pill tables (static-src/js/version.js, hub/assets/dashboard.html) gained delayed with the amber draining styling; the hub history badge now counts only non-window misses newer than the last done.

Docs-currency (low): src/docs/operator-reference.md:1348 still lists missed as covering "the schedule gate's hold reason while the window is still open", :1352 says outcome (done / missed), and :1354 says the hub pill appears for … verifying and missed and the spoke chip shows Missed window while a hold is active. All three now describe the new delayed state/outcome. Worth a short edit in this PR — note #15731 (split the operator reference) is open and would move these lines, so whichever lands second needs a rebase there.

— hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read the diff against the tree at aa824df, plus the callers; this looks correct to me.

What I checked:

  • Relabelling window+missed rows to delayed is safe. UpgradeHistoryForDisplay (src/pkg/hub/spoke/upgrade_history.go:15-16) only rewrites rows whose Trigger == UpgradeTriggerWindow, and that trigger is documented as "not an attempt at all" (spoke/upgrade_progress.go:58-60) and is written only by windowMissed (hub/upgrade_history.go:203); start() never uses it. So real attempts that failed inside a window keep missed.
  • Hub live-hold vs. history row agree. windowMissed stamps the hold row EndedAt = at (hub/upgrade_history.go:194,204), so once recorded the new guard in.HoldAt.After(last.EndedAt) (hub/upgrade_progress.go:142) is false and the pill falls through to the history branch, which emits the same delayed state, Reason, NextWindowAt and "delayed: …" detail (hub/upgrade_progress.go:154-168). No visible flicker between the two paths.
  • Audit log untouched. Only recent() (hub/upgrade_history.go:75) applies the display filter; byID/persistLocked keep the raw rows, and recent()'s two callers (heartbeat_upgrade_policy.go:66, upgrade_progress.go:219) are both display/heartbeat paths. The spoke re-applies the same filter idempotently with its own Outcome.CompletedAt, which is what clears a hold before the next heartbeat.
  • dashboard.html's missed counter now excludes trigger === 'window' rows and anything older than the last done, matching the Go side.
  • go build ./pkg/hub/... ./pkg/dashboard/ is clean at the head; gofmt clean on the touched Go files.

Body claims match the diff (Delayed state, cleared holds, audit log preserved).

— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive
hivecommons-hive Bot merged commit 0ad9d56 into v5 Oct 30, 2026
75 of 76 checks passed
@kubestellar-prow
kubestellar-prow Bot deleted the fix/upgrade-delay-20261030 branch October 30, 2026 12:40
@hivecommons-hive

Copy link
Copy Markdown
Contributor
Evidence bundle `#157472448337` sha256:e9c23d2b065240c7d40caad5f743a65add1d2ff83078c8cc7881f6d859790cbb — download from the dashboard or `hivectl review evidence #15747`.

@hivecommons-hive

Copy link
Copy Markdown
Contributor

merged by Hive trusted-author auto-merge: author @clubanderson holds role owner and GitHub permission admin on repo; CI green at 2448337.

@github-actions

Copy link
Copy Markdown
Contributor

Thank you for your contribution! Your PR has been merged.

We'd love to hear how your experience was: share feedback

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. dco-signoff: yes Indicates the PR's author has signed the DCO. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant