Repository navigation
✨ dashboards: upgrade progress pill, history and Upgrade now (interrupt turns) - #14968
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Open PR overlap checkBase branch: Conflicts are checked with every PR merged onto the current tip of
|
9785a94 to
783df80
Compare
|
@clubanderson is auto-promoted for contributor-lane merging on this PR: the last 5 merged PRs by this author in this hive's repositories have no revert or sentinel-alert evidence. CI, reviewer verdict, diff fences, soak, holds and branch protection still apply. |
Generated by changelog-fragment-guard from the PR title (#5675). Edit the wording freely, or replace this file with the no-changelog label if the change is not user-visible. Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: clubanderson <407614+clubanderson@users.noreply.github.com>
|
@clubanderson this PR is held at Why:
Next: the hive does not push to your branch. Push a new head (or reply here); CI and the review swarm re-run automatically and the PR returns to the normal path. |
|
@clubanderson this PR is held at Failing checks:
Next: the hive does not push to your branch. Push a new head (or reply here); CI and the review swarm re-run automatically and the PR returns to the normal path. |
|
Supersession sweep found that this PR's claimed issue is already closed by another merged PR.
Decision needed (@hivecommons): #14953 already merged and closed the claimed issue, but this PR touches files the merged PR did not. Recommended: keep open until the extra changes are reviewed, or close as superseded if they are not wanted. Another option: merge anyway. Files the merged PR did not touch: This comment is edited in place by the supersession sweep; it is not duplicated. |
There was a problem hiding this comment.
Read the second commit (783df80) against the tree. The interrupt path reads correctly: upgradeInterrupts is a leaf store with TTL eviction in pending, the flag is attached in both UpgradeTo branches that can reach a draining spoke (server.go:3234, :3245) and withheld on the pause/digest-pin branches, it is cleared when the latch closes (timeline.go:550) or on a plain re-arm (saas_upgrade.go:623), and the spoke resets the atomic on every heartbeat so a flag never outlives the instruction that carried it. cmd/hive/main.go only zeroes drainTimeout when there are in-flight turns, so a quiet spoke is unaffected.
Since this is stacked on #14953, it inherits the one defect I found there: pkg/hub/upgrade_progress.go:61 checks UpgradeFailed before Upgrading, and beginUpgrade does not clear UpgradeFailed, so a retry after a failed upgrade renders missed: <old error> through draining/pulling/rolling — which is exactly the moment the new Upgrade now (interrupt turns) button is gated to appear (draining or missed); the button will show for the wrong reason on that path. Fixing it in PR A fixes it here on rebase.
No other verified issues in the UI commit. The red changelog-fragment-guard is already flagged to you via the stage label; I did not dig into it.
— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88
There was a problem hiding this comment.
Reviewed the PR-B commit (783df80) plus the fragment commit on top, against the stacked #14953 base. The interrupt mechanism reads correctly: arm/clear under a leaf mutex, 2 h TTL in pending, the flag only rides when resp.UpgradeTo is set (server.go:3234, :3245), and the spoke stores it immediately before onUpgrade (spoke/heartbeat.go:1416) so a later instruction without it resets the atomic before the drain decision runs. handleUpgradeHive keeps the existing owner/admin check in front of ?interrupt=1.
Two concrete things:
1. changelog-fragment-guard is red because this PR now carries two fragments. 783df80 added changelog.d/added-upgrade-progress-ui.md and 73b496d added changelog.d/added-14968.md; changelog.d/README.md:1 is "one changelog fragment per PR". Drop one (the added-14968.md one already carries the (#14968) suffix the compiler wants).
2. Inherits the open-row gap I flagged on #14953, and it reaches the interrupt too. upgradeInterrupts.clear is called only from recordUpgradeHistoryTransition (timeline.go:550) on the old.Upgrading && !new.Upgrading edge. The six clearUpgradeLatch sites that mutate the registry outside that diff (server.go:3012, :3097, saas_upgrade.go:1113/1140/1191, digest_pin.go:344) never fire it, so an interrupt armed for an upgrade that lands through one of those paths stays pending for up to 2 h and rides along with the next UpgradeTo — appliesTo (upgrade_interrupt.go) matches when the new target differs only if r.Target is empty, so it is mostly bounded by the SHA check, but a re-arm to the same target within 2 h would cut turns the owner did not ask to cut. Low severity given the TTL + target match; fixing the gap at the clear sites in #14953 resolves it here for free.
I did not dig into the test (rest 3/4) / coverage failures — those may just be the merge-base skew of a stacked PR; worth a look once rebased onto v5.
— hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88
|
@clubanderson this PR is held at Why:
Next: the hive does not push to your branch. Push a new head (or reply here); CI and the review swarm re-run automatically and the PR returns to the normal path. |
…pt turns)
Renders the scheduled-upgrade state machine from the previous change on
both surfaces and adds the owner action that resolves a stalled drain.
Hub hive card (dashboard.html):
- A pill for draining/pulling/rolling/verifying/missed with the drain
countdown and the agents holding the upgrade ("Draining · 2/5 agents
in turn: reviewer 38m, scanner 12m · 14m left"); a quiet scheduled row
renders exactly as before.
- A "history" badge whose tooltip lists the hub's recent attempts; red
with a miss count when any window was missed.
- "Upgrade now (interrupt turns)" while draining or missed. Same POST as
Upgrade now with ?interrupt=1, same owner/admin check, hiveConfirm
(no native dialog), audit-logged with interrupt_turns=true and
recorded on the hive timeline.
Hub → spoke: the interrupt is held in memory per hive (2h TTL, cleared
when the upgrade closes, dropped on hub restart so the spoke falls back
to draining) and rides on the heartbeat beside upgrade_to as
upgrade_interrupt. The spoke's drain decision reads it and proceeds
immediately instead of waiting out the drain window.
Spoke operator dashboard: /api/version carries upgradeProgress and
upgradeHistory; the version menu gains a Progress row with the pill and
an Upgrade history list, and the header chip shows Draining/Missed
window while either state holds. Styled with .oc-version-* classes.
Tests: interrupt store arm/pending/TTL/clear and target matching,
request parsing, heartbeat response carries the flag only with
upgrade_to, spoke dispatch exposes it to the upgrade callback,
/api/version payload fields, and markup tests for both dashboards.
Refs #14950
Signed-off-by: Andrew Anderson <andy@clubanderson.com>
…nt bullet The style ratchet counts inline style= attributes and raw px/rem values; the new pill, history badge and interrupt action now use .upgrade-progress-* (hub) and .oc-version-* (spoke) classes built on tokens.css, and the progress line shares the existing stacked Version line. Operator raw-colour baseline lowered to the measured 135. Changelog fragment starts with the required '- ' bullet. Signed-off-by: Andrew Anderson <andy@clubanderson.com>
Keeps the Version cell's one-element-per-line assembly intact for TestVersionCellPutsEachElementOnItsOwnLine. Signed-off-by: Andrew Anderson <andy@clubanderson.com>
9078b25 to
814639d
Compare
There was a problem hiding this comment.
Scoped read: I traced the Upgrade now (interrupt turns) path end to end, since it is the only part of this PR that changes what the spoke does to running agents. I did not read the pill/history rendering in dashboard.html / index.html beyond the interrupt action.
What I verified (at 814639d)
- Authorization: the interrupt is a flag on the existing owner/admin-gated
handleUpgradeHive(src/pkg/hub/saas_upgrade.go:617-623); no new endpoint, no new trust surface. It is audit-logged withinterrupt_turnsand recorded on the timeline (:627-634). - Delivery:
handleHeartbeatonly setsUpgradeInterruptwhen an unexpired requestappliesTothe target actually being instructed (src/pkg/hub/server.go:3234,:3245), so a stale request cannot ride a different SHA. - Lifecycle: cleared when the
Upgradinglatch drops (src/pkg/hub/timeline.go:550), on a later non-interrupt request (saas_upgrade.go:623), or after the 2h TTL (upgrade_interrupt.go:14,:55-58). Hub restart drops it — safe direction (spoke drains).operator-reference.md's description of this matches the code. - Consumption:
spoke.setUpgradeInterruptis written on every heartbeat that carriesUpgradeTo(spoke/heartbeat.go:1416), so the atomic reflects the latest instruction;src/cmd/hive/main.go:5181zeroesdrainTimeoutonly when the flag is set and turns are in flight, then goes through the normalselfUpgradeDrain.decide. Looks correct to me.
One nit (info): the comment on upgradeInterruptRequested (src/pkg/hub/upgrade_interrupt.go:87-88) says it accepts a JSON body {"interrupt":true}; the function only reads the ?interrupt query parameter. The dashboard sends ?interrupt=1 so nothing is broken — either drop the body clause from the comment or implement it.
The PR currently conflicts with v5 and will need a rebase before CI can judge it.
— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88
|
@clubanderson this PR is held at Failing checks:
Next: the hive does not push to your branch. Push a new head (or reply here); CI and the review swarm re-run automatically and the PR returns to the normal path. |
|
merged by Hive trusted-author auto-merge: author @clubanderson holds role owner and GitHub permission admin on repo; CI green at 814639d. |
|
Thank you for your contribution! Your PR has been merged. We'd love to hear how your experience was: share feedback |
Refs #14950 — PR B of the scheduled-upgrade visibility feature. Motivation: #14856 (FMA hive silently missed two windows) and #14925 (drain latch).
Builds on #14953 (PR A — data model, history, payloads), now merged; this PR is rebased on
v5.What this adds
Hub hive card
draining/pulling/rolling/verifying/missedwith the drain countdown and the agents holding the upgrade —Draining · 2/5 agents in turn: reviewer 38m, scanner 12m · 14m left,Missed window · drain lapsed. A quietscheduledrow renders exactly as before.historybadge; tooltip lists the hub-persisted attempts (red + miss count when any window was missed).drainingormissed. SamePOST /api/saas/hives/{id}/upgradewith?interrupt=1, same owner/admin check,hiveConfirm(no native dialog), audit-logged (interrupt_turns=true) and on the hive timeline.Hub → spoke: interrupt held per hive in memory (2 h TTL, cleared when the upgrade closes, dropped on hub restart → spoke falls back to draining); delivered beside
upgrade_toasupgrade_interrupton the heartbeat; the spoke's drain decision reads it and proceeds at once.Spoke operator dashboard:
/api/versionnow carriesupgradeProgress+upgradeHistory; the version menu gains a Progress row and an Upgrade history list; the header chip showsDraining 14m/Missed window..oc-version-*classes, no inline styles.Docs: operator-reference "Scheduled-upgrade progress and history" — where it shows, and the interrupt action. Changelog
added-upgrade-progress-ui.md.Tests
upgradeInterruptsarm/pending/TTL/clear,appliesTotarget matching,?interrupt=parsing.upgrade_interruptonly alongsideupgrade_to, and only for the armed hive./api/versionpayload fields; markup tests for both dashboards (labels for every state,hiveConfirm, no inline styles in the history list).time.Sleepadded.go test ./pkg/hub/ ./pkg/hub/spoke/ ./pkg/dashboard/ ./cmd/hive/green locally.