Repository navigation
π hub: pull-only clusters use heartbeat health; neutral awaiting-heartbeat state - #10580
hivecommons-hive[bot] wants to merge 2 commits into
Conversation
β¦ awaiting state Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: scanner <scanner@hive.kubestellar.io> Co-authored-by: clubanderson <407614+clubanderson@users.noreply.github.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
hive/awaiting-ci: CI pending β sweep will merge when green. π Hive Agent: β hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88 |
Open PR overlap checkBase branch:
Conflicts are checked with every PR merged onto the current tip of
|
|
Supersession sweep found that this PR's claimed issue is already closed by another merged PR.
this PR touches files the merged PR did not. A human should decide whether to close this PR. This comment is edited in place by the supersession sweep; it is not duplicated. |
β¦atchet The awaiting-heartbeat and stale notes added inline style attributes with raw rgba() colors and px padding, pushing the hub static pages over the TestStyleRatchet baseline (961/456/167). Move them to a token-only .cluster-note class so the ratchet counts stay exact. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: adjudicator <adjudicator@hive.kubestellar.io>
|
PR needs rebase. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
There was a problem hiding this comment.
[reviewer] recommend close: superseded β issue #10559 was closed 2026-10-05T14:19Z by human-authored PR #10567 (fix/pull-only-heartbeat-health-10559, commit 4abc571) merged to v5 before this PR could land. #10567 implements the same fix: pull-only clusters get heartbeat-reported health, a PerClusterHealth status/note (awaiting_heartbeat / stale_heartbeat), a neutral dashboard message class, and the boot_seams wiring. This branch edits the same regions of saas_cluster_health.go, boot_seams.go, main.go and dashboard.html with a parallel design (awaiting_heartbeat/pull_only bool fields), so merging it would duplicate fields and rendering paths. evidence: git log origin/v5 -S awaiting_heartbeat β 4abc571; merge db4d5c2 of #10567; gh issue view 10559 β CLOSED/COMPLETED at 14:19:09Z. tests: CI failure (TestStyleRatchet: +2 inline style=, +2 raw rgba colors, +2 raw padding over the 961/456/167 baseline) was confirmed with a scratch build of the ratchet counters and is fixed on this branch by commit 1f0e2b7, but that is moot given supersession. remaining risk: none β the shipped fix in v5 covers the issue; the one difference (#10567 does not expose a pull_only bool) is not needed by the dashboard.
β hive: agent=adjudicator backend=copilot model=claude-fable-5.1 copilot=1.0.88
β Escalation cleared β back in the automated laneCI is green on the current head β returning this PR to the automated lane. The |
Root cause (code-level)
For a pull-only cluster,
buildSingleClusterHealthreturnserrClusterPullOnlyby design, and the hub falls back toheartbeatHealth[clusterID]. That entry is only written when a heartbeat carriescluster_health.Spoke-side gap:
cmd/hiveattachedClusterHealthonly when theHIVE_CLUSTER_IDenv var was set. The payload'scluster_id, which the hub uses as theheartbeatHealthkey, comes from the resolved id:hub.cluster_idfrom hive.yaml, overridden by the env var. A firewalled spoke configured through hive.yaml (hub.cluster_id: vllm-d) therefore heartbeated withcluster_id=vllm-dbut never sent node health. The hub had nothing to fall back to and showed0 nodesplus the red error.I checked the hub-side key path too. On write,
entry.ClusterIDcomes from the SaaS record'sClusterID, then the payloadcluster_id, thendefaultClusterID. On read, the registry id is used. The two match in code.heartbeatHealthStalenessonly marks entries stale; nothing is evicted. Nothing skips the report when the hive count is 0.Fix
heartbeatClusterHealth(clusterID, collect)inboot_seams.gogates collection on the resolvedb.cfg.Hub.ClusterIDinstead of the env var alone. With no cluster id it still sends nothing and doesn't run the collector.PerClusterHealth{PullOnly, AwaitingHeartbeat, LastSeen}with noError.LastSeenis the newestLastHeartbeatof any registry hive on that cluster, ornever. Heartbeat-backed pull-only entries also carrypull_only.pull-only Β· awaiting spoke heartbeat (last seen: never/<age>)line. Red is kept for real errors and for heartbeat data that went stale (data_stale), which the dashboard didn't show before.src/pkg/hub/cluster_health_pull_only_test.go,src/cmd/hive/boot_heartbeat_cluster_health_test.go.changelog.d/fixed-10559.md.Remains (needs-human)
The card's
0 hivesmeans no registry hive is currently keyed tovllm-dat all. The spoke-side gap above is real, but it doesn't explain that on its own. Live hub logs are needed to confirm:cluster_idit is filed under. A SaaS record whosecluster_idwas defaulted tohive-okebyensureClusterIDForClaimwould override the spoke's own report;CollectClusterHealthreturns nil silently, at debug level only, when it isn't.With this change, the new
last seenvalue on the card answers the first question directly.Refs #10559 (needs-human: live hub logs needed to confirm whether a vllm-d spoke is heartbeating)
β hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88 requested_by=@clubanderson