Repository navigation
✨ feature(spoke): publish bounded activity digest from heartbeat - #16024
Conversation
… coalesced updates Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor. hive/awaiting-ci: pushed 86d6a16 raising the pkg/dashboard internal-import ratchet (56→57) for pkg/hubactivity — the only failure in the dashboard shards (TestDashboardInternalImportCountRatchet). CI pending — sweep will merge when green. 🐝 Hive Agent: — hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88 Review findings from Intent alignment (#15963) — approve, one advisory. Read at head 86d6a16. The diff covers every bullet of #15963: in-process build from the memoised tracker ( Advisory (in-scope, non-blocking): the issue says private-repository items are dropped, but the implementation defines "private" as not operator-declared public: Review findings from docs-currency review of 86d6a16 (merge-base d8e1d41).
Verified against the code: the defaults, ceiling ( Review findings from Style review (head
Otherwise the new code follows the package's idioms (package-level Review findings from Style review (read at head e150c77; diff plus
Minor: |
Open PR overlap checkBase branch: Conflicts are checked with every PR merged onto the current tip of
|
…wlist pkg/hubactivity import TestAgentRuntimeConfigGolden: hub.activity is a non-pointer block so it marshals as an empty object; record it in the golden (deliberate schema change for #15963). TestDashboardImportRatchet: pkg/dashboard builds the digest from its own stage tracker and needs the shared wire contract in pkg/hubactivity (a leaf package with no hive imports); allowlist it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Two CI failures on this head (86d6a16) are reproducible from the diff, not infra. Both look like small fixes; the design (digest riding the heartbeat, coalesced 15 s updates on the same bearer) reads correctly against heartbeat.go and the #16005 hub side.
1. pkg/hub/spoke hangs 20 min — test race in TestActivityUpdatesCoalesceToOnePer15sWithBackpressure (high, blocks merge)
CI goroutine dump: test parked at src/pkg/hub/spoke/activity_publisher_test.go:343 (<-waits), run parked at src/pkg/hub/spoke/activity_publisher.go:175 (case <-p.kick).
Mechanism: fire <- clock.Now() (test:338) is a synchronous handoff to run's <-p.after(wait). The test then calls p.notify() (test:342) immediately. If that kick lands before run reaches the non‑blocking drain at activity_publisher.go:186-189, the drain swallows it, update() runs the unchanged pass once and returns false, and run goes back to <-p.kick with nothing pending — so p.after is never called again and <-waits at test:343 blocks forever. The comment at test:339-341 ("receiving that wait also proves the unchanged pass has finished") assumes an ordering nothing enforces. Production code is fine (a kick merged into an in-progress pass is the intended behaviour); the test needs a sync point that the unchanged update() pass has completed before the final notify() — e.g. a test-only post-update hook on activityPublisher, or have rec.post/build signal completion. It failed in both test (rest 1/4) jobs on this run, so it is not a flake.
2. TestSettingsDialogReferencesEveryHiveYAMLLeaf fails (medium)
settings_parity_test.go:33: hub.activity.enabled, hub.activity.max_items, hub.activity.share_private (from src/pkg/config/hub_activity.go:17-23, wired at config.go:4436) are neither referenced by the Settings UI nor listed in src/pkg/dashboard/settings_parity_allowlist.txt. Either add a Settings UI reference or allowlist the three leaves with a reason.
Verified OK: pkg/hubactivity already exists at the merge base (d8e1d41), so the import-allowlist/ratchet bump to 57 is just bookkeeping; stageTrackerPayloadFrom(..., markServed=false) correctly avoids counting the in-process reader as a served client.
— hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88
…wlist hub.activity settings TestActivityUpdatesCoalesceToOnePer15sWithBackpressure raced: a notify sent right after the timer fired could be swallowed by run's kick drain ahead of the unchanged pass, leaving the test blocked on waits until the 20m package timeout (test (rest 1/4) on #16024). Observe each digest build and only notify once the drain is past. Allowlist hub.activity.{enabled,max_items,share_private} for the Settings parity guard; they are yaml-only like hub.heartbeat_omit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Read the hub receiver, the spoke publisher and the heartbeat integration against the tree at 86d6a16; the trust-boundary parts look correct to me.
Verified:
src/pkg/hub/activity_update.go:38-83mirrorshandleHeartbeat(server.go:1845-1900) step for step: bearer captured → bodyLimitReader(maxPayloadBytes)→sanitizeHeartbeatField+isValidNameon the claimedhive_id→verifyHeartbeatBearerbound to that id → signedHeartbeatResponse. It additionally refuses unregistered hives (404), so an update cannot create registry state./api/heartbeatis registered bare atserver.go:1711, so routing the marked request ahead of the mux inHandler()bypasses no middleware.- Spoke side (
activity_publisher.go:298-340) sends only thehubactivity.Digest, applieshub.heartbeat_omitviaredactActivityand re-stamps the etag, and rejects an unsigned/invalid hub response before applying the ack. Cadence is coalesced to ≥15 s (run, line 173) with a backoff when the hub predates #15963. heartbeat.goholdspub.muacross the full beat (andupdate()across its POST) so a beat and an out-of-band update cannot interleave their acks — deliberate per the comment, and bounded byheartbeatTimeout(10 s), so worst case a beat waits ≤10 s behind an update. Not a defect, just noting the serialization is intentional.
I did not read pkg/dashboard/hub_activity.go (the tracker-side source) in the same depth.
— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88
|
Thank you for your contribution! Your PR has been merged. We'd love to hear how your experience was: share feedback |
Closes #15963
Part of #15861 (hub-wide Activity Monitor RFC; remaining children #15964/#15965 track the hub aggregation and UI). Thanks to @hanthor for the RFC.
The hub side of the bounded activity digest (#16005) can only show what spokes send, and nothing populated the
activityblock yet. This makes the spoke publish it, built in process from the stage tracker it already keeps, so no extra GitHub calls or self-fetch are needed.Per the owner decision on #15861, the digest rides the normal heartbeat, with an unchanged etag sending only the marker, plus coalesced activity-only updates (at most one per 15 s, with backpressure) on the same endpoint and bearer. #16005 only reads
activityfrom a full heartbeat, so a small hub-side guard accepts the marked activity-only update for already-registered hives and touches nothing but the in-memory digest.Scope: this is the spoke publisher plus the config surface.
hub.activity.enableddefaults on only for hosted spokes; self-hosted stays opt-in. Nothing is activated or rolled out by this PR.— hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88 requested_by=@hanthor
Dashboard import ratchet
pkg/dashboardnow importspkg/hubactivity(added tointernal/testutil/dashboard_import_allowlist.txt). Justification:pkg/hubactivityis the leaf wire contract (ceilings, urgency order, sanitiser) shared by the spoke builder and hub re-validation; the dashboard builds the digest from its own stage tracker, so there is no existing seam that already carries these types without a circular import.