Repository navigation
π fix(hub): bound activity event retention for low-volume hives - #16091
Conversation
Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Open PR overlap checkBase branch: Conflicts are checked with every PR merged onto the current tip of
|
|
Hive status β updated in place; edits do not notify. Full history is in the Activity Monitor. hive/awaiting-ci: CI pending β sweep will merge when green. π Hive Agent: β hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88 Evidence bundle `#16091089a82e` sha256:a1b964e0917b6d65b3337c62d502ec640aabc9ee2b18f7983a4a5ad97af0f244 β download from the dashboard or `hivectl review evidence #16091`. |
There was a problem hiding this comment.
The retention bound itself is right: forgetOldLocked runs on every record (src/pkg/hub/activity.go:83,142), so the new pass in activity_events.go:165-177 drops any entry older than 24h on the next beat from any hive, and the test's 30 hourly heartbeats exercise exactly that path. One thing I'd change before this lands:
Expiry is counted as a drop, so events_dropped stops meaning loss (medium) β src/pkg/hub/activity_events.go:173-175. Every event that is not evicted under the cap now ages out after 24h, and each one increments h.dropped. In steady state that counter becomes "every event this hive ever sent, minus the ones currently held" rather than "events that never reached, or were pushed out of, the ring". A hive that sends 50 events/day and never hits the cap reports events_dropped: 1500 after a month with nothing lost. The field's own doc still says it is what "the spoke's or the hub's per-block cap dropped before they reached the ring" (activity.go:273-274) and activityEventHive.dropped says the same (activity_events.go:133-134); hub-server.md was updated to say expired events count too, so this is deliberate, but it removes the only signal an API consumer has that data went missing. Nothing in the hub UI reads events_dropped yet (grep events_dropped src/pkg --include=*.js finds nothing), so changing it now is cheap. Suggest: drop expired entries without touching h.dropped (retention is not loss), flip the test's b.dropped["hive-b"] != 1 to != 0, and revert the "expired events are counted in events_dropped too" clause in hub-server.md:49 and the changelog fragment. If the counting is wanted, the two field comments need to say so.
Everything else: forget inside the byHive range loop is pre-existing and safe in Go; the doc edits in telemetry.md:75 and hub-server.md:49 match the new behaviour; changelog fragment name follows fixed-<slug>.md.
β hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88
Every retained event eventually passes the 24-hour retention bound, so counting expiry as a drop turned events_dropped into a running total of all events ever held and hid real cap drops. Expiry is retention, not loss; only spoke disclosures, the hub per-block cap and fair-share ring evictions count. Field comments and the API reference now say so. Hive-Fix-Attempt: 1/6 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Read the ring and store at 089a82e, not just the diff. The fix is correct; one thing in the PR body is stale.
Verified
forgetOldruns underst.mufromrecord()on every heartbeat (src/pkg/hub/activity.go:83,:136-143), so the newDeleteFunconreceivedAt(src/pkg/hub/activity_events.go:174-176) ages events out for every hive on any spoke's beat β exactly the low-volume-but-heartbeating case #16072 describes.- Ageing leaves
byHive[hive].lastSequntouched, so a spoke resending an aged-out batch is still rejected atactivity_events.go:78and nothing is stored twice. - The response
ETagis computed from the body (activity.go:387), so an event expiring changes it without any extra plumbing. - The new test does fail without the change:
ingestwith an empty batch still refresheslastSeen(activity_events.go:75), so before this PR hive-b was never forgotten and its event atactivityT0stayed in the ring through hour 30. - Not counting age-outs in
droppedis the right call for the reason the comment gives (activity_events.go:166-168); the field docs inactivity.go:273-276and both markdown pages were updated to match.
Body mismatch (info) β the body says ageing is "counting it in that hive's events_dropped". The code, the EventsDropped comment, telemetry.md and hub-server.md all say the opposite, and the opposite is what ships. Worth a one-line edit of the body so the merge commit does not record the wrong semantics; no code change needed.
Observation, not a defect: ageing only happens inside record(), so if every spoke stops beating the ring keeps up to 24h-old events until the next heartbeat. That is consistent with how forgetOld already worked for whole hives and is outside #16072's scope.
Looks correct to me.
β hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88
|
Thank you for your contribution! Your PR has been merged. We'd love to hear how your experience was: share feedback |
Why
After #16071 the activity event ring evicts only from the hive holding the most events. A low-volume hive's events were therefore never evicted while it kept heartbeating (
forgetOldonly fires after 24h of silence), removing the old guarantee that every event ages out. Data is in-memory and auth-filtered, so there is no exposure change; this restores a retention bound.Fix
forgetOld(already run on every record) now also drops any event held longer thanactivityForgetAfter(24h), for every hive, counting it in that hive'sevents_dropped. This is the simplest additive option: it reuses the existing constant and leaves fair-share eviction untouched.Docs (
telemetry.md,hub-server.md) now describe "at most 1,000 events, none held longer than 24 hours" instead of "the last 1,000 events". Adds a test where a heartbeating hive's old event ages out, plus a changelog fragment.Closes #16072
β hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88