Skip to content

fix(nodes): calculate analytics rates over available history in full and summary endpoints #485

Closed

Description

@dborup

Assessment baseline: fork master b6852ba0876b02426b6200c1c79c7e0ebe3b9389, checked on 16 October 2026. Recheck master and competing work before implementation.

Problem

Node analytics can divide availability and packets/day by the requested 30/365-day range even when the in-memory store contains much less history. That presents missing history as inactivity. The fork's full analytics and lightweight summary endpoint share an accumulator, so both must change together; fixing only the upstream-shaped full endpoint would break their parity.

Priority: high — correctness of user-visible statistics. This is an adaptation task, not approval to copy the source patch unchanged.

Fork-specific implementation

  • Derive a conservative coverage boundary from configured retention, actual startup/background-load progress, load caps and memory-pressure eviction. Do not infer it from the first transmission's FirstSeen: an old transmission heard again can still be loaded recently.
  • Filter the numerator and denominator to the same supported interval. Define empty-store, zero-duration and partial-hour behavior explicitly; do not produce NaN/Inf or claim complete coverage while loading.
  • Share the boundary and rate calculation between full analytics and summary; preserve all existing response fields and their parity. Return explicit partial-coverage metadata and show a clear UI notice.
  • Normalize comparisons to UTC and preserve inclusive boundary behavior. Any tolerance around the retention edge must be documented and tested.
  • This does not add older SQL history or silently change sender/recipient/relay membership semantics.

Acceptance criteria

  1. Deterministic tests cover full coverage, a longer requested range, empty/loading/capped stores, pressure eviction, re-heard old transmissions, exact boundaries and non-UTC host time zones.
  2. Full and summary endpoints return matching computed statistics at the same clock instant; bins before the effective boundary are excluded.
  3. Browser tests verify the partial-data notice and its absence for genuinely complete ranges; API documentation explains requested versus effective coverage.
  4. A regression test fails on the current calculation and passes with the fix. Independent review checks denominator semantics and startup/eviction behavior.
  5. No per-request full SQL scan; measure relevant latency and allocations. No staging/production change or automatic deployment is part of this issue.

Source reference (literal text to avoid cross-repository notifications)

Repository: Kpa-clawbot/CoreScope
Pull request: 2211
Head: 78f82a6122af4e512d04702093158a6dfce3f08e
Merge: 927a8eec8a430b8813e8afa1668d536ca62ed3a0
Changed files:
.github/workflows/deploy.yml
cmd/server/chunked_load.go
cmd/server/node_analytics_coverage_test.go
cmd/server/node_hop_analytics.go
cmd/server/store.go
cmd/server/types.go
public/node-analytics.js
public/style.css
scripts/non-unit-tests.json
test-all.sh
tests/e2e/test-issue-748-node-analytics-coverage-e2e.js
tests/unit/test-issue-748-node-analytics-coverage.js

Activity

  1. dborup commented on Oct 16, 2026

    @dborup
    OwnerAuthor

    Fixed by #494 (merged as babafdf3876852774eae44550d9f958cf0f290f9).Fixed by #494: node analytics (full and summary) are computed over the available retained history, with explicit coverage metadata and a partial-history notice. Verified on staging: coverage.from tracked eviction smoothly over an hour (48 h window, no jumps). APPROVE after round 2, CI green. Follow-up for the late_evicted_observations semantics is filed separately.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions