Skip to content

perf(channels): stale-while-revalidate for region-filtered channel lists (cold 15 s after #499) #516

Closed

Description

@dborup

Problem

After #499 (region-correct channel counts), the cold region-filtered channel list got slower, as #499 disclosed (+34–57%). Measured on staging v0.2.6-rc1 against production v0.2.5:

/api/channels?region=DK cold warm
production v0.2.5 9.8 s 0.25 s
staging v0.2.6-rc1 15.0 s 0.2 s

The channel-list cache has a 60 s TTL with singleflight. The first request after expiry waits for the full cold query, so roughly one user per minute per region can wait up to 15 s.

Task

Add stale-while-revalidate to the channel-list cache, covering GetChannels(region) and GetEncryptedChannels(region), as proposed in #379:

  • When an entry has expired, serve it immediately and start one bounded background refresh per key, coalesced with singleflight.
  • Only a truly empty cache (first request after start, or an evicted key) waits synchronously.
  • Keep the cache keys bounded, keep error handling (a failed refresh keeps the stale entry and retries later, without a stampede) and keep the server read-only. No SQL under PacketStore.mu (perf(store): move runtime SQL outside packet-store locks #495 rule).
  • Bound staleness. For example, serve stale for at most N× TTL, and wait synchronously after that.
  • Optional: speed up the cold query itself (query plan or index), if that can be done without breaking the fix(channels): keep regional counts and latest message consistent #499 correctness guarantees.

Tests and evidence

  • Test that an expired entry is served immediately while exactly one refresh runs under concurrent requests.
  • Test the bounded-staleness cutoff.
  • Test that a failed refresh keeps serving stale.
  • Measure before/after locally on a realistic fixture. The coordinator re-measures on staging.

Acceptance criteria

After warm-up, no region-filtered channel-list request waits for a cold query on a cache expiry. The #499 correctness tests stay green.

Activity

  1. dborup commented on Oct 21, 2026

    @dborup
    OwnerAuthor

    Fixed by #517 (merged as 6585f5f4c2ad880d55dcfe630b7939c0ecf9427f).Fixed by #517: region-filtered channel lists use stale-while-revalidate (an expired entry is served at once while one bounded refresh runs; staleness bounded; failures keep the stale entry). APPROVE with nits, CI green.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions