You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
perf(nodes): region-filtered /api/nodes cold build takes ~1 min per region set (build per IATA once, union per request) #618
A region-filtered Nodes request (/api/nodes?region=…) takes about a minute the first time each region set is requested after a restart. During that time the Nodes page shows nothing. Behind the reverse proxy the request can time out, and the page shows a 502 with no visible error. A region-group quick pick (#565) or any new combination of codes is a new set, so it is cold again.
Measured from the [nodes-region] … full build log lines:
Instance
Retention / DB
Cold build per region set
Production (2 vCPU / 3.8 GB)
about 67.8M observation ids
RKE 70.2 s, RNN 71.3 s
Canary (8 GB)
7 d retention
RNN 53.4 s, AAL,AAR 52.6 s, AAR 34.2 s
Either, warm
about 6 ms
This is pre-existing: cmd/server/nodes_region_cache.go has not changed between v0.2.5 and v0.2.6. #612 (#488 M2) touched it only after v0.2.6.
Cause
scanNodeRegionKeys does a full history scan per region set: the region's observers ⋈ observations ⋈ ADVERT transmissions. The cache is keyed by the joined set of codes, so N quick picks or combinations cost N full scans of the whole retained history.
Proposal
Build the heard membership per IATA code in one shared scan:
one pass over ADVERT observations grouped by observer IATA;
serve any region set as the union of the per-code key sets;
keep the existing watermark/delta and rebuild semantics per code, and the singleflight.
This is the same shape #488 M2's placement aggregate already uses (one shared full scan for all regions). Reuse that machinery instead of duplicating it where possible.
Also:
Warm the per-code sets in the background after startup, outside any request and outside PacketStore.mu, so the first user request does not pay for the scan.
Bound memory: about 15 codes × a few thousand keys.
Acceptance
After warmup, any region set (single, multi, quick-pick group) answers in milliseconds.
The cold path (warmup not finished) never blocks longer than today, and concurrent identical requests share one build.
The result is identical to today's per-set build for every set: a property test over the fixture plus a generated large DB.
Activity
test(nodes): cold region requests wait for the warmup build; fallback…
fix(nodes): a cold region request waits for the warmup's index build (#…
dborup commentedon Oct 26, 2026
Fixed by #619 (merged as
960c910cc2270e489b1fb92c404e8c8058115da0).