Skip to content

Upgrade glassnode-api to 1.0.0; recover rate-limited startup metadata - #22

Merged
planadecu merged 3 commits into
mainfrom
chore/glassnode-api-1.0
Sep 29, 2026
Merged

planadecu merged 3 commits into
mainfrom
chore/glassnode-api-1.0

Conversation

@planadecu

@planadecu planadecu commented Sep 29, 2026 •

Copy link
Copy Markdown
Collaborator

What changed

glassnode-api 0.29.3 → 1.0.0, its first stable release. I checked each breaking change in its changelog against the code:

  • BulkEntry['v'] can now be null (0.30.0). This is the only source change needed. sortAssetsByMcap skips null market caps and treats them like missing ones. Before, a single null made the whole market-cap fetch throw, and the asset list silently fell back to alphabetical order (Startup has no fallback when API responses drift from glassnode-api schemas #17).
  • MetricMetadata.parameters, refs and queried are now optional. Every parameters read already used ?. with a fallback, and nothing reads refs or queried.
  • modified is now unix seconds. It isn't used.
  • Removed exports (MetricTier, MetricDataType, FetchFn): not used.
  • maxRetries now defaults to 2 in API-key mode; x402 mode stays at 0.
  • Node ≥ 22 is required. That matches the release PR (Release 0.2.0 #10).

Startup metadata was being rate-limited. Startup makes 1,817 getMetricMetadata calls. Measured live, about 16% came back 429 (rate limited) even with 1.0's two retries, and they were dropped silently. That lost those metrics' display names and supported-asset lists. It also leaked 160 point-in-time (_pit) variants into the metric list, because the is_pit filter needs each metric's metadata.

The fix is a dedicated startup client with batches of 10 instead of 20. It retries up to 5 times and waits for the rate-limit window to reset. The API's 429s carry x-rate-limit-reset (seconds until its 1,200-per-minute window resets) but no Retry-After. rateLimitAwareFetch copies that value into Retry-After, which glassnode-api honours, with maxRetryDelay raised to 60 s. It exists only in API-key mode. In x402 mode calls are paid, and a retry could pay twice, so they stay at 0 retries. Interactive calls keep 1.0's default, so a rate limit doesn't stall the UI.

Live comparison of the metadata fetch (earlier settings, before the reset-aware retries):

Setting Metadata failures (of 1,817) Time
2 retries, batches of 20 296 74 s
5 retries, batches of 20 39 68 s
5 retries, batches of 10 10 85 s

Added after review:

  • Point-in-time variants are hidden by their _pit suffix as well as by is_pit, so one whose metadata failed to load can't leak into the list. Every is_pit metric's path ends in _pit, and no other metric's does.
  • The startup cache key is now startup-data-v2, so a day-old cache that lost metadata to rate limits isn't reused after upgrading.
  • ink-uplot bumped to 0.2.16.

Verification

  • tsc passes, 43 tests pass, and pnpm audit is clean. New tests cover:
    • sortAssetsByMcap with a null market cap
    • the startup client's retry settings (API-key mode)
    • rateLimitAwareFetch
    • listedMetrics, which hides point-in-time variants without their metadata
  • Final live fresh start, with the reset-aware retries:
    • 1,817 of 1,817 metadata entries
    • 931 metrics listed, with no point-in-time variants
    • 1,743 assets in market-cap order
    • about 75 s to a ready UI, cached afterwards
  • An independent review (Opus 5.5) found it safe to merge. Its low-severity findings are addressed above. This PR should merge before Release 0.2.0 #10, which raises engines.node to 22 to match 1.0.
  • An earlier live fresh start with 1.0.0 (separate HOME, so the cache starts empty) in a pseudo-terminal:
    • 1,743 assets in market-cap order (BTC, ETH, USDT, BNB, XRP)
    • 1,807 of 1,817 metadata entries (1,485 before this change)
    • the same 931 regular metrics, and 5 point-in-time variants leaked instead of 160
    • about 89 s to a ready UI on first start; the data is then cached for a day
  • The metadata failures that remain were all 429 Rate limit exceeded. None came from 1.0's stricter response checks.

Partly addresses #17: null market caps and rate-limited metadata are fixed. Still open there: a fallback when the asset metadata call itself fails.

planadecu and others added 3 commits September 30, 2026 01:33
- glassnode-api 0.29.3 -> 1.0.0. The only source change it needs: bulk values
  (BulkEntry['v'], since 0.30) can be null — sortAssetsByMcap skips them
  (treated like a missing value) instead of the whole market-cap fetch failing
  and silently falling back to alphabetical order. MetricMetadata.parameters is
  now optional; every read already used `?.` with a fallback. Node >= 22 matches
  the release PR.
- Startup metadata burst: of 1,817 getMetricMetadata calls, ~16% came back
  429 (rate limited) even with 1.0's default 2 retries, and were silently
  dropped — losing display names, and leaking 160 point-in-time variants into
  the metric list (the is_pit filter needs the metadata). A dedicated startup
  client (API-key mode only; x402 stays at 0 retries so a paid call is never
  repeated) retries up to 5 times, and batches drop from 20 to 10.
  Measured live: 1,807/1,817 metadata kept (was 1,485), PIT leak 160 -> 5,
  ~89 s first startup (cached for a day).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PzJHgXssByMwBXH1PGbXCL
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PzJHgXssByMwBXH1PGbXCL
… suffix

From review of this PR:
- The API's 429s carry x-rate-limit-reset (seconds until its 1,200/minute window
  resets) but no Retry-After, so retries used short jittered backoff and could
  give up before the reset. rateLimitAwareFetch (startup client only) exposes
  the reset as Retry-After, which glassnode-api honours, capped at a 60 s
  maxRetryDelay. Live: 1,817/1,817 metadata (was 1,807), first startup 75 s.
- listedMetrics hides point-in-time variants by `_pit` suffix as well as
  is_pit, so a variant whose metadata failed to load can't leak into the list.
- Startup cache key -> startup-data-v2 so a day-old cache that lost metadata to
  rate limits isn't reused after upgrading.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PzJHgXssByMwBXH1PGbXCL
@planadecu
planadecu force-pushed the chore/glassnode-api-1.0 branch from 82d9b73 to 556c316 Compare September 29, 2026 23:41
@planadecu
planadecu merged commit 68875d4 into main Sep 29, 2026
3 checks passed
@planadecu
planadecu deleted the chore/glassnode-api-1.0 branch September 29, 2026 23:42
@planadecu planadecu mentioned this pull request Sep 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant