Skip to content

[DRAFT] Optimize query-path caching/memoization - #1060

Draft
cjonas9 wants to merge 20 commits into
bypass-jrpc2-marshalfrom
cache-getLatestLedger
Draft

cjonas9 wants to merge 20 commits into
bypass-jrpc2-marshalfrom
cache-getLatestLedger

Conversation

@cjonas9

@cjonas9 cjonas9 commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Supersedes #1003, which GitHub auto-closed as merged into bypass-jrpc2-marshal when the stack was reordered to put this PR first. What little review history exists for this PR lives there.

What

Adds latestMemo[T] in methods/latest_memo.go, an atomic pointer to a {seq, value} entry plus a miss mutex. A new ledger invalidates it by moving the key, so there is no ingestion hook.
This is used so that...

  • getLatestLedger memoizes its rendered response.** The fully rendered JSON is cached keyed by ledger sequence.
  • getNetwork and getVersionInfo also benefit from this, as getProtocolVersion now memoizes the result by sequence with the same latestMemo.

You can compare the optimized performance seen in coordinator runs here to the base perf measured in #1018. The results in the coordinator runs here include all recent optimization work -- XDR views adoption (#945, #962, #941, #1035, all validated by tests in #976, #982), as well as more recent tangential optimizations (see the stack, particularly #1036, 1037, and this PR).

Measurements

Handler-level, from the perf-eval go-bench leg on this branch (pubnet-sized ledger):

time B/op allocs/op
GetLatestLedger, uncached 8.8 ms 11.2 MiB 40
GetLatestLedger, cached 1.06 µs 164 B 5

getNetwork over the pubnet sqlite fixture (BenchmarkLedgerReads harness; v1 sqlite / v2 hot). getVersionInfo tracks it within noise:

time B/op allocs/op
before 4.2 ms / 4.4 ms 10.5 MB / 9.2 MB 103,000
header view read 175 µs / 264 µs 1.5 MB / 8.7 KB ~200
view + memo 10 µs / 13 µs 5 KB / 6 KB ~120

The memo row is getHealth cost. The 103K allocations per call were what made these endpoints fold under concurrency.

Tests

  • getLatestLedger: serves the memo until the ledger advances, an older view does not evict a newer render, 32 concurrent misses render once, errors are not memoized, and the rendered bytes are byte-equal to marshaling the response struct.
  • Protocol version: same memo behaviour, plus getNetwork doing one raw read across two requests.
  • BenchmarkGetLatestLedger gained cached and uncached rows.

Why

getLatestLedger, getNetwork and getVersionInfo are what SDKs and wallets poll continuously, and they were the first endpoints to buckle under load even though their answers change once per ledger. Proportionate to what they return, they were made needlessly expensive because every request touched the 2 MB latest ledger. This is a full decode for one field or a base64 render of the whole thing.

Known limitations / additional notes

  • Per-handler memos, not one shared cache. getLatestLedger, getNetwork and getVersionInfo each hold their own. A shared one would need wiring through specs.go, and a getNetwork miss would then either trigger the 2 MB render or need per-field laziness.

@cjonas9
cjonas9 added this pull request to stack #1061 October 2, 2026 23:29
@cjonas9 cjonas9 linked an issue Oct 5, 2026 that may be closed by this pull request
3 of 5 tasks
@cjonas9
cjonas9 removed this pull request from stack #1061 October 5, 2026 22:57
@cjonas9
cjonas9 changed the base branch from feature/full-history to bypass-jrpc2-marshal October 5, 2026 23:00
@cjonas9
cjonas9 force-pushed the bypass-jrpc2-marshal branch from 4490bc6 to 731856f Compare October 5, 2026 23:12
@cjonas9
cjonas9 force-pushed the cache-getLatestLedger branch from 406cf48 to d7df9c9 Compare October 5, 2026 23:12
@cjonas9
cjonas9 added this pull request to stack #1093 October 5, 2026 23:47
@cjonas9
cjonas9 removed this pull request from stack #1093 October 5, 2026 23:48
@cjonas9
cjonas9 added this pull request to stack #1094 October 5, 2026 23:48
@cjonas9
cjonas9 force-pushed the cache-getLatestLedger branch 3 times, most recently from 5c6163a to 0983f01 Compare October 6, 2026 23:10
@cjonas9
cjonas9 force-pushed the cache-getLatestLedger branch from 0983f01 to f932dc1 Compare October 7, 2026 18:17
@cjonas9
cjonas9 force-pushed the bypass-jrpc2-marshal branch from 45d4c4c to 3caa253 Compare October 7, 2026 18:18
@cjonas9
cjonas9 force-pushed the cache-getLatestLedger branch from f932dc1 to 7660e51 Compare October 7, 2026 19:53
@cjonas9
cjonas9 force-pushed the bypass-jrpc2-marshal branch from 99f86ed to b99e8eb Compare October 8, 2026 16:41
@cjonas9
cjonas9 removed this pull request from stack #1094 October 8, 2026 18:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cache/memoize per-ledger work in the endpoint handlers for performance

1 participant