Skip to content

Cache/memoize per-ledger work in the endpoint handlers for performance #989

Description

@cjonas9

What

Adjacent to the query-path optimizations in epic #732, several endpoints redo per-request work whose answer only changes once per ledger close. getLatestLedger was the slowest no-parameter endpoint by over 2x (see @tamirms's measurements here, or any release eval run), and getNetwork / getVersionInfo decoded the whole ~2 MB latest LedgerCloseMeta to read one 4-byte protocol version. Under load these cheap-looking endpoints were the first to buckle.

This issue tracks caching/memoization of that per-ledger work in the endpoint handlers. Anything that only changes when the ledger sequence changes can be memoized keyed by sequence: a new ledger invalidates the entry by moving the key, so no ingestion hook is needed, and one bounded single-entry memo amortizes the work down to once per close across all polls.

Implemented in #1003 as a generic latestMemo[T] (atomic pointer to a {seq, value} entry plus a miss mutex, so a burst at a ledger boundary computes once, an older read view cannot evict a newer ledger's value, and errors are never memoized), used by:

  • getLatestLedger: the fully rendered JSON response, keyed by sequence.
  • getNetwork and getVersionInfo: the protocol version, via a shared protocolVersionCache. Use views in getProtocolVersion #1035 (merged) first made the miss path cheap by reading the version off the header view instead of decoding the ledger.

Acceptance criteria

Measurements

Handler-level, perf-eval go-bench leg, pubnet-sized ledger:

time B/op allocs/op
getLatestLedger, uncached 8.8 ms 11.2 MiB 40
getLatestLedger, cached 1.06 µs 164 B 5

getNetwork over the pubnet sqlite fixture (v1 sqlite / v2 hot); getVersionInfo tracks it within noise:

time B/op allocs/op
before 4.2 ms / 4.4 ms 10.5 MB / 9.2 MB 103,000
header view read (#1035) 175 µs / 264 µs 1.5 MB / 8.7 KB ~200
view + memo (#1003) 10 µs / 13 µs 5 KB / 6 KB ~120

Out of scope

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions