You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Cache/memoize per-ledger work in the endpoint handlers for performance #989
Adjacent to the query-path optimizations in epic #732, several endpoints redo per-request work whose answer only changes once per ledger close. getLatestLedger was the slowest no-parameter endpoint by over 2x (see @tamirms's measurements here, or any release eval run), and getNetwork / getVersionInfo decoded the whole ~2 MB latest LedgerCloseMeta to read one 4-byte protocol version. Under load these cheap-looking endpoints were the first to buckle.
This issue tracks caching/memoization of that per-ledger work in the endpoint handlers. Anything that only changes when the ledger sequence changes can be memoized keyed by sequence: a new ledger invalidates the entry by moving the key, so no ingestion hook is needed, and one bounded single-entry memo amortizes the work down to once per close across all polls.
Implemented in #1003 as a generic latestMemo[T] (atomic pointer to a {seq, value} entry plus a miss mutex, so a burst at a ledger boundary computes once, an older read view cannot evict a newer ledger's value, and errors are never memoized), used by:
getLatestLedger: the fully rendered JSON response, keyed by sequence.
getNetwork and getVersionInfo: the protocol version, via a shared protocolVersionCache. Use views in getProtocolVersion #1035 (merged) first made the miss path cheap by reading the version off the header view instead of decoding the ledger.
simulateTransaction's getLatestLedgerPreflightInfo stops full-decoding the latest ledger for three header fields (protocol version, bucket list size, close time): read them off the view and memoize by sequence like the above
[optional] decide whether the three per-handler memos should become one shared latest-ledger cache wired through specs.go; per-field laziness would be needed so a getNetwork miss does not trigger the 2 MB render
What
Adjacent to the query-path optimizations in epic #732, several endpoints redo per-request work whose answer only changes once per ledger close.
getLatestLedgerwas the slowest no-parameter endpoint by over 2x (see @tamirms's measurements here, or any release eval run), andgetNetwork/getVersionInfodecoded the whole ~2 MB latestLedgerCloseMetato read one 4-byte protocol version. Under load these cheap-looking endpoints were the first to buckle.This issue tracks caching/memoization of that per-ledger work in the endpoint handlers. Anything that only changes when the ledger sequence changes can be memoized keyed by sequence: a new ledger invalidates the entry by moving the key, so no ingestion hook is needed, and one bounded single-entry memo amortizes the work down to once per close across all polls.
Implemented in #1003 as a generic
latestMemo[T](atomic pointer to a{seq, value}entry plus a miss mutex, so a burst at a ledger boundary computes once, an older read view cannot evict a newer ledger's value, and errors are never memoized), used by:getLatestLedger: the fully rendered JSON response, keyed by sequence.getNetworkandgetVersionInfo: the protocol version, via a sharedprotocolVersionCache. Use views in getProtocolVersion #1035 (merged) first made the miss path cheap by reading the version off the header view instead of decoding the ledger.Acceptance criteria
getLatestLedgercaches its rendered response, invalidated by ledger sequence ([DRAFT] Optimize query-path caching/memoization #1003)getNetworkandgetVersionInfomemoize the latest ledger's protocol version by sequence ([DRAFT] Optimize query-path caching/memoization #1003, on top of Use views in getProtocolVersion #1035)simulateTransaction'sgetLatestLedgerPreflightInfostops full-decoding the latest ledger for three header fields (protocol version, bucket list size, close time): read them off the view and memoize by sequence like the abovespecs.go; per-field laziness would be needed so agetNetworkmiss does not trigger the 2 MB renderMeasurements
Handler-level, perf-eval go-bench leg, pubnet-sized ledger:
getLatestLedger, uncachedgetLatestLedger, cachedgetNetworkover the pubnet sqlite fixture (v1 sqlite / v2 hot);getVersionInfotracks it within noise:Out of scope