Skip to content

Full-history query benchmark commands — reads across all data types  #856

Open

Description

@marwen-abid

TL;DR

#771 was re-scoped to ingestion benchmarks; this issue carries the query half. Rewire the rpc-hack read benches onto the production store readers as a bench-query subcommand next to bench-ingest, reusing #771's CSV sink and reporting format. Together with #771 this completes the original scope (and #730's).

Scope

  • The eight read benches from the prototype: {hot,cold} × {ledgers, txpage, txhash, events}.
  • Measure the production read paths in internal/fullhistory/storage/stores/ — e.g. eventstore/query.go + cold_reader.go, ledger/cold_reader.go + hot_store.go, txhash/read_assembly.go + cold_reader.go — not the prototype's own reader implementations (same rewiring principle as Full-history benchmark commands — ingest across all data types #771).
  • Cold event/txhash benches must measure the complete correct read, including the MPHF post-filter — an index probe alone benchmarks a correctness bug, not a query.
  • --query-concurrency sweeps and the grid runner (concurrency × data type matrix).
  • Events query-corpus auto-generation: derive realistic contract-ID/topic terms from the ingested data rather than hand-picked constants.
  • XDR-views only (the decided path; no views-vs-decode comparison).
  • CSV output in Full-history benchmark commands — ingest across all data types #771's schema (stage,n,n_items,total_ns,p50_ns,p90_ns,p99_ns,max_ns) so Run full-history benchmark commands in CI to catch perf regressions #775 can consume ingest and query reports uniformly.
  • Land on feature/full-history at mergeable quality.

Source

rpc-hack scripts/bench-fullhistory/: bench_{cold,hot}_*.go, bench_grid.go, bench_concurrent_runner.go, corpus.go, and the cache-eviction helpers (cold read numbers are meaningless against a warm page cache). Carry the harness and methodology; leave the prototype's reader code behind.

Reuses from #771

internal/fullhistory/bench/ — the cobra subcommand pattern, the CSV/percentile sink, and the fixture/source plumbing. Benchmark inputs (a populated hot DB / frozen cold chunks) can be produced by bench-ingest itself.

Out of scope

Dependencies

Pinned by marwen-abid

Activity

  1. self-assigned this
    on Jul 11, 2026
  2. moved this from To Do to No status in Platform Scrumon Jul 11, 2026
  3. changed the title [-]Full-history query benchmark commands — reads across all data types (split from #771)[/-] [+]Full-history query benchmark commands — reads across all data types [/+] on Jul 21, 2026
  4. removed this from the platform sprint 74 milestone on Jul 21, 2026
  5. 14 remaining items

  6. moved this from In Progress to Needs Review in Platform Scrumon Oct 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

Relationships

None yet

Development

No branches or pull requests

Issue actions