Skip to content

bench: add a divan harness and the first hot-path benchmarks - #248

Open
BenjaminDEMAILLE wants to merge 1 commit into
mainfrom
feat/bench-harness
Open

bench: add a divan harness and the first hot-path benchmarks#248
BenjaminDEMAILLE wants to merge 1 commit into
mainfrom
feat/bench-harness

Conversation

@BenjaminDEMAILLE

Copy link
Copy Markdown
Contributor

Closes #204.

Several open dependency questions (#162, #202, #205, #208) all start with "measure first", and each was about to invent its own measurement. This is the shared one.

Divan rather than criterion

Both were on the table in the issue. Divan because the questions that prompted a harness compare peak memory as much as speed (sufr/libsais against caps-sa is explicitly about genomeGenerate wall time or peak RSS), and divan reports allocation counts next to wall time with no extra setup. A full run also takes seconds, which keeps it usable inside a pull request rather than only in a nightly job.

Criterion is the better choice if the CI integration in #245 wants its report format. Swapping is a benches/ change, not an API one, so this does not foreclose it.

What is measured

Three groups, picked as the functions the open questions would actually replace:

All are pure functions with no genome index to build, so a run takes seconds and a regression is attributable to one function rather than to a pipeline.

Sample output (--sample-count 3, an M-series laptop):

seed_scan_full_match     50    3.0 ns
                        250    8.1 ns
gene_overlap_query      100   101.5 ns
                      20000   189.1 ns

The bench feature

find_stop and the GTF record type are private to the crate. Rather than widen the published API for a benchmark, both are exposed under a bench feature that only benches/ enables, marked #[doc(hidden)]. Nothing changes for a normal build or for crates.io consumers.

The DEPENDENCIES.md row for divan belongs with #246, which introduces that file; the rationale above will move there rather than being duplicated.

cargo test, cargo clippy --all-targets -- -D warnings on a cold cache, and cargo fmt --check are green.

🤖 Generated with Claude Code

Several open dependency questions (#162, #202, #205, #208) all start with
"measure first", and each was about to invent its own measurement. This is the
shared one.

Divan rather than criterion: the questions that prompted a harness compare
peak memory as much as speed, and divan reports allocation counts next to wall
time without extra setup; a full run also takes seconds, so it stays usable in
a pull request rather than only in a nightly job. Criterion is the better
choice if the CI integration in #245 needs its report format, and swapping is
a benches/ change rather than an API one.

Three groups, chosen as the functions the open questions would replace:
seed-extension scanning (`find_stop`, what a portable-SIMD crate would swap),
the gene-overlap query and the annotation build (what an interval crate would
swap). All are pure functions with no genome index to build, so a run takes
seconds and a regression is attributable to one function.

`find_stop` and the GTF record type are private to the crate; rather than
widen the published API for a benchmark, both are exposed under a `bench`
feature that only benches/ enables.

Closes #204.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

dev-deps: add a benchmark harness (divan or criterion) — there is no benches/ today

1 participant