bench: add a divan harness and the first hot-path benchmarks - #248
Open
BenjaminDEMAILLE wants to merge 1 commit into
Open
bench: add a divan harness and the first hot-path benchmarks#248BenjaminDEMAILLE wants to merge 1 commit into
BenjaminDEMAILLE wants to merge 1 commit into
Conversation
Several open dependency questions (#162, #202, #205, #208) all start with "measure first", and each was about to invent its own measurement. This is the shared one. Divan rather than criterion: the questions that prompted a harness compare peak memory as much as speed, and divan reports allocation counts next to wall time without extra setup; a full run also takes seconds, so it stays usable in a pull request rather than only in a nightly job. Criterion is the better choice if the CI integration in #245 needs its report format, and swapping is a benches/ change rather than an API one. Three groups, chosen as the functions the open questions would replace: seed-extension scanning (`find_stop`, what a portable-SIMD crate would swap), the gene-overlap query and the annotation build (what an interval crate would swap). All are pure functions with no genome index to build, so a run takes seconds and a regression is attributable to one function. `find_stop` and the GTF record type are private to the crate; rather than widen the published API for a benchmark, both are exposed under a `bench` feature that only benches/ enables. Closes #204.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #204.
Several open dependency questions (#162, #202, #205, #208) all start with "measure first", and each was about to invent its own measurement. This is the shared one.
Divan rather than criterion
Both were on the table in the issue. Divan because the questions that prompted a harness compare peak memory as much as speed (
sufr/libsaisagainstcaps-sais explicitly aboutgenomeGeneratewall time or peak RSS), and divan reports allocation counts next to wall time with no extra setup. A full run also takes seconds, which keeps it usable inside a pull request rather than only in a nightly job.Criterion is the better choice if the CI integration in #245 wants its report format. Swapping is a
benches/change, not an API one, so this does not foreclose it.What is measured
Three groups, picked as the functions the open questions would actually replace:
seed_scan_full_match/seed_scan_stop_at_half—simd_scan::find_stop, the innermost loop of seed extension and the function a portable-SIMD crate (Dependency discussion: portable SIMD crate (wide/pulp) vs the hand-rolled intrinsics in simd_scan.rs #205) would swap. Two shapes, because a full match and an early stop have different costs.gene_overlap_query— the segment-tree overlap query, per read and twice forGeneFull, and what an interval crate (Dependency discussion: interval-overlap crates (superintervals / coitrees / rust-lapper) #208) would swap. Sized at 100 / 2 000 / 20 000 genes.gene_annotation_build— the allocation-heavy half of the same question.All are pure functions with no genome index to build, so a run takes seconds and a regression is attributable to one function rather than to a pipeline.
Sample output (
--sample-count 3, an M-series laptop):The
benchfeaturefind_stopand the GTF record type are private to the crate. Rather than widen the published API for a benchmark, both are exposed under abenchfeature that onlybenches/enables, marked#[doc(hidden)]. Nothing changes for a normal build or for crates.io consumers.The
DEPENDENCIES.mdrow fordivanbelongs with #246, which introduces that file; the rationale above will move there rather than being duplicated.cargo test,cargo clippy --all-targets -- -D warningson a cold cache, andcargo fmt --checkare green.🤖 Generated with Claude Code