bench: replay a rustar-aligner shaped segmented build from a fixture - #14
Merged
rob-p merged 2 commits intoAug 14, 2026
Merged
Conversation
rustar-aligner's `sa_build` is the workload that decides whether a change to this crate is worth having, and it does not look like a flat-text build: N-starting suffixes are never sorted, and the `LimitProvider` stops every comparison at a chromosome or junction-flank boundary. On a chr21-shaped fixture 98.3% of LCP calls resolve inside 16 bytes. Measuring a candidate on raw FASTA instead is how an optimization that regresses this workload gets proposed. This example replays the exact call shape `dispatch_caps_sa_segmented` makes (the spacer-bearing `u8` text verbatim, `SegmentedText::from_ends` over the spacer-run ends, STAR's boundary order, and the ACGT-only streaming filter through `build_ext_mem_for_filter_with`) from a fixture directory holding `text.bin` and `ends.u64`. It reports the caps-sa build wall time, the emitted entry count, and an order-sensitive checksum of the emitted position stream, so an A/B is correctness-gated by construction: identical checksum, identical suffix array. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This was referenced Aug 13, 2026
Open
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a standalone replay of the rustar-aligner-shaped segmented suffix-array build, so caps-sa changes can be screened on the production workload before they become optimization PRs.
This is deliberately a benchmark/documentation PR only: it changes no library API or construction algorithm. PR #15 is stacked on this harness and contains the packed-prefix optimization.
What the harness preserves
examples/rustar_segmented_bench.rsreplays the call shape made by rustar-aligner'ssa_build::dispatch_caps_sa_segmented:2 * n_genomeencodedu8text (A..T = 0..=3,N = 4, spacer= 5), including the annotation-derived junction append;SegmentedText::from_endsover ruSTAR's spacer-run boundaries;spacer-as-largestboundary convention and ascending-position tie-break;build_ext_mem_for_filter_with;u64output positions on production-scale inputs.It reports build wall time, emitted-position count, and a 128-bit order-sensitive checksum. Matching counts and checksums are a strong regression signal, not a mathematical proof of equality; release validation should compare streams exactly or use a direct suffix comparator on a smaller fixture.
Reproducible fixture format
The harness consumes:
text.bin: the encoded text handed to caps-sa;ends.u64: strictly increasing cumulative segment ends in little-endianu64, ending attext.bin's length.bench/README.mdnow documents the precise dump point indispatch_caps_sa_segmented, a minimal dump snippet, the fixture invariants, and the command for a pinned 32-core replay. This keeps the fixture tied to the selected rustar-aligner revision, FASTA, GTF,sjdbOverhang, parsing, junction deduplication, padding, and forward/reverse-complement layout.The CLI now rejects missing option values, unknown flags, zero thread counts, and zero repetitions with a usage message. It also checks that the fixture is non-empty and uses ruSTAR's encoded alphabet.
Why this workload matters
On the original chr21 + GENCODE fixture (95,847,502 symbols, 9,952 segments, 82,167,238 retained suffixes), 972,463,189 observed LCP calls had this result-length distribution:
This is materially different from raw FASTA: N-starting positions are filtered before sorting, while segment limits stop comparisons at chromosome and splice-flank boundaries. Optimizations with per-call overhead therefore need to justify that cost on the overwhelmingly short-comparison common case.
Validation
cargo package --locked --offline --allow-dirtybuilds and verifies the packaged crate.