Benchmark devirt on real-world dispatch: tantivy 3.5× faster, tracing ~0%#26
Open
Benchmark devirt on real-world dispatch: tantivy 3.5× faster, tracing ~0%#26
Conversation
git-subtree-dir: benchmarks/tracing git-subtree-split: 4fb9ca34f9abb1dc1aaf70203eccee5c74d63635
Benchmarks measuring devirt's impact on the tracing `Subscriber` dispatch pattern. The real `tracing_core::Subscriber` trait dispatches 7 methods through `&dyn Subscriber` on every span/event. This reproduces that exact dispatch shape and measures devirt vs plain vtable. Results on this machine: - Single enabled() call: 0.94ns (devirt) vs 1.60ns (plain) — 1.7x faster - Span lifecycle (3 calls): 1.45ns vs 4.65ns — 3.2x faster - Event pipeline (2 calls): 1.42ns vs 2.81ns — 2.0x faster - Shuffled n=1000 enabled: 1.06µs vs 1.85µs — 1.7x faster - Shuffled n=1000 event: 1.53µs vs 2.91µs — 1.9x faster - Shuffled n=1000 span: 1.54µs vs 4.89µs — 3.2x faster Also adds tokio-rs/tracing as a git subtree under benchmarks/tracing/ for reference, and excludes benchmarks/ from the devirt workspace. https://claude.ai/code/session_017PoPwVjGgzzMWdDnPh5WWR
📝 WalkthroughSummary by CodeRabbit
WalkthroughAdds a benchmarks directory and benchmark crate, excludes benchmarks from the Cargo workspace, adds gitignore and README for benchmarks, and introduces a tantivy-style Criterion benchmark comparing devirtualized vs. vtable trait dispatch. Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Patches the actual tracing-core Subscriber trait with devirt (feature-gated
behind `devirt-bench`) and benchmarks using real `tracing::info!()` and
`tracing::span!()` macros through the full dispatch pipeline.
Real-world results comparing plain vtable vs devirt dispatch:
Benchmark | baseline | devirt | change
-------------------------------|------------|------------|--------
tracing::info!("hello") | 11.70 ns | 11.92 ns | +1.2% (noise)
tracing::info!(x=42, y="t") | 12.18 ns | 12.24 ns | -0.1% (noise)
tracing::debug!() (filtered) | 367.7 ps | 365.1 ps | -3.3%
span create+enter+exit | 31.21 ns | 32.81 ns | +5.1%
nested spans (2 deep) | 63.83 ns | 67.97 ns | +4.1%
span + 3 events | 49.88 ns | 51.01 ns | +5.2%
Key finding: devirt provides no measurable benefit for tracing because
dispatch overhead is a small fraction of total cost. The tracing hot path
is dominated by callsite registration, field construction, and subscriber
bookkeeping — not the vtable call itself. The extra vtable comparison
in the devirt shim adds slight overhead that isn't recouped.
This confirms devirt is most effective when dispatch overhead is
proportionally large (many cheap method calls in tight loops), as shown
by the 1.7-3.2x speedups in the microbenchmarks.
Run instructions:
cd benchmarks/tracing
cargo bench --bench devirt_event # baseline
cargo bench --bench devirt_event --features devirt-bench # with devirt
https://claude.ai/code/session_017PoPwVjGgzzMWdDnPh5WWR
Kab1r
changed the title
Benchmarks reproducing tantivy's per-document scorer dispatch pattern. tantivy MUST use dyn dispatch here — the scorer type is determined by the user's search query parsed at runtime. The hot loop calls scorer.score() (BM25: ~10 arithmetic ops) and scorer.advance() (index + branch) through &mut dyn Scorer on every matching document — potentially millions per query. Results (TermScorer with BM25 scoring): n docs | devirt | plain vtable | Speedup ----------|------------|-------------|-------- 1,000 | 3.1 µs | 9.9 µs | 3.2× 10,000 | 27.8 µs | 98.9 µs | 3.6× 100,000 | 276 µs | 961 µs | 3.5× 1,000,000 | 3.54 ms | 10.3 ms | 2.9× Shuffled 80/20 hot/cold (100 scorers × 1000 docs each): devirt: 333 µs vs plain: 821 µs — 2.5× faster ConstScorer (trivially cheap score()) shows ~0% difference — the method is so cheap LLVM optimizes both paths identically. https://claude.ai/code/session_017PoPwVjGgzzMWdDnPh5WWR
Kab1r
changed the title
The tracing benchmarks showed ~0% improvement because dispatch isn't the bottleneck in tracing's pipeline. The tantivy scorer benchmark demonstrates the real value: 3.5× speedup on mandatory dyn dispatch where the method call IS the work. https://claude.ai/code/session_017PoPwVjGgzzMWdDnPh5WWR
git-subtree-dir: benchmarks/tantivy git-subtree-split: 2e16243f9a9fd15cd147d505c661077a57711d54
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@benchmarks/tantivy-devirt/benches/scorer.rs`:
- Around line 147-174: Move the helper methods for ConstScorer so they appear
before the devirt impl to match TermScorer's ordering: relocate the functions fn
do_advance(&mut self) -> u32 and fn do_doc(&self) -> u32 from their current
position after impl Scorer for ConstScorer to immediately above the
#[devirt::devirt] impl block for Scorer on ConstScorer, preserving their
signatures and #[inline] annotations so the devirt impl's calls to
self.do_advance() and self.do_doc() remain valid and unchanged.
- Around line 31-36: The devirt attribute on the Scorer trait only marks
TermScorer as the hot type (#[devirt::devirt(TermScorer)]) but both TermScorer
and ConstScorer carry #[devirt::devirt] on their impls; decide which behavior
you want and make it consistent: either remove the #[devirt::devirt] attribute
from the ConstScorer impl to keep it as plain dynamic dispatch (leave TermScorer
as the sole hot type), or add ConstScorer to the trait attribute
(#[devirt::devirt(TermScorer, ConstScorer)]) so both are treated as hot; update
the annotations on the Scorer trait and the impls (symbols: Scorer, TermScorer,
ConstScorer) accordingly.
In `@benchmarks/tantivy-devirt/Cargo.toml`:
- Around line 17-19: The [profile.bench] block duplicates workspace settings
(lto and codegen-units); fix by making the benchmark crate a member of the
workspace so it inherits the workspace [profile.bench] and then remove the
duplicated [profile.bench] section (remove the lto and codegen-units keys) from
this crate's Cargo.toml; alternatively, if you must keep the crate excluded from
the workspace, add a short comment above [profile.bench] documenting why it
diverges and keep only the necessary keys to minimize future maintenance.
- Line 11: Update the benchmark crate's dependency entry for criterion to match
the workspace version: change the current line for criterion to use version
"0.8" (e.g. criterion = { version = "0.8", features = ["html_reports"] }), then
run cargo update to refresh the lockfile and verify the benchmark builds and its
HTML reports feature still works.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: da448295-ff08-4bee-acaf-004beec6976a
📒 Files selected for processing (6)
Cargo.tomlbenchmarks/.gitignorebenchmarks/README.mdbenchmarks/tantivy-devirt/Cargo.tomlbenchmarks/tantivy-devirt/benches/scorer.rsbenchmarks/tantivy-devirt/src/lib.rs
Comment on lines
+31
to
+36
| #[devirt::devirt(TermScorer)] | ||
| trait Scorer { | ||
| fn score(&mut self) -> f32; | ||
| fn advance(&mut self) -> u32; | ||
| fn doc(&self) -> u32; | ||
| } |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Consider clarifying hot type specification.
The trait definition specifies only TermScorer as the hot type via #[devirt::devirt(TermScorer)], but later both TermScorer and ConstScorer receive #[devirt::devirt] impl attributes (lines 137-145, 147-155). Since the PR objectives indicate ConstScorer shows ~0% benefit from devirtualization (it's not a hot path), consider whether the ConstScorer impl should use devirt or remain as plain dynamic dispatch.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@benchmarks/tantivy-devirt/benches/scorer.rs` around lines 31 - 36, The devirt
attribute on the Scorer trait only marks TermScorer as the hot type
(#[devirt::devirt(TermScorer)]) but both TermScorer and ConstScorer carry
#[devirt::devirt] on their impls; decide which behavior you want and make it
consistent: either remove the #[devirt::devirt] attribute from the ConstScorer
impl to keep it as plain dynamic dispatch (leave TermScorer as the sole hot
type), or add ConstScorer to the trait attribute (#[devirt::devirt(TermScorer,
ConstScorer)]) so both are treated as hot; update the annotations on the Scorer
trait and the impls (symbols: Scorer, TermScorer, ConstScorer) accordingly.
Comment on lines
+147
to
+174
| #[devirt::devirt] | ||
| impl Scorer for ConstScorer { | ||
| #[inline] | ||
| fn score(&mut self) -> f32 { self.the_score } | ||
| #[inline] | ||
| fn advance(&mut self) -> u32 { self.do_advance() } | ||
| #[inline] | ||
| fn doc(&self) -> u32 { self.do_doc() } | ||
| } | ||
|
|
||
| impl ConstScorer { | ||
| #[inline] | ||
| fn do_advance(&mut self) -> u32 { | ||
| self.cursor += 1; | ||
| if self.cursor >= self.doc_ids.len() { | ||
| return TERMINATED; | ||
| } | ||
| self.doc_ids[self.cursor] | ||
| } | ||
| #[inline] | ||
| fn do_doc(&self) -> u32 { | ||
| if self.cursor >= self.doc_ids.len() { | ||
| TERMINATED | ||
| } else { | ||
| self.doc_ids[self.cursor] | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Inconsistent method definition order.
The ConstScorer devirt impl (lines 147-155) calls self.do_advance() and self.do_doc(), but these methods are defined later (lines 159-173). In contrast, TermScorer defines all its helper methods (lines 77-103) before the devirt impl (lines 137-145).
For consistency and readability, consider moving the ConstScorer helper methods (lines 157-174) to appear before the devirt impl (before line 147).
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@benchmarks/tantivy-devirt/benches/scorer.rs` around lines 147 - 174, Move the
helper methods for ConstScorer so they appear before the devirt impl to match
TermScorer's ordering: relocate the functions fn do_advance(&mut self) -> u32
and fn do_doc(&self) -> u32 from their current position after impl Scorer for
ConstScorer to immediately above the #[devirt::devirt] impl block for Scorer on
ConstScorer, preserving their signatures and #[inline] annotations so the devirt
impl's calls to self.do_advance() and self.do_doc() remain valid and unchanged.
| devirt = { path = "../../crates/core" } | ||
|
|
||
| [dev-dependencies] | ||
| criterion = { version = "0.5", features = ["html_reports"] } |
There was a problem hiding this comment.
Criterion version mismatch with workspace.
The workspace specifies criterion = "0.8" but this benchmark uses "0.5". While this won't cause build errors (since benchmarks are excluded from the workspace), the version inconsistency could lead to confusion, API incompatibilities, or different benchmark behavior.
Consider updating to match the workspace version:
-criterion = { version = "0.5", features = ["html_reports"] }
+criterion = { version = "0.8", features = ["html_reports"] }📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| criterion = { version = "0.5", features = ["html_reports"] } | |
| criterion = { version = "0.8", features = ["html_reports"] } |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@benchmarks/tantivy-devirt/Cargo.toml` at line 11, Update the benchmark
crate's dependency entry for criterion to match the workspace version: change
the current line for criterion to use version "0.8" (e.g. criterion = { version
= "0.8", features = ["html_reports"] }), then run cargo update to refresh the
lockfile and verify the benchmark builds and its HTML reports feature still
works.
Comment on lines
+17
to
+19
| [profile.bench] | ||
| lto = "thin" | ||
| codegen-units = 1 |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Profile configuration duplicates workspace settings.
These benchmark profile settings are identical to the workspace's [profile.bench] (lines 25-27 in root Cargo.toml). Since the benchmarks directory is excluded from the workspace, this duplication is necessary, but it creates a maintenance burden—changes to workspace settings must be manually synchronized here.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@benchmarks/tantivy-devirt/Cargo.toml` around lines 17 - 19, The
[profile.bench] block duplicates workspace settings (lto and codegen-units); fix
by making the benchmark crate a member of the workspace so it inherits the
workspace [profile.bench] and then remove the duplicated [profile.bench] section
(remove the lto and codegen-units keys) from this crate's Cargo.toml;
alternatively, if you must keep the crate excluded from the workspace, add a
short comment above [profile.bench] documenting why it diverges and keep only
the necessary keys to minimize future maintenance.
tantivy already manually devirtualizes its hot paths — TermWeight::for_each calls concrete TermScorer directly (no vtable), and BooleanWeight uses SpecializedScorer::TermUnion with block WAND. The dyn Scorer fallback only runs for uncommon query types (phrase, regex, fuzzy). This is actually a validation of devirt's approach: tantivy's manual specialization (downcast + concrete calls) is exactly what devirt automates via vtable-pointer comparison. The tantivy-devirt microbenchmark is kept — it faithfully reproduces tantivy's fallback dyn Scorer path and shows 3.5× speedup. https://claude.ai/code/session_017PoPwVjGgzzMWdDnPh5WWR
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@benchmarks/README.md`:
- Around line 1-38: Update the benchmarks/README.md to add a new section
summarizing the tracing subscriber benchmark (mentioning the ~0% improvement)
and a subsection reporting the ConstScorer results (explicitly stating
"ConstScorer (very cheap score()) shows ~0% difference"), include brief numeric
or qualitative results and a short explanation why devirt had no effect, and add
a one‑paragraph conclusion synthesizing both findings (e.g., "devirt helps when
dyn calls dominate; no benefit when vtable dispatch is dwarfed by surrounding
work"); reference existing tantivy-devirt content and use the exact names
"tracing subscriber" and "ConstScorer" so readers can correlate with the PR
objectives.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: d962b419-e3b1-45aa-9a0b-edc525aa7d0e
📒 Files selected for processing (1)
benchmarks/README.md
Comment on lines
+1
to
+38
| # Real-World Devirt Benchmarks | ||
|
|
||
| Benchmarks measuring `devirt`'s impact on dispatch patterns from real Rust projects. | ||
|
|
||
| ## tantivy search engine — scorer dispatch (`tantivy-devirt/`) | ||
|
|
||
| Reproduces tantivy's per-document scoring loop. tantivy iterates over | ||
| matching documents calling `scorer.score()` (BM25: ~10 arithmetic ops) | ||
| and `scorer.advance()` through `&mut dyn Scorer` — mandatory dynamic | ||
| dispatch since the scorer type comes from the user's search query parsed | ||
| at runtime. | ||
|
|
||
| Reference: [tantivy `src/query/weight.rs`](https://github.com/quickwit-oss/tantivy/blob/main/src/query/weight.rs) | ||
|
|
||
| Note: tantivy already manually devirtualizes its hottest paths | ||
| (`TermWeight::for_each` calls the concrete `TermScorer` directly). | ||
| The `dyn Scorer` fallback runs for uncommon query types (phrase, regex, | ||
| fuzzy). This benchmark measures the speedup devirt would provide on that | ||
| fallback path, and demonstrates what tantivy achieves manually that devirt | ||
| could automate. | ||
|
|
||
| ### Results (TermScorer with BM25 scoring) | ||
|
|
||
| | n documents | devirt | plain vtable | Speedup | | ||
| |-------------|--------|-------------|---------| | ||
| | 1,000 | 3.1 µs | 9.9 µs | **3.2×** | | ||
| | 10,000 | 27.8 µs | 98.9 µs | **3.6×** | | ||
| | 100,000 | 276 µs | 961 µs | **3.5×** | | ||
| | 1,000,000 | 3.54 ms | 10.3 ms | **2.9×** | | ||
|
|
||
| Shuffled 80/20 hot/cold (100 scorers × 1000 docs): 333 µs vs 821 µs — **2.5×** | ||
|
|
||
| ### Running | ||
|
|
||
| ```bash | ||
| cd benchmarks/tantivy-devirt | ||
| cargo bench | ||
| ``` |
There was a problem hiding this comment.
Document the tracing benchmarks and ConstScorer results mentioned in the PR objectives.
The PR objectives explicitly describe two benchmark case studies:
- tantivy (documented here)
- tracing subscriber showing ~0% improvement (not documented)
Additionally, the PR mentions that "ConstScorer (very cheap score()) shows ~0% difference," which is valuable context missing from the README.
The negative results are just as important as the positive ones—they help users understand when devirt is and isn't beneficial. Consider adding:
- A section documenting the tracing benchmarks and their ~0% improvement
- Discussion of the ConstScorer results showing when devirt doesn't help
- A conclusion section synthesizing both findings: "devirt helps when dyn calls dominate; no benefit when vtable dispatch is dwarfed by surrounding work"
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@benchmarks/README.md` around lines 1 - 38, Update the benchmarks/README.md to
add a new section summarizing the tracing subscriber benchmark (mentioning the
~0% improvement) and a subsection reporting the ConstScorer results (explicitly
stating "ConstScorer (very cheap score()) shows ~0% difference"), include brief
numeric or qualitative results and a short explanation why devirt had no effect,
and add a one‑paragraph conclusion synthesizing both findings (e.g., "devirt
helps when dyn calls dominate; no benefit when vtable dispatch is dwarfed by
surrounding work"); reference existing tantivy-devirt content and use the exact
names "tracing subscriber" and "ConstScorer" so readers can correlate with the
PR objectives.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Benchmarks measuring devirt's impact on dispatch patterns from real Rust projects. Two case studies showing when devirt helps and when it doesn't.
tantivy search engine — 3.5× speedup (mandatory dyn dispatch)
tantivy's per-document scoring calls
scorer.score()(BM25: ~10 arithmetic ops) andscorer.advance()through&mut dyn Scoreron every matching document. This dispatch is mandatory — the scorer type comes from the user's search query parsed at runtime. Cannot be replaced by generics or enums.Shuffled 80/20 (100 scorers × 1000 docs): 333 µs vs 821 µs — 2.5×
tracing subscriber — ~0% change (dispatch not the bottleneck)
Patched the real
tracing_core::Subscribertrait with devirt. Fulltracing::info!()/tracing::span!()pipeline shows no benefit because the vtable call is dwarfed by callsite registration, field construction, and bookkeeping.tracing::info!("hello")When devirt helps vs. when it doesn't
Devirt shines when the
dyn Traitmethod call IS the bottleneck: tight loops dispatching cheap methods (search scoring, per-element processing).Devirt doesn't help when the vtable call is dwarfed by surrounding work (tracing's callsite registration, complex bookkeeping).
Test plan
cd benchmarks/tantivy-devirt && cargo bench— tantivy scorer benchmarkscd benchmarks/tracing-devirt && cargo bench— tracing microbenchmarkscd benchmarks/tracing && cargo bench --bench devirt_event— real tracing baseline + devirthttps://claude.ai/code/session_017PoPwVjGgzzMWdDnPh5WWR