test: worker correctness tests + hand-crafted engine vs ArroyoSketch comparison#232
Merged
milindsrivastava1997 merged 6 commits intomainfrom Apr 7, 2026
Merged
Conversation
zzylol
force-pushed
the
pr/D-benchmarks-scalability
branch
from
4c740c5 to
a145bd7
Compare
zzylol
force-pushed
the
pr/E-correctness-tests-comparison
branch
from
8c04bc5 to
47d279e
Compare
zzylol
force-pushed
the
pr/D-benchmarks-scalability
branch
from
a145bd7 to
1648f1e
Compare
zzylol
changed the title
zzylol
force-pushed
the
pr/E-correctness-tests-comparison
branch
from
60d90f6 to
443c7e0
Compare
…comparison - Add CapturingOutputSink test utility for capturing precomputed outputs in-memory - Add worker correctness unit tests covering Sum and KLL accumulator pipelines - Add comparison test asserting precompute engine matches ArroyoSketch results - Add test that loads aggregation config from real streaming_config YAML - Expand design doc with sliding window behaviour, accumulator lifecycle, load balancing trade-offs, and fault tolerance sections Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
zzylol
force-pushed
the
pr/E-correctness-tests-comparison
branch
from
443c7e0 to
31906ef
Compare
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Revert broken streaming_engine conditional in precomputed_output.rs (variable doesn't exist in scope) - Replace .inner.sketch.get_n() with .inner.count() in KLL comparison test (KllSketch no longer has a .sketch field) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- create_precompute_from_bytes is only called from the arroyo deserialization path, so use deserialize_from_bytes_arroyo for MultipleSumAccumulator - Inline short assert_eq for KLL count comparison Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…uild The sketchlib-rust repo (ProjectASAP/sketchlib-rust) renamed its package from "sketchlib-rust" to "asap_sketchlib" on main. Without a pinned rev, Cargo fails to find the package by name. Pin to the same rev (4404274) used by asap-query-engine/Cargo.toml where the package is still named "sketchlib-rust". Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
milindsrivastava1997
approved these changes
Apr 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
CapturingOutputSinktest utility that captures precomputed outputs in-memorystreaming_configYAMLThis is PR E of 6 stacked PRs splitting #162
Stacking order:
Test plan
cargo test -p query_engine_rust precomputepasses all worker teststest_precompute_engine_matches_arroyo_sum,test_precompute_engine_matches_arroyo_kll) passes🤖 Generated with Claude Code