Repository navigation
perf: check cluster routing with one read of the state - #453
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cluster mode served about 320k requests a second whether or not the client pipelined, against 870k (no pipeline) and 2.7M (pipeline of 16) standalone. Every command locked the migration state, a tokio mutex shared by every connection, and read the cluster state's async RwLock up to three times.
std::sync::RwLock. no holder awaits while holding it, and the per-command check only reads itClusterCoordinator::check_commandmakes the whole decision from one read of the cluster state: paused writes, replica redirects, the majority fence, CROSSSLOT and the slot check. it replacesis_replica/primary_addr_for_slot/reaches_majority_of_primaries/check_slot_with_migration/check_crosssloton the hot pathcheck_command, and a new one checks that a replica sends writes to its primary with MOVED and serves readsOne-node cluster owning every slot against standalone, 50 clients, 1M requests, 32 cores:
With a pipeline of 16 cluster mode is still about 15% below standalone, from the one shared read lock each command takes.