Repository navigation
Faster in-place union of a few containers into many - #908
Merged
Merged
Conversation
roaring_bitmap_or_inplace walked all the containers of its first argument, and merging rebuilt its array of containers, even when the second argument had a single container. When the second argument has fewer than 1/32 as many containers, we now find each of its keys by galloping, union the matching containers in place, and insert the other ones by moving the containers of the first argument once, block by block. Adding one container to a bitmap of 51,000 containers goes from 84 us to 4.5 us.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
roaring_bitmap_or_inplacewalked all the containers of its first argument, and merging rebuilt its whole container array, even when the second argument had a single container. The cost was linear in the size of the first bitmap.When the second argument has fewer than 1/32 as many containers as the first, we now:
Adding one container to a bitmap with 51,000 containers (Apple M-series, release build):
roaring_bitmap_or_inplaceThe remaining cost is the block move, the same as
roaring_bitmap_remove_rangeremoving a container.This matters when a small bitmap is unioned into a large one repeatedly, for example when overwriting a byte range of a large bitmap with
remove_rangefollowed byor_inplace.The new test
test_or_inplace_few_containerschecks the result againstroaring_bitmap_orover 200 trials:All 27 test suites pass under ASan/UBSan. Formatting was checked with clang-format 17.