Repository navigation
Negate sparse arrays into run containers; SIMD in-place array union - #906
Merged
Merged
Conversation
When the first array has room for the result, array_array_container_inplace_union merged it in place with the scalar union_uint16, since the vectorized union cannot write over its own input. Copy the first array to a stack buffer instead and use fast_union_uint16. Repeated in-place unions (or_inplace over several bitmaps) reach this path: four-way OR of sparse bitmaps (1% density) gets about twice as fast.
When an array container is negated over a range large enough that the result exceeds 4096 values, the result used to be a bitset (8 KB). The negation has at most cardinality + 1 runs, so when these runs take less room than a bitset we now produce a run container instead. This makes flipping sparse bitmaps much faster and the result much smaller.
container_not and container_inot, used when a flip covers whole chunks, always produced a bitset from an array container. They now go through array_container_negation_range_to_any like the partial flips.
The runs of the negation are the gaps between the values in the range and the values themselves outside of it, so we write them in one pass instead of converting the array to runs and negating those. Flipping 100M values at 1% density goes from 4.1 ms to 0.8 ms (1.3 ms with bitsets before). The test now also covers values that touch the ends of the range.
When the negation of a whole array container is a bitset, build it in one pass (set all bits, clear the values) as before, instead of copying the array into a bitset and flipping it.
lemire
added a commit
to lemire/valkey
that referenced
this pull request
Oct 15, 2026
Update CRoaring to 5.2.4 with two pending changes (RoaringBitmap/CRoaring#906 and valkey-io#907): sparse array containers are negated into run containers, and roaring_bitmap_range_to_words() writes a range of a bitmap as 64-bit words. GET, GETRANGE, BITFIELD and BITPOS 0 now read the bitmap as words instead of iterating over the values or parsing the serialization. On a 100M-bit string with half of the bits set, GETRANGE of 1 KB goes from 22 us to 0.45 us and BITFIELD GET from 0.42 us to 0.22 us. BITOP NOT on sparse bitmaps produces run containers: with 100k random 32-bit offsets it goes from 101 ms to 8 ms. BITOP NOT no longer run-optimizes its result, BITOP OR and XOR over many keys combine the inputs pairwise when the result stays sparse, and BITFIELD does not write back a window that did not change. Signed-off-by: Daniel Lemire <daniel@lemire.me>
lemire
added a commit
to lemire/valkey
that referenced
this pull request
Oct 16, 2026
Vendor the CRoaring development branch after 5.2.4, which includes RoaringBitmap/CRoaring#906 and valkey-io#908, plus valkey-io#907, where roaring_bitmap_range_to_words() is now named roaring_bitmap_word_range_to_bitset(). Signed-off-by: Daniel Lemire <daniel@lemire.me>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two small performance changes.
Negating sparse array containers. When an array container is negated over a range large enough that the result has more than 4096 values,
container_not_rangeandcontainer_inot_rangeproduced a bitset container (8 KB). The negation of an array has at most cardinality + 1 runs, so when those runs take less room than a bitset (cardinality up to 2046), the result is now a run container. This applies to partial flips (container_not_range,container_inot_range) and to whole-container flips (container_not,container_inot). The runs are written in one pass: in the range, they are the gaps between the values; outside, the values themselves.Flipping 100M values (
roaring_bitmap_flip(r, 0, 100000000)), Apple M-series,-O3:The result is also smaller (about 2.6 KB instead of 8 KB per container at 1%). The new function is
array_container_negation_range_to_any;array_container_negation_rangeis unchanged. Test:test_flip_sparse_array_containerintoplevel_unit.c(copy and in-place flips, several cardinalities and ranges, values touching the ends of the range, checked value by value).In-place array union with spare capacity.
array_array_container_inplace_union, when the first array has enough capacity, used amemmovefollowed by the scalarunion_uint16. It now copies the first array to a stack buffer and usesfast_union_uint16(SIMD when available). Test:array_array_inplace_union_spare_capacity_testinmixed_container_unit.c.All tests pass with sanitizers.