Skip to content

Faster in-place union of a few containers into many - #908

Merged
lemire merged 1 commit into
masterfrom
or-inplace-few
Oct 15, 2026
Merged

lemire merged 1 commit into
masterfrom
or-inplace-few

Conversation

@lemire

@lemire lemire commented Oct 15, 2026

Copy link
Copy Markdown
Member

roaring_bitmap_or_inplace walked all the containers of its first argument, and merging rebuilt its whole container array, even when the second argument had a single container. The cost was linear in the size of the first bitmap.

When the second argument has fewer than 1/32 as many containers as the first, we now:

  • find each of its keys by galloping;
  • union the matching containers in place;
  • insert the other ones by moving the containers of the first bitmap once, block by block, from the back.

Adding one container to a bitmap with 51,000 containers (Apple M-series, release build):

before after
roaring_bitmap_or_inplace 84 us 4.5 us

The remaining cost is the block move, the same as roaring_bitmap_remove_range removing a container.

This matters when a small bitmap is unioned into a large one repeatedly, for example when overwriting a byte range of a large bitmap with remove_range followed by or_inplace.

The new test test_or_inplace_few_containers checks the result against roaring_bitmap_or over 200 trials:

  • array, bitset, run and full containers;
  • keys before, between, matching and after the existing ones;
  • with and without copy-on-write, including containers shared with a copy.

All 27 test suites pass under ASan/UBSan. Formatting was checked with clang-format 17.

roaring_bitmap_or_inplace walked all the containers of its first argument,
and merging rebuilt its array of containers, even when the second argument
had a single container. When the second argument has fewer than 1/32 as
many containers, we now find each of its keys by galloping, union the
matching containers in place, and insert the other ones by moving the
containers of the first argument once, block by block.

Adding one container to a bitmap of 51,000 containers goes from 84 us
to 4.5 us.
@lemire
lemire merged commit 20997d1 into master Oct 15, 2026
43 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant