Prioritize L0 compaction with bounded slices - #35
Draft
byeongsu-hong wants to merge 11 commits into
Draft
Conversation
Older MANIFEST files were swept only in open(); a long-running process rotates a manifest every few minutes and never reopens, so stale generations accumulate without bound — measured 441 files / 441 MB in one production store after 31 hours of uptime (~14 GB across its 32 stores, ~10% of the volume). save() now sweeps generations below the one it just flipped CURRENT to. Best-effort by design: a crash mid-sweep leaves work for the next save, and newer generations (pre-flip crash artifacts) are deliberately left for the open-time sweep, which already handles them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
fix(manifest): prune stale generations at save, not only at open
…entry allocs The pinned index held one heap Vec<u8> per block entry; against ~40-byte keys the Vec header + malloc header + size-class rounding roughly doubled resident bytes, and a 148 GB production index pays that across ~30 M entries (~3 GB observed). One concatenated keys arena + u32 ends + a BlockRef vector keeps the same binary search over the same keys with no per-entry allocations. Existing multi-block tests (256-byte blocks, 500 keys, every key looked up, forward/reverse iteration and seeks) cover the lookup equivalence. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
perf(table): flat-arena block index — half the pinned heap
…e heap Every open table pinned its whole bloom filter for life. At 10 bits/key that is the single largest pinned population on a big store — measured 2.76 GB across one production deployment's 2,481 tables — and it grows with data size, not load, so no cache budget ever governed it. The filter now loads through the shared block cache under its natural (file_id, filter.off) key: hot filters stay resident by being used, cold tables cost nothing, and block_cache_size finally bounds every block the reader touches. may_contain_ukey becomes fallible (a cache miss re-reads the block); the two non-reader callers — the run-level check and the compaction tombstone-drop predicate — propagate the error instead of swallowing it. Corruption still surfaces at open via a verify-and-drop read. Quick bench probe (single run, examples/bench): cold gets -6.5% (the added cache probe per lookup), everything else within noise. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
perf(table): bloom filters ride the block cache instead of pinning the heap
Compression grows a Zstd variant (codec byte 2, level fixed at 3 —
measured on real store data, level 8 buys ~0.5% for several times the
CPU) and Options grows bottom_compression: the codec for compaction
outputs landing in the deepest level, where ~90% of a store's bytes
live and rewrites are rarest. None means "same as compression", so
nothing changes without opting in.
Measured on two production tables (154 MB raw): zstd-3 stores 33% of
raw at 32 KiB blocks vs lz4's 41% — about a quarter less disk — at
2.3 GB/s single-thread decompress, which is noise on the read path.
Tables carrying a zstd block bump to format 3, so pre-zstd readers
reject them at open ("unsupported table format") instead of failing
mid-read on a codec byte. Per-block store-raw-if-not-smaller behavior
matches lz4, and the zstd payload is size-prefixed like lz4's.
The placement flag is wired at all three Job constructions and proven
through the production compaction path: with a 2-level tree every
output is the bottom, so the test asserts format-3 tables appear with
bottom_compression set and never without it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
feat(table): zstd codec + bottom-level compression placement
Land the production line: sorted bulk loading + four storage fixes
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
compact_allthrough the same schedulerWhy
A long deep-level rewrite held the only compaction executor until completion. L0 could reach its stall threshold during that window, blocking writers even though automatic compaction was running.
Impact
The default 1 MiB slice reduced median writer stall from 278.2 ms to 4.73 ms in a five-run 256 MiB L1-to-L2 compaction canary. Median total drain time was 291 ms before and 288 ms after; median peak RSS was 25.51 MiB before and 25.63 MiB after.
Jobs still publish atomically, no on-disk format changes, and no concurrent compaction workers are introduced.
Checks
cargo test --workspacecargo test -p fluent31 --lib(73 passed)cargo test --workspace --no-run