Repository navigation
[PR] perf(storage): index session GC companion suffix lookups - #6611
Merged
ranxianglei merged 1 commit intoNov 14, 2026
Merged
Conversation
Owner
|
Superseded by #6677 (merged as Landed into master via the real merge commit The fix for #6609 is now in master. Closing this PR as superseded — thanks for the careful, well-tested fix. |
This was referenced Nov 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #6609
Problem
The content-store co-deletion guard and orphan sweep scan all session basenames for each companion. A directory with 5,000 live sessions and 5,000 old but still-referenced companions therefore performs millions of repeated basename/suffix checks on every GC sweep, despite retaining every file.
Change
Build a per-sweep reversed-basename index lazily when a companion actually needs matching. Binary search finds the existing arbitrary suffix matches; matching paths are restored to the original filesystem-walk order before the existing twin checks run. This preserves legacy names and cross-namespace fallback without assuming a hash length or an underscore separator.
The index is built once in O(N log N) time and uses O(N) entries plus reversed filename strings. A lookup costs O(log N + K log K) for K matches, including restoration of probe order. Sweeps that never query companions do not build the index. Existing deleted-session filtering, readable-own-store checks, age/token gates, resident-session protection, and deletion/audit behavior remain in place.
Verification
BILI_STORAGE_JSON_WRITER=1 BILI_PERSIST_ZSTD=0to exercise the file-persistence contract expected by that suite.git diff --checkpass.Reproducible measurements
The included benchmark creates real session and content-store files, warms up once, and times complete
gcSessionFilescalls. Setup and SHA-256 verification are outside the timed region. Every filename and file-content byte is hashed before and after the sweeps, and the full GC result is checked.Seven fresh-process baseline/candidate pairs per workload, alternating order, on Node 24.22.1 / Linux x86_64:
The large companion case improves in all seven pairs (7.81–15.65× paired wall-time ratio). Median CPU time is 12,390.16 → 1,006.53 ms; wall-time ranges are 11,706.59–14,870.34 ms and 842.60–1,499.01 ms. All fourteen input/output hashes match within each workload, with every expected path retained.
Both control medians increase. Their ranges overlap; the no-companion candidate is slower in six of seven pairs, while the small control has mixed results. These controls do not establish a speedup or a no-regression guarantee. The measured benefit is specifically the many-companion matching workload. This is not a claim about general request latency, all GC layouts, or overall memory use.
Run the same driver on baseline
e1f31ad9d5b47500362699a9867819b861c7de96and the candidate. No dependency, configuration surface, schema, or kernel changes.