Repository navigation
[PR] perf: batch message bodies for storage readers - #6733
Conversation
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) Dedup check: not a duplicate. No other open [PR] row targets this thread (#6732 / #6733); the related prior work cited in the PR body (#6618 block-summary readers, #4765 pointer-argument bounding) is different scope — neither batches the per-row CAS body lookups. Labels: pr, enhancement, p2, scope/storage (perf improvement to the storage read path; the underlying issue is measurable slowdown on full exports / uncovered-history reads, not a correctness defect). Review in progress — I'll post the structured 8-question review once I've read the diff against current master. |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) Review of PR #6733 — perf: batch message bodies for storage readers Independently verified (not taken from the PR description on faith):
Note (not a blocker): master moved 2 merges past the PR base ( CI (read live from the mirror cache just now): 0 failed, 13 queued/pending, e2e lanes skipped (gated). The full-suite verdict is still pending on the pushed head. Verdict: mergeable — pending CI green. It is a 结构化八问(PR #6733)
判定:可合并(待 CI 全绿;master 已前进 2 个合并但与本 PR 无文件重叠,rebase 可选)。外部提交 ⑤b 清单全项通过;测试有牙(语句预算断言在 base 上必挂);diff 干净,无跑题文件。 |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) Correction to the CI line in my review above. My review's CI statement ("0 failed, 13 queued/pending") was read from the mirror cache, which was showing a label-triggered skip run, not the real matrix result. The mirror label sync (my triage labels) fired a The real core matrix run (push-triggered, run 43599536491, head
This is a documented Windows load-starvation flake, not a PR regression:
Action taken: I re-ran just the failed job (run 43599536491 → Adjusted verdict: the code-review verdict in my review above stands unchanged (mergeable; no regression; no cache/config/wire impact — the 8-question checklist there is unaffected). The sole CI red is an environmental Windows flake in a cell the PR's diff cannot reach, not a blocker. If the re-run goes green the PR is fully green and ready to merge; if it flakes again that further confirms the environmental signature. |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) Addendum (local verification): ran |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) Final CI verdict: green. The re-run of the failed windows job (run 43599536491, attempt 2) completed success at 16:05Z — every job in the core matrix is now green (only The original red is confirmed to be the environmental Windows load-starvation flake: the same |
Closes #6732
Problem
Message readers select the requested metadata and then issue one separate content-addressed-store query for every message body. Full JSONL exports and uncovered-history reads pay this cost for thousands of rows. The existing 500-pointer limit bounds SQL arguments but still performs 500 extra body lookups per chunk; the MySQL implementation also serially awaits each lookup.
Change
The bound is on body rows per temporary batch, not on the bytes of one legitimate large message. The existing APIs still return their complete requested output. No storage schema, dependency, setting, or persisted data changes.
Verification
The new module has 18 cases across the SQLite driver variants and MySQL query/connection contract. On unchanged base, six statement-budget regressions fail, five semantic controls pass, and seven optional-driver cases skip. For 1,003 ordinary messages, full reads fall from 1,004 statements to four; pointer reads fall from 1,006 to three. A 501-row uncovered selection falls from 504 to five. Tests also require each body-fetch result to hold at most 500 rows.
The complete focused group passes 79 tests with 16 explicit skips across 13 modules after rebasing onto current master. It covers fields and ordering, retirement/sequence filters, missing and empty CAS, invalid UTF-8, exact Unicode/NUL data, split image/media hydration, public view loading, and snapshot connection ownership. Skips are unavailable optional better-sqlite3 and live-MySQL configuration; the MySQL regression executes production SELECT statements through a typed adapter backed by real SQLite, so it proves the query/connection contract rather than a live MySQL server.
Full
npm run typecheck,npm run build, standalone strict checking of the benchmark script, andgit diff --checkpass. An independent read-only review checked both backend implementations, all new cases and the final bounded-batch design; no blocker was found. The reviewer did not rerun tests.Reproducible measurement
scripts/bench-message-content-read.tscreates 10,000 real SQLite messages, warms each public operation once, and measures only the operation. Fixture creation and exact output checks are outside the timer. Three alternating baseline/fix pairs on Node 24.22.1/Linux/TZ=UTC produced these elapsed medians:Every operation's complete output has the same SHA-256 in all six runs. Timing varies: full-read base/fix ranges are 203.77–323.18 / 75.71–83.43 ms, and full JSONL ranges are 201.98–242.85 / 104.59–149.06 ms. These are synthetic local SQLite measurements, not a network-MySQL benchmark or a claim about every proxy request. Timing used master
3858b6956; the subsequenta8bbb8d23update changes none of the storage/export files involved here, and final regressions/types/build were rerun on that new base.History check
Checked the full Afloat16 issue index and actual related PR patches, including a fresh read of #6618's current files. That change optimizes block summary/one readers only. #4765 bounds message pointer queries but retains the per-message CAS reads. Related identity, retirement, archive, and snapshot patches do not batch these message bodies. Focused all-state searches found no previous equivalent change.
Base: master
a8bbb8d23108ba9148d09512927041364625d910, bili 0.1.214. Hosted CI remains authoritative for the full suite on the exact pushed head.