Skip to content

[PR] perf: batch message bodies for storage readers - #6733

Merged
ranxianglei merged 1 commit into
ranxianglei:masterfrom
Afloat16:2026-11-14_join-message-content
Nov 14, 2026
Merged

ranxianglei merged 1 commit into
ranxianglei:masterfrom
Afloat16:2026-11-14_join-message-content

Conversation

@Afloat16

Copy link
Copy Markdown
Contributor

Closes #6732

Problem

Message readers select the requested metadata and then issue one separate content-addressed-store query for every message body. Full JSONL exports and uncovered-history reads pay this cost for thousands of rows. The existing 500-pointer limit bounds SQL arguments but still performs 500 extra body lookups per chunk; the MySQL implementation also serially awaits each lookup.

Change

  • Join content into the already bounded 500-pointer reads in SQLite and MySQL.
  • Keep full-history and uncovered metadata queries unchanged, then fetch their captured content hashes in batches of at most 500 rows. This removes per-row queries without holding every historical BLOB alongside all decoded strings.
  • Preserve selected-row order, missing-content fallbacks, UTF-8 decoding, empty bodies, retired/sequence filters, and both media-hydration formats. All MySQL snapshot reads, including batched CAS and media, remain on the caller's connection.

The bound is on body rows per temporary batch, not on the bytes of one legitimate large message. The existing APIs still return their complete requested output. No storage schema, dependency, setting, or persisted data changes.

Verification

The new module has 18 cases across the SQLite driver variants and MySQL query/connection contract. On unchanged base, six statement-budget regressions fail, five semantic controls pass, and seven optional-driver cases skip. For 1,003 ordinary messages, full reads fall from 1,004 statements to four; pointer reads fall from 1,006 to three. A 501-row uncovered selection falls from 504 to five. Tests also require each body-fetch result to hold at most 500 rows.

The complete focused group passes 79 tests with 16 explicit skips across 13 modules after rebasing onto current master. It covers fields and ordering, retirement/sequence filters, missing and empty CAS, invalid UTF-8, exact Unicode/NUL data, split image/media hydration, public view loading, and snapshot connection ownership. Skips are unavailable optional better-sqlite3 and live-MySQL configuration; the MySQL regression executes production SELECT statements through a typed adapter backed by real SQLite, so it proves the query/connection contract rather than a live MySQL server.

Full npm run typecheck, npm run build, standalone strict checking of the benchmark script, and git diff --check pass. An independent read-only review checked both backend implementations, all new cases and the final bounded-batch design; no blocker was found. The reviewer did not rerun tests.

Reproducible measurement

scripts/bench-message-content-read.ts creates 10,000 real SQLite messages, warms each public operation once, and measures only the operation. Fixture creation and exact output checks are outside the timer. Three alternating baseline/fix pairs on Node 24.22.1/Linux/TZ=UTC produced these elapsed medians:

Public operation Rows Base Change
getMessages 10,000 260.96 ms 78.63 ms
getMessagesAt 5,000 130.76 ms 37.11 ms
uncoveredRows 5,000 140.64 ms 52.08 ms
renderSessionJsonlFromTables 10,000 214.49 ms 132.74 ms

Every operation's complete output has the same SHA-256 in all six runs. Timing varies: full-read base/fix ranges are 203.77–323.18 / 75.71–83.43 ms, and full JSONL ranges are 201.98–242.85 / 104.59–149.06 ms. These are synthetic local SQLite measurements, not a network-MySQL benchmark or a claim about every proxy request. Timing used master 3858b6956; the subsequent a8bbb8d23 update changes none of the storage/export files involved here, and final regressions/types/build were rerun on that new base.

node --import ./tests/preload-test-env.mjs --import ./tests/preload-storage-baseline.mjs --import tsx scripts/bench-message-content-read.ts 10000
node --import ./tests/preload-test-env.mjs --import ./tests/preload-storage-baseline.mjs --import tsx --test --test-concurrency=1 tests/storage-message-content-join.test.ts tests/issue4739-storage-read-snapshot.test.ts tests/storage-backend-contract.test.ts tests/storage-block-content-join.test.ts tests/storage-mysql-seam.test.ts tests/media-cas.test.ts tests/storage-media-sidecar-split.test.ts tests/media-split-digest-parity.test.ts tests/storage-unified.test.ts tests/storage-bounded-load.test.ts tests/d5-5-render-window.test.ts tests/storage-review-p2s.test.ts tests/uncovered-range-bounds.test.ts

History check

Checked the full Afloat16 issue index and actual related PR patches, including a fresh read of #6618's current files. That change optimizes block summary/one readers only. #4765 bounds message pointer queries but retains the per-message CAS reads. Related identity, retirement, archive, and snapshot patches do not batch these message bodies. Focused all-state searches found no previous equivalent change.

Base: master a8bbb8d23108ba9148d09512927041364625d910, bili 0.1.214. Hosted CI remains authoritative for the full suite on the exact pushed head.

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

Dedup check: not a duplicate. No other open [PR] row targets this thread (#6732 / #6733); the related prior work cited in the PR body (#6618 block-summary readers, #4765 pointer-argument bounding) is different scope — neither batches the per-row CAS body lookups.

Labels: pr, enhancement, p2, scope/storage (perf improvement to the storage read path; the underlying issue is measurable slowdown on full exports / uncovered-history reads, not a correctness defect).

Review in progress — I'll post the structured 8-question review once I've read the diff against current master.

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

Review of PR #6733 — perf: batch message bodies for storage readers

Independently verified (not taken from the PR description on faith):

  • Checked out head a2fa9891b5 (base a8bbb8d231) and read the full diff: 5 files, +329/−24.
  • Schema: content.hash is PRIMARY KEY in both backends (src/storage/schema.ts:155, src/storage/mysql.ts:146) → the new LEFT JOIN content is 1:1, no row multiplication.
  • Decoding equivalence: the new batch paths use exactly the old per-row getContent decoding — SQLite Buffer.from(bytes).toString("utf8") (store.ts:3369 vs store.ts:3024), MySQL str(bytes) (mysql.ts:1619 vs mysql.ts:620); missing content → "" on all four reader paths.
  • Tests: new module 18 cases → 11 pass / 7 skip (better-sqlite3 absent here) / 0 fail. PR's 12-module focused group → 77 tests, 68 pass / 9 skip / 0 fail. Combined: 79 pass + 16 skips across 13 modules, matching the PR's claim.
  • npm run typecheck clean on the head.
  • Benchmark A/B, run by me (base a8bbb8d231 vs head, 10,000 rows): output SHA-256 identical for all four operations; wall in my env: getMessages 170.8→107.0 ms, getMessagesAt 80.5→37.5 ms, uncoveredRows 94.7→63.7 ms, jsonlExport 192.2→136.7 ms (1.4–2.1×; the PR's 2.2–3.3× was on a quieter machine).
  • Statement budget probed: 1,003 rows → getMessages = 4 SELECTs (1 metadata + 3×500-row content batches), getMessagesAt = 3 (3 JOIN chunks), 501-row uncovered = 5 (3 gap queries + 2 content batches). Matches the claimed 1004→4 / 1006→3 / 504→5.
  • Snapshot consistency: the SQLite sessionViewRows JOIN runs inside the pinned WAL transaction; MySQL batched CAS + media hydration stay on the caller's snapshot connection (the new test asserts every query's source is connection and the pool query throws on escape).
  • messageRowReads (fix: bound ordinary session loading and avoid full reloads for lazy fields #4739 instrumentation) still increments per materialized row; callers checked (src/export.ts:308, src/persist.ts:2540, shadow verifier/builder 32-seq batches) consume the unchanged API shape.
  • Diff hygiene: git diff --check clean; no version bump, no lockfile churn, no config surface; scripts/ is not in the npm files list, so the benchmark script ships nowhere.

Note (not a blocker): master moved 2 merges past the PR base (a8bbb8d231 → 8964c1fe2d: named-config feature + e2e self-update tests) — no file overlap with this PR, no conflict; rebase before merge is optional.

CI (read live from the mirror cache just now): 0 failed, 13 queued/pending, e2e lanes skipped (gated). The full-suite verdict is still pending on the pushed head.

Verdict: mergeable — pending CI green. It is a perf: change (not a pure fix:), so it stays on the human-merge track per §7.4.


结构化八问(PR #6733)

  1. 是什么问题:存储层消息读取器(getMessages / getMessagesAt / uncoveredRows,SQLite 与 MySQL 双后端)在 500 指针批查询之外,仍对每一行消息单独发一条 CAS 内容查询——1,003 条消息要 1,004 条 SELECT;全量 JSONL 导出与未覆盖历史物化在大会话上付出数千次冗余查询/网络往返。
  2. 什么原因导致的:元数据查询与内容获取是两步分离的设计——[PR] fix: bound ordinary session loading and avoid full reloads for lazy fields #4765 只把指针参数批量化,内容获取仍是逐行 getContent(hash),MySQL 端还串行 await。根因是内容获取没有跟随既有的批处理纪律。
  3. 影响范围与严重程度:所有走统一存储读路径的会话——JSONL 导出、未覆盖历史读取、视图加载、shadow 校验/构建器。特定条件下有问题(大会话/全量导出时延迟放大),小会话无感;性能瑕疵级,非正确性缺陷。
  4. 怎么修复的:两条策略——有界指针读(getMessagesAt/sessionViewRows)把内容 LEFT JOIN 进已有 500 指针批查询(content.hash 是主键,1:1 不放大行数);无界全量/未覆盖读保持元数据查询不变,把捕获的 content_hash 按 ≤500 行分批 IN 查询取回,避免把整段历史的 BLOB 与全部解码字符串同时驻留。
  5. 是否有回归问题:逐项过——行序(ORDER BY seq / 最终 sort)、retired/seqs/fromSeqExclusive 过滤、缺失内容回退 ""、空 BLOB、非法 UTF-8 解码、两种 media 水合形态、MySQL 快照连接归属、fix: bound ordinary session loading and avoid full reloads for lazy fields #4739 messageRowReads 计数——全部保持;本环境 A/B 实测四个公开操作输出 SHA-256 与 base 完全一致。高危模式(语义漂移、fail-fast 变吞错、阈值/时序/输出格式漂移、缓存失效)逐项无命中。唯一行为变化 = 语句数下降(本 PR 的目的本身,已在正文披露)。
  6. 是否破坏缓存:不涉及——纯读路径,无缓存/重试/增量游标/去重键参与。
  7. 是否改了设计方向:不涉及——不新增模块/依赖/配置,不碰 wire 与两种压缩模式;沿用既有 500-chunk 批处理纪律与双后端平行实现风格。
  8. 是否引入了新的配置:不涉及——无新 env/config/CLI/默认值。

判定:可合并(待 CI 全绿;master 已前进 2 个合并但与本 PR 无文件重叠,rebase 可选)。外部提交 ⑤b 清单全项通过;测试有牙(语句预算断言在 base 上必挂);diff 干净,无跑题文件。

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

Correction to the CI line in my review above.

My review's CI statement ("0 failed, 13 queued/pending") was read from the mirror cache, which was showing a label-triggered skip run, not the real matrix result. The mirror label sync (my triage labels) fired a labeled event; ci.yml's plan job treats any non-ci:full label as scope=skip, so that run skipped every test/static/mysql leg and concluded success — shadowing the real red in the cache. (This exact shadowing is documented under #4599, floor 16: read the push-event run, not the labeled run.)

The real core matrix run (push-triggered, run 43599536491, head a2fa9891b5, completed 14:07Z): every job green except one — test (windows-latest, node 22, shard 2):

  • abort x proxy-openai-sse: client abort mid-stream destroys the upstream request → upstream socket(s) must be destroyed after the client abort (closed=0), 21.9s
  • Sibling cells in the same run all passed: anthropic-sse 120ms, responses-sse 75ms, json 413ms

This is a documented Windows load-starvation flake, not a PR regression:

  1. The test file's own comment (tests/wire-exit-matrix.test.ts:31-64) records this exact cell (openai-sse, closed=0) flaking repeatedly on windows-latest node22 shard 2 across multiple master/ci-fixes heads (c78ecb4, 9dc4c94, c37293b, 23a826d) — the failing lane here is that exact lane. The audit noted there concluded the residual latency is runner-side event-loop starvation, not a teardown regression.
  2. The PR's diff touches only the storage readers (src/storage/store.ts, src/storage/mysql.ts). This test runs with storage disabled (SessionStore({ enabled: false }), line 192), so the PR's code is unreachable in this test.
  3. All four abort cells share the same relay→destroy path; if the PR had broken teardown, all four would fail — only the one windows cell did.

Action taken: I re-ran just the failed job (run 43599536491 → rerun-failed-jobs, accepted 201) to get a clean signal.

Adjusted verdict: the code-review verdict in my review above stands unchanged (mergeable; no regression; no cache/config/wire impact — the 8-question checklist there is unaffected). The sole CI red is an environmental Windows flake in a cell the PR's diff cannot reach, not a blocker. If the re-run goes green the PR is fully green and ready to merge; if it flakes again that further confirms the environmental signature.

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

Addendum (local verification): ran tests/wire-exit-matrix.test.ts on the PR head a2fa9891b5 (Linux, node 22) — all 8 tests pass, including the abort x proxy-openai-sse cell that flaked on CI. Confirms the test is not broken by the PR on a healthy platform; the CI red is the Windows-only load-starvation signature, consistent with the re-run now in progress.

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

Final CI verdict: green. The re-run of the failed windows job (run 43599536491, attempt 2) completed success at 16:05Z — every job in the core matrix is now green (only red-signal skipped, which is its normal no-op state). All other lanes (e2e, guards, PR build artifact, windows codex overlay) were already green.

The original red is confirmed to be the environmental Windows load-starvation flake: the same openai-sse cell passed on re-run with no code change. The PR is fully green and ready to merge — the code-review verdict in my review above (mergeable, no regression, no cache/config/wire impact) stands.

@ranxianglei
ranxianglei merged commit becb595 into ranxianglei:master Nov 14, 2026
111 of 148 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Message readers perform one body lookup per row even inside bounded pointer batches

2 participants