Skip to content

[PR] fix: route oversized kv values to the content CAS instead of throwing (#7378) - #7389

Open
ranxianglei wants to merge 1 commit into
masterfrom
2026-11-20_kv-cas-routing
Open

ranxianglei wants to merge 1 commit into
masterfrom
2026-11-20_kv-cas-routing

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Summary

Fixes #7378 — the session-order skeleton (179 entries, ~4-8 MiB) exceeded the 4 MiB inline extension-value cap (EXT_VALUE_MAX_BYTES), so serializeExtValue threw on every start and the hydrate fell back to the legacy file path for the process lifetime. The error message itself said "store large payloads in the content CAS instead" — this PR implements that.

What changed

setKv now routes values over the inline cap to the content CAS: the kv row stores a small {"$biliCas": "<sha256>"} ref and the payload lives in the refcount-managed content table.

  • sqlite (UnifiedStore) and MySQL backends both get the routing.
  • getKv/listKv resolve the ref transparently (a missing CAS blob falls back to the ref with a warning — never throws, so one corrupt row can't break the whole hydrate).
  • deleteKv releases the ref before deleting so the blob is reclaimed (refcount balanced, delete-on-zero → no leak).
  • setSessionExt is unchanged (still inline, still throws over the cap) — the issue scope is kv_store.

⚠️ Behavior change

setKv no longer throws for over-cap values — it stores a CAS ref instead. Old: throw "extension value exceeds 4194304 bytes". New: store a small CAS ref + the payload in the content table. This is the intended fix (the error message was the TODO). setSessionExt is unaffected.

Tests

  • New tests/kv-cas-routing.test.ts (7 tests): oversized setKv no-throw + CAS routing, byte-exact round-trip, listKv resolution, small-value inline, overwrite refcount balance, deleteKv ref release, whole-namespace delete.
  • tests/order-unified-store.test.ts: new test — oversized skeleton (>4 MiB) migrates into the kv via the CAS and survives a restart.
  • Reworked tests/extension-storage-contract.test.ts + tests/storage-ext.test.ts to assert the new behavior (setKv routes, setSessionExt still rejects).

GC / leak safety

The content CAS gc() only sweeps refcount <= 0 rows; putContent bumps, releaseContent drops + deletes-on-zero. So a live CAS ref is GC-safe and a released ref is reclaimed — no leak.

Closes #7378

…#7378)

The session-order skeleton (179 entries, ~4-8 MiB) exceeded the 4 MiB inline
extension-value cap (EXT_VALUE_MAX_BYTES), so serializeExtValue threw on every
start and the hydrate fell back to the legacy file path for the process
lifetime. The error message itself said "store large payloads in the content
CAS instead" — this fix implements that.

setKv now routes values over the inline cap to the content CAS: the kv row
stores a small {"$biliCas": "<sha256>"} ref and the payload lives in the
refcount-managed content table (GC-safe: gc only sweeps refcount<=0;
delete-on-zero keeps it leak-free). getKv/listKv resolve the ref transparently;
deleteKv releases the ref so the blob is reclaimed. Both the sqlite
(UnifiedStore) and MySQL backends.

Behavior change: setKv no longer throws for over-cap values — it stores a CAS
ref. setSessionExt is unchanged (still inline, still throws over the cap).

Fixes #7378
@ranxianglei ranxianglei added bug Something isn't working p2 scope/storage labels Nov 20, 2026
@ranxianglei ranxianglei changed the title fix: route oversized kv values to the content CAS instead of throwing (#7378) [PR] fix: route oversized kv values to the content CAS instead of throwing (#7378) Nov 20, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

查重: 不重复 — Only this PR (#7389 / mirror #7390) targets #7378; no open parallel PR fixes the same thread.

Labels: Source issue #7378 and PR #7389 are already labeled (bug, p2, scope/storage) — pickup labeling confirmed, no changes needed.

Review in progress. I've read the full diff and traced the mechanism end-to-end:

  • setKv routes values over the 4 MiB inline cap (EXT_VALUE_MAX_BYTES) to the content CAS; the kv row keeps a small {"$biliCas": "<sha256>"} ref. getKv/listKv resolve the ref transparently; deleteKv releases the ref before deleting; setSessionExt is unchanged (still inline, still throws over the cap) — correct, the issue scope is kv_store.
  • Refcount discipline is sound: putContent bumps (1 for new / +1 for existing), releaseContent drops with a refcount > 0 floor + delete-on-zero. Overwrite (different payload → old blob reclaimed; same payload → net 0) and delete are balanced in both the SQLite and MySQL backends.
  • Orphan-fallback is graceful: when the CAS blob is missing, resolveKvValue returns the ref object with a warning (never throws). The session-order consumer parseStoreDoc handles it by returning null → empty skeleton, and the one-time migration write still re-merges legacy-file entries — so one corrupt row can't break the whole hydrate.

CI: the test shards are still in progress on the pushed head. I'll post the full 8-question review once the lanes are green.

Minor observations (non-blocking):

  1. MySQL setKv is not wrapped in a transaction (the SQLite twin is). A crash mid-sequence could leak a refcount (slow leak, not a correctness issue — the refcount > 0 floor prevents negatives). This is a pre-existing pattern in the MySQL backend, not a regression introduced here.
  2. casRefOf's doc comment slightly overstates the guard: a user value that exactly matches the shape (a plain object with a $biliCas key whose value is 64-char lowercase hex) would be misread as a ref. The residual collision risk is negligible.

Full 8-question review follows once CI is green.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

八问

  1. 是什么问题:session-order skeleton(179 条目,~4–8 MiB)超过 4 MiB 的 inline extension-value 上限(EXT_VALUE_MAX_BYTES),serializeExtValue 每次启动都抛错,hydrate 回退到 legacy 文件路径并持续整个进程生命周期。本 PR 把超上限的 kv 值路由到 content CAS,不再抛错。

  2. 什么原因导致的:serializeExtValue(src/storage/store.ts:375-384)在 JSON 字节数 > EXT_VALUE_MAX_BYTES(4194304)时 throw。session-order skeleton 超过该上限,导致每次启动都抛错、hydrate 回退 legacy。错误信息本身就写着 "store large payloads in the content CAS instead"——本 PR 实现了这个 TODO。

  3. 影响范围与严重程度:受影响的 session(skeleton > 4 MiB)在启动时无法从统一存储 hydrate session-order skeleton,回退到 legacy 文件路径;persona resolution(Compression fails in long sessions while new sessions work fine | 压缩失败.新会话反而没事 #3397)可能无法在重启后存活。能用但有明显问题(p2)——session 仍可用,只是部分持久化路径降级。

  4. 怎么修复的:setKv 把超过 cap 的值路由到 content CAS——kv 行存一个小的 {"$biliCas": "<sha256>"} ref,payload 存在 refcount 管理的 content 表。getKv/listKv 透明解析 ref;deleteKv 先释放 ref 再删除(refcount 平衡,delete-on-zero 不泄漏);setSessionExt 不变(仍 inline,仍抛错)——issue 范围是 kv_store,正确。SQLite 和 MySQL 两个后端都实现了路由。

  5. 是否有回归问题:逐项审过——

    • 行为变更(已披露):setKv 不再对超 cap 值抛错(旧:throw "extension value exceeds 4194304 bytes";新:存 CAS ref)。这是有意的修复,不是回归。
    • setSessionExt 不变(仍抛错)——正确,scope 是 kv_store。
    • refcount 纪律:putContent bump(新 1 / 已有 +1),releaseContent drop(floor 0)+ delete-on-zero。overwrite(不同 payload → 旧 blob 回收;相同 payload → 净 0)和 delete 在两个后端都平衡。
    • orphan fallback:CAS blob 缺失时 resolveKvValue 返回 ref 对象 + 警告(不抛错);session-order 的 parseStoreDoc(order-persist.ts:91)对 ref 对象返回 null → 空 skeleton,migration write 仍重新合并 legacy 文件——一行坏数据不会拖垮整个 hydrate。
    • 无 fail-fast 变吞错、无阈值/时序/输出格式漂移、无缓存/重试策略静默失效。
    • 高危模式逐项过:无。
  6. 是否破坏缓存:不涉及。content CAS 是 refcount 管理的独立表,不影响 prompt cache 或任何缓存失效策略。

  7. 是否改了设计方向:没有。只是把超 cap 的 kv 值路由到已有的 content CAS 机制(putContent/releaseContent/getContent 都是现成的),没有改变架构/边界/数据流。

  8. 是否引入新配置:没有。

CI 状态与结论

  • 21 个 test 分片全绿(ubuntu/windows × node 22/24 × shards),2 个 skipped,1 个 FAILURE。
  • FAILURE = build-artifact:这是 npm publish 404(404 Not Found - PUT https://registry.npmjs.org/billion-context),基础设施瞬态问题,非代码问题——build 本身成功(tarball 已生成,11.5 MB),失败在 publish 步骤。同一 job 在 2026-11-19 合并的 PR [PR] kernel: move illegally-positioned directive-only systems to the trailing run (#7270) #7281 上是绿的,本次 404 是 npm registry 瞬态拒绝。
  • 判定:代码变更 sound,refcount 纪律正确,orphan fallback 优雅,test 全绿。build-artifact 红条是 npm publish 瞬态 404,与 PR 代码无关,重触发即可转绿。ready-to-merge(等 build-artifact 重触发转绿,或 owner 判定忽略该瞬态红条)。

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working p2 scope/storage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

session-order skeleton exceeds 4 MiB extension value limit — hydrate fails, falls back to legacy file path

1 participant