Skip to content

[PR] fix(session-order): keep newest skeletons within each carrier's byte budget - #6477

Open
ranxianglei wants to merge 1 commit into
masterfrom
2026-11-13_session-order-snapshot-capacity
Open

ranxianglei wants to merge 1 commit into
masterfrom
2026-11-13_session-order-snapshot-capacity

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

What & why

The session-order snapshot producer (src/order-persist.ts) had two capacity defects in the same write path:

  1. Byte budget dropped the newest conversations. normalizeForWrite() only sorted by recency when the entry count exceeded 1,024. Below that — the common case — the union stayed in array (write) order, so the 8 MiB byte cap kept the oldest records and silently dropped the newest ones. After a restart those conversations have no persisted skeleton (the resolver re-runs the cold-start race, Compression fails in long sessions while new sessions work fine | 压缩失败.新会话反而没事 #3397).

  2. 8 MiB producer budget vs 4 MiB store ceiling. The same producer used its 8 MiB legacy-file budget for unified KV writes, but the store rejects extension values over 4 MiB (EXT_VALUE_MAX_BYTES). A snapshot in the 4–8 MiB band made the one-time migration (hydrateOrderUnified) throw, demoted the process to the legacy file path for the whole run, and raised per-flush write volume (the file is plaintext JSON).

Fix

One bounded-snapshot producer fix — no new config, schema, or fragment truncation:

  • sort the union by recency before both caps (count and byte);
  • measure serialized UTF-8 bytes (the on-disk size), not UTF-16 units;
  • use the store's 4 MiB ceiling for KV migration and steady-state writes;
  • keep the legacy file's 8 MiB ceiling and the single-oversized-record fallback (the first entry is always kept).

EXT_VALUE_MAX_BYTES is now exported from src/storage/store.ts so the producer and the store share one ceiling constant.

Behavior change (old → new)

  • KV snapshot cap: 8 MiB → 4 MiB (the store ceiling the producer previously ignored; now respected so the KV write can't throw).
  • Byte-budget eviction order: array/write order → recency (newest kept).
  • Byte measurement: UTF-16 units → UTF-8 bytes (stricter — keeps slightly fewer entries for non-ASCII content, never throws).

Validation

  • Targeted: tests/order-persist.test.ts + tests/order-unified-store.test.ts → 10/10 pass (3 new regression tests pin both facets, each in both directions).
  • npm run typecheck clean; npm run build green.
  • Full-suite verdict is the remote CI test lanes on this head.

Duplicate screening: no open/closed issue or PR makes this snapshot producer respect its carrier's size limit or keep recent entries under it. #3405 introduced the producer, #5757 migrated it to KV, #6186 fixes the write lifecycle, #6203 fixes generic extension validation — none touch the capacity ordering/ceiling.

Reported and proposed by @Afloat16 (issue #6255); this is a clean rebase of that fix onto current master with the regression tests.

Fixes #6255

…budget (#6255)

The session-order snapshot producer (src/order-persist.ts) applied its byte
budget in array (write) order, so with fewer than 1,024 entries — where the
recency sort was skipped — the byte cap kept the OLDEST records and silently
dropped the newest conversations. It also used its 8 MiB legacy-file budget
for unified KV writes, but the store rejects extension values over 4 MiB, so
a large snapshot made the one-time migration throw, demoted the process to
the legacy file path for the whole run, and raised per-flush write volume.

Fix: sort the union by recency before BOTH caps, measure serialized UTF-8
bytes, and use the store's 4 MiB ceiling for KV migration and steady-state
writes (the legacy file keeps its 8 MiB ceiling). No new config, schema, or
fragment truncation.

Closes #6255
@ranxianglei ranxianglei changed the title fix(session-order): keep newest skeletons within each carrier's byte budget [PR] fix(session-order): keep newest skeletons within each carrier's byte budget Nov 13, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-11-13_session-order-snapshot-capacity (f1dcb9e)

Latest build — install this exact version

npm install -g billion-context@0.0.9422-pr.6477.968e203

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
f1dcb9e billion-context@0.0.9422-pr.6477.968e203 build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr6477.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

查重: 不重复 — 检索 open+closed PR/issue: 唯一针对 #6255 的在途 PR 就是本 PR #6477;Afloat16 的前置工作 #6186(写生命周期:await unified snapshot writes)已 closed 且修的是另一面,本 PR 是 #6255 修复向当前 master 的干净 rebase,无并行第二 PR 修同一 thread。

标签: bug + scope/storage + p2(会话顺序快照的容量/预算缺陷,属持久化域;能用但有明确问题——丢最新骨架 + KV 写失败降级,未到数据丢失/不可用级,故 p2 非 p1;无特定客户端绑定,不打客户端标签)。

这是带 ework-agent-pr 标记的 [PR] issue,按流程不重做,改为:拉取分支核对代码 + 确认 CI 状态 + 报告结论。正在审。

@ranxianglei ranxianglei added bug Something isn't working p2 scope/storage labels Nov 13, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

Review verdict: APPROVE — code is correct, the regression tests have teeth, CI is fully green on head.

What I verified (independent, from the branch — not the issue body):

  • Fetched the PR branch and read the full 4-file diff. One bounded-snapshot producer fix: normalizeForWrite now sorts by recency before both caps (count + byte); byte measurement is UTF-8 (Buffer.byteLength, the on-disk size) not UTF-16; the KV path (steady-state writes + the one-time hydrateOrderUnified migration) uses the store's 4 MiB ceiling (EXT_VALUE_MAX_BYTES, now exported from src/storage/store.ts); the legacy file path keeps its 8 MiB ceiling and the single-oversized-record fallback (first entry always kept).
  • Regression audit: in-memory order untouched (sort on a copy); count-cap behavior unchanged below 1024; lastSeen ties keep array order (V8 stable sort, pinned in test); mergeWithDisk unchanged. Both disclosed behavior changes are intentional per the PR body.
  • Local on head: npm run build green; targeted tests/order-persist.test.ts + tests/order-unified-store.test.ts = 10/10 pass; npm run typecheck clean.
  • Teeth check: ran the PR's test files against master (pre-fix) → the 2 new regression tests FAIL (they reproduce the bug), 8 pass. The tests genuinely pin both facets, not tautologies.

CI (head f1dcb9e):

Follow-up observation (noted here, not filed, per thread rule): serializeExtValue in src/storage/store.ts checks json.length (UTF-16 units) > EXT_VALUE_MAX_BYTES, not UTF-8 bytes. The producer's UTF-8 measurement is strictly stricter, so the store can never throw for this producer — but the store's own guard is named "bytes" while counting UTF-16, so a non-ASCII extension value could land up to ~4x more on disk than the 4 MiB ceiling implies. Pre-existing, out of scope for this PR; worth a follow-up if you want it tracked.

Labels applied to the PR (judged from its own diff): bug, p2, scope/storage.


八问(中文):

  1. 是什么问题:session-order 快照写入器在字节预算内丢的是最新的会话骨架(旧代码保留最旧、丢弃最新);且 KV 写入沿用了 8 MiB 预算,但存储层上限是 4 MiB,4–8 MiB 区间的快照会让一次性迁移抛错、进程整轮降级到 legacy 明文文件路径。
  2. 什么原因导致的:根因是快照生产者没有尊重各载体的真实上限——(a) 字节预算只在条目数 >1024 时才按 recency 排序,常见小规模场景保持写入顺序,8 MiB 上限因此保留最旧记录;(b) 同一生产者对 KV 迁移/稳态写入沿用 legacy 文件的 8 MiB 预算,忽略了 EXT_VALUE_MAX_BYTES(4 MiB);(c) 字节度量用 UTF-16 .length 而非落盘的 UTF-8 字节数。
  3. 影响范围与严重程度:重启后被字节预算挤出的最新会话没有持久化骨架,resolver 重跑冷启动竞态(Compression fails in long sessions while new sessions work fine | 压缩失败.新会话反而没事 #3397);4–8 MiB 快照让迁移抛错、整轮降级到明文 JSON 文件路径、放大每次 flush 写量。属"能用但有明显问题"——p2。
  4. 怎么修复的:单一有界快照生产者修复——两个上限(条数 + 字节)之前都按 recency 排序;度量改用落盘 UTF-8 字节;KV 迁移/稳态写入改用存储层 4 MiB 上限(EXT_VALUE_MAX_BYTES 从 store 导出共用);legacy 文件保留 8 MiB 上限与单条超大记录回退(首条恒保留)。无新配置、无 schema 变更、无碎片截断。
  5. 是否有回归问题:无。内存顺序未动(排序在拷贝上);条数上限行为在 <1024 时不变;lastSeen 平局保持数组顺序(V8 稳定排序,测试钉死);mergeWithDisk 未动。两处披露的行为变更(KV 上限 8→4 MiB、逐出顺序 写入序→recency、度量 UTF-16→UTF-8)均为 PR 正文有意变更。
  6. 是否破坏缓存:不涉及——未触碰缓存/重试/游标/去重键逻辑,只改快照落盘前的容量裁剪。
  7. 是否改了设计方向:不涉及——同一有界快照生产者,只是让它尊重各载体的真实上限。
  8. 是否引入了新的配置:不涉及——无新 env/config/默认值;EXT_VALUE_MAX_BYTES 从 store 导出是共用既有常量,非新配置。

判定:APPROVE,可合并。 代码正确、回归测试有牙(master 上复现 bug、PR 上转绿)、CI 在 head 全绿(flake 重跑后转绿)、标签已打。留一个跟进观察(store 的 serializeExtValue 用 UTF-16 .length 而非 UTF-8 字节做 4 MiB 判定,非 ASCII 扩展值可能超上限落盘)——本 PR 外,是否单开由你定。合并留给人工按 §7.4。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[ework] CI fix verified green ✓

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working p2 scope/storage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug][session-order] Snapshot capacity handling drops newest conversations and exceeds the KV limit

1 participant