Skip to content

[PR] storage: move six JSON sidecar states onto the unified sqlite store (#5753) - #5757

Merged
ranxianglei merged 5 commits into
masterfrom
2026-11-05_json-state-to-unified
Nov 10, 2026
Merged

ranxianglei merged 5 commits into
masterfrom
2026-11-05_json-state-to-unified

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

What

The six remaining process-lifetime JSON state files now ride the unified
storage (kv_store rows), following the #5548 notice-receipt pattern:

file kv namespace/key module
session-order.json session-order / v1 src/order-persist.ts
prefix-affinity.json prefix-affinity / v1 src/affinity-persist.ts
rollout-id.json rollout / install-id src/rollout.ts
resign-pending.json resign-pending / pending src/apig-resign.ts
plugin-conversations.json plugin-conversations / table src/plugin.ts
host-plugin-notices.json host-plugins / <plugin id> src/host-plugin-notice.ts

Carrier rules everywhere: kv row is primary once hydrated; the legacy
file stays only as a pre-hydration read fallback and as the sole carrier for
legacy-only instances (BILI_ENCRYPTION_KEY opt-out — no unified sidecar);
a one-time import lands existing file content on first hydration, after
which a unified instance never writes the file again. Persistence stays
best-effort warn-once (never throws into a request path), mirroring #5548.

How it is wired

  • Boot: cli.ts hydrates the five process-lifetime states right next to
    hydrateNoticeReceipts() (no-op on legacy-only instances and before the
    storage singleton exists); server.ts hydrates the plugin conversation
    table immediately after loadConversations().
  • Shutdown: SIGTERM/SIGINT drain, the eaddrinuse die paths, and both
    restart hand-over routes (restart.ts) now await the async flushes
    (flushOrderSkeletons/flushPrefixAffinity became Promise<void>; new
    flushResignPending()), so a graceful exit loses nothing recorded this run.

Deliberate non-changes

Tests

Six new hermetic suites (full mode pinned in-suite, mirroring
tests/notice-unified-store.test.ts): one-time migration exactly once,
restart survival via kv with memory + file wiped, pre-hydrate fallback to
the file with late import, idempotent re-hydration (no row churn), plus the
plugin LRU cap enforced during the merge. The CI storage-baseline preload
pins the broad corpus to legacy, so existing file-assertion suites
(apig-resign, affinity-persist, rollout, persona-restart-stability, …)
keep covering the other direction of the gate untouched.

Pre-flight: typecheck clean; targeted regression batches green
(71 + 98 + 71 tests across the touched entry points); unused-export gate
and test-import-graph clean. No kernel/pi-subagents changes.

@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-11-05_json-state-to-unified (0968d50)

Latest build — install this exact version

npm install -g billion-context@0.0.8842-pr.5757.a8525a4

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
0968d50 billion-context@0.0.8842-pr.5757.a8525a4 build
24ec71c billion-context@0.0.8638-pr.5757.7f9ba77 build
56aeb4c billion-context@0.0.8635-pr.5757.d7a3c80 build
fb65337 billion-context@0.0.8303-pr.5757.997951b build
4732e77 billion-context@0.0.8280-pr.5757.c301bc2 build
4d74613 billion-context@0.0.8278-pr.5757.9661c16 build
c275b1e billion-context@0.0.8272-pr.5757.78c30bd build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr5757.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei ranxianglei changed the title storage: move six JSON sidecar states onto the unified sqlite store (#5753) [PR] storage: move six JSON sidecar states onto the unified sqlite store (#5753) Nov 5, 2026
ranxianglei added a commit that referenced this pull request Nov 5, 2026
…ew fixes (#5753)

Addresses the self-review findings on #5757 and the windows-CI failure it
caused (tests/snapshot-render-chain.test.ts installs a PARTIAL store stub —
tail-view methods only — and drove every proxy-mode render through
rolloutInstallId, which hit setKvAsync on the stub and crashed).

- persist.ts: new kvStore() probe — returns the singleton only when it is
  non-legacy AND actually exposes getKv/setKv/listKv; partial test stubs now
  degrade to the legacy file path instead of throwing
- host-plugin-notice.ts: pre-hydration dismissals no longer block the one-time
  file→kv migration (hydrate unions into the existing lazy cache and latches
  its own dismissHydrated flag); mid-hydration dismissals survive the final
  cache assignment
- apig-resign.ts: legacy-gate direction restores master parity (no sticky
  pendingCache when not riding kv — sibling-lane/manual file edits stay
  visible); failed unified writes demote back to the FILE path (the old
  discipline) instead of looping retries on sqlite
- order/affinity: same demote-on-failure on RMW errors; order adopts the
  presence-based hadRow check so an empty-but-present row is not re-uploaded
  on every boot
- plugin.ts: failed unified conversation-table writes demote to the legacy
  file path instead of retrying into the same dead chain every 300ms
- restart.ts/server.ts: withExitDeadline() bounds every graceful-exit storage
  flush at 5s — a wedged volume can no longer hold SIGTERM/SIGINT drain,
  eaddrinuse die paths or restart hand-over hostage (pre-migration flushes
  were sync and always returned)

Tests: two new regression cases (pre-hydration dismissal still migrates;
partial stub degrades without throwing). Typecheck clean; snapshot-render-chain
+ all six unified suites + affected regressions green; unused-export and
test-import-graph gates clean.

@ranxianglei ranxianglei left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self-review pass over c275b1e before merge. Findings and their resolution (fixes all in 4d74613):

Fixed

  1. [MED] Partial store stubs crashed every unified-mode caller. tests/snapshot-render-chain.test.ts installs a stub with only tail-view methods; rolloutInstallId() (hot path via featureEnabled) hit setKvAsync on it → TypeError — this is exactly what killed both windows-22 CI shards. Fix: new persist.ts kvStore() probe returns the store only when non-legacy AND actually exposing getKv/setKv/listKv; partial stubs now degrade to the legacy file path. All six callers switched from raw getStore() to it, plus a regression test in tests/rollout-identity-unified.test.ts.
  2. [MED] host-plugin-notice: a dismissal landing before the boot hydration lazily created the cache, which made the hydrate bail — blocking the one-time file→kv migration and hiding sibling rows for the process lifetime (inherited pattern flaw — see follow-up issue for the #5548 twin). Now: hydrate unions into the existing lazy cache, latches its own dismissHydrated flag, and mid-hydration dismissals survive the final assignment. Regression test added.
  3. [MED-LOW] apig-resign sticky cache broke the legacy-gate direction (master re-read the file on every op; sibling-lane/manual edits stayed invisible until restart after first local mutation). The legacy direction no longer caches at all = exact master parity. Also: failed unified writes now demote back to the FILE path (old discipline) instead of retry-looping on a dead chain every mutation.
  4. [LOW] order-persist used content-based hadRow while the affinity twin used presence-based — an empty-but-present row triggered re-upload + "migrated N" log on every boot. Aligned to presence check.
  5. [LOW] No timeout on exit drains: a wedged sqlite handle could hold SIGTERM/SIGINT drain, eaddrinuse die paths, and restart hand-over hostage forever (pre-migration flushes were sync and always returned). New withExitDeadline(label, op) in restart.ts bounds every flush chain at 5s, warn-logged, proceeds with what landed (next boot re-hydrates). Applied to finishShutdown/finishDrain, all three die paths, and both restart hand-over routes.
  6. [LOW] Misleading degradation warnings (order/affinity/plugin promised "legacy file path for this process" but kept retrying kv forever once the ready flag latched). The demote-on-failure change in #3 makes the file fallback real and the messages true again.

Accepted as-is (with rationale)

  • Cross-instance concurrent first-boot: migration does a blind whole-row setKv; the losing lane's unique rows are transiently gone from kv until its next mutation re-writes them (self-healing, bounded, requires two instances racing first boot). RMW-under-chain would complicate six modules for a self-healing window; noted here rather than changed.
  • _reset*ForTest() seams swap chains without draining in-flight ops: test-only ghost-write risk, accepted.
  • order/affinity do read-modify-write persists while resign/plugin keep whole-row overwrite: this mirrors each module's PRE-change file-writer semantics (all four files were whole-map last-writer-wins); not an inconsistency introduced by this PR.
  • Informational: early-exit paths flush conversations+sessions only (parity with master); killed in-flight migrations self-heal via re-import.

Follow-up (filed separately): the finding #2 pattern flaw also exists in #5548's src/notice-store.ts (addDismissedKeyInMemory + if (cache !== null || hydrating) return) — same pre-hydration-blocked-migration window for upgrade-notice receipts. Not touched in this PR to keep scope clean.

@ranxianglei

Copy link
Copy Markdown
Owner Author

Round-2 CI triage found a branch regression (not an infra flake): two auto-restart subtests failed here while passing at c275b1e. Root cause — one line of import graph:

MARKER_FILE in src/restart.ts freezes when that module first evaluates, and tests/auto-restart.test.ts re-points XDG_CACHE_HOME after its static imports, importing ../src/restart.ts dynamically on purpose (documented in the suite header). Commit 4d74613 added import { withExitDeadline } from "./restart.js" to src/server.ts; the suite statically imports server.ts, so restart.ts moved into the static phase and MARKER_FILE froze under the preload root — the suite's clears/writes then went to the env-dynamic path, hence "stale marker" + "cannot see own writes".

Fix in 4732e77: withExitDeadline moves to a new src/exit-deadline.js (body unchanged); server.ts and restart.ts both import it from there. The server → restart edge is gone; ordering is back: local repro 13/13, auto-restart file + snapshot-render-chain 22/22, six unified-store suites + regressions 88/88, typecheck + both repo gates clean.

@ranxianglei

Copy link
Copy Markdown
Owner Author

Round-3 result: green. The lone round-2 failure was test (windows-latest, 22, 5) → one subtest in tests/snapshot-render.test.ts, "D on but storage unavailable: legacy fallback, request still served", dying with undici fetch failed at 15.4s — immediately after its 80.9s churn sibling ran back-to-back on the same loaded runner. It has no contact surface with this branch's diff (none of the 17 changed files sit in that path), passes 4/4 locally here (including three runs of both the failing subtest and its heavy neighbor), and the failed-job re-run just went green with no code change. Classified as runner-load flake, consistent with the #4161 timing-window family rather than a regression.

State of the PR at 4732e77 (three commits: c275b1e migration · 4d74613 review fixes + partial-store-stub hardening · 4732e77 import-graph fix): all CI green, review comments addressed above, follow-up filed at #5761 for the notice-store twin inherited from #5548. Ready for human merge whenever.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

CI 车道收尾确认(本轮):独立复核 round-3 判定——head 4732e77 全绿,零代码改动。证据链(本轮逐条重查过,不是转述):

  1. 唯一红项 = test (windows-latest, 22, 5)(job 127373744290),其余 29 项绿/skipped。
  2. 拉了该 job 日志,唯一失败:not ok 1498 - D on but storage unavailable: legacy fallback, request still served(tests/snapshot-render.test.ts,harness turn() 处 undici fetch failed),duration 15437ms;紧邻前一个用例 ok 1497(D-off churn 文档用例)在同一 runner 上连跑 80940ms——典型的负载下时序窗口签名。
  3. 该 shard 总计 2578 pass / 1 fail;同文件其余子测试(1492–1497、1499 及之后)全绿。
  4. 与 PR 的 17 个改动文件逐一比对:失败测试文件与 D 车道渲染模块都不在 diff 里(diff 只覆盖六个 sidecar 状态迁移 + partial-store 加固 + server.ts import graph)。
  5. 终态独立核验(workflow run 42064154219 @ head 4732e77)= completed / success——failed-jobs 重跑即绿,同一份代码,无任何代码变更。镜像缓存当时滞后于上游,属轮询延迟,不影响结论。

八问:

  1. 是什么问题:本 PR 的 CI 单点红——windows-latest/node22/shard5 上一个子测试挂掉。
  2. 什么原因导致的:runner 资源争抢/时序窗口——80.9s 重负载用例连跑在先,下一用例起步时 fetch 超时(fetch failed,15.4s);非代码逻辑错误。
  3. 影响范围与严重程度:只影响那一次 CI 运行;用户面/产品行为零影响,同一代码重跑即过。
  4. 怎么修复的:无需修复——验证为 flake,failed-jobs 自动重跑转绿;本车道未推任何 commit。
  5. 是否有回归问题:无。diff 不触碰失败路径;static-gates、mysql-contract 及其余 29 项全绿。
  6. 是否破坏缓存:不涉及(无代码变更)。
  7. 是否改了设计方向:不涉及。
  8. 是否引入了新的配置:不涉及。

披露:我先前那条收尾评论因编号解析错误误发到了姊妹线程(GitHub PR #5755,增量 refs diff,镜像 #5757),已在那边补了误发说明楼层,请忽略那边那条。

判定:head 4732e77 全 CI 绿,本车道收工,等待人工合并。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重:不重复。mirror 上无其他 [PR] 指向 #5753「六个进程级 JSON sidecar 迁统一存储」主题——这是该 issue 的唯一 PR。(注:mirror↔GitHub 编号错位已确认,mirror #5757 是另一个无关 PR;对外一律引 GitHub 号 5757。)

标签:pr / scope/storage / p2 —— 跨车道存储基础设施重构(非缺陷故不打 bug;触及持久状态但按设计安全+载体可重建,判 p2)。

结论先行:可合。 当前 head 4732e77ac CI 30/30 绿;方向成立、回归安全、代码整洁、测试兜底、diff 干净。我不动 merge(人 only)。下面按八问展开。

① 是什么问题/需求:把六个进程级 JSON sidecar(session-order / prefix-affinity / rollout-id / resign-pending / plugin-conversations / host-plugin-notices)从各自独立文件迁到统一 sqlite(kv_store 行),沿用 #5548 notice-receipt 的载体规则。是存储收敛重构,不是修 bug。

② 什么原因导致的(为何改):六个文件各一套 load/save/debounce/torn-write 防护,与 #5548 已建好的统一存储载体规则重复;其中 plugin-conversations.json 还是唯一 in-place 写的文件(#406 才有 pid-tmp+fsync+rename 防 torn-write)。迁 sqlite 后原子提交直接消掉那一类,并统一「kv 主 / 文件仅 pre-hydration fallback + legacy-only 载体 / 一次性导入」纪律。

③ 影响范围与严重程度:

  • 触发面:所有启用统一存储(BILI_STORAGE_UNIFIED=full)实例,六个载体落点从 <state>/xxx.json 切到 kv 行。legacy-only 实例(BILI_ENCRYPTION_KEY opt-out)完全不变(纯文件路径逐字节保持)。
  • 严重度:本身非缺陷。风险点在「持久化格式变更」,但设计上有完整回退(一次性导入保旧数据、失败 sticky demote 回文件、best-effort warn-once 不进请求路径),且六个载体都是 routing/advisory 级状态(非主会话内容、非凭据)。判 p2。

④ 怎么修复的(机制):

  • 共享门 persist.ts kvStore():store 存在且真暴露 getKvAsync/setKvAsync/listKvAsync 才返回(部分测试 stub 安全落 null→文件回退)。
  • 每模块:kvReady 门控路由;写走链式 RMW(kvChain 串行 + 跨 lane union + 重套上限);失败 sticky demote 回文件(warn-once);hydrate*Unified() 一次性把文件行导入 kv 并覆盖内存;flush*() 变 async 等在途写入。
  • 两个刻意不变保真:apig-resign 可见性保持同步(调用方立刻断言 pendingCache,只有载体写排队)+ 文件写仍同步(hot-path 拒绝当下必须已落盘);rollout-id 保留 wx-file bootstrap race(两 fresh-install lane 副本先经文件竞态收敛到同一 id,再各自镜像进 kv)。
  • 五条退出路径(die/busy-port/crash/graceful-finishShutdown/finishDrain)全用 withExitDeadline(5s deadline)包 async flush——wedged volume 不绑架优雅退出。

⑤ 是否有回归问题(逐项审过既有行为):

  • legacy-only 纯文件路径:逐字节保持(unifiedLive()/kvReady=false 时走原 writeSnapshotLegacy/writeConversationsFile/readPendingRefusalsRaw)✓
  • apig-resign 同步可见性 + 文件同步写:未动 ✓
  • rollout-id bootstrap race:未动(generation 仍每装一次写一次文件)✓
  • plugin LRU cap:合并期重新施加(1030→1024 有测试钉住)✓
  • shutdown 时序:从「同步立即返回」变为「deadline 内 await」(最坏 +5s,wedged 时不挂死)——见下方披露
  • 唯一真实回归(auto-restart MARKER_FILE 冻结):本 PR 曾引入——server.ts 新增 import { withExitDeadline } from './restart.js',而 auto-restart.test.ts 静态 import server.ts,把 restart.ts 拖进静态阶段,MARKER_FILE 在 preload root 冻结,marker round-trip 读错位置。已在同 PR 内修复(withExitDeadline 拆到 src/exit-deadline.js,去掉 server→restart import 边),我读代码独立确认该机制成立。净回归:无。

⑥ 是否破坏缓存:不涉及模型 prefix-cache 语义。这六个是持久化载体不是 KV/prefix cache;迁移只换存储后端,内容(entries/table/id/map)原样搬运 + 校验函数守畸形数据。prefix-affinity/session-order 仍是同一份 snapshot,只是落点变了。

⑦ 是否改了设计方向:延续而非转向——把 #5548 已确立的「统一存储 + 载体规则」推广到剩余六个进程级状态。数据流清晰:boot hydrate(文件 warm-up→kv overlay→一次性上迁)→ steady-state kv 主 → exit flush。边界干净,未污染请求热路径(best-effort warn-once)。

⑧ 是否引入了新配置:无新 config/env/CLI flag(config-surface-guard 绿)。复用既有 BILI_STORAGE_UNIFIED / BILI_ENCRYPTION_KEY 语义,用户无需迁移动作(一次性导入自动发生)。

行为变更披露(Deliverables 第 4 条):

  1. 持久化载体变更:六个状态从文件改为 kv 行(仅启用统一存储的实例)。已迁移后 unified 实例不再写这些文件——回滚到旧版本会读到迁移时点的静态文件快照,post-migration 写入不可见。这是任何格式迁移的固有代价,Unified Announcement Panel Design (Approved): Built-in declarations + local JSON + remote GitHub raw — a single renderer | 公告面统一设计(已批准):内置声明 + 本地 JSON + 远程 GitHub raw,三来源一套渲染器 #5548 notice-receipt 同款先例(owner 已认可该模式);六个载体均为可重建的 routing/advisory 状态,非主会话数据,影响可控。
  2. 优雅退出时序:die/busy-port/crash/graceful 现在 await 存储 flush,受 5s deadline 约束(wedged volume 最坏多等 5s 后放行,之前同步立即返回)。可用性取向(不丢本轮记录 > 立即退出),timer unref 不阻塞 loop。

代码质量:六模块模式一致;校验函数(parseStoreDoc/parseConvTable/parsePendingDoc)守畸形持久化数据;无 as any/console.error;注释讲 why(载体规则 + #5757 修复点)。轻微观察(不拦):六模块各复制一套 kvReady/kvChain/warnOnce/hydrate/resetForTest 样板——因各域形状(array-entries vs table-map vs single-id vs pending-map)与合并规则(cross-lane union vs LRU re-insertion vs id-adoption)都不同,抽公共 helper 会牺牲清晰度,判为「结构平行」而非「重复逻辑」,不强制抽取。

测试有效性:6 个 hermetic 套件断言有牙——migration-once、restart-survival-via-kv(memory+file 双清)、pre-hydrate-fallback→late-import、idempotent re-hydration(no row churn)、LRU-cap-during-merge(1030→1024,oldest 从 front evict,newest 存活)。CI storage-baseline preload 把大语料钉 legacy,既有文件断言套件继续盖另一方向,gate 双向都有行。非假测试。

diff 整洁度:干净。17 文件全目的相关(6 模块 + cli/server wiring + persist plumbing + 新 exit-deadline.ts + 6 测试),真实 diff 无 lockfile churn、无空白噪音、无跑题文件;3 commit 逻辑分组(migration / hardening / test-isolation 修复)。

证据说明:CI 当前 head 30/30 绿为权威编译+测试判定(§7.2)。behind-master 风险已查:PR 落后 origin/master 37 commit,唯一两侧都碰的热文件是 src/server.ts,而 master 那处只是 +5 行的 dedupeAdjacentOutputConfig re-export(line~2555),与本 PR 全部改动区(import 头 / loadConversations 后 / 五条 shutdown 路径)完全不相交 → 无语义冲突,GitHub mergeable=True。(本地未单独重跑 typecheck——目标分支 worktree 被占用、成本不成比例;以 CI 绿为准。)

判定:可合。 建议合并前 rebase 到当前 master(落后 37,虽可干净合并)后让 CI 在 rebased head 重跑一遍即发。我不动 merge。

@ranxianglei

Copy link
Copy Markdown
Owner Author

Round-4 triage (run on fb65337): single red was test (ubuntu-latest, 22, 2) → one subtest in tests/msgid-lane-flow.test.ts (:139, "an edit lands as a FRESH row": got 3 rows, expected 4). That file is untouched by every commit in this PR — including fb65337, which only refines the resign-pending hydration import (src/apig-resign.ts + its own suite). Locally the whole suite passes repeatedly (6×6), and the failed-job re-run just went green with zero code change. It's the same race family this suite already documents in its own settle() comment (PR #5644 win22-s2 red): on a loaded runner the debounced save read lands before the edit generation persists, despite the afterNextFlush verdict. Classification: pre-existing CI flake in the msgid lane suite, not a regression from this branch. If it recurs enough to bite merges, the fix belongs in that suite's settle (wait for the specific row/seq, not just "a flush happened") — filed nothing since it's out of this PR's scope; happy to open a tracking issue if you want one.

Status at fb65337: all CI green again, otherwise unchanged from the round-3 note.

@ranxianglei ranxianglei added the ci:full Run the FULL CI matrix (all e2e lanes + 12-leg test matrix) on this PR label Nov 6, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

Round-4 独立复核(head fb65337):分类成立——msgid lane 套件的既有 CI flake,不是本分支回归。本轮逐条独立验证过(不是转述):

  1. 终态:镜像缓存新鲜拉取,sha=fb65337cd…eea,30/30 项 checks 全绿(failed 0 / pending 0)。
  2. 新提交范围:fb65337cd(per-scheme legacy import for resign-pending hydration)只动 src/apig-resign.ts(+9/-3)+ tests/apig-resign-unified-store.test.ts(+18)——两者都在原 17 文件 diff 内;base..HEAD 变更文件总数仍为 17,无新增文件。
  3. 接触面:tests/msgid-lane-flow.test.ts 不在分支任何提交的 diff 里(全窗 --name-only grep 零命中)。
  4. 签名核对:该文件第 139 行正是 assert.equal(rows.length, 4, "an edit lands as a FRESH row …")——与报告的「got 3 rows, expected 4」逐字对上。
  5. race 家族佐证:套件自己的 settle() 注释记录了固定 sleep 曾在 loaded windows runner 上与 debounced save 的 fsync 竞态(PR [PR] fix: surface failed watcher registration cause/state and report heartbeat recovery #5644 win22-s2 red),改为等 afterNextFlush 落盘判决;本次红项同属该家族(负载下判决之后、只读连接读到的仍是编辑前快照),且 debounceMs: 15 在负载下进一步放大窗口。
  6. 零代码改动:本车道本轮未推任何 commit;本地分支已同步至 fb65337,工作树干净。

八问:

  1. 是什么问题:round-4 的 CI 单点红——ubuntu-latest/node22/shard2,msgid-lane-flow :139 子测试。
  2. 什么原因导致的:runner 负载时序窗口(套件自述 [PR] fix: surface failed watcher registration cause/state and report heartbeat recovery #5644、CI: windows-latest node-22 shard 2 intermittently red on timing-sensitive e2e tests under load (rotating flakes: issue760, #4079, storage-full-flip #3818) #4161 同族),非代码逻辑错误。
  3. 影响范围与严重程度:仅影响那一次 CI 运行;用户面零影响,重跑即绿。
  4. 怎么修复的:无需修复——验证为 flake,failed-jobs 重跑转绿;零 commit。
  5. 是否有回归问题:无。diff 与失败路径零接触面;其余 29 项全绿。
  6. 是否破坏缓存:不涉及(无代码变更)。
  7. 是否改了设计方向:不涉及。
  8. 是否引入了新的配置:不涉及。

跟踪问题处置:按 CI 协调政策(2026-10-24)不为该 flake 开新 issue——结论已按政策补评到 #4599(CI 总控);它属于既有 #4161 时序窗口家族,单独立项只会触发重复调度。若后续复发到影响合并,修法方向如正文所述:settle() 从「等一个 flush 发生过」收紧为「等特定 row/seq 可见」,届时对 msgid lane 套件单独走一个小 PR。

判定:head fb65337 全 CI 绿,本车道收工,等待人工合并。

@ranxianglei
ranxianglei force-pushed the 2026-11-05_json-state-to-unified branch from fb65337 to 56aeb4c Compare November 8, 2026 12:03
ranxianglei added a commit that referenced this pull request Nov 8, 2026
…ew fixes (#5753)

Addresses the self-review findings on #5757 and the windows-CI failure it
caused (tests/snapshot-render-chain.test.ts installs a PARTIAL store stub —
tail-view methods only — and drove every proxy-mode render through
rolloutInstallId, which hit setKvAsync on the stub and crashed).

- persist.ts: new kvStore() probe — returns the singleton only when it is
  non-legacy AND actually exposes getKv/setKv/listKv; partial test stubs now
  degrade to the legacy file path instead of throwing
- host-plugin-notice.ts: pre-hydration dismissals no longer block the one-time
  file→kv migration (hydrate unions into the existing lazy cache and latches
  its own dismissHydrated flag); mid-hydration dismissals survive the final
  cache assignment
- apig-resign.ts: legacy-gate direction restores master parity (no sticky
  pendingCache when not riding kv — sibling-lane/manual file edits stay
  visible); failed unified writes demote back to the FILE path (the old
  discipline) instead of looping retries on sqlite
- order/affinity: same demote-on-failure on RMW errors; order adopts the
  presence-based hadRow check so an empty-but-present row is not re-uploaded
  on every boot
- plugin.ts: failed unified conversation-table writes demote to the legacy
  file path instead of retrying into the same dead chain every 300ms
- restart.ts/server.ts: withExitDeadline() bounds every graceful-exit storage
  flush at 5s — a wedged volume can no longer hold SIGTERM/SIGINT drain,
  eaddrinuse die paths or restart hand-over hostage (pre-migration flushes
  were sync and always returned)

Tests: two new regression cases (pre-hydration dismissal still migrates;
partial stub degrades without throwing). Typecheck clean; snapshot-render-chain
+ all six unified suites + affected regressions green; unused-export and
test-import-graph gates clean.
@ranxianglei

Copy link
Copy Markdown
Owner Author

已 rebase 到最新 master(24ec71c69),CI 全绿,mergeStateStatus: CLEAN。

rebase 解决的两处冲突(src/plugin.ts 导入行,均取并集):

  • 主体迁移 commit:master 新加的 CORRUPT_SIBLING_CAP/pruneCorruptSiblings(plugin-conversations 损坏备份清理)与本 PR 的 unifiedKvActive 共存;
  • 硬化 commit:本 PR 把 plugin.ts 的探测从 unifiedKvActive() 换成 kvStore()(能力探测),master 的 corrupt-siblings 导入保留。

rebase 后新增的一个修复(24ec71c69):master 新落地的 architecture-docs guard(#5930,要求 reference/architecture.md 的 src 文件树全覆盖)报 FAIL: reference/architecture.md missing: src/exit-deadline.ts——本 PR 拆出的 src/exit-deadline.ts 没进文件树。已按 guard 要求补一行,本地 guard 全绿后推送。

本地验证(worktree,干净 XDG state 目录):

  • npm run typecheck ✓
  • 6 个新 unified-store 测试 + notice-unified-store:34/34 ✓
  • 定向回归(apig-resign、affinity-persist、rollout、persona-restart-stability、plugin-native-binding、instance-governance、auto-restart):128/128 ✓
  • check:unused-exports、check:test-import-graph ✓

与 #5901 快照表的关系不变:两条线正交,可各合各的。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

Rebase 轮独立复核完成(head 24ec71c698):结论与正文一致——CI 全绿、无冲突、内容零漂移。逐条独立验证(非转述):

  1. 终态:镜像缓存对该 sha 两次拉取一致——30/30 checks 全绿(failed 0 / pending 0),mergeable_state: clean。(收到本条时运行尚未结束:18/30 在跑、0 失败;稍后复查转全绿。)
  2. rebase 保真度:git range-diff 对比 rebase 前后两串提交——第 3、4 个提交(aa3c888a8f、56aeb4cf09)逐字节相同(=);第 1、2 个提交的差异只有 src/plugin.ts 导入行的并集(与正文描述的两处冲突解法完全一致);新增第 5 个提交 24ec71c698 仅动 reference/architecture.md +1 行(exit-deadline.ts # 5s cap on the exit flush path (EXIT_FLUSH_DEADLINE_MS, #5753)),满足 architecture-docs guard([Bug][config] 0.1.212: effectiveConfigEpoch stamps the live file hash onto a snapshot resolved from startup-captured opts — config edits silently keep the pre-edit chain until reload #5930)。
  3. 并集语义:corrupt-siblings 清理只活在 loadConversations() 的坏文件 catch 分支(legacy 文件域),与 kv-primary 路径正交——kv 激活时该分支仍只在「文件存在且损坏」时触发,union 无交互缺陷。
  4. master 再前进的观察(信息项,rebase 基之后 master 又落了 [PR] Over-window requests no longer sent raw upstream: three forwarding gates metered by actual outbound body size + 32 MiB c | 超窗请求不再裸发上游:三道转发闸按真实出站体计量 + 大请求 32MiB 成本带准入 (#5000 #4543 #5885 #5794) #5932 等 3 个提交,动了 src/server.ts):两侧 hunk 区域完全不相交(本 PR 的 server.ts 改动集中在 ≤1743 行,[PR] Over-window requests no longer sent raw upstream: three forwarding gates metered by actual outbound body size + 32 MiB c | 超窗请求不再裸发上游:三道转发闸按真实出站体计量 + 大请求 32MiB 成本带准入 (#5000 #4543 #5885 #5794) #5932 集中在 ≥3733 行),GitHub 判 clean,无需再次 rebase。
  5. 零代码改动:本车道本轮未推任何 commit;本地跟踪分支已同步至 24ec71c698,工作树干净。

八问:

  1. 是什么问题:本轮无缺陷——是 rebase 后的 CI 确认轮。
  2. 什么原因导致的:不涉及(无红项)。
  3. 影响范围与严重程度:不涉及。
  4. 怎么修复的:不适用——零变更,只做独立复核。
  5. 是否有回归问题:未发现。range-diff 证明内容与 rebase 前等价(除两处声明过的导入并集 + 一行文档);并集语义正交;新 master 提交与本 PR 接触面零重叠。
  6. 是否破坏缓存:不涉及(无代码变更)。
  7. 是否改了设计方向:不涉及。
  8. 是否引入了新的配置:不涉及。

判定:24ec71c698 全 CI 绿 + CLEAN,与 #5901 两条线正交的说法成立,等待人工合并。

@ranxianglei

Copy link
Copy Markdown
Owner Author

评估性能问题

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

性能评估完成。结论:无实质性能风险——六处状态的写频率全部有界、无一在请求关键路径上被等待、单次写成本与旧文件方案持平(本环境实测)且机制上更省。证据如下:

1. 写触发频率(逐状态)

状态 触发点 频率上限 文档大小上限
prefix-affinity 请求处理器内 chain 变更(src/server/handle.ts:1660/1691) trailing debounce 500ms / max-delay 5s → 持续流量下 ≤1 flush/5s MAX_TRACKED_SESSIONS LRU
session-order session resolve 事件(src/session-resolve.ts:239) 同上 500ms/5s 1024 条 / 8MiB 字节预算
plugin-conversations 会话 map 变更 debounce 300ms LRU 上限
resign-pending 上游拒绝事件(稀有错误路径) 无 debounce 但频率极低、best-effort warn-once 32 条(RESIGN_PENDING_MAX_ENTRIES)
host-plugins dismiss 用户关闭通知动作 事件驱动,显式 fire-and-forget 单行
rollout install-id 每安装一次 一次性 单行

2. 热路径判定:六处写全部 fire-and-forget(模块级 promise chain:scheduleOrderPersist/scheduleAffinityPersist/persistPendingNow 均返回 void),请求→上游→流式响应关键路径不 await 任何 kv 写;只有 shutdown/restart 路径 await chain(flushOrderSkeletons/flushResignPending/flushConversations)。affinity 是唯一请求相邻的触发点,但旧文件方案同样在此调度 debounce 写——变的只是载体,不是调度。

3. 单次写 IO 特性:kv_store 为 WITHOUT ROWID 表、(namespace,key) 主键;库开 PRAGMA journal_mode=WAL + busy_timeout=5000(src/storage/schema.ts:105/107)。旧方案每次 flush = tmp 文件 writeFileSync+fsyncSync+rename(src/order-persist.ts writeSnapshotLegacy);新方案 = 1 次小索引 upsert(WAL append,fsync 由 checkpoint 摊销)。本机对照微基准(node:sqlite、同 pragmas,probe 存于工作区 .tmp/perf-probe.mjs):

  • 6KB 文档:旧 11.02ms vs 新 RMW(select+upsert)11.02ms(n=200)
  • ~900KB 文档:旧 11.79ms vs 新 12.23ms(n=50)
    两组在噪声内持平(此环境存储延迟地板 ~11ms;本地 NVMe 上两者都会降到亚毫秒级)。结论:单次 flush 成本不升,新方案还省去 tmp 文件创建/rename 的额外 syscall。

4. 最坏情形边界:唯一理论成本 = 封顶文档每 flush 一次的 JSON 往返(order 快照上限 8MiB)——但同一 normalizeForWrite 上限旧文件方案原样适用(diff 未触碰任何常量:debounce 500/5000、1024 条、8MiB 均为既有值),非新增暴露面;队列争用方面,持续流量下三状态合计每秒 ≤~2 次小 RMW 并入统一存储 shadow 写队列,相对消息 ingest 量可忽略,锁等待由 busy_timeout 5s 兜底。

5. 诚实边界(行为 delta,已披露):resign-pending 的持久化时序从「记录时同步落盘」(旧契约:on-disk by any exit)变为「异步 enqueue + shutdown await」;SIGKILL 硬杀下的丢失窗口从 ≈0 扩大到「上次入队写落地之后」——但该事件稀有、≤32 条封顶,且持久 store 失败会立刻降级回文件纪律(#5757 review fix)。这是 PR 内的设计取舍,不是静默漂移。

八问:

  1. 是什么问题:评估任务——本 PR(六个 sidecar 状态迁到统一 sqlite 存储)有无性能问题;非缺陷报告。
  2. 什么原因导致的:不涉及(未发现缺陷)。
  3. 影响范围与严重程度:结论=无实质风险;唯一行为 delta(resign-pending 硬杀丢失窗口)仅影响稀有的拒绝记录场景,严重度可忽略。
  4. 怎么修复的:不适用——零代码改动,静态代码路径分析 + 两载体受控微基准。
  5. 是否有回归问题:未发现——频率/debounce/封顶参数逐项与旧方案一致(diff 未动常量);单次写成本实测持平;热路径不变(fire-and-forget);WAL 替换 fsync+rename 不劣化。
  6. 是否破坏缓存:不涉及(无任何缓存相关变更)。
  7. 是否改了设计方向:不涉及——载体更换即本 PR 设计意图本身。
  8. 是否引入了新的配置:不涉及。

判定:性能面通过,无阻塞项;叠加此前各轮 CI 全绿 + CLEAN,等待人工合并。

@maozhuoshushu

Copy link
Copy Markdown

我们在一块 QLC(Crucial P3 Plus 1TB,标称 TBW 220 TB)上做了 31 天实地测量,结论是:这条 PR 应该提到 #3049 那个级别。它是目前唯一能把这些写入归零的路径,本机数据可以直接当作验收依据。

我们量到的(0.1.213 / 133f3525c)

C 盘 SMART Data Units Written:2026-10-09 33,713,640(17.26 TB / POH 4,124)→ 2026-11-09 75,017,044(38.40 TB / POH 4,736)。

30.99 天里 +41,303,404 units = +21.15 TB = 682.6 GB/天,是该盘前 4,124 个通电小时历史均速(4.19 GB/h)的 8.25 倍;Percentage Used 4% → 7%。

治理后(storage.jsonWriter=off + 快照接 zstd + debounce 30 s + WAL 20→100)降到 190 GiB/天,但文件系统层只能解释 SMART 的 1.4%:

SMART 实测                        684.6 MB / 300 s
可归因(目录增长 + 文件重写 + WAL 追加)   9.8 MB

剩下的全在 index.db 页级重写里(mtime 变、size 不变)。

六个 sidecar 在本机的实测数据(治理前后)

文件 治理前原大小 zstd 后 重写间隔(debounce 500 ms)
session-order.json 6.65 MB 2.52 MB(2.64×) 中位 13.6 s,min 6.2 s
prefix-affinity.json 1.68 MB 0.76 MB(2.17×) 同上
plugin-conversations.json 137,003 B 34,303 B(3.99×) 300 ms debounce,300 s 内 31 次
rollout-id.json 51 B — 静态
port-zone.json 24 B — 启动一次
proxy-origin 364 B — 启动一次

单 session-order.json 外推 ≈188 GB/天,与实测同量级。迁移后这 188 GB/天 + 2.69 MB/s 的 affinity 直接归零。

三个会影响这个 PR 的实测细节

1. session-order 的体积几乎全是 frags,而且没有时间淘汰。
57 条 entry 里 frags 占 83.7%;最大单条 12,357 个 frag / 942,513 B。normalizeForWrite2 只按 lastSeen 降序截断 MAX_ORDER_ENTRIES=1024 / MAX_SNAPSHOT_BYTES2=8 MB,没有任何按时间丢弃——本机 prefix-affinity 满 1024 条,最早一条是 27 天前。迁库时顺手加一个「超过 N 天丢弃」能立刻削掉存量,不必等 GC。

2. 迁进 kv_store 本质上还是整行重写,别把 JSON 那个放大原封不动搬过去。
这正是 #5599 里 mariopasquali 量的那个数:28 MiB mirror + 6 MiB metadata,三次变长小字段更新产生 215,385,392 B WAL,而定长编码只要 12,392 B。session-order 是典型的「长数组 + 尾部追加」,如果按单个 kv_store.value_json 整份 JSON.stringify 再 UPDATE,会从「每 13.6 s 重写 6.65 MB 的 JSON 文件」变成「每 13.6 s 重写 6.65 MB 的 SQLite 行」,总量基本不变。建议按 session 分 key(或只存 delta/frag 尾部),让每次 UPDATE 的 B-tree 页脏化只覆盖真正变化的部分。

3. port-zone.json / proxy-origin 的排除是对的,别改。
#5753 里已经把这两个排除掉了(launcher spawn 热路径)。我们这边确认 proxy-origin 只有 364 B、启动写一次,port-zone.json 24 B——顺手迁了不会省任何写入,只会把 spawn 可用性绑到 db 上。

顺带一个部署侧的坑(如果这个 PR 改了写路径)

我们用 BILI_SNAP_DEBOUNCE_MS 覆盖 debounce 时完全无效,原因是 bili 是 DSH host 的子进程,Windows 子进程继承的是父进程环境块,不是注册表 User 环境——host 已经跑起来之后再 setx 的新变量,新 spawn 的子进程收不到。最后只能改代码默认值。任何「运行时靠 env 调参」的设计都会踩这个(PERSIST_DEBOUNCE_MS2 是硬编码 var,反而没这个问题,但它也就完全不可配)。

完整报告(测量方法、四道治理的收益与代价、复现与回滚脚本):https://gist.github.com/maozhuoshushu/11331b6e9c3388d431365f2fffd1df92

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

🔶 需要决策

收到实地测量报告。三条技术判断我逐条对着 PR head 24ec71c698 的代码核过,全部成立;其中第 ② 条也暴露了我上一轮性能评估的口径盲区,一并修正。这属于设计决策(持久化布局 / 淘汰策略),按车道规则停下等拍板——本轮零代码改动,已挂 needs-decision 标签。

逐条核实(代码证据)

① session-order 体积以 frags 为主、无时间淘汰 —— 成立。

② 迁进 kv_store 仍是整行重写、写入量基本不变 —— 成立;这也是我上轮评估没讲透的地方。

  • 新路径 persistUnifiedNow(src/order-persist.ts:163)是单行 RMW:setKvAsync("session-order","v1",{version:1,entries})——整份文档塞进一个 value_json;旧路径每次 flush 也是整文件 rewrite+fsync+rename。两个载体每次 flush 都写整份文档,本 PR 对这些状态的累计写入量是中性的(不增不减)。
  • 修正:我此前的性能评估口径是「单次 flush 延迟」(微基准两路径同档 ~11ms);换成累计字节数(QLC 寿命指标)这个维度,结论应为:迁移本身归零不了这些写入,真正削量要靠分 key/增量编码 + 淘汰策略。你引用的 Migration design review: legacy JSON to SQLite-only, wire/cache contract, duplicate routes and durability fixes #5599 WAL 放大数字(215 MB vs 12 KB)我未独立复测,按你的实测引用。
  • 「按 session 分 key(或只存 frag 尾部 delta)」方向技术上成立:kv_store 是 WITHOUT ROWID,更新一行只脏化该行覆盖的页,分片后每次 UPDATE 的脏页数 ∝ 变化量。但这改的是持久化布局——human-gated 区,值得独立设计周期而不是塞进本 PR。

③ port-zone.json / proxy-origin 排除是对的 —— 同意,维持现状。
六个迁入状态不含这两个;PR diff(name-only)对它们零触碰,相关代码仍在 src/port-owner.ts、src/launcher.ts 等 spawn 热路径一侧。不动。

部署侧坑(Windows env 继承)

机制属实:bili 作为 DSH host 的子进程,继承的是父进程启动时的环境块,host 跑起来后 setx 的新变量到不了新 spawn 的子进程。规避方式:运行时调参走文件配置(~/.config/billion-context/billion-context.json,knobs 解析顺序 env>file>default,文件渠道不受 spawn 继承影响),不要依赖 env。PERSIST_DEBOUNCE_MS=500 / PERSIST_MAX_DELAY_MS=5000 目前是硬编码常量(src/order-persist.ts:28-29),本就不可配——若想要可配,需按配置面纪律走 owner 审批,我不擅自加 key。

数据可信度

算术抽查通过:41,303,404 units × 512 B ≈ 21.15 TB;POH 加权速率 21.15 TB / 612 h ≈ 34.6 GB/h vs 历史 4.19 GB/h ≈ 8.26×(与你的 8.25× 一致)。这份数据(SMART units 增量 + GB/day 前后对照 + gist 复现脚本)适合作为后续削量工作的验收基线——是否采纳由 owner 定。优先级提到 #3049 那个级别同样是 owner 的判断,我这里只能确认:底层问题真实、有量化数据支撑。

选项(🔶 需要决策)

  • A(推荐,默认):本 PR 按现状推进(载体迁移目标已达成、CI 30/30 绿 + mergeable CLEAN @ 24ec71c、写入量中性 = 合入不新增磨损);另立 issue 做写入量削减(N 天年龄淘汰 + 按 session 分 key/增量编码),以本报告为规格与验收依据。
  • B:扩入本 PR——合并前把淘汰 + 分片做进来。代价:持久化格式变更(human-gated)、范围膨胀、拖住当前已干净的 PR。
  • C:暂缓合并,等削量方案定稿再一起上。代价:一致性收益被一个不带来寿命收益的事拖延(本 PR 不恶化写入)。

推荐 A,理由:可用性/一致性优先于性能(#3128 原则);写放大是独立的失败域,按粒度政策归并进一个后续 issue 管理(子缺陷清单:淘汰策略 / 分片布局 / 可选压缩);现场数据直接当验收基线。未拍板时默认 A:本车道不再动这个 PR,PR 保持 green+CLEAN 等人工合并。

八问

  1. 是什么问题:实机测量显示六个 sidecar 的整文档重写是 QLC 盘磨损的主要贡献者之一(682.6 GB/天,历史均速 8.25 倍),疑问是本 PR 迁 sqlite 后能否消除这些写入。
  2. 什么原因导致的:两条写路径(legacy 整文件 / kv 单行整文档)每次 debounce flush 都重写整份文档,且无时间淘汰(多实例共享 sessions 目录的重启竞态:丢失更新、前缀缓存击穿与会话跨实例暴露(#394 扩展) #1724 明确设计为永不过期、仅 LRU 上限)——换载体不改字节数,只改字节落点。
  3. 影响范围与严重程度:长跑实例 + 低耐久盘(QLC)用户;慢速寿命消耗(Percentage Used 31 天 +3%),非可用性故障;本 PR 本身写入量中性,合入不新增磨损。
  4. 怎么修复的:本轮零代码改动(设计决策挂起);推荐 A:本 PR 原样推进,削量另立 issue(年龄淘汰 + 分 key/增量编码),以现场数据为验收基线。
  5. 是否有回归问题:无——本轮零改动;PR 本体已在前几轮审完(rebase 保真 range-diff、30/30 绿、CLEAN);量中性意味着无寿命回归。
  6. 是否破坏缓存:不涉及(零改动)。
  7. 是否改了设计方向:本轮没有;若选 B,则持久化布局转向(单文档行 → 按 key 分片),属 human-gated 区。
  8. 是否引入了新的配置:无;附注:Windows 下 DSH host 子进程收不到 setx 的 env,调参请走文件配置;新增 knob 需 owner 批(配置面纪律)。

判定:等 owner 在 A/B/C 中拍板;在此之前本 PR 保持现状(green + CLEAN @ 24ec71c),本车道停手。

ranxianglei pushed a commit that referenced this pull request Nov 9, 2026
… the sessions dir (#2960)

#5988 stopped the CCR content-store sidecar write under the off switch,
but the contract "the SQLite index is the ONLY persistent copy" had no
mechanical guard: any future save path that starts writing JSON there
would silently regress it (the sidecar itself is exactly that shape of
violation). This adds the gate.

tests/storage-json-writer-off-json-writes.test.ts:
- snapshot-diff design: every *.json file in the store dir is hashed
  before/after each phase; added/modified entries are checked against the
  whitelist. No fs interception — whatever API a write path uses shows up
  in the diff, so a new writer cannot hide behind an un-intercepted call.
- Phases: A first save (ZERO .json writes), B restart adopt + save
  (ZERO), C replica + restore (the restored id's own materialized files
  ONLY — envelope + CCR sidecar, named via relPathFor/
  contentStoreRelPathFor; session files are hash-named, so the concrete
  per-run set is derived from the restored id, not a regex), plus an
  on-control leg proving the snapshot mechanism actually sees writes.
- JSON_WRITE_WHITELIST documents the allowed category (restore
  materialization: the index is not a restore source and does not carry
  forked CCR stores); extending it requires the code path and the reason
  the index cannot carry the bytes.
- Mutation-verified: disabling the #5988 sidecar gate makes phase A fail
  with the offender file listed — the gate catches exactly the regression
  #5988 fixed.

Scope: the sessions directory (persistence layer). CLI-level state files
under stateDir() are handled separately by #5757 (kv_store migration).

Verification: typecheck ✓, gate suite 2/2 ✓, sibling off suites +
issue5805 9/9 ✓, storage regression battery 110/110 ✓,
check:unused-exports ✓, check:test-import-graph ✓,
gen-config-docs check-coverage ✓, check-architecture-docs ✓
@ranxianglei

Copy link
Copy Markdown
Owner Author

@ranxianglei 应你之请,把这条核实报告里的三处对着代码和算术重核了一遍。A/B/C 的判断本身站得住,但记录里有三处要更正:

1. 算术行的单位差 1000 倍(结论不变,但单位口径要落实)
"41,303,404 units × 512 B ≈ 21.15 TB" 照字面算是 21.15 GB,不是 TB。21.15 TB 只在每 unit ≈ 512 kB(十进制)时成立——原报告两个端点(33,713,640 → 17.26 TB、75,017,044 → 38.40 TB)正是这个口径,所以 34.6 GB/h、8.25×(比值,与单位无关)都对,但单位口径本身需要测量方确认:NVMe 标准 "Data Units Written" 是 512 字节单位,若按标准,绝对写入量低 1000 倍(约 0.68 GB/天),"按 220 TBW 约 10 个月磨完盘"的紧迫性不成立。交叉核对:21.15 TB 是 220 TBW 的 9.6%,而 Percentage Used 31 天只走了 4%→7%(≈6.6 TB 当量),两个计数器差约 3.2 倍——磨损指标粗糙/非线性,不足以定案,但值得让测量方把工具的换算单位确认一句,再给削量 issue 定优先级。

2. debounce knob:键已存在,只是没接线
"PERSIST_DEBOUNCE_MS/PERSIST_MAX_DELAY_MS 硬编码不可配,若要可配需按配置面纪律加 key" 这句不准确:persist.debounceMs / BILI_PERSIST_DEBOUNCE_MS 已经存在(src/knobs.ts:424-426,process.yaml 有列),并且对主会话落盘 writer 生效(src/persist.ts:4198)。真正没接线的是 order-persist.ts / affinity-persist.ts 这两个 sidecar persister 用自己的 const 500、完全不 import knobs.ts——所以"做成可配"是把现有 knob 接进这两个模块,不是新增配置键。连带地,"运行时调参走文件配置"的规避说法只对主 writer 成立,表里 500 ms 那两行(session-order / prefix-affinity)用文件配置同样改不动。另:原报告的 BILI_SNAP_DEBOUNCE_MS 在整个代码库里不存在,所以它"完全无效"至少还有一个更直接的根因;Windows setx 环境继承坑是真的,但不是这次失效的已验证根因。

3. 合并状态已过期
"CI 30/30 绿 + mergeable CLEAN @ 24ec71c" 是 11-08 的状态。今天 master 又进了若干 PR,本 PR 当前是 BEHIND + MERGEABLE(可自动合并但 base 落后),最后一次 CI 跑在 master 移动之前。按既定顺序:两个叠层 storage PR(#5988、#5992)合并后,本 PR 最后 rebase 一次再合。

结论维持 A 合理(迁移写入量中性——两条路径每次 flush 都写整份文档,已对代码核实,合入不新增磨损;削量是独立的 human-gated 布局变更,值得单独 issue + 现场数据做验收基线)。唯一前置:第 1 点的单位确认;若单位真是 512 B,削量 issue 的优先级应相应下调。

ranxianglei and others added 5 commits November 10, 2026 00:48
session-order.json, prefix-affinity.json, rollout-id.json,
resign-pending.json, plugin-conversations.json and host-plugin-notices.json
now persist through the unified storage (kv_store rows), following the #5548
notice-receipt pattern: the legacy file stays only as a pre-hydration read
fallback and as the sole carrier for legacy-only instances; a one-time import
lands existing file content on first hydration, after which a unified
instance never writes the file again.

- cli.ts boot hydrates the five process-lifetime states (no-op on legacy-only
  instances and before anything created the storage singleton)
- server.ts hydrates the plugin conversation table right after
  loadConversations(); shutdown paths (SIGTERM/SIGINT drain, eaddrinuse dies,
  restart hand-over) await the new async flushes so a graceful exit loses
  nothing recorded this run
- apig-resign keeps SYNCHRONOUS visibility on record/clear (callers assert
  the pending map immediately); only the carrier write is queued — the FILE
  path still writes sync, a new flushResignPending() drains the kv path on
  every exit route
- the wx-file creation race in rollout-id is kept BY DESIGN: two fresh-install
  lane copies must converge on one id before either mirrors into sqlite
- SQLite atomicity also removes the pid-tmp torn-write class from the plugin
  conversation table (the #406 workaround stays inert there)

Tests: six new hermetic suites (one-time migration / restart survival via
kv / pre-hydrate file fallback with late import / idempotent re-hydration /
LRU cap). The CI storage-baseline preload pins the broad corpus to legacy, so
existing file-assertion suites keep covering the other direction of the gate.
…ew fixes (#5753)

Addresses the self-review findings on #5757 and the windows-CI failure it
caused (tests/snapshot-render-chain.test.ts installs a PARTIAL store stub —
tail-view methods only — and drove every proxy-mode render through
rolloutInstallId, which hit setKvAsync on the stub and crashed).

- persist.ts: new kvStore() probe — returns the singleton only when it is
  non-legacy AND actually exposes getKv/setKv/listKv; partial test stubs now
  degrade to the legacy file path instead of throwing
- host-plugin-notice.ts: pre-hydration dismissals no longer block the one-time
  file→kv migration (hydrate unions into the existing lazy cache and latches
  its own dismissHydrated flag); mid-hydration dismissals survive the final
  cache assignment
- apig-resign.ts: legacy-gate direction restores master parity (no sticky
  pendingCache when not riding kv — sibling-lane/manual file edits stay
  visible); failed unified writes demote back to the FILE path (the old
  discipline) instead of looping retries on sqlite
- order/affinity: same demote-on-failure on RMW errors; order adopts the
  presence-based hadRow check so an empty-but-present row is not re-uploaded
  on every boot
- plugin.ts: failed unified conversation-table writes demote to the legacy
  file path instead of retrying into the same dead chain every 300ms
- restart.ts/server.ts: withExitDeadline() bounds every graceful-exit storage
  flush at 5s — a wedged volume can no longer hold SIGTERM/SIGINT drain,
  eaddrinuse die paths or restart hand-over hostage (pre-migration flushes
  were sync and always returned)

Tests: two new regression cases (pre-hydration dismissal still migrates;
partial stub degrades without throwing). Typecheck clean; snapshot-render-chain
+ all six unified suites + affected regressions green; unused-export and
test-import-graph gates clean.
…port phase (#5753)

Round-2 CI caught two auto-restart subtests failing on this branch while they
pass at c275b1e: the held-CONNECT-tunnel repro read a stale marker, and the
tolerant-marker test could not see its own writes. Both trace to one
mechanism. MARKER_FILE is frozen when src/restart.ts first evaluates; the
auto-restart suite re-points XDG_CACHE_HOME after its static imports and
relies on importing ../src/restart.ts dynamically so the constant freezes
under the suite's own sandbox. Commit 4d74613 added an
"import { withExitDeadline } from './restart.js'" edge in src/server.ts —
and tests/auto-restart.test.ts statically imports server.ts — which dragged
restart.ts forward into the static phase, freezing MARKER_FILE under the
preload root while the suite's clears/writes went to the env-dynamic path.

withExitDeadline now lives in its own src/exit-deadline.js (body unchanged);
server.ts imports it from there, restart.ts imports it for its two default
flush hooks. The server->restart edge is gone and the dynamic-import
ordering the suite documents holds again: local repro 13/13, full
auto-restart file + snapshot-render-chain 22/22, the six unified-store
suites + apig-resign/affinity/persistence regressions 88/88, typecheck and
both repo gates clean.
The one-time file-to-kv import only fired while the kv row was empty or
absent. Once the row held any other scheme, a late file entry (cross-lane
drift window: a legacy-only sibling still writing the file) could never
migrate. Align with the per-entry trigger the sibling modules already use
(order-persist, affinity-persist), plus a regression test.
The rebase onto master picked up the new architecture-docs guard
(#5930) which requires reference/architecture.md to list every tracked
src/ file. exit-deadline.ts (split out in the exit-flush hardening
commit of this PR) was missing.
@ranxianglei
ranxianglei force-pushed the 2026-11-05_json-state-to-unified branch from 24ec71c to 0968d50 Compare November 9, 2026 17:10
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

三处更正我逐条独立重核过,全部成立,记录按此修正。其中第 ① 条是我上一轮的算术错误,明确认领。另外:既定顺序的前置已满足(两个叠层 storage PR 均已合入 master),本轮把最后这次 rebase 做完了——详见下文。

更正确认(逐条重核证据)

① 算术单位 —— 你是对的,我错了。
41,303,404 units × 512 B = 21.15 GB,不是 TB;我上轮「算术抽查通过」一行是照抄了报告方的 TB 口径、没有重算,这行作废。三种口径重算:

换算 端点1(33,713,640) 端点2(75,017,044) 31 天增量速率
512 B(NVMe 字面) 17.26 GB 38.41 GB 0.68 GB/天
1000 B(NVMe 规范口径) 33.71 GB 75.02 GB 1.33 GB/天
512 kB(报告方工具实际口径) 17.26 TB 38.41 TB 682.4 GB/天

报告方两个端点与 512 kB/unit 精确吻合(17.26 / 38.41 TB),即其 SMART 工具按 512 kB 每 unit 换算。交叉核对复算无误:21.15/220 = 9.6%,Percentage Used 走 3% ≈ 6.6 TB 当量,差 3.2×。补一个佐证测量方需要确认单位的旁证:若按 512 B 或 1 kB 口径,该盘 4,124 POH 的累计写入只有 17–34 GB,对一块长跑系统盘低得反常(暗示计数器重置或厂商私有换算)。不受此争议影响的部分:8.25× 是比值、与单位无关,仍成立;FS 层直测(session-order.json 6.65 MB 整文件重写、中位间隔 13.6 s ≈ 42 GB/天主机写入)不经过 SMART 换算,机制与其 FS 量级都站得住。受影响的只是总量估计、「约 10 个月磨完盘」的紧迫性框架、以及削量 issue 的定级。同意你的处置:先让测量方确认工具换算单位,再定削量 issue 优先级;若真是 512 B,优先级相应下调。

② debounce knob —— 你是对的,我表述不准。
代码核实:persistDebounceMs() 已存在(src/knobs.ts:424-427,BILI_PERSIST_DEBOUNCE_MS > persist.debounceMs > 500),主会话落盘 writer 已接线(src/persist.ts:4197-4199 defaultDebounce()),website/config-reference/process.yaml:302-305 有列。真正没接线的只有 order-persist / affinity-persist 两个 sidecar persister(各自本地 const 500/5000,零 knobs import)。所以「做成可配」= 把现有键接进这两个模块,不是新增配置键——我上轮「需按配置面纪律加 key」的说法作废。连带更正:BILI_SNAP_DEBOUNCE_MS 全仓 grep(src/tests/scripts/docs)= 零命中,报告方「完全无效」更直接的根因是这个变量根本不存在(与平台无关);Windows setx 继承坑普遍为真,但不是那次失效的已验证根因,我上轮「机制属实」的措辞收窄为上述两点。处置建议:「persist.debounceMs 接入两个 sidecar persister」并入后续削量 issue 的子缺陷清单(同一机制族:这些状态的写频控制),不单独动本 PR。

③ 合并状态过期 —— 核实属实,且前置已满足。
PR head 未动(@ 24ec71c);origin/master 已到 4446301(较 merge-base 前进 232 commits),BEHIND 确认。(注:镜像 check 缓存冻在 11-05 快照、sha 还是旧的 5fd07ad,故本轮状态用单次 API 拉取核对,非轮询。)关键进展:#5988(jsonwriter-off-ccr-sidecar,→ 8efaa4c)与 #5992(jsonwriter-off-json-write-gate,→ fb7c607)均已在 origin/master——你定的顺序「叠层合并后本 PR 最后 rebase 一次再合」的前置条件已经满足。

本轮执行:最后 rebase(既定顺序的下一步)

结论与挂起项

A 维持(你的裁定)。挂起两项,都不阻塞本 PR 合并:① 测量方确认 SMART 工具换算单位 → 决定削量 issue 优先级(512 B 则下调);② 削量 issue 立项时带上子缺陷清单(年龄淘汰策略 / 分 key 或增量编码布局 / persist.debounceMs 接入两个 sidecar persister),以现场数据为验收基线。needs-decision 标签保留到单位确认闭环,也可由你直接摘除。

八问

  1. 是什么问题:对我上轮核实报告的三处更正(算术单位、knob 现状、合并状态过期)+ 既定顺序中「最后 rebase」的执行。
  2. 什么原因导致的:① 我照抄了报告方 TB 口径未重算(41,303,404×512 B=21.15 GB);② 写报告时未查 knobs.ts 已有 persist.debounceMs;③ 11-08 验证后 master 又进 232 commits,且镜像缓存冻结在 11-05 快照。
  3. 影响范围与严重程度:① 影响削量 issue 的紧迫性判断与定级(绝对写入量最多差 1000 倍),不影响机制本身(FS 层直测独立成立);② 纯措辞更正,无代码行为影响;③ base 落后已由本次 rebase 消除。
  4. 怎么修复的:更正记录;按既定顺序执行最终 rebase(stack 双 PR 已合入=前置满足),零冲突重放 + 本地门禁过后推送,CI 全绿收口。
  5. 是否有回归问题:无——rebase 为零冲突纯重放(merge-tree 预探干净);本地 typecheck/build/32 定向测试全绿;CI 64 项全绿(63 success+1 skipped)。
  6. 是否破坏缓存:不涉及(未改任何缓存相关逻辑,rebase 只带入 master 侧变更)。
  7. 是否改了设计方向:无;A 维持,knob 接线归入后续 issue 清单。
  8. 是否引入了新的配置:无(persist.debounceMs 为既有键,未接线部分属后续 issue 条目,非本 PR 引入)。

判定:PR @ 0968d50 全绿、base 最新,等人工合并;本车道完成本轮工作后停手。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

本轮完整报告已在上方楼层(三处更正确认 + 最终 rebase 执行记录)。一句话收口:更正全部成立并已修正记录;既定顺序前置(#5988/#5992 已合入)满足,最终 rebase 已推——新 head 0968d50,本地门禁(typecheck/build/定向 32 测试)与 CI(64 项全绿)均通过,PR 现处 base 最新、可人工合并。挂起两项不阻塞合并:① 测量方确认 SMART 工具换算单位 → 定削量 issue 优先级;② 削量 issue 子缺陷清单(年龄淘汰 / 分 key 或增量布局 / persist.debounceMs 接入两个 sidecar persister)。本车道停手待命。

@github-actions

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.1135-master.082143e (rolling master tag).

npm install -g billion-context@0.0.1135-master.082143e  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

ranxianglei pushed a commit that referenced this pull request Nov 12, 2026
… legacy→kv migration (#5761)

A dismissal landing before the cli-boot hydration lazily created the receipts
cache, which made hydrateNoticeReceipts bail on `cache !== null` — skipping
the one-time legacy JSON→kv migration for that process's lifetime (re-import
churn on every boot) and hiding sibling lanes' kv rows until restart; a
dismissal landing inside the hydrate's await gap was clobbered by the final
cache assignment.

Twin of the host-plugin-notice fix that landed in the #5757 line (commit
4d74613), ported onto master's API surface (unifiedKvActive/getStore; the
kvStore() probe from that line is not merged yet):

- hydrate unions into the existing lazy cache instead of bailing; a new
  hydrated latch (distinct from lazy-init) gates re-entry
- the live-cache snapshot moves to just before the publish with no awaits in
  between, so mid-hydration dismissals survive the final assignment
- publish only on success: a failed listKvAsync now leaves the cache null
  (legacy-file fallback stays live, next call retries) instead of clobbering
  it with an empty/partial map for the process lifetime

Tests: two regression cases mirroring the #5757 shapes — both fail on the
pre-fix code and pass with it; adjacent notice suites green.

(cherry picked from commit cf85588)
ranxianglei pushed a commit that referenced this pull request Nov 20, 2026
… legacy→kv migration (#5761)

A dismissal landing before the cli-boot hydration lazily created the receipts
cache, which made hydrateNoticeReceipts bail on `cache !== null` — skipping
the one-time legacy JSON→kv migration for that process's lifetime (re-import
churn on every boot) and hiding sibling lanes' kv rows until restart; a
dismissal landing inside the hydrate's await gap was clobbered by the final
cache assignment.

Twin of the host-plugin-notice fix that landed in the #5757 line (commit
4d74613), ported onto master's API surface (unifiedKvActive/getStore; the
kvStore() probe from that line is not merged yet):

- hydrate unions into the existing lazy cache instead of bailing; a new
  hydrated latch (distinct from lazy-init) gates re-entry
- the live-cache snapshot moves to just before the publish with no awaits in
  between, so mid-hydration dismissals survive the final assignment
- publish only on success: a failed listKvAsync now leaves the cache null
  (legacy-file fallback stays live, next call retries) instead of clobbering
  it with an empty/partial map for the process lifetime

Tests: two regression cases mirroring the #5757 shapes — both fail on the
pre-fix code and pass with it; adjacent notice suites green.

(cherry picked from commit cf85588)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci:full Run the FULL CI matrix (all e2e lanes + 12-leg test matrix) on this PR needs-decision p2 scope/storage 马上合并

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants