Repository navigation
[PR] fix: storage tail defects from the #3130 review (#3135) - #3144
ranxianglei wants to merge 7 commits into
Conversation
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.4162-pr.3144.d671dc8This exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr3144.tgz
npm install -g packageInstall instructions are refreshed on each push. |
(ai: vllm-qwen/qwen3.8-27b) 独立审核完成(未照抄 issue 侧结论,逐条对代码重新核过)。先说明一个关键点:本地跟踪分支是旧的(停在 结构化八问1. 是什么问题:#3130 发布评审发现的 6 个存储层小缺陷的收尾——自禁用那一拍可能丢最后一轮、BILI_PERSIST=0 误报「存储降级」且理由错、storageMode 盖戳虚报请求值、digest 漂移不上报、坏引擎缓存永不重下、限量启动看不见纯索引会话;外加一条 WAL 断电持久性文档说明。 2. 什么原因导致的(逐项根因,已对代码核实):
3. 影响范围与严重程度:均为特定条件下的正确性/误导面,无数据损坏。①仅 jsonWriter=off + sqlite 连续 3 次写失败 + 恰在翻转拍崩溃才丢最后一轮(极窄);②BILI_PERSIST=0 用户常驻假横幅+错误诊断理由(误导报障);③full 请求但 sidecar 开失败时 web 面板谎称有 index.db 副本;④full 模式漂移会话静默供旧数据(shadow 模式 JSON 是事实源,不受影响);⑤引擎缓存损坏后每次 boot 永久降级横幅;⑥限量启动下 jsonWriter=off 纯索引会话不可见。严重度:体验瑕疵~窄窗口丢一轮,无完全不可用场景。 4. 怎么修复的(机制层面,已核对实现):
5. 是否有回归问题:未发现无意回归。 我把每个触碰路径的既有行为清点了:
6. 是否破坏缓存:不涉及。prefix-cache 路径、更新节流、增量游标、去重键均未触碰;⑤ 的 7. 是否改了设计方向:不涉及。六项都在既有机制内补齐(同拍保证、opt-out 记录通道、构造期定戳、per-session 漂移门、下载阶梯既有冷却、限额内并集),无架构转向。 8. 是否引入新配置:无新 config/env/CLI。仅内部测试 seam 独立发现(低危,非阻塞,供你拍板)
次要观察(可忽略):① 的翻转贴片只在 测试有效性六个修复各有真断言兜底(非恒真):① strike-3 当拍 CI 状态(head
|
(ai: vllm-qeen/qwen3.8-27b) 先给结论:这条「上游 PR 冲突」通知我没有照单执行,而是先独立核实了真实状态再判断。 核实下来冲突是真的,但它是语义级、需要设计取舍的合并,不是机械解决;再加上这条通知是经一个未标记信任的外部通道转发来的、且要求 force-push——我按仓库 Git Safety 规则没有强推,也没有在没跑通存储测试的前提下把猜测性的合并结果推到共享分支。下面是我独立查到的事实和处置。 一、我独立核实的状态(不依赖那条转发的文字)
二、两处语义冲突(需设计判断,不是 pick-one-side)1.
2.
三、我的处置与理由
四、请你拍板(二选一)
另:仓库规范是结构化八问,通知里写的「七问」疑似模板笔误,不影响本次判断。等你一句方向我就接着做完。 |
(ai: vllm-qwen/qwen3.8-27b) 补充确认(完整独立核实与两处语义冲突分析见上一条):本次「上游 PR 冲突」我已独立查证为真实但语义级的冲突——本 PR #3144 与 master 新并入的 #3140 storage-delta-ingest 撞在同一批存储文件上。合并已在隔离 worktree 摊开备着。因涉及存储层设计取舍、且该通知来自未验证外部通道并索要 force-push,我没有强推、也没有擅自推送猜测性的合并结果。 一句话结论:PR 现值 |
060a0cb to
064b3c2
Compare
(1) jsonWriter=off: the save tick on which the sidecar hits its 3rd
consecutive failure previously armed neither the JSON write nor a
successful index write - that turn was memory-only until the next
save. The writerOff branches now re-arm the legacy JSON write in
the same tick when the disable flip happens mid-build; flushSync
returns the real file-write result instead of an unconditional true.
(2) BILI_PERSIST=0 no longer reports a permanent 'storage degraded'
banner with the bogus 'no SQLite engine available' reason:
persistence-disabled is recorded as an intentional opt-out, exactly
like the encryption block (new recordPersistDisabled seam).
(3) The per-file / live-session storageMode stamp now reflects the mode
that ACTUALLY opened, frozen once at store construction (stampMode):
a full request whose sidecar failed to open stamps 'legacy' instead
of claiming an index.db copy that does not exist. The requested-mode
resolver effectiveFileStorageMode() is gone - every buildRecord call
site passes the instance stamp explicitly.
(4) Digest drift is wire-visible: verifyOk/verifyDrift ride the
/__bili/status + overview storage payload and the diagnostics blob.
A drifted session in dual-write full mode serves the legacy-file
view again until a clean ingest resets it; under jsonWriter=off the
index view stays (sole copy), drift surfaced via stats.
(5) An unloadable-but-complete cached engine copy is removed and
re-downloaded once (sha512-pinned, so safe) instead of returning
'cache-hit' and degrading forever; a freshly downloaded copy that
fails to load arms the 6h cooldown marker instead of re-failing on
every boot.
(6) Bounded boot runs the same index union under its own limits
(count + decoded-byte budget from summary rows), so jsonWriter=off
index-only sessions are visible under warm-start limits; lazy reload
re-imports the envelope when the JSON copy is newer than the index
row before serving the unified view (mirrors buildForLoadAll).
Docs: CONFIGURATION en/zh jsonWriter row - fail-safe wording updated to
the same-tick guarantee, plus a WAL durability note for the off lane
(synchronous=NORMAL: crash-safe, power loss can drop the last checkpoint
interval). No new config surface.
064b3c2 to
fc144a3
Compare
|
ci失败 |
Union of intent on the three textual conflicts: - src/persist.ts: keep BOTH new field groups — this PR's stampMode (#3135③ construction-time mode stamp) and master's #3406 host-owned debounce fields (saveTimers/saveBuilders/debounceMs). All other hunks auto-merged disjointly; verified each PR hunk (writerOffBuild flip re-arm, residentFoldAllowed gate, loadSync staleness re-import, bootBounded index union) landed intact against master's ownership-stamp rewrite, and that effectiveFileStorageMode removal + session.ts getStore().stampMode applied cleanly. - CONFIGURATION.md/.zh-CN.md: master's sqliteAutoDownload line (mirror list #3338 + Node upgrade hint #3385) + this PR's jsonWriter line (same-tick fail-safe wording #3135① + WAL durability note). Tree left generator-stable: tools/gen-config-docs.mjs generate produces no drift.
(ai: vllm-qwen/qwen3.8-27b) 查重:不适用——本条是 PR #3144 自身上游冲突的续办任务(解自己与 master 的冲突),无重复条目。 冲突已解决并推送:merge 提交 一、CI 红的定位(你的「ci失败」)旧 head 二、冲突怎么解的(3 处文本冲突,全部按「意图并集」处理)
其余文件 git 自动合并不代表语义正确,逐一核过:
三、本地定向验证(全量判定以远程 CI 为准)
结构化八问
收工判定冲突已清、CI 红因已明(随本次合并转绿)、本地定向验证全过:待新 head |
The six-racer teardown SIGTERMs the children but never waits for them to actually exit before rmSync-ing the fixture dir. On Windows a killed child's tsx-cache handles outlive the signal, so unlink defers and rmdir hits ENOTEMPTY mid-cleanup. All lease assertions had already passed — this is a teardown race, not a product defect (#3144 CI, windows-22 shard 1, job 117338243865).
(ai: vllm-qwen/qwen3.8-27b) 查重:不重复(本 PR 跟踪项;新发现的两个独立问题已分别立项 #3530 / #3532,均带来源标记)。 「ci失败」已逐轮查清。冲突同步后 CI 先后红了三轮,三个不同原因,现已全部处置完毕,head 红①(ubuntu-22-2,上一轮已报):AC12 测试墙钟 flake,master 已由 #3334 修掉,合并 master 即消。 红②(windows-22-1): 红③(windows-22-2,两轮):lane-decisions-matrix 级联挂起("Promise resolution is still pending",约 9 个兄弟测试被连带取消)。第一轮是我分支基线早于 #3490 的加固合入——合并 master( 附带发现(每 push 一个幽灵红叉):#3416 今天进 master 后,每次 push 都多一条零 job、以原始路径命名的失败 run( 结构化八问
判定:head |
|
(ai: zhipuai-lb/glm-5.3) 陈旧票清理(第三批动作①):目标 issue #3135 已关。为避免重复劳动关闭本 PR;如判断有误请直接 reopen 并留言。 |
Closes #3135
All six items + the INFO note from the PR #3130 pre-release review of the SQLite storage layer. Host-side only: no
kernel/**changes, no new config surface.Fixes
① 3-strike self-disable tick lost one save (jsonWriter=off) —
src/persist.ts: the writerOff branches of scheduleSave/writeNow/flushSync checkedwriterOffActive()BEFORE running the builder, but the disable flip happens INSIDE the builder (shadow.record). On that tick neither the JSON write nor the index write happened — the turn was memory-only until the next save. The branches now re-arm the legacy JSON write in the same tick when the flip occurs mid-build (writerOffBuildhelper); flushSync returns the real file-write result instead of an unconditionaltrue(which let callers evict a never-persisted session). Strikes 1–2 stay transient by design (the next save retries sqlite).② BILI_PERSIST=0 → permanent false "degraded" banner with a wrong reason —
src/storage/runtime-status.ts+ constructor: persistence-disabled is recorded as an intentional opt-out via a newrecordPersistDisabledseam (same path as the encryption block):degraded=false,intentionalOptOut=true, no bogus "no SQLite engine available" reason; the diagnostics blob names the real cause ("persistence disabled" vs "BILI_ENCRYPTION_KEY opt-out").③ storageMode stamp said the requested mode, not the opened mode —
src/persist.ts/src/session.ts: newSessionStore.stampModeis decided once at construction: if the sidecar did not open (open failure, encryption block, persistence off) it islegacyeven thoughfullwas requested. Files written while the sidecar is dead and live-session metadata now stamplegacyinstead of claiming an index.db copy that does not exist. The requested-mode resolvereffectiveFileStorageMode()is removed; every buildRecord call site passes the instance stamp explicitly.④ digest drift invisible; full mode served drifted views —
src/storage/shadow.ts+ runtime-status: per-session drift tracking; a drifted session in dual-write full mode serves the legacy-file view again until a clean ingest resets it (the file is truth while dual-writing); under jsonWriter=off the index view keeps serving (sole copy), drift surfaced via stats.verifyOk/verifyDriftride the/__bili/status+ overview storage payload and the diagnostics blob.⑤ unloadable cached engine copy never re-downloaded —
src/storage/engine-bootstrap.ts: a complete-but-unloadable cached copy (e.g. ABI mismatch) is removed and falls through to the normal ladder (one sha512-pinned re-download) instead of returning "cache-hit" and degrading forever; a freshly downloaded copy that fails to load arms the 6h cooldown marker instead of re-failing every boot.⑥ bootBounded skipped the index union and the staleness re-import —
src/persist.ts: bounded boot runs the same index union under its own limits (count + decoded-byte budget from summary rows), so jsonWriter=off index-only sessions are visible under warm-start limits; lazy reload (loadSync) re-imports the envelope when the JSON copy is newer than the index row before serving the unified view (mirrors buildForLoadAll; importRecord never throws; closed-sidecar handled via try/catch around sessionUpdatedAt).INFO — CONFIGURATION.md en+zh
storage.jsonWriterrow: fail-safe wording updated to the same-tick guarantee, plus a WAL durability note (with off active the index.db is the sole copy; synchronous=NORMAL is crash-safe but a power loss can drop commits since the last checkpoint).Behavior changes (old → new)
cache-hit+ a permanent degraded banner every boot → removed + self-healing re-download (one fetch); post-download load failure was warn-only → now arms the 6h cooldown marker (status staysdownloaded).legacy).verifyOk/verifyDrifton the/__bili/status+ overview storage payload; drifted dual-write full-mode sessions serve the legacy-file view instead of the index view (writerOff behavior unchanged).recordPersistDisabled.Verification
npm run typecheckgreen.