Repository navigation
[PR] fix(storage): in pure SQLite mode, confirmed index corruption no longer silently falls back to JSON; now fails loudly on | fix(storage): 纯 SQLite 姿态下索引证实损坏不再静默降级 JSON,改为响亮拒启 (#7156) - #7424
Conversation
… index instead of degrading to legacy JSON (#7156) In the pure-sqlite stance the SQLite index is the ONLY persistent copy of the sessions — but a proven corruption (this boot's own quick_check, or the first post-open write on a large store with a cached verdict) still took the historical #4626-era contract: loud alarm, start WITHOUT the index, serve the JSON envelopes. Here those envelopes are a near-empty, non-persistent view, so the degradation silently loses the durable history — worse than the corruption itself (2026-11-21 production incident: 3.2GB index.db, panel showed requests:2/context 0 while the real 1,563 requests / 124k context / cost sat inside the index). Now, when the effective stance is pure-sqlite (effective mode full + sqlite backend + no encryption + jsonWriter off — the 2026-11-13 default flip or an explicit off), a proven corruption makes bili REFUSE TO START with an actionable manual fix (back up index.db{,-wal,-shm}, move them aside by hand, start again). The files stay byte-identical (no move/copy/rebuild, the #4609/#4626 no-auto-touch policy is untouched). Every stance where the envelopes remain a real recovery source (explicit dual-write full, shadow, legacy, network backends, encryption) keeps the historical alarm-and-degrade contract byte-for-byte. BILI_STORAGE_ALLOW_LEGACY_DEGRADE=1/true/on is the conscious escape hatch that restores the degrade path in the pure-sqlite stance; it is never the default and never implied by any config value. The stance decision is a pure function (storageStancePureSqlite) using the exact same knobs/precedence as the persist constructor's jsonWriterOff switch, so the two can never disagree. The refusal is recorded on the diagnostics surface (REFUSING TO START (pure-sqlite stance) prefix on the sidecar open error, visible in /__bili/status and the diagnostics dump) and applies again on every boot. Docs: docs/configuration.md + configuration.zh-CN.md (BILI_STORAGE_QUICK_CHECK row split by stance, new BILI_STORAGE_ALLOW_LEGACY_DEGRADE row), CHANGELOG [Unreleased] Fixed. Regression: tests/issue7156-pure-sqlite-refusal.test.ts (new: stance truth table, corrupt-index refusal in both pure-sqlite env spellings + file tier, escape hatch, repeat refusal, byte-identity assertions, dual-write stance keeps degrading), tests/issue4816-integrity-throw-quarantine.test.ts, tests/acp-status-storage-degraded.test.ts, tests/web-storage-banner.test.ts, tests/storage-durability.test.ts and the full storage suite green.
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.11133-pr.7424.8f14692This exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr7424.tgz
npm install -g packageInstall instructions are refreshed on each push. |
…+ lifecycle ledger (#7156) The #7156 fix added the BILI_STORAGE_ALLOW_LEGACY_DEGRADE escape-hatch env var to src/ but only hand-edited the generated en/zh config docs. The seed (website/config-reference/env-only.yaml — the source of truth) and docs/lifecycle.md were never updated, so three doc gates went red: docs-coverage (undocumented env var), docs-sync (hand-edited docs drift from regenerated output), lifecycle-ledger-guard (no ledger entry). Fix: add the seed entry (en/zh/ja) and split the BILI_STORAGE_QUICK_CHECK row by stance in all three languages, then regenerate (only the ja doc picks up the new rows — en/zh already matched the seed), and register the new var in lifecycle §2.4 as a permanent operator escape hatch.
The new refusal test pinned the quick_check corruption branch (truncateHalf) but not the first-post-open-write branch (async mode / large store / cached verdict) — the path a >512 MiB store like the incident's 3.2 GB index.db is most likely to hit. The dual-write stance's first-write branch is pinned by issue7156-storage-corruption.test.ts (FIX 4), but the pure-sqlite stance was untested there: same recipe (zeroed page 2 + forced async mode), now asserting the REFUSAL throw, the latch, and the first-write error wording.
(ai: vllm-qwen/qwen3.8-27b) 查重: 不重复。检索 open PR 无其他 PR 指向同一「纯 SQLite 姿态证实损坏拒启」修复;最接近的是 #5255(双写者根因政策:把第二个长驻 index.db 写者降级为 legacy-only,#5064)——那是防损坏的根因决策,属 open owner 决策(#5250/#5281),本 PR 明确不碰(「不碰的」一节),机制不同(损坏后响应 vs 损坏前预防)。 标签: bug + dsh + scope/storage + p1(数据丢失类:静默丢持久历史;事故宿主为 DSH;修复在存储层)。 审核结论:代码正确、回归干净、测试到位。已独立读完全量 diff、核对姿态公式与 persist 开关同一性、走查拒启的完整启动路径、本地定向验证 59 用例 0 失败。CI 3 红项已由 CI 工程师车道修复(见下),最终 head 全 CI 绿即可合并。 结构化八问
顺手修(1 处,commit CI 状态(live 核实):3 红项(docs-coverage / docs-sync / lifecycle-ledger-guard)同一根因——新 env var 判定:代码正确、回归干净、测试到位(含我补的首写分支钉测);最终 head 全 CI 绿即可合并。 |
(ai: vllm-qwen/qwen3.8-27b) Fixed the three red doc gates on this PR. All three are now green on the live mirror cache (head Root cause. The #7156 fix added the
Fix (one commit, Note: a concurrent agent pushed a test-pin commit ( 七问:
判定:3 个文档门禁已全绿,本 run 收工。 |
…rded-refusal cases (#7156) The status wire's `requested` resolves from the REAL process env (the defensive fallback used before the first SessionStore build records the mode). Under a CI environment where the ambient unified selection resolves to legacy, `degraded` is false and the wire `reason` is blank — so the /WITHOUT the index/ and /REFUSES TO START/ assertions failed on the CI test lane even though the refusal/degrade mechanism itself fired correctly (every expected REFUSING/WITHOUT log was present in the CI output). Tests 2 and 5 already pinned process.env.BILI_STORAGE_UNIFIED (and passed on CI); the escape-hatch and recorded-refusal cases were the oversight. Pin it the same way (save/restore in finally) so the wire `requested` is deterministic regardless of the ambient environment — matching production, where recordRequestedMode always sets the requested mode.
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) CI 失败根因 + 修复(#7424 / 本 PR) 上一轮 head 根因:状态线 修复(commit 新 head |
…ult now refuses, #7156) The #5188 suite pins the round-5 degrade contract (a proven-corrupt index alarms and boots envelope-served, reopen hooks never re-scan). Its 'actual boot' ran under the ambient/default stance, which is now pure-sqlite (the 2026-11-13 default flip) — and the #7156 change makes a proven-corrupt pure-sqlite index REFUSE to start instead of degrading, so the boot threw and the test failed. Pin the boot to the explicit dual-write stance (BILI_STORAGE_UNIFIED=full) to preserve the test's intent (the round-5 degrade contract holds for dual-write). The pure-sqlite refusal is pinned separately in tests/issue7156-pure-sqlite-refusal.test.ts. Closes the CI regression on the #7156 PR head (run 44855025220).
…s abandoned-handle lock The first-write corruption branch intentionally abandons the opened store handle (close() would run the close-time integrity-cache refresh and re-stamp the poisoned verdict), and the background quick_check may still hold the file. On Windows the abandoned handle keeps index.db locked until GC runs, so the test's rmSync cleanup hit EBUSY (CI run 44855025220, windows-22 shard 1). Retry the removal briefly (50ms x 10); if it still fails with EBUSY, leave the temp dir (the OS reclaims it) — the test's assertions already ran. Non-EBUSY errors still throw.
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 上一轮我提交的 8dc4d68 触发的 CI(run 44855025220)红了,三处失败已逐一定位并修复,新 head = 92044e6,CI 重跑中。 失败 1:#5188 测试被本 PR 的行为变更打破(真回归,已修)
失败 2:我新增的 #7156 首写损坏测试在 Windows 清理时 EBUSY(测试卫生,已修)
失败 3:e2e #3384(windows-22 shard 1,疑似 flaky,未改)
提交
本地已验证:#5188 2/2、#7156 5/5(clean + legacy ambient 双环境)、 |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 新 head 92044e6 的 CI 已全绿(run 44858027262):28 个 job = 27 success + 1 skipped,0 失败;11 个 test matrix job(ubuntu/windows × node 22/24 × 分片)全部通过,上一轮红的 #5188、#7156 EBUSY 均已消除,疑似 flaky 的 e2e #3384 本轮未复现。 本 PR 的审核与 CI 收敛工作到此完成:
判定:可合并。 等 owner 合并即可(合并是 human-only,我不代合)。 |
…eage and lifecycle stats (#7156 follow-up) A save made while the process served from JSON envelopes (degraded window) carried the envelope's coarser block structure into importRecord, which treated the incoming set as authoritative: the index's 103 fine blocks were deleted and replaced by 32 coarse ones, and lifecycle stats regressed (requests 1563->2). ingestSession gains an envelopeImport flag (set by UnifiedShadow.importRecord/importRecordAsync only): with a stored block lineage present the import adopts only messages that continue after the stored max(seq) and skips block upserts/stale sweeps entirely; stats_json keeps the stored value when stored requests >= incoming (import path only, zero overhead for organic saves). First-open imports into an empty index keep the existing adopt-the-envelope behavior.
修复什么问题
2026-11-21 生产事故(bili 0.1.216,DSH host):3.2GB 的 index.db 被同 lane 多进程共写损坏(根因另案,见 #7156/#5281)。进程检测到「索引证实损坏」后走了旧契约——响亮告警 + 不挂索引照常启动、改用 JSON 信封服务。但当时的配置是纯 SQLite 姿态(index.db 是唯一持久副本,信封只是近乎为空的非持久视图),结果「降级」= 静默丢持久历史:面板显示 requests:2 / 上下文 0,而真实的 1,563 请求、124k 上下文、$12.41 成本全在损坏的索引里。降级比写坏更糟——写坏还能救,降级让「能救」这件事悄悄过期。
改了什么
storageStancePureSqlite(effective mode full + sqlite 后端 + 无加密 + jsonWriter off,与 persist 的 jsonWriterOff 同一公式,两开关永不分歧)。BILI_STORAGE_ALLOW_LEGACY_DEGRADE=1/true/on:有意识恢复旧降级路径;永不缺省、不被任何配置值隐含。/__bili/status与 diagnostics 可见REFUSING TO START (pure-sqlite stance)前缀),每次启动重新生效(latch 只短路进程内重扫)。回归
新增 tests/issue7156-pure-sqlite-refusal.test.ts(姿态真值表 + 两种纯 sqlite 拼法与 file 层的损坏拒启/逃生阀/重复拒启/字节不变断言 + 双写姿态保留降级契约);tests/issue4816-integrity-throw-quarantine.test.ts、tests/acp-status-storage-degraded.test.ts、tests/web-storage-banner.test.ts、tests/storage-durability.test.ts 及 storage 全量套件(69 文件 465 用例)绿,
tsc --noEmit0 错。不碰的