Skip to content

[PR] fix(storage): close the sqlite handle when quick_check throws on a corrupt header (#4357) - #4358

Merged
ranxianglei merged 1 commit into
masterfrom
2026-10-23_corrupt-index-handle-leak
Oct 23, 2026
Merged

ranxianglei merged 1 commit into
masterfrom
2026-10-23_corrupt-index-handle-leak

Conversation

@ranxianglei

@ranxianglei ranxianglei commented Oct 23, 2026 •

Copy link
Copy Markdown
Owner

Fixes #4357

一句话:损坏头的 index.db 在 quick_check 抛错路径上泄漏 sqlite 句柄——Windows 上删不掉该文件(CI 全部 windows 车道确定性红 + 用户无法按 #2771 指引重置索引),Linux 被 unlink-open 语义掩盖。

改动(已 rebase 到当前 master,替代原 head d8352db)

验证(rebase 后的 head 本地定向跑)

  • Linux 复现:修复前 open() 抛错后残留 1 个 index.db fd;修复后 0 个。
  • 双向回归:还原 store.ts 修复只留守卫 → fdGuard 精确失败(open handle(s) into the corrupt index dir after close: .../index.db);带修复 tests/storage-engine-proofs.test.ts 12/12 绿。
  • tests/web-storage-banner.test.ts 5/5 绿(另一条 CI 失败测试,同一根因自愈)。
  • npm run typecheck 干净。

时间线:399c8bc66d(最后绿)无 quick_check → 1da8eca(#4175)引入 → 9e2d217 起所有 windows 车道红。该子家族与 #4347 的 AV/外部锁 flake 是两回事;#4350 的有界重试机制保留,继续服务真 flake。

@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-23_corrupt-index-handle-leak (1369811)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.5247-pr.4358.3b8edac

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr4358.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei ranxianglei changed the title fix(storage): close the sqlite handle when quick_check throws on a corrupt header (#4357) [PR] fix(storage): close the sqlite handle when quick_check throws on a corrupt header (#4357) Oct 23, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重: 不重复 —— 与已关闭的 #4347 / #4338 同症状家族(Windows 上 corrupt-db 测试 teardown EBUSY unlink),但机制不同:#4347/#4338 假设是外部 AV/锁 flake(已由 #4350 有界重试覆盖),本 PR 是 bili 进程自身在 quick_check 抛错路径泄漏 sqlite 句柄——同一 runner 确定性复现、Linux 侧 fd 可钉,与外部锁无关。两者并存不冲突。

标签: bug scope/storage p1(全部 windows CI 车道确定性红 = 验证管线停摆;且用户按 #2771 指引无法在不重启 bili 的情况下删除损坏 index.db)——无客户端标签(跨车道存储层)。

审核进展(进行中):

  1. 根因独立复核成立:对照代码确认 Storage Reliability Trio: open-time self-check (quick_check) + online backup (VACUUM INTO) + second-process probe on the | 存储可靠性三件套:开库自检 quick_check + 在线备份 VACUUM INTO + 同库第二进程探测 #4175(1da8eca6)把 PRAGMA quick_check 插进 UnifiedStore.open() 的既有 close-on-error catch 之外——损坏头时 new DatabaseSync(path) 成功(句柄已开),db.prepare("PRAGMA quick_check").all() 从语句本身抛 file is not a database,该路径无任何 db.close()。降级行为([Bug][storage] corrupt index.db bricks SessionStore construction in shadow/full mode (violates the sidecar never-breaks-legacy policy) #2771)本身没坏,坏的是句柄。store.ts 的修法(try/catch → recordIntegrityOutcome(false) + close() + rethrow)打到了产生泄漏的那一行,方向正确。
  2. 发现合并冲突,正在处理:master 在这 73 个提交里已经把该测试区重构成子进程方案(Windows CI lane stuck red: corrupt-index.db tests cannot delete their temp dirs (EBUSY) — blocks every PR merge #4338,tests/corrupt-index-child.ts),本 PR 的测试改动与之文本冲突。两点判断:
    • store.ts 修复 master 仍然没有(冲突只在测试文件),根因修复必须保留;
    • master 的 rename guard 在 POSIX 上是 no-op(Linux 测不出泄漏),本 PR 的 /proc/self/fd 守卫在 Linux 上有真实检测力——两者意图都要保留。
    • 处置:rebase 到当前 master,把 fd 守卫适配进子进程流程(句柄泄漏发生在子进程内,父进程扫不到;guard 移到子进程内、作为命名 check 加入),本地定向验证后推回本分支。
  3. CI 现状:head 上现有 run 多为被取代的 cancelled/skipped,无有效红绿——以 rebase 后新 head 的 CI 为准。

稍后补完整八问结论。

@ranxianglei ranxianglei added bug Something isn't working p1 scope/storage labels Oct 23, 2026
@ranxianglei
ranxianglei force-pushed the 2026-10-23_corrupt-index-handle-leak branch from d8352db to c71a93c Compare October 23, 2026 11:47
…rrupt header (#4357)

#4175 put PRAGMA quick_check between openSqliteDatabase() and the existing
close-on-error catches. On a corrupt header the constructor succeeds (handle
open) and quick_check .all() throws 'file is not a database' from the
statement itself — a path with no db.close(). The #2771 degrade then stays
functionally correct but leaks the handle: Linux unlink-open masks it, while
Windows deterministically burns the whole tmp-rm retry ladder on unlink EBUSY
(every windows CI leg since #4175, root cause of the corrupt-db sub-family of
#4347, reproduced via /proc/self/fd on Linux).

Rebased onto current master (replaces d8352db): the corrupt-index proof now
runs in a spawned child (#4338, tests/corrupt-index-child.ts), so the
portable fd guard lands there as a named check (fdGuard): after the degraded
construction and final close, no open fd may reference the index dir on
Linux (/proc/self/fd). The rename guards are POSIX no-ops, so this is the
only deterministic detector on Linux; Windows is covered end-to-end by the
teardown rmrf no longer hitting EBUSY. Negative control verified: reverting
the store.ts fix makes fdGuard fail naming the leaked index.db.
@ranxianglei
ranxianglei force-pushed the 2026-10-23_corrupt-index-handle-leak branch from c71a93c to 1369811 Compare October 23, 2026 11:50
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

冲突已解决,rebase 到最新 master 并验证后推回本分支。 新 head 1369811af(替代 d8352db)。冲突是纯文本性的——master 在 #4338 里把该测试区重构成了子进程方案,两者意图按并集保留:

  • store.ts 根因修复逐字保留(master 仍缺这个修复,冲突只在测试文件);
  • fd 守卫移入子进程(tests/corrupt-index-child.ts,命名 check fdGuard):泄漏发生在子进程内,父进程扫不到;master 的 rename guard 在 POSIX 上是 no-op(Linux 测不出泄漏),/proc/self/fd 守卫是 Linux 侧唯一确定性检测器——互补而非重复。

结构化八问(独立审核,非复读 issue):

  1. 是什么问题:损坏头的 index.db 在 quick_check 抛错路径上泄漏 sqlite 句柄——降级照常工作但句柄一直开着,Windows 上该文件删不掉(Storage Reliability Trio: open-time self-check (quick_check) + online backup (VACUUM INTO) + second-process probe on the | 存储可靠性三件套:开库自检 quick_check + 在线备份 VACUUM INTO + 同库第二进程探测 #4175 起所有 windows CI 车道确定性红;真实用户按 [Bug][storage] corrupt index.db bricks SessionStore construction in shadow/full mode (violates the sidecar never-breaks-legacy policy) #2771 指引重置索引必须重启 bili)。
  2. 什么原因导致的:Storage Reliability Trio: open-time self-check (quick_check) + online backup (VACUUM INTO) + second-process probe on the | 存储可靠性三件套:开库自检 quick_check + 在线备份 VACUUM INTO + 同库第二进程探测 #4175 把 PRAGMA quick_check 插在 openSqliteDatabase()(句柄已开)与既有 close-on-error catch 之间;损坏头时异常从语句本身抛出(file is not a database),该路径无任何 db.close()。Linux 被 unlink-open 语义掩盖,只在 Windows 显形。
  3. 影响范围与严重程度:全部 windows CI 车道(验证管线停摆)+ 真实 Windows 用户损坏索引场景(不重启进程无法按指引删除文件)。p1。
  4. 怎么修复的:在产生泄漏的那一行修——quick_check 的 prepare/all 包 try/catch,抛错时先 recordIntegrityOutcome(false) + db.close() 再原样 rethrow,[Bug][storage] corrupt index.db bricks SessionStore construction in shadow/full mode (violates the sidecar never-breaks-legacy policy) #2771 降级行为零变化;测试侧在子进程内加命名 fd 守卫(降级构造+最终 close 后,/proc/self/fd 不得有任何指向 index 目录的句柄)。
  5. 是否有回归问题:逐项审过——异常类型/消息不变(rethrow 原 err)、quarantine/降级路径不变、BILI_STORAGE_QUICK_CHECK=0 逃生口不变;唯一行为增量是该抛错路径现在也会向 runtime-status 记一条 integrity 失败(与既有 verdict≠ok 分支一致,纯观测增益)。无 fail-fast→吞错、无阈值/时序/输出格式漂移。
  6. 是否破坏缓存:不涉及(存储打开路径,与前缀缓存/增量游标无关)。
  7. 是否改了设计方向:无。测试结构顺从 master 的 Windows CI lane stuck red: corrupt-index.db tests cannot delete their temp dirs (EBUSY) — blocks every PR merge #4338 子进程方案,未回退。
  8. 是否引入新配置:无。

验证(本地定向跑,全量交 CI):

  • 带修复:tests/storage-engine-proofs.test.ts 12/12、tests/web-storage-banner.test.ts 5/5、npm run typecheck 干净(对最新 master 跑过两轮);
  • 负向对照(双向门禁):还原 store.ts 修复只留守卫 → fdGuard 精确失败(open handle(s) into the corrupt index dir after close: …/index.db,点名泄漏文件);恢复修复转绿;
  • rebase 备注:工作期间 master 又前进 6 个提交(zcode MCP dialect + CI concurrency,与本 PR 文件零交集),二次 rebase 干净通过。

判定:根因成立、修复打到核心、双向回归钉住、diff 干净(2 文件 +43/-2,每行相关)——可合,等新 head 1369811af 的全量 CI 绿,结果出来我再补一条。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

补一条状态:新 head 1369811af 的 21 个 check run 目前在共享 runner 队列里排队(推送 ~20 分钟后仍全部 queued,池子被其他在途 PR 占着),暂无有效红绿。本侧工作已全部完成:冲突解决 + 两轮 rebase 到最新 master + 本地定向验证(12/12、5/5、typecheck、负向对照)均已在上条结论中列明。CI 若红会再唤醒处理;全绿即可由你合并——合并权限在你。

@ranxianglei
ranxianglei merged commit 389de69 into master Oct 23, 2026
8 of 21 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.763-master.389de69 (rolling master tag).

npm install -g billion-context@0.0.763-master.389de69  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

1 participant