Skip to content

fix: align SQLite engine self-download timeout with the updater (10s -> 60s) - #3348

Merged
ranxianglei merged 1 commit into
masterfrom
2026-10-16_fix-3338-sqlite-bootstrap-timeout
Oct 17, 2026
Merged

ranxianglei merged 1 commit into
masterfrom
2026-10-16_fix-3338-sqlite-bootstrap-timeout

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

What & why

Fixes #3338. On Node ≤ 21 (no builtin node:sqlite) with no host-installed better-sqlite3, bili self-downloads the ~11 MB better-sqlite3@13.0.3 tarball to enable unified storage. That download was capped at 10 s (BOOTSTRAP_TIMEOUT_MS), while the self-updater budgets 60 s (src/update.ts:1666, fetchWithEgress) for its own comparable multi-MB tarball pull. On a non-trivial network (corporate proxy / slow link) 10 s forces >1.1 MB/s sustained before TLS-handshake / CONNECT overhead, so a single slow boot aborted the fetch — producing exactly the reporter's reason="The operation was aborted due to timeout" — and armed the 6 h cooldown, locking the machine into legacy-only storage for hours even though a working engine was obtainable.

Change

Align BOOTSTRAP_TIMEOUT_MS with the updater's 60 s budget in src/storage/engine-bootstrap.ts. The download path now reads bootstrapTimeoutOverrideMs ?? BOOTSTRAP_TIMEOUT_MS, where the override is a test-only seam (null in production).

Behavior change (old → new)

  • SQLite engine self-download whole-request timeout: 10 s → 60 s.
  • Only affects the narrow path where ALL hold: unified storage requested AND no builtin/host engine AND auto-download allowed AND cache miss. Bounded by the existing 6 h failure cooldown to at most once per window per machine. No other path, config key, wire shape, or persistence format changes.

Tests

Added a test seam + three regression tests in tests/storage-engine-bootstrap.test.ts:

node --import tsx --test tests/storage-engine-bootstrap.test.ts: 25/25 green; full typecheck clean on the rebased head. Full-suite verdict = CI test lanes.

Review notes

  • Single-module fix: scoped to src/storage/engine-bootstrap.ts; no unrelated churn, no lockfile change.
  • No new config surface — deliberately did NOT add a storage.* knob; the existing BILI_UPDATE_REGISTRY mirror plus the aligned default cover the report. If you'd rather expose the timeout as an explicit knob, say so and I'll propose it separately.
  • Human review required: this is a threshold/default-value change, which §7.4 keeps out of auto-merge.

Closes #3338

…3338)

The engine self-bootstrap capped its ~11 MB better-sqlite3 fetch at 10 s while
the self-updater budgets 60 s for its own comparable tarball pull. On a
non-trivial network (corporate proxy / slow link) that forced >1.1 MB/s
sustained before TLS/CONNECT overhead, so a single slow boot on Node<=21 (no
builtin node:sqlite) aborted the download and armed the 6 h cooldown — locking
the machine into legacy-only storage for hours even though a working engine was
obtainable.

Align BOOTSTRAP_TIMEOUT_MS with the updater's 60 s budget. Only behavioral
change: the startup wait ceiling in the narrow path where unified storage is
requested AND no engine exists AND auto-download is allowed AND cache misses;
the 6 h cooldown bounds it to at most once per window per machine.

Adds a test seam + three regression tests: a floor guard on the production
budget, an abort path (budget < download time), and a success path
(budget > download time).
@github-actions

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-16_fix-3338-sqlite-bootstrap-timeout (13d91f3)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.3948-pr.3348.7278e97

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr3348.tgz
npm install -g package

Install instructions are refreshed on each push.

@rahul05ranjan rahul05ranjan left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Traced this end to end — the fix is sound and the failure story in #3338 checks out.

What I verified:

  • BOOTSTRAP_TIMEOUT_MS (src/storage/engine-bootstrap.ts:65) is the only cap on the tarball fetch in downloadAndInstall (line 302). The old 10 s forced the ~11 MB better-sqlite3@13.0.3 tarball through at >1.1 MB/s sustained — one slow boot behind a proxy and the fetch aborts. I confirmed the abort path on a live Node: AbortSignal.timeout rejects with TimeoutError, message "The operation was aborted due to timeout" — exactly the reporter's reason — which flows into the catch at line 251, writes .bootstrap-failed.json, and arms the 6 h cooldown. The "one bad boot → 6 h legacy-only" lock-in is real, and this timeout was the trigger.
  • The new value matches the updater's own tarball budget (src/update.ts:1666, 60 s for a comparable multi-MB download). I also swept the other hardcoded timeouts for the same class of drift: the bulk-download paths are consistent (5 min for cold-install exec, dsh-channel.ts:25 / pi-channel.ts:34), and everything else in the 1.5–15 s range is health-check/handshake/manifest traffic, not multi-MB fetches. No other spot has this problem.
  • The override seam is clean: bootstrapTimeoutOverrideMs ?? BOOTSTRAP_TIMEOUT_MS is null in production and resetAllSeams() clears it. The abort-path test's /abort|time/i assertion is grounded — I ran a minimal fetch + AbortSignal.timeout(300) against a stalled local server and got exactly that message.
  • The floor-guard test (>= 30_000, citing #3338) is the right shape — it pins the regression direction without freezing the exact value.

Two nits (not blocking):

  • The updater's 60 s at src/update.ts:1666 is itself an inline literal, so the two values are "aligned" only by comment. The floor guard catches the bootstrap side regressing, but nothing pins the pair together — a shared named constant both import would close that for good.
  • Residual risk worth confirming as intended: 60 s still needs ~150 KB/s sustained for ~11 MB, and a timeout arms the same 6 h cooldown with no in-attempt retry. Fine as a scoped default change (and the body already rules out a config knob), just flagging that genuinely terrible links will still land in the cooldown — only less often.

I couldn't run the test file locally (no deps in my scratch clone), so the 25/25 claim is static-verified only — but the three new tests read correctly against the seam.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重: 不重复。 open+closed issue/PR 按 "SQLite engine self-download timeout / better-sqlite3 bootstrap" 及相近签名检索,无并行条目;本 PR 是 #3338 的唯一修复入口。相关背景项(#2825 回退契约、#2833 自下载特性、#3040 状态面板)与本变更不重叠。

结论: 代码与行为审查通过,可合并。 按 §7.4,阈值/默认值变更需人工合并。CI 唯一红分片是无关测试的 harness 级偶发取消(证据见下),不是本 PR 回归;已触发失败 job 重跑并立项跟踪(#3356)。

审查细节(独立重做,非复读 issue)

① 方向: 两端都实测核对——自更新器同类多 MB tarball 拉取用 AbortSignal.timeout(60_000)(src/update.ts:1666);引擎自下载原为整请求 10 s 预算。报告者的失败签名(reason="The operation was aborted due to timeout" + 6h cooldown)与 engine-bootstrap.ts catch 路径(err.message → writeCooldown → {status:"failed"})逐字吻合。修预算不一致 = 修根因,不是治标。

② 回归清点(触碰路径的既有行为逐项):

  • 探测梯顺序(builtin → host → cache → download)不变;
  • 任何下载失败的 fail-safe(catch → writeCooldown(reason) → failed+detail)不变,detail 文本与报告者日志逐字一致;
  • sha512 校验后再 rename 发布的次序不变——中断下载仍不会留下坏缓存目录(新测试钉住);
  • 6h cooldown 机制不变;启动控制流不变(src/cli.ts:601 仍同步 await ensureSqliteEngine);
  • 唯一有意变更:窄路径(unified 请求 AND 无 builtin/host 引擎 AND 允许自动下载 AND 缓存未命中)整请求超时 10 s → 60 s,PR 正文已披露旧值→新值。未发现语义漂移、吞错、输出格式漂移。

③ 代码: 单模块 fix,seam 遵循既有 *ForTests 命名惯例,null-in-production 语义清晰;注释讲 why;无 as any/死代码。60 s 窗口覆盖 DNS/TLS/CONNECT + 完整 body 流(AbortSignal 同时中止 reader 循环,downloadAndInstall 通读确认)。

④ 测试有效性: 三个新测试都有牙——floor guard 把生产值钉在 ≥30 s(引 #3338,防静默回落到秒级);abort 路径钉住 fail-safe 机制(failed + cooldown marker + 无缓存目录);success 路径钉住"慢但能完成"的下载现在能完成。非恒真断言,bug 回归时抓得住。

⑤ diff 整洁度: 2 文件 +87/−6,每行都服务于本 PR 目的;无 lockfile/生成物连带变更。

并发冲突: PR #3144 也改 src/storage/engine-bootstrap.ts(+18/−11,位于 runBootstrap 缓存命中/不可加载副本区 ~206-245 行)+ 其测试(+83,插在 happy-path 测试后 ~344 行)——与本 PR 的 hunk(常量 65 行、seam ~98-105、signal 一行 302、offline 测试前追加)正交。两者谁后合谁 rebase,干净;无语义交互。

CI(run 38852815261 @ head 13d91f3): 全部 gate 绿(static-gates 含 typecheck/build、kernel-guard、config-surface-guard、unused-exports、lock-sync 等);5/6 测试分片绿。唯一红 test (windows-latest, 22, 1) 取证:TAP fail 0 / cancelled 2,仅有的两条 not ok 是 #1647 BILI_STREAM_KEEPALIVE_MS=0 disables the hold 与 #1647 line-boundary guard…,均 failureType: 'cancelledByParent'(error Promise resolution is still pending but the event loop has already resolved),位置 tests/stream-keepalive.test.ts——本 PR 未触碰的文件,同一代码在其余 5 分片全绿;该分片总耗时 612s,判定为负载敏感的 harness 级取消(flake),非断言失败。本 PR 三个新测试在该红分片内也是绿的(ok 2856–2858)。已处置:① rerun-failed-jobs(HTTP 201);② 立项 #3356 跟踪(带来源标记)。

八问

  1. 是什么问题: 没有内置 node:sqlite 且主机未装 better-sqlite3(Node ≤21)的机器上,bili 自下载 ~11 MB SQLite 引擎 tarball 的整请求超时只有 10 s,代理/慢网下必然中途被掐,写入 6 h cooldown,把机器锁进 legacy-only 存储数小时。
  2. 什么原因导致的: 两个同类多 MB registry tarball 拉取的预算不一致——自更新器给 60 s(src/update.ts:1666),引擎 bootstrap 只给 10 s;10 s 要求 TLS/CONNECT 开销之前持续 >1.1 MB/s,一次慢启动即不满足。
  3. 影响范围与严重程度: 触发面 = Node ≤21 + 无 host better-sqlite3 + 首次下载 >10 s 的用户;严重度 = 特定条件下功能降级(unified 索引不可用,回退 legacy JSON 仍正常工作,无数据丢失)。修复后残留:网络不可达/黑洞场景下最坏首启等待从 ~10 s 变为 ~60 s(每 6h 窗口至多一次;ECONNREFUSED 类快速失败不受影响)。
  4. 怎么修复的: 单常量对齐到自更新器既有 60 s 预算(消除预算不一致这一根因);加 test-only override seam(生产恒 null)使"预算 vs 下载时长"关系无需真实慢网即可测;floor guard 钉住生产值 ≥30 s。
  5. 是否有回归问题: 触碰路径既有行为逐项过(探测序、fail-safe 映射、sha 校验+rename 发布、cooldown 机制、启动流)全部保持;唯一行为变化即已披露的 10 s→60 s。无 fail-fast 变吞错、无阈值/时序/输出格式漂移。
  6. 是否破坏缓存: 否。压缩/前缀缓存不涉及;引擎缓存目录布局与 .bootstrap-failed.json marker 格式不变。
  7. 是否改了设计方向: 否——启动保持同步阻塞。非阻塞启动是独立决策项(见下),不进本 PR。
  8. 是否引入新配置: 否——刻意不加 storage.* knob;既有 BILI_UPDATE_REGISTRY mirror + 对齐后的默认值覆盖报告场景。

待你拍板的两个决策(沿用 #3338,建议在此一并定)

  1. 60 s 还是 30 s? 我的建议:60 s。与自更新器对齐正是本次修复的核心逻辑(同类产物、同网络环境、应同预算);floor guard 只钉量级(≥30 s),若日后想收紧到 30 s 再单独改常量即可。
  2. 启动非阻塞化是否单开 issue? 我的建议:开。启动目前同步 await ensureSqliteEngine(src/cli.ts:601),引擎不可用时阻塞整个进程启动——这与本 PR(下载预算)是不同机制(启动控制流 + store 生命周期),风险面更大,不应塞进这个最小修复。你点头我就带来源标记单立。

判定: 可合(代码审查通过;§7.4 请你手动合并;红分片已证与本 PR 无关,等重跑转绿再合或直接合均可)。

@ranxianglei
ranxianglei merged commit 74a79ab into master Oct 17, 2026
55 of 56 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.515-master.74a79ab (rolling master tag).

npm install -g billion-context@0.0.515-master.74a79ab  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

ranxianglei pushed a commit that referenced this pull request Oct 17, 2026
The pinned-registry single source made the first-start engine download
abort on any slow/blocked network (10s budget at the time; 60s since
#3348): one registry down or throttled meant legacy-only until the 6h
cooldown expired.

When no registry is explicitly configured (BILI_UPDATE_REGISTRY /
update.registry), the self-download now walks a built-in public chain -
registry.npmjs.org then registry.npmmirror.com - each source under its
own timeout budget. An explicitly configured registry stays exclusive:
corporate egress must never silently fall through to public mirrors.

Per-source failure is logged with its cause and only an all-sources
failure arms the existing 6h cooldown marker, whose reason now names
every source tried. The sha512 pin applies per source, so a tampered
first mirror quarantines itself and the next mirror still serves.

Docs: CONFIGURATION.md en/zh updated on both touched rows plus the
BILI_UPDATE_REGISTRY env row (its 'leave unset in production' note was
stale for the engine-download path). Tests: candidate-resolution matrix,
first-mirror-failure fallthrough, all-fail cooldown naming both sources,
per-source sha gate.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SQLite unified storage degraded to legacy-only (bili v0.1.197, node v20.19.5, win32-x64)

2 participants