Repository navigation
fix: align SQLite engine self-download timeout with the updater (10s -> 60s) - #3348
Conversation
…3338) The engine self-bootstrap capped its ~11 MB better-sqlite3 fetch at 10 s while the self-updater budgets 60 s for its own comparable tarball pull. On a non-trivial network (corporate proxy / slow link) that forced >1.1 MB/s sustained before TLS/CONNECT overhead, so a single slow boot on Node<=21 (no builtin node:sqlite) aborted the download and armed the 6 h cooldown — locking the machine into legacy-only storage for hours even though a working engine was obtainable. Align BOOTSTRAP_TIMEOUT_MS with the updater's 60 s budget. Only behavioral change: the startup wait ceiling in the narrow path where unified storage is requested AND no engine exists AND auto-download is allowed AND cache misses; the 6 h cooldown bounds it to at most once per window per machine. Adds a test seam + three regression tests: a floor guard on the production budget, an abort path (budget < download time), and a success path (budget > download time).
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.3948-pr.3348.7278e97This exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr3348.tgz
npm install -g packageInstall instructions are refreshed on each push. |
rahul05ranjan
left a comment
There was a problem hiding this comment.
Traced this end to end — the fix is sound and the failure story in #3338 checks out.
What I verified:
BOOTSTRAP_TIMEOUT_MS(src/storage/engine-bootstrap.ts:65) is the only cap on the tarball fetch indownloadAndInstall(line 302). The old 10 s forced the ~11 MB better-sqlite3@13.0.3 tarball through at >1.1 MB/s sustained — one slow boot behind a proxy and the fetch aborts. I confirmed the abort path on a live Node:AbortSignal.timeoutrejects withTimeoutError, message "The operation was aborted due to timeout" — exactly the reporter'sreason— which flows into the catch at line 251, writes.bootstrap-failed.json, and arms the 6 h cooldown. The "one bad boot → 6 h legacy-only" lock-in is real, and this timeout was the trigger.- The new value matches the updater's own tarball budget (src/update.ts:1666, 60 s for a comparable multi-MB download). I also swept the other hardcoded timeouts for the same class of drift: the bulk-download paths are consistent (5 min for cold-install exec, dsh-channel.ts:25 / pi-channel.ts:34), and everything else in the 1.5–15 s range is health-check/handshake/manifest traffic, not multi-MB fetches. No other spot has this problem.
- The override seam is clean:
bootstrapTimeoutOverrideMs ?? BOOTSTRAP_TIMEOUT_MSis null in production andresetAllSeams()clears it. The abort-path test's/abort|time/iassertion is grounded — I ran a minimalfetch+AbortSignal.timeout(300)against a stalled local server and got exactly that message. - The floor-guard test (
>= 30_000, citing #3338) is the right shape — it pins the regression direction without freezing the exact value.
Two nits (not blocking):
- The updater's 60 s at src/update.ts:1666 is itself an inline literal, so the two values are "aligned" only by comment. The floor guard catches the bootstrap side regressing, but nothing pins the pair together — a shared named constant both import would close that for good.
- Residual risk worth confirming as intended: 60 s still needs ~150 KB/s sustained for ~11 MB, and a timeout arms the same 6 h cooldown with no in-attempt retry. Fine as a scoped default change (and the body already rules out a config knob), just flagging that genuinely terrible links will still land in the cooldown — only less often.
I couldn't run the test file locally (no deps in my scratch clone), so the 25/25 claim is static-verified only — but the three new tests read correctly against the seam.
(ai: vllm-qwen/qwen3.8-27b) 查重: 不重复。 open+closed issue/PR 按 "SQLite engine self-download timeout / better-sqlite3 bootstrap" 及相近签名检索,无并行条目;本 PR 是 #3338 的唯一修复入口。相关背景项(#2825 回退契约、#2833 自下载特性、#3040 状态面板)与本变更不重叠。 结论: 代码与行为审查通过,可合并。 按 §7.4,阈值/默认值变更需人工合并。CI 唯一红分片是无关测试的 harness 级偶发取消(证据见下),不是本 PR 回归;已触发失败 job 重跑并立项跟踪(#3356)。 审查细节(独立重做,非复读 issue)① 方向: 两端都实测核对——自更新器同类多 MB tarball 拉取用 ② 回归清点(触碰路径的既有行为逐项):
③ 代码: 单模块 fix,seam 遵循既有 ④ 测试有效性: 三个新测试都有牙——floor guard 把生产值钉在 ≥30 s(引 #3338,防静默回落到秒级);abort 路径钉住 fail-safe 机制(failed + cooldown marker + 无缓存目录);success 路径钉住"慢但能完成"的下载现在能完成。非恒真断言,bug 回归时抓得住。 ⑤ diff 整洁度: 2 文件 +87/−6,每行都服务于本 PR 目的;无 lockfile/生成物连带变更。 并发冲突: PR #3144 也改 src/storage/engine-bootstrap.ts(+18/−11,位于 runBootstrap 缓存命中/不可加载副本区 ~206-245 行)+ 其测试(+83,插在 happy-path 测试后 ~344 行)——与本 PR 的 hunk(常量 65 行、seam ~98-105、signal 一行 302、offline 测试前追加)正交。两者谁后合谁 rebase,干净;无语义交互。 CI(run 38852815261 @ head 13d91f3): 全部 gate 绿(static-gates 含 typecheck/build、kernel-guard、config-surface-guard、unused-exports、lock-sync 等);5/6 测试分片绿。唯一红 八问
待你拍板的两个决策(沿用 #3338,建议在此一并定)
判定: 可合(代码审查通过;§7.4 请你手动合并;红分片已证与本 PR 无关,等重跑转绿再合或直接合均可)。 |
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.515-master.74a79ab # this exact build
npm install -g billion-context@master # always the newest merged build
|
The pinned-registry single source made the first-start engine download abort on any slow/blocked network (10s budget at the time; 60s since #3348): one registry down or throttled meant legacy-only until the 6h cooldown expired. When no registry is explicitly configured (BILI_UPDATE_REGISTRY / update.registry), the self-download now walks a built-in public chain - registry.npmjs.org then registry.npmmirror.com - each source under its own timeout budget. An explicitly configured registry stays exclusive: corporate egress must never silently fall through to public mirrors. Per-source failure is logged with its cause and only an all-sources failure arms the existing 6h cooldown marker, whose reason now names every source tried. The sha512 pin applies per source, so a tampered first mirror quarantines itself and the next mirror still serves. Docs: CONFIGURATION.md en/zh updated on both touched rows plus the BILI_UPDATE_REGISTRY env row (its 'leave unset in production' note was stale for the engine-download path). Tests: candidate-resolution matrix, first-mirror-failure fallthrough, all-fail cooldown naming both sources, per-source sha gate.
What & why
Fixes #3338. On Node ≤ 21 (no builtin
node:sqlite) with no host-installedbetter-sqlite3, bili self-downloads the ~11 MBbetter-sqlite3@13.0.3tarball to enable unified storage. That download was capped at 10 s (BOOTSTRAP_TIMEOUT_MS), while the self-updater budgets 60 s (src/update.ts:1666,fetchWithEgress) for its own comparable multi-MB tarball pull. On a non-trivial network (corporate proxy / slow link) 10 s forces >1.1 MB/s sustained before TLS-handshake / CONNECT overhead, so a single slow boot aborted the fetch — producing exactly the reporter'sreason="The operation was aborted due to timeout"— and armed the 6 h cooldown, locking the machine into legacy-only storage for hours even though a working engine was obtainable.Change
Align
BOOTSTRAP_TIMEOUT_MSwith the updater's 60 s budget insrc/storage/engine-bootstrap.ts. The download path now readsbootstrapTimeoutOverrideMs ?? BOOTSTRAP_TIMEOUT_MS, where the override is a test-only seam (null in production).Behavior change (old → new)
Tests
Added a test seam + three regression tests in
tests/storage-engine-bootstrap.test.ts:>= 30 s, citing SQLite unified storage degraded to legacy-only (bili v0.1.197, node v20.19.5, win32-x64) #3338) so it can't silently regress back to seconds;failed+ cooldown marker armed, no cache dir);downloaded).node --import tsx --test tests/storage-engine-bootstrap.test.ts: 25/25 green; full typecheck clean on the rebased head. Full-suite verdict = CItestlanes.Review notes
fix:scoped tosrc/storage/engine-bootstrap.ts; no unrelated churn, no lockfile change.storage.*knob; the existingBILI_UPDATE_REGISTRYmirror plus the aligned default cover the report. If you'd rather expose the timeout as an explicit knob, say so and I'll propose it separately.Closes #3338