Repository navigation
fix(storage): builtin npm+npmmirror mirror fallback + Node upgrade hint for the SQLite engine download (#3338) - #3485
Conversation
…nt for the SQLite engine download (#3338) Directives 2 and 3 of #3338 (owner ruling in the #3385 dup thread): 2. The engine tarball download now walks a builtin mirror list: [configured registry (BILI_UPDATE_REGISTRY / update.registry), https://registry.npmjs.org, https://registry.npmmirror.com] dedup-preserving-order. Per-mirror fetch + sha512 verify; the first mirror that yields a verified tarball wins; extraction/publishing is once and mirror-independent. Each failed mirror logs a warn and the loop advances; full exhaustion names every attempted mirror in the cooldown reason ('... (tried mirrors: a, b, c)') so the #3385-style 'aborted due to timeout' boxes can see WHERE the network died. The self-downloaded artifact stays sha512-pinned, so a mirror serving tampered bytes still quarantines (#2833 contract unchanged). 3. runtime-status degrade reasons and the bootstrap-failure log line now append an explicit upgrade lever on Node < 22.5 (engines allows >= 20, but node:sqlite ships at 22.5 - the #3385 node v20.20.2 box had no builtin engine and a download that could not complete): 'node:sqlite needs Node.js >= 22.5; upgrade Node.js for the builtin engine (engine tarball mirrors tried in order: npm + npmmirror; set BILI_UPDATE_REGISTRY or update.registry to pin one)' Tests (tests/storage-engine-bootstrap.test.ts): - mirror fallback: HTTP-500 first mirror -> next mirror serves -> downloaded - mirror exhaustion: cooldown reason lists every attempted mirror - configured registry attempted before the builtins (hermetic: the succeeding mock must be the one registry hit) - nodeSqliteUpgradeHint pins the >= 22.5 threshold + levers - withRegistry now pins the mirror list to its single mock so the pre-mirror-era tests stay hermetic (a failing mock must fail the bootstrap instead of falling through to the real registries) - resetAllSeams clears the new mirrors seam Docs: storage.sqliteAutoDownload seed (website/config-reference/ process.yaml, EN+ZH) + regenerated tables document the mirror walk.
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.4135-pr.3485.62dd5e6This exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr3485.tgz
npm install -g packageInstall instructions are refreshed on each push. |
(ai: vllm-qwen/qwen3.8-27b) 查重: 不重复。 本 PR 是 #3338 owner 指令 2+3 的唯一实现入口(指令 1 的 60s 预算已由 #3348 合入);#3385 已在其自身线程判定为 #3338 的重复项,将随本 PR 的 ① 是什么问题:Node 20 机器(node:sqlite 未内置)自下载 SQLite 引擎 tarball 失败后,机器因冷却期长时间锁死在 legacy-only 存储;且用户完全不知道升级 Node.js 就能直接解决。本 PR 落地两条 owner 指令:内置多镜像回退(npm → npmmirror)、Node < 22.5 升级提示。 ② 什么原因导致的:原下载逻辑只尝试单一 registry(用户配置或默认 npmjs),该 registry 不可达/超时就整体失败并进 6h 冷却——单点网络故障直接放大成存储降级;degrade reason 只说「引擎不可用」,没有给出可操作出口。 ③ 影响范围与严重程度:影响所有需要自下载引擎的主机(Node < 22.5,#3338/#3385 的 Node 20 win32 盒子是典型)。旧行为下单一 registry 不通 = 存储 legacy 锁死 6h;新行为下 CN 网络可落 npmmirror,失败信息能定位到具体死在哪个镜像、以及如何钉住。严重度:能力不可用 → 现在有真实恢复路径。 ④ 怎么修复的:
⑤ 是否有回归问题:逐项审过既有行为,未破坏:
⑥ 是否破坏缓存:不涉及。冷却标记/快照/去重键均未变;多镜像只影响下载尝试路径。 ⑦ 是否改了设计方向:不涉及。仍是「按需自下载 + 失败冷却」,只是下载更健壮、诊断更可操作,架构未转向。 ⑧ 是否引入了新的配置:无。复用既有 行为变更披露(有意):
已知小限制(不阻塞):提示阈值用 22.5,而早期 22.x minor 的 node:sqlite 还挂在 --experimental-sqlite 后面(driver.ts 注释有载);22.5–22.12 用户看不到提示,但本来就能走新下载路径拿到引擎,阈值也与文档「Node >= 22.5」解析顺序表述一致,不改。 结论:代码审查通过——机制正确、回归清点干净、文档同步、hermeticity 保持。唯一阻塞是上面的无关 Windows shard flake,重跑 |
|
The one red leg on the first run (windows-latest, 22, 2) was a lane-decisions-matrix flake, not this change:
If the google-lane promise-pending hang recurs on windows runners it deserves its own issue (likely the fake Gemini upstream outliving the test under load); nothing here changes its inputs. |
…e fix (#3385 / #3338 directive 3) The Node >= 22.5 upgrade lever and mirror list live only in lane logs and the web banner today — surfaces the user never sees. acp_status is the one surface the MODEL reads every session, so mirror the degrade reason line there ("relay this to the user"): STORAGE — storage: shadow (unified-first reads, dual-write) ⚠ degraded — no SQLite engine available (...) — upgrade hint (relay this to the user) Healthy storage and the intentional BILI_ENCRYPTION_KEY legacy opt-out stay quiet. Tests build a real store with engines forced unavailable (the recording seams fire exactly as in production) and pin _setStoreForTest so ccrLoopConfig cannot rebuild a healthy store mid-call and clobber the degraded state.
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.524-master.82138fa # this exact build
npm install -g billion-context@master # always the newest merged build
|
Implements owner directives 2 and 3 from #3338 (after #3348 landed directive 1, the 60 s budget):
Directive 2 — builtin multi-mirror
The engine tarball download walks a mirror list instead of a single registry:
[storage] ... mirror <base> failed (<reason>) — trying next mirror... (tried mirrors: a, b, c)— a SQLite unified storage degraded to legacy-only (bili v0.1.197, node v20.20.2, win32-x64) #3385-style box can now see WHERE the network diedDirective 3 — Node upgrade hint
On Node < 22.5 with no engine, the degrade reason and the bootstrap-failure log append:
(Rationale:
enginesallows Node >= 20, but node:sqlite ships at 22.5 — the #3385 reporter rode node v20.20.2 with no builtin and a download that could not complete, locked into a 6 h cooldown loop for days.)Tests
downloadedwithRegistrynow pins the mirror list to its single mock so the pre-mirror-era tests stay hermetic (a failing mock must fail the bootstrap instead of falling through to the real registries). All 29 engine-bootstrap tests green; storage neighbors (proofs, web-storage-banner, overview-cold-scan, write-amplification) 22/22 green; tsc clean; docs seed + regenerated tables updated (EN+ZH).Closes #3385 (dup of #3338; directive 1 already landed via #3348).