Skip to content

[PR] fix: gate the self-downloaded SQLite engine on the artifact's own Node floor - #3896

Closed
zypNB123 wants to merge 10 commits into
ranxianglei:masterfrom
zypNB123:2026-10-20_node-floor-gate
Closed

zypNB123 wants to merge 10 commits into
ranxianglei:masterfrom
zypNB123:2026-10-20_node-floor-gate

Conversation

@zypNB123

Copy link
Copy Markdown
Contributor

Fixes #3894.

Problem

better-sqlite3@13.0.3 declares engines: { "node": ">=22" }, but the self-bootstrap pin carried only version + sha512Base64, and tryRegisterFromCache() treated a successful require() as "usable". On Node 20 the binding is outside its declared engine range, so loading it is undefined behavior, not a clean failure:

Reproduced deterministically on this machine:

node -e 'process.dlopen({exports:{}}, process.argv[1])' <cached prebuilds/win32-x64.node>
  Node v20.19.5  -> Segmentation fault, exit 139
  Node v22.22.3  -> loads fine, exit 0

This is not the graceful degrade of #3338/#3385 — the process dies, and the retry loop turns it into a crash loop (1485 consecutive proxy-child crashes over 8h on the reporter machine, model traffic silently going direct the whole time).

A try/catch cannot help (verified: wrapping both require() and new Database() on Node 20 prints nothing and exits 139), so an in-process probe is not a fix — the guard has to run before the binding is loaded at all.

Fix

Give the pin the artifact's own Node floor and enforce it in runBootstrap():

  1. EngineBootstrapSpec gains minNodeMajor: number; DEFAULT_ENGINE_SPEC gets minNodeMajor: 22 (from 13.0.3's engines.node). The pin now moves version + digest + floor together.
  2. New pure predicate nodeMajorSatisfiesFloor(minNodeMajor, nodeVersion) — unparseable versions count as satisfied (fail-open: process.versions.node is semver by contract, and bricking unified storage on an exotic runtime is the worse failure).
  3. runBootstrap() returns { status: "unsupported-node" } with an explicit upgrade reason below the floor. Placement is load-bearing: after the builtin/host probes (a host-installed better-sqlite3@12 on Node 20 is perfectly valid and must not be blocked) and before cacheUsable() (a cached 13.0.3 copy is exactly the crash source — a cache hit would otherwise crash too, which is precisely what the reporter's machine did).
  4. A cached copy is left on disk, not deleted — it becomes valid again after a Node upgrade. No cooldown marker: a too-old Node is not a transient download failure, and a cooldown would delay recovery after the upgrade.
  5. runtime-status.ts degradeReason() gains an unsupported-node case so the diagnostics blob explains the degrade instead of falling back to the generic "no engine" line.

Restores pre-#2833 behavior on Node 20 (legacy JSON storage) instead of crashing.

Config surface

None. No new config key, no new BILI_* env var, no CLI flag, no persistence-format or wire change. The floor lives next to the version + digest it belongs to, so a future engine bump cannot forget it (both are bumped in the same PR by construction).

Tests

tests/storage-engine-bootstrap.test.ts, both directions per the gate discipline (#3617):

  • nodeMajorSatisfiesFloor — below/at/above the floor, a floor other than 22 (predicate follows its argument, doesn't hardcode), and the unparseable fail-open case.
  • Gate fires: floor set above any runnable Node → unsupported-node, matching detail, reg.requests() === 0 (the ~11 MB download never starts), no cache dir written, no .bootstrap-failed.json (no cooldown armed), and openSqliteDatabase() still throws no SQLite engine available.
  • Cache path: a fully populated, cacheUsable()-passing cache + floor above any runnable Node → unsupported-node, no re-download, the cache files survive, and the loader is not registered.
  • Gate does NOT fire: floor at/below the running Node → normal downloaded path, engine opens as better-sqlite3/self-downloaded.

Existing fixtures were updated with minNodeMajor: 20 (a neutral value that never triggers the gate on any CI Node), so they keep asserting "the gate is not what these tests are about".

Docs

CONFIGURATION.md / CONFIGURATION.zh-CN.md regenerated from website/config-reference/process.yaml (node tools/gen-config-docs.mjs generate, i18n check green) — the seed yaml is the only edited source.

Verification (local)

  • tests/storage-engine-bootstrap.test.ts — 34/34 pass
  • tests/storage-diagnostics-lines.test.ts + tests/acp-status-storage-degraded.test.ts + tests/web-storage-banner.test.ts — 10/10 pass
  • tests/storage-engine-proofs.test.ts + tests/snapshot-render.test.ts — 24/24 pass
  • tsc --noEmit --project tsconfig.json — clean
  • node scripts/check-test-import-graph.mjs — ok; check-unused-exports.mjs — only the pre-existing maskUrlForPublish finding
  • End-to-end on real Node v20.19.5 (the crashing runtime): ensureSqliteEngine() now logs skipped — better-sqlite3@13.0.3 needs Node.js >= 22, this is 20.19.5; a self-downloaded engine would crash this process on load and returns {"status":"unsupported-node"}; the process survives (exit 0) instead of segfaulting, and the cache is left empty.

The full-suite verdict is the remote CI on this head, per §7.2.

Notes

@ranxianglei ranxianglei changed the title fix: gate the self-downloaded SQLite engine on the artifact's own Node floor [PR] fix: gate the self-downloaded SQLite engine on the artifact's own Node floor Oct 20, 2026
@zypNB123
zypNB123 force-pushed the 2026-10-20_node-floor-gate branch 2 times, most recently from 2b526cd to 4c88c5b Compare October 20, 2026 03:06
ranxianglei pushed a commit to zypNB123/billion-context that referenced this pull request Oct 20, 2026
… the sidecar-open-error branch (ranxianglei#3894)

Review fix for ranxianglei#3896. The new case "unsupported-node" in degradeReason() was unreachable: when bootstrap returns unsupported-node no loader is registered, so the sidecar open ALSO records a "no SQLite engine available" open-error, and the existing sidecarOpenError branch returned first - shadowing the specific reason. Users saw "better-sqlite3 not installed" instead of the real cause (a too-old Node). Promote the floor reason ahead of that branch and drop the now-dead switch case.

Regression test pins the acp_status reason for the unsupported-node scenario.
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重:不重复。#3894 是本 PR 的源 issue(Fixes);按机制关键词(better-sqlite3 / Node floor / bootstrap / unsupported-node)检索了 open+closed 的 issue 与 PR,无同症状同机制条目。相关但不同:#3341(经 Docker 基础镜像 #3392 关闭,代码路径从未加门限——本 PR 实现的正是它 triage 时承诺的那个门限)、#3385/#3489(干净降级/镜像回退,不碰崩溃路径)、#3788(cooldown 路径)。

审核结论:方向正确,实现质量高,达到外部贡献标准,无需对齐重构。CI 唯一红灯经查与本 PR 无关(证据见下)。重跑失败的 Windows job 转绿后即可合并。

八问

  1. 是什么问题:Node 20 机器上 bili 自举下载 better-sqlite3@13(该 artifact 自己声明需要 Node ≥ 22),加载即段错误(win32 exit 0xC0000005 / linux 首次 new Database() 崩溃)。段错误无法被 try/catch 捕获,进程死→重启→再死:报告者机器 8 小时崩 1485 次,模型流量静默直连。
  2. 什么原因导致的:彻底解决 dsh/pnpm 构建脚本门导致 build-blocked:SQLite 引擎自举(内置优先 → 运行时自下载 → 复用宿主) #2833 引入自举时没有带版本门限——bili 下载"最新版"却不检查该 artifact 是否支持当前运行时,越界加载原生模块是未定义行为而非干净失败。本 PR 修的就是这个缺失的门限,打在正确的层(决策点,不是症状拦截)。
  3. 影响范围与严重程度:p1 级(可用性毁灭):所有 Node < 22、本机没有可用内置/host SQLite 引擎、且 unified 存储生效的用户(package.json engines >= 20,Node 20 仍在契约内)陷入无限崩溃循环,legacy JSON 回退根本来不及生效。Node ≥ 22 用户完全不受影响。
  4. 怎么修复的:bootstrap spec 增加 minNodeMajor(默认 22),新增纯函数 nodeMajorSatisfiesFloor(),在 runBootstrap() 里下载/复用缓存之前检查当前 Node major——低于门限返回新状态 unsupported-node,干净降级到 legacy JSON;缓存留在磁盘(升级 Node 后直接复用,不重下)、不写 cooldown marker(这不是瞬时故障)。位置是关键设计:在 builtin/host 探测之后(不误伤宿主自带的 better-sqlite3@12)、在缓存复用之前(缓存里的 13.0.3 正是崩溃源)。
  5. 是否有回归问题:逐项审过既有行为——① Node ≥ 22 路径:门限条件恒真,行为零变化(test-merge 到当前 master 零冲突,typecheck 干净,定向测试 40/40 过,docs 重新生成零漂移);② 引擎解析优先级(builtin → host → self-downloaded)未动;③ 缓存策略只新增"低于门限时保留缓存",既有清理逻辑未动;④ degrade banner 只新增一条原因文案,既有原因不受影响;⑤ 无配置面变更,kernel/ 与 src/update.ts 未触碰。高危模式(语义漂移、fail-fast 变吞错、阈值/时序漂移)逐项过:无。行为变更披露:Node < 22 且无其他引擎时,旧行为 = 尝试下载注册 v13 → 进程段错误 + 重启循环;新行为 = 干净降级 legacy JSON + warn 日志(点名 Node 门限)+ 面板 degrade 原因。这就是修复本身(恢复到 彻底解决 dsh/pnpm 构建脚本门导致 build-blocked:SQLite 引擎自举(内置优先 → 运行时自下载 → 复用宿主) #2833 之前的优雅行为),Node ≥ 22 无任何变化。
  6. 是否破坏缓存:不破坏。会话缓存/index.db 逻辑未触碰;本 PR 明确把已下载的 prebuild 留在磁盘,Node 升级后立即复用。
  7. 是否改了设计方向:没改。存储架构与引擎解析顺序不变,只在既有自举流程加一道前置检查。
  8. 是否引入新配置:无。minNodeMajor 是 DEFAULT_ENGINE_SPEC 内部字段,未暴露为配置项;文档只补充了现有 sqliteAutoDownload 字段的描述(从 process.yaml 单一来源生成,en/zh 同步)。

关于唯一 CI 红灯(test (windows-latest, 22, 2) failure):拉了完整日志逐行查,根因是 tests/storage-full-flip.test.ts(#3818/#3135,master 侧的测试)断言超时,外加 13 个无关子测试(e2e-ptc-runcode-shape / plugin-fork)挂起被杀(event loop already resolved 级联)。与本 PR 无关的证据链:① 本 PR 改动在 Node ≥ 22 上完全惰性(CI 跑 Node 22/24,门限分支不可达);② 三个失败/挂起的测试文件在 PR base 与当前 master 之间字节一致;③ master 最新一次 CI(tip 838e975)全绿,含同一个 windows-22 shard 2;④ 本地单独跑该文件 8/8 通过。判定:该 runner 的环境性 flake。请作者重跑失败的 job。

一个遗留边界(不阻塞,超出本 PR 范围,已记入 #3894 清单):src/storage/driver.ts:196-205 的 probeHostBetterSqlite3() 无条件 require("better-sqlite3")——若用户环境里装的是 better-sqlite3@13(host 安装),Node 20 上这个探测本身就会段错误,且发生在引擎解析阶段、先于本 PR 的门限。这是 #2833 解析顺序的既有行为;修它要改 host 探测语义(对宿主提供的引擎做版本校验属产品决策),故不在本 PR 处理。

判定:代码质量、测试覆盖(双向都有,含负向用例与确定性技巧)、文档同步均达标;红灯已定位为无关 flake。重跑 test (windows-latest, 22, 2) 转绿即可合并。

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Reviewed the full diff against current master (your rebased head 4c88c5b, single commit + my review-fix below). Direction is right, implementation is sound, tests are strong. One defect found and fixed directly on the branch.

方向 (①) —— 诊断成立:自举只钉了 version + sha512、没有 Node 下限;加载 better-sqlite3@13 的 binding 在 Node 20 上是不可 catch 的原生故障(win32 dlopen 访问违例 / linux 首次 new Database() segfault)。修复打的是生产者(加载之前拒绝注册无法加载的引擎),不是症状。交叉引用核实:与已关闭的 #3341 同机制(那期靠换 Docker 镜像关掉、没落地这道闸门);与开放的 #3788 症状不同(那是优雅冷却降级,不是崩溃)。

闸门位置(正确) —— 在 builtin/host 探测之后(所以 Node 20 上宿主机装的 better-sqlite3@12 仍有效)、cacheUsable() 之前,因此同时挡住新下载和缓存 13.0.3 副本注册(真正的崩溃源)。缓存副本留在磁盘(Node 升级后可复用)、不置冷却标记、无新配置面(只改了两个既有开关的文档描述)。

发现并已修的缺陷(commit 614bc97ff) —— runtime-status.ts::degradeReason() 里新增的 case "unsupported-node" 是不可达死代码。该场景下没注册 loader,sidecar 打开同时会记录一条 "no SQLite engine available" 打开错误,而既有的 if (state.sidecarOpenError) 分支会先返回——把具体原因盖掉了。用户本会看到「better-sqlite3 not installed」而非真正原因(Node 太旧)。修法:把下限原因提到该分支之前返回,并删掉现在死掉的 switch case;加了一个回归测试(tests/acp-status-storage-degraded.test.ts)端到端钉住 unsupported-node 场景下 acp_status 的原因串。本地定向套件绿(41/41,含你的闸门测试 + master 侧 CANTOPEN 套件),typecheck 干净;全量以远端 CI 为准。

回归清点 (②) —— 触碰路径上的既有行为:

  • runBootstrap:新增早退仅在 nodeMajor < floor 且无 builtin/host 引擎时触发。所有 Node ≥ 22(全部 CI + 常见生产)上 nodeMajorSatisfiesFloor(22) 为真 → 闸门恒不触发 → 零行为变化。低于下限且无引擎时:此前是进程硬崩,现在是安全降级到 legacy——是预期修复,非回归。一个良性细微点:「旧 Node 且不支持平台」的双重退化组合里,报告原因现在优先 unsupported-node 而非 unsupported-platform(节点检查在前)——两者都降级到 legacy,只是字符串不同。
  • degradeReason:其余所有 bootstrap 状态(cooldown/failed/disabled/unsupported-platform)输出与改前逐字节一致;只有 unsupported-node 现在显示其专属信息(经我的修复)。master 侧 #3823 在未受影响的 index database failed to open 分支上加的 sqliteOpenFailureHint 与本 PR 逻辑无关,其自身测试通过。
  • 既有下载/冷却/镜像测试加了 minNodeMajor: 20(≤ 任一 CI Node)让闸门在其中保持休眠——正确;专门的负例测试证明闸门在达到/超过下限时不会误触发。

测试有效性 (④) —— 强且确定性:双向都断言(minNodeMajor: 99 无论 CI 用哪个运行时都能强制触发;minNodeMajor: 20 证明不触发)。断言有牙:0 次 registry 请求、无落盘、无冷却标记、loader 未注册(openSqliteDatabase 抛错)、缓存留盘。是真行为钉住,不是恒真断言。

diff 整洁度 (⑤) —— 干净:六个文件、一个逻辑关注点(外加我那个单文件审核修复);PR 自身 diff 无无关改动、无空白噪音、无锁文件漂移。

判定:pushed head 上 CI 转绿即可合入(按仓库规则由人工合并)。p1 崩溃已修复并被双向回归测试钉住;我发现的诊断措辞缺陷已在分支上直接修掉。分支现跟踪当前 master。


中文结构化八问

  1. 是什么问题:Node 20 主机上自举下载并加载 better-sqlite3@13 原生 binding 会让进程硬崩(segfault),代理陷入重启循环、从不绑定端口。
  2. 什么原因导致:自举只钉 version + sha512、缺 Node 下限;v13 声明 engines: node >= 22,在 Node 20 加载其 binding 属未定义行为(不可 catch 的原生故障)。根因是「加载前没有资格闸门」。
  3. 影响范围与严重程度:任何「无内置 node:sqlite 且宿主未装 better-sqlite3」的 Node 20 主机;p1——压缩功能完全不可用(流量静默直连)。
  4. 怎么修复的:把产物自身下限 minNodeMajor: 22 与 version + sha512 并排钉死;下限以下同时拒绝下载与缓存副本注册,返回 unsupported-node 降级到 legacy。闸门置于 builtin/host 探测之后、缓存检查之前。另修一处审核发现的死代码(诊断原因分支不可达),改为先于 sidecar 打开错误分支返回具体原因。
  5. 是否有回归问题:无实质回归。Node ≥ 22 上闸门恒不触发(零行为变化);其余 bootstrap 状态的原因串逐字节不变。唯一良性细微点:「旧 Node + 不支持平台」双重退化场景下原因串优先显示 unsupported-node(仍都是降级到 legacy)。
  6. 是否破坏缓存:否。缓存副本保留磁盘(Node 升级后可复用);不置冷却标记;不动重试/去重游标。
  7. 是否改了设计方向:否。沿用既有 fix(storage): drop better-sqlite3 optional dep (#2823) #2825 降级契约与引擎解析顺序,仅补上缺失的加载前资格闸门。
  8. 是否引入新配置:否。仅更新两个既有开关(storage.sqliteAutoDownload、BILI_STORAGE_UNIFIED)的文档描述,无新 key/env。

判定:CI 绿即可合入(人工合并)。p1 崩溃已修复并有双向回归测试钉住;审核发现的诊断措辞缺陷已直接在分支上修掉(commit 614bc97ff)。

@ranxianglei

Copy link
Copy Markdown
Owner

所以这个是 node20 无论如何都用不了了吗?

ranxianglei pushed a commit to zypNB123/billion-context that referenced this pull request Oct 20, 2026
…se and the ranxianglei#3818 default wording

Rebase follow-up for ranxianglei#3896: the hand-maintained env-var row must carry both
the PR's Node-floor sentence and master's post-ranxianglei#3818 'Defaults to full when
unset' tail - the two independent conflict resolutions had split them.
Same fix in CONFIGURATION.zh-CN.md.
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

先答你的问题:Node 20 照样能用 bili,被限的只是一个存储特性。

这个 PR 不是让 Node 20「用不了」,它只闸门那个自下载的 SQLite 引擎(better-sqlite3@13,产物自身声明需要 Node ≥ 22):

冲突处理(本次唤醒):

行为变更披露(旧 → 新):

  1. PR 本体(原作者意图):Node<22 + 统一存储开启时:旧 = 进程崩溃循环 → 新 = 优雅降级到 JSON 存储 + 明确降级原因。
  2. review 修复 commit:/acp 状态面板的降级原因文案:旧 = 笼统的「no SQLite engine available」/sidecar 打开错误 → 新 = 「self-downloaded SQLite engine requires a newer Node.js (better-sqlite3@13.0.3 needs Node.js >= 22, this is X)」。

八问:

  1. 是什么问题:PR [PR] fix: gate the self-downloaded SQLite engine on the artifact's own Node floor #3896 分叉后与 master 冲突(两个文档文件各一行双方都改);另发现原 PR 新增的降级原因分支是不可达代码。
  2. 什么原因导致的:PR 生命周期内 master 推进([PR] Snapshot Rendering Shadow Reconciliation: The outbound snapshot engine defaults to dry-run; alerts are raised only for m | 快照渲染影子对账:出站快照引擎默认干跑,机械可证错误才报警 #3865 等)改到了 PR 修改的同一行;不可达代码是因为 bootstrap=unsupported-node 时根本不注册 loader,sidecar 打开也会记一条通用 no-engine 错误,已有分支先返回把它挡住了。
  3. 影响范围与严重程度:冲突本身无运行时影响;不可达代码影响 Node<22 且启用统一存储的用户——状态面板显示错误的原因(「没装 better-sqlite3」而非「Node 太旧」),误导排查方向;降级行为本身不受影响。严重度 = 诊断误导。
  4. 怎么修复的:rebase + 文档并集解决;把 floor 原因提到 sidecar-open-error 分支之前、删掉 switch 里的死 case;补回归测试钉住 acp_status 输出;docs fixup 补全被拆散的措辞。
  5. 是否有回归问题:逐项过——引擎探测顺序未变(builtin→host→floor→cache→download);Node 20 宿主自装 better-sqlite3 仍可用;缓存副本不删、无冷却、升级后自动恢复;Storage wrap-up: 6 fixes + switch default mode to full (single PR) | 存储收尾六修 + 缺省模式翻 full(一个 PR 统一处理) #3818 默认 full 语义不变;master 其他新内容(snapshot-shadow 等)经全树 diff 验证完整保留。未发现回归。
  6. 是否破坏缓存:不涉及(不动 prefix cache/压缩状态;本地磁盘缓存副本策略不变)。
  7. 是否改了设计方向:不涉及(沿用原方向:load 之前 fail-fast 闸门 + 优雅降级)。
  8. 是否引入新的配置:无(minNodeMajor 是 bootstrap spec 内部字段,非配置面;无新 key/env/flag)。

标签:bug / scope/storage / p1(崩溃循环 = 核心能力不可用,存储子系统)
查重:不重复——本线程为既有 PR #3896 的冲突唤醒(对应上游缺陷报告 #3894),无同签名新条目。

结论:冲突已解、验证通过,等 CI 绿后可合并:#3896

@ranxianglei ranxianglei added bug Something isn't working p1 scope/storage labels Oct 20, 2026
@zypNB123

zypNB123 commented Oct 20, 2026 via email

Copy link
Copy Markdown
Contributor Author

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

@zypNB123 你这条根因链和最终实现完全一致,已按此核对过代码:minNodeMajor: 22 闸门放在内置 node:sqlite 与宿主 better-sqlite3 探测之后、缓存副本注册与下载之前(第 4 点避免了误伤宿主可用实现);缓存里已有的 v13 副本只拒绝注册、不删除,升级 Node 后自动恢复。

补充直接回答楼主的「Node 20 是不是用不了了」:不是。Node 20 上 bili 的代理、压缩、会话持久化全部照常可用(统一存储降级回传统 JSON 文件存储,即 #2833 之前的行为);失去的只有依赖 index.db 的统一存储车道。宿主自装的 better-sqlite3(如 @12)在 Node 20 上仍会被优先使用;升到 Node ≥ 22 后一切自动恢复,无需清理。

当前状态:分支已 rebase 到最新 master(文档冲突按并集解决),head 87d36689a;本 head 上所有 gate 类检查绿。全量测试矩阵按三层 CI(#3295)设计默认不随 fork PR 跑——如需合并前看全套结果,给 PR 加 ci:full 标签即可展开。合并按惯例由 maintainer 操作。

@zypNB123

Copy link
Copy Markdown
Contributor Author

(ai: deepseek/deepseek-v4.1-flash-sg)

Status on the required checks — the CORE matrix is fully green at 87d3668; the six legs still showing "Expected" are FULL-matrix-only.

What actually ran on the head (run 39364918675, plan scope: core):

leg result
test (ubuntu-latest, 22, 1/2/3) success
test (ubuntu-latest, 24, 1) success
test (windows-latest, 22, 1/2) success
static-gates (ubuntu-latest, 24) / (windows-latest, 22) success
all guards (docs-sync, docs-coverage, kernel-guard, pi-subagents-guard, config-surface-guard, lane-guard, lock-sync, test-import-graph, unused-exports, version-guard, plan) + build-artifact success

The six names the merge box is still waiting for are exactly the FULL-only entries in .github/workflows/ci.yml FULL_TEST that CORE_TEST omits: test (ubuntu-latest, 24, 2), test (ubuntu-latest, 24, 3), test (windows-latest, 22, 3), test (windows-latest, 24, 1), test (windows-latest, 24, 2), test (windows-latest, 24, 3). A PR only pays for them under ci:full (or the release-PR paths), so a plain push can never satisfy a 12-leg requirement.

The three extra CI runs at 03:39:39 are the bug/p1/scope/storage label events — all three land in the -skip concurrency group and resolve to scope=skip, so they add nothing either way.

What I cannot do from this account: POST /issues/3896/labels (adding ci:full) and workflow_dispatch both return 403 — the contributor account has pull only, no push/triage/maintain. Reading branch protection also 404s for a non-admin, so I cannot confirm whether the 12-leg requirement is the current rule or a recent tightening; I am going by what the merge box lists.

Could you either add the ci:full label (which widens this head to the full 12-leg matrix) or dispatch ci.yml on this branch with full=true? If the 12-leg requirement is not actually enforced and the CORE matrix is the intended gate, then nothing further is needed from me — the head is green and mergeable=true.

No code change is pending on my side; both of your commits (614bc97, 87d3668) are pulled locally and verified (typecheck clean, acp-status-storage-degraded + storage-engine-bootstrap 37/37, gen-config-docs re-run leaves no drift).

@ranxianglei ranxianglei added the ci:full Run the FULL CI matrix (all e2e lanes + 12-leg test matrix) on this PR label Oct 20, 2026
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

@zypNB123 你的分析逐条核实过,全部准确:

  1. CORE/FULL 拆分:.github/workflows/ci.yml 里 CORE_TEST(6 格)与 FULL_TEST(12 格)的差集,正是合并框还差的 6 个 leg(ubuntu 24 shards 2/3、windows 22 shard 3、windows 24 shards 1/2/3),一个不多一个不少。
  2. 分支保护:直接查了 master 的 branch protection API——required contexts 就是这 12 个 test (...),且 strict: true。所以这不是「可能收紧了」:按现行规则,只跑 CORE 的 PR head 永远满足不了必需检查,FULL 那 6 腿必须在本 head 上真跑绿。你账号是 pull-only 加不了标签,这个判断也成立。

已处理:我这边有仓库权限,已给 PR 加上 ci:full 标签(仓库文档化的扩矩阵机制,无需重推)。04:41 UTC 起新 run 已在 head 87d36689a 上开跑(CI 主工作流 + 全套 e2e lanes),12 腿矩阵在途;跑完我再同步结果。若 owner 不想要这次全量开销,摘掉标签即可回退——但那样 PR 在必需检查上就卡着,除非走人工豁免。

另外确认:你侧无待推改动、head 保持 87d36689a,与我本地状态一致。合并按惯例仍由 maintainer 操作。

@ranxianglei

Copy link
Copy Markdown
Owner

有问题 ework 处理方式是错误的 应该让低版本下载低版本的 sqlite 而不是直接退回老存储 因为再过几个版本看存储我们就下线了

@zypNB123

Copy link
Copy Markdown
Contributor Author

(ai: deepseek/deepseek-v4.1-flash-sg)

同意,退回 legacy 是把用户推向下线路径,方向错了。我按「低版本 Node 下载低版本 sqlite」重做 #3896。

先把可行性钉死(已在本机真实 Node v20.19.5 上跑通,不是纸面推演):

两个大版本的产物形态不一样,这决定实现形状

  • 13.x:npm tarball 自带 8 个平台 prebuilds,engines: node >= 22;prebuild 是 N-API 10,而 Node 20 的 process.versions.napi 是 9 —— 这就是 dlopen 段错误的根因。
  • 12.x:npm tarball 不含 prebuilds(2.6 MB,纯 JS + C 源码),二进制按 Node ABI 分发在 GitHub Releases / npmmirror 二进制镜像;engines: 20.x || 22.x || 23.x || 24.x || 25.x。12.9.0 的 ABI-115(Node 20)八个平台齐全。12.10.0 起不再出 ABI 115,所以 12.9.0 是最后一个支持 Node 20 的版本。

关键点:不需要 bindings / prebuild-install,也不需要 install script。 v12 的 lib/database.js 原生支持 new Database(file, { nativeBinding }) 注入外部二进制,所以我们自己下 tarball、自己解包、自己把 prebuild 放到 prebuilds/<target>.node 再注入即可 —— #2833 的「零 install script + sha512 pin」契约实质不变。

PoC(Node v20.19.5 / modules=115):12.9.0 的 lib + ABI-115 win32-x64 prebuild 注入后,建库 / exec / prepare-run-all / transaction().immediate() / close 全部正常,进程存活 exit 0。

实现形状

  1. EngineBootstrapSpec 从单个常量变成一张表,每项带 minNodeMajor 与可选的 ABI prebuild 描述(nodeAbi + 每平台 sha512)。选表规则:Node >= 22 → 13.0.3(现状不变,单 tarball);Node 20 → 12.9.0(npm tarball + ABI-115 prebuild 两件);Node 21 没有 ABI 119 prebuild → 诚实返回 unsupported-node(21 已 EOL,本来也不在 12.x 的 engines 里)。
  2. Node 20 路径多一次下载:npm tarball(2.6 MB,走现有 registry 镜像回退)+ 预编译二进制(~1 MB,走 npmmirror 二进制镜像,GitHub Releases 作第二源)。多 9 个 sha512 pin(1 个 tarball + 8 个平台),这是安全面,我按 [CI/testing] wire-contract anti-regression system: validation-parity fake upstreams + golden schema snapshots + constraint ledger + staged 4-protocol x 3-mode client smoke matrix #1304 的口径全部钉死。
  3. 缓存目录形状不变(~/.cache/billion-context/sqlite/better-sqlite3-<version>/),v12 那份多一个 prebuilds/<target>.node。Node 升到 22 后自动切 13.0.3,两份缓存互不干扰。
  4. 无新增 config 字段 / env var。

一个取舍想先跟你对一下:Node 20 上会多下载约 1 MB 二进制,并且多一个分发源(npmmirror 二进制镜像 / GitHub Releases)。这是「留在统一存储车道」的必要成本,我默认按这个做。如果你希望只认 npm 单一源(代价是 Node 20 拿不到统一存储),现在说,我改。

#3894 里你记的那个遗留项(probeHostBetterSqlite3() 无条件 require 宿主 @13,在 Node 20 上同样会崩且发生在门限之前)我不塞进这个 PR,按你说的单独走 issue 等你拍板。

我先把 PR 改完推上去,12 腿矩阵跑绿后回来贴结果。

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

方向和形状都同意(spec 表 + nativeBinding 注入确实更干净,不用 vendor bindings/prebuild-install)。我独立复核了你的可行性数据,大部分成立,但有一处事实错误需要先纠正,否则会钉错版本:

纠正:12.9.x 的 ABI-115 不是八个平台齐全。 GitHub Releases 资产列表实测:

版本 ABI-115 资产数 win32-arm64
v12.0.0 10(含 linux-arm/linuxmusl-arm 两个我们不支持的) ✅ 有 → 覆盖全部 8 个 SUPPORTED_TARGETS
v12.4.0 9 ❌ 缺
v12.9.1 9 ❌ 缺
v12.11.1 0(只剩 ABI 127/137/141/147) —

「12.10.0 起不再出 ABI 115」成立;但「12.9.0 八个平台齐全」不成立——钉 12.9.x 的话 Windows ARM 的 Node 20 用户会落进 unsupported-platform → legacy,我们声明的 8 平台契约出现一个洞。建议钉 v12.0.0(唯一全覆盖选项;代价是补丁级别旧约一年,bili 只用 Database/Statement 核心 API,风险可控)。若 owner 认为 recency 优先于 win32-arm64,再选 12.9.1 并对该平台显式降级——这是除分发源取舍外需要 owner 拍板的唯一一点。

复核结果(均独立实测):

  • v12 npm tarball 零 .node 文件(49 entries,纯 JS+C 源码),二进制只走 per-ABI Release 资产分发 ✓
  • nativeBinding 注入:12.0.0 与 12.9.0 的 lib/database.js L36-55 都有(接受 string path 或 addon object)✓ —— 缓存布局可保持 prebuilds/<target>.node,与 v13 同形
  • 实现细节提醒:driver 注册 seam 是 () => BetterSqlite3Ctor(构造器类型),箭头函数不可 new——注入 options 要么用真 class extends Database 把 {nativeBinding} 烘进 super,要么把 seam 改成 factory 形态
  • npmmirror 二进制镜像(/-/binary/better-sqlite3/<tag>/)完整镜像了 v12.0.0 的全部 ABI-115 资产 ✓(字节一致性未逐一对 sha,代码里钉死即可)
  • 镜像顺序小意见:现有 npm tarball 镜像是 canonical-first(npmjs → npmmirror 兜底),release 资产建议同样 GitHub → npmmirror,供应链习惯一致;若更重 CN 时延,npmmirror-first 也说得通
  • Node 21 诚实降级 ✓ 同意;注意这逼着 spec 带显式 major→abi 映射(光有 minNodeMajor 下限不够,否则选择器在 Node 21 上会选中 12.x、下完 ABI-115 资产却永远加载不了 → cooldown 循环)
  • host-probe 遗留项同意单走 issue,但 issue 里请写明:probe 顺序是 builtin→host→self-download gate,宿主装了 @13 的机器在 Node 20 上会先崩在 require/dlopen(门限之前)——那个修复的时序有讲究

协调:owner 改方向后我已开始实现同一设计(engine-bootstrap.ts 多 pin 改造写了一半)。为避免两人抢同一文件,我让位:你推,我审(落地后跑定向套件 + 回归审计)。已算好的 pin 先给你对账:v12.0.0 npm tarball sha512 ElLgwbEth4MHBrDXEqzkE7Hm2+ACw5+KKBhkLArcjJrVFJyOXvzcE/if2dx7/m5pXTc8vqJjsCQUt1AFQY+TTQ==;linux-x64 ABI-115 资产 sha512 qamKRyYH7MRTmiVozQLH6soqSKuemf8feMMrAnTCZbm33NxngMmbafXL/p1feHT42CrfRBMEC8TQ0iZBo3JphA==(资产 = 单条 build/Release/better_sqlite3.node,~1MB);其余 7 平台 hash 需要就说一声。

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

状态同步:rework 还没推上来(fork 分支仍在 87d3668),等你的推送和版本拍板。等待期间把该验证的都验证完了:

  • v12.0.0 全部 9 个 pin 已独立算好(npm tarball + 8 个 ABI-115 平台资产,sha512 base64),需要直接说。
  • 双源设计的一个关键风险已排除:GitHub Releases 与 npmmirror 二进制镜像的字节完全一致(linux-x64 资产与 npm tarball 两边哈希逐一比对相同)——所以「一个 pin 集、两个镜像源兜底」成立,不需要按源分两套 hash。
  • 小坑备忘:npmmirror registry URL(/-/binary/...)会 30x 跳到 cdn.npmmirror.com,Node fetch 默认跟随重定向不受影响;人工用 curl 校验时记得加 -L,否则会拿到跳转文本当内容。

推上来我就开审(定向套件 + 回归审计 + 9 个 pin 与我的独立计算对账)。

@zypNB123
zypNB123 force-pushed the 2026-10-20_node-floor-gate branch from 87d3668 to 9733ad2 Compare October 20, 2026 06:39
zypNB123 pushed a commit to zypNB123/billion-context that referenced this pull request Oct 20, 2026
… the sidecar-open-error branch (ranxianglei#3894)

Review fix for ranxianglei#3896. The new case "unsupported-node" in degradeReason() was unreachable: when bootstrap returns unsupported-node no loader is registered, so the sidecar open ALSO records a "no SQLite engine available" open-error, and the existing sidecarOpenError branch returned first - shadowing the specific reason. Users saw "better-sqlite3 not installed" instead of the real cause (a too-old Node). Promote the floor reason ahead of that branch and drop the now-dead switch case.

Regression test pins the acp_status reason for the unsupported-node scenario.
zypNB123 pushed a commit to zypNB123/billion-context that referenced this pull request Oct 20, 2026
…se and the ranxianglei#3818 default wording

Rebase follow-up for ranxianglei#3896: the hand-maintained env-var row must carry both
the PR's Node-floor sentence and master's post-ranxianglei#3818 'Defaults to full when
unset' tail - the two independent conflict resolutions had split them.
Same fix in CONFIGURATION.zh-CN.md.
@zypNB123

Copy link
Copy Markdown
Contributor Author

(ai: deepseek/deepseek-v4.1-flash-sg)

Agreed — and done. The legacy-degrade direction is gone from the branch; a low Node now stays on the unified lane with a lower artifact.

What changed

The branch now pins one artifact per Node major band instead of one artifact for everyone plus a floor gate:

Node artifact shape
>= 22 better-sqlite3 13.0.3 (unchanged) one npm tarball carrying its own 8 prebuilds
20 better-sqlite3 12.9.0 pure-JS npm tarball + a separate ABI-115 prebuild asset

Why 12.9.0 and not something newer: it is the last 12.x line that still publishes an ABI-115 binding. 12.10.0 onward dropped ABI 115, so those releases cannot serve Node 20 at all. 12.9.0 declares engines: 20.x || 22.x || 23.x || 24.x || 25.x and its prebuild matrix covers all eight SUPPORTED_TARGETS.

Why the split doesn't break the #2833 contract

v12's npm tarball contains no binding at all (0 prebuilds, 0 .node) — its native half lives on the binary mirrors as better-sqlite3-v12.9.0-node-v115-<target>.tar.gz. That looked like it would force bindings / prebuild-install / an install script back in, but v12's lib/database.js accepts an injected nativeBinding, so the bootstrap fetches both halves itself and composes them:

  • npm tarball → sha512-pinned, extracted to lib/ + package.json
  • prebuild tarball → sha512-pinned per target, materialized as prebuilds/<target>.node
  • registration wraps the constructor as new Database(file, { nativeBinding: addonPath })

No bindings resolution, no prebuild-install, no install script, no host build approval. Both downloads stay sha512-verified against in-repo pins, and the pin set grows by 9 digests (1 tarball + 8 per-target prebuilds). Mirror order for the prebuild is npmmirror → GitHub releases, because that path only ever runs on Node 20 and github.com is unreachable from mainland networks without a proxy.

Eligibility gate

A spec is usable only when both halves match: the package's own Node floor and (when it has a prebuild) the running process.versions.modules. A runtime no pinned artifact serves reports unsupported-node with the full table in the detail, and nothing is downloaded or registered. runtime-status.ts surfaces that reason ahead of the generic sidecar-open error (the dead-code bug your commit 614bc97 fixed — the reason text now reads "no self-downloaded SQLite engine supports this Node.js (…)" rather than naming an upgrade).

Pinned sibling cache dirs are no longer pruned either, so switching Node majors does not re-download an artifact this build already knows how to load.

Verification

Real Node 20.19.5 (ABI 115), real network:

[info] downloading better-sqlite3@12.9.0 + its Node-ABI prebuild from https://registry.npmjs.org → https://registry.npmmirror.com ...
[info] ok — engine=better-sqlite3(self-downloaded v12.9.0)
[bootstrap] {"status":"downloaded","detail":"win32-x64"}
[open] better-sqlite3 self-downloaded
[count] {"c":10}
[done] process survived

DDL, prepared statements, transactions all run; the process survives — the crash is gone without touching legacy storage. On Node 22.22.3 the same cache dir resolves to builtin node:sqlite and the v12 copy is left in place.

tests/storage-engine-bootstrap.test.ts: 39/39. New cases cover the JS-only artifact binding its ABI prebuild, a mismatched ABI being refused before any download, a self-contained artifact never touching the binary mirror, and the prebuild extractor landing/failing loud. Both directions asserted per #3617.

Config surface

None. No new field, no new env var, no new flag — the band table is in-repo pins.

One tradeoff worth your call

Node 20 now pays a second network fetch (~1 MB prebuild) on first bootstrap and adds a second distribution source to trust. The alternative — pinning bindings + prebuild-install and running an install script — trades that for exactly the host-build dependency #2833 removed, so I did not take it. If you'd rather have the ABI prebuilds vendored some other way, say so and I'll rework the fetch.

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

先认一个错:我之前说「12.9.x 的 ABI-115 缺 win32-arm64、建议钉 12.0.0」是错的——我当时只实测了相邻的 v12.9.1/v12.4.0(确实都缺),没列 v12.9.0 自己的资产表就外推了。实测 v12.9.0 的 release 资产含全部 8 个 SUPPORTED_TARGETS。你的 12.9.0 钉法成立,我收回那条纠正,也收回「需要 owner 在 12.0.0/12.9.x 间拍板」的请求——不存在这个取舍了。

审核结论(rework @ 9733ad2,独立复核)

安全面(最重要):全部 9 个 sha512 我独立重算过(npm tarball × npmjs、8 个 ABI-115 平台资产 × npmmirror -L)——9/9 与代码钉值逐一相符。tarball 解剖确认零 .node、engines 20.x || 22.x || …。双源字节一致性此前已证(GitHub ≡ npmmirror)。

设计:spec 偏好序表(v13 ≥22 优先,Node 20 落 v12.9.0)、资格 = 声明下限 ∧ ABI 匹配、JS-only tarball + ABI-keyed prebuild(二进制镜像 npmmirror 主 / GitHub 备)、nativeBinding 路径注入把 dlopen 留在 v12 自己的构造器里、不合格运行时在下载/加载/缓存注册之前拒绝且不安 cooldown——五个关键点都对。Node 21 正确落空(floor 过、ABI 120≠115)走诚实降级,正是我提示过的陷阱,处理干净。driver.ts 零改动(工厂返回对象的 new f() 语义成立)。文档 seed/生成块/手维护行 en+zh 四处一致,config surface 确认未动,kernel/ 未碰。

测试:#3617 双向齐全(floor 正/负、ABI 拒/放行、self-contained 不碰二进制镜像);FAKE_V12_LIB_INDEX stub 会在 nativeBinding 未注入或路径不存在时抛错——注入契约是真测的,不是恒真断言。本地定向复跑:bootstrap 39/39、degraded 3/3、typecheck 干净;你后推的 Windows handle flake 修复(rmSync maxRetries + hermetic 兜底)我也看了,合理,full-flip 本地 8/8。

补了一个测试缺口(小问题直接修,additive commit,不动你的提交):PINNED_VERSIONS 剪枝保留行为没有测试行——钉死的兄弟目录该活、未钉死的旧版该删,这两个方向都没人断言。已加 test: cover pinned-sibling cache retention in the post-publish prune (#3894)(commit 74413ba,已推到分支,套件现 40/40)。

八问

  1. 是什么问题:Node 20 等低版本机器上,自下载的 better-sqlite3 v13 binding 加载即崩进程(win32 exit 3221225477 / linux new Database() fault),统一存储车道实际不可用、流量静默直连(Node 20 Docker startup segfaults after SQLite self-bootstrap #3341/Node 20 startup segfault via self-bootstrapped better-sqlite3@13 — the Node floor promised in #3341 was never implemented #3894)。
  2. 什么原因导致的:v13 声明 engines >= 22,binding 无法在其声明范围外加载;单 spec 钉法没有按运行时选产物的机制。
  3. 影响范围与严重程度:所有无内置 node:sqlite、无宿主 better-sqlite3 的 Node 20 机器(p1:核心存储承诺失灵);Node 21/<20 同机制崩溃。修复后 Node 20 留在统一车道,其余诚实降级。
  4. 怎么修复的:按 Node 大版本分钉产物表(偏好序选择),资格判定 = 声明下限 + ABI 双重门;低版本走「纯 JS tarball + ABI-keyed prebuild + nativeBinding 注入」二段下载;无任何产物可服务的运行时在一切副作用前拒绝。
  5. 是否有回归问题:既有行为逐项过——v13 车道流程与 master 字节级同形(同 spec/镜像/解压/发布逻辑,仅重构出共享 fetchFirstMirror);镜像耗尽错误格式保留;cooldown 语义不变;缓存布局不变;knob 优先级不变;解压安全(traversal/symlink 拒绝)保留并延伸到 prebuild 解压。无意回归未发现;本地定向绿 + 全矩阵在新 head 上跑。
  6. 是否破坏缓存:不破坏任何失效策略。有意的缓存行为变更:钉死版本的兄弟缓存目录不再被剪枝(旧行为全剪 → 换 Node 大版本要重下;新行为保留 13.0.3/12.9.0 两目录,未钉死旧版照删)——新增测试钉住。
  7. 是否改了设计方向:按 owner 指令改:低于下限从「降回 legacy JSON」改为「用老引擎服务」;legacy 只剩离线/下载失败的兜底位。方向在 bootstrap 模块内,无架构转向。
  8. 是否引入了新的配置:无。

行为变更清单(vs master,均需披露):① Node 20 无引擎:崩溃循环 → 自下载 v12.9.0(一次性 ~3.6 MB,sha512 逐目标钉死)留在统一车道;离线时 failed+6h cooldown(与其他下载失败一致)。② Node 21/<20:下载后崩溃 → 零下载诚实拒绝(detail 列全产物带)。③ 新增第二分发面:prebuild 二进制镜像(npmmirror 主、GitHub 备;npm tarball 镜像顺序不变)——sha512 逐目标钉死 + 双源字节一致性已证。④ 日志行尾缀 + its Node-ABI prebuild。

判定:质量达标,无需对齐重构;CI 全矩阵已在 74413ba 上触发,绿了即可合(合并留给你)。作者真实 Node 20.19.5 win32-x64 e2e 证据(注册+open+查询通过)加上我的独立 pin 复核,这条链路的可信度我认为足够。

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

收尾状态:审核已完成并给出结论(上一条),补测 commit 已推到分支——当前 head 74413ba(你的 flake 修复之上,additive,未动你的提交)。全矩阵 CI 已在该 head 触发;绿了即可合,合并留给你。CI 若红我定向排查。

@zypNB123
zypNB123 force-pushed the 2026-10-20_node-floor-gate branch from 74413ba to 764f667 Compare October 20, 2026 07:31
zypNB123 pushed a commit to zypNB123/billion-context that referenced this pull request Oct 20, 2026
… the sidecar-open-error branch (ranxianglei#3894)

Review fix for ranxianglei#3896. The new case "unsupported-node" in degradeReason() was unreachable: when bootstrap returns unsupported-node no loader is registered, so the sidecar open ALSO records a "no SQLite engine available" open-error, and the existing sidecarOpenError branch returned first - shadowing the specific reason. Users saw "better-sqlite3 not installed" instead of the real cause (a too-old Node). Promote the floor reason ahead of that branch and drop the now-dead switch case.

Regression test pins the acp_status reason for the unsupported-node scenario.
@zypNB123
zypNB123 force-pushed the 2026-10-20_node-floor-gate branch from 4a71ecb to 346813d Compare October 20, 2026 11:21
zypNB123 pushed a commit to zypNB123/billion-context that referenced this pull request Oct 20, 2026
… the sidecar-open-error branch (ranxianglei#3894)

Review fix for ranxianglei#3896. The new case "unsupported-node" in degradeReason() was unreachable: when bootstrap returns unsupported-node no loader is registered, so the sidecar open ALSO records a "no SQLite engine available" open-error, and the existing sidecarOpenError branch returned first - shadowing the specific reason. Users saw "better-sqlite3 not installed" instead of the real cause (a too-old Node). Promote the floor reason ahead of that branch and drop the now-dead switch case.

Regression test pins the acp_status reason for the unsupported-node scenario.
zypNB123 pushed a commit to zypNB123/billion-context that referenced this pull request Oct 20, 2026
…se and the ranxianglei#3818 default wording

Rebase follow-up for ranxianglei#3896: the hand-maintained env-var row must carry both
the PR's Node-floor sentence and master's post-ranxianglei#3818 'Defaults to full when
unset' tail - the two independent conflict resolutions had split them.
Same fix in CONFIGURATION.zh-CN.md.
@ranxianglei

Copy link
Copy Markdown
Owner

ci红

zypNB123 and others added 10 commits October 20, 2026 19:56
…e floor

better-sqlite3 v13 declares engines: node >= 22, but the bootstrap pinned only
version + sha512 and registered any copy that require() accepted. On Node 20 the
win32-x64 prebuild faults inside process.dlopen (access violation) and the
linux-x64 one faults on the first new Database() - uncatchable either way, so a
bootstrap probe cannot defend against it. The crash loops the host process
instead of degrading to legacy storage.

The pin now carries the artifact's own Node major floor and runBootstrap()
refuses BOTH the download and a cached-copy registration below it, returning
unsupported-node with an explicit upgrade reason. The gate sits after the
builtin/host probes (a host-installed better-sqlite3@12 on Node 20 is fine) and
before the cache check (the cached 13.0.3 copy is the crash source). A cached
copy is left on disk so it becomes usable again after a Node upgrade.

No new config surface, no cooldown (a too-old Node is not a transient download
failure).

Fixes ranxianglei#3894. Refs ranxianglei#3341.
… the sidecar-open-error branch (ranxianglei#3894)

Review fix for ranxianglei#3896. The new case "unsupported-node" in degradeReason() was unreachable: when bootstrap returns unsupported-node no loader is registered, so the sidecar open ALSO records a "no SQLite engine available" open-error, and the existing sidecarOpenError branch returned first - shadowing the specific reason. Users saw "better-sqlite3 not installed" instead of the real cause (a too-old Node). Promote the floor reason ahead of that branch and drop the now-dead switch case.

Regression test pins the acp_status reason for the unsupported-node scenario.
…se and the ranxianglei#3818 default wording

Rebase follow-up for ranxianglei#3896: the hand-maintained env-var row must carry both
the PR's Node-floor sentence and master's post-ranxianglei#3818 'Defaults to full when
unset' tail - the two independent conflict resolutions had split them.
Same fix in CONFIGURATION.zh-CN.md.
…ne artifact

better-sqlite3 v13 declares `engines: node >= 22` and compiles its prebuilds
for N-API 10, while Node 20 only provides N-API 9. Loading that binding below
its floor is undefined behavior, not a clean failure: the win32-x64 prebuild
faults inside `process.dlopen` (access violation, uncatchable) and the
linux-x64 one faults on the first `new Database()` (ranxianglei#3341, ranxianglei#3894). The
self-bootstrap registered such a copy whenever it happened to be cached, which
crash-looped the proxy child (1485 spawns, exit 0xC0000005) on Node 20.

The previous revision of this branch refused the artifact below its floor and
degraded to legacy JSON storage. That is the wrong answer: legacy storage is
scheduled for removal, so "low Node" must stay on the unified lane with a
lower artifact rather than fall off it.

Pin one artifact per Node major band instead:

  * Node >= 22 -> better-sqlite3 13.0.3, unchanged: one tarball that carries
    its own eight prebuilds.
  * Node 20 -> better-sqlite3 12.9.0, the last 12.x line that publishes an
    ABI 115 binding. Its npm tarball is pure JS (no binding, no prebuilds);
    the native half is a separate ABI-keyed asset
    (`better-sqlite3-v12.9.0-node-v115-<target>.tar.gz`) that the bootstrap
    now fetches from the binary mirror and materializes as
    `prebuilds/<target>.node`. v12's `lib/database.js` accepts an injected
    `nativeBinding`, so the two halves compose without `bindings`,
    `prebuild-install` or any install script — the ranxianglei#2833 contract holds:
    pinned artifacts, sha512 verified on every download, no host build
    approval.

A spec is eligible only when its Node floor AND (when it has one) its prebuild
ABI match the running Node. A runtime no pinned artifact serves reports
`unsupported-node` with the full table in the detail instead of registering a
binding that would crash the process. Pinned sibling cache dirs are no longer
pruned, so switching Node majors does not re-download an artifact this build
already knows how to load.

Verified on a real Node 20.19.5 (ABI 115): downloads 12.9.0 + its ABI 115
prebuild, binds through nativeBinding, runs DDL/DML/transactions, process
survives. On Node 22.22.3 the same cache resolves to builtin node:sqlite and
the v12 copy is left untouched.

Tests: 5 new cases (JS-only artifact binds its ABI prebuild; a mismatched ABI
is refused before any download; a self-contained artifact never touches the
binary mirror; the prebuild extractor lands and fails loud). Both directions of
the gate are asserted per ranxianglei#3617.

Fixes ranxianglei#3894. Refs ranxianglei#3341.
The `test (windows-latest, 24, 3)` leg went red on
`ranxianglei#3818 review: reap-shape records ... stay envelope-served across restart`
with `EPERM, Permission denied` from `rmrf()` — not an assertion failure.
Windows can still hold an ephemeral handle on `index.db`/`-wal` right after a
store closes (Defender real-time scan, delayed close), and a bare
`rmSync({recursive:true, force:true})` throws instead of waiting.

Retry with the same `maxRetries`/`retryDelay` shape the other Windows-touching
suites already use (`tests/export-jsonl.test.ts`,
`tests/identity-registry.test.ts`), then swallow the residual error: the dir
lives under `HERMETIC`, which the `after` hook clears anyway, so a leaked
handle can only leak a temp dir — never fail an otherwise green run.

Refs ranxianglei#3894.
…anxianglei#3894)

Additive review commit (ai: vllm-qwen/qwen3.8-27b): the PINNED_VERSIONS
prune-retention behavior introduced by the per-major artifact table had no
test row - assert the other pinned version' dir survives a successful
download while an unpinned legacy version is still removed.
The per-major band table from the previous commit only asked two questions -
the package's own engines floor and the ABI of the binary - and that leaves a
hole big enough to crash the process: better-sqlite3 v13 declares
`engines.node >= 22`, but its binding.gyp compiles with NAPI_VERSION=10, and
Node 22.0 through 22.12 ship N-API 9. Those releases satisfy the major floor,
pass every existing check, and then fault uncatchably inside process.dlopen
when the v13 prebuild is loaded. `minNodeMajor: 22` cannot express this: the
gap is in the N-API axis, not the major axis.

Serving is therefore decided on three axes now - engines floor, N-API version
the binding was compiled against, and the ABI of the binary we would register:

  * `EngineBootstrapSpec.minNapiVersion` carries the artifact's NAPI_VERSION.
    v13.0.3 declares 10. A runtime below it is refused with a reason naming
    the gap instead of being handed an addon it cannot load.
  * `EnginePrebuild.sha512Base64ByAbi` replaces the single-ABI
    `nodeAbi`/`sha512Base64ByTarget` pair with an ABI-keyed digest table, so
    v12.9.0 now serves both Node 20 (ABI 115) and the Node 22.0-22.12 band
    (ABI 127). Eight new pins, taken from the npmmirror binary mirror.
  * The cache directory is ABI-keyed (`better-sqlite3-12.9.0-abi115`): a
    runtime that switches Node minors must fetch the binding that matches it
    rather than inherit the previous one. The prune keeps every pinned ABI
    variant, so switching back does not re-download.

`pinnedEngineVersionFor()` is exported as a pure selector so the whole band
matrix is testable without a download.

Verified end to end on real runtimes against the real registry: Node 22.0.0
(ABI 127, N-API 9) now downloads v12.9.0 plus its ABI 127 prebuild, opens a
database, runs DDL/DML and survives - the same binary that segfaults when the
v13 prebuild is loaded. Node 20.19.5 stays on the ABI 115 lane; Node 22.13+
and 22.14+ take the builtin or the v13 single-tarball lane as before.

Fixes ranxianglei#3894. Refs ranxianglei#3341.
…om the sqliteAutoDownload description (ranxianglei#3894)

In-passing nit from review of bb53400: the rev3 prose stated the
refusal clause twice in a row (short form + detailed form) in both
languages. Keep the detailed sentence only; regenerate CONFIGURATION.md
en/zh from the seed.
ranxianglei#3853 landed three solver-backed verif tests without the availability gate the
other verif suites carry, so on a runner with neither cvc5 nor z3 they fail
instead of skipping - `no solver available (set VERIF_SOLVER_CMD /
VERIF_PYTHON)`. That contradicts the module's own contract (AGENTS.md rule 14:
an environment without a solver must skip, never go red) and it is what turned
every `test` leg red on this head, on ubuntu and windows alike.

Adds the same probe-then-skip the sibling suites use (verif-bug3769,
verif-generic-append-stability, verif-symbolic-smoke) to bug3676, bug3691 and
the induction step. With a solver present nothing changes; without one the
three cases report skipped and the rest of the suite still runs.

Verified locally: the 8 verif files report 5 pass / 10 skipped / 0 fail with no
solver on PATH, and the probe returns null as expected.
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

ci红已定位:红的不是我们审核过的 head,是作者刚把 master merge 进分支产生的新 commit(c66665779),失败与 PR 内容无关。

事实链:

  1. 我盯的 head 4a71ecb(rev3+文档小修)全矩阵全绿(48 success + 1 skipped,watcher 11:15 确认)。之后作者推了两个 commit:merge master(c66665779)+ 一个测试修复(0a308262c)。
  2. c666657 上 12 个 test leg 红了 10 个。翻 job 日志,失败分两类,都不是本 PR 的文件:
  3. PR 本体无恙:五个核心文件 + seed 与我审核过的版本逐字节一致;CONFIGURATION 对 master 的 delta 恰好只有本 PR 自己的行。

现状:新 head 0a30826 的 CI 刚开始排队(查询时 0 个 check-run 落地)。继续盯着:绿了报你;deferral 测试若再 flake 就重跑该 leg;verif 若还有漏跳的文件会点名报出来。

另注:master tip 上 ci-gate-plugins 工作流也是红的——这是已知存量问题(#3711,修复 PR #3715 在途),与本 PR 无关,不混进来处理。

@zypNB123
zypNB123 force-pushed the 2026-10-20_node-floor-gate branch from 0a30826 to f53889c Compare October 20, 2026 12:01
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

进展更新:作者在我定位红因之后又把分支整体重建到了最新 master(现 head f53889c,线性栈,含之前全部 commit + verif-skip + 一个文档重新生成小修)。新基线带进来了 #3932 storage-backend-seam(src/storage/backend.ts,sqlite 改走注册表)——这和本 PR 的 engine-bootstrap 同域,我专门本地验证过交互:

CI 在 f53889c 上跑着:目前 34/48 绿、14 在跑、零失败。watcher 盯着,绿了报你;那个 deferral flake 若再现就重跑该 leg。

@ranxianglei

Copy link
Copy Markdown
Owner

Rebased onto current master (the storage PR wave #3911/#3927/#3932–#3955 had conflicted this branch) — continuation at #3963, all 9 commits kept with @zypNB123's original authorship, zero engineering changes on top (only a CHANGELOG entry). Local verification on the rebased branch: engine-bootstrap 43/43, full-flip 8/8, acp-status-degraded 3/3, unified 14/14, tsc clean, config coverage ok. Suggest closing this one in favor of #3963.

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

CI 心跳:f53889c9e 上目前 34/48 绿、14 个 leg 在跑(含主矩阵分片)、零失败。没有新变化,绿了或红了第一时间报。

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

CI 红已定位:不是本 PR 的问题,是 master 存量回归,我已在 #3970 立项。

排查链(全部实测):

  1. 红的四个 leg(shard 3 × ubuntu/windows × 22/24)是同一个测试:tests/advisory.test.ts:627「checkForUpdate: a forced manual update bypasses the first-check deferral」,断言 calls === 2 实际 3。
  2. 与 Node 20 startup segfault via self-bootstrapped better-sqlite3@13 — the Node floor promised in #3341 was never implemented #3894 无关的证明:在 origin/master tip(detached)和本 PR 的 merge base(1cc1cb85e)上本地跑同一文件——同样 5/5 稳定失败。本 PR 只动 storage bootstrap 域,完全不碰 update/advisory。
  3. 机制(fetch 探针钉死):强制更新路径的实际 fetch 序列 = packument → auto-update-policy.json(Automatic staggered rollout: default random upgrade interval of 15 min – 48 h + GitHub remote policy file (emergency kil | 自动升级撒点:默认 15 分钟~48 小时随机升级 + GitHub 远程策略文件(紧急刹车/按版本调速),给事故留响应窗 #3619 的策略门,force=true 也照拉,代码注释明确 manual updates still resolve pinned/blocked) → tarball。该测试写于 [updater] #1588 rollback gate: post-restart first-check race lets auto-update install the affected version before the advisory loop arms #2456 时期(Automatic staggered rollout: default random upgrade interval of 15 min – 48 h + GitHub remote policy file (emergency kil | 自动升级撒点:默认 15 分钟~48 小时随机升级 + GitHub 远程策略文件(紧急刹车/按版本调速),给事故留响应窗 #3619 于 10-18 合入之前),断言停在 2,没跟上第三个 fetch。策略 fetch 被 mock 拒绝后按设计 fail-open,运行时自更新行为不受影响,纯测试断言滞后。
  4. 时间线疑点也写进 issue 了:Automatic staggered rollout: default random upgrade interval of 15 min – 48 h + GitHub remote policy file (emergency kil | 自动升级撒点:默认 15 分钟~48 小时随机升级 + GitHub 远程策略文件(紧急刹车/按版本调速),给事故留响应窗 #3619 合入后仍有多个 PR 正常合入(CI 必过),说明该测试在部分 CI 运行中还能过——疑似 loadUpdatePolicy 进程级 memoization 与测试顺序/环境交互,修复者需干净环境验证,别把「改断言为 3」修成另一种 flake。

#3970 给了两个修法:A=仅测试(withFetch 注册策略路由 + 断言改 3,不动 src/update.ts,倾向);B=force 路径整体跳过策略 fetch(改自更新语义,你拍板)。建议独立小修走 master 侧 PR,不混进 #3894。

对本 PR 的影响与选项:master 自己的 CI 现在也是红在同一测试上(tip 71d0721 run = failure),所以——① 你若现在就想合 #3894:这条红 leg 已完整归因并立项,可视为已知存量问题放行;② 若要全绿再合:先修 #3970 落 master,作者 rebase 后重跑。内容本身我的审核结论不变(rev3 全部通过)。

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

无新变化(分支仍在 f53889c)。球在你这边:#3894 是「已知存量红放行」还是「等 #3970 修完 rebase 全绿再合」,你定;定了我就收尾。

@zypNB123

Copy link
Copy Markdown
Contributor Author

(ai: deepseek/deepseek-v4.1-flash-sg)

Closing in favor of #3963, which is merged (74b894d). Confirmed the continuation kept all 9 commits with original authorship and made no engineering changes on top.

Thanks for carrying it across the storage PR wave — the rebase against #3911/#3927/#3932–#3955 was the part I could not do safely from a fork.

One note for the record: the merged head leaves docs-sync red on master (node tools/gen-config-docs.mjs generate deletes two stale duplicate rows — storage.backend step-1 wording and the pre-#3388 storage.sqliteAutoDownload row — from both CONFIGURATION files). That is a merge-resolution artifact, not a code defect. I will send a small follow-up PR that only regenerates the two generated files.

The host-provided probeHostBetterSqlite3() case from #3894 (src/storage/driver.ts unconditionally require("better-sqlite3"), so a host-installed v13 still faults on Node 20 before any floor check) remains open as its own item — it needs your call on the host-provided-engine semantics.

@zypNB123 zypNB123 closed this Oct 20, 2026
Zhishulo pushed a commit to Zhishulo/billion-context that referenced this pull request Oct 20, 2026
… the sidecar-open-error branch (ranxianglei#3894)

Review fix for ranxianglei#3896. The new case "unsupported-node" in degradeReason() was unreachable: when bootstrap returns unsupported-node no loader is registered, so the sidecar open ALSO records a "no SQLite engine available" open-error, and the existing sidecarOpenError branch returned first - shadowing the specific reason. Users saw "better-sqlite3 not installed" instead of the real cause (a too-old Node). Promote the floor reason ahead of that branch and drop the now-dead switch case.

Regression test pins the acp_status reason for the unsupported-node scenario.
Zhishulo pushed a commit to Zhishulo/billion-context that referenced this pull request Oct 20, 2026
…se and the ranxianglei#3818 default wording

Rebase follow-up for ranxianglei#3896: the hand-maintained env-var row must carry both
the PR's Node-floor sentence and master's post-ranxianglei#3818 'Defaults to full when
unset' tail - the two independent conflict resolutions had split them.
Same fix in CONFIGURATION.zh-CN.md.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working ci:full Run the FULL CI matrix (all e2e lanes + 12-leg test matrix) on this PR p1 scope/storage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Node 20 startup segfault via self-bootstrapped better-sqlite3@13 — the Node floor promised in #3341 was never implemented

2 participants