Repository navigation
[PR] fix(dist): load mysql2 lazily at backend-open time + generalize the dist external-import gate (#4008) - #4013
Merged
Conversation
added 2 commits
October 21, 2026 00:14
… self-contained (#4008) src/storage/mysql.ts held a TOP-LEVEL static value import of mysql2/promise. openBackend's dynamic import of ./mysql.js is an internal-module import that tsup's single-file bundle (splitting:false) hoists into the bundle body, and mysql2 (a package.json dependency) is auto-externalized by tsup, so the driver import landed as a static import at the top of dist/index.js and dist/agent/pi-native.js. ESM resolves static imports at module load regardless of use: every install without a node_modules tree (dsh profile bare copies, bili's own e2e rigs) crashed at startup with ERR_MODULE_NOT_FOUND before the proxy bound its port, and model traffic silently went direct, uncompressed (the 'real dsh headless profile lane' reds on #4003 ci:full). The driver now loads only inside openBackend("mysql", dsn): a literal dynamic import of the external specifier survives bundling as a real runtime import (same discipline as the jimp lazy load, #3516), and the resolved module is injected into MysqlBackend's constructor. Boots that select sqlite never resolve mysql2; a thin install that selects mysql fails at open time with a contextual error that the existing documented degrade paths already handle (shadow: warn + legacy-only; export: notice + sqlite fallback), instead of dying at process start.
scripts/check-dist-external-zod.mjs only matched the literal bare specifier 'zod', so the mysql2 static import from #4008 sailed through every build. The gate now scans every dist artifact for ANY external bare import (all static forms incl. minified import{x}from"y" / export-from / require, plus dynamic import()) and fails unless the specifier is on an explicit allowlist whose entries carry a reason AND a sanctioned kind set: jimp + mysql2/promise are dynamic-only (their lazy-load contract is what keeps thin installs alive), react is confined to agent/dsh-native-client.js (dsh web-profile browser bundle). Node builtins and relative paths stay exempt; opaque variable specifiers remain invisible by design (the sanctioned host-zod path). Both directions are pinned in tests/check-dist-external.test.ts (#3617 discipline): the exact #4008 minified shape fails, every zod form still fails (#3511 pin preserved), a STATIC form of an allowlisted dynamic-only entry still fails, unknown new packages fail in all six forms, and every sanctioned shape passes. Renamed to scripts/check-dist-external.mjs to match its actual scope; build wiring + stale references updated.
Contributor
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.4793-pr.4013.c49346eThis exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr4013.tgz
npm install -g packageInstall instructions are refreshed on each push. |
Contributor
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.689-master.ab7e941 # this exact build
npm install -g billion-context@master # always the newest merged build
|
This was referenced Oct 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #4008
What's broken
Thin installs (a copy of
dist/+package.jsonwithout anode_modulestree — dsh profile bare copies, bili's own e2e rig) crash at proxy startup:ERR_MODULE_NOT_FOUND: Cannot find package 'mysql2' imported from .../dist/index.js. The port never binds and model traffic silently goes direct/uncompressed. In CI this showed up as the two red assertions inreal dsh headless profile lane(#2241 A / #2791 C "both turns stamped ONE conversation id" — actually 0, because no request ever reached bili).Root cause
src/storage/mysql.ts:30held a top-level static value import ofmysql2/promise.openBackend()'sawait import("./mysql.js")is an internal-module dynamic import: tsup's single-file bundle (splitting:false) hoists it into the bundle body, dragging the driver import along to the top ofdist/index.jsanddist/agent/pi-native.js.mysql2is a package.json dependency → tsup auto-externalizes it → the import stays external in dist → ESM resolves static imports at module load regardless of which code path runs → every thin-install boot crashed even when sqlite was selected."zod", so this sailed through; the thin-install smoke test does a registry install (which fetches mysql2), so it cannot catch this class either.Fix (two commits)
fix:) — the driver now loads only insideopenBackend("mysql", dsn): a literal dynamic import of an external specifier survives bundling as a real runtime import (same discipline as the jimp lazy load, v0.1.198 release blockers found in final pre-release review | v0.1.198 发版前终审发现的阻塞项清单 #3516), and the resolved module is injected intoMysqlBackend's constructor. Boots that select sqlite never resolve mysql2. A thin install that selects mysql fails at open time with a contextual error that the existing documented degrade paths already handle (shadow: warn + legacy-only; export: notice + sqlite fallback).ci:) —check-dist-external-zod.mjs→scripts/check-dist-external.mjs: scans every dist artifact for ANY external bare import (all static forms incl. minifiedimport{x}from"y"/ export-from / require, plus dynamicimport()) and fails unless on an explicit allowlist with a reason AND a sanctioned kind set:jimp+mysql2/promiseare dynamic-only,reactconfined toagent/dsh-native-client.js. Both directions pinned in tests (Test Blind Spot Retrospective: Reinforcing Negative Assertion Discipline, Tightening Fake Upstream Strictness, Thin Inst | 测试盲区复盘:补反向断言纪律、假上游严格化、薄安装冒烟、形态矩阵与生命周期覆盖 #3617): the exact dist/index.js crashes on startup: mysql2 is statically inlined into the bundle, so deployment modes without node_modules | dist/index.js 启动即崩:mysql2 被静态打进 bundle,不带 node_modules 依赖的安装形态(dsh e2e、dist-only 嵌入)直接挂 #4008 minified shape fails, every zod form still fails (197-master 发版最后 #3511 pin preserved), a STATIC form of an allowlisted dynamic-only entry still fails, unknown new packages fail in all six forms, every sanctioned shape passes.Behavior changes (old → new)
ERR_MODULE_NOT_FOUND), port never bindsstorage.backend=mysqlselectedopenBackend("mysql")rejects with a contextual message naming both knobs; existing degrade paths apply"zod"onlyNo config-surface change. No kernel change. No wire change (storage layer only; both compression modes unaffected).
Verification
npm run typecheckclean; targeted suites green:tests/check-dist-external.test.ts(both directions),tests/storage-backend-knob.test.ts,tests/storage-backend-contract.test.ts,tests/cli-export.test.ts,tests/storage-unified.test.ts,tests/storage-backend-mysql-e2e.test.ts(DSN lanes skipped locally — no MySQL server here).mysql2/promisenow appears dynamic-only indist/index.js+dist/agent/pi-native.js; zero static externals anywhere else (jimp dynamic, react confined to the browser bundle); new gate passes the build.dist/+package.jsoninto a scratch dir with nonode_modules→node dist/index.js --versionexits 0 and the proxy binds its port (before: immediateERR_MODULE_NOT_FOUND).real dsh headless profile laneneeds dsh ≥ 0.2.0-rc.2 (local binary is older), so the two red assertions are CI's call on the pushed head.