From c01fe3ae45295784e216e3e57b1ebca0720b89bc Mon Sep 17 00:00:00 2001 From: awork Date: Tue, 20 Oct 2026 01:33:15 +0800 Subject: [PATCH 1/2] =?UTF-8?q?docs:=20regenerate=20config=20docs=20?= =?UTF-8?q?=E2=80=94=20drop=20stale=20duplicate=20storage=20rows=20left=20?= =?UTF-8?q?by=20a=20bad=20merge=20resolution?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The generated param-ref block in CONFIGURATION.md / CONFIGURATION.zh-CN.md carried two duplicated rows (storage.unified, storage.sqliteAutoDownload): the pre-#3818 spellings alongside the regenerated ones. Re-running tools/gen-config-docs.mjs generate removes exactly those four stale lines; every key stays documented once with current text. Fixes the docs-sync gate red on master since 2026-10-19. --- CONFIGURATION.md | 2 -- CONFIGURATION.zh-CN.md | 2 -- 2 files changed, 4 deletions(-) diff --git a/CONFIGURATION.md b/CONFIGURATION.md index 09e953c4c5..d59b7e9cc6 100644 --- a/CONFIGURATION.md +++ b/CONFIGURATION.md @@ -174,8 +174,6 @@ This index is generated from `website/config-reference/*.yaml` — edit the seed | `persist.epermAlertRepeatMs` | number | 0 (no repeats) | BILI_PERSIST_EPERM_ALERT_REPEAT_MS | Repeat interval for the EPERM alert; 0 logs it once. | | `storage` | object | {} (all defaults) | — | Unified SQLite storage sidecar settings (#2671). | | `storage.unified` | legacy\|shadow\|full | unset (full) | BILI_STORAGE_UNIFIED | Tiered storage-migration switch (#2671): shadow = unified SQLite double-write dry-run with digest reconciliation; full = drop-in read path (unified-first loads with organic import, legacy fallback). Absent everywhere now defaults to FULL (#3818 flip - the shadow dry-run reconciled cleanly since #2815, so the index is the load path; JSON envelopes keep being written, and legacy remains the one-line escape hatch). File tier is strict legacy/shadow/full; the env tier also accepts 1/true (=shadow) and unified (=full) and wins over the file. Resolved once per lane boot - hot edits need a restart. Encryption (BILI_ENCRYPTION_KEY) hard-blocks the sidecar. | -| `storage.sqliteAutoDownload` | boolean | true | BILI_SQLITE_AUTO_DOWNLOAD | Gate for the one-time SQLite engine self-download (#2833): when no engine is available in-process (no built-in node:sqlite, no host-installed better-sqlite3) and the unified sidecar is wanted on this platform, bili downloads the pinned better-sqlite3 prebuilt tarball from the npm registry once (~11 MB, sha512-verified) into the cache dir and reuses it on every later start. Engine resolution order: built-in node:sqlite (Node >= 22.5) -> host-provided better-sqlite3 -> cached self-download -> self-download (gated by this switch) -> none (legacy-only, the #2825 degrade contract). The switch gates only the network fetch - a cached copy is used even when it is false; unsupported platforms never download. The tarball download walks a builtin mirror list (#3338): the configured registry (BILI_UPDATE_REGISTRY / update.registry) is tried first, then https://registry.npmjs.org, then https://registry.npmmirror.com - the first mirror that yields a sha512-verified tarball wins; a Node < 22.5 with no engine gets an explicit Node upgrade hint in the degrade reason (#3385). | -| `storage.unified` | legacy\|shadow\|full | unset (shadow) | BILI_STORAGE_UNIFIED | Tiered storage-migration switch (#2671): shadow = unified SQLite double-write dry-run with digest reconciliation; full = drop-in read path (unified-first loads with organic import, legacy fallback). Absent everywhere now defaults to shadow (JSON stays the sole source of truth; the index is digest-reconciled telemetry - set legacy to opt out). File tier is strict legacy/shadow/full; the env tier also accepts 1/true (=shadow) and unified (=full) and wins over the file. Resolved once per lane boot - hot edits need a restart. Encryption (BILI_ENCRYPTION_KEY) hard-blocks the sidecar. | | `storage.sqliteAutoDownload` | boolean | true | BILI_SQLITE_AUTO_DOWNLOAD | Gate for the one-time SQLite engine self-download (#2833): when no engine is available in-process (no built-in node:sqlite, no host-installed better-sqlite3) and the unified sidecar is wanted on this platform, bili downloads the pinned better-sqlite3 prebuilt tarball from the npm registry once (~11 MB, sha512-verified) into the cache dir and reuses it on every later start. Engine resolution order: built-in node:sqlite (Node >= 22.5) -> host-provided better-sqlite3 -> cached self-download -> self-download (gated by this switch) -> none (legacy-only, the #2825 degrade contract). The switch gates only the network fetch - a cached copy is used even when it is false; unsupported platforms never download. The tarball download walks a builtin mirror list (#3338): the configured registry (BILI_UPDATE_REGISTRY / update.registry) is tried first, then https://registry.npmjs.org, then https://registry.npmmirror.com - the first mirror that yields a sha512-verified tarball wins; a Node < 22.5 with no engine gets an explicit Node upgrade hint in the degrade reason (#3385). A failed download arms a 6-hour retry cooldown persisted as a marker file under the cache dir (billion-context/sqlite/.bootstrap-failed.json); the storage diagnostics blob shows the marker's absolute expiry time and path - delete that file and restart bili to force an immediate retry instead of waiting out the whole window (#3788). | | `storage.jsonWriter` | on\|off | on | BILI_STORAGE_JSON_WRITER | Phase 3 envelope switch (#2960): off stops writing the legacy JSON session envelope while storage.unified is full - the SQLite index becomes the only persistent copy (reads are already unified-first; boot enumerates the index union, so index-only sessions load normally). Only honored in effective-full mode: under shadow/legacy (or with encryption active) the request is ignored with a one-time warning because the JSON file remains the sole source of truth. Env tier 0/false/off = off and 1/true/on = on wins over the strict file spellings. Fail-safe: if the sidecar disables itself mid-run, envelope writes resume on the next save. Rollback: unset the knob - writing resumes, index-only sessions keep loading via the boot union, and replicas/restore re-materialize portable JSON envelopes. Downgrading the binary (or losing the SQLite engine) makes index-only sessions invisible to v0.1.196 and older until a >=v0.1.197 binary runs again - the data stays intact in index.db, it is just not read by the older code. The per-session content-store sidecar keeps being written either way. A jsonWriter=off lane must be the SOLE writer of its sessions directory: an off lane reads unified-first and never falls back to the file, so any peer instance running legacy/shadow against the same directory diverges silently (its JSON copy stops receiving updates, and the off lane never sees the peer's writes). | | `storage.walJournalSizeLimitBytes` | number | 8388608 | BILI_STORAGE_WAL_JOURNAL_SIZE_LIMIT_BYTES | WAL reclamation cap for index.db-wal (#3250), in bytes: every completed checkpoint truncates the WAL file back to this size, bounding disk use and crash-recovery replay in long-lived processes. NOT a per-session data quota - data lands in index.db as it is written and the WAL only stages frames between checkpoints, so session count/size does not scale this; a large burst can overshoot temporarily until the next completed checkpoint truncates back. Higher = rarer truncate IO but more steady disk use and a longer crash-replay tail. Env tier (positive integer) wins over the file tier; a garbage or non-positive env value falls through to the file/default tier instead of failing boot. Applied when the store opens at lane boot - hot edits need a restart. The effective value appears on the unified-enable log line and in the storage diagnostics blob. | diff --git a/CONFIGURATION.zh-CN.md b/CONFIGURATION.zh-CN.md index 758505a848..457bc4cbb3 100644 --- a/CONFIGURATION.zh-CN.md +++ b/CONFIGURATION.zh-CN.md @@ -174,8 +174,6 @@ | `persist.epermAlertRepeatMs` | number | 0 (no repeats) | BILI_PERSIST_EPERM_ALERT_REPEAT_MS | EPERM 告警的重复间隔;0 只记录一次。 | | `storage` | object | {} (all defaults) | — | 统一 SQLite 存储边车设置(#2671)。 | | `storage.unified` | legacy\|shadow\|full | unset (full) | BILI_STORAGE_UNIFIED | 分层存储迁移开关(#2671):shadow = 统一 SQLite 双写干跑 + digest 对账;full = 平替读路径(unified-first 加载 + 有机导入 + legacy 兜底)。两处均未设置时默认 full(#3818 翻转 —— shadow 干跑自 #2815 起对账一直干净,索引因此成为默认加载路径;JSON 信封照写,legacy 仍是一行逃生口)。文件层只接受 legacy/shadow/full;环境变量层还接受 1/true(=shadow)与 unified(=full)并优先于文件。每个 lane 启动时解析一次 —— 热改动需重启生效。加密(BILI_ENCRYPTION_KEY)会硬阻断边车。 | -| `storage.sqliteAutoDownload` | boolean | true | BILI_SQLITE_AUTO_DOWNLOAD | SQLite 引擎一次性自下载的闸门(#2833):进程内没有可用引擎(无内置 node:sqlite、宿主未装 better-sqlite3)且本平台希望启用统一边车时,bili 从 npm registry 一次性下载钉死的 better-sqlite3 预编译 tarball(约 11 MB,sha512 校验)到缓存目录,之后每次启动复用。引擎解析顺序:内置 node:sqlite(Node >= 22.5)-> 宿主机已装的 better-sqlite3 -> 缓存的自下载副本 -> 自下载(本开关控制)-> 无引擎(仅 legacy,即 #2825 降级契约)。开关只控制网络下载 —— 为 false 时缓存副本仍会使用;不支持的平台永不下载。 tarball 下载内置镜像列表兜底(#3338):先试配置的 registry(BILI_UPDATE_REGISTRY / update.registry),再试 https://registry.npmjs.org、https://registry.npmmirror.com —— 第一个通过 sha512 校验的镜像生效;Node < 22.5 且无引擎时,降级原因会给出明确的 Node 升级提示(#3385)。 | -| `storage.unified` | legacy\|shadow\|full | unset (shadow) | BILI_STORAGE_UNIFIED | 分层存储迁移开关(#2671):shadow = 统一 SQLite 双写干跑 + digest 对账;full = 平替读路径(unified-first 加载 + 有机导入 + legacy 兜底)。两处均未设置时默认 shadow(JSON 仍是唯一事实源,索引只做对账遥测 —— 设 legacy 可退出)。文件层只接受 legacy/shadow/full;环境变量层还接受 1/true(=shadow)与 unified(=full)并优先于文件。每个 lane 启动时解析一次 —— 热改动需重启生效。加密(BILI_ENCRYPTION_KEY)会硬阻断边车。 | | `storage.sqliteAutoDownload` | boolean | true | BILI_SQLITE_AUTO_DOWNLOAD | SQLite 引擎一次性自下载的闸门(#2833):进程内没有可用引擎(无内置 node:sqlite、宿主未装 better-sqlite3)且本平台希望启用统一边车时,bili 从 npm registry 一次性下载钉死的 better-sqlite3 预编译 tarball(约 11 MB,sha512 校验)到缓存目录,之后每次启动复用。引擎解析顺序:内置 node:sqlite(Node >= 22.5)-> 宿主机已装的 better-sqlite3 -> 缓存的自下载副本 -> 自下载(本开关控制)-> 无引擎(仅 legacy,即 #2825 降级契约)。开关只控制网络下载 —— 为 false 时缓存副本仍会使用;不支持的平台永不下载。 tarball 下载内置镜像列表兜底(#3338):先试配置的 registry(BILI_UPDATE_REGISTRY / update.registry),再试 https://registry.npmjs.org、https://registry.npmmirror.com —— 第一个通过 sha512 校验的镜像生效;Node < 22.5 且无引擎时,降级原因会给出明确的 Node 升级提示(#3385)。下载失败会置一个 6 小时的重试冷却,以标记文件形式持久化在缓存目录下(billion-context/sqlite/.bootstrap-failed.json);storage 诊断块会显示该标记的绝对到期时间与路径 —— 删除该文件并重启 bili 即可强制立即重试,不必等完整个冷却窗口(#3788)。 | | `storage.jsonWriter` | on\|off | on | BILI_STORAGE_JSON_WRITER | Phase 3 信封开关(#2960):off 时若 storage.unified 为 full 则不再写 legacy JSON 会话信封 —— SQLite 索引成为唯一持久化副本(读路径本就 unified-first;boot 枚举索引并集,纯索引会话可正常加载)。仅在有效 full 模式下生效:shadow/legacy(或加密激活)下请求会被忽略并警告一次,因为 JSON 文件仍是唯一事实源。环境变量层 0/false/off = off、1/true/on = on,优先于严格拼写的文件层。失败安全:若边车运行中自禁用,下次保存即恢复信封写入。回滚:去掉该开关 —— 写入恢复,纯索引会话经 boot 并集继续可加载,replicas/restore 可重新物化可移植的 JSON 信封。降级二进制(或丢失 SQLite 引擎)会让纯索引会话在 v0.1.196 及更早版本上不可见,直到重新运行 >=v0.1.197 —— 数据在 index.db 中完好无损,只是旧代码不读它。每会话的 content-store 边车无论开关如何都照写。jsonWriter=off 的 lane 必须是其会话目录的唯一写者:off lane 读路径 unified-first、永不回退到文件,因此同一目录上跑 legacy/shadow 的对等实例会静默分叉(它的 JSON 副本收不到更新,off lane 也看不到对等实例的写入)。 | | `storage.walJournalSizeLimitBytes` | number | 8388608 | BILI_STORAGE_WAL_JOURNAL_SIZE_LIMIT_BYTES | index.db-wal 的 WAL 回收上限(#3250),单位字节:每次完成的 checkpoint 把 WAL 文件截回该大小,约束长驻进程的磁盘占用与崩溃恢复重放。不是每会话数据配额 —— 数据写入时就已落进 index.db,WAL 只是两次 checkpoint 之间的帧暂存区,会话数量/大小不会放大它;大 burst 可能暂时超出,直到下一次完成的 checkpoint 截回。调大 = truncate IO 更稀疏,但稳态磁盘占用更多、崩溃恢复重放尾更长。环境变量层(正整数)优先于配置文件层;env 值非法或非正数时静默落到文件/默认层,boot 不崩。lane 启动、store open 时生效 —— 热改动需重启。实际取值会出现在 unified-enable 日志行与 storage 诊断块中。 | From bdad5b0226beb0935c6e733a7cef440bdfa055d4 Mon Sep 17 00:00:00 2001 From: awork Date: Tue, 20 Oct 2026 01:39:19 +0800 Subject: [PATCH 2/2] fix(ci): drop orphaned export on maskUrlForPublish (unused-exports gate) a4f273074 (#3811) introduced maskUrlForPublish with an export, but nothing outside src/log-mask.ts imports it (only maskUrlsForPublish below calls it), so scripts/check-unused-exports.mjs fails with: INTERNAL_ONLY src/log-mask.ts: maskUrlForPublish Dropping the export keyword makes the gate pass again; no behavior change. Same treatment as #2315 applied to maskUrlsInText. --- src/log-mask.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/log-mask.ts b/src/log-mask.ts index ac2440b279..d34a619b1b 100644 --- a/src/log-mask.ts +++ b/src/log-mask.ts @@ -118,7 +118,7 @@ export function maskUrlsInText(text: string): string { * preference must not weaken the publish boundary (same stance as * redactSecretsInText, which is never gated by the switch). Userinfo, query * and hash are dropped exactly as in maskUrlForLog; the path is kept. */ -export function maskUrlForPublish(url: string): string { +function maskUrlForPublish(url: string): string { let u: URL; try { u = new URL(url);