diff --git a/CHANGELOG.md b/CHANGELOG.md index d0e66f8e17..0217bf0b2b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,10 @@ Versions follow the merge of a `*_release-v*` branch; CI publishes to npm on tag ## [Unreleased] +### Added + +* **#5460(P1):实时性能采集落地(SQLite 存储)**。近期所有性能事故(#5403 OOM×6、#4528 主线程卡死、#5385 前缀钉死)都是用户先发现、CI 零预警。现在 DB 存储模式(sqlite/mysql)下每处理一个请求,就在会话目录旁的新小库 `perf.sqlite` 落一行(消息数/进出字节/本地耗时/会话号),另有每 30 秒一条内存水位采样(rss/heap/external)——OOM 类事故从此有爬坡曲线可查。写路径与服务完全隔离:任何写失败只停采集(一次性警告),绝不影响服务;纯 JSON 存储模式冻结零足迹(owner 定)。`BILI_PERF_TELEMETRY=0` 整体关闭。后续:P2 场景夹具+`bili bench`(只记录不判)、P3 基线容差带接 FULL/夜间 CI(普通 PR CI 零性能 job)、P4 `/stats`+Web 面板性能页+外部看门狗——跟踪单 #5460。 + ### Removed - **compress-at-fold-point 整功能移除(owner 指令 2026-11-10,#6132;v1 #5471 + 未发布的 v2 #5949)**:删除 `compress.compressAtFoldPoint` / `compress.compressAtFoldPointClearThinking` 配置键、`BILI_COMPRESS_AT_FOLD_POINT` / `BILI_COMPRESS_AT_FOLD_POINT_CLEAR_THINKING` env、kernel 的 `relocate-compress-carriers` 管线节点与 `relocateCompressCarriers` / `planRelocateCompressCarriers` 导出、`Config.reasoningDroppable`、v2 的 `already-folded-echo` 解析 kind、web 配置行与 i18n 文案、config-reference/站点文档与博客文章(owner 指令直接删除)。移除原因:9 天内在其目标模型家族(qwen3-27b 系)出现两次不同的真实事故——2026-11-08 v1 引发的自发重复压缩循环(#5949 现场报告,v2 的设计动机)与 2026-11-10 生产会话(v0.1.213,v1 语义)中的作者误认:同轮「回答 + compress」被重放时,压缩对挪位后该轮回答文本裸挂在 transcript 尾部(无 pending 工具调用、无工具结果、无用户输入),27B 模型把自己的最后一条发言当成用户消息(m00376 链),重复回答了自己的答案且错误认知被固化进折叠块主题。v2 从未发布(最新 tag v0.1.213 仍为 v1,`git tag --contains` 为空),移除零迁移成本:开关关闭时布局就是移除后的布局(字节级一致)。旧配置残留的键:一次性 INERT 警告(`warnInertRemovedCompressKeys`,含 top-level 残留)并忽略值。kernel 0.0.192 → 0.0.193。回归面:`tests/removed-compress-keys-inert.test.ts`(新增移除键用例)、`tests/compress-reject-reasons.test.ts`(删除 echo 用例)、`kernel/tests/parse-compress-input.test.ts`(删除两个 pointer-echo 用例)、`kernel/tests/pipeline.test.ts`(节点清单)。 @@ -54,6 +58,7 @@ Versions follow the merge of a `*_release-v*` branch; CI publishes to npm on tag - **中断后/usage 丢失后的「幻影紧急压缩」:本轮拿不到计费样本时,压力档压缩延后一轮再决定**:中断的流(用户 ESC)或上游短 TTL 缓存把 usage 报告弄丢时,本轮的「计量表针」只能落回字符级本地估算——它量的是即将被改写丢掉的入站字节,实测比真实计费高约 3 倍(0.1.212 当天两起真实现场:ESC 的会话从 9478 跳成 228376、长跑会话被连续两轮注入 153% 幻影压力带,徒劳改写并烧掉前缀缓存)。此前这种坏表针直接按 EMERGENCY/OVER-LIMIT 扣扳机。现在:内核收到以「估算值」计量的压力带读数(宿主标记 tokenCountSource="estimate")时,本轮压力档注压延后一轮——下一次请求必然带着新鲜计费报告,拿真数再决定,代价≈0;计费级读数("usage")与不传 source 的老调用方字节不变,非压力档(增长/节拍)完全不受影响,硬溢出兜底臂全程照旧武装。宿主侧提前广告臂(emergencyNudge)同步镜像该位,不会拿着同一个坏读数抢跑。逃生阀:`compress.requireUsageSampleForPressure: false` 恢复旧的估算驱动行为。tests:kernel/tests/nudge-pressure-no-sample-hold.test.ts(6 例)、tests/nudge-pressure-no-sample-armed.test.ts(3 例);kernel 既有套件 1582/1582 绿。 - **External-summary mode no longer reports protected or already-folded ranges as failed compression.** An unchanged kernel preview now returns the normal idempotent no-op or kernel-error receipt without a paid summarizer call; validated internal probe summaries prevent an irrelevant submitted-hint length from blocking that receipt. Probe-created blocks with no recoverable content still refuse rather than folding an unreviewed hint. Regression: `tests/external-summary-runtime.test.ts` (24/24). +- **#5460(P1 评审发现):Node <22.5 上整个 CLI 不再因性能采集找不到 SQLite 而启动即死。** P1 的 perf.sqlite 在模块顶层直接 `import "node:sqlite"`——该内置模块 Node ≥22.5 才有,而 package.json 仍声明 engines >=20;静态导入让**整个进程**在模块加载期崩溃(不可用的是全部 bili,不只是遥测)。现在与 index.db 走同一条引擎解析链(`openSqliteDatabase`:builtin node:sqlite → 宿主 better-sqlite3 → 自下载,#2833/#2845):老 Node 自动落到原生引擎,全缺时按既有契约一次性警告、只降级停用遥测,服务可用性不受影响;同时满足 dist 自包含门禁(零新增外部 specifier)。 * **#5412: 异步存储后端的「压缩冻结会话」现在也能自愈。** #5408 事故(一个会话连续 17 次摘要漂移后被保护机制冻住、8 小时无人知晓)的主修复 #5409 只教了同步存储路径(SQLite 直写)自救:漂移挂起期间保存会自动升级为全量重导入并清掉索引里的残留行。异步路径(opt-in 网络后端,如 mysql)那份孪生代码没搬——同样被冻住的会话在那上面永远救不回来,每次保存都重写不完整视图、下轮采样继续漂移。本 PR 在 `src/storage/shadow.ts` 的异步入队路径补上同一行对齐(`completeLog = hasRawTail || driftedSessions.has(id)`),行为与同步路径逐字一致;回归测试同时钉住正门(latch 挂起的折叠保存清掉外来行)与负门(未挂起保存不清扫)。 - **中断频繁的会话压缩提醒几乎不触发,积到几十万 token 才压一次(#5415,0.1.210 引入)**:#5293 的修复把每次用户中断后的增长参照点直接抬到当前读数,连「中断前已实测的真实增长」也一起清零;高频中断的客户端(实测一份日志 4.5 小时 85 次中断)永远攒不到增长台阶,30 万可压内容 idle 数小时,最后一次性压 33 万。现在吸收时把被清零的实测增长记入「欠账」(`nudge.absorbedDebt`)并累计吸收次数(`nudge.absorbCount`),两者都只在**第 2 次起**的中断后生效:单次中断字节级保持 #5293 的安静(不打扰用户刚改的主意——欠账再多也不当场弹提醒);第二次中断才证明会话是「高频中断型」而非「改主意」,欠账从此与新增量一起计入增长台阶,恢复 209 时代的节奏。另加「大积压排水」兑底:欠账>0 且可压量≥2×阈值且当前无未消化提醒时不再等增长台阶(与 #194 同哲学);无欠账/单次中断的会话决策字节不变。压缩成功时欠账与计数随其它基线一起清零;idle 日志同步显示 `growth X + absorbed Y across N aborts < floor Z`(单次中断显示 `held (single abort, #5293)`)便于排查。tests:kernel/tests/issue5415-absorb-debt-nudge.test.ts(欠账付台阶/单次中断钉死不弹/排水兑底/忽略提醒不重发/无欠账字节不变/压缩后清账+清计数);tests/issue5415-absorb-debt.test.ts(吸收记账算术、欠账+计数、累次累计、参照点已越过 anchor 不记账也计数、陈旧 flag 丢弃)。 - **长会话每回合把 Node 主线程卡死十几秒、网页面板冻结(#5337)**:每回合的压缩筛查把历史切成若干可折叠单元,逐个跑「能否折叠」判定——而每次判定都从零重扫全量消息数组(id 查找无索引,五处线性 `.find()`/`includes`),还重复重算全套整史统计(受保护/过期工具调用集合、工具配对表、末尾锚点、完整性预扫)。总工作量≈单元数×全史长度=历史规模平方级,且同步跑在主线程:2.3万条核心消息实测单回合约17秒(事件循环被占死,请求排队;#5304 同签名报告)。修复=把这批 O(n) 推导提升为每轮筛查**一次**构建(`buildProjectionShared`)、共享给同一轮所有单元;id→序号一次性建表替五处线性查找;「已丢弃」成员判断从 includes-in-some 改为 Set。判定逻辑本身一字未动(#3236 单一判定纪律),只是输入到达判定的方式变了。同机同语料实测:2万条消息完整推荐链 9028ms→约572ms(≈16×),回落到线性量级。单次调用路径(apply/report)保持原有内联计算,输出逐字节不变。kernel v0.0.175。tests/issue5337-shared-screening.test.ts(共享/非共享投影逐字段等价+降级边界一致+n=20000 耗时上界钉);既有 issue3406 曲线钉与 #3236 一致性回归不变。 diff --git a/docs/configuration.ja.md b/docs/configuration.ja.md index 215a6ffc3e..b918fe8a00 100644 --- a/docs/configuration.ja.md +++ b/docs/configuration.ja.md @@ -390,6 +390,7 @@ | `BILI_STORAGE_ARCHIVE_AFTER_DAYS` | integer | 365 | — | GC 年齢スウィープウィンドウ (日) (#3463): これより長くアイドル状態のセッションはアーカイブされる (単一のアーカイブログ (archive/archive.jsonl) に 1 行の JSON が追記 + トombstone 行 + コンテンツ/メディア参照の解放)。0 で年齢スウィープを無効化; 手動エントリ POST /__bili/storage/gc は常に実行される。設定ファイルキー storage.archiveAfterDays が永続形式である。 | | `BILI_STORAGE_QUICK_CHECK` | string | on | — | オープン時のストレージ整合性セルフチェック (#4175): すべての index.db オープン時にマイグレーション前に PRAGMA quick_check を実行; このブートのスキャンによって破損が証明された場合、何も自動的に移動、コピー、再構築されない (自動回復は #4609/#4626 時代に撤回: BILI_STORAGE_JSON_WRITER=off ではインデックスが唯一の永続コピーであり、自動的な移動または再構築では完全なデータ回復を保証できないため) かつ bili はそれでも起動する — JSON エンベロープからサーブし、手動修正 (バックアップしてから index.db/-wal/-shm を手動で退避し、再起動 — 新しいインデックスは JSON エンベロープから再構築される) を名指しする大きくて実行可能なアラームを発する。成功した PASS はキャッシュされ、後の起動で再スキャンをスキップする; FAIL 判定は決して永続化されない (2026-10-30 オーナーポリシー): ランタイム/バックグラウンドの発見はアラームのみ (大きなログ + ランタイムステータス、プロセスはサーブを継続 — jsonWriter が on の間に書かれた JSON エンベロープが回復ソースのまま) であり、すべてのブートは現在のファイルセットの新しいスキャンから再決定するため、古いバージョンが残した古い FAIL が将来のブートをブロックすることは決してない。自動リネーム/修復もレガシー専用フォールバックもない: レガシー専用で続行すると、JSON エンベロープが死んだインデックスから永遠に分岐する (決して調整不能)。一時的なオープン失敗 (SQLite エンジンなし / CANTOPEN / 読み取り専用 / バックエンド DSN) は、実行可能なエラーで bili が大きく起動を拒否する — 環境問題に対するサイレントレガシーフォールバックもない (2026-10-30 オーナーポリシー; ライブな第二ライターがこの拒否リストを 2026-10-31 に残した — 1 つのローカルディスク index.db を共有することはサポートされるトポロジーであり、情報提供のみ、決してブロックしない); 真に JSON のみでサーブしたいデプロイは明示的に設定する (BILI_STORAGE_UNIFIED=legacy — 未設定はピュア SQLite のデフォルトであり legacy ではない)。チェックのコストが重要なブート時感度の高い巨大デプロイでのみ 0 を設定する。 | | `BILI_SNAPSHOT_SHADOW` | boolean | true (on) | — | スナップショットレンダリングのシャドウ観測可能性(#3756 のフォローアップ): アームされたターンごとに送信スナップショットビューを計算するが使用せず、レガシー送信との差分を分類し、セッションごとのカウンターを Web パネルおよびスナップショットシャドウ診断トピックに表示する。0/false/off で無効化、1/true/on で有効化、それ以外の場合は設定ファイルの snapshot.shadow ブール値が適用される(デフォルトは on)。レンダリングレーン自体はロールアウトゲート(BILI_ROLLOUT_SNAPSHOT_RENDER)で制御されるため、シャドウレーンは送信バイト列を変更しない - これは後日のロールアウトを切り替えるためのドライラン証拠である。 | +| `BILI_PERF_TELEMETRY` | boolean-ish | on | — | リアルタイム性能テレメトリのキルスイッチ (#5460)。DB 系ストレージモード (sqlite/mysql) ではデフォルトでオン: 処理された各リクエストが index.db と同じディレクトリの perf.sqlite に 1 行 + 30 秒ごとのメモリウォーターマークサンプルを記録する — 独立したファイルで、ユーザー自身のストアからクラッシュ/サイズ両面で分離; 純 JSON ストレージモードは凍結され、決して収集しない。0/false で収集を完全に無効化。テレメトリのオープン/挿入失敗は収集をデグレードするのみ (1 件の warn + dead フラグ) で、サービングには決して影響しない。 | | `BILI_CONFIG_FILE` | string | unset (XDG config path) | — | 設定ファイルのパスを上書きする(パスの再配置)。スラッシュ `/` を含まない名前(NAME)のみは命名設定 `/NAME.json` を選択します。名前が誤っている場合は利用可能な名前を列挙して即座にエラーとなり、明示的なパスが誤っている場合は従来の空設定の挙動を維持します。 | | `BILI_SESSIONS_DIR` | string | unset (XDG state path) | — | セッションレコードの保存先(パスの再配置)。 | | `BILI_ENCRYPTION_KEY` | string | unset | — | 暗号化された永続化ペイロード用の鍵材料(シークレット)。 | diff --git a/docs/configuration.md b/docs/configuration.md index 6eaf046056..62808a8441 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -393,6 +393,7 @@ This index is generated from `website/config-reference/*.yaml` — edit the seed | `BILI_STORAGE_ARCHIVE_AFTER_DAYS` | integer | 365 | — | GC age sweep window in days (#3463): sessions idle longer than this are archived (one JSON line appended to the single archive log (archive/archive.jsonl) + tombstone row + content/media refs released). 0 disables the age sweep; the manual entry POST /__bili/storage/gc always runs. Config-file key storage.archiveAfterDays is the persistent form. | | `BILI_STORAGE_QUICK_CHECK` | string | on | — | Storage integrity self-check at open (#4175): every index.db open runs PRAGMA quick_check BEFORE migrations; if a corruption is PROVEN by THIS BOOT's own scan NOTHING is moved, copied or rebuilt automatically (auto-recovery was withdrawn #4609/#4626-era: with BILI_STORAGE_JSON_WRITER=off the index is the SOLE durable copy, so an automatic move or rebuild cannot guarantee full data recovery) and bili STARTS ANYWAY — serving from the JSON envelopes with a loud, actionable alarm naming the manual fix (back up then move index.db/-wal/-shm aside BY HAND, then start again — a fresh index rebuilds from the JSON envelopes). A successful PASS is cached so later starts skip the rescan; a FAIL verdict is NEVER persisted (2026-10-30 owner policy): runtime/background findings are alarms only (loud logs + runtime-status, the process keeps serving — JSON envelopes written while the jsonWriter was on remain the recovery source) and every boot decides again from a fresh scan of the CURRENT file set, so a stale FAIL left behind by an older version can never block future boots. There is NO automatic rename/repair AND no legacy-only fallback: continuing legacy-only would fork the JSON envelopes from the dead index forever (never reconcilable). TRANSIENT open failures (no SQLite engine / CANTOPEN / readonly / backend DSN) make bili REFUSE TO START loudly with an actionable error — no silent legacy fallback for environment problems either (2026-10-30 owner policy; a live second writer left this refusal list on 2026-10-31 — sharing one local-disk index.db is the supported topology, informational only, never blocking); deployments that genuinely want JSON-only serve explicitly (BILI_STORAGE_UNIFIED=legacy — leaving it unset is the pure-sqlite default, not legacy). Set 0 only for boot-time-sensitive huge deployments where the check's cost matters. | | `BILI_SNAPSHOT_SHADOW` | boolean | true (on) | — | Snapshot-render shadow observability (#3756 follow-up): compute the outbound snapshot view on every armed turn WITHOUT using it, classify the diff against the legacy outbound, and surface per-session counters in the web panel + the snapshot-shadow diagnostics topic. 0/false/off disables; 1/true/on enables; otherwise the config-file snapshot.shadow boolean applies (default on). The render lane itself stays rollout-gated (BILI_ROLLOUT_SNAPSHOT_RENDER), so the shadow lane never changes outbound bytes - it is the dry-run evidence that flips the rollout later. | +| `BILI_PERF_TELEMETRY` | boolean-ish | on | — | Kill switch for #5460 realtime perf telemetry. On by default in DB-backed storage modes (sqlite/mysql): every handled request writes one row plus a 30 s memory-watermark sample into perf.sqlite next to index.db — a separate file, crash/size-isolated from the user's own store; pure-JSON storage mode is frozen and never collects. 0/false disables collection entirely. A telemetry open/insert failure degrades collection (one warn + dead flag) without ever affecting serving. | | `BILI_CONFIG_FILE` | string | unset (XDG config path) | — | Override the config file path (path relocation). A bare NAME (no `/`) selects the named config `/NAME.json`; a missing NAME fails fast listing the available names, a missing explicit PATH keeps its historical empty-config behavior. | | `BILI_SESSIONS_DIR` | string | unset (XDG state path) | — | Where session records are stored (path relocation). | | `BILI_ENCRYPTION_KEY` | string | unset | — | Key material for encrypted persisted payloads (secret). | diff --git a/docs/configuration.zh-CN.md b/docs/configuration.zh-CN.md index 6209c06540..d0d8bfd9ce 100644 --- a/docs/configuration.zh-CN.md +++ b/docs/configuration.zh-CN.md @@ -393,6 +393,7 @@ | `BILI_STORAGE_ARCHIVE_AFTER_DAYS` | integer | 365 | — | GC 年龄清扫窗口(天,#3463):闲置超过该时长的会话被归档(单文件归档日志 archive/archive.jsonl 追加一行 + 墓碑行保留标签 + 释放内容/媒体引用)。0 关闭年龄清扫;手动入口 POST /__bili/storage/gc 始终可用。配置文件键 storage.archiveAfterDays 为持久形式。 | | `BILI_STORAGE_QUICK_CHECK` | string | on | — | 存储开库自检(#4175):每次 index.db 打开在迁移之前先跑 PRAGMA quick_check;一旦由本次启动自己的扫描证实损坏,文件原样保留在原地——不做任何自动搬移、复制或重建(自动恢复已撤回:BILI_STORAGE_JSON_WRITER=off 时索引是唯一的持久副本,自动操作无法保证完整数据恢复)——bili 照常启动,改用 JSON 信封服务,并给出响亮、可操作的告警指明手工修复步骤(备份后把 index.db/-wal/-shm 手工移开再启动——全新索引会从 JSON 信封重建)。PASS 判定会被缓存,之后的启动跳过重扫;FAIL 判定永不持久化(2026-10-30 owner 策略):运行期/后台发现只是告警(响亮日志 + runtime-status,进程继续服务——jsonWriter 开启期间所写的 JSON 信封是恢复来源),每次启动都对当前文件集做新鲜扫描再判定,旧版本遗留的陈旧 FAIL 永远不能阻断后续启动。既没有自动改名/修复,也没有 legacy-only 兜底(继续以 legacy-only 运行会让 JSON 信封与死掉的索引永久分叉、再无对账可能)。瞬态打开失败(无引擎 / CANTOPEN / 只读 / 后端 DSN)仍响亮地拒绝启动并给出可操作的错误——环境类问题同样不允许静默走 legacy 兜底(活的第二写者已于 2026-10-31 移出该拒绝清单:多进程共享一个本地盘 index.db 是受支持拓扑,只提示、永不阻断启动);真正需要纯 JSON 服务的部署请显式设置 BILI_STORAGE_UNIFIED=legacy(注意:留空现在是纯 SQLite 缺省,不是 legacy)。仅当超大部署对启动耗时敏感、检测成本构成问题时才设 0 跳过。 | | `BILI_SNAPSHOT_SHADOW` | boolean | true (on) | — | 快照渲染影子对账(#3756 后续):每轮已折叠会话都用引擎现算一次出站快照视图但不使用,与 legacy 实际出站比对分类,逐会话计数进 web 面板与 snapshot-shadow 诊断主题。0/false/off 关闭;1/true/on 开启;否则读配置文件 snapshot.shadow 布尔(默认开)。渲染车道本身仍受灰度开关控制(BILI_ROLLOUT_SNAPSHOT_RENDER),影子车道绝不改变出站字节 —— 它是后续翻转灰度的干跑证据。 | +| `BILI_PERF_TELEMETRY` | boolean-ish | on | — | #5460 实时性能采集总开关。DB 存储模式(sqlite/mysql)下默认开启:每处理一个请求落一行,外加每 30 秒一条内存水位采样,写入 index.db 旁边的独立小库 perf.sqlite —— 独立文件,与用户自己的库崩溃/体积隔离;纯 JSON 存储模式冻结、永不采集。0/false 整体关闭采集。任何打开/插入失败只降级采集(一次性告警 + 死标记),绝不影响服务。 | | `BILI_CONFIG_FILE` | string | unset (XDG config path) | — | 覆盖配置文件路径(路径重定位)。不带 `/` 的纯名字 NAME 选择命名配置 `/NAME.json`;名字写错立即报错并列出可用名字,显式路径写错保持历史空配置行为。 | | `BILI_SESSIONS_DIR` | string | unset (XDG state path) | — | 会话记录的存放目录(路径重定位)。 | | `BILI_ENCRYPTION_KEY` | string | unset | — | 加密持久化载荷的密钥材料(机密)。 | diff --git a/docs/lifecycle.md b/docs/lifecycle.md index 97c4400148..627c32cbb9 100644 --- a/docs/lifecycle.md +++ b/docs/lifecycle.md @@ -143,6 +143,7 @@ | `BILI_STORAGE_QUICK_CHECK` | 默认开——index.db 开库在迁移前跑 PRAGMA quick_check 自检;失败=影子层隔离文件组(.corrupt-)+重建新库(JSON 信封为恢复源);=0 跳过(超大库启动时间敏感场景,#4175) | | `BILI_STORAGE_WAL_JOURNAL_SIZE_LIMIT_BYTES` | storage.walJournalSizeLimitBytes——SQLite WAL 回收字节上限(#3250;垃圾/非正值落回文件/默认层,不让 typo 崩 boot) | | `BILI_MYSQL_DSN` | storage.mysql.dsn——mysql 引擎连接串(env > 文件 > "";空 = mysql 不可用走 sqlite,#3388/#3931) | +| `BILI_PERF_TELEMETRY` | `=0/false` 关闭 #5460 实时性能采集(缺省开)。DB 存储模式(sqlite/mysql)下每请求一行 + 30s 内存采样 → sessions 目录旁 `perf.sqlite`;纯 JSON 存储模式冻结不采集;写失败一次性降级停采,绝不影响服务 | ### 2.5 诊断 diff --git a/src/perf-telemetry.ts b/src/perf-telemetry.ts new file mode 100644 index 0000000000..314d24365c --- /dev/null +++ b/src/perf-telemetry.ts @@ -0,0 +1,241 @@ +// #5460 (P1): realtime performance telemetry, SQLite-backed. +// +// Design constraints (owner decisions, 2026-11-01, issue #5460): +// * Metrics land ONLY in SQLite / database-backed storage modes — never in +// JSON files. The pure-JSON ("legacy-only") storage mode is frozen: this +// system has zero footprint there (init("json") is permanently inert). +// * Telemetry must never break serving: every insert is guarded, and a +// failure DEGRADES telemetry (one warn + dead flag), never the request +// path (#5327 adopt discipline: observe without endangering). +// * The metrics DB is a separate file (perf.sqlite) next to index.db, not a +// migration of the user's own store: crash/size isolation from index.db, +// zero schema-version coupling, trivially deletable by the owner. +// * BILI_PERF_TELEMETRY=0 force-disables (registered in LIFECYCLE.md §2.4). +// * Engine acquisition goes through the shared storage driver seam +// (src/storage/driver.js, #2833 order: builtin node:sqlite → host +// better-sqlite3 → self-downloaded prebuilt) — NEVER a direct static +// node:sqlite import: that builtin is absent on Node <22.5 while +// package.json claims engines >=20, and a static import would crash the +// whole CLI at module load instead of degrading telemetry alone. +import * as path from "node:path"; +import { openSqliteDatabase, type SqliteDatabase } from "./storage/driver.js"; +import { sessionsDir } from "./paths.js"; +import { log as loggerLog } from "./logger.js"; + +export const PERF_DB_FILE = "perf.sqlite"; +const PERF_TABLE = "perf_events"; +// Amortized retention sweep: keep the observability file bounded (~tens of +// MB at most). Every Nth insert deletes the oldest rows past the cap — one +// indexed range delete per batch, negligible against an 80 MiB parse pass. +const RETENTION_CAP_ROWS = 200_000; +const RETENTION_SWEEP_EVERY = 500; +const RING_SIZE = 500; +const DEFAULT_MEM_INTERVAL_MS = 30_000; +interface InitOptions { dir?: string; memIntervalMs?: number; log?: (level: "warn" | "error" | "info", msg: string) => void; } + +interface PerfState { + db: SqliteDatabase | null; // lazily opened on first record + mode: string; // "sqlite" | "mysql" | "json"(off) + dir: string | null; + dead: boolean; // first insert/open failure → degrade forever + warned: boolean; + insertsSinceSweep: number; + memTimer: NodeJS.Timeout | null; + log: ((level: "warn" | "error" | "info", msg: string) => void) | null; +} + +const state: PerfState = { + db: null, mode: "", dir: null, dead: false, warned: false, + insertsSinceSweep: 0, memTimer: null, log: null, +}; + +const ring: Array> = []; +function pushRing(row: Record): void { + ring.push(row); + if (ring.length > RING_SIZE) ring.shift(); +} + +function perfEnabledByEnv(): boolean { + const raw = process.env.BILI_PERF_TELEMETRY; + return !(raw === "0" || raw === "false"); +} + +/** #5460 schema stability guard target — a change here must bump this text + * deliberately (same culture as the storage goldens). */ +export const PERF_EVENTS_DDL = + "CREATE TABLE IF NOT EXISTS perf_events (\n" + + " id INTEGER PRIMARY KEY AUTOINCREMENT,\n" + + " ts INTEGER NOT NULL,\n" + + " kind TEXT NOT NULL,\n" + + " session_id TEXT,\n" + + " msgs INTEGER,\n" + + " inbound_bytes INTEGER,\n" + + " outbound_bytes INTEGER,\n" + + " total_ms REAL,\n" + + " parse_ms REAL,\n" + + " payload TEXT\n" + + ")"; + +function dbPath(dir: string): string { return path.join(dir, PERF_DB_FILE); } +// Canonical sessions dir (BILI_SESSIONS_DIR > dataDir()/sessions) — the same +// root index.db lives under; perf.sqlite keeps company without touching it. +const defaultDir = sessionsDir; + +function warnDegrade(w: (level: "warn" | "error" | "info", msg: string) => void, what: string): void { + if (state.warned) return; + state.warned = true; + try { w("warn", `[perf] telemetry degraded: ${what} — metrics collection stopped for this process (serving unaffected)`); } catch { /* last resort */ } +} + +function ensureDb(): SqliteDatabase | null { + if (state.db) return state.db; + // Degrade contract (#5327 discipline): one failure → dead forever. Without + // this check a broken dir would retry opening the DB on EVERY request + // and every mem tick instead of stopping collection once. + if (state.dead) return null; + if (!state.dir) return null; + try { + const { db } = openSqliteDatabase(dbPath(state.dir)); + // synchronous=NORMAL matches index.db (src/storage/schema.ts): under WAL + // the SQLite default (FULL) would fsync on EVERY commit — one per request + // row on the hot path. Telemetry data is re-collectable; batch the fsyncs. + db.exec(`PRAGMA journal_mode=WAL; PRAGMA synchronous=NORMAL; ${PERF_EVENTS_DDL}; CREATE INDEX IF NOT EXISTS idx_perf_events_ts ON ${PERF_TABLE}(ts);`); + state.db = db; + return db; + } catch (e) { + state.dead = true; + warnDegrade(state.log ?? loggerLog, `open failed: ${e instanceof Error ? e.message : String(e)}`); + return null; + } +} + +function insertRow(db: SqliteDatabase, row: Record, w: (level: "warn" | "error" | "info", msg: string) => void): void { + if (state.dead) return; + try { + db.prepare( + `INSERT INTO ${PERF_TABLE}(ts, kind, session_id, msgs, inbound_bytes, outbound_bytes, total_ms, parse_ms, payload) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)`, + ).run( + row.ts as number, row.kind as string, + (row.session_id as string) ?? null, + (row.msgs as number) ?? null, + (row.inbound_bytes as number) ?? null, + (row.outbound_bytes as number) ?? null, + (row.total_ms as number) ?? null, + (row.parse_ms as number) ?? null, + (row.payload_json as string | undefined) ?? null, + ); + pushRing(row); + if (++state.insertsSinceSweep >= RETENTION_SWEEP_EVERY) { + state.insertsSinceSweep = 0; + db.prepare(`DELETE FROM ${PERF_TABLE} WHERE id IN (SELECT id FROM ${PERF_TABLE} ORDER BY id DESC LIMIT -1 OFFSET ?)`).run(RETENTION_CAP_ROWS); + } + } catch (e) { + state.dead = true; + warnDegrade(w, `insert failed: ${e instanceof Error ? e.message : String(e)}`); + } +} + +/** Initialize telemetry for the process. Call ONCE per storage-mode decision + * (SessionStore construction paths); later calls are no-ops (idempotent by + * design — the retry/cold-open path may re-run). mode: + * - "sqlite": unified sidecar active (or the local engine default) + * - "mysql" : network backend lane — metrics still persist locally + * - "json" : pure-JSON storage mode — FROZEN/deprecated, permanently off */ +export function perfTelemetryInit(mode: "sqlite" | "mysql" | "json", opts?: InitOptions): boolean { + // Idempotence FIRST: once armed, every later call (any mode) is a no-op. + // (#5460: a stray init("json") from a secondary path must not downgrade + // an already-running telemetry instance.) + if (state.mode === "sqlite" || state.mode === "mysql") return true; + if (!perfEnabledByEnv()) { state.mode = "off"; return false; } + if (mode === "json") { state.mode = "json"; return false; } + state.mode = mode; + state.dir = opts?.dir ?? defaultDir(); + const log = opts?.log ?? loggerLog; + state.log = log; + state.memTimer = setInterval(() => perfRecordMemSample(log), opts?.memIntervalMs ?? DEFAULT_MEM_INTERVAL_MS); + state.memTimer.unref?.(); + return true; +} + +/** Record one handled request (wired into logRequestCost — the single + * observation point every lane already flows through, #5408). */ +export function perfRecordRequest(r: { + sessionId: string | null; + msgs?: number | null; + inboundBytes?: number | null; + outboundBytes?: number | null; + totalMs?: number | null; + parseMs?: number | null; + extra?: Record; +}, w?: (level: "warn" | "error" | "info", msg: string) => void): void { + if (state.mode === "off" || state.mode === "json") return; + const logFn = w ?? loggerLog; + try { + const db = ensureDb(); + if (!db) return; + const row: Record = { + ts: Date.now(), kind: "req", + session_id: r.sessionId ?? null, + msgs: r.msgs ?? null, + inbound_bytes: r.inboundBytes ?? null, + outbound_bytes: r.outboundBytes ?? null, + total_ms: r.totalMs ?? null, + parse_ms: r.parseMs ?? null, + payload_json: r.extra ? JSON.stringify({ engine: state.mode, ...r.extra }) : JSON.stringify({ engine: state.mode }), + }; + insertRow(db, row, logFn); + } catch (e) { + // Serving-immune contract (#5327/#5460): a malformed `extra` payload + // (BigInt / circular ref) makes JSON.stringify throw — degrade telemetry, + // never propagate into the request hot path that called us. + state.dead = true; + warnDegrade(logFn, `record failed: ${e instanceof Error ? e.message : String(e)}`); + } +} + +/** Memory watermark row — the OOM class (#5403 ×6) finally shows its ramp. */ +export function perfRecordMemSample(w?: (level: "warn" | "error" | "info", msg: string) => void): void { + if (state.mode === "off" || state.mode === "json") return; + const logFn = w ?? loggerLog; + try { + const db = ensureDb(); + if (!db) return; + const m = process.memoryUsage(); + const row: Record = { + ts: Date.now(), kind: "mem", + session_id: null, msgs: null, inbound_bytes: null, outbound_bytes: null, + total_ms: null, parse_ms: null, + payload_json: JSON.stringify({ rss: m.rss, heapUsed: m.heapUsed, external: m.external, arrayBuffers: m.arrayBuffers }), + }; + insertRow(db, row, logFn); + } catch (e) { + state.dead = true; + warnDegrade(logFn, `mem-sample failed: ${e instanceof Error ? e.message : String(e)}`); + } +} + +/** Close everything (process shutdown / tests). Idempotent. */ +export function perfStopAll(): void { + if (state.memTimer) { clearInterval(state.memTimer); state.memTimer = null; } + if (state.db) { try { state.db.close(); } catch { /* best effort */ } state.db = null; } +} + +/** Full state reset — unit tests only (module singleton across test cases in + * one process). */ +export function perfTelemetryResetForTests(): void { + if (state.memTimer) { clearInterval(state.memTimer); state.memTimer = null; } + if (state.db) { try { state.db.close(); } catch { /* best effort */ } state.db = null; } + state.mode = ""; + state.dir = null; + state.dead = false; + state.warned = false; + state.insertsSinceSweep = 0; + state.log = null; + ring.length = 0; +} + +/** In-memory ring of the most recent rows — feeds /stats and the panel + * page without a round-trip to disk (P4 surface). */ +export function perfRecentRows(): readonly Record[] { return ring; } +/** Which engine tag rows carry (also the /stats "storage mode" answer). */ +export function perfActiveMode(): string { return state.mode; } diff --git a/src/persist.ts b/src/persist.ts index 2746ad49f4..54264a823d 100644 --- a/src/persist.ts +++ b/src/persist.ts @@ -39,6 +39,7 @@ import { restampRefsToLedger, restampContentStore } from "acp-kernel"; import { LazyBlockContents, isDroppedContentStore, isDroppedMetadata, lazyContentStore, lazyMetadata, peekSession, sessionWeight } from "./session.js"; import type { Session, BlockContent, BlockView } from "./session.js"; import { orphanContents, resolveStorageMode, UnifiedShadow, hadIntegrityQuickCheckFailure, type StorageMode } from "./storage/shadow.js"; +import { perfTelemetryInit } from "./perf-telemetry.js"; import { noteColdLoad } from "./attach-heartbeat.js"; import { resolveBackendName } from "./storage/backend.js"; import { toLegacyLike, type LegacySessionLike } from "./storage/ingest.js"; @@ -707,6 +708,10 @@ export class SessionStore { fileStorageMysqlDsn(), ).then((sh) => { this.shadow = sh ?? undefined; + // #5460 P1: DB-backed mode → telemetry armed (local perf.sqlite). + // Pure-JSON ("legacy") mode stays frozen even when it rides this + // branch (a mysql backend with storage.unified=legacy) — zero footprint. + perfTelemetryInit(storageMode === "legacy" ? "json" : "mysql", { dir: this.sessionsDir, log: baseLog }); }); // #5046 chain (2026-10-30): openAsync now REJECTS on backend failure // (no degradation in any case). Boot awaits ready() and exits @@ -728,8 +733,14 @@ export class SessionStore { // closed for a full cooldown window. this.syncSidecarOpen = { legacyLoad }; this.shadow = UnifiedShadow.maybeOpen(this.sessionsDir, process.env, legacyLoad, fileStorageUnified(), fileStorageWalJournalSizeLimit(), fileStorageBackend()) ?? undefined; + // #5460 P1: DB-backed mode → telemetry armed (local perf.sqlite). + // Pure-JSON ("legacy") mode flows through this branch too + // (maybeOpen returns null for it) but stays frozen — zero footprint. + perfTelemetryInit(storageMode === "legacy" ? "json" : "sqlite", { dir: this.sessionsDir, log: baseLog }); } else { + // #5460: pure-JSON storage mode is frozen — telemetry stays off there. this.shadow = undefined; + perfTelemetryInit("json", { log: baseLog }); } } this.store = new StateStore({ diff --git a/src/server/request-cost.ts b/src/server/request-cost.ts index d3dfb69bd0..f165bb9a73 100644 --- a/src/server/request-cost.ts +++ b/src/server/request-cost.ts @@ -1,4 +1,5 @@ import { noteRequest } from "../attach-heartbeat.js"; +import { perfRecordRequest } from "../perf-telemetry.js"; import { currentFetchTransport } from "../fetch-transport.js"; // #903: per-request local-cost line, logged at every point where bili finishes @@ -39,7 +40,11 @@ export function _resetSlowSessionWarnedForTest(): void { export function logRequestCost(log: (level: string, msg: string) => void, sessionId: string, msgs: number | null, inboundBytes: number, t0: number, outbound?: string | Buffer): void { const ms = Math.max(0, Math.round(performance.now() - t0)); - const outboundField = outbound !== undefined ? `, outbound=${formatBytes(Buffer.byteLength(outbound))}` : ""; + const outboundBytes = outbound !== undefined ? Buffer.byteLength(outbound) : null; + // #5460 P1: structured mirror of this line into perf.sqlite (inert unless + // a DB-backed storage mode armed the module; degrades, never throws). + perfRecordRequest({ sessionId, msgs, inboundBytes, outboundBytes, totalMs: ms }); + const outboundField = outboundBytes !== null ? `, outbound=${formatBytes(outboundBytes)}` : ""; const view = currentFetchTransport() ? ", view=ws-expanded" : ""; log("info", `[${sessionId}] request: ${msgs ?? "?"} msgs, inbound=${formatBytes(inboundBytes)}${outboundField}, local=${ms}ms${view}`); // #5408: this line is THE observation point — a hot-at-attach session that diff --git a/tests/issue5460-perf-telemetry.test.ts b/tests/issue5460-perf-telemetry.test.ts new file mode 100644 index 0000000000..f599fb74ab --- /dev/null +++ b/tests/issue5460-perf-telemetry.test.ts @@ -0,0 +1,256 @@ +// #5460 (P1): realtime perf telemetry — SQLite-backed, DB-modes only. +// +// Schema-stability culture mirrors the storage goldens: the DDL text is +// pinned, the file layout is pinned, and every degradation path must prove +// it leaves serving untouched (telemetry NEVER throws into the request path). +import { test, after } from "node:test"; +import assert from "node:assert/strict"; +import { existsSync, mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import * as os from "node:os"; +import * as path from "node:path"; +import { DatabaseSync } from "node:sqlite"; +import { + perfTelemetryInit, perfRecordRequest, perfRecordMemSample, + perfStopAll, perfRecentRows, perfActiveMode, perfTelemetryResetForTests, + PERF_DB_FILE, PERF_EVENTS_DDL, +} from "../src/perf-telemetry.js"; + +const SANDBOXES: string[] = []; +function sandbox(): string { + const dir = mkdtempSync(path.join(os.tmpdir(), "bili-5460-")); + SANDBOXES.push(dir); + return dir; +} + +function resetModule(): void { + perfTelemetryResetForTests(); + delete process.env.BILI_PERF_TELEMETRY; +} + +test("#5460 P1: sqlite mode records request rows into /perf.sqlite", () => { + resetModule(); + const dir = sandbox(); + try { + assert.equal(perfTelemetryInit("sqlite", { dir }), true); + assert.equal(perfActiveMode(), "sqlite"); + const beforeTs = Date.now(); + perfRecordRequest({ sessionId: "sess-x", msgs: 42, inboundBytes: 123456, outboundBytes: 789, totalMs: 12.4, extra: { lane: "dsh" } }); + const db = new DatabaseSync(path.join(dir, PERF_DB_FILE)); + try { + const row = db.prepare("SELECT * FROM perf_events WHERE kind='req'").get() as Record; + assert.equal(row.session_id, "sess-x"); + assert.equal(row.msgs, 42); + assert.equal(row.inbound_bytes, 123456); + assert.equal(row.outbound_bytes, 789); + assert.equal(row.total_ms, 12.4); + assert.ok(Number(row.ts) >= beforeTs && Number(row.ts) <= Date.now()); + const payload = JSON.parse(String(row.payload)) as Record; + assert.equal(payload.engine, "sqlite"); + assert.equal(payload.lane, "dsh"); + } finally { db.close(); } + } finally { perfStopAll(); } +}); + +test("#5460 P1: schema is pinned (DDL text + ts index + WAL) — golden discipline", () => { + resetModule(); + const dir = sandbox(); + try { + perfTelemetryInit("sqlite", { dir }); + perfRecordRequest({ sessionId: "s" }); + const db = new DatabaseSync(path.join(dir, PERF_DB_FILE)); + try { + const ddl = db.prepare("SELECT sql FROM sqlite_master WHERE type='table' AND name='perf_events'").get() as { sql: string }; + // sqlite_master stores the statement verbatim minus the idempotence + // prefix, so normalize both sides to the CREATE TABLE body. + assert.equal(ddl.sql.trim().replace("IF NOT EXISTS ", ""), PERF_EVENTS_DDL.trim().replace("IF NOT EXISTS ", "")); + const idx = db.prepare("SELECT name FROM sqlite_master WHERE type='index' AND name='idx_perf_events_ts'").get(); + assert.ok(idx, "ts index must exist"); + const mode = db.prepare("PRAGMA journal_mode").get() as { journal_mode: string }; + assert.equal(String(mode.journal_mode).toLowerCase(), "wal"); + } finally { db.close(); } + } finally { perfStopAll(); } +}); + +test("#5460 P1: pure-JSON storage mode is frozen — permanently inert", () => { + resetModule(); + const dir = sandbox(); + try { + assert.equal(perfTelemetryInit("json", { dir }), false); + assert.equal(perfActiveMode(), "json"); + perfRecordRequest({ sessionId: "s", msgs: 1 }); + perfRecordMemSample(); + assert.ok(!existsSync(path.join(dir, PERF_DB_FILE)), "json mode must never create a metrics file"); + assert.equal(perfRecentRows().length, 0); + } finally { perfStopAll(); } +}); + +test("#5460 P1: BILI_PERF_TELEMETRY=0 kill switch wins over any mode", () => { + resetModule(); + const dir = sandbox(); + process.env.BILI_PERF_TELEMETRY = "0"; + try { + assert.equal(perfTelemetryInit("sqlite", { dir }), false); + assert.equal(perfActiveMode(), "off"); + perfRecordRequest({ sessionId: "s" }); + perfStopAll(); + assert.ok(!existsSync(path.join(dir, PERF_DB_FILE))); + } finally { + perfTelemetryResetForTests(); + delete process.env.BILI_PERF_TELEMETRY; + } +}); + +test("#5460 P1: open failure degrades ONCE — no throw into the request path, no retry", () => { + resetModule(); + const dir = sandbox(); + // A non-SQLite regular file at the db path makes the first exec throw + // deterministically ("file is not a database") — the open-failure path. + writeFileSync(path.join(dir, PERF_DB_FILE), "not a sqlite database"); + const warns: string[] = []; + try { + assert.equal(perfTelemetryInit("sqlite", { dir, log: (l, m) => warns.push(`${l}:${m}`) }), true); + perfRecordRequest({ sessionId: "s1" }); + perfRecordRequest({ sessionId: "s2" }); + perfRecordMemSample(); + const degraded = warns.filter((w) => w.startsWith("warn:") && w.includes("telemetry degraded")); + assert.equal(degraded.length, 1, `exactly ONE degrade warn, got ${degraded.length}`); + assert.ok(degraded[0].includes("open failed"), "warn must name the open failure"); + assert.equal(perfRecentRows().length, 0, "no rows may be recorded while dead"); + } finally { perfStopAll(); } +}); + +test("#5460 P1: init is idempotent — later calls never re-arm or throw", () => { + resetModule(); + const dir = sandbox(); + try { + perfTelemetryInit("sqlite", { dir }); + assert.equal(perfTelemetryInit("mysql", {}), true, "already armed → no-op true"); + assert.equal(perfTelemetryInit("json", {}), true); + assert.equal(perfActiveMode(), "sqlite"); + } finally { perfStopAll(); } +}); + +test("#5460 P1: mem sampler rows land with real memory numbers", async () => { + resetModule(); + const dir = sandbox(); + try { + perfTelemetryInit("sqlite", { dir, memIntervalMs: 5 }); + await new Promise((r) => setTimeout(r, 80)); + perfStopAll(); + const db = new DatabaseSync(path.join(dir, PERF_DB_FILE)); + try { + const rows = db.prepare("SELECT payload FROM perf_events WHERE kind='mem'").all() as Array<{ payload: string }>; + assert.ok(rows.length >= 2, `expected repeated mem samples, got ${rows.length}`); + const p = JSON.parse(rows[0].payload) as Record; + assert.ok(p.rss > 0 && p.heapUsed > 0 && p.external >= 0); + } finally { db.close(); } + } finally { perfStopAll(); } +}); + +test("#5460 P1: ring buffer serves in-process consumers (/stats, panel P4)", () => { + resetModule(); + const dir = sandbox(); + try { + perfTelemetryInit("sqlite", { dir }); + perfRecordRequest({ sessionId: "a", inboundBytes: 1 }); + perfRecordRequest({ sessionId: "b", inboundBytes: 2 }); + const ring = perfRecentRows(); + assert.equal(ring.length, 2); + assert.equal((ring[1] as Record).session_id, "b"); + } finally { perfStopAll(); } +}); + +test("#5460 P1: logRequestCost wires structured rows through the single choke point", async () => { + // Late import keeps the heavy server module out of the fast paths above; + // importing ../src/server.js in tests is an established pattern. + const { logRequestCost } = await import("../src/server/request-cost.js"); + resetModule(); + const dir = sandbox(); + try { + assert.equal(perfTelemetryInit("sqlite", { dir }), true); + const logs: string[] = []; + logRequestCost((lvl, msg) => logs.push(`${lvl}:${msg}`), "sess-wired", 7, 2048, performance.now(), "body-out"); + perfStopAll(); + const db = new DatabaseSync(path.join(dir, PERF_DB_FILE)); + let row: Record | undefined; + try { + row = db.prepare("SELECT * FROM perf_events WHERE kind='req' AND session_id='sess-wired'").get() as Record; + } finally { db.close(); } + assert.ok(row, "logRequestCost must emit a telemetry row"); + assert.equal(row!.msgs, 7); + assert.equal(row!.inbound_bytes, 2048); + assert.equal(row!.outbound_bytes, Buffer.byteLength("body-out")); + assert.ok(Number(row!.total_ms) >= 0); + assert.ok(logs.some((l) => l.includes("request:")), "original human line untouched"); + } finally { perfStopAll(); } +}); + +test("#5460 P1: retention sweep caps perf_events at the 200k-row window — oldest drop first", () => { + resetModule(); + const dir = sandbox(); + try { + assert.equal(perfTelemetryInit("sqlite", { dir }), true); + // Stop EXACTLY on a sweep boundary past the cap (a multiple of the + // 500-insert sweep period): the sweep that just fired must leave the + // table at precisely the cap, newest kept. Between sweeps the table + // may hold up to cap+period-1 (amortized by design). + for (let i = 0; i < 200_500; i++) { + perfRecordRequest({ sessionId: `r${i}`, inboundBytes: i }); + } + perfStopAll(); + const db = new DatabaseSync(path.join(dir, PERF_DB_FILE)); + try { + const n = db.prepare("SELECT COUNT(*) AS n FROM perf_events").get() as { n: number }; + assert.equal(Number(n.n), 200_000, "retention cap must hold at the sweep boundary"); + const newest = db.prepare("SELECT session_id FROM perf_events ORDER BY id DESC LIMIT 1").get() as { session_id: string }; + assert.equal(newest.session_id, "r200499", "newest rows survive, oldest drop"); + } finally { db.close(); } + } finally { perfStopAll(); } +}); + +test("#5460 P1: legacy (pure-JSON) storage mode leaves ZERO telemetry footprint — owner constraint ②", async () => { + // The sqlite arming branch runs for every enabled store — including + // storage.unified=legacy ones where maybeOpen() returns null. This pins + // that such a store still never creates perf.sqlite. + const { SessionStore } = await import("../src/persist.js"); + resetModule(); + const savedUnified = process.env.BILI_STORAGE_UNIFIED; + const savedKey = process.env.BILI_ENCRYPTION_KEY; + delete process.env.BILI_ENCRYPTION_KEY; + process.env.BILI_STORAGE_UNIFIED = "legacy"; + const dir = sandbox(); + try { + new SessionStore({ enabled: true, dir }); + perfStopAll(); + assert.equal(perfActiveMode(), "json", "legacy mode must arm the frozen (inert) engine"); + assert.ok(!existsSync(path.join(dir, PERF_DB_FILE)), "pure-JSON mode must never create a metrics file"); + } finally { + if (savedUnified === undefined) delete process.env.BILI_STORAGE_UNIFIED; else process.env.BILI_STORAGE_UNIFIED = savedUnified; + if (savedKey === undefined) delete process.env.BILI_ENCRYPTION_KEY; else process.env.BILI_ENCRYPTION_KEY = savedKey; + perfTelemetryResetForTests(); + } +}); + +test("#5460 P1: a malformed `extra` payload degrades telemetry, never throws into the caller", () => { + resetModule(); + const dir = sandbox(); + const warns: string[] = []; + try { + const rec: (level: "warn" | "error" | "info", msg: string) => void = (l, m) => warns.push(`${l}:${m}`); + assert.equal(perfTelemetryInit("sqlite", { dir, log: rec }), true); + // BigInt is not JSON-serializable: JSON.stringify throws. The record + // path must swallow it as a one-shot degrade, not propagate it into + // the caller (which is the request hot path via logRequestCost). + assert.doesNotThrow(() => { + perfRecordRequest({ sessionId: "s-bigint", extra: { bad: 10n } }, rec); + }); + const degraded = warns.filter((w) => w.startsWith("warn:") && w.includes("telemetry degraded")); + assert.equal(degraded.length, 1, "exactly ONE degrade warn for a malformed payload"); + assert.ok(degraded[0].includes("record failed"), "warn must name the record failure"); + // Dead now: subsequent records are dropped silently (no throw, no row). + perfRecordRequest({ sessionId: "s-after" }, rec); + assert.equal(perfRecentRows().length, 0, "no rows may be recorded while dead"); + } finally { perfStopAll(); } +}); + +after(() => { for (const d of SANDBOXES.splice(0)) rmSync(d, { recursive: true, force: true }); }); diff --git a/tests/preload-test-env.mjs b/tests/preload-test-env.mjs index 4f7a373f12..91208d649c 100644 --- a/tests/preload-test-env.mjs +++ b/tests/preload-test-env.mjs @@ -51,3 +51,10 @@ for (const name of ["sessions", "data", "state", "cache", "config"]) mkdirSync(p // tests/message-timestamps-default-on.test.ts). Storage-layer stamping runs // regardless of this flag. process.env.BILI_MESSAGE_TIMESTAMPS = "0"; +// #5460: pin realtime perf telemetry OFF suite-wide. SessionStore auto-arms +// it (src/persist.ts), and its persistent WAL handle makes Windows runners +// fail every sandbox teardown that rimrafs a sessions dir +// (EBUSY …/sessions/perf.sqlite — the whole w22 matrix, run of 2026-11-01). +// Tests that need it opt in per-file by DELETING the key (the dedicated +// tests/issue5460-perf-telemetry.test.ts does exactly that). +process.env.BILI_PERF_TELEMETRY = "0"; diff --git a/website/config-reference/env-only.yaml b/website/config-reference/env-only.yaml index b34c86ce13..f34f4bd1cf 100644 --- a/website/config-reference/env-only.yaml +++ b/website/config-reference/env-only.yaml @@ -83,6 +83,15 @@ entries: desc_en: "Snapshot-render shadow observability (#3756 follow-up): compute the outbound snapshot view on every armed turn WITHOUT using it, classify the diff against the legacy outbound, and surface per-session counters in the web panel + the snapshot-shadow diagnostics topic. 0/false/off disables; 1/true/on enables; otherwise the config-file snapshot.shadow boolean applies (default on). The render lane itself stays rollout-gated (BILI_ROLLOUT_SNAPSHOT_RENDER), so the shadow lane never changes outbound bytes - it is the dry-run evidence that flips the rollout later." desc_zh: "快照渲染影子对账(#3756 后续):每轮已折叠会话都用引擎现算一次出站快照视图但不使用,与 legacy 实际出站比对分类,逐会话计数进 web 面板与 snapshot-shadow 诊断主题。0/false/off 关闭;1/true/on 开启;否则读配置文件 snapshot.shadow 布尔(默认开)。渲染车道本身仍受灰度开关控制(BILI_ROLLOUT_SNAPSHOT_RENDER),影子车道绝不改变出站字节 —— 它是后续翻转灰度的干跑证据。" desc_ja: "スナップショットレンダリングのシャドウ観測可能性(#3756 のフォローアップ): アームされたターンごとに送信スナップショットビューを計算するが使用せず、レガシー送信との差分を分類し、セッションごとのカウンターを Web パネルおよびスナップショットシャドウ診断トピックに表示する。0/false/off で無効化、1/true/on で有効化、それ以外の場合は設定ファイルの snapshot.shadow ブール値が適用される(デフォルトは on)。レンダリングレーン自体はロールアウトゲート(BILI_ROLLOUT_SNAPSHOT_RENDER)で制御されるため、シャドウレーンは送信バイト列を変更しない - これは後日のロールアウトを切り替えるためのドライラン証拠である。" + - key: BILI_PERF_TELEMETRY + type: boolean-ish + default: "on" + kind: env-only + audience: user + group: envonly + desc_en: "Kill switch for #5460 realtime perf telemetry. On by default in DB-backed storage modes (sqlite/mysql): every handled request writes one row plus a 30 s memory-watermark sample into perf.sqlite next to index.db — a separate file, crash/size-isolated from the user's own store; pure-JSON storage mode is frozen and never collects. 0/false disables collection entirely. A telemetry open/insert failure degrades collection (one warn + dead flag) without ever affecting serving." + desc_zh: "#5460 实时性能采集总开关。DB 存储模式(sqlite/mysql)下默认开启:每处理一个请求落一行,外加每 30 秒一条内存水位采样,写入 index.db 旁边的独立小库 perf.sqlite —— 独立文件,与用户自己的库崩溃/体积隔离;纯 JSON 存储模式冻结、永不采集。0/false 整体关闭采集。任何打开/插入失败只降级采集(一次性告警 + 死标记),绝不影响服务。" + desc_ja: "リアルタイム性能テレメトリのキルスイッチ (#5460)。DB 系ストレージモード (sqlite/mysql) ではデフォルトでオン: 処理された各リクエストが index.db と同じディレクトリの perf.sqlite に 1 行 + 30 秒ごとのメモリウォーターマークサンプルを記録する — 独立したファイルで、ユーザー自身のストアからクラッシュ/サイズ両面で分離; 純 JSON ストレージモードは凍結され、決して収集しない。0/false で収集を完全に無効化。テレメトリのオープン/挿入失敗は収集をデグレードするのみ (1 件の warn + dead フラグ) で、サービングには決して影響しない。" - key: BILI_CONFIG_FILE type: string default: "unset (XDG config path)"