Skip to content

[PR] fix: [instances] warning judges by sessions directory, not lane alone (#5046) - #5075

Closed
ranxianglei wants to merge 9 commits into
masterfrom
2026-10-29_instance-warn-dir-criterion
Closed

ranxianglei wants to merge 9 commits into
masterfrom
2026-10-29_instance-warn-dir-criterion

Conversation

@ranxianglei

@ranxianglei ranxianglei commented Oct 29, 2026 •

Copy link
Copy Markdown
Owner

(ai: vllm-qwen/qwen3.8-27b)

Closes #5046 — owner decision (2026-10-29, ruled in #5068): B — a second instance opening the same index.db degrades to legacy-only at open time; S2 threshold stays 3.

Scope (3 labeled commits)

1. [instances] warning judged by sessions directory, not lane alone

2. Second-writer degrade (decision B)

3. README en+zh: single-machine multi-install isolation section (BILI_SESSIONS_DIR + BILI_CONFIG_FILE recipe).

Behavior change disclosure

  1. Isolated (different-dir) coexisting instances: [instances] notice warn→info, new wording (kills the false positive from SQLite cache has no graceful degradation when the driver is unavailable: llm/stream throws `Cannot read properties of un | sqlite 缓存在驱动不可用时没有降级:llm/stream 每回合抛 Cannot read properties of undefined (reading 'prepare')(DSH/Electron 打包环境) #5046 field evidence). Same-dir or undeterminable: byte-identical existing warn.
  2. Same-dir same-lane (or unknown-lane) coexisting instances: "error log + keep running as second writer" → "error log + degrade to legacy-only". Intended (this is the corruption path SQLite cache has no graceful degradation when the driver is unavailable: llm/stream throws `Cannot read properties of un | sqlite 缓存在驱动不可用时没有降级:llm/stream 每回合抛 Cannot read properties of undefined (reading 'prepare')(DSH/Electron 打包环境) #5046 reports); reversible by isolation alone (no file changes, no migration).
  3. Status wire: the condition already surfaced as the structured secondProcess field; now the same fact ALSO flips degraded=true / effective=legacy with an actionable reason (before, the install reported healthy unified while double-writing).

Verification

  • tsc clean on head 950bfaa (base master 5594c13).
  • Targeted local: tests/issue5046-second-writer-degrade.test.ts — 6 rows: live same-lane ⇒ degrade + foreign lock untouched + actionable log + status wire; different-lane ⇒ still opens (topology unchanged); stale lock ⇒ normal open+takeover; clean dir ⇒ original fast path; old marker without lane ⇒ conservative degrade; direct UnifiedStore.open ⇒ report-only unchanged. Plus storage-durability / storage-engine-proofs / storage-ingest-watermark / instance-governance / launcher = 400 pass / 1 gated-skip / 0 fail.
  • Full-suite verdict: remote CI on this head.

ci-check added 2 commits October 29, 2026 07:58
…ry, not lane alone (#5046)

The storage-side second-writer guard keys on the sessions directory while
the [instances] guard keyed on lane overlap — after isolating a second
install via BILI_SESSIONS_DIR, the warn kept claiming 'both processes will
write the same sessions directory', a factually wrong false positive that
drowns the real signal. Registry markers now carry the resolved sessions
dir; when both sides report different dirs the notice drops to info level
(unknown dir on either side stays conservative = warn). Both the startup
warning and the #2401 slow-tick rescan use the same criterion.
…#5046)

Two installs on one machine (e.g. two DSH profiles) default to the same
XDG sessions dir + config; with storage.unified on they double-write one
index.db (SQLITE_BUSY → failed turns). Document the BILI_SESSIONS_DIR +
BILI_CONFIG_FILE isolation recipe in both languages.
@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-29_instance-warn-dir-criterion (1e26f38)

Latest build — install this exact version

npm install -g billion-context@0.0.7176-pr.5075.16f5e17

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
1e26f38 billion-context@0.0.7176-pr.5075.16f5e17 build
9d8b15e billion-context@0.0.6834-pr.5075.238da73 build
41b8697 billion-context@0.0.6764-pr.5075.07c3b5b build
6f44206 billion-context@0.0.6759-pr.5075.9f9b76e build
7d02228 billion-context@0.0.6673-pr.5075.c408538 build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr5075.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei ranxianglei changed the title fix: [instances] warning judges by sessions directory, not lane alone (#5046) [PR] fix: [instances] warning judges by sessions directory, not lane alone (#5046) Oct 29, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重: 不重复。 关联项: 父 issue #5046(本 PR 即其"不依赖决策部分"的修复);#5015(修 #5014,storage 面板 banner 侧的姊妹改动,不同表面,互补——见下);#4882(增量车道会话级双写标记,机制不同)。无重复在途 PR。

标签: 应为 bug dsh scope/storage p3(误报=日志噪音;证据来自 DSH 双安装;sessions dir 域)。镜像 label API 按文档格式(labelIds/names/labels/裸数组/add-remove,POST+PUT)全部尝试后均未生效(POST 返回空 current、PUT method not allowed)——平台 shim 限制,请 owner 手动补打。

Review 结论 — 结构化八问

本 PR 是我此前处理 #5046 时开出的(fix commit 1120610eb + docs commit 7d02228c0,分支已 rebase 到当前 master 5594c1315,零落后)。本轮独立重审了 diff、两个调用点、wire 兼容与测试,并本地跑定向测试 + 读 CI 缓存。

  1. 是什么问题: 同一台机器上两个 bili 安装用 BILI_SESSIONS_DIR 隔离 sessions 目录后,[instances] 警告仍声称"both processes will write the same sessions directory"——事实错误的误报,还带着错误的"stop one"建议。
  2. 什么原因导致的: 无单实例守卫:多个 bili 代理进程并存共写 sessions 目录,伴随孤儿 ref、有注册表无 transcript、计数自相矛盾的污染簇 #394 instances 守卫只判 lane 重叠、不判状态位置;而 storage 侧守卫(The same sessionId is served by two bili processes simultaneously: interleaved logs from global + pi-native dual instanc | 同一 sessionId 被两个 bili 进程同时服务:全局 + pi-native 双实例日志交织、重复 cold-load(split-brain) #3923/SQLite unified storage degraded to legacy-only (bili v0.1.202, node v24.21.0, darwin-x64) index.db corrupted | SQLite unified storage degraded to legacy-only (bili v0.1.202, node v24.21.0, darwin-x64) index.db损坏 #3959)按实际 index.db/sessions dir 判。隔离后两进程共享任何状态都不再成立,但 instances 守卫的判据没跟上 → 两道守卫说不同的话。
  3. 影响范围与严重程度: 仅"单机多安装且至少一侧已隔离 sessions dir"的机器(如 SQLite cache has no graceful degradation when the driver is unavailable: llm/stream throws `Cannot read properties of un | sqlite 缓存在驱动不可用时没有降级:llm/stream 每回合抛 Cannot read properties of undefined (reading 'prepare')(DSH/Electron 打包环境) #5046 的 DSH desktop+bot 拓扑)。严重度: 纯日志噪音+误导性建议,无数据风险、无功能损坏。p3。
  4. 怎么修复的: registry marker 增加可选 sessionsDir 字段(仅 registry marker——proxy-origin discovery 文件形状不变,atomicWriteInstanceFile(instanceRecord) 在 src/server.ts:1190 写的仍是无该字段的记录);两侧都有值且归一化后不同(win32 大小写不敏感)→ 对那个 peer 的通知降为 info(新措辞 "different sessions directories … no shared state, informational only");任一侧无值(修复前的旧 marker)→ 保守维持原 warn。启动注册与 [Bug][State] stale-write guard preserves newer disk record but stale resident keeps serving old in-memory fold state #2401 的 60s rescan 两个调用点用同一判据(sessionsDirsDiffer)。
  5. 是否有回归问题: 逐项过既有行为——同目录 peer: warn 措辞逐字节不变 ✓;旧构建 marker(无 dir): 保守 warn 不变 ✓(新增测试行 d 钉住);跨 lane 静默: lanesOverlap 门未动 ✓;每 peer 只告警一次(warnedPeers 语义): 返回值列表未变 ✓;旧读新 marker: coerceEntry 只挑已知字段,多余字段忽略 ✓;新读旧: undefined → 保守 ✓。高危模式排查: 无 fail-fast 变吞错、无阈值/时序漂移;输出格式变化仅限 intended 的 different-dir 分支(PR 正文已披露旧值→新值)。一个诚实边界: 比较是字面的(path.resolve,不解析符号链接)——若两条路径经 symlink 指向同一物理目录会判成"different"降为 info;但该情形下 storage 侧文件级守卫(src/storage/store.ts:443)打开时仍会报 error,最坏结果 = 报错时机从启动推迟到 open,严格优于现状 → 判定可接受,不加 realpathSync(会引入 ENOENT 失败模式,收益边际)。验证: 本地定向 tests/instance-governance.test.ts + tests/launcher.test.ts = 362 pass / 0 fail;CI(head 7d02228c0)= 14 success / 16 gated-skip / 0 failed / 0 pending(skip 均为门控 e2e/theorem 车道,设计内)。
  6. 是否破坏缓存: 不涉及(不触碰前缀缓存/压缩缓存/重试游标/去重键)。
  7. 是否改了设计方向: 否。无架构变化,只是让 instances 守卫与 storage 守卫对齐到同一判据(lane 重叠 ∧ 目录可证不同才降级)。SQLite cache has no graceful degradation when the driver is unavailable: llm/stream throws `Cannot read properties of un | sqlite 缓存在驱动不可用时没有降级:llm/stream 每回合抛 Cannot read properties of undefined (reading 'prepare')(DSH/Electron 打包环境) #5046 的决策部分(second-writer 处置策略 degrade vs keep-running)仍在 #5064 等 owner 拍板,本 PR 不碰,因此不关 SQLite cache has no graceful degradation when the driver is unavailable: llm/stream throws `Cannot read properties of un | sqlite 缓存在驱动不可用时没有降级:llm/stream 每回合抛 Cannot read properties of undefined (reading 'prepare')(DSH/Electron 打包环境) #5046。
  8. 是否引入了新的配置: 无。BILI_SESSIONS_DIR/BILI_CONFIG_FILE 均为既有 env 变量,本 PR 仅在 README en+zh 的"单机多安装共存(SQLite cache has no graceful degradation when the driver is unavailable: llm/stream throws `Cannot read properties of un | sqlite 缓存在驱动不可用时没有降级:llm/stream 每回合抛 Cannot read properties of undefined (reading 'prepare')(DSH/Electron 打包环境) #5046)"小节做了文档化;sessionsDir 是 registry 内部字段,非用户配置面。

与 #5015 的关系(合并顺序提示): 两者表面互补——#5015 改 storage 面板 banner 的分类(已知 pid → benign/info),本 PR 改 [instances] 日志行;都碰 src/instance.ts 但是不同函数/字段,无语义冲突,可能有文本冲突。建议先合任一、另一个 rebase 即可。

测试有效性: 新增两个测试双向断言齐备(#3617 纪律): different-dir → info(正例)、same-dir → 原 warn(反例)、legacy marker → 全保守 warn(反例)、marker 落盘 round-trip、rescan 路径同判据。断言有牙,bug 回归时抓得住。

判定: 可合。 CI 绿、定向测试绿、diff 干净(2 个标注清晰的独立 commit,fix/docs 分离)、行为变更已披露。mergeable_state=blocked 来自 branch protection 要求(checks 本身零失败),合并由 owner 操作。

ci-check added 2 commits October 29, 2026 17:45
…/B degrade policy

store.ts second-process detection conflicted between two owner decisions:
- #5091 (master): a live foreign pid opening index.db is always informational
  (SQLite WAL multi-process open is safe on a local disk); no corruption alarm.
- #5046/B (this branch): the unified shadow opts into secondWriterPolicy:degrade
  and refuses to become a persistent second writer when the peer is same-lane or
  lane-unknown (the risky double-write shape), degrading to legacy-only.

Reconciliation: the degrade gate keys on cross-lane safety (a local decision);
reporting stays informational for every holder and wire benign stays true (#5091).
Aligns the two #5046/B test assertions that still referenced pre-#5091 wording.
@LHN-xiao-hai-tun

Copy link
Copy Markdown

撤回对 B 的拍板,理由写在 #5046:#5046 (comment)

要点(针对本 PR):提交 3 950bfaac(secondWriterPolicy:"degrade" + StorageSecondWriterError + tests/issue5046-second-writer-degrade.test.ts)建立在「打开同一个 index.db = 第二持久写者 = 损坏」这个前提上,而该前提已被 #5082 否掉(store.ts L441 现文把多实例共用 sessions dir 定为 supported topology,L442 记 benign: true)。本 PR 现在 CONFLICTING,冲突全部来自提交 3;提交 1 1120610e 与提交 2 7d02228c 与 master 三方合并实测 0 冲突。

建议:回退到 7d02228c,即只保留「按 sessions 目录判 [instances] 警告」+ README 多安装隔离指南两项。提交 1 修的正是最初 issue 里「两道守卫判据不一致」(目录 vs lane)那条腿,与 #5082 不冲突。

另外订正一处:我先前说「本机第二安装已不存在」是错的——D:\RJ\ai\AI厂商应用\DSH 社区\社区 DS\ 完好,_bili-state\sessions 676 文件 / 809.5 MB。

我没有本仓库 push 权限(push=false),以上是请求,动手权在仓库侧。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

冲突已解并推送。 新 head 6f44206ed(merge commit,parents = 本分支 950bfaace + master 1fad21de8),普通 push(非 force,快进)。

这次冲突不是纯文本,是两条 owner 决策撞在同一段代码上

冲突只落在 src/storage/store.ts 的「第二进程检测」块;其余文件全部自动合并。根因:本分支从旧 base 切出、未包含已合入 master 的 #5091,于是两边在同一段代码上各自改了不同方向——

我的调和方式(含一个判断点,请确认)

两个概念解耦,各归各:

  1. 报告路径(report 默认,CLI/export 等直开)→ 采纳 [PR] Second process opening index.db falsely reports "will corrupt data" — change to a shared-mode warning (#5082) | 第二个进程打开 index.db 误报「会损坏数据」——改为共享形态的提示(#5082) #5091:外来 pid 一律 info + benign:true。不回退已合入的 [PR] Second process opening index.db falsely reports "will corrupt data" — change to a shared-mode warning (#5082) | 第二个进程打开 index.db 误报「会损坏数据」——改为共享形态的提示(#5082) #5091 修复。
  2. 降级路径(仅 unified shadow 显式选入)→ 保留 SQLite cache has no graceful degradation when the driver is unavailable: llm/stream throws `Cannot read properties of un | sqlite 缓存在驱动不可用时没有降级:llm/stream 每回合抛 Cannot read properties of undefined (reading 'prepare')(DSH/Electron 打包环境) #5046/B:用一个本地变量 crossLaneSafe 判定——跨 lane 共享 = 支持拓扑 → 照开;同 lane 或对方 lane 未知(inline 车道从不注册 / 旧 marker 无 lane)= 风险双写形态 → 抛 StorageSecondWriterError → 降级。
  3. wire 上的 benign 字段恒 true([PR] Second process opening index.db falsely reports "will corrupt data" — change to a shared-mode warning (#5082) | 第二个进程打开 index.db 误报「会损坏数据」——改为共享形态的提示(#5082) #5091 口径),与「要不要降级」这个本地判断完全分离。

为什么不是「任何外来持有者都降级」也不是「完全按 #5091 不降级」:两边的测试契约把答案钉死了——#5046/B 的测试明确断言「跨 lane 仍要能开」(supported topology)+「默认 report 只报告不降级」;#5091 的测试断言「report 模式一律 info + benign:true」。调和后两套全绿。我改了两处 #5046/B 测试里还引用 pre-#5091 旧措辞的断言(informational only / keeps running… → #5091 的 supported topology),语义断言(开/关、锁归属、降级 wire 态)原样保留。

⚠️ 唯一要你点头的判断点:降级门槛用「cross-lane-safe」(跨 lane 放行、同 lane/未知降级)。这是本分支原实现 + 其测试契约的方向,也与你 B 拍板里「存活外来持有者⇒降级」的意图兼容(canonical 双 DSH 安装就是同 lane)。若你更倾向「任何外来持有者都降级」或「彻底让 #5091 覆盖、撤掉 degrade」,说一声我一行翻转——PR 未合并,改动面很小。

行为变更披露

验证

typecheck 干净(kernel + pi-subagents 构建通过);定向 4 个测试文件 36 pass / 0 fail / 0 skip,覆盖 issue5046-second-writer-degrade(全 6 条双向)、issue-split-brain-surface D2、storage-durability #5082、instance-governance。全量回归以 CI 新 head 为准。

结构化八问

  1. 是什么问题:上游 PR 与本分支在 store.ts 第二进程检测段产生 merge 冲突——本质是「外来 pid 打开 index.db 该怎么处置」这条逻辑被 [PR] Second process opening index.db falsely reports "will corrupt data" — change to a shared-mode warning (#5082) | 第二个进程打开 index.db 误报「会损坏数据」——改为共享形态的提示(#5082) #5091 和本分支的 B 各自独立改了方向。
  2. 什么原因导致的:本分支切自不含 [PR] Second process opening index.db falsely reports "will corrupt data" — change to a shared-mode warning (#5082) | 第二个进程打开 index.db 误报「会损坏数据」——改为共享形态的提示(#5082) #5091 的旧 base,两边在同一代码段做了语义不同的修改([PR] Second process opening index.db falsely reports "will corrupt data" — change to a shared-mode warning (#5082) | 第二个进程打开 index.db 误报「会损坏数据」——改为共享形态的提示(#5082) #5091 放宽为纯 info;B 新增 degrade 拒绝路径),git 无法自动合并该段。
  3. 影响范围与严重程度:仅 unified storage 的第二写者处理路径;单实例用户不受影响(无外来持有者时新分支永不触发)。属合并期内的语义协调,非线上缺陷。
  4. 怎么修复的:报告与降级两职责解耦——report 走 [PR] Second process opening index.db falsely reports "will corrupt data" — change to a shared-mode warning (#5082) | 第二个进程打开 index.db 误报「会损坏数据」——改为共享形态的提示(#5082) #5091 的 info/benign:true,degrade 用本地 cross-lane-safe 门槛保留 B 的拒绝+降级;wire benign 字段与降级判断分离。
  5. 是否有回归问题:逐项过——单实例快路径不变;stale-lock 接管不变;跨 lane 共享拓扑不变(仍照开,[PR] Second process opening index.db falsely reports "will corrupt data" — change to a shared-mode warning (#5082) | 第二个进程打开 index.db 误报「会损坏数据」——改为共享形态的提示(#5082) #5091 支持拓扑语义保留);report 措辞变为 [PR] Second process opening index.db falsely reports "will corrupt data" — change to a shared-mode warning (#5082) | 第二个进程打开 index.db 误报「会损坏数据」——改为共享形态的提示(#5082) #5091 口径(有意变更,见披露);split-brain 同会话告警路径未触碰;无 fail-fast 变吞错、无阈值/时序漂移。
  6. 是否破坏缓存:否。不碰 prefix cache / 重试 / 增量游标 / 去重键;wire 仅 benign 恒 true(形状兼容)。
  7. 是否改了设计方向:不转向。「单一会话目录一个持久写者」姿态不变,只是把 [PR] Second process opening index.db falsely reports "will corrupt data" — change to a shared-mode warning (#5082) | 第二个进程打开 index.db 误报「会损坏数据」——改为共享形态的提示(#5082) #5091(文件共开安全→info)与 B(避免持续双写 BUSY→降级)这两条正交结论落到同一段代码上互不打架。
  8. 是否引入新配置:否。复用既有 BILI_SESSIONS_DIR / BILI_CONFIG_FILE / BILI_LAUNCHER_LANE,无新 key。

收工判定:冲突已解、已推,CI 会在新 head 重跑。等你确认上面那个 cross-lane-safe 判断点(默认按现状即可);CI 绿后走仓库侧合并。

ci-check added 2 commits October 29, 2026 18:38
The new tests/issue5046-second-writer-degrade.test.ts spawns one bare
'node -e' dummy child purely as a live PID holder for its lock-file
fixtures — no ports, no servers, so the zone band is irrelevant. The
entry is append-only (slot = index drives zone bands) and exists for
the SPAWN_RE tripwire, same shape as the #3890/#5114 entries. Without
it every test shard dies at pretest before running a single test.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

CI auto-fix lane (attempt 1/2): fixed directly on the branch, new head 41b8697. Details below.

What was red, exactly

All 11 test shards died in ~20s at pretest, before running a single test, with one identical signature on every shard (ubuntu 22/24 + windows 22):

process-spawning test files missing from LIVE_FILES in scripts/run-ci-tests.mjs:
tests/issue5046-second-writer-degrade.test.ts. Register them there after reviewing
their concurrency assumptions (#1248).

Two independent causes stacked:

  1. This PR's own defect. The new suite tests/issue5046-second-writer-degrade.test.ts spawns a process (one bare node -e dummy child used purely as a live-PID holder for its lock-file fixtures), which trips the SPAWN_RE tripwire in scripts/run-ci-tests.mjs. By design (tests/claude-native.test.ts linux live-E2E subtests fail/hang in shared multi-agent sandboxes #1248) every process-spawning suite must be registered in LIVE_FILES (scripts/live-files.mjs) after a concurrency review — this file was never registered, so every shard failed before any test ran. Consequence: this PR's actual test code had never been verified by CI (every prior run died at the same spot).
  2. Inherited master red. Base 1fad21d was itself red (run 40852517737, 6 shards): the hermetic preload pinned BILI_CONFIG_FILE to an absent file, shadowing fixtures that write to the XDG default config path (claude-native attach-gate/lane-priority/policy/url + native lanes suspected of not reusing instances: one proxy per process, instances only grow (v0.1.197 / opencode v2) | [Bug?] native 车道疑似不复用实例:一进程一代理,实例只增不减(v0.1.197 / opencode v2) #3436 compat-check spawn timeout), plus Windows leak-guard snapshot keys built with native path.relative() backslashes. Both mechanisms are already fixed on ci-fixes@cf99d935f (sentinel convergence, carrier PR [PR] ci-fixes: master-CI fix convergence branch (#4599) #5133) — per the CI Control Center: Centralized CI Failure Handling & Status Sync (Comment all CI issues here; do not open new issues) | CI 总控: CI 失败集中处理与状态同步(所有 CI 问题在此评论, 勿开新 issue) #4599 policy I merged origin/ci-fixes into the branch instead of re-diagnosing or duplicating.

What I did (2 commits on 2026-10-29_instance-warn-dir-criterion)

Local targeted verification (full-suite verdict = remote CI on the pushed head)

  • Tripwire scan replicated locally against the full tests/ set: PASS (no unregistered spawners, no stale entries).
  • Both new suites: 24/24 pass (tests/issue5046-second-writer-degrade.test.ts 6/6 + tests/instance-governance.test.ts 18/18) — including the positive degrade case, cross-lane negative, stale-lock takeover, clean fast path, mixed-version fleet, and report-only default-policy rows.
  • Existing UnifiedShadow coverage over the changed paths: 21/21 pass (issue3919-shadow-drift-forensics, storage-ingest-watermark, issue3407-shadow-blockcontents-order).
  • test-suite-lock.test.ts (touched by the ci-fixes merge): 31/31 pass.

Coordinated in #4599 (one-line status posted). Awaiting the CI re-run on 41b8697.


结构化八问:

  1. 是什么问题:PR 的 11 个测试 shard 全部在 pretest 阶段被哨兵杀掉,一个测试都没跑到。
  2. 什么原因导致的:两层叠加——①PR 自己新增的进程派生测试文件 tests/issue5046-second-writer-degrade.test.ts 未按 tests/claude-native.test.ts linux live-E2E subtests fail/hang in shared multi-agent sandboxes #1248 流程登记进 LIVE_FILES,触发 SPAWN_RE 哨兵(按设计 fail-fast);②base 继承了 master 自身的红(BILI_CONFIG_FILE 预载钉死遮蔽 fixture 配置 + Windows 泄漏守卫路径分隔符),该红已在 ci-fixes 修复。
  3. 影响范围与严重程度:仅 CI——此前该 PR 的全部测试从未真正被执行过(每轮都死在同一哨兵处);我的改动本身只加一行调度表数据,无运行时行为面。
  4. 怎么修复的:在 live-files.mjs 末尾追加登记项(slot=index 规则,append-only,不重排任何既有 band),并发假设审查结论写进条目注释;同时 merge ci-fixes 把继承性红的修复带进本分支,不重复诊断、不重复开修。
  5. 是否有回归风险:审过既有行为——append-only 登记不改变任何既有文件的 zone band(slot 序号不变);ci-fixes 合入只动测试基建(preload-test-env / e2e native 套件 / test-suite-lock),不碰 src/ 与 wire;本地定向跑了新套件 + 既有 shadow 路径覆盖 + 合入触及的 test-suite-lock,全绿。
  6. 是否破坏缓存:不涉及(纯测试基建)。
  7. 是否改了设计方向:不涉及。
  8. 是否引入了新的配置:不涉及。

判定:修复已推送(新 head 41b8697),等 CI 重跑;若转绿即可正常 review/merge。

@ranxianglei

Copy link
Copy Markdown
Owner Author

(ai: sglang-gpu0/qwen3.8-27b)

① 已同步最新 origin/master(09a5b5b7):全仓唯一文本冲突是 scripts/live-files.mjs 的测试注册列表——两侧各自追加了条目,按并集解决,一行未删。最终都含的新注册项:本 PR 侧 "tests/issue5046-second-writer-degrade.test.ts";master 侧 "tests/event-loop-watchdog.test.ts"(#4762)、"tests/issue5012-stdout-leg-mute.test.ts"(#5012)、"tests/issue5152-owner-beat-survives-host-stall.test.ts"(#5152)。合并后核对零重复条目(数组共 57 条)。merge commit 234e382a。

② 顺手对齐了 shadow 降级日志措辞(src/storage/shadow.ts 外层 catch 的 error 行,单独一笔提交 9d8b15e3 便于回退):原句 "two processes on one index.db corrupt it" 暗示「两进程共享必然损坏」,与 #5091 后同文件相邻位置的「跨 lane 共享是受支持拓扑」现行表述矛盾。改为准确的风险描述:跨 lane 共享仍是受支持拓扑(#5082/#5091),真实风险是同一 sessions dir 被两个进程双写造成 split-brain(#3923/#3959),本实例拒绝共写、降 legacy-only 属防御纵深。既有要素全部保留:双方 pid、db 路径、损失面(CCR/checkpoints/unified fast-read OFF)、修复配方(set BILI_SESSIONS_DIR,配置不同则加 BILI_CONFIG_FILE,然后 restart)、(#5046)/(#5068) 引用号。

③ 本地验证:tsc --noEmit 零输出干净;node --import tsx --test tests/issue5046-second-writer-degrade.test.ts tests/instance-governance.test.ts ⇒ tests 24 / pass 24 / fail 0 / skipped 0。全量回归以推送后 CI 为准。

④ 两件留给 owner 的事:
a) PR body 尾缀是 Closes #5046——但审计发现 #5046 的 prepare 症状证据已在 thread 中撤回(根因归 dsh-tools pnpm hoisting),合并会自动关闭该 issue,请确认关闭口径(若不想自动关,现在直接把 body 尾缀改掉即可);
b) LHN 指出的 shadow 自禁用仅内存态(每次重启都会重跑一遍 BUSY 循环再禁用)不在本次范围,建议另开 issue 跟踪 first-BUSY degrade / persistent marker 方向。

@ranxianglei

Copy link
Copy Markdown
Owner Author

(ai: vllm-qwen/qwen3.8-27b)

Merge-guard 冲突核查结论:无需再推。 分支 head 9d8b15e3 已包含最新 origin/master(09a5b5b7,merge-base 祖先核验 + 本地试合并返回 "Already up to date")——冲突已由先前同步提交 234e382a(live-files.mjs 注册表并集解冲突)解决。GitHub API 现报 mergeable=true / mergeable_state=blocked(blocked = 等待 required checks,非冲突);CI 镜像缓存 @9d8b15e3:0 failed / 10 pending。无设计决策点,未开 decision-request。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

CI auto-fix lane (attempt 2/2): no branch-side fix exists or is needed — this round's red is the inherited #5144 flake. Rerun of the 3 failed jobs triggered; details below.

What was red

Run 40912373654 on head 9d8b15e3 (owner's log-alignment commit + fresh master merge on top of my 41b86978): exactly 3 shards, one failure each, byte-identical signature:

not ok … HTTP a cold-restored non-retained parent serves its intact raw tail immediately (#5140)
location: tests/plugin-fork.test.ts:1473
error: 409 !== 200   (ERR_ASSERTION)
  • test (ubuntu-latest, 22, 3) · job 123598812448
  • test (windows-latest, 22, 4) · job 123598812325
  • test (ubuntu-latest, 24, 3) · job 123598812177

All other checks in the run passed; the tripwire/LIVE_FILES registration from attempt 1 held (tests actually ran this time).

Why it's inherited, not ours (independent verification, not just citing the thread)

  1. The failing test is master's own new code. tests/plugin-fork.test.ts:1461 ([Bug][fork] Cold restore cannot use an intact unified raw tail for a non-retained parent (#4266 item 4) #5140, landed via [PR] fix: recover non-retained parent fork snapshot from intact unified raw tail after cold restart (#5140) #5144 "cold raw-tail fork" merged into master 10-29) asserts that after a cold restore from the unified index, GET /__bili/plugin/snapshot?conversationId=parent answers 200 immediately. Under CI load the endpoint intermittently answers 409 — a timing-sensitive cold-restore path. This PR's diff (instance-governance warning criterion, second-writer degrade in UnifiedShadow.maybeOpen, lock handling, status wire, +1 line log wording) does not touch the plugin snapshot/fork/cold-restore paths at all.
  2. Master itself is red on the same subtest. Per CI Control Center: Centralized CI Failure Handling & Status Sync (Comment all CI issues here; do not open new issues) | CI 总控: CI 失败集中处理与状态同步(所有 CI 问题在此评论, 勿开新 issue) #4599 floor 486: master HEAD 09a5b5b7 carried 8 failing checks with this exact subtest, and earlier 083594fa carried 3 — all appearing only after [PR] fix: recover non-retained parent fork snapshot from intact unified raw tail after cold restart (#5140) #5144 merged. A flake that reproduces on master without our PR cannot be caused by this PR.
  3. Local A/B on OUR exact head: tests/plugin-fork.test.ts → 54/54 pass, three consecutive runs (plus one initial), no failures. The same-sha rerun evidence on master (floor 486: multiple all-green runs of 09a5b5b7) confirms intermittency.
  4. Fleet precedent is unanimous (CI Control Center: Centralized CI Failure Handling & Status Sync (Comment all CI issues here; do not open new issues) | CI 总控: CI 失败集中处理与状态同步(所有 CI 问题在此评论, 勿开新 issue) #4599 floors 485/486/491: PRs [PR] fix(#5142): account expected host rebases out of fold-drift #5150/[Bug] Drifted open-tag names (acpt/acct/accent…) and mid-text draft envelopes reach the client verbatim — two thinking-leak shapes #5175/[PR] fix(#5175): strip drifted open-tag names and mid-turn draft envelopes #5178/[PR] fix: bound ordinary session loads and narrow lazy field reads #4750 hit the identical signature and were cleared as the inherited [PR] fix: recover non-retained parent fork snapshot from intact unified raw tail after cold restart (#5140) #5144 flake).

State & actions

结构化八问

  1. 是什么问题:PR 的 3 个测试分片红,每片各挂同一条断言(409 !== 200),其余全部通过。
  2. 什么原因导致的:继承性阵发 flake——master 经 [PR] fix: recover non-retained parent fork snapshot from intact unified raw tail after cold restart (#5140) #5144 新合入的测试 [Bug][fork] Cold restore cannot use an intact unified raw tail for a non-retained parent (#4266 item 4) #5140(冷恢复父会话后 snapshot 端点应立即 200)在 CI 并发负载下偶发返回 409;master 自身在同一 subtest 上就有 8 个失败 check,与本 PR 代码无关。
  3. 影响范围与严重程度:仅 CI 验证面;本 PR 功能本身已被上一轮全矩阵绿 + 本轮本地 A/B(3×54/54)反复验证,无运行时行为缺陷。
  4. 怎么修复的:分支侧无可修点(ci-fixes 已全含、哨兵尚未把该 flake 收敛进 ci-fixes);按在档先例直接重跑 3 个失败 job(run 40912373654),并在 CI Control Center: Centralized CI Failure Handling & Status Sync (Comment all CI issues here; do not open new issues) | CI 总控: CI 失败集中处理与状态同步(所有 CI 问题在此评论, 勿开新 issue) #4599 登记一行;若重跑再现同签名,归 [PR] fix: recover non-retained parent fork snapshot from intact unified raw tail after cold restart (#5140) #5144 侧/ci-fixes 维护方处置。
  5. 是否有回归问题:本轮零代码改动,不引入任何回归;既有行为核对:本 PR diff 不触及 plugin snapshot/fork/冷恢复路径,与失败点无因果交集。
  6. 是否破坏缓存:不涉及。
  7. 是否改了设计方向:不涉及。
  8. 是否引入了新的配置:不涉及。

判定:无分支侧动作可再推,等 rerun 结果——转绿即可正常 review/merge;若同签名再现,属 #5144 侧待收敛项(#4599 已登记)。

@LHN-xiao-hai-tun

Copy link
Copy Markdown

核实了你这条评论的四项依据,逐条对账。结论仍成立,但两处依据已过期,且 rerun 结果永远等不到了。

一、逐条对账

你的说法 核验 出处
3 个分片同签名红(409 !== 200) ✅ 属实 job 123598812448 / 123598812325 / 123598812177,tests/plugin-fork.test.ts:1473
本 PR diff 不触及 plugin snapshot/fork/冷恢复 ✅ 属实 10 个改动文件,plugin-fork / snapshot / cold 零命中
该 flake 在 master 上就红(#4599 floor 486) ✅ 属实 master 09a5b5b7 携带同 subtest 失败
已触发 rerun(HTTP 201) ✅ 属实 但结果不是预期的

二、rerun 已被取消,信号为零

run 40912373654  attempt=2  conclusion=cancelled
  01:13:17Z 开始  →  01:15:17Z 取消   (只跑了 2 分钟)
  三个失败 job  conclusion=cancelled   ← 不是 success,也不是 failure
  job 日志  BlobNotFound(已被清理)

triggering_actor: ranxianglei,但 attempt 2 只存活 120 秒就被 cancel。

我先怀疑是 #5187 的 ci-janitor 干的,查完发现时间对不上,收回这条推测:

#5187 merged_at  = 2026-10-30T05:36:53Z
attempt2 取消    = 2026-10-30T01:15:17Z   ← janitor 晚 4h21m,不可能

取消者我没有查到确证,不猜。客观后果是确定的:「等 rerun 结果」这条路已经走不通了,没有结果会在路上到来。

三、master 已经加强了这条 flake 的诊断,PR 应把它带进来

master 在 10-30T03:12:58Z(这条评论发出前 1.5 小时)已有提交:

b5385a7a  test(plugin-fork): capture the 409 body so the #5140 cold-tail
          flake names its refusal branch

diff:

- assert.equal((await h.request("/__bili/plugin/snapshot?conversationId=parent")).status, 200);
+ const coldSnap = await h.request("/__bili/plugin/snapshot?conversationId=parent");
+ assert.equal(coldSnap.status, 200, JSON.stringify(coldSnap.body));

位置核验:

这直接否掉你的第 4 条推论「降级为等 rerun」。带着 b5385a7 再跑,flake 重现时日志会直接写出是哪条 forkSnapshot 拒绝分支,这比现在「裸状态码 409、日志已被清」的可诊断性高一个量级,也是 #4599 sentinel 收 409 body 的原意。

四、关于「本地 3×54/54 全绿」

这条不能作为「不是本 PR 引起」的证据:CI 并发负载下的时序 flake,本地全绿是预期行为而非反证。真正有力的仍是你的第 2 条(master 自身红)与第 4 条(fleet 同签名先例),这两条我核过,成立。

五、请求的动作

  1. 把 origin/master(dcf76075)merge 进本分支 —— 一举两得:解掉 behind 状态,并带上 b5385a7a 的 409 body 诊断。
  2. merge 后重跑 CI。
  3. 若同签名再现:日志里会有 forkSnapshot 的具体拒绝分支,据此判归 [PR] fix: recover non-retained parent fork snapshot from intact unified raw tail after cold restart (#5140) #5144 侧还是别处,而不是继续以「继承性 flake」结案。
  4. 若再现且 body 指向本 PR 之外:我会撤回「继承」判断并协助定位。

备注:b5385a7a 只加诊断不改行为,merge 它不会影响本 PR 的语义。

@ranxianglei

Copy link
Copy Markdown
Owner Author

(ai: vllm-qwen/qwen3.8-27b)

Behind 已同步并推送。 origin/master(44067a5b)merge 进本分支,零冲突;本地 typecheck 干净。新 head 1e26f3879;GitHub 现报 mergeable=true / blocked(blocked = 等待新 head 的 required checks,非冲突)。全量绿红以该 head 的远程 CI 为准。

@ranxianglei

Copy link
Copy Markdown
Owner Author

[bot] 🏷
(ai: vllm-qwen/qwen3.8-27b)

状态确认(无新动作):behind 已在上条解决——origin/master 44067a5b 零冲突合入,typecheck 干净,head 1e26f3879;GitHub mergeable=true / blocked(仅等新 head 的 required checks)。线程无新的用户输入需要处理。

@ranxianglei

Copy link
Copy Markdown
Owner Author

已并入 #5213(随 D 类整合,人工合并入 master)。实例判据按 sessions 目录的修复已在 master 生效。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants