You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Storage banner falsely warns «second process writing index.db» when same-lane bili instances share the machine-global sessions dir | 同车道多实例共享存储误报损坏告警 #5014
同一台机器上同时运行多个同车道 bili 实例(日常形态:每个终端一个 bili pi -r,各自 spawn 代理),它们共享机器级 sessions 目录(~/.local/share/billion-context/sessions/,设计如此——AGENTS.md「machine-global facilities stay SHARED」)。任何一个实例在另一个存活实例之后打开统一存储时,Web 面板弹出红色告警:
Same lane ⇒ peer.lane !== ownLane is false ⇒ the "two processes on one index.db corrupt it" branch, exactly as the root cause in this issue predicts. So the trigger is not multiple terminals spawning proxies; a single host instance appears to be enough to produce the collision (possibly the +1 port ladder discussed in #4568).
It is not only a banner
While both were up, the store on 18795 reported a real degradation, not just noise:
storage.requested: full
storage.effective: legacy
storage.reason: index database failed to open (Error: database is locked)
It recovered to effective: full later, so this is transient lock contention rather than corruption — which supports the issue's premise that the same-lane case is benign at the database level. But in 0.1.209 the contention still lands the serving instance in legacy mode for a while, so the cost of misclassifying it is not cosmetic. (Related: #5324 covers the silent-degrade path; master reportedly refuses to start instead — this lane is on 0.1.209.)
/__bili/status also flagged a split-brain claim on one session id from both instances, i.e. the second, independent channel this issue asks to keep unchanged.
Question
Is one host spawning two instances on consecutive ports (18795 + 18796) the intended topology, or a launch-ladder leak? If intended, the same-lane benign fix here matters more than the "multiple terminals" framing suggests, because every DSH user would hit it by default.
Data point from the dsh lane (Windows, 0.1.209, 2026-11-01 18:49 local UTC+8) — the banner may be benign, but same-lane sharing has a real consequence once the other instance is an unreachable orphan:
18:49:48 [storage] unified sidecar failed to open …\sessions\index.db (Error: database is locked)
— staying legacy-only; delete the file to reset the index
18:49:51 port 18789: registry twin 0f60d531 (pid 20724) no longer answers as itself — the socket moved on
while it drained; treating the port as foreign-held instead of dying twin-collision (#4982)
18:49:52 [instances] another bili instance is running (pid 20724, http://127.0.0.1:18789) on lane "dsh" …
18:52:10 [storage] second-process pid 20724 has exited — stale warning cleared ← only after I killed it
Two things this adds to the "false positive" framing:
legacy-only is not just a banner — the replacement latched off the unified store for its entire life, so shadow verification and cross-process dedupe were gone for the whole session (which then also produced a unified shadow DIGEST DRIFT error once it re-opened).
[instances] another bili instance is running (pid N) is reported for a peer that is provably unreachable and whose owner is dead. Knowing it is a known instance is not enough information here: the benign/malicious split needs a third state — known but wedged.
Suggestion: when the peer pid maps to a known instance but its health has been unreachable for N seconds and its owner pid is not alive, say that explicitly (peer <pid> is wedged — takeover recommended) instead of folding it into either "benign" or "corruption", and don't fall back to legacy-only silently. Background and the reaping failure that produced the orphan: #5449.
Both points verified against master — thanks for the clean log capture.
1. legacy-only latch: confirmed permanent per process. The sidecar opens exactly once at store construction (src/persist.ts:498-500, one-shot UnifiedShadow.maybeOpen); the open-failure catch (src/storage/shadow.ts:237-254) logs the warn you saw, records the failure, and returns null — no retry, ever. The periodic recheck that printed your 18:52:10 line (src/storage/runtime-status.ts:272) only clears the warning when the peer pid exits; it never reopens the store. So yes: after you killed pid 20724, that instance stayed legacy-only until manual restart, even though the lock contention had already cleared. Your later DIGEST DRIFT on reopen is the known multi-instance consequence channel (shadow.ts:423 names a shared sessions dir as the usual cause).
2. Third state "known but wedged": agreed in principle — with one important interaction. Today (your 0.1.209) that orphan reads as the red corruption alarm; once PR #5015 merges, the same scenario flips to benign info ("informational only") — i.e. the fix removes the over-alarm but would leave misleading "safe to ignore" advice for exactly this case, where the actionable fact is kill/takeover. Two feasibility notes from the code:
the instance registry does not record an owner/parent pid (RegistryEntry, src/instance.ts:473-499), so "owner pid is not alive" as stated is not evaluable today without a new field;
buildable signals already exist: health probing via the entry's origin (healthOf(origin), instance.ts:273 — already used by lane attach) plus the portReleasedAt marker ([PR] release v0.1.209 #4982), which is set exactly when a twin releases its socket while still draining — the machine-readable form of your "the socket moved on while it drained" line. A wedge verdict of "registry-known + health-unreachable-for-Ns (+ portReleasedAt)" needs no new data.
Filed as issue #5452 (labels: bug / dsh / scope/storage / p2 — silent capability loss with a workaround, not data loss): checklist items D1 missing third state, D2 one-shot sidecar open with no recovery path, D3 dead-pid-only lock takeover (deliberately NOT "unreachable ⇒ takeover" — a slow-but-live writer is still the legitimate shared-dir topology; grabbing its lock is real corruption). Options A/B/C are listed there for owner decision.
One correction: #5449 does not resolve to the reaping-failure background you cite — in this repo that number is an unrelated release-workflow PR (emergency publish channel restore). Orphan production (host dies mid-drain, child keeps running while holding locks) is a distinct upstream dependency of this whole scenario; if you have the right tracking number, drop it into #5452 so the chain closes. Until then the issue carries it as an open dependency.
Workaround meanwhile: kill the orphan (as you did) and restart the replacement — the replacement will not pick up the unified store by itself.
Activity
ranxianglei commentedon Oct 28, 2026
(ai: google/gemini-2.5-pro)
修复已提:PR #5015(Closes #5014)。主题类别:E 存储层。
ranxianglei commentedon Oct 28, 2026
(ai: vllm-qwen/qwen3.8-27b)
收工摘要(完整验证结论见上一条回复):
issue-split-brain-surface5/5、storage-durability5/5、web-storage-banner5/5)+ typecheck 干净。fix(storage): a registry hit whose lock boot stamp predates the live …
aweds13 commentedon Oct 31, 2026
Same-lane collision here is not "one terminal each" — one host process spawned both
Confirming the
benigncondition described above bites even when there is exactly one user-visible host, and adding the consequence I could measure.Environment: bili 0.1.209 (39a0ada), Windows, single DSH (DeepSeek Harness) process.
Both children of the same parent pid 29300; no second terminal, no
bili pi -r, no build change. On 18795,/__bili/statusreports:Same lane ⇒
peer.lane !== ownLaneis false ⇒ the "two processes on one index.db corrupt it" branch, exactly as the root cause in this issue predicts. So the trigger is not multiple terminals spawning proxies; a single host instance appears to be enough to produce the collision (possibly the +1 port ladder discussed in #4568).It is not only a banner
While both were up, the store on 18795 reported a real degradation, not just noise:
It recovered to
effective: fulllater, so this is transient lock contention rather than corruption — which supports the issue's premise that the same-lane case is benign at the database level. But in 0.1.209 the contention still lands the serving instance in legacy mode for a while, so the cost of misclassifying it is not cosmetic. (Related: #5324 covers the silent-degrade path; master reportedly refuses to start instead — this lane is on 0.1.209.)/__bili/statusalso flagged a split-brain claim on one session id from both instances, i.e. the second, independent channel this issue asks to keep unchanged.Question
Is one host spawning two instances on consecutive ports (18795 + 18796) the intended topology, or a launch-ladder leak? If intended, the same-lane benign fix here matters more than the "multiple terminals" framing suggests, because every DSH user would hit it by default.
ranxianglei commentedon Oct 31, 2026
(ai: vllm-qwen/qwen3.8-27b)
感谢这份证据——单宿主双实例的形态和实测降级都很关键。逐点回复:
1. 你的 status JSON 正是本 issue 诊断的代码路径。同车道 peer(
lane: "dsh")在旧代码里peer.lane !== ownLane为假 →benign: false→ 红色损坏告警。PR #5015 修掉后,你这个场景(两个 pid 都在实例注册表里、都是 bili 子进程)会判为 benign → info 日志 + 信息样式横幅,不再报「corrupt it」。**2. 回答你的问题:不是预期拓扑,是已知缺陷,且已有修复在途。**分两层说:
37f3229c,CI 重跑中);配套的 Part 1(宿主缓存 hop 指向死端口导致 turn 全死)PR [PR] fix: drop stale host proxy hops from intercepted dispatches (#4566 part 1) #4878 CI 已全绿。这两个 PR 合入后,你看到的单宿主双实例状态本身就不会再产生。3. 你实测的降级是真实的,且已有记录:
storage.effective: legacy+database is locked这个签名在 #4566 的证据评论里一字不差地出现过(第二实例共目录的直接数据面后果)。两点澄清:src/storage/shadow.ts:251),不是拒启——和你观察到的「后来自己恢复」一致。4. split-brain 标记:独立通道按设计工作(#3952)——两个实例确实提交了同一个 session id,这正是本 issue 明确保留不动的那条告警。#5064 落地后双实例状态消失,这条也不会再触发。
版本提示:你在 0.1.209 上;#5015 / #5064 / #4878 都会随后续 release 发布。在那之前,你这套 Windows 单宿主出现双实例属于上述已知缺陷,不是配置问题。
ssamurias commentedon Nov 1, 2026
Data point from the dsh lane (Windows, 0.1.209, 2026-11-01 18:49 local UTC+8) — the banner may be benign, but same-lane sharing has a real consequence once the other instance is an unreachable orphan:
Two things this adds to the "false positive" framing:
legacy-onlyis not just a banner — the replacement latched off the unified store for its entire life, so shadow verification and cross-process dedupe were gone for the whole session (which then also produced aunified shadow DIGEST DRIFTerror once it re-opened).[instances] another bili instance is running (pid N)is reported for a peer that is provably unreachable and whose owner is dead. Knowing it is a known instance is not enough information here: the benign/malicious split needs a third state — known but wedged.Suggestion: when the peer pid maps to a known instance but its health has been unreachable for N seconds and its owner pid is not alive, say that explicitly (
peer <pid> is wedged — takeover recommended) instead of folding it into either "benign" or "corruption", and don't fall back tolegacy-onlysilently. Background and the reaping failure that produced the orphan: #5449.ranxianglei commentedon Nov 1, 2026
(ai: vllm-qwen/qwen3.8-27b)
Both points verified against master — thanks for the clean log capture.
1.
legacy-onlylatch: confirmed permanent per process. The sidecar opens exactly once at store construction (src/persist.ts:498-500, one-shotUnifiedShadow.maybeOpen); the open-failure catch (src/storage/shadow.ts:237-254) logs the warn you saw, records the failure, and returns null — no retry, ever. The periodic recheck that printed your 18:52:10 line (src/storage/runtime-status.ts:272) only clears the warning when the peer pid exits; it never reopens the store. So yes: after you killed pid 20724, that instance stayed legacy-only until manual restart, even though the lock contention had already cleared. Your laterDIGEST DRIFTon reopen is the known multi-instance consequence channel (shadow.ts:423names a shared sessions dir as the usual cause).2. Third state "known but wedged": agreed in principle — with one important interaction. Today (your 0.1.209) that orphan reads as the red corruption alarm; once PR #5015 merges, the same scenario flips to benign info ("informational only") — i.e. the fix removes the over-alarm but would leave misleading "safe to ignore" advice for exactly this case, where the actionable fact is kill/takeover. Two feasibility notes from the code:
RegistryEntry,src/instance.ts:473-499), so "owner pid is not alive" as stated is not evaluable today without a new field;healthOf(origin),instance.ts:273— already used by lane attach) plus theportReleasedAtmarker ([PR] release v0.1.209 #4982), which is set exactly when a twin releases its socket while still draining — the machine-readable form of your "the socket moved on while it drained" line. A wedge verdict of "registry-known + health-unreachable-for-Ns (+ portReleasedAt)" needs no new data.Filed as issue #5452 (labels: bug / dsh / scope/storage / p2 — silent capability loss with a workaround, not data loss): checklist items D1 missing third state, D2 one-shot sidecar open with no recovery path, D3 dead-pid-only lock takeover (deliberately NOT "unreachable ⇒ takeover" — a slow-but-live writer is still the legitimate shared-dir topology; grabbing its lock is real corruption). Options A/B/C are listed there for owner decision.
One correction: #5449 does not resolve to the reaping-failure background you cite — in this repo that number is an unrelated release-workflow PR (emergency publish channel restore). Orphan production (host dies mid-drain, child keeps running while holding locks) is a distinct upstream dependency of this whole scenario; if you have the right tracking number, drop it into #5452 so the chain closes. Until then the issue carries it as an open dependency.
Workaround meanwhile: kill the orphan (as you did) and restart the replacement — the replacement will not pick up the unified store by itself.