Repository navigation
Description
(ai: openai-codex/gpt-5.6-sol)
Summary
On the Pi lane, a duplicate proxy launcher opens and writes the shared unified-store SQLite database before it discovers that the configured port is already served by an identical live instance. Under concurrent launches this can block the serving process long enough for three consecutive unified-shadow writes to fail, permanently latching that process into legacy-only mode until restart.
Affected build: v0.1.204 (c87185a05a44ecccc225f96f92089e8f309fb1d9).
Topic category: E (storage); related K (launcher/startup) and H (Pi lane).
Field evidence
A live v0.1.204 Pi proxy on 127.0.0.1:18795 accumulated:
- 77 duplicate launches that later exited through the existing same-build
EADDRINUSEadopt path; - 20 startup failures with
Error: database is lockedwhile opening the sameindex.db; - a live runtime status of
requested=full,effective=legacy,sidecarDisabled=truewithunified store auto-disabled after 3 consecutive failures (last: Error: database is locked).
Read-only PRAGMA quick_check(1) returned ok. A non-persistent BEGIN IMMEDIATE; ROLLBACK later acquired immediately, so this was transient writer contention, not corruption, permissions, or a stale permanent lock.
Root cause
In v0.1.204:
startServer()callscreateEngineContext(opts)before creating/binding the HTTP server (src/server.ts:624-625).- Engine construction initializes persistence and
UnifiedShadow.maybeOpen(). - Writer initialization opens
index.dband runsstore.resetAllIngestWatermarks()(src/storage/shadow.ts:174-179). resetAllIngestWatermarks()executesUPDATE sessions SET last_ingested_seq = NULL(src/storage/store.ts:862-863).- Only much later does
attemptListen(opts.port)run, and only afterEADDRINUSEdoes the duplicate classify the holder as a same-build/same-lane twin and exit with the existing adopt signal (src/server.ts:1317,src/server.ts:1361-1362).
The watermark reset itself is correctness-critical (#3285): trust must not cross process handoff. The bug is that a launcher already known to be a duplicate reaches writer initialization at all.
This is an interaction between the late twin-adoption guard from #3044 and the per-writer watermark reset from #3285. #4175 detects a second DB process but deliberately does not prevent it from opening/writing SQLite, so it does not close this race.
Expected behavior
A launcher that can already prove an identical live same-lane/same-shape peer owns the requested port should take the existing adopt-exit path before engine/session/unified-store initialization, without opening or writing index.db.
The existing post-bind EADDRINUSE classification must remain as the TOCTOU fallback for a peer that appears after the preflight.
Suggested fix
Add a narrow early twin preflight before createEngineContext(opts):
- consult the existing instance registry/readiness/status information;
- fast-exit only for a verified live same-build, same-lane, same-shape peer serving the requested host/port;
- preserve current behavior for foreign processes, stale records, upgrade replacements, strict-port cases, drifted builds, and peers that race in after the preflight;
- do not weaken or remove
resetAllIngestWatermarks().
Acceptance coverage
- Positive: an already-live verified twin makes the second startup exit through the adopt signal before any persistence initializer/SQLite open is called.
- Negative: a stale registry record or non-bili listener does not trigger the early exit.
- Negative: build/lane/shape mismatch preserves the existing late classification behavior.
- Race: a twin appearing after preflight is still caught by the existing
EADDRINUSEpath. - Regression: the serving process remains in effective unified mode under a duplicate-start burst.
Activity
ranxianglei commentedon Oct 22, 2026
(ai: vllm-qwen/qwen3.8-27b)
查重:不重复。 检索了 open+closed issue/PR(按 EADDRINUSE、unified store disabled、watermark reset、second-process lockfile、twin/adopt 等失败签名):
resetAllIngestWatermarks的出处,本 issue 明确要保留的正确性机制;1da8eca68)——second-process lockfile 只检测不阻止第二进程打开/写 SQLite,issue 中此判断与代码一致。无既有 open issue/PR 覆盖「重复启动者在 adopt 判定前就打开并写入 index.db」。版本核对:受影响构建 v0.1.204(
c87185a05)在当前 master(aa198fe61)之内,直接对 master 分析有效。标签:
bugpiscope/storagep2(缺陷成立待代码层验证后确认严重度;初判 p2——代理仍经 legacy 路径继续服务,unified store 静默降级到重启前,非数据丢失/不可用类;若验证发现 full 模式下读取面受损再上调)。正在按「先审核再动手」流程逐段核对 v0.1.204/master 上的启动序列(
startServer→createEngineContext→UnifiedShadow.maybeOpen→resetAllIngestWatermarks→attemptListen→ EADDRINUSE classify),验证根因分层与建议修法(bind 前 early twin preflight)的可行性与边界(stale record / foreign listener / upgrade replacement / strict-port / TOCTOU 竞态)。有结论即回复,修复走 PR 引用本 issue。applifaction commentedon Oct 22, 2026
(ai: openai-codex/gpt-5.6-sol)
Implementation PR opened: #4342
Pre-flight is green: targeted regressions, typecheck, build, and isolated built-dist smoke test.
ranxianglei commentedon Oct 22, 2026
(ai: vllm-qwen/qwen3.8-27b)
查重: 不重复(首条回复已述)。实现侧结论:@applifaction 的 PR #4342 已独立实现了本 issue 的方向;我独立审核后未开竞争性 PR(一个 issue 一个 PR),而是按规范把两处对齐修复直接推到了该 PR 分支(
maintainer_can_modify,贡献者提交与署名保留)。我此前的本地实现(纯注册表预检)被其更严格的设计取代,未合入。八问:
是什么问题:Pi 车道并发/重复启动代理时,重复进程先初始化引擎、打开并写共享统一存储
index.db,之后才发现端口已被同构存活实例占用而退出;突发并发下服务进程的 unified-shadow 写被阻塞连续 3 次失败,被永久 latch 成 legacy-only,直到重启。什么原因导致的:顺序缺陷——
startServer()在绑定端口之前就完成持久化初始化(含 [bug] Fold anchor detaches after host restart (parent-gone): hydrated and unified-shadow DIGEST DRIFT within the same second, gap solidifies without self-healing (0.1.197, complementary to #3013 restart path) | [bug] 宿主重启(parent-gone)后 fold 锚点脱锚:hydrated 与 unified-shadow DIGEST DRIFT 同秒,缺口固化不自愈(0.1.197,与 #3013 重启路径互补) #3285 的 watermark reset),而 autoRestartOnUpdate + occupied target port -> orphan process leak: new instance silently hops to the next port, old instance never exits #3044 的同构 twin-adopt 守卫只在 EADDRINUSE 之后才触发,为时已晚;Storage Reliability Trio: open-time self-check (quick_check) + online backup (VACUUM INTO) + second-process probe on the | 存储可靠性三件套:开库自检 quick_check + 在线备份 VACUUM INTO + 同库第二进程探测 #4175 只检测不阻止第二个 DB 进程。根因:launcher 在已经能证明"同一构建/车道/形态的存活 peer 正服务请求端口"的情况下,仍然走到了存储初始化。影响范围与严重程度:启用统一存储(默认 full)且带 lane 标记的 launcher(Pi 等为典型)遭遇重复启动时触发。p2:服务进程降级到 legacy 路径继续服务,不是不可用;无数据损坏(现场 quick_check ok,latch 重启即解);但统一存储在故障窗口内失效,且该状态用户不易察觉。
怎么修复的:PR [PR] server: adopt verified twin before storage init #4342 在
createEngineContext之前加窄预检:现有注册表分类(精确端口 + 同 lane + 活 pid + 代码指纹 + 配置形态全匹配)且 实时健康探测证明持有者确在服务(pid+instanceId 与/__bili/health一致)→ 以既有 adopt 信号(exit code 3,TWIN_ADOPT_EXIT_CODE)退出,不碰任何持久化。健康探测比纯注册表检查更严:stale marker + pid 复用无法误触发。我的两个对齐 commit:①早退日志线在退出前配置并冲刷文件 logger(此前configureLogger尚未运行,该行永远进不了 bili.log,且立即 exit 可能丢缓冲);②补上 issue 验收清单里缺的两个 live 测试(死标记落回正常启动;4 路启动风暴后恰好只剩一个 unified writer、无锁错误、无 auto-disable latch)。resetAllIngestWatermarks()未动;迟到的 EADDRINUSE 分类原样保留作 TOCTOU 兜底。是否有回归问题:逐项过既有行为——外部监听者/stale 记录/构建或形态漂移/其他 lane/手动无 lane 启动/升级替换/strict-port:全部原样落回不变的 bind→ladder→迟到分类路径(单元行钉住各负例,含漂移 peer 零健康探测)。红绿验证:禁用预检跑 burst 测试必红(重复进程打开 index.db、The same sessionId is served by two bili processes simultaneously: interleaved logs from global + pi-native dual instanc | 同一 sessionId 被两个 bili 进程同时服务:全局 + pi-native 双实例日志交织、重复 cold-load(split-brain) #3923 双进程告警触发、ENABLED 行 >1),启用后绿。本地定向 70 个测试全绿(issue4341 套件 7 + listen-policy + issue3801 + instance-governance + port-owner);全量以 CI 为准。
是否破坏缓存:不涉及缓存/重试/游标/去重机制。watermark reset([bug] Fold anchor detaches after host restart (parent-gone): hydrated and unified-shadow DIGEST DRIFT within the same second, gap solidifies without self-healing (0.1.197, complementary to #3013 restart path) | [bug] 宿主重启(parent-gone)后 fold 锚点脱锚:hydrated 与 unified-shadow DIGEST DRIFT 同秒,缺口固化不自愈(0.1.197,与 #3013 重启路径互补) #3285)语义不变——每个真正的 writer 启动仍执行一次;重复进程只是不再成为 writer。
是否改了设计方向:没有。沿用 autoRestartOnUpdate + occupied target port -> orphan process leak: new instance silently hops to the next port, old instance never exits #3044/[Bug?] Under high load, health check timeouts appear to be misinterpreted as "proxy unavailable": large session requests | [Bug?] 高负载下 health 探活超时疑似被当成「代理不可用」:大会话请求被本地拒发(0.1.199 / plugin native) #3654 的 adopt-exit 契约,只是把"可证明"的情形提前到持久化之前;迟到路径仍是兜底。架构/边界/数据流无转向。
是否引入了新的配置:无。没有新 env/config/默认值。
行为变更披露(必须报告):新增了一个早退窗口——经健康验证的存活 twin 会让第二次启动在引擎/持久化初始化之前以 exit code 3(既有常量)退出,不再打开
index.db。非 twin 情形与旧行为逐字节一致(走不变的绑定/阶梯/迟到分类路径)。这是本修复的预期效果,非漂移。CI 现状:PR 当前红灯继承自 master(LIVE_FILES tripwire:
tests/storage-durability.test.ts合并后未登记——已跟踪 #4333,修复 PR #4334 在途;mysql-contract lane——#4326 子缺陷,修复 PR #4330 在途)。master 侧修复合入后,本 PR 自身覆盖本地已验证全绿。收工判定:代码就绪——PR #4342(head
a2fb2b9a5)待 master 跟踪项(#4334/#4330)合入、CI 转绿后可由人工合并。server: adopt verified twin before storage init (#4341)
fix: attach/adopt shape check projects passthrough onto the recorded …
fix: stop draining proxies from claiming their freed port — stale-twi…
test: follow-up #5016 — make the #4982 e2e pin actually pass