Skip to content

[PR] fix(ws-bridge): re-arm wss lazily after a post-bind ladder re-listen - #7047

Open
ranxianglei wants to merge 1 commit into
masterfrom
2026-11-17_ws-bridge-ladder-relisten
Open

ranxianglei wants to merge 1 commit into
masterfrom
2026-11-17_ws-bridge-ladder-relisten

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Summary

Fixes #7046

The #2941 post-bind ladder closes the bound port and re-listens the SAME http.Server object. The bridge's 'close' backstop runs wss.close(), and the ws library never reopens a closed WebSocketServer — every later upgrade 503s (completeUpgrade: _state > RUNNING) for the rest of the process, permanently killing the WS compression lane (opencode/codex responses).

Fix

Re-arm lazily in the upgrade handler: track a wssDead flag set by the 'close' backstop; on the first upgrade after a re-listen, create a fresh wss before handleUpgrade. The 'listening' event can't be used because attemptListen (server.ts) strips all 'listening' listeners before each listen. The ladder path has already cut every connection (closeAllConnections), so no live session is lost.

Regression test

tests/ws-bridge.test.ts: installWebSocketBridge → listen(0) → close + closeAllConnections → re-listen(0) (mirroring attemptListen's removeAllListeners('listening')) → real loopback WS upgrade → assert echo. Fails with 503 before the fix, passes after.

@ranxianglei ranxianglei added bug Something isn't working p2 labels Nov 17, 2026
@github-actions

github-actions Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-11-17_ws-bridge-ladder-relisten (a3d80e4)

Latest build — install this exact version

npm install -g billion-context@0.0.10792-pr.7047.256b3ff

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
a3d80e4 billion-context@0.0.10792-pr.7047.256b3ff build
32b7136 billion-context@0.0.10590-pr.7047.2d42bdd build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr7047.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重: 同家族但机制不同 — 本 PR (#7047) 修 #7046(ladder re-listen 后 WS 车道永久 503,启动期换端口触发);与 #7018 同属「WS bridge 车道失效」家族、同改 installWebSocketBridge,但机制不同:#7046 是 #2941 post-bind ladder 复用同一 server 对象 re-listen 后 wss.close() 永不恢复(启动期换端口),#7018 是 shutdown 期 pending upgrade socket 泄漏(关服时序)。两者独立,不重复。

独立核验(不照抄 issue/前次结论)

拉了 PR 分支 2026-11-17_ws-bridge-ladder-relisten (32b71368dd),对 base 301e1a5459 (0.1.215) 与 head 分别做定向验证:

  • Bug 真实存在:base 上跑 PR 新增回归测试 → not ok … error: 'Unexpected server response: 503'(精确复现 issue 的 503);head 上 → 4/4 全过。
  • 机制逐层钉死:
    • src/server.ts:1539-1540(master 现行):ladder 路径 server.close(() => attemptListen(nextPort)) + closeAllConnections(),复用同一 http.Server 对象;
    • src/server.ts:1568:attemptListen 每次 listen 前 removeAllListeners("listening") → 用 'listening' 事件 re-arm 不可行(PR 取舍理由成立);
    • node_modules/ws/lib/websocket-server.js:385:if (this._state > RUNNING) return abortHandshake(socket, 503) — 已 close 的 WebSocketServer 对所有升级 503,且 ws 无 reopen API → 「重建 wss」是唯一可行 re-arm 方式(issue 里「重置内部状态」选项在 ws API 下不可实现,该取舍被库约束强制)。
  • CI 绿(head 32b71368dd 主 CI run 44045569705):27 success / 1 skipped / 0 fail,11 个 test 分片全过。
  • diff 干净:2 文件 +67/−1,每行都与本 PR 目的相关;master 自 base 以来未触碰 src/ws-bridge.ts / tests/ws-bridge.test.ts / server.ts ladder 区 → 虽落后 master 26 个提交,合并无冲突。

结构化八问

  1. 是什么问题:启动时端口被外部监听者占用、bili 走 Windows: concurrent lane proxies can double-bind the same zone port (SO_REUSEADDR semantics) — instance pointer diverges from the socket owner #2941 post-bind ladder 换端口重试后,WebSocket 压缩车道(opencode/codex responses lane)整进程永久失效,后续所有 WS 升级全部被拒。
  2. 什么原因导致的:bridge 的 server.on("close") backstop 执行 wss.close(),而 ws 库不会重开已关闭的 WebSocketServer(completeUpgrade 里 _state > RUNNING → 503);Windows: concurrent lane proxies can double-bind the same zone port (SO_REUSEADDR semantics) — instance pointer diverges from the socket owner #2941 ladder 复用同一 server 对象 re-listen,所以 wss 死后整进程所有 WS 升级都 503。因果链:ladder close → 'close' 事件 → backstop 执行 wss.close() → wss 永久死亡 → 后续升级全部 503。
  3. 影响范围与严重程度:触发条件 postBindCanLadder = hasLaunchToken && !strictPort && !upgradeReplacement(launcher 启动双开竞态)。触发后 WS 车道整进程失效,客户端退回 HTTP 通道(有 fallback 的车道)。体验降级但不致命(p2)。
  4. 怎么修复的:upgrade handler 里 lazy re-arm——wssDead flag 由 'close' backstop 置位,首次升级检测到 wssDead 时重建 wss 再 handleUpgrade。不用 'listening' 事件(被 attemptListen 剥掉);ladder 已 closeAllConnections() 切断所有连接,重建不丢活跃会话。re-arm 与 handleUpgrade 在 await 之后同步执行,无并发竞态;重复 ladder 也成立(闭包变量重读,backstop 对重建后的 wss 仍正确 terminate peers + close)。
  5. 是否有回归问题:无。正常路径 wssDead 恒 false,新增 if (wssDead) 是 no-op;shutdown 路径 backstop 行为不变(仅多置一个 flag,进程退出无后续升级)。高危模式逐项:无 fail-fast 变吞错、无阈值/时序/输出格式漂移。唯一行为变化 = 本 bug 修复本身(ladder re-listen 后车道恢复而非永久死亡),已在 PR 描述披露。
  6. 是否破坏缓存:不涉及。
  7. 是否改了设计方向:不涉及。修复在 bridge 内部完成,无架构/边界/数据流转向。
  8. 是否引入了新的配置:不涉及。

⚠️ 与 #7018 的交叉(需 owner 定合并顺序)

#7018(open,同 base 301e1a5459)给同一函数加了单向 closing latch:server.close 包装与 'close' backstop 都会置 closing = true,upgrade handler 里 if (closing) { socket.destroy(); return true; }。ladder 路径调的正是 server.close() → 若 #7018 先合入,ladder 后所有升级会被 latch 在 re-arm 之前拒掉,WS 车道照样永久死(死因从 503 变成 socket destroy)。本 PR 的 re-arm 不重置该 latch。

两者同改 installWebSocketBridge、同 base,都合并必然要协调。此交叉已在 #7046 正文与 #7018 审核评论里记录过,非新发现。建议:先合 #7047(自包含、对 master 正确、CI 绿);合 #7018 时让其 closing latch 对 ladder re-listen 感知(re-arm 时重置,或仅在真 shutdown 置位)。

判定

#7047 本身:修复正确、回归测试有牙(base 503 / head 全过,本地复现)、CI 绿、diff 干净、标签齐(bug/p2)。可合(建议先于 #7018)。

…7046)

The #2941 post-bind ladder closes the bound port and re-listens the SAME
http.Server object. The bridge's 'close' backstop runs wss.close(), and the
ws library never reopens a closed WebSocketServer — every later upgrade
503s (completeUpgrade: _state > RUNNING) for the rest of the process,
permanently killing the WS compression lane (opencode/codex responses).

Re-arm lazily in the upgrade handler: track a wssDead flag set by the
'close' backstop; on the first upgrade after a re-listen, create a fresh
wss before handleUpgrade. The 'listening' event can't be used because
attemptListen (server.ts) strips all 'listening' listeners before each
listen. The ladder path has already cut every connection
(closeAllConnections), so no live session is lost.

Regression test: installWebSocketBridge → listen(0) → close +
closeAllConnections → re-listen(0) (mirroring attemptListen's
removeAllListeners('listening')) → real loopback WS upgrade → assert echo.
Fails with 503 before the fix, passes after.
@ranxianglei
ranxianglei force-pushed the 2026-11-17_ws-bridge-ladder-relisten branch from 32b7136 to a3d80e4 Compare November 18, 2026 10:26
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Conflict resolved and pushed. Rebased the PR branch onto current master (548e396022) and force-pushed the conflict rebase (the only force-push this lane is allowed). New head: a3d80e4291. PR merge state is no longer dirty — mergeable: true (currently blocked = waiting on CI checks for the new head).

Where it conflicted: src/ws-bridge.ts only. This PR (wss re-arm after a #2941 post-bind ladder re-listen) and master commit f8f58d3ab1 (fix: close pending WebSocket upgrades during server shutdown) both rewrote the same upgrade-handler region: the PR turns const wss into let wss + a wssDead re-arm; master adds a pendingUpgrades set + a closing shutdown latch. tests/ws-bridge.test.ts merged cleanly.

Important — the pure textual union was NOT enough. After uniting both intents, the PR's own regression test failed locally (socket hang up / ECONNRESET): master's closing latch is set by the close backstop and never reset, so the #2941 ladder's close+re-listen of the same server object leaves it latched forever — every later WS upgrade hits if (closing) { socket.destroy(); return true; }. Same pathology this PR exists to kill ("WS lane permanently dead after a ladder re-listen"), just through the new latch instead of the closed wss. On top of master, the PR's fix alone would not have restored the lane.

What I did about it: moved the re-arm to the top of the upgrade handler (it must run BEFORE the closing check, not inside the async IIFE) and gated it on wssDead && server.listening: an upgrade can only arrive after the 'close' backstop if the server re-listened ('close' fires only once every connection is gone; I verified empirically that server.listening flips to false synchronously on close() and back to true on re-listen). On that detection both latches are reset (wssDead = false, closing = false) and a fresh wss is created. This keeps f8f58d3ab1's shutdown behavior intact — during a real shutdown close server.listening === false, so the latch is never wrongly cleared (its 5-test pending-close suite passes).

Local verification (targeted, per local-test discipline): npm run typecheck clean; node --import tsx --test tests/ws-bridge.test.ts tests/ws-bridge-pending-close.test.ts → 6/6 pass, including the PR's regression test (which fails without the latch reset). Full-suite verdict is the remote CI on the new head.


结构化八问:

  1. 是什么问题:PR 分支与 master 冲突(双方都改了 ws-bridge 升级处理器的同一区域);且纯文本合并后 PR 的修复不完整——master 新加的关机闩锁(closing)在 ladder re-listen 后同样会把 WS 车道永久打死。
  2. 什么原因导致的:两个独立修复(本 PR 的 wss 懒重建;f8f58d3ab1 的 pending-upgrades 关机闩锁)都假设 server 对象 close 后不会再 listen,但 Windows: concurrent lane proxies can double-bind the same zone port (SO_REUSEADDR semantics) — instance pointer diverges from the socket owner #2941 post-bind ladder 恰恰会对同一个 server 对象 close+re-listen;closing 闩锁没有任何重置路径。
  3. 影响范围与严重程度:所有命中 Windows: concurrent lane proxies can double-bind the same zone port (SO_REUSEADDR semantics) — instance pointer diverges from the socket owner #2941 端口冲突 ladder 的实例(绑定端口被外部监听者占用时):re-listen 后整条 WS 压缩车道(opencode/codex responses)永久死亡——原病灶是 503(wss 已关),纯合并后变成 ECONNRESET(闩锁打死 socket);属"特定条件下车道不可用",条件即端口冲突。
  4. 怎么修复的:rebase + 双方意图取并集 + 最小语义胶水——在升级处理器顶部检测「close 兜底已跑过(wssDead)且 server 正在 listen」⇒ 判定发生了 re-listen ⇒ 重置两个闩锁并重建 wss;改动自包含于 src/ws-bridge.ts,无 API/配置面变化。
  5. 是否有回归问题:逐项审过——(a) f8f58d3ab1 的关机行为:真实关机 close 期间 server.listening===false,闩锁不会被误清,in-flight/pending 升级照旧被 destroy(pending-close 测试 5/5 通过);(b) PR 的 re-arm 语义保持,只是提前到同步段(closing 检查之前);(c) 其余文件零改动,无阈值/时序/输出格式漂移。
  6. 是否破坏缓存:不涉及——不触碰缓存、重试、增量游标、去重键。
  7. 是否改了设计方向:没有——沿用 PR 既有的懒重建思路,未引入新表面。
  8. 是否引入了新的配置:无。

判定:冲突已解、已推送(a3d80e4291),PR 恢复可合并状态;最终绿/红以新 head 的远程 CI 为准。

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working p2 root-cause

Projects

None yet

1 participant