Skip to content

fix: wait out a same-lane upgrade predecessor before laddering zone ports (#1723) - #1726

Merged
ranxianglei merged 1 commit into
masterfrom
2026-09-30_port-zone-restart-race
Sep 30, 2026
Merged

ranxianglei merged 1 commit into
masterfrom
2026-09-30_port-zone-restart-race

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Fixes #1723

Problem

An upgrade-restart overlap (the new build's child binding while the OLD build's instance of the same lane is still draining) made the EADDRINUSE handler ladder +1 without identifying the holder, and settleZonePort wrote the drifted port back sticky. Result: a monotonic ratchet burning one port per auto-update — the sticky guarantee failing systematically exactly at the version boundary, where it matters most.

Approach

Issue directions 1 + 3 adopted (identify-before-ladder, bounded wait); direction 2 (launcher drain barrier) rejected — the launcher cannot know when the old build finished flushing, and the bounded wait already degrades gracefully; direction 4 (version epoch) deferred — the fingerprint identity check covers the real case without a new sticky schema.

  • src/instance.ts: findSameLanePredecessor(entries, port, lane, ownFingerprint) — identifies the EADDRINUSE holder from registry markers: a LIVE same-lane instance running DIFFERENT code (codeFingerprint, lane-aware instance discovery — single proxy-origin file breaks same-client reattach under concurrent multi-client use #1232) is a being-replaced predecessor; a missing fingerprint (pre-lane-aware instance discovery — single proxy-origin file breaks same-client reattach under concurrent multi-client use #1232 marker) counts as a different build; a same-code peer or a foreign/undeclared-lane holder is never a wait target.
  • src/server.ts: on EADDRINUSE at the lane's port, when the holder is such a predecessor, wait for it to release (500ms ticks, 5s total budget) and rebind the SAME port. Budget exhaustion falls through to the existing +1 ladder, so worst case equals today's behavior plus a bounded delay. A lane'd launch settling ABOVE its preferred port now logs a drift warning instead of drifting silently.
  • Tests: unit decision table for the predicate (tests/port-zone.test.ts, 7 cases) + live e2e against real dual processes with a byte-different build copy (tests/port-zone-live.test.ts): takeover rebinds the same port and sticky stays put; a holder that never leaves still ladders +1 as before.
  • Docs: README / CONFIGURATION / TECHNICAL-NOTES, en+zh.

Behavior change (old -> new)

  • OLD: overlapping same-lane restart silently laddered +1 and permanently rewrote the sticky record (unbounded ratchet; every auto-update burned a port).
  • NEW: the new process waits out a same-lane predecessor running different code (bounded: 10 x 500ms) and rebinds the SAME port; sticky is left untouched. If the predecessor never releases within the budget, the old +1 ladder runs exactly as before. One new warn log line when a lane'd launch settles above its preferred port.
  • Wire/protocol: untouched (listen-phase only, both compression modes unaffected). Config surface: NO CHANGE (no field added/renamed/re-semanticsed; nothing under ~/.config/billion-context/config.json, no BILI_* env var, no CLI flag).

Pre-flight (rebased head: master d27dc34 + this commit 33d0acb)

  • npm run typecheck — clean
  • npm test — 3290 pass / 0 fail / 19 skipped
  • npm run build — OK
  • ACP_TEST_CLAUDE_NATIVE=1 node --import tsx --test tests/port-zone-live.test.ts — 7/7 pass (incl. both new port-zone 重启竞态:sticky 端口单调漂移,升级依附与无界自增互相矛盾(#1660 后续) #1723 cases and the unchanged squatter-ladder case)
  • codex e2e: NOT runnable locally at commit time — the local sglang fixture behind the default E2E_UPSTREAM_URL refused connections mid-run (ECONNREFUSED; the proxy under test booted fine and correctly surfaced the upstream failure in its 502 detail). This PR touches src/server.ts, so ci-e2e.yml auto-runs on the PR and is the authoritative e2e signal here.

…orts (#1723)

An upgrade-restart overlap (the new build's child binding while the OLD build's instance of the same lane is still draining) made the EADDRINUSE handler ladder +1 without identifying the holder, and settleZonePort wrote the drifted port back sticky — a monotonic ratchet burning one port per auto-update.

- instance.ts: findSameLanePredecessor() identifies the EADDRINUSE holder from the registry markers (lane + codeFingerprint + pid): a live same-lane instance running DIFFERENT code is a being-replaced predecessor (missing fingerprint = pre-#1232 = different build); a same-code peer or a foreign/undeclared-lane holder never is.
- server.ts: on EADDRINUSE at the lane's port, wait for the predecessor to release (500ms ticks, 5s total budget) and rebind the SAME port; budget exhaustion falls through to the existing +1 ladder, so worst case equals today's behavior plus a bounded delay. A lane'd launch settling ABOVE its preferred port now logs a drift warning instead of drifting silently.
- Tests: unit decision-table for the predicate + live e2e against real dual processes with a byte-different build copy (takeover rebinds the same port, sticky stays put; a holder that never leaves still ladders +1 as before).
- Docs: README/CONFIGURATION/TECHNICAL-NOTES, en+zh.
@github-actions

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-09-30_port-zone-restart-race (33d0acb)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.1678-pr.1726.a22fad3

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr1726.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 PR Review #1726 — 可合并 ✅

一句话中文摘要:修复了升级重启竞态导致的 sticky 端口单调棘轮(EADDRINUSE 时凭 lane+codeFingerprint 识别同 lane 旧 build,限时 5s 等其释放后重绑原端口,超时落回原阶梯),已按五关审完、在当前 master 上独立复验全绿,可以合并。


① 原 issue(方向关)— 通过

#1723 的机制分析准确:棘轮由两个生产者构成——(a) EADDRINUSE 决策点不识别持有者,把「正在退场的同 lane 前代」当外部占用者阶梯 +1;(b) settleZonePort 无条件回写。本 PR 修在决策点 (a),不动 sticky 语义、不动阶梯本体——判别信号(registry marker 里的 lane + codeFingerprint,#1232/#1225 已有)现成可用,同一根因换形状出现时修复依然成立(新持有者类型不匹配谓词即回落原阶梯)。方向 2(drain barrier)拒绝理由成立(跨版本进程间无可靠 flush 完成信号),方向 4(epoch)暂缓理由成立(fingerprint 已编码 build 身份,不值得为它动持久化格式)。

② 回归审查(既有行为清点)

Diff 触碰路径的既有行为逐项判定:

既有行为 结论
launcher 子进程 EADDRINUSE +1 阶梯(≤17 次→ephemeral) 保持——等待分支不消耗 listenAttempts,预算耗尽后阶梯计数与今天完全一致,最坏情况 = 现状 + ≤5s 延迟
手工 bili start fail-fast(无 launchToken)、strict-port 拒绝(#964) 保持——等待分支在外层 launchToken && !strictPort 门内,两条路径不可达
settleZonePort 粘性回写 保持——机制未改;等待成功时落定点==首选口,回写是幂等的;fallback 路径与今天相同
跨 build attach 拒绝(#1225/#1232 fingerprint 门) 保持——谓词只读 registry,不改 attach 逻辑(live 测试日志可见 native-attach: skip … code-fingerprint-mismatch 照常工作)
死 lane 记录保留 有意不变(#1723 floor 2 已定:无害,复活走有界阶梯)
wire/协议、两种压缩模式 零触碰(listen 阶段)
config surface 零变化——docs diff 仅改写既有 BILI_ZONE_PORT 条目描述,无新字段/env/flag

唯一新增输出:一行 drift warn(lane'd launch 落在 preferred 之上时)——纯日志增量,每进程至多一次(单次成功 bind 触发一次 announceListening)。PR 正文的行为变更披露完整(旧→新、wire/config 声明均准确)。

③ 代码审查(主关)

  • findSameLanePredecessor(src/instance.ts:141-167):纯函数、分支齐全。缺 lane 或缺自身 fingerprint 时直接放弃等待(安全降级到阶梯);isPidAlive 过滤死记录(崩溃未 unregister 的 marker 也安全);同端口至多一个活持有者,多 entry 无歧义;pid 复用需同时命中同 port+同 lane+存活,窗口内概率可忽略。missing fingerprint 按「不同 build」处理(pre-lane-aware instance discovery — single proxy-origin file breaks same-client reattach under concurrent multi-client use #1232 marker)是有意的保守取舍——最坏对 pre-lane-aware instance discovery — single proxy-origin file breaks same-client reattach under concurrent multi-client use #1232 的同 build peer 多等 5s,已在注释与 PR 中披露。
  • 等待循环(src/server.ts:782-786):每个 EADDRINUSE 事件重新求值谓词(前代释放后若被第三方抢占,下一 tick 自然转阶梯);setTimeout 不加 unref 正确(启动期进程本就该活着等 bind);predecessorWaitAnnounced 防日志刷屏。
  • restart.ts 自重启路径覆盖成立:replacement 继承 BILI_LAUNCHER_LANE/BILI_LAUNCH_TOKEN env,新 dist fingerprint 不同、旧 listener 还在 drain——正是同一竞态,命中谓词。
  • 规范合规:无 as any/@ts-ignore,用 loggerLog,注释全部带 issue 锚点且解释 why。

④ 测试有效性

  • 单测决策表(tests/port-zone.test.ts,7 例):覆盖谓词全部分支(异码=前任 / 缺 fp=旧 build / 同码 peer≠目标 / 混合持有者 / 错 port·错 lane·未声明 lane / 死 pid / 无 lane·无 fp·空 registry),断言都有具体期望值——若有人日后删掉 fingerprint 比较或误把同码 peer 纳入等待,「same-code peer」与「mixed holders」两例会抓住。真测试。
  • live e2e(tests/port-zone-live.test.ts,2 新增例):真实双进程 + 字节级不同的 build 副本(dist 尾部加 marker,content hash 不同——正是升级在磁盘上产生的东西)。断言钉的是行为本身(h2.port === ZONE_PORT_BASE 且 readZonePort() 不变;fallback 例钉 ZONE_PORT_BASE + 1),不是恒真式。两条路径(接管重绑 / 超时落回阶梯)都被兜住。
  • 小观察(不构成阻塞):takeover 例的 setTimeout(1500) 后再杀旧进程是时序依赖,但 5s 预算余量充足,实际稳定。

⑤ diff 整洁度

单 commit,10 文件 +238/-11,每一行都与本 PR 目的相关(谓词 + 接入 + 测试 + en/zh 文档镜像);无空白噪音、无 lockfile/生成物连带、无 version 触碰。干净。

合并前验证(我在当前 master 上独立复跑)

未发现需要补修的问题,无需追加 issue。建议合并。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

port-zone 重启竞态:sticky 端口单调漂移,升级依附与无界自增互相矛盾(#1660 后续)

1 participant