Repository navigation
fix: self-restart replacement binds-or-dies on the freed port instead of hopping (#3044) - #3055
Conversation
793ba8d to
2a86aca
Compare
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.3644-pr.3055.61026d5This exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr3055.tgz
npm install -g packageInstall instructions are refreshed on each push. |
[bot] 🏷 冲突已解决并重新推送,PR 现在 Rebase 与冲突:master 在 PR 切出后前进了很多(含同域新机制 #2947 port-owner-guard、#2945 restart 隧道强关)。3 处文本冲突全部是「双方各自新增」的意图合并:CHANGELOG 条目位置(#3044 条目置顶,保留 master 全部条目)、 审核中发现并已修的 2 个问题(不修则本 PR 的保证不完整):
结构化八问
本地证据: 收工判定:冲突已解、可合并状态、定向验证绿;等 CI 全绿后请人工合并(我不代合并):#3055 |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) CI triage 完成:之前的 3 类红全部归因——一类是我们自己的(已修),另两类是 master 今日合入的 #2731 自己坏了,已单独立 issue。 ① 我们的:unused-exports 门禁(已修) ② 不是我们的:master 自 #2731 起编译就是红的(已立 #3064) 为什么 master 自己的 CI 没抓到:push 事件的工作流只跑静态守卫, 处置:已立 #3064(带来源标记),给出两个修复方向请 owner 拍板(补一行 import,或让 google lane 像其余三条 wire 一样走 session-resolve 的 wiring adapter),外加一条改进项(push 事件至少跑一次 typecheck,否则同类漏 import 会继续静默进 master)。handle.ts 的修法没有混进本 PR(保持 diff 范围干净);这几条车道要等 master 经 #3064 修好后自然转绿,本分支无需重推。 当前 head 结论不变:本 PR 代码侧已完成(rebase 到当前 master + 两处评审修复 + 门禁修复,共 4 个干净 commit),等 #3064 在 master 修好、CI 全绿后人工合并。 |
… of hopping (#3044) With autoRestartOnUpdate, a replacement spawned while the old instance still held its port inherited the launcher's EADDRINUSE ladder and silently drifted to P+1, P+2, ...: idle orphans serving no client while the prior instance never exited (#3044 reported 14 orphans / ~8 GB private memory over 9 h). - New src/listen-policy.ts: pure EADDRINUSE decision table. A spawn marked BILI_UPGRADE_REPLACEMENT=1 (set by performSelfRestart at spawn time) keeps the bounded same-lane predecessor wait (#1723) and then fails fast instead of laddering. All non-replacement paths are byte-identical to the old inline handler. - performSelfRestart verifies the handover against the replacement's own registry record (pid + port it actually bound) instead of a raw TCP answer: wrong port -> terminate + resume; another live bili instance owns the port -> treat the upgrade as landed there and exit cleanly instead of resuming an old-code copy that would ladder-drift into its own zombie; foreign non-bili squatters keep today's resume behavior. Kill results are verified and a surviving child escalates loudly with a manual-cleanup hint. Tests: tests/listen-policy.test.ts (policy matrix), auto-restart.test.ts (marker env, misplaced termination, supersede-exit, unkillable escalation), port-zone-live.test.ts (live fail-fast end-to-end, CI lane). Internal env BILI_UPGRADE_REPLACEMENT documented in CONFIGURATION.md en+zh via the generator seed.
… x #2947) Cross-branch gap found while rebasing #3044 onto master: #2947's post-bind port-ownership check (src/port-owner.ts resolvePortClaim) gives any launch-token child the right to move +1 off a contested just-bound port. A self-restart replacement inherits its parent's BILI_LAUNCH_TOKEN, so on socket semantics that tolerate a concurrent bind onto an actively-listening port it could still hop P+1 after a 'successful' bind — the exact orphan shape #3044 kills, via the second hop path. The EADDRINUSE handler was already closed by this branch; this closes the claim path too: postBindCanLadder() in the same pure policy module, pinned in tests/listen-policy.test.ts.
…#3044 x #2895) The new handover verification's foreign-peer search excluded only the child's pid. The dying parent bili process has its OWN registry record for this very port (written at its announceListening), still 'live' while we drain — so a healthy replacement that bound the port but had not yet announced read as superseded by us and got killed, resuming old service. Master's #2895 tunnel-handover test (real in-process bili server) pins it: superseded-by-peer instead of a clean handover. Exclude process.pid, the same way the terminal decision already does.
EaddrinuseAction is only referenced inside src/listen-policy.ts (decideEaddrinuse's return type); nothing imports it, so the unused-exports gate flagged it INTERNAL_ONLY. Make it module-local. Behavior unchanged.
5d63510 to
b33ca09
Compare
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 上游冲突已处理:rebase 到最新 master( 这次发生了什么:上一轮 rebase 之后 master 又合入了 4 个 PR(#2731 后续、#3058 kernel preserveRecentMessages 5→3、#3056 codex compact、#3052 dsh exports、以及关键的 #3067),PR 再次与基线冲突。 重要发现——上一轮的满屏红其实都是 master 自己的伤,且已全部修好:#3067(「unblock master lanes」)一次性修了两件事—— 结构化八问:
判定:代码侧就绪,等本轮 CI 全绿后人工合并。 |
… of hopping (#3044) Second trigger path from the 2026-10-13 recurrence on 0.1.197: a host respawn (dsh plugin channel) races an attach scan and spawns while an identical instance already serves the lane's port; that child laddered to P+1 and idled as an orphan while traffic re-pointed to it. findSameLaneTwin recognizes a live same-lane SAME-build holder (the mirror of findSameLanePredecessor, which only matches different builds). Any non-strict lane'd spawn hitting it now fails fast with exit code 1 and a loud log line instead of laddering: the duplicate loses, the serving instance keeps serving, and the next launch attaches to it. The rule is deliberate and deterministic — tearing down a live-serving instance for an identical fresh one buys nothing, and no admin endpoint or cross-process kill authority is needed. Strict-port and manual launches are unaffected (they already fail fast).
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.419-master.bd2aa6e # this exact build
npm install -g billion-context@master # always the newest merged build
|
Follow-up to the conflict takeover: master advanced (ranxianglei#3055 port-hop fixes, ranxianglei#3165 per-lane attach policy, ranxianglei#3072 codex WS egress fidelity) after the first merge. Conflicts confined to files the branch never semantically touched; resolved verbatim to origin/master.
Self-restart handover over an occupied port: bind-or-die, never hop (#3044)
Closes #3044
What was wrong
With
autoRestartOnUpdate, when the old instance still held its port at spawn time (or a racing process took it during the drain window), the replacement inherited the launcher's EADDRINUSE ladder (BILI_LAUNCH_TOKENset, not strict-port) and silently drifted to P+1, P+2, …: idle processes serving no client while the prior instance never exited. The issue reports 14 such orphans over 9 h (~8 GB private memory combined),stalenever converging, and the host never learning the upgrade didn't land.A 2026-10-13 recurrence on a clean host restart (v0.1.197, predating this fix) confirmed the leak repeats on every plugin-channel update and exposed a second trigger path in the same failure domain: a host respawn (dsh plugin channel) raced its own attach scan against a LIVE same-build instance — cmdline
--port 18812landed LISTENing on18813while the 18812 instance kept serving, then became the orphan. Launcher-spawned children carry no replacement marker AND the predecessor wait only matches DIFFERENT-build holders, so a same-build twin sent them straight down the ladder. Observed side effects of the coexistence: the sticky lane-port cursor (port-zone.json) ratcheted permanently with each hop, and two live instances contended on the shared state dir (persist stale-snapshot rejections).Four sub-defects in one mechanism family (bind/handover over an occupied port):
child.kill()results were never verified (the surviving orphan is the reported leak), and the parent's resume-after-failure went through the same ladder, so an OLD-code copy could drift onto P+1 as its own zombie.The fix
src/listen-policy.ts— pure EADDRINUSE decision table (decideEaddrinuse). A spawn markedBILI_UPGRADE_REPLACEMENT=1keeps the bounded same-lane predecessor wait (port-zone 重启竞态:sticky 端口单调漂移,升级依附与无界自增互相矛盾(#1660 后续) #1723) and then fails fast instead of laddering; every non-replacement, non-twin path is byte-identical to the old inline handler (pinned by the policy matrix test + untouched legacy assertions).findSameLaneTwininsrc/instance.ts; first check indecideEaddrinuse): a live same-lane registry entry running OUR build holds the port → the newcomer exits 1 loudly, outranking every other branch, for marked AND unmarked spawns alike. Log line: "port N is already served by a same-build bili instance (pid …, lane …) — exiting instead of hopping: the duplicate loses and the serving instance keeps serving; the next launch will attach to it". Deterministic rule chosen: the older/serving instance wins — tearing down a live-serving instance for an identical fresh one buys nothing, and no admin endpoint or cross-process kill authority is needed. Different-build overlap stays on the existing path (bounded predecessor wait; marked replacements fail fast; the old instance's own update cycle handles true upgrades).performSelfRestartmarks the child at spawn (env merge, does not mutate the parent env) and verifies the handover against the replacement's own registry record (pid + port it actually bound): wrong port → terminate, kill verified, service resumes; another live bili instance already owns P → the upgrade counts as landed there and the parent exits cleanly instead of resuming an old-code copy that would ladder-drift into a zombie; foreign non-bili squatter → today's resume behavior kept (deliberate boundary). Kill verification escalates loudly with a manual-cleanup hint when SIGTERM doesn't take.postBindCanLadder, autoRestartOnUpdate + occupied target port -> orphan process leak: new instance silently hops to the next port, old instance never exits #3044 x fix: verify sole port ownership after bind; guard proxy-origin against clobbering a live holder (#2941) #2947): the post-bind claim check (resolvePortClaim) no longer lets a marked replacement move +1 off a contested just-bound port.Behavior changes (old → new)
child.kill()fire-and-forgetconfig surface
One new internal env var, resolved through the existing env-var convention (spawn-channel category, like
BILI_LAUNCH_TOKEN/BILI_STRICT_PORT):BILI_UPGRADE_REPLACEMENT=1— set automatically byperformSelfRestarton the replacement's spawn env only; never meant to be set by users or hosts. Documented in CONFIGURATION.md en+zh via the generator seed (website/config-reference/env-only.yaml; blocks regenerated withtools/gen-config-docs.mjs generate). No config-file field, no CLI flag, no default change anywhere else.Tests
tests/listen-policy.test.ts(new) — full policy matrix: twin collision outranks replacement/strict/ladder (and applies token-independent); replacement never ladders (incl. exhausted wait budget); legacy launcher-child, strict-port, and manual-launch behaviors unchanged when no twin holds.tests/auto-restart.test.ts— marker env carried without mutating the parent env; misplaced replacement terminated + service resumed + child verified dead; supersede-exit path (no resume, clean finish); unkillable replacement escalated loudly.tests/port-zone-live.test.ts— live end-to-end (CI laneACP_TEST_CLAUDE_NATIVE=1): (1) marked replacement against a squatted zone base exits 1 and nothing lands on base+1; (2) unmarked lane'd spawn against a LIVE same-build twin exits 1, nothing lands on base+1, and the twin keeps serving — a direct replay of the 2026-10-13 20:12 event.Out of scope (owner-decision follow-up)
port-zone.jsonsettles to is a persistence-semantics/product decision ([Design] 定死端口 lane(claude 48787 / zcode 48789)端口被占时的 0 配置解法 #1660 deliberately made sticky follow drift so repeat launches find the daemon fast). Options are laid out in the issue thread; awaiting owner/user call — deliberately NOT changed here.