Repository navigation
[PR] fix(launcher): adopt proven-healthy child at spawn timeout instead of killing it (#4994) - #5010
Conversation
…tead of killing it (#4994) When the 60s spawn budget expires, the spawner killed its own spawned child unconditionally. If the health probe on the preferred port already answered with pid == our child's pid (the instance handshake merely never matched), that kill destroyed a proxy that was bound and serving — and the lane then degraded to direct uncompressed traffic until a host restart. - diagnosePort now returns { text, selfHealthy } so the timeout path can act on the same evidence it already collected; all user-facing strings are byte-identical to before. - Rung 1: selfHealthy (health pid match on the preferred port) -> settle the zone sticky port, start the owner heartbeat, attach and return — no kill. - Rung 2: the child may have laddered to another port after EADDRINUSE (src/server.ts ladder); find it via its own pid-tagged listen line in the per-port log (bytes written since spawn) and adopt it with the same strict proof (health ok + pid match). - Unproven cases keep today's behavior exactly: burn the budget, taskkill /F /T, throw the same error text.
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.6807-pr.5010.812f1acThis exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr5010.tgz
npm install -g packageInstall instructions are refreshed on each push. |
(ai: vllm-qwen/qwen3.8-27b) 查重:不重复。 本 PR 修的是 #4994 的 facet ④(60s 预算耗尽后超时路径无条件 标签: bug / dsh / scope/wiring / p1 —— 压缩车道整体停摆到宿主重启 = 可用性类 p1;触发场景为 dsh 车道,修复点在 launcher bring-up 接线层。 本 PR 带 |
… timeout adoption rung 2 can find it (#4994) Review finding on 5ae293d: rung 2 of the timeout adoption parses the child's pid-tagged listen line out of the per-port log, but post-#2864 a launcher-spawned child mutes its stderr leg into bili.log once the file leg is live - so the banner (the only producer of that line) never reaches the port log and rung 2 was unreachable for a healthy child in the primary starvation shape (clobbered records, healthy state dir). The listen-failed paths already carry this explicit write for exactly this reason; mirror it on the success path (one short line per spawn, best-effort append). Behavior change (additive): launcher-spawned children now append ONE listen-success line to %TMPDIR%/bili-proxy-<port>.log per spawn (previous content unchanged; rotation at PORT_LOG_MAX_BYTES untouched). No config surface, wire, or persistence-format changes.
(ai: vllm-qwen/qwen3.8-27b) CI 终局补记:最终 head |
…meout-adopt-healthy-child
…meout-adopt-healthy-child
…meout-adopt-healthy-child
(ai: vllm-qwen/qwen3.8-27b) Push landed once connectivity recovered — head is now 7c397e1 (origin/master @ b08fbf3 merged in cleanly, #4994 changes intact). Post-push state: mergeable, no conflicts, behind→blocked (required checks running on the new head, none failed). |
# Conflicts: # scripts/live-files.mjs
|
(ai: sglang-gpu0/qwen3.8-27b) Synced this branch to current master. The only conflict was Local verdict on the merged head |
…meout-adopt-healthy-child
(ai: vllm-qwen/qwen3.8-27b) Verified the sync above: |
# Conflicts: # scripts/live-files.mjs
# Conflicts: # scripts/live-files.mjs
The previous sync commit resolved the live-files.hunk to the master side and dropped this PR's own registration row; CI's tripwire would have red every shard for it.
|
(ai: sglang-gpu0/qwen3.8-27b) 同步一条:本分支已并入当前 master 推了 9180053(无冲突;scripts/live-files.mjs 五条 spawn 测试条目 #4982/#4994/#5012/preload-sandbox-dirs/event-loop-watchdog(如 master 侧已有则保留不重复) 校验无重复,共 54 条)。本地 tsc 干净、issue4994-spawn-timeout-adopt 4/4。顺带澄清此前一处读取坑:用 8 位短 SHA 查 actions/runs?head_sha= 会返回空,需完整 40 位——按完整 SHA 查,上一头 e331f44 的主 CI 其实是绿的(它 14:55:04Z 触发时 master 已含 cf99d93)。本轮 CI 结果以新 push 为准。 |
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.916-master.3a5d532 # this exact build
npm install -g billion-context@master # always the newest merged build
|
Closes #4994
What this fixes
Facet ④ of #4994 (the untracked one — facets ①②③ are covered by #3890/#4572 and the merged PR #4825): when the 60s bring-up budget expires, the spawner killed its own spawned child unconditionally. In the reported incident the health probe on the preferred port was already answering with
pid == our child's pid("port 18790 finally answers healthy from our child (pid 50948) but its instance handshake never matched") — i.e. the child was bound and serving, only the launch-token handshake never matched. The spawner thentaskkill /F /T'd that healthy proxy, nothing retried the bootstrap in that window, and the lane degraded to direct uncompressed traffic until a host restart.Behavior change (old → new)
killAbandonedChild(child)+ threwproxy did not become healthy within 60000ms.diagnosePortalready collected proves the victim is our own healthy child (healthokANDpid === child.pidon the preferred port), the spawner settles the zone sticky port, starts the owner heartbeat, attaches to it and returns it as the handle — no kill. A second rung covers the EADDRINUSE-ladder case (child re-bound on another port persrc/server.ts:1381-1420): the child's own pid-tagged listen line in its per-port log (bytes written since spawn) locates the real origin, adopted under the same strict proof.diagnosePortnow returns{ text, selfHealthy }; all user-facing prose is identical). No config surface changes. No kernel changes.Regression tests
New
tests/issue4994-spawn-timeout-adopt.test.ts(fake-clock driven, modeled on the #2954 suite): rung-1 adopt, rung-2 ladder adopt, unproven-still-killed, foreign-responder-never-adopted. Verified the two adopt rows FAIL against pre-fix code (child SIGKILL'd) and pass with the fix; the two negative rows pass on both. Adjacent suites green locally: issue2954 / issue2187 / issue3116 / issue4850 (20/20), typecheck + build clean. Full-suite verdict = remote CI.