You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
bug: cross-process startup-window race — two concurrent launches each spawn a proxy (double writers over one sessions dir) #707
ensureProxyRunning (#394/#417 attach semantics) only attaches when it finds a healthy instance at probe time. Two processes calling ensureProxyRunning inside the probe→healthy window (child spawn + health polling, ~1s–several s) both see "no healthy instance" and each spawn a child. Result:
the "orphaned" extra proxy is not reaped early: its parent-pid watchdog tracks ITS OWN spawner, so it lives until that client exits.
Repro
Launch the same client twice within ~1s of each other while the first proxy is still coming up (e.g. bili codex ×2 concurrently, or — since #706 — two bare pi terminals opened back-to-back, which now self-bootstrap at extension load). Observe two node dist/index.js start children and two "proxy started" log entries.
Before #706 the window was reachable only via deliberate concurrent bili <client> launches. Native mode makes self-bootstrap the default path for every bare pi start, so back-to-back terminal opens become a realistic trigger.
Impact
Concurrent writes over one sessions dir (state corruption risk), duplicated proxy processes, instance-file flapping between the two origins.
Suggested fix
Publish a "starting" marker BEFORE spawning (e.g. write the instance file with the launchToken up front, or a separate lockfile held across the spawn window). Concurrent probes then find an in-progress bring-up and WAIT for health instead of spawning. In-process callers are already covered by the singleFlight() wrapper added in #706's review commit (3cb0331); this issue is the cross-process half.
The race window is exactly as described. Cross-launch coordination happens entirely through the instance file (<state>/proxy-origin), and that file is only written by the proxy child after it successfully binds (announceListening, src/server.ts:430). Between spawnImpl (src/launcher.ts:1509) and the child's first successful listen, there is zero on-disk evidence that a bring-up is in flight:
a concurrent process's probeExistingInstance (src/launcher.ts:1399) finds no record → falls through to spawn;
Plan (matches your suggested fix — the cross-process half only):
New <state>/proxy-starting marker claimed with O_CREAT|O_EXCL (atomic across processes — a rename-based last-writer-wins file would leave the same race at the marker level). Carries {token, pid, host, port, startedAt} and is written by the starter before spawning.
A concurrent caller that sees an active marker (owner pid alive AND age < TTL = SPAWN_WAIT_MS + 30s) waits for the starter's instance instead of spawning — same probe as attach (pid alive + /__bili/health + instanceId match), bounded by SPAWN_WAIT_MS. Marker disappearance (starter gave up / crashed) breaks the wait early so the caller spawns immediately.
Starter clears the marker on every terminal path (healthy / child-exit / timeout) with a token-checked clear; a hard crash leaves a stale marker that the pid/TTL checks treat as dead.
Incompatible-config semantics preserved: if the waited-for instance isn't compatible with our launch options we fall through and spawn anyway (same as today's incompatible-healthy-instance behavior).
Explicit manual bili start deliberately bypasses all of this (no claim, no wait) — it's a user action, not auto-bootstrap. In-process dedup (singleFlight, 3cb0331) stays as-is.
Reproducing empirically (two concurrent launches over one shared state dir) before and after the fix, then I'll open a PR.
Reproduced as described. Two concurrent launches sharing one state dir (fake client kept alive so the window stays open) produced exactly the reported failure: two node dist/index.js start children, two "started proxy" log lines, two registry markers under instances/, and proxy-origin flapping last-writer-wins between the two origins. Root cause confirmed where you pointed: the instance file is written by the child only after its first successful bind (announceListening, src/server.ts), so between spawnImpl and first listen there was zero on-disk evidence of an in-flight bring-up for concurrent probes to find.
Fix (per your suggested approach). The starter claims <state>/proxy-starting with O_EXCL before spawning — {token, pid, host, port, startedAt}. A concurrent caller that finds an active marker (owner pid alive AND younger than SPAWN_WAIT_MS + 30s) waits for the starter's instance using the same probe as attach (pid alive + /__bili/health + instanceId match), bounded by SPAWN_WAIT_MS, then attaches. Marker disappearance (starter gave up / crashed) breaks the wait early so the caller spawns immediately instead of burning the full window. The starter clears the marker on every terminal path via token-checked clear; a hard crash leaves a stale marker that the dead-owner/TTL checks render inert. Incompatible-config fall-through is preserved (a waited-for instance that doesn't match our launch options still spawns, matching today's incompatible-healthy behavior), explicit bili start bypasses the marker entirely, and the in-process singleFlight dedup from #706 is untouched — this PR is the cross-process half.
Verified post-fix. Same repro now yields exactly one proxy; the second launcher logs another bili launch is bringing up a proxy — waiting for it instead of spawning a second and then attaches to the same origin/pid. Preflight: typecheck clean, npm test 1376 pass / 0 fail (2 skipped = gated e2e), build ok. Nine new tests: 7 launcher scenarios (wait+attach without spawn; dead-owner cleanup+spawn; expired-marker cleanup+spawn; early bail <2s; claim-before-spawn assert; failure-path marker cleanup; incompatible fall-through) plus 2 marker-mechanics unit tests (O_EXCL exclusivity, token-checked clear, garbage-file degrade path). One e2e caveat noted in the PR body: the gated codex e2e needs an upstream not reachable from this environment; the change touches only the launcher bootstrap path (src/launcher.ts + src/instance.ts), not the request pipeline.
Activity
ranxianglei commentedon Sep 11, 2026
Confirmed — root cause verified in code.
The race window is exactly as described. Cross-launch coordination happens entirely through the instance file (
<state>/proxy-origin), and that file is only written by the proxy child after it successfully binds (announceListening, src/server.ts:430). BetweenspawnImpl(src/launcher.ts:1509) and the child's first successful listen, there is zero on-disk evidence that a bring-up is in flight:probeExistingInstance(src/launcher.ts:1399) finds no record → falls through to spawn;Plan (matches your suggested fix — the cross-process half only):
<state>/proxy-startingmarker claimed withO_CREAT|O_EXCL(atomic across processes — a rename-based last-writer-wins file would leave the same race at the marker level). Carries{token, pid, host, port, startedAt}and is written by the starter before spawning./__bili/health+ instanceId match), bounded by SPAWN_WAIT_MS. Marker disappearance (starter gave up / crashed) breaks the wait early so the caller spawns immediately.Explicit manual
bili startdeliberately bypasses all of this (no claim, no wait) — it's a user action, not auto-bootstrap. In-process dedup (singleFlight, 3cb0331) stays as-is.Reproducing empirically (two concurrent launches over one shared state dir) before and after the fix, then I'll open a PR.
(中文摘要:已确认跨进程启动窗口竞态——实例文件只在子进程成功绑定后才写入,spawn→healthy 之间磁盘上没有任何"正在启动"的标记,因此两个并发启动会各自 spawn 一个代理、双写同一 sessions 目录。修复方案:spawn 前用 O_EXCL 原子抢占
<state>/proxy-starting标记,并发方看到活跃标记就等待健康实例而不是重复 spawn。)ranxianglei commentedon Sep 11, 2026
Fixed — PR #709: #709
Reproduced as described. Two concurrent launches sharing one state dir (fake client kept alive so the window stays open) produced exactly the reported failure: two
node dist/index.js startchildren, two "started proxy" log lines, two registry markers underinstances/, andproxy-originflapping last-writer-wins between the two origins. Root cause confirmed where you pointed: the instance file is written by the child only after its first successful bind (announceListening, src/server.ts), so betweenspawnImpland first listen there was zero on-disk evidence of an in-flight bring-up for concurrent probes to find.Fix (per your suggested approach). The starter claims
<state>/proxy-startingwith O_EXCL before spawning —{token, pid, host, port, startedAt}. A concurrent caller that finds an active marker (owner pid alive AND younger thanSPAWN_WAIT_MS + 30s) waits for the starter's instance using the same probe as attach (pid alive +/__bili/health+ instanceId match), bounded bySPAWN_WAIT_MS, then attaches. Marker disappearance (starter gave up / crashed) breaks the wait early so the caller spawns immediately instead of burning the full window. The starter clears the marker on every terminal path via token-checked clear; a hard crash leaves a stale marker that the dead-owner/TTL checks render inert. Incompatible-config fall-through is preserved (a waited-for instance that doesn't match our launch options still spawns, matching today's incompatible-healthy behavior), explicitbili startbypasses the marker entirely, and the in-processsingleFlightdedup from #706 is untouched — this PR is the cross-process half.Verified post-fix. Same repro now yields exactly one proxy; the second launcher logs
another bili launch is bringing up a proxy — waiting for it instead of spawning a secondand then attaches to the same origin/pid. Preflight: typecheck clean,npm test1376 pass / 0 fail (2 skipped = gated e2e), build ok. Nine new tests: 7 launcher scenarios (wait+attach without spawn; dead-owner cleanup+spawn; expired-marker cleanup+spawn; early bail <2s; claim-before-spawn assert; failure-path marker cleanup; incompatible fall-through) plus 2 marker-mechanics unit tests (O_EXCL exclusivity, token-checked clear, garbage-file degrade path). One e2e caveat noted in the PR body: the gated codex e2e needs an upstream not reachable from this environment; the change touches only the launcher bootstrap path (src/launcher.ts+src/instance.ts), not the request pipeline.中文摘要: 已按 issue 建议实现跨进程 starting 标记(O_EXCL 原子占位、等待健康后 attach、token 校验清理、死主/TTL 失效),并发启动不再双开代理、消除同一 sessions 目录的双写者,修复在 PR #709,可以合并。
fix(launcher): make the O_EXCL claim the arbiter — close read→claim T…
Merge pull request #709 from ranxianglei/2026-09-11_fix-707-startup-race
fix(launcher): cross-process startup-window race — starting marker pr…
fix: native-hook attach gate — refuse lifecycle-less proxies by defau…
fix: merge master into #4622 — unify #4994 dual-axis watchdog with #4125
fix(launcher,storage): #7156 fixes 1+2 — never spawn a second same-la…