Wait out a broker that is refusing connections, and stop three tests racing the runner

[Edo771977]

Two commits: the broker change this PR started as, and three pre-existing flaky tests the review work surfaced.

335/335 idle (run twice), 334/335 under load average 36 — the one failure there is a fourth pre-existing flake, unrelated and reproduced identically on main (see the end). tsc clean.

1. Wait out a broker that is refusing connections, not one that is missing

ensureBrokerSession() probes the persisted broker for 150ms and, when that fails, replaces it — spawning a second broker with its own app-server and every MCP server under it. What the failed probe meant was never asked, and the two answers deserve opposite treatment: "nothing is there" and "something is there but would not take this connection right now" are the same boolean.

probeBrokerEndpoint() now reports the outcome, not just a verdict:

probe outcome	meaning	what happens
EAGAIN (full accept backlog), EBUSY (Windows named pipe with no free instance), timeout	listening, refusing right now	second window (1500ms, overridable), reused if it starts accepting
ECONNREFUSED, ENOENT, unparsable	nothing is there	straight to the replacement, as before — no wait

Liveness at that gate is now pinned to the identity recorded at spawn time, like the shutdown path, so a reused pid reads as gone instead of keeping an abandoned record alive.

What the first version of this PR got wrong

It waited whenever the pid was alive, on the theory that a broker mid-turn keeps its event loop busy and misses the probe. That theory is wrong on POSIX and two adversarial reviews caught it. connect() to a Unix socket is completed by the kernel as soon as the peer is listening — the server never has to call accept(). A server blocked in a 5-second spin loop still answers:

probe while the server's event loop is blocked: {"outcome":"connect","ms":1}
probe while the server's event loop is blocked: {"outcome":"connect","ms":0}
after the server is gone:                       {"outcome":"ECONNREFUSED","ms":1}

So the old gate fired only when nothing was listening — the "broker is gone" case it claimed to be free for. What it actually caught was the broker's own graceful shutdown: runShutdown() closes the listener before its first await and the process then lives through two 5s graces, so for up to ten seconds the pid is alive and nothing listens. Measured, same outcome either way:

before: {"elapsedMs":368,  "replacementSpawned":true}
after:  {"elapsedMs":2033, "replacementSpawned":true}

1500ms of that was spent holding the broker lock. It was also, for any live-pid record, exactly the cost this PR cited as its reason for not taking upstream #768's blanket 3s probe. Gating on the outcome removes it. (openai#768's other two fixes — never tearing down a live broker, and serializing the check-then-create window — were already in this fork.)

Tests

Four, deciding on injected probe outcomes rather than wall-clock races — a Unix connect() cannot be made to report EAGAIN on demand without a saturated backlog, and EBUSY needs Windows:

sabotage	which test fails
no busy window at all	refusing-broker reuse, and keeps-refusing replacement
outcome gate removed (anything alive waits)	gone-broker never pays
shipped default 1500 → 15	refusing-broker reuse
liveness gate removed	dead record is reclaimed, not waited on

2. Three pre-existing tests that raced the runner

All three pass idle and fail under load; all three were reproduced here in a full-suite run at load average 36, and each asserted something other than what its name promises.

shutdown request always uses a finite deadline failed on unreachable: true, not on time. It called sendBrokerShutdown() with timeoutMs: 0, clamped to 1ms — and a loaded machine does not complete connect() in 1ms, so the deadline fires before the connection and the outcome correctly reports a broker it never reached. One loop was covering two properties, so they are now separate: timeoutMs: 0 asserts only that the call settles, raced against a watchdog (socket.setTimeout(0) disables the timer outright, and the clamp is the only thing between that and waiting forever); the accepted-but-silent case gets a budget the connect can win, plus an assertion that the server did accept — otherwise the test was not measuring its own scenario.

The two interrupt-starvation tests failed because the SessionEnd hook splits one 2200ms budget between the reaper's dead-turn interrupts and the session's own, and a hung interrupt consumes its whole slice. Under load nothing was left for the second job, and the test reported a starvation that had not happened. The budget is now tunable through CODEX_TURN_INTERRUPT_BUDGET_MS, like the other lifecycle limits, default unchanged; the two tests raise it to 8000ms. The sharing math — the property under test — is identical at any budget, and the knob stands on its own for a workspace with many active jobs.

Both fixes still fail on the defect they guard: removing the zero-timeout clamp makes the deadline test report "never settled", and removing the per-job share makes both starvation tests fail.

Known, not fixed here

Under the same load, task logs subagent reasoning and messages with a subagent prefix fails: the subagent's thread/started name never arrives before the turn ends, so every log line falls back to the thread id (Subagent thr_2 reasoning: instead of Subagent design-challenger reasoning:). Reproduced 4/4 on main and 4/4 on this branch under 32 spinners, so it is pre-existing and untouched by these commits. It looks like a notification-ordering race between the child thread's thread/started and the parent's collabAgentToolCall item, which is a deeper fix than this PR should carry.

🤖 Generated with Claude Code

https://claude.ai/code/session_01UXfvnjSC72HsM6EEPVt2Tg