fix: bind test servers to the IPv4 loopback they dial - #703
Merged
Conversation
The dev-platform route tests failed intermittently with a 401 `devplatform.unauthorized` on a request carrying valid session headers, and with a 404 on a repo the test had just registered. Neither the routes nor the fakes were at fault. `app.listen(0)` binds the IPv6 wildcard `[::]`. On macOS/BSD that socket is IPV6_V6ONLY, so the kernel reserves the port in the IPv6 ephemeral space only, while every harness hands out `http://127.0.0.1:<port>` -- an IPv4 URL. The two spaces are independent, so the port the test dials is never reserved at all and any unrelated process holding that IPv4 port receives the request and answers it. A 1800-attempt probe on a developer machine caught three foreign responders: a local MCP server (401 "provide valid authorization token"), a Flask dev server (404 `<!doctype html>`), and a non-HTTP peer that surfaced as `HTTPParserError: Response does not match the HTTP/1.1 protocol`. Binding 127.0.0.1 explicitly makes the reserved port and the dialled port the same port, so the OS refuses a colliding bind with EADDRINUSE instead of silently shadowing the harness. Applied to every test listener that already waits for its `listening` callback (36 sites, both dev-platform harnesses, plus the updater sidecar's server test). The remaining 55 `app.listen(0)` sites read `server.address().port` synchronously on the next line; passing a host makes the bind asynchronous, so those need an await added first and are left for follow-up. harnessLoopbackBinding.test.ts guards the property.
…s-401-flake # Conflicts: # docs/CHANGELOG.md
The standalone guard file added 7 lines naming Dev Platform identifiers, which raised the epic #470 core-decoupling ratchet (1029 -> 1036 in middleware/test). The reference count is the definition of "extraction finished", so the right move is to not add references rather than to raise the baseline. The same three assertions now live in a neutrally-named `harness socket binding` suite inside devPlatformRoutes.test.ts and devRunnerApi.test.ts, which already import their harness. Nothing in the added lines matches a ratchet pattern, so the count is flat at 3294. Reverting either harness still turns all three red.
This was referenced Aug 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Symptom
devPlatformRoutes.test.tsfailed intermittently with401where404was expected — on a request carrying perfectly validx-sub/x-roleheaders. Sibling tests in the same file failed the same way with404 !== 201and404 !== 202.Root cause — not port reuse, not keep-alive
app.listen(0)binds the IPv6 wildcard[::]. On macOS/BSD that socket isIPV6_V6ONLY, so the kernel reserves the port in the IPv6 ephemeral space only. Every harness then hands outhttp://127.0.0.1:<port>— an IPv4 URL.The two ephemeral port spaces are independent. The port the test dials was therefore never reserved at all, and any unrelated process on the machine holding that IPv4 port receives the request and answers it. The test sees that foreign server's response.
Evidence
server.address()afterapp.listen(0)family=IPv6 address=::baseUrlhanded tofetchhttp://127.0.0.1:${port}— IPv4fetchreturned 404 with no request ever reaching our serverHTTPParserError: Response does not match the HTTP/1.1 protocol401 … provide valid authorization token, a Flask/Werkzeug404 <!doctype html>, and one empty bodyThe MCP server's 401 is the reported symptom exactly.
Deterministic reproduction
With a foreign IPv4 listener occupying port P:
100% reproducible. The third line is the property the fix buys: once the harness binds the address it dials, shadowing is impossible rather than unlikely.
The fix
Bind
127.0.0.1explicitly wherever the listener already waits for itslisteningcallback — 36 call sites, including both dev-platform harnesses and the updater sidecar's server test.A neutrally-named
harness socket bindingsuite indevPlatformRoutes.test.tsanddevRunnerApi.test.tsguards it: reverting either harness turns all 3 assertions red (verified).These guards deliberately live inside the existing suites rather than in a file of their own. A standalone file had to name the harnesses in its imports, doc comment, and test titles — 7 lines matching the epic #470 core-decoupling ratchet, which pushed
middleware/testfrom 1029 to 1036 and failed CI. The reference count is the definition of "extraction finished", so the fix was to stop adding references, not to raise the baseline. The count is flat at 3294.Deliberately not changed
55 sites use
const server = app.listen(0);and readserver.address().portsynchronously on the next line. Passing a host makeslisten()bind asynchronously, so adding the host alone breaks them withTypeError: Cannot read properties of null (reading 'port')— measured: a blanket sweep produced 48 failures / 119 cancelled. Each of those sites needs anawaitonlisteningfirst. They remain exposed to this failure mode; noted in the changelog for follow-up.Measured failure rate
<!doctypeJSON parse error inadminProvidersRoute.test.ts— the same bug in another file)The pre-fix full-suite failure is worth calling out: it reproduced this defect in a file the original report never mentioned, which is why the fix is not confined to
devplatform/.Gates
npm run test— green ×2 (6431 tests, 0 fail), on top of currentmaincore decoupling ratchet (#470)— held at 3294, no baseline changenpm run test:updater— 95/95npm run typecheck:test— ratchet holds at baseline 406, no regressionsnpm run lint— cleanNo retries, sleeps, timeout bumps, or skips were added. Lockfile untouched.