Skip to content

Orphaned server spins on CPU and writes unbounded logs: uncaughtException handler re-enters via console.error on a broken pipe #7

Description

@sadraphael

Summary

When the parent process dies, an orphaned server enters a self-sustaining exception loop: the uncaughtException handler installed by the esbuild banner writes to the same broken pipe that caused the exception. Each write raises a new async EPIPE, which becomes a new uncaught exception, which calls the handler again.

The process never exits, burns ~33% of a CPU core indefinitely, and appends to its log file at roughly 27,000 iterations per second.

Impact (measured on one machine)

  • 12 orphaned server processes accumulated over several days, each holding ~150 MB RSS and burning 31–37% of a core — about 4 of 20 cores permanently busy, enough to keep a laptop's fans running continuously.
  • 90.4 GB of log files written by the loop: ~/.pii_shield/audit/ner_init.log at 34.3 GB and %TEMP%\piish-banner-debug.log at 56.1 GB. This took the disk from 31% free to 12.5% free.
  • Every log line is the same stack trace:
[UNCAUGHT] Error: EPIPE: broken pipe, write
    at Socket._write (node:internal/net:75:18)
    at writeOrBuffer (node:internal/streams/writable:570:12)
    at _write (node:internal/streams/writable:499:10)
    at Writable.write (node:internal/streams/writable:508:10)
    at __DBG (file:///.../server.bundle.mjs:31:24)
    at process.<anonymous> (file:///.../server.bundle.mjs:49:44)
    at process.emit (node:events:521:24)
    at process._fatalException (node:internal/process/execution:159:25)

The frames tell the whole story: process._fatalException → the banner's uncaughtException handler → a write to the broken stream → and back around.

Root cause

In nodejs-v2/esbuild.config.mjs:

// line 57
'function __earlyLog(msg) {',
'  try {',
'    ... appendFileSync(... "ner_init.log" ...)',
'  } catch (_) {}',
'  try { console.error(msg); } catch (_) {}',   // line 63
'}',
'process.on("uncaughtException", (err) => { __earlyLog("[UNCAUGHT] " + (err && err.stack || err)); });',  // line 65

Two things combine:

  1. The try/catch on line 63 does not help. EPIPE on a broken pipe is delivered as an asynchronous error event on the stream, not as a synchronous throw. The catch never sees it.
  2. Neither process.stdout nor process.stderr has an error listener. With no listener, Node routes the stream error to process._fatalException, which fires uncaughtException — whose handler writes to that same stream. The loop is closed.

Because each iteration is a fresh tick rather than a re-entrant throw, Node's usual "exception inside the exception handler" bail-out never triggers, and the loop runs until the process is killed externally.

Nothing tells the server to shut down when its client disappears, so it also survives indefinitely: SIGTERM/SIGINT are not delivered on Windows when a parent dies, and StdioServerTransport.start() registers only data and error on stdin — no end/close.

Reproduction of the mechanism

The full end-state is timing-dependent, but the mechanism reproduces deterministically in isolation. A child that mimics the banner (an uncaughtException handler that writes to stderr), with its read end destroyed by the parent:

Variant Uncaught exceptions in 2.5 s Errors delivered to listener
No error listener on stdout/stderr 67,589 0
With error listener 0 134

And on the real server, same scenario, same timings:

Variant Result
Stock build wrote 6.2 MB of [UNCAUGHT] EPIPE in a few seconds
With the fix below exited cleanly, code 0, ~1.2 s after the pipe broke

Suggested fix

Attaching an error listener to the output streams is enough to break the loop, since the error is then delivered to the listener instead of process._fatalException. Exiting on a dead pipe also stops the orphan from lingering — if stdout is gone, the server has no one left to answer:

'var __onStreamError = function (e) {',
'  var code = (e && e.code) || "";',
'  if (code === "EPIPE" || code === "ERR_STREAM_DESTROYED" || code === "ECONNRESET") {',
'    try { process.exit(0); } catch (_) {}',
'  }',
'};',
'try { process.stdout.on("error", __onStreamError); } catch (_) {}',
'try { process.stderr.on("error", __onStreamError); } catch (_) {}',
// clean shutdown when the client closes the channel gracefully:
'try { process.stdin.on("end", function () { try { process.exit(0); } catch (_) {} }); } catch (_) {}',
'try { process.stdin.on("close", function () { try { process.exit(0); } catch (_) {} }); } catch (_) {}',

Applying this to the installed bundle stopped the loop; the server starts normally afterwards and still reports all 17 tools.

Two things you may want to consider alongside it:

  • A size cap on __earlyLog. Even without this loop, an unbounded append-only log in a user data directory can grow without limit.
  • A per-process beacon path. All instances write the same server_status.json via tmp.<pid> + rename, so with several instances alive the status file reflects whichever wrote last rather than any particular server. Stale server_status.json.tmp.<pid> files from instances killed mid-write were still present from previous months.

Environment

  • PII Shield v2.2.0 (as reported in startup.log)
  • Node v24.15.0, Windows 11 Pro 26200, x64
  • Launched as an MCP stdio server by Claude Desktop / Claude Code

One note in case it matters: the installed bundle self-reports v2.2.0 but contains __DBG instrumentation that writes a second log to %TEMP%\piish-banner-debug.log, which I don't see in the v2.2.0 tag of esbuild.config.mjs. That extra instrumentation is what produced the larger of the two files. The defect itself is identical in the tagged source — a stock build would simply write to one destination instead of two.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions