From eba8431178e629b5ce5a2fe9908d3848486aeb1f Mon Sep 17 00:00:00 2001 From: ework-agent Date: Sat, 3 Oct 2026 18:33:45 +0800 Subject: [PATCH 1/2] fix(server): linger for peer close signal on post-response client closes (#1982) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit When the proxy initiates the client-side close right after a completed response (Connection: close), Node's destroySoon() destroys the socket in the same tick as end(), so the final bytes (incl. TLS close_notify) can still be unACKed when close(fd) runs and the kernel sends RST instead of FIN — observed downstream in #1982. Intercept destroy() on client-facing sockets (plain-TCP and MITM TLS legs alike): when the close follows a completed response and no error/drain state applies, resume() the recv side and hold the fd until the peer's close signal — TCP FIN or TLS close_notify, which can only arrive after our last byte was received+ACKed — then close cleanly. A bounded budget (BILI_POST_RESPONSE_LINGER_MS, default 5000, nginx lingering_time parity) caps silent-peer exposure; backstop closes are classified reason=linger-backstop (warn line). Structurally-destroyed raw legs of normally-closed MITM connections are classified paired-clean; genuinely abortive closes are elevated from debug to warn with an ABORTIVE marker so downstream RST reports reconcile against bili.log directly. Error-driven closes, pre-handshake teardown, the clientError drain (#1529) and the idle reaper are deliberately unchanged. --- CONFIGURATION.md | 1 + CONFIGURATION.zh-CN.md | 1 + README.md | 4 + README.zh-CN.md | 4 + src/mitm.ts | 7 + src/server.ts | 152 +++++++++- tests/fixtures/graceful-close-bili.mts | 47 ++++ tests/graceful-close.test.ts | 369 +++++++++++++++++++++++++ 8 files changed, 574 insertions(+), 11 deletions(-) create mode 100644 tests/fixtures/graceful-close-bili.mts create mode 100644 tests/graceful-close.test.ts diff --git a/CONFIGURATION.md b/CONFIGURATION.md index f64eb05267..244ca9fd05 100644 --- a/CONFIGURATION.md +++ b/CONFIGURATION.md @@ -821,6 +821,7 @@ Environment variables take precedence over the config file. They are useful for | `BILI_KEEP_ALIVE_TIMEOUT_MS` | Keep-alive timeout (ms) for client-facing sockets (default `5000`, matching Node's implicit default; #1452). Idle client connections are reaped with a clean FIN by Node's built-in reaper; this makes the previously implicit value explicit and configurable, and the reap is classified as `reason=idle-timeout` in the connection lifecycle ledger (debug log). Non-numeric or non-positive values fall back to `5000`. | | `BILI_EXPOSURE_LOG_INTERVAL_MS` | Interval (ms) for the periodic long-lived-exposure telemetry line `[exposure] uptime=… liveConns=… tcpHandles=… handles=… sessions=… blindTunnels=… inFlight=…` (default `3600000` = hourly; #1452). `0` disables. Exists so socket-handle leaks and zombie connections surface in long-running logs instead of post-mortem forensics. | | `BILI_CLIENT_ERROR_BACKSTOP_MS` | Terminal backstop (ms) for the clientError drain path (#1529, follow-up to #1452 item 1): after the 300ms drain bail has `end()`ed a connection whose peer never sends FIN, the socket would otherwise sit half-open on our side indefinitely — the keep-alive reaper keys off completed responses and Node enables no SO_KEEPALIVE by default. After this much post-bail silence the proxy destroys the socket instead, classified as `reason=clienterror-backstop` in the connection lifecycle ledger with its own warn marker. Safe against the #1452 RST signature: the socket has been `resume()`-drained for the whole window, so the destroy carries no unread residual bytes. Default `30000`; `0` restores the hold-until-peer-death status quo. Non-numeric or negative values fall back to `30000`. | +| `BILI_POST_RESPONSE_LINGER_MS` | Graceful-close budget (ms) after a completed response (#1982): when the proxy initiates closing a client connection right after the last response (e.g. `Connection: close`), it holds the socket open for up to this budget waiting for the peer's close signal — a TCP FIN or TLS close_notify, which can only arrive after our final byte was received+ACKed — then closes cleanly; if no signal arrives within the budget the socket is destroyed anyway and classified as `reason=linger-backstop` (warn line). This removes the abortive destroy that could race unACKed bytes into an RST on the client side (nginx `lingering_time` parity). Default `5000`; non-numeric or non-positive values fall back to `5000`. Error-driven closes, pre-handshake teardown, the clientError drain (#1529), and the idle reaper are intentionally unaffected. | | `ACP_SESSION_HEADER` | Conversation-id header name (default `x-acp-session`). | | `ACP_REASONING_KEEP` | Responses API only: set `none` to drop all reasoning items. Default routes reasoning through the compression pipeline so it is hidden automatically once its turn is summarized (prevents the unbounded accumulation that broke Codex's prompt-cache prefix). | | `ACP_RENDER_NONE` | Set to `1` to stop injecting per-message render tags (the `` `` `` markers carrying `mNNNNN` refs) into outgoing request history — applies to every wire format (OpenAI chat, Anthropic, Responses) and compact rebuilds (#933). Default is `text-only`: the model reads these refs to cite messages in `compress` calls, so only disable them once you've confirmed your workflow doesn't need ref-based compression (e.g. tag echoes leaking into client-visible output). Previously this variable was honored only on the Responses path and compact; #933 extended it to all paths. | diff --git a/CONFIGURATION.zh-CN.md b/CONFIGURATION.zh-CN.md index d916721e59..86f0cfe240 100644 --- a/CONFIGURATION.zh-CN.md +++ b/CONFIGURATION.zh-CN.md @@ -818,6 +818,7 @@ | `BILI_KEEP_ALIVE_TIMEOUT_MS` | 客户端侧套接字的 keep-alive 超时(毫秒,默认 `5000`,与 Node 隐式默认一致;#1452)。空闲客户端连接由 Node 内建回收器以干净 FIN 回收;此开关把原先隐式的值显式化并可配置,回收在连接生命周期台账(debug 日志)中分类为 `reason=idle-timeout`。非数字或非正值回退到 `5000`。 | | `BILI_EXPOSURE_LOG_INTERVAL_MS` | 长驻暴露遥测行 `[exposure] uptime=… liveConns=… tcpHandles=… handles=… sessions=… blindTunnels=… inFlight=…` 的周期(毫秒,默认 `3600000` 即每小时;#1452)。`0` 关闭。目的是让套接字句柄泄漏与僵尸连接在长期运行日志中现形,而不是靠事后取证。 | | `BILI_CLIENT_ERROR_BACKSTOP_MS` | clientError 排空路径的终局兜底(毫秒)(#1529,#1452 第 1 项后续):排空 bail(300ms)对连接调用 `end()` 后,若对端始终不发 FIN,该套接字否则会在我方无限期半开滞留——keep-alive 回收器以已完成响应为键,且 Node 默认不开 SO_KEEPALIVE。bail 后静默超过此值时,代理改为销毁该套接字,在连接生命周期台账中分类为 `reason=clienterror-backstop` 并带独立的 warn 标记。对 #1452 的 RST 签名安全:整个窗口内套接字一直处于 `resume()` 排空状态,销毁时不携带未读残留字节。默认 `30000`;`0` 恢复「持有直到对端死亡」的旧行为。非数字或负值回退到 `30000`。 | +| `BILI_POST_RESPONSE_LINGER_MS` | 响应完成后的优雅关闭预算(毫秒)(#1982):代理在最后一个响应结束后主动关闭客户端连接(如 `Connection: close`)时,最多为此预算时长持有该套接字,等待对端的关闭信号——TCP FIN 或 TLS close_notify(它只能在我方最后字节被接收并 ACK 之后到达)——然后干净地关闭;预算内无信号则照常销毁套接字,分类为 `reason=linger-backstop`(warn 行)。此举消除了可能在客户端侧把未 ACK 字节竞态成 RST 的激进销毁(与 nginx `lingering_time` 对齐)。默认 `5000`;非数字或非正值回退到 `5000`。错误驱动的关闭、握手前拆除、clientError 排空(#1529)与空闲回收器均刻意不受影响。 | | `ACP_SESSION_HEADER` | 会话 id 请求头名称(默认 `x-acp-session`)。 | | `ACP_REASONING_KEEP` | 仅 Responses API:设 `none` 丢弃全部 reasoning 项。默认让 reasoning 走压缩管道,其轮次被摘要后自动隐藏(避免无限累积破坏 Codex 的 prompt-cache 前缀)。 | | `ACP_RENDER_NONE` | 设为 `1` 停止向出站请求历史注入逐消息渲染标签(承载 `mNNNNN` ref 的 `` `` `` 标记)——适用于所有线格式(OpenAI chat、Anthropic、Responses)及 compact 重建(#933)。默认 `text-only`:模型靠这些 ref 在 `compress` 调用中引用消息,只有确认自己的工作流不需要基于 ref 的压缩(例如标签回声泄漏到客户端可见输出)后才应禁用。此前该变量仅在 Responses 路径与 compact 上生效;#933 扩展到了所有路径。 | diff --git a/README.md b/README.md index c70e4f563c..6b349efeab 100644 --- a/README.md +++ b/README.md @@ -375,6 +375,10 @@ By default the proxy binds `127.0.0.1` and only accepts loopback connections. To All logs tee to `~/.local/state/billion-context/bili.log` by default (XDG state dir) and still print to stderr. Override with `"logFile"` in config or `ACP_LOG_FILE` (`off` disables the file). Auto-rotates at 10 MB (`bili.log.old`). Per-request cache-hit stats log as `[acp-usage] round N input=X cached=Y (cache hit Z%)` so you can measure prefix-cache health directly from the log. +### Connection lifecycle tuning (#1982) + +Client-facing connections close gracefully after the final response: when the proxy initiates the close (`Connection: close`), it waits up to `BILI_POST_RESPONSE_LINGER_MS` (default `5000`) for the client's close signal before releasing the socket, so pooled clients see a clean EOF instead of a possible RST from racing bytes. Related knobs: `BILI_KEEP_ALIVE_TIMEOUT_MS` (idle-reap budget, default `5000`) and `BILI_CLIENT_ERROR_BACKSTOP_MS` (terminal backstop for the error-drain path, default `30000`) — full semantics in [CONFIGURATION.md](CONFIGURATION.md#environment-variables). + ### Self-update The proxy checks npm on startup and every 3 minutes; a newer version is installed in place and a notice is logged — **restart `bili` to pick it up**, unless you enable opt-in self-restart (`--auto-restart-on-update` flag, env `ACP_AUTO_RESTART_ON_UPDATE=1`, or `"autoRestartOnUpdate": true` in config — default OFF): with zero in-flight requests it verifies the new install, stops accepting connections, drains, spawns a replacement on the same port, and exits once it accepts connections (clients reconnect automatically; session state survives on disk). Safety gates: zero in-flight through the drain window, an install sanity check before re-exec, and a 10-minute cooldown marker so a flapping version can never loop-restart; any failure resumes the original listener and falls back to the plain reminder. diff --git a/README.zh-CN.md b/README.zh-CN.md index c22439ab00..a3cb3d084d 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -295,6 +295,10 @@ bili --no-auto-update # 本次启动禁用自动更新 所有日志**默认同时写入文件**:`~/.local/state/billion-context/bili.log`(XDG state 目录),同时仍打印到 stderr。覆盖用配置的 `"logFile"` 或 `ACP_LOG_FILE`(`off` 关闭文件)。超过 10 MB 自动轮转(`bili.log.old`)。每个请求的缓存命中统计以 `[acp-usage] round N input=X cached=Y (cache hit Z%)` 打印,可直接从日志衡量前缀缓存健康度。 +### 连接生命周期调优(#1982) + +客户端侧连接在最终响应结束后优雅关闭:代理主动发起关闭(`Connection: close`)时,最多等待 `BILI_POST_RESPONSE_LINGER_MS`(默认 `5000`)毫秒的对端关闭信号才释放套接字,池化客户端因此看到的是干净的 EOF,而非字节竞态可能产生的 RST。相关旋钮:`BILI_KEEP_ALIVE_TIMEOUT_MS`(空闲回收预算,默认 `5000`)与 `BILI_CLIENT_ERROR_BACKSTOP_MS`(错误排空路径终局兜底,默认 `30000`)——完整语义见 [CONFIGURATION.zh-CN.md](CONFIGURATION.zh-CN.md)。 + ### 自动更新 代理启动时和每 3 分钟检查 npm 是否有新版本。发现新版本就原位安装并打印通知 —— **重启 `bili` 才能生效**,除非启用可选自重启(`--auto-restart-on-update` 参数 / `ACP_AUTO_RESTART_ON_UPDATE=1` 环境变量 / 配置 `"autoRestartOnUpdate": true`,默认关闭):零在途请求时校验新安装、停止接收连接、排空、在同一端口拉起替代进程并在其开始接受连接后退出(客户端自动重连;会话状态在磁盘上保留)。安全门:排空窗口全程零在途、re-exec 前安装完整性检查、10 分钟冷却标记防止版本抖动循环重启;任何失败恢复原监听器并回落到普通提醒。运行进程落后于磁盘安装("stale")时,Web UI 显示横幅,`GET /__bili/status` 返回 `{version, diskVersion, stale, autoRestartOnUpdate, advisory, inFlight}` 供脚本使用(`advisory` 为生效中的严重缺陷公告或 `null`,见下节)。永久禁用:配置(`"autoUpdate": false`)或环境变量(`ACP_AUTO_UPDATE=0`)。 diff --git a/src/mitm.ts b/src/mitm.ts index bd055c2ca2..9c139ecfc6 100644 --- a/src/mitm.ts +++ b/src/mitm.ts @@ -23,6 +23,12 @@ export const DEFAULT_MITM_DOMAINS = [ * host the CONNECT tunnel targeted. */ export const MITM_UPSTREAM_KEY = "__biliMitmUpstream"; +/** Socket marker: the raw TCP socket backing a MITM TLS leg. The connection + * ledger (#1982) pairs the two records so the raw leg's close — which Node's + * TLSWrap.close() structurally turns into destroy() even on fully graceful + * TLS closes — can be classified with knowledge of how its TLS leg closed. */ +export const MITM_RAW_SOCKET_KEY = "__biliMitmRawSocket"; + // A client that rejects our MITM cert (root CA not trusted) can flood the log // with hundreds of identical handshake failures. Warn once, with the fix. let warnedCertRejected = false; @@ -274,6 +280,7 @@ function doMitm( // sees the decrypted request as a plain POST /api/anthropic/v1/messages — // with this marker it routes to https:// instead of the default. (tlsSocket as unknown as Record)[MITM_UPSTREAM_KEY] = `https://${host}`; + (tlsSocket as unknown as Record)[MITM_RAW_SOCKET_KEY] = clientSocket; // A TLS handshake error (client rejects our cert, abrupt disconnect, // reset) emits "error" on the TLSSocket. Without a listener Node treats // it as an uncaught exception and crashes the whole proxy. Destroy the diff --git a/src/server.ts b/src/server.ts index ec2d76ae1f..a308f23314 100644 --- a/src/server.ts +++ b/src/server.ts @@ -113,7 +113,7 @@ import { prefixAffinity, type AnonymousAffinity } from "./prefix-affinity.js"; import { maybeAdoptForkBlocks, maybeAdoptResume } from "./fork-adoption.js"; import { flushPrefixAffinity, hydratePrefixAffinity, scheduleAffinityPersist } from "./affinity-persist.js"; import { consumePluginRegisterFor, flushConversations, handlePluginCompact, handlePluginManifest, handlePluginRegister, handlePluginRuntimeInfo, handlePluginStatus, handlePluginTool, isPluginFoldCallId, loadConversations, pipePluginChatWithStrip, pipePluginJson, pipePluginResponsesWithStrip, pluginAgentHeader, pluginConversationHeader, pluginHeadersMatchModel, pluginReportedContextWindow, pluginReportedMaxOutput, pluginRequestAgentHeader, pluginRuntimeInfoFor, pluginRuntimeInfoForConversation, recordChainVerdict, recordPluginSession, rememberPluginMessages, resolveConversation, runtimeConversationId, takePendingPluginRegister } from "./plugin.js"; -import { setupMitm, readMitmUpstream, getBlindTunnelStats, liveBlindTunnels } from "./mitm.js"; +import { setupMitm, readMitmUpstream, getBlindTunnelStats, liveBlindTunnels, MITM_RAW_SOCKET_KEY } from "./mitm.js"; import { evaluateChain, extractChainCarriers, stampOutbound, stripEmbeddedChainCarriers } from "./chain-checkpoint.js"; import type { BiliMessage } from "acp-kernel/wire"; import { appendSystemText, BILI_PASSTHROUGH_HEADER, BILI_PLUGIN_BYPASS_HEADER, hardenOpenaiAssistantContent, isLoopbackAddress, inspectContextOverflow, reserveOutputHeadroom, resolveOutputHeadroomCap, shouldReserveOutputHeadroom, systemToUser, usageOutputTotal, usageTotals, type ContextOverflowInfo, type WireProtocol } from "./util.js"; @@ -367,6 +367,17 @@ export function googleModelFromPath(urlPath: string): string | undefined { } +// #1982: budget for the post-response close linger (see installPostResponseLinger). +// Default 5s mirrors nginx's lingering_time: a peer that FINs promptly costs one +// RTT of extra hold; a silent peer costs at most this window per fd. +// Env-overridable like BILI_MITM_HANDSHAKE_TIMEOUT_MS (tests + operator tuning); +// non-numeric or non-positive values fall back to the default. +const POST_RESPONSE_LINGER_MS_DEFAULT = 5_000; +function postResponseLingerMs(): number { + const v = Number.parseInt(process.env.BILI_POST_RESPONSE_LINGER_MS ?? "", 10); + return Number.isFinite(v) && v > 0 ? v : POST_RESPONSE_LINGER_MS_DEFAULT; +} + export async function startServer(opts: ProxyOptions): Promise { // Configure the tee logger (file + stderr) BEFORE any logging so the very // first line (persist status) lands in the file too. @@ -512,9 +523,92 @@ export async function startServer(opts: ProxyOptions): Promise { drainArmed: boolean; /** #1529: performance.now() when the post-bail backstop destroyed the socket (peer never FINned). */ backstopAt: number | null; + /** #1982: server-side end() was called on this socket (destroySoon stamp; distinguishes it from bare destroys). */ + ended: boolean; + /** #1982: TLS handshake completed (plain-TCP legs start true — nothing to wait for). */ + secured: boolean; + /** #1982: the other leg of a MITM connection (raw TCP ↔ terminated TLS); null elsewhere. */ + paired: ConnRecord | null; + /** #1982: performance.now() when the post-response linger backstop destroyed the socket (peer never FINned). */ + lingerBackstopAt: number | null; } const connRecords = new Map(); let connSeq = 0; + // #1982: turn the proxy-initiated post-response close from abortive into + // graceful. Node's destroySoon() — the Connection: close disposition in + // resOnFinish — calls end() then destroy() on the SAME TICK (for flushed + // responses writableFinished is already true, so destroy is not deferred), + // while the response tail / TLS close_notify may still be unACKed in + // flight. A kernel closing an fd with unacked send bytes (or unread recv + // residual) answers RST instead of FIN; pooled downstream clients surface + // it as ECONNRESET (#1982: 196 occurrences measured over two weeks on a + // Windows downstream, two of them crashing its process). + // Detection: end() is intercepted to stamp rec.ended — prefinish cannot be + // used (it fires async, AFTER the same-tick destroy). The peer's close + // signal (a TCP FIN, or a TLS close_notify that can only follow ours) is + // proof our last byte was received+ACKed — it cannot be sent before + // processing ours — so: intercept the destroy, resume() to drain the recv + // side, wait for that signal, then destroy; a silent peer costs at most + // one fd for the budget window (backstop). Plain-TCP and MITM TLS legs + // share the same destroySoon race and get the same treatment. + // Deliberately NOT applied to: error-driven destroys (the peer is already + // gone — nothing left to protect), pre-handshake teardown, sockets owned + // by the clientError drain (#1529), and bare destroys without end() (kat + // reaper on idle sockets — empty queues, already clean). + const installPostResponseLinger = (socket: net.Socket, rec: ConnRecord): void => { + let armed = false; + let backstopTimer: ReturnType | undefined; + const origDestroy = socket.destroy.bind(socket); + // Node's end() overloads don't compose under .call; flatten to one + // signature at this interception boundary (all three call shapes covered). + // bind() is load-bearing: called unbound, Socket.end reads + // this._writableState off undefined (crash inside destroySoon). + const origEnd = socket.end.bind(socket) as unknown as (chunk?: string | Uint8Array, enc?: BufferEncoding, cb?: () => void) => typeof socket; + const wrappedEnd = (chunk?: string | Uint8Array, encOrCb?: BufferEncoding | (() => void), cb?: () => void): typeof socket => { + rec.ended = true; + if (typeof encOrCb === "function") return origEnd(chunk, undefined, encOrCb); + return origEnd(chunk, encOrCb, cb); + }; + Object.defineProperty(socket, "end", { value: wrappedEnd, writable: true, configurable: true }); + const finishLinger = (why: "peer-fin" | "backstop" | "error"): void => { + if (!armed) return; + armed = false; + if (backstopTimer) clearTimeout(backstopTimer); + if (socket.destroyed) return; + if (why === "backstop") { + rec.lingerBackstopAt = performance.now(); + log("warn", `[conn#${rec.id}] ${rec.kind} linger backstop: no peer close signal ${postResponseLingerMs()}ms after post-response close — destroying (peer may see RST/ECONNRESET)`); + } else if (why === "peer-fin") { + log("debug", `[conn#${rec.id}] ${rec.kind} linger complete: peer close signal received — closing cleanly`); + } + origDestroy(); + }; + const wrappedDestroy = (err?: Error): net.Socket => { + if (armed) return socket; + if (err || rec.errored !== null || rec.drainArmed || !rec.secured || !rec.ended || rec.lastResponseEndAt === null) { + return origDestroy(err); + } + // Peer closed first: its FIN already proved delivery, so the destroy + // is clean — and waiting for an 'end' that already fired would only + // dead-lock into the backstop. + if (rec.peerFinAt !== null || socket.readableEnded) { + return origDestroy(); + } + armed = true; + log("info", `[conn#${rec.id}] ${rec.kind} post-response close: lingering for peer close signal (budget ${postResponseLingerMs()}ms)`); + // http leaves the socket paused between requests; without resume() + // the peer's EOF would never reach us and every linger would run + // out on the backstop. Draining also removes unread recv residual + // (the Linux RST trigger) across the whole window. + socket.resume(); + socket.once("end", () => finishLinger("peer-fin")); + socket.once("error", () => finishLinger("error")); + backstopTimer = setTimeout(() => finishLinger("backstop"), postResponseLingerMs()); + backstopTimer.unref?.(); + return socket; + }; + Object.defineProperty(socket, "destroy", { value: wrappedDestroy, writable: true, configurable: true }); + }; server.on("connection", (socket) => { const rec: ConnRecord = { id: ++connSeq, @@ -527,8 +621,24 @@ export async function startServer(opts: ProxyOptions): Promise { peerFinAt: null, drainArmed: false, backstopAt: null, + ended: false, + secured: !(socket instanceof tls.TLSSocket), + paired: null, + lingerBackstopAt: null, }; connRecords.set(socket, rec); + if (socket instanceof tls.TLSSocket) { + socket.once("secure", () => { rec.secured = true; }); + // doMitm stamps the raw TCP leg onto the TLS socket; pair the two + // ledger records (raw leg always arrives first — real accept) so + // each leg's close classifies with knowledge of the other. + const rawLeg = (socket as unknown as Record)[MITM_RAW_SOCKET_KEY] as net.Socket | undefined; + const rawRec = rawLeg ? connRecords.get(rawLeg) : undefined; + if (rawRec) { + rec.paired = rawRec; + rawRec.paired = rec; + } + } // prefinish fires when end() fully flushes — never on destroy(). That // makes it the reliable "server-initiated close" marker without patching // the socket object. performance.now() (µs) rather than Date.now(): @@ -556,19 +666,39 @@ export async function startServer(opts: ProxyOptions): Promise { // budget rather than the end marker; the requests>0 guard keeps // request-less closes out (the reaper only arms post-response). const idleForBudget = rec.requests > 0 && rec.lastResponseEndAt !== null && now - rec.lastResponseEndAt >= keepAliveTimeoutMs; + // #1982: the raw TCP leg of a MITM connection is structurally + // destroyed by Node's TLSWrap.close() even when the TLS leg closed + // fully gracefully — classify it by what its PAIRED tls leg did + // instead of reporting a false abortive "destroyed". + const pairedClean = rec.paired !== null && rec.paired.secured && rec.paired.errored === null && rec.paired.lingerBackstopAt === null; const reason = rec.backstopAt !== null ? "clienterror-backstop" - : rec.errored - ? `error(${rec.errored})` - : rec.peerFinAt !== null && (rec.serverEndAt === null || rec.peerFinAt <= rec.serverEndAt) - ? "peer-fin" - : idleForBudget - ? "idle-timeout" - : rec.serverEndAt !== null - ? "server-end" - : "destroyed"; - log("debug", `[conn#${rec.id}] ${rec.kind} closed reason=${reason} age=${now - rec.openedAt}ms reqs=${rec.requests}`); + : rec.lingerBackstopAt !== null + ? "linger-backstop" + : rec.errored + ? `error(${rec.errored})` + : rec.kind === "tcp" && rec.paired !== null + ? (pairedClean ? "paired-clean" : "destroyed") + : rec.peerFinAt !== null && (rec.serverEndAt === null || rec.peerFinAt <= rec.serverEndAt) + ? "peer-fin" + : idleForBudget + ? "idle-timeout" + : rec.serverEndAt !== null + ? "server-end" + : "destroyed"; + // #1982: a bare "destroyed" means nobody ended the socket and no + // other marker explains the close — the fd went away possibly with + // bytes still in flight, i.e. the peer may have seen RST/ECONNRESET. + // Elevate to warn (was debug) so a downstream "RST at T" report + // reconciles against this line directly (#1982 request 2); every + // intentional path carries its own dedicated marker above. + if (reason === "destroyed") { + log("warn", `[conn#${rec.id}] ${rec.kind} closed reason=destroyed age=${now - rec.openedAt}ms reqs=${rec.requests} [ABORTIVE — peer may see RST/ECONNRESET]`); + } else { + log("debug", `[conn#${rec.id}] ${rec.kind} closed reason=${reason} age=${now - rec.openedAt}ms reqs=${rec.requests}`); + } }); + installPostResponseLinger(socket, rec); }); // #1452: Node's default client-error disposition (no listener) writes a // bare `HTTP/1.1 400 Bad Request` / Connection: close reply and then diff --git a/tests/fixtures/graceful-close-bili.mts b/tests/fixtures/graceful-close-bili.mts new file mode 100644 index 0000000000..f8d434b5f3 --- /dev/null +++ b/tests/fixtures/graceful-close-bili.mts @@ -0,0 +1,47 @@ +// Child-process fixture for tests/graceful-close.test.ts (#1982): a full bili +// server with MITM enabled for "localhost". It runs in its OWN process because +// NODE_EXTRA_CA_CERTS — which makes the proxy's upstream fetch trust the +// mock-upstream cert signed by bili's test CA — is only read at process +// startup, so an in-process test can never make it take effect. Prints +// "READY " on stdout once listening; all bili logs go to stderr +// (logFile: "off"), which the parent test asserts against. +import { defaultConfig } from "acp-kernel"; +import { startServer } from "../../src/server.js"; +import type { ProxyOptions } from "../../src/config.js"; + +const tlsUp = Number(process.env.GC_TLS_UPSTREAM_PORT); +const plainUp = Number(process.env.GC_PLAIN_UPSTREAM_PORT); +if (!Number.isInteger(tlsUp) || !Number.isInteger(plainUp)) { + console.error("fixture requires GC_TLS_UPSTREAM_PORT and GC_PLAIN_UPSTREAM_PORT"); + process.exit(2); +} +const opts: ProxyOptions = { + port: 0, + host: "127.0.0.1", + upstream: "http://127.0.0.1", + routes: { + [`https://localhost:${tlsUp}`]: { models: { "gpt-test": { context: 400_000 } } }, + [`http://127.0.0.1:${plainUp}`]: { models: { "gpt-test": { context: 400_000 } } }, + }, + modelContextLimit: 400_000, + kernelConfig: defaultConfig(400_000), + compress: { injectTool: true, injectNudge: true }, + promptCache: { routing: "auto" }, + sessionHeader: "x-acp-session", + log: true, + logFile: "off", + debug: false, + passthrough: false, + autoUpdate: false, + compat: { roles: {} }, + passthroughSource: null, + autoRestartOnUpdate: false, + updateTag: "latest", + mitm: { enabled: true, domains: ["localhost"] }, +}; +startServer(opts).then((server) => { + server.on("listening", () => { + const port = (server.address() as { port: number }).port; + console.log(`READY ${port}`); + }); +}); diff --git a/tests/graceful-close.test.ts b/tests/graceful-close.test.ts new file mode 100644 index 0000000000..fdf8ce9734 --- /dev/null +++ b/tests/graceful-close.test.ts @@ -0,0 +1,369 @@ +import test from "node:test"; +import assert from "node:assert/strict"; +import fs from "node:fs"; +import os from "node:os"; +import path from "node:path"; +import tls from "node:tls"; +import net from "node:net"; +import http from "node:http"; +import https from "node:https"; +import { once } from "node:events"; +import { spawn, type ChildProcess } from "node:child_process"; +import { fileURLToPath } from "node:url"; +import { ensureRootCA, rootCaPath, mintHostCert, _resetForTest } from "../src/ca.js"; +import { rmrf } from "./tmp-rm.ts"; + +// #1982: the proxy-initiated post-response close must be graceful +// (close_notify + FIN) instead of an abortive destroy that races unACKed bytes +// into an RST. Asserted against a REAL bili server in a child process (see +// tests/fixtures/graceful-close-bili.mts for why), with the connection +// lifecycle ledger lines captured from its stderr. +// +// Single top-level test with explicit step sequencing: node:test's root +// test.after hook fires after the FIRST awaited top-level test completes, not +// after all of them — hook-based teardown would SIGKILL the child mid-suite. +// +// Child env pins: BILI_POST_RESPONSE_LINGER_MS=400 (backstop step stays fast; +// happy paths FIN in milliseconds so the shorter budget never bites them), +// BILI_KEEP_ALIVE_TIMEOUT_MS=300 (idle-reaper regression guard stays fast), +// BILI_MITM_HANDSHAKE_TIMEOUT_MS=600 (pre-handshake teardown step stays fast). + +let caDir = "", stateDir = "", homeDir = ""; +let caPath = "", rootPem = ""; +let leafCert = "", leafKey = ""; +let child: ChildProcess | undefined; +let monTimer: ReturnType | undefined; +let biliPort = 0, tlsUpPort = 0, plainUpPort = 0; +let tlsUp: tls.Server | undefined, plainUp: http.Server | undefined; +let stdoutBuf = ""; +const logLines: string[] = []; + +const FIXTURE = fileURLToPath(new URL("./fixtures/graceful-close-bili.mts", import.meta.url)); + +function sleep(ms: number): Promise { return new Promise((r) => setTimeout(r, ms)); } +// once(server, "close") rejects/hangs if the server is already closed — guard with a timer. +function closeServer(s: net.Server | undefined): Promise { + return new Promise((resolve) => { + if (!s) { resolve(); return; } + s.closeAllConnections?.(); + const t = setTimeout(resolve, 2_000); + s.on("close", () => { clearTimeout(t); resolve(); }); + // close() stops accepting — without it the server handle keeps the + // test process alive long after the suite ends. + s.close(); + }); +} +const PHASE_T0 = Date.now(); +function phase(msg: string): void { process.stderr.write(`[gc-test] t=${Date.now() - PHASE_T0}ms ${msg}\n`); } +async function onceCap(target: T, ev: string, ms: number): Promise { + let to: ReturnType; + const timeout = new Promise((_, reject) => { to = setTimeout(() => reject(new Error(`${ev} timeout ${ms}ms`)), ms); }); + try { return await Promise.race([once(target, ev), timeout]); } finally { clearTimeout(to!); } +} +function mark(): number { return logLines.length; } +function since(m: number): string[] { return logLines.slice(m); } +async function waitFor(pred: () => boolean, ms: number, what: string): Promise { + const t0 = Date.now(); + while (Date.now() - t0 < ms) { + if (pred()) return; + await sleep(25); + } + throw new Error(`timeout waiting for: ${what}\nrecent log:\n${logLines.slice(-30).join("\n")}`); +} + +/** Send a raw CONNECT and resolve once the status line arrives. */ +async function rawConnectStatus(port: number, target: string): Promise<{ statusLine: string; socket: net.Socket }> { + const socket = net.connect(port, "127.0.0.1"); + socket.write(`CONNECT ${target} HTTP/1.1\r\nHost: ${target}\r\n\r\n`); + let buf = ""; + await new Promise((resolve, reject) => { + const timer = setTimeout(() => reject(new Error("CONNECT timeout")), 5_000); + const onData = (c: Buffer): void => { + buf += c.toString("utf8"); + if (buf.includes("\r\n\r\n")) { + clearTimeout(timer); + socket.off("data", onData); + resolve(); + } + }; + socket.on("data", onData); + socket.once("error", reject); + }); + return { statusLine: buf.slice(0, buf.indexOf("\r\n")), socket }; +} + +interface MitmResult { body: string; sawReset: boolean; sock: tls.TLSSocket; } + +/** One model request through the MITM leg. allowHalfOpen keeps the client from + * auto-FINning when bili's close_notify arrives, so each step decides exactly + * WHEN (or whether) to send its FIN. */ +async function mitmRequest(tag: string, conn: string | undefined): Promise { + phase(`mitmRequest(${tag}): connect`); + const target = `localhost:${tlsUpPort}`; + const { statusLine, socket } = await rawConnectStatus(biliPort, target); + phase(`mitmRequest(${tag}): CONNECT ok`); + assert.match(statusLine, /^HTTP\/1\.1 200/, `CONNECT must be accepted: ${statusLine}`); + const sock = tls.connect({ socket, ca: rootPem, servername: "localhost", allowHalfOpen: true }); + await onceCap(sock, "secureConnect", 5_000); + phase(`mitmRequest(${tag}): TLS ok`); + let sawReset = false; + sock.once("error", (e: NodeJS.ErrnoException) => { if (e.code === "ECONNRESET") sawReset = true; }); + const body = JSON.stringify({ model: "gpt-test", messages: [{ role: "user", content: "ping" }] }); + const connHdr = conn ? `Connection: ${conn}\r\n` : ""; + sock.write( + `POST /v1/chat/completions HTTP/1.1\r\n` + + `Host: ${target}\r\n` + + `Content-Type: application/json\r\n` + + `x-acp-session: gc-${tag}\r\n` + + connHdr + + `Content-Length: ${Buffer.byteLength(body)}\r\n\r\n${body}`, + ); + let buf = ""; + sock.on("data", (c: Buffer) => { buf += c.toString("utf8"); }); + const full = (): string | undefined => { + const i = buf.indexOf("\r\n\r\n"); + if (i < 0) return undefined; + const head = buf.slice(0, i); + const mcl = /content-length:\s*(\d+)/i.exec(head); + if (mcl) { + return buf.length >= i + 4 + Number(mcl[1]) ? buf.slice(i + 4, i + 4 + Number(mcl[1])) : undefined; + } + return /\r\n0\r\n\r\n$/.test(buf) ? buf : undefined; + }; + const t0 = Date.now(); + while (full() === undefined) { + if (Date.now() - t0 > 8_000) throw new Error(`response incomplete:\n${buf.slice(0, 500)}`); + await sleep(10); + } + phase(`mitmRequest(${tag}): response complete`); + return { body: full()!, sawReset, sock }; +} + +interface PlainResult { body: string; sawReset: boolean; sock: net.Socket; } + +/** One model request through the PLAIN-TCP direct leg (/bili//... prefix). + * Raw socket instead of the http client: the agent would close a + * Connection: close socket on its own schedule, but each step must own + * exactly WHEN (or whether) the FIN goes out. halfOpen=true keeps the + * writable side open after receiving our FIN (Node's default duplex + * politely auto-ends it — see endWritableNT), which is what makes a + * genuinely silent peer possible. */ +async function plainRequest(tag: string, conn: string | undefined, halfOpen?: boolean): Promise { + phase(`plainRequest(${tag}): connect`); + const target = `127.0.0.1:${plainUpPort}`; + const sock = net.connect({ host: "127.0.0.1", port: biliPort, allowHalfOpen: halfOpen ?? false }); + await onceCap(sock, "connect", 5_000); + let sawReset = false; + sock.once("error", (e: NodeJS.ErrnoException) => { if (e.code === "ECONNRESET") sawReset = true; }); + const body = JSON.stringify({ model: "gpt-test", messages: [{ role: "user", content: "ping" }] }); + const connHdr = conn ? `Connection: ${conn}\r\n` : ""; + sock.write( + `POST /bili/http://${target}/v1/chat/completions HTTP/1.1\r\n` + + `Host: ${target}\r\n` + + `Content-Type: application/json\r\n` + + `x-acp-session: gc-${tag}\r\n` + + connHdr + + `Content-Length: ${Buffer.byteLength(body)}\r\n\r\n${body}`, + ); + let buf = ""; + sock.on("data", (c: Buffer) => { buf += c.toString("utf8"); }); + const full = (): string | undefined => { + const i = buf.indexOf("\r\n\r\n"); + if (i < 0) return undefined; + const head = buf.slice(0, i); + const mcl = /content-length:\s*(\d+)/i.exec(head); + if (mcl) { + return buf.length >= i + 4 + Number(mcl[1]) ? buf.slice(i + 4, i + 4 + Number(mcl[1])) : undefined; + } + return /\r\n0\r\n\r\n$/.test(buf) ? buf : undefined; + }; + const t0 = Date.now(); + while (full() === undefined) { + if (Date.now() - t0 > 8_000) throw new Error(`response incomplete:\n${buf.slice(0, 500)}`); + await sleep(10); + } + phase(`plainRequest(${tag}): response complete`); + return { body: full()!, sawReset, sock }; +} + +await test("#1982 graceful client-side close (MITM + plain-TCP legs)", async () => { + phase("setup: start"); + caDir = fs.mkdtempSync(path.join(os.tmpdir(), "bili-gc-ca-")); + stateDir = fs.mkdtempSync(path.join(os.tmpdir(), "bili-gc-state-")); + homeDir = fs.mkdtempSync(path.join(os.tmpdir(), "bili-gc-home-")); + // Generate the test CA under an isolated XDG_DATA_HOME BEFORE spawning the + // child, so the child's bili reuses it and NODE_EXTRA_CA_CERTS can point at it. + const prevXdg = process.env.XDG_DATA_HOME; + process.env.XDG_DATA_HOME = caDir; + try { + _resetForTest(); + ensureRootCA(); + caPath = rootCaPath(); + rootPem = fs.readFileSync(rootCaPath(), "utf8"); + const leaf = mintHostCert("localhost"); + leafCert = leaf.certPem; + leafKey = leaf.keyPem; + } finally { + if (prevXdg === undefined) delete process.env.XDG_DATA_HOME; + else process.env.XDG_DATA_HOME = prevXdg; + _resetForTest(); + } + const upstreamHandler = (req: http.IncomingMessage, res: http.ServerResponse) => { + req.resume(); + req.on("end", () => { + res.writeHead(200, { "content-type": "application/json" }); + res.end(JSON.stringify({ + id: "r1", + object: "chat.completion", + created: 1, + model: "gpt-test", + choices: [{ index: 0, message: { role: "assistant", content: "hi" }, finish_reason: "stop" }], + usage: { prompt_tokens: 10, completion_tokens: 1, total_tokens: 11 }, + })); + }); + }; + // https (not tls) createServer: the listener must be a request handler, + // not a secure-connection handler. + phase(`setup: ca ready (${caPath})`); + tlsUp = https.createServer({ key: leafKey, cert: leafCert }, upstreamHandler); + await new Promise((r) => tlsUp!.listen(0, "127.0.0.1", r)); + tlsUpPort = (tlsUp.address() as { port: number }).port; + plainUp = http.createServer(upstreamHandler); + await new Promise((r) => plainUp!.listen(0, "127.0.0.1", r)); + plainUpPort = (plainUp.address() as { port: number }).port; + + phase(`setup: upstreams up tls=${tlsUpPort} plain=${plainUpPort}, spawning child`); + child = spawn(process.execPath, ["--import", "tsx", FIXTURE], { + env: { + ...process.env, + XDG_DATA_HOME: caDir, + XDG_STATE_HOME: stateDir, + HOME: homeDir, + NODE_EXTRA_CA_CERTS: caPath, + GC_TLS_UPSTREAM_PORT: String(tlsUpPort), + GC_PLAIN_UPSTREAM_PORT: String(plainUpPort), + BILI_POST_RESPONSE_LINGER_MS: "400", + BILI_KEEP_ALIVE_TIMEOUT_MS: "300", + BILI_MITM_HANDSHAKE_TIMEOUT_MS: "600", + BILI_ADVISORY_CHECK: "0", + }, + stdio: ["ignore", "pipe", "pipe"], + }); + child.stdout.on("data", (d: Buffer) => { stdoutBuf += d.toString("utf8"); }); + child.stderr.on("data", (d: Buffer) => { + const text = d.toString("utf8"); + for (const l of text.split("\n")) if (l.trim()) logLines.push(l); + process.stderr.write(text); + }); + child.on("exit", (code, sig) => phase(`child exited code=${code} sig=${sig}`)); + const readyT0 = Date.now(); + let m: RegExpExecArray | null; + while (!(m = /^READY (\d+)$/m.exec(stdoutBuf))) { + if (Date.now() - readyT0 > 30_000) throw new Error(`child did not report READY\n${logLines.slice(-30).join("\n")}`); + await sleep(50); + } + biliPort = Number(m[1]); + phase(`setup: child READY on ${biliPort}`); + // Liveness watchdog: silent while healthy, dumps state only when the child + // stops accepting — the failure mode this suite guards against. + monTimer = setInterval(() => { + const c = child; + if (!c) return; + const probe = net.connect(biliPort, "127.0.0.1"); + const done = (res: string): void => { + try { probe.destroy(); } catch { /* already gone */ } + if (res !== "ok") phase(`[mon] exitCode=${c.exitCode} killed=${c.killed} port=${res}`); + }; + const pt = setTimeout(() => done("timeout"), 600); + probe.once("connect", () => { clearTimeout(pt); done("ok"); }); + probe.once("error", (e: NodeJS.ErrnoException) => { clearTimeout(pt); done(e.code ?? "err"); }); + }, 750); + monTimer.unref?.(); + + try { + phase("step1(happy): start"); + { + const m0 = mark(); + const { body, sawReset, sock } = await mitmRequest("happy", "close"); + assert.match(body, /chat\.completion/, "mock upstream response must pass through unchanged in shape"); + // The client decides to FIN now (a pooled client tearing down the connection). + // allowHalfOpen suppresses the automatic 'close' after EOF; and on loopback + // bili's close_notify can land BEFORE any 'end' listener is registered, so + // prove the clean shutdown via the readableEnded state — race-free. + sock.end(); + await waitFor(() => sock.readableEnded, 5_000, "client EOF (readableEnded)"); + await waitFor(() => since(m0).some((l) => /post-response close: lingering for peer close signal/.test(l)), 3_000, "linger arm line"); + await waitFor(() => since(m0).some((l) => /linger complete: peer close signal received/.test(l)), 3_000, "linger completion line"); + await waitFor(() => since(m0).some((l) => /\] tls closed reason=(server-end|idle-timeout)/.test(l)), 3_000, "tls leg close line"); + await waitFor(() => since(m0).some((l) => /\] tcp closed reason=paired-clean/.test(l)), 3_000, "raw leg paired-clean line"); + assert.equal(sawReset, false, "client must not observe ECONNRESET on the graceful close"); + assert.ok(!since(m0).some((l) => /\[ABORTIVE/.test(l)), `no abortive close expected in this window:\n${since(m0).filter((l) => /\[ABORTIVE/.test(l)).join("\n")}`); + try { sock.destroy(); } catch { /* already gone */ } + } + + phase("step2(backstop): start"); + { + const m0 = mark(); + const { body, sock } = await plainRequest("backstop", "close", true); + assert.match(body, /chat\.completion/); + // Hold the socket open without FINning (a stuck/zombie client): + // allowHalfOpen suppresses Node's polite auto-FIN, so the budget + // must run out. A well-behaved TLS stack cannot stay silent either + // — RFC 8446 §6.1 makes the close_notify reply mandatory (that + // auto-reply is what completes step 1's linger instantly), so the + // TLS-leg backstop is unreachable by design; the mechanism itself + // is leg-agnostic. + await waitFor(() => since(m0).some((l) => /linger backstop: no peer close signal/.test(l)), 3_000, "linger backstop warn"); + await waitFor(() => since(m0).some((l) => /\] tcp closed reason=linger-backstop/.test(l)), 3_000, "tcp leg linger-backstop close line"); + try { sock.destroy(); } catch { /* already gone */ } + } + + phase("step3(idle-reaper): start"); + { + const m0 = mark(); + const { body, sock } = await mitmRequest("idle", undefined); + assert.match(body, /chat\.completion/); + // Hold open past kat (300ms pin + Node's 1s keepAliveTimeoutBuffer) without FINning. + await waitFor(() => since(m0).some((l) => /\] tls closed reason=idle-timeout/.test(l)), 4_000, "idle-timeout close line"); + assert.ok(!since(m0).some((l) => /post-response close: lingering/.test(l)), "kat reaper (bare destroy on empty idle queues) must stay immediate, not linger"); + try { sock.destroy(); } catch { /* already gone */ } + } + + phase("step4(pre-handshake): start"); + { + const m0 = mark(); + const { statusLine, socket } = await rawConnectStatus(biliPort, `localhost:${tlsUpPort}`); + assert.match(statusLine, /^HTTP\/1\.1 200/); + // Never send a ClientHello — the slowloris pattern; the handshake timeout (600ms pin) destroys both legs. + await waitFor(() => since(m0).some((l) => /TLS handshake timeout/.test(l)), 3_000, "handshake timeout line"); + await waitFor(() => since(m0).some((l) => /\[ABORTIVE — peer may see RST\/ECONNRESET\]/.test(l)), 3_000, "abortive warn line"); + assert.ok(!since(m0).some((l) => /post-response close: lingering/.test(l)), "pre-handshake teardown must not linger"); + try { socket.destroy(); } catch { /* already gone */ } + } + + phase("step5(plain-tcp): start"); + { + const m0 = mark(); + const { body, sawReset, sock } = await plainRequest("direct", "close"); + assert.match(body, /chat\.completion/, "mock upstream response must pass through unchanged in shape"); + // Pooled-client teardown: explicit FIN now. + sock.end(); + await waitFor(() => since(m0).some((l) => /\] tcp post-response close: lingering for peer close signal/.test(l)), 3_000, "tcp linger arm line"); + await waitFor(() => since(m0).some((l) => /\] tcp linger complete: peer close signal received/.test(l)), 3_000, "tcp linger completion line"); + await waitFor(() => since(m0).some((l) => /\] tcp closed reason=(server-end|idle-timeout)/.test(l)), 3_000, "tcp leg close line"); + assert.equal(sawReset, false, "client must not observe ECONNRESET on the graceful close"); + try { sock.destroy(); } catch { /* already gone */ } + } + } finally { + phase("teardown: start"); + if (monTimer) { clearInterval(monTimer); monTimer = undefined; } + if (child && child.exitCode === null && !child.killed) { + child.kill("SIGKILL"); + await onceCap(child, "exit", 5_000).catch(() => {}); + } + await Promise.all([closeServer(tlsUp), closeServer(plainUp)]); + for (const d of [caDir, stateDir, homeDir]) { + if (d) { try { rmrf(d); } catch { /* best-effort */ } } + } + } +}); From 86b8fe7e697c7f5ddfb6418d243fc740c3f9846b Mon Sep 17 00:00:00 2001 From: ework-agent Date: Sat, 3 Oct 2026 18:34:01 +0800 Subject: [PATCH 2/2] fix(mitm): preserve non-default CONNECT port in upstream key doMitm stamped the upstream key as https://, dropping the CONNECT port, so every non-443 MITM tunnel was misrouted to :443 (connect-refused against local gateways). Discovered while building the #1982 graceful-close test suite, which exercises a non-default-port mock upstream through the MITM leg. The 443 path stays byte-identical. --- src/mitm.ts | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/src/mitm.ts b/src/mitm.ts index 9c139ecfc6..16fd96c503 100644 --- a/src/mitm.ts +++ b/src/mitm.ts @@ -278,8 +278,10 @@ function doMitm( } // Mark the socket so resolveUpstream() can recover the real origin. handle() // sees the decrypted request as a plain POST /api/anthropic/v1/messages — - // with this marker it routes to https:// instead of the default. - (tlsSocket as unknown as Record)[MITM_UPSTREAM_KEY] = `https://${host}`; + // with this marker it routes to https://[:port] instead of the default. + // The port must be preserved: dropping it silently misroutes every + // non-443 CONNECT to :443 (connect-refused for local gateways). + (tlsSocket as unknown as Record)[MITM_UPSTREAM_KEY] = `https://${host}${port === 443 ? "" : `:${port}`}`; (tlsSocket as unknown as Record)[MITM_RAW_SOCKET_KEY] = clientSocket; // A TLS handshake error (client rejects our cert, abrupt disconnect, // reset) emits "error" on the TLSSocket. Without a listener Node treats