Skip to content
Closed
1 change: 1 addition & 0 deletions CONFIGURATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -822,6 +822,7 @@ Environment variables take precedence over the config file. They are useful for
| `BILI_EXPOSURE_LOG_INTERVAL_MS` | Interval (ms) for the periodic long-lived-exposure telemetry line `[exposure] uptime=… liveConns=… tcpHandles=… handles=… sessions=… blindTunnels=… inFlight=…` (default `3600000` = hourly; #1452). `0` disables. Exists so socket-handle leaks and zombie connections surface in long-running logs instead of post-mortem forensics. |
| `BILI_CLIENT_ERROR_BACKSTOP_MS` | Terminal backstop (ms) for the clientError drain path (#1529, follow-up to #1452 item 1): after the 300ms drain bail has `end()`ed a connection whose peer never sends FIN, the socket would otherwise sit half-open on our side indefinitely — the keep-alive reaper keys off completed responses and Node enables no SO_KEEPALIVE by default. After this much post-bail silence the proxy destroys the socket instead, classified as `reason=clienterror-backstop` in the connection lifecycle ledger with its own warn marker. Safe against the #1452 RST signature: the socket has been `resume()`-drained for the whole window, so the destroy carries no unread residual bytes. Default `30000`; `0` restores the hold-until-peer-death status quo. Non-numeric or negative values fall back to `30000`. |
| `BILI_POST_RESPONSE_LINGER_MS` | Graceful-close budget (ms) after a completed response (#1982): when the proxy initiates closing a client connection right after the last response (e.g. `Connection: close`), it holds the socket open for up to this budget waiting for the peer's close signal — a TCP FIN or TLS close_notify, which can only arrive after our final byte was received+ACKed — then closes cleanly; if no signal arrives within the budget the socket is destroyed anyway and classified as `reason=linger-backstop` (warn line). This removes the abortive destroy that could race unACKed bytes into an RST on the client side (nginx `lingering_time` parity). Default `5000`; non-numeric or non-positive values fall back to `5000`. Error-driven closes, pre-handshake teardown, the clientError drain (#1529), and the idle reaper are intentionally unaffected. |
| `BILI_WS_IDLE_WARN_SECONDS` | Idle-retention warn threshold (seconds) for the WebSocket bridge (#1926 option (c)): an idle client peer pins one live upstream connection plus both checkpoints with no reclamation until it closes; once a connection's client-side silence crosses this threshold, ONE `warn` line per idle episode is logged (tagged `(#1926)`, reset by any client activity) and per-connection `idleMs` / `retainedBytes` / `inFlight` surface under `wsBridge` in `GET /__bili/stats`. Pure observability — nothing is closed. Default `1800` (30 min); `0` disables the warn scan (stats stay live); non-numeric or negative values fall back to `1800`. Scan cadence is fixed at 60 s. |
| `ACP_SESSION_HEADER` | Conversation-id header name (default `x-acp-session`). |
| `ACP_REASONING_KEEP` | Responses API only: set `none` to drop all reasoning items. Default routes reasoning through the compression pipeline so it is hidden automatically once its turn is summarized (prevents the unbounded accumulation that broke Codex's prompt-cache prefix). |
| `ACP_RENDER_NONE` | Set to `1` to stop injecting per-message render tags (the `` `<acp>` `` markers carrying `mNNNNN` refs) into outgoing request history — applies to every wire format (OpenAI chat, Anthropic, Responses) and compact rebuilds (#933). Default is `text-only`: the model reads these refs to cite messages in `compress` calls, so only disable them once you've confirmed your workflow doesn't need ref-based compression (e.g. tag echoes leaking into client-visible output). Previously this variable was honored only on the Responses path and compact; #933 extended it to all paths. |
Expand Down
1 change: 1 addition & 0 deletions CONFIGURATION.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -819,6 +819,7 @@
| `BILI_EXPOSURE_LOG_INTERVAL_MS` | 长驻暴露遥测行 `[exposure] uptime=… liveConns=… tcpHandles=… handles=… sessions=… blindTunnels=… inFlight=…` 的周期(毫秒,默认 `3600000` 即每小时;#1452)。`0` 关闭。目的是让套接字句柄泄漏与僵尸连接在长期运行日志中现形,而不是靠事后取证。 |
| `BILI_CLIENT_ERROR_BACKSTOP_MS` | clientError 排空路径的终局兜底(毫秒)(#1529,#1452 第 1 项后续):排空 bail(300ms)对连接调用 `end()` 后,若对端始终不发 FIN,该套接字否则会在我方无限期半开滞留——keep-alive 回收器以已完成响应为键,且 Node 默认不开 SO_KEEPALIVE。bail 后静默超过此值时,代理改为销毁该套接字,在连接生命周期台账中分类为 `reason=clienterror-backstop` 并带独立的 warn 标记。对 #1452 的 RST 签名安全:整个窗口内套接字一直处于 `resume()` 排空状态,销毁时不携带未读残留字节。默认 `30000`;`0` 恢复「持有直到对端死亡」的旧行为。非数字或负值回退到 `30000`。 |
| `BILI_POST_RESPONSE_LINGER_MS` | 响应完成后的优雅关闭预算(毫秒)(#1982):代理在最后一个响应结束后主动关闭客户端连接(如 `Connection: close`)时,最多为此预算时长持有该套接字,等待对端的关闭信号——TCP FIN 或 TLS close_notify(它只能在我方最后字节被接收并 ACK 之后到达)——然后干净地关闭;预算内无信号则照常销毁套接字,分类为 `reason=linger-backstop`(warn 行)。此举消除了可能在客户端侧把未 ACK 字节竞态成 RST 的激进销毁(与 nginx `lingering_time` 对齐)。默认 `5000`;非数字或非正值回退到 `5000`。错误驱动的关闭、握手前拆除、clientError 排空(#1529)与空闲回收器均刻意不受影响。 |
| `BILI_WS_IDLE_WARN_SECONDS` | WebSocket 桥接的空闲保留告警阈值(秒)(#1926 选项 (c)):一个空闲的客户端对端会固定占用一条活跃的上游连接及两个检查点,且在其关闭前不会被回收;当某条连接的客户端侧静默时长超过该阈值时,每个空闲事件仅记录一行 `warn` 日志(标记 `(#1926)`,任何客户端活动即重置),同时每连接的 `idleMs` / `retainedBytes` / `inFlight` 指标会在 `GET /__bili/stats` 的 `wsBridge` 下暴露。纯可观测性——不会关闭任何连接。默认值 `1800`(30 分钟);设为 `0` 则禁用告警扫描(stats 仍保持实时);非数字或负值回退为 `1800`。扫描周期固定为 60 秒。 |
| `ACP_SESSION_HEADER` | 会话 id 请求头名称(默认 `x-acp-session`)。 |
| `ACP_REASONING_KEEP` | 仅 Responses API:设 `none` 丢弃全部 reasoning 项。默认让 reasoning 走压缩管道,其轮次被摘要后自动隐藏(避免无限累积破坏 Codex 的 prompt-cache 前缀)。 |
| `ACP_RENDER_NONE` | 设为 `1` 停止向出站请求历史注入逐消息渲染标签(承载 `mNNNNN` ref 的 `` `` `` 标记)——适用于所有线格式(OpenAI chat、Anthropic、Responses)及 compact 重建(#933)。默认 `text-only`:模型靠这些 ref 在 `compress` 调用中引用消息,只有确认自己的工作流不需要基于 ref 的压缩(例如标签回声泄漏到客户端可见输出)后才应禁用。此前该变量仅在 Responses 路径与 compact 上生效;#933 扩展到了所有路径。 |
Expand Down
15 changes: 15 additions & 0 deletions docs/opencode-responses-websocket.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,21 @@ Unknown protocols cannot be compressed (folding requires knowing where history
lives in the wire format); they stay on the #1472 transparent passthrough lane
and remain untouched by this bridge.

## Idle-socket observability (#1926, option (c))

Idle client peers each pin one live upstream connection plus both checkpoints,
with no reclamation until the client closes (idle-close (a)/(b) awaits
verified retry-full host recovery — see #1926). Until then the lane surfaces
the retention:

- `GET /__bili/stats` gains a `wsBridge` object: per live connection — codec,
connection id, `idleMs`, `retainedBytes` (client + upstream checkpoints),
`inFlight`.
- A scan (default every 60s) logs one `warn` per idle episode per connection
once idle age crosses the threshold (default 30min;
`BILI_WS_IDLE_WARN_SECONDS` overrides, `0` disables), tagged `(#1926)` and
reset by client activity. Pure observability — nothing is closed.

## Verification

Use random loopback ports and fake Responses WebSocket upstreams, with no real
Expand Down
16 changes: 16 additions & 0 deletions src/responses-ws.ts
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,13 @@ export class ResponsesWsHistory {
clear(): void {
this.checkpoint = undefined;
}

/** #1926 observability: bytes retained by the live checkpoint. */
bytes(): number {
const checkpoint = this.checkpoint;
if (!checkpoint) return 0;
return Buffer.byteLength(JSON.stringify(checkpoint.request)) + Buffer.byteLength(JSON.stringify(checkpoint.output));
}
}

// The pipeline emits JSON bodies as strings (prepare* re-serialization) or
Expand All @@ -135,6 +142,11 @@ export class ResponsesWsUpstream {
this.log("debug", `upstream checkpoint reset reason=${reason}`);
}

/** #1926 observability: bytes retained by the upstream-leg checkpoint. */
retainedBytes(): number {
return this.history.bytes();
}

close(reason = "transport-close"): void {
const socket = this.socket;
this.socket = undefined;
Expand Down Expand Up @@ -393,6 +405,10 @@ class ResponsesWsSession implements WsBridgeSession {
context.log("info", "Responses socket connected (ACP request pipeline)");
}

stats(): { retainedBytes: number; inFlight: boolean } {
return { retainedBytes: this.history.bytes() + this.transport.retainedBytes(), inFlight: this.busy };
}

onMessage(data: Buffer, binary: boolean): void {
const peer = this.context.peer;
if (binary) { peer.close(1003, "Responses requires text frames"); return; }
Expand Down
15 changes: 11 additions & 4 deletions src/server.ts
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,12 @@ import { applyOutputSteering, applyOutputSteeringJson } from "./output-steering.
import { bodyDumpEnabled, getUnrecognizedPathStats, isModelDiscoveryPath, logDumpFailure, logUnrecognizedPath } from "./server/observability.js";
import { BILI_HOP_HEADER, anthropicBetaContextWindow, capRegistryWindowByStandard, expandedContextSuffixWindow, LAUNCHER_MODEL_WINDOWS, LAUNCHER_MODEL_MAX_OUTPUTS, launcherContextWindow, launcherMaxOutput, parseLauncherModelWindows, windowSourceLogged } from "./server/context-window.js";
import { buildForwardHeaders, connectionNamedHeaders, NO_IDENTITY_MESSAGE, RESPONSE_ONLY_STRIP_HEADERS, safeSessionId, UPSTREAM_HOP_HEADERS } from "./server/headers.js";
import { installWebSocketBridge } from "./ws-bridge.js";
import { installWebSocketBridge, type WsBridgeStats } from "./ws-bridge.js";

// #1926 observability: the WS bridge handle's stats accessor, published once
// the server installs the bridge so `GET /__bili/stats` can report live WS
// connections, idle age, and retained checkpoint bytes per connection.
let wsBridgeStats: (() => WsBridgeStats) | undefined;
import { codexResponsesCodec, responsesCodec } from "./responses-ws.js";
import { currentFetchTransport } from "./fetch-transport.js";
import { isSideRequest, outputBudgetField, restoreOutputBudget, SIDE_REQUEST_MAX_TOKENS, sideRequestGuard, stripLeakedBiliTools } from "./server/side-request.js";
Expand Down Expand Up @@ -468,9 +473,11 @@ export async function startServer(opts: ProxyOptions): Promise<http.Server> {
};
const server = http.createServer(dispatch);
// Generic WebSocket bridge: protocol codecs claim upgrades here (#1467
// phase-2 shell); the Responses codec is the first (and currently only)
// entry. Unclaimed upgrades still fall through to the 426 contract below.
// phase-2 shell); the Responses codecs (opencode plugin lane + codex
// prefix lane) are the current entries. Unclaimed upgrades still fall
// through to the 426 contract below.
const wsUpgrade = installWebSocketBridge(server, dispatch, log, [responsesCodec, codexResponsesCodec]);
wsBridgeStats = wsUpgrade.stats;
// Unclaimed upgrades retain the immediate HTTP fallback contract.
// An explicit 'upgrade' listener is
// required: without one Node's behavior is version-dependent (some
Expand Down Expand Up @@ -7302,7 +7309,7 @@ function sendStats(res: http.ServerResponse): void {
};
});
res.writeHead(200, { "content-type": "application/json" });
res.end(JSON.stringify({ sessions, blindTunnels: getBlindTunnelStats(), unrecognizedPaths: getUnrecognizedPathStats(), conflicts: summarizeConflicts(all) }, null, 2));
res.end(JSON.stringify({ sessions, blindTunnels: getBlindTunnelStats(), unrecognizedPaths: getUnrecognizedPathStats(), conflicts: summarizeConflicts(all), wsBridge: wsBridgeStats?.() ?? { connections: [] } }, null, 2));
}

/** Stale-install state for the web UI badge (#811): whether the on-disk
Expand Down
83 changes: 77 additions & 6 deletions src/ws-bridge.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,12 @@ export interface WsBridgeSession {
onClose(code: number): void;
/** The server is shutting down; fail in-flight exchanges and drop upstream transports. */
shutdown(reason: string): void;
/**
* Optional resource accounting for the #1926 observability surface
* (`/__bili/stats` + the idle-retention warn line). Implementations report
* the bytes their checkpoints retain and whether an exchange is in flight.
*/
stats?(): { retainedBytes?: number; inFlight?: boolean };
}

export interface WsBridgeContext {
Expand Down Expand Up @@ -76,29 +82,82 @@ export interface WsBridgeCodec {
createSession(context: WsBridgeContext): WsBridgeSession;
}

export interface WsBridgeStats {
connections: { codec: string; connection: number; idleMs: number; retainedBytes?: number; inFlight?: boolean }[];
}

export interface WsBridgeHandle {
/** The upgrade handler; wire into the http server's "upgrade" event. */
(req: http.IncomingMessage, socket: Duplex, head: Buffer): boolean;
/** #1926 observability: live connections, idle age, retained bytes. */
stats(): WsBridgeStats;
}

export interface WsBridgeOptions {
/** Clock injection for tests. */
now?: () => number;
/** Idle-scan cadence (ms); default 60_000. */
scanMs?: number;
/** Idle threshold (ms) before the retention warn line fires; default 30 min, 0 disables. */
idleWarnMs?: number;
}

interface BridgeEntry {
session: WsBridgeSession;
peer: WebSocket;
codec: string;
connection: number;
label: string;
lastActivityAt: number;
idleWarned: boolean;
}

export function installWebSocketBridge(
server: http.Server,
dispatch: (req: http.IncomingMessage, res: http.ServerResponse) => Promise<void>,
log: (level: string, message: string) => void,
codecs: readonly WsBridgeCodec[],
): (req: http.IncomingMessage, socket: Duplex, head: Buffer) => boolean {
options: WsBridgeOptions = {},
): WsBridgeHandle {
const wss = new WebSocketServer({ noServer: true, maxPayload: MAX_REQUEST_BYTES, perMessageDeflate: false });
const sessions = new Set<{ session: WsBridgeSession; peer: WebSocket }>();
const sessions = new Set<BridgeEntry>();
let connectionId = 0;
const now = options.now ?? Date.now;
const scanMs = options.scanMs ?? 60_000;
const idleWarnSeconds = Number(process.env.BILI_WS_IDLE_WARN_SECONDS ?? 1800);
// Non-numeric or negative env values fall back to the 30-minute default instead of silently disabling the scan (sibling-knob convention, e.g. BILI_MODEL_INFO_RETRY_MS).
const idleWarnMs = options.idleWarnMs ?? (Number.isFinite(idleWarnSeconds) && idleWarnSeconds >= 0 ? idleWarnSeconds : 1800) * 1000;
// #1926: idle client peers pin one live upstream connection plus both
// checkpoints each, with no reclamation until the client closes. Until
// idle-close (a)/(b) is verified against the retry-full contract, surface
// the retention: one warn line per idle episode per connection (reset on
// activity), and per-connection stats via `stats()` below.
const scanner = idleWarnMs > 0 && scanMs > 0 ? setInterval(() => {
for (const entry of sessions) {
const idleMs = now() - entry.lastActivityAt;
if (entry.idleWarned || idleMs < idleWarnMs) continue;
entry.idleWarned = true;
const retained = entry.session.stats?.().retainedBytes;
log("warn", `${entry.label} idle ${Math.round(idleMs / 1000)}s: socket pins a live upstream connection and checkpoints${retained === undefined ? "" : ` (retained≈${retained} bytes)`} — no reclamation until client close (#1926)`);
}
}, scanMs) : undefined;
scanner?.unref();
const close = server.close.bind(server);
server.close = callback => {
const live = [...sessions];
sessions.clear(); // exactly-once: the 'close' event backstop below must not re-run
scanner?.close();
for (const { session } of live) session.shutdown("server-close");
for (const peer of wss.clients) peer.terminate();
return close(callback);
};
server.on("close", () => {
scanner?.close();
for (const { session } of sessions) session.shutdown("server-close");
for (const peer of wss.clients) peer.terminate();
wss.close();
});
return (source, socket, head) => {
const upgrade = (source: http.IncomingMessage, socket: Duplex, head: Buffer): boolean => {
const admitted = isLoopbackAddress(source.socket.remoteAddress);
const claim = admitted ? codecs.flatMap(candidate => {
const conversation = source.headers[candidate.conversationHeader ?? "x-bili-plugin-conversation"];
Expand All @@ -125,22 +184,34 @@ export function installWebSocketBridge(
}
if (socket.destroyed) return;
wss.handleUpgrade(source, socket, head, peer => {
const label = `[${codec.name}] [conn=${++connectionId}] [session=${JSON.stringify(conversationId.slice(0, 128))}]`;
const connection = ++connectionId;
const label = `[${codec.name}] [conn=${connection}] [session=${JSON.stringify(conversationId.slice(0, 128))}]`;
const trace: WsBridgeLog = (level, message) => log(level, `${label} ${message}`);
const session = codec.createSession({ log: trace, peer, source, upstreamUrl: upstream, dispatch, codec });
const entry = { session, peer };
const entry: BridgeEntry = { session, peer, codec: codec.name, connection, label, lastActivityAt: now(), idleWarned: false };
sessions.add(entry);
peer.on("error", () => {});
peer.on("close", (code: number) => {
session.onClose(code);
sessions.delete(entry);
});
peer.on("message", (data: Buffer, binary: boolean) => session.onMessage(data, binary));
peer.on("message", (data: Buffer, binary: boolean) => {
entry.lastActivityAt = now();
entry.idleWarned = false;
session.onMessage(data, binary);
});
});
})().catch(() => {
log("warn", `[${codec.name}] upgrade failed`);
if (!socket.destroyed) socket.end("HTTP/1.1 502 Bad Gateway\r\nConnection: close\r\nContent-Length: 0\r\n\r\n");
});
return true;
};
const stats = (): WsBridgeStats => ({
connections: [...sessions].map(entry => {
const entryStats = entry.session.stats?.();
return { codec: entry.codec, connection: entry.connection, idleMs: Math.max(0, now() - entry.lastActivityAt), retainedBytes: entryStats?.retainedBytes, inFlight: entryStats?.inFlight };
}),
});
return Object.assign(upgrade, { stats });
}
Loading
Loading