From de68fc62c215eaac99cf6fc4fa00ecd22170ce3c Mon Sep 17 00:00:00 2001 From: awork Date: Thu, 15 Oct 2026 16:10:27 +0800 Subject: [PATCH 1/7] feat: in-request model-content deadline with three-level cascade (#3254) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit In-request summarization had no bili-side wait bound of its own: a stalled relay that kept the TCP connection alive (byte-level idle timer rearmed by pings/comments) could hold a model attempt open until the CLIENT's own content watchdog fired — issue #3254 measured 775s + 773s + 27s for one fold, with bili logs showing only keep-alive notices and zero abort records. Add two opt-in model-content deadlines, each cascading model > provider route > global network.* (explicit 0 wins at its level): - firstContentTimeoutMs: response start -> first model content event (text/reasoning delta, tool call, redacted thinking, standalone signature); - contentGapTimeoutMs: silence between content events after the first arrived, re-armed on every content event (absent -> inherits the resolved initial value). On expiry the loop aborts the stalled upstream stream (interruptible-body wrapper: ReadableStream.cancel() on a locked stream rejects, so the kill goes through the underlying source's controller) and ends the turn with one clean in-band error in the #1453 shape — context intact, resend your last message. No internal blind refetch (owner decision): the client's retry ladder re-arms from the in-band error, which it handles natively (verified against codex-webui's handle_response_stream_error/retry_delay). Deliberately file-only config (no env layer): per-route/per-model values cannot be expressed as one process-wide variable. Strict validation (throw on non-integer/negative) at load time AND per request, unlike the lenient network.* numeric siblings — a typo must not silently disable a safety mechanism. Absent everywhere = byte-identical legacy behavior. Tests: tests/issue3254-first-content-deadline.test.ts (initial/gap fire, healthy-stream no-fire, legacy pin, validator unit, cascade table, load-time validation via ACP_PROVIDERS file path). Docs mirrored en/zh + config-reference yamls. Config-surface guard green. --- CONFIGURATION.md | 8 +- CONFIGURATION.zh-CN.md | 8 +- src/config.ts | 91 +++++- src/loop/core.ts | 148 ++++++++++ src/server/relay.ts | 8 +- .../issue3254-first-content-deadline.test.ts | 276 ++++++++++++++++++ website/config-reference/process.yaml | 16 +- website/config-reference/providers.yaml | 20 ++ 8 files changed, 565 insertions(+), 10 deletions(-) create mode 100644 tests/issue3254-first-content-deadline.test.ts diff --git a/CONFIGURATION.md b/CONFIGURATION.md index f398f1d02f..598ba07987 100644 --- a/CONFIGURATION.md +++ b/CONFIGURATION.md @@ -152,6 +152,8 @@ This index is generated from `website/config-reference/*.yaml` — edit the seed | `network.maxShrinkPerCompress` | number | unset | BILI_MAX_SHRINK_PER_COMPRESS | Cap on how much one compression may shrink history (anti-collapse guard). | | `network.proxyKeepAliveMaxMs` | number | 55000 (0 = one-shot connections) | BILI_PROXY_KEEPALIVE_MAX_MS | Max lifetime of a connection through an outbound egress proxy before forcing reconnection (avoids stale NAT mappings). | | `network.postResponseLingerMs` | number | 5000 | BILI_POST_RESPONSE_LINGER_MS | Graceful-close budget for proxy-initiated closes after the response completed: the socket is held waiting for the peer's FIN/TLS close_notify and destroyed at the deadline (reason=linger-backstop). | +| `network.firstContentTimeoutMs` | number | unset (off) | — | Model-content deadline, phase 1 (#3254): max ms an upstream attempt may go with zero model content (text/reasoning deltas, tool calls, signatures) from response start before bili aborts it and ends the turn with an in-band error. Also per route (`providers[]`) and per model (`providers[].models[]`); cascade model > route > global, explicit 0 wins at its level. File-only by design (per-route values can't be one process-wide env var). | +| `network.contentGapTimeoutMs` | number | unset (inherits firstContentTimeoutMs) | — | Model-content deadline, phase 2 (#3254): max ms between two consecutive model-content events after the first arrived; re-armed on every content event (pings/comments never count). Absent → inherits the resolved `firstContentTimeoutMs`; 0 disables the gap phase only. Same three-level cascade and file-only rationale as `firstContentTimeoutMs`. | | `network.allowPrivateHttpProviders` | boolean | false | BILI_ALLOW_PRIVATE_HTTP_PROVIDERS | Opt-in (#2984): admit plain-HTTP providers on private networks (RFC1918, 127/8, link-local, ULA, *.local/.lan/.internal) into the agent-registry intake and summary-endpoint guards. Off by default — plaintext stays loopback-only; public IPs/domains and CGNAT (100.64/10) stay refused regardless. 1/true enables. | | `network.receiveBodyBytes` | number | 5368709120 | BILI_RECEIVE_BODY_BYTES | Max inbound wire bytes read into memory per request before a 413 at the receive stage. Raised from a hardcoded 100 MiB to 5 GiB (#3128 owner decision) so multi-gigabyte single requests are accepted instead of the socket being reset. Also the threshold above which a decoded request consumes one of two shared large-decode admission slots. WebSocket lanes keep the fixed 100 MiB surface. | | `network.decodedBodyBytes` | number | 10737418240 | BILI_DECODED_BODY_BYTES | Max DECODED bytes after content-encoding expansion (gzip/deflate/br/zstd), checked incrementally during decode (decode-stage 413). Default is 2x the receive budget, mirroring the legacy decode=2x-wire relationship; guards against decompression bombs. | @@ -787,6 +789,8 @@ Since #2030 every pure-behavior knob has a config-file key alongside its env var "preflightDeadEndCooldownMs": 300000, // cooldown after a dead-end preflight verdict "proxyKeepAliveMaxMs": 55000, // upstream-proxy connection reuse cap; 0 uncapped "postResponseLingerMs": 5000, // post-response close linger budget (#1982) + "firstContentTimeoutMs": 180000, // #3254: max ms without model content from response start; off by default; per-route/model overrides in providers + "contentGapTimeoutMs": 600000, // #3254: max ms between content events after the first; absent = inherits firstContentTimeoutMs "allowPrivateHttpProviders": false // opt-in: private-network plain-HTTP providers (#2984) }, @@ -845,10 +849,10 @@ Since #2030 every pure-behavior knob has a config-file key alongside its env var ### `network` -- **Type:** `{ upstreamTimeoutMs?: number; requestWatchdogMs?: number; replayRetryMax?: number; replayRetryBaseMs?: number; maxShrinkPerCompress?: number; keepAliveTimeoutMs?: number; clientErrorBackstopMs?: number; exposureLogIntervalMs?: number; streamKeepAliveMs?: number; preflightHoldMs?: number; preflightDeadEndCooldownMs?: number; proxyKeepAliveMaxMs?: number; postResponseLingerMs?: number; allowPrivateHttpProviders?: boolean }` +- **Type:** `{ upstreamTimeoutMs?: number; requestWatchdogMs?: number; replayRetryMax?: number; replayRetryBaseMs?: number; maxShrinkPerCompress?: number; keepAliveTimeoutMs?: number; clientErrorBackstopMs?: number; exposureLogIntervalMs?: number; streamKeepAliveMs?: number; preflightHoldMs?: number; preflightDeadEndCooldownMs?: number; proxyKeepAliveMaxMs?: number; postResponseLingerMs?: number; firstContentTimeoutMs?: number; contentGapTimeoutMs?: number; allowPrivateHttpProviders?: boolean }` - **Default:** `{}` (built-ins: `720000`, `2× upstream timeout`, `3`, `1500`, unset, `5000`, `30000`, `3600000`, `15000`, `30000`, `300000`, `55000`, `5000`, `false`) - **Status:** ACTIVE -- **Description:** Process-wide transport and retry timing (ms). Each key resolves env > file > default via its twin (`BILI_UPSTREAM_TIMEOUT_MS`, `BILI_REQUEST_WATCHDOG_MS`, `BILI_REPLAY_RETRY_MAX`, `BILI_REPLAY_RETRY_BASE_MS`, `BILI_MAX_SHRINK_PER_COMPRESS`, `BILI_KEEP_ALIVE_TIMEOUT_MS`, `BILI_CLIENT_ERROR_BACKSTOP_MS`, `BILI_EXPOSURE_LOG_INTERVAL_MS`, `BILI_STREAM_KEEPALIVE_MS`, `BILI_PREFLIGHT_HOLD_MS`, `BILI_PREFLIGHT_DEAD_END_COOLDOWN_MS`, `BILI_PROXY_KEEPALIVE_MAX_MS`, `BILI_POST_RESPONSE_LINGER_MS`, `BILI_ALLOW_PRIVATE_HTTP_PROVIDERS`). Semantics: `upstreamTimeoutMs` bounds time-to-first-byte and inter-chunk silence end-to-end (#551); `requestWatchdogMs` is the total-request budget (a negative value opts out); `replayRetryMax` counts attempts after transient upstream rejections (#189/#1688); `maxShrinkPerCompress` caps how much one compress may shrink the request before bili steers toward smaller ranges (#189); `streamKeepAliveMs` emits an SSE comment when the response has written zero bytes for that long (#1647); `proxyKeepAliveMaxMs` caps connection reuse through an upstream proxy below its recycle cadence (#1263), `0` uncaps; `postResponseLingerMs` is the graceful-close budget after a proxy-initiated post-response close — the socket is held waiting for the peer's FIN/TLS close_notify and destroyed at the deadline (`reason=linger-backstop`) (#1982); `allowPrivateHttpProviders` is an opt-in (#2984): when true, plain-HTTP provider baseUrls on private networks (RFC1918, 127/8, link-local, IPv6 ULA/link-local, `*.local`/`.lan`/`.internal`) are admitted by the agent-registry intake and summary-endpoint guards that otherwise trust loopback plaintext only — public IPs/domains and CGNAT (100.64/10) stay refused regardless, and the default `false` keeps pre-#2984 behavior byte-for-byte. Full details in the [env table](#config-file-keys-for-environment-knobs-2030). +- **Description:** Process-wide transport and retry timing (ms). Every key except `firstContentTimeoutMs`/`contentGapTimeoutMs` (#3254, file-only — see below) resolves env > file > default via its twin (`BILI_UPSTREAM_TIMEOUT_MS`, `BILI_REQUEST_WATCHDOG_MS`, `BILI_REPLAY_RETRY_MAX`, `BILI_REPLAY_RETRY_BASE_MS`, `BILI_MAX_SHRINK_PER_COMPRESS`, `BILI_KEEP_ALIVE_TIMEOUT_MS`, `BILI_CLIENT_ERROR_BACKSTOP_MS`, `BILI_EXPOSURE_LOG_INTERVAL_MS`, `BILI_STREAM_KEEPALIVE_MS`, `BILI_PREFLIGHT_HOLD_MS`, `BILI_PREFLIGHT_DEAD_END_COOLDOWN_MS`, `BILI_PROXY_KEEPALIVE_MAX_MS`, `BILI_POST_RESPONSE_LINGER_MS`, `BILI_ALLOW_PRIVATE_HTTP_PROVIDERS`). Semantics: `upstreamTimeoutMs` bounds time-to-first-byte and inter-chunk silence end-to-end (#551); `requestWatchdogMs` is the total-request budget (a negative value opts out); `replayRetryMax` counts attempts after transient upstream rejections (#189/#1688); `maxShrinkPerCompress` caps how much one compress may shrink the request before bili steers toward smaller ranges (#189); `streamKeepAliveMs` emits an SSE comment when the response has written zero bytes for that long (#1647); `proxyKeepAliveMaxMs` caps connection reuse through an upstream proxy below its recycle cadence (#1263), `0` uncaps; `postResponseLingerMs` is the graceful-close budget after a proxy-initiated post-response close — the socket is held waiting for the peer's FIN/TLS close_notify and destroyed at the deadline (`reason=linger-backstop`) (#1982); `allowPrivateHttpProviders` is an opt-in (#2984): when true, plain-HTTP provider baseUrls on private networks (RFC1918, 127/8, link-local, IPv6 ULA/link-local, `*.local`/`.lan`/`.internal`) are admitted by the agent-registry intake and summary-endpoint guards that otherwise trust loopback plaintext only — public IPs/domains and CGNAT (100.64/10) stay refused regardless, and the default `false` keeps pre-#2984 behavior byte-for-byte; `firstContentTimeoutMs`/`contentGapTimeoutMs` are FILE-ONLY model-content deadlines (#3254): phase 1 bounds how long an upstream attempt may go with zero model content (text/reasoning deltas, tool calls, signatures) from response start, phase 2 (`contentGapTimeoutMs`, re-armed on every content event — pings/comments never count) bounds silence between content events after the first arrived; on expiry bili aborts the stalled upstream stream and ends the turn with one clean in-band error (context intact, resend your last message) — no internal refetch, so the client's own retry ladder re-arms; cascade model > provider route > global `network.*`, explicit 0 wins at its level, gap inherits the resolved initial when absent; deliberately no env layer because per-route/per-model values cannot be expressed as one process-wide variable. Full details in the [env table](#config-file-keys-for-environment-knobs-2030). ### `persist` diff --git a/CONFIGURATION.zh-CN.md b/CONFIGURATION.zh-CN.md index 316a208b36..4d529ba838 100644 --- a/CONFIGURATION.zh-CN.md +++ b/CONFIGURATION.zh-CN.md @@ -152,6 +152,8 @@ | `network.maxShrinkPerCompress` | number | unset | BILI_MAX_SHRINK_PER_COMPRESS | 单次压缩允许收缩历史的上限(防塌缩护栏)。 | | `network.proxyKeepAliveMaxMs` | number | 55000 (0 = one-shot connections) | BILI_PROXY_KEEPALIVE_MAX_MS | 经出站代理的连接最大存活时长,到期强制重建(规避陈旧 NAT 映射)。 | | `network.postResponseLingerMs` | number | 5000 | BILI_POST_RESPONSE_LINGER_MS | 代理主动发起的响应后关闭的优雅关闭预算:socket 挂起等待对端 FIN/TLS close_notify,到点销毁(reason=linger-backstop)。 | +| `network.firstContentTimeoutMs` | number | 未设置(关闭) | — | 模型内容期限,阶段一(#3254):上游一次尝试从响应开始到首个模型内容(文本/推理增量、工具调用、签名)之间的最长毫秒数;超时则 bili 中止卡死流并以带内错误结束回合。也可按线路(`providers[]`)与按模型(`providers[].models[]`)设置;级联 model > route > global,显式 0 在所在层生效。按设计仅配置文件可设(每线路取值无法用单一进程级环境变量表达)。 | +| `network.contentGapTimeoutMs` | number | 未设置(继承 firstContentTimeoutMs) | — | 模型内容期限,阶段二(#3254):首个内容到达之后,相邻两个模型内容事件之间的最长毫秒数;每个内容事件重新计时(ping/注释不算内容)。缺省继承解析后的 `firstContentTimeoutMs`;0 仅禁用阶段二。三级级联与仅文件可设的理由同 `firstContentTimeoutMs`。 | | `network.allowPrivateHttpProviders` | boolean | false | BILI_ALLOW_PRIVATE_HTTP_PROVIDERS | 可选开关(#2984):允许私网(RFC1918、127/8、link-local、ULA、*.local/.lan/.internal)明文 HTTP provider 进入 agent-registry 接收与摘要端点守卫。默认关——明文仅回环受信;公网 IP/域名与 CGNAT(100.64/10)无论如何都拒。1/true 启用。 | | `network.receiveBodyBytes` | number | 5368709120 | BILI_RECEIVE_BODY_BYTES | 单请求读入内存的入站 wire 字节上限,超限在 receive 阶段返回 413。由硬编码 100 MiB 提升到 5 GiB(#3128 owner 决定),使多 GB 单请求被接受而非断连。同时也是解码请求占用两个共享大解码准入槽位的阈值。WebSocket lane 保持固定 100 MiB 表面。 | | `network.decodedBodyBytes` | number | 10737418240 | BILI_DECODED_BODY_BYTES | content-encoding 解压(gzip/deflate/br/zstd)后的解码字节上限,解码过程中增量检查(decode 阶段 413)。默认为接收预算的 2 倍,沿用旧的 decode=2×wire 关系;用于防解压炸弹。 | @@ -787,6 +789,8 @@ "preflightDeadEndCooldownMs": 300000, // preflight 死路判定后的冷却 "proxyKeepAliveMaxMs": 55000, // 经上游代理的连接复用上限;0 不限 "postResponseLingerMs": 5000, // 响应后关闭的优雅关闭预算(#1982) + "firstContentTimeoutMs": 180000, // #3254:响应开始后无模型内容的上限毫秒数;默认关闭;providers 内可按线路/模型覆盖 + "contentGapTimeoutMs": 600000, // #3254:首个内容之后相邻内容事件间的上限毫秒数;缺省继承 firstContentTimeoutMs "allowPrivateHttpProviders": false // 私网明文 HTTP provider 可选放行(#2984) }, @@ -845,10 +849,10 @@ ### `network` -- **类型:** `{ upstreamTimeoutMs?: number; requestWatchdogMs?: number; replayRetryMax?: number; replayRetryBaseMs?: number; maxShrinkPerCompress?: number; keepAliveTimeoutMs?: number; clientErrorBackstopMs?: number; exposureLogIntervalMs?: number; streamKeepAliveMs?: number; preflightHoldMs?: number; preflightDeadEndCooldownMs?: number; proxyKeepAliveMaxMs?: number; postResponseLingerMs?: number; allowPrivateHttpProviders?: boolean }` +- **类型:** `{ upstreamTimeoutMs?: number; requestWatchdogMs?: number; replayRetryMax?: number; replayRetryBaseMs?: number; maxShrinkPerCompress?: number; keepAliveTimeoutMs?: number; clientErrorBackstopMs?: number; exposureLogIntervalMs?: number; streamKeepAliveMs?: number; preflightHoldMs?: number; preflightDeadEndCooldownMs?: number; proxyKeepAliveMaxMs?: number; postResponseLingerMs?: number; firstContentTimeoutMs?: number; contentGapTimeoutMs?: number; allowPrivateHttpProviders?: boolean }` - **默认:** `{}`(内置值依次为:`720000`、`2× 上游超时`、`3`、`1500`、未设置、`5000`、`30000`、`3600000`、`15000`、`30000`、`300000`、`55000`、`5000`、`false`) - **状态:** ACTIVE -- **说明:** 进程级传输与重试时序(毫秒)。每个键按 env > file > default 解析,对应环境变量分别为 `BILI_UPSTREAM_TIMEOUT_MS`、`BILI_REQUEST_WATCHDOG_MS`、`BILI_REPLAY_RETRY_MAX`、`BILI_REPLAY_RETRY_BASE_MS`、`BILI_MAX_SHRINK_PER_COMPRESS`、`BILI_KEEP_ALIVE_TIMEOUT_MS`、`BILI_CLIENT_ERROR_BACKSTOP_MS`、`BILI_EXPOSURE_LOG_INTERVAL_MS`、`BILI_STREAM_KEEPALIVE_MS`、`BILI_PREFLIGHT_HOLD_MS`、`BILI_PREFLIGHT_DEAD_END_COOLDOWN_MS`、`BILI_PROXY_KEEPALIVE_MAX_MS`、`BILI_POST_RESPONSE_LINGER_MS`、`BILI_ALLOW_PRIVATE_HTTP_PROVIDERS`。要点:`upstreamTimeoutMs` 端到端约束首字节时间与 chunk 间静默(#551);`requestWatchdogMs` 是单请求总预算(负值 = 退出该约束);`replayRetryMax` 统计瞬态上游拒绝后的尝试次数(#189/#1688);`maxShrinkPerCompress` 限制单次压缩对请求的最大缩减比例,超限则引导模型选择更小的范围(#189);`streamKeepAliveMs` 在响应零字节写出超过该时长时发一条 SSE 注释行(#1647);`proxyKeepAliveMaxMs` 把经上游代理的连接复用窗口压在上游回收周期之下(#1263),`0` 取消上限;`postResponseLingerMs` 是代理主动发起的响应后关闭的优雅关闭预算 —— socket 被挂起等待对端 FIN/TLS close_notify,到点销毁(`reason=linger-backstop`)(#1982);`allowPrivateHttpProviders` 是可选开关(#2984):为 true 时,私网(RFC1918、127/8、link-local、IPv6 ULA/link-local、`*.local`/`.lan`/`.internal`)上的明文 HTTP provider baseUrl 被 agent-registry 接收与摘要端点守卫放行——这些守卫默认只信任回环明文;公网 IP/域名与 CGNAT(100.64/10)无论如何都拒;默认 false 保持 #2984 之前的行为逐字不变。完整细节见[环境变量表](#环境变量的配置键对照-2030)。 +- **说明:** 进程级传输与重试时序(毫秒)。除 `firstContentTimeoutMs`/`contentGapTimeoutMs`(#3254,仅文件可设——见下)外,每个键按 env > file > default 解析,对应环境变量分别为 `BILI_UPSTREAM_TIMEOUT_MS`、`BILI_REQUEST_WATCHDOG_MS`、`BILI_REPLAY_RETRY_MAX`、`BILI_REPLAY_RETRY_BASE_MS`、`BILI_MAX_SHRINK_PER_COMPRESS`、`BILI_KEEP_ALIVE_TIMEOUT_MS`、`BILI_CLIENT_ERROR_BACKSTOP_MS`、`BILI_EXPOSURE_LOG_INTERVAL_MS`、`BILI_STREAM_KEEPALIVE_MS`、`BILI_PREFLIGHT_HOLD_MS`、`BILI_PREFLIGHT_DEAD_END_COOLDOWN_MS`、`BILI_PROXY_KEEPALIVE_MAX_MS`、`BILI_POST_RESPONSE_LINGER_MS`、`BILI_ALLOW_PRIVATE_HTTP_PROVIDERS`。要点:`upstreamTimeoutMs` 端到端约束首字节时间与 chunk 间静默(#551);`requestWatchdogMs` 是单请求总预算(负值 = 退出该约束);`replayRetryMax` 统计瞬态上游拒绝后的尝试次数(#189/#1688);`maxShrinkPerCompress` 限制单次压缩对请求的最大缩减比例,超限则引导模型选择更小的范围(#189);`streamKeepAliveMs` 在响应零字节写出超过该时长时发一条 SSE 注释行(#1647);`proxyKeepAliveMaxMs` 把经上游代理的连接复用窗口压在上游回收周期之下(#1263),`0` 取消上限;`postResponseLingerMs` 是代理主动发起的响应后关闭的优雅关闭预算 —— socket 被挂起等待对端 FIN/TLS close_notify,到点销毁(`reason=linger-backstop`)(#1982);`allowPrivateHttpProviders` 是可选开关(#2984):为 true 时,私网(RFC1918、127/8、link-local、IPv6 ULA/link-local、`*.local`/`.lan`/`.internal`)上的明文 HTTP provider baseUrl 被 agent-registry 接收与摘要端点守卫放行——这些守卫默认只信任回环明文;公网 IP/域名与 CGNAT(100.64/10)无论如何都拒;默认 false 保持 #2984 之前的行为逐字不变;`firstContentTimeoutMs`/`contentGapTimeoutMs` 是仅文件可设的模型内容期限(#3254):阶段一约束上游一次尝试从响应开始到首个模型内容(文本/推理增量、工具调用、签名)的最长等待,阶段二(`contentGapTimeoutMs`,每个内容事件重新计时——ping/注释不算内容)约束首个内容之后相邻内容事件间的静默;到期则 bili 中止卡死的上游流并以一条干净的带内错误结束回合(上下文完好,重发最后一条消息即可继续)——不做内部盲重取,客户端自身的重试梯照常接管;级联 model > provider route > global `network.*`,显式 0 在所在层生效,gap 缺省继承解析后的 initial;刻意不设 env 层,因为按线路/按模型的取值无法用单一进程级变量表达。完整细节见[环境变量表](#环境变量的配置键对照-2030)。 ### `persist` diff --git a/src/config.ts b/src/config.ts index 4b3980e165..5cbd43a5e2 100644 --- a/src/config.ts +++ b/src/config.ts @@ -133,8 +133,27 @@ export type ProviderRoute = { * - `max: ` — clamp the applied factor to ≤ max (e.g. `1` = * allow deflation learning only, never inflation). Must be > 0 and ≤ 4 * (higher is inert anyway — the publication clamp already caps at 4). - * Invalid values reject the config load loudly (strict #1909 discipline). */ + * Invalid values reject the config load loudly (strict #1909 discipline). */ calibration?: { disabled?: boolean; max?: number }; + /** #3254 level-2 (global → provider → model cascade): per-route in-request + * model-content deadline. When armed, EVERY upstream attempt of the + * compression loop must see MODEL CONTENT (text/reasoning delta, tool + * call, thinking-family block) within `firstContentTimeoutMs` of the + * response headers; relay pings / SSE keep-alive comments never count — + * they are exactly what hid the 775s stall. On fire the loop aborts the + * live upstream stream and exits with an in-band protocol-native error + * (context intact; the host/client retry ladder takes over) instead of + * letting a stalled-but-alive stream ride the client's own content + * watchdog. Absent everywhere = feature off (byte-identical legacy); + * explicit 0 disables the phase AT THIS LEVEL (falls back to nothing, + * not to the lower level). Invalid values reject the config load loudly + * (strict #1909 discipline). See contentGapTimeoutMs for the mid-stream + * phase. */ + firstContentTimeoutMs?: number; + /** #3254 level-2: inter-content deadline — re-armed on every model content + * event after the first (mid-stream stalls included). Unset inherits the + * resolved firstContentTimeoutMs value. Same levels/semantics. */ + contentGapTimeoutMs?: number; }; export type ProviderRoutes = Record; // key = upstream URL prefix (the /bili/ string) @@ -153,6 +172,10 @@ export type ModelEntry = { * (CODEARTS_BENEFIT_FALLBACK mirror). Env BILI_RESIGN_BENEFIT wins over * this like every other knob. */ benefit?: boolean; + /** #3254 level-3 (deepest): per-model overrides of the provider route's + * model-content deadlines. Same semantics as the route-level fields. */ + firstContentTimeoutMs?: number; + contentGapTimeoutMs?: number; }; /** User-facing compression tuning. Configurable at three levels — global @@ -1248,6 +1271,11 @@ export function loadOptions(env: NodeJS.ProcessEnv = process.env): ProxyOptions const host = rawHost === "localhost" ? "127.0.0.1" : rawHost; const upstream = (env.ACP_UPSTREAM ?? fileConfig.upstream ?? "https://api.anthropic.com").replace(/\/$/, ""); const routes = loadRoutes(env); + // #3254: the global network.* deadline tier is not covered by route + // parsing — validate it at startup too (a broken safety bound must not + // boot silently disabled). + parseDeadlineMs("network.firstContentTimeoutMs", fileConfig.network?.firstContentTimeoutMs); + parseDeadlineMs("network.contentGapTimeoutMs", fileConfig.network?.contentGapTimeoutMs); const namedProviders = loadNamedProviders(env); warnCcrPluginDivergences(routes, fileConfig.compress); const passthrough = passthroughState(env); @@ -1757,6 +1785,22 @@ type FileConfig = { proxyKeepAliveMaxMs?: number; /** Post-response close linger budget (#1982; was env-only BILI_POST_RESPONSE_LINGER_MS; default 5000). */ postResponseLingerMs?: number; + /** #3254 global in-request model-content deadline (level 1 of the + * global → provider → model cascade): no MODEL CONTENT (text/reasoning + * delta, tool call, thinking-family block) within this many ms of the + * response headers → the compression loop aborts the live upstream + * stream and exits with an in-band protocol-native error (context + * intact; the client's retry ladder takes over) instead of letting a + * stalled-but-alive stream ride the client's content watchdog. Relay + * pings / SSE keep-alive comments never count. Deliberately FILE-ONLY + * (no env layer): the per-route cascade needs a file shape env cannot + * express. 0 = explicitly disabled; default unset = feature off + * (byte-identical legacy). Invalid values reject loudly (strict + * #1909), unlike the lenient numeric siblings above. */ + firstContentTimeoutMs?: number; + /** #3254 inter-content phase: re-armed on every model content event; + * unset inherits firstContentTimeoutMs (see that field for levels). */ + contentGapTimeoutMs?: number; /** Opt-in plain-HTTP trust for private networks (#2984): when true, private hosts * (RFC1918, 127/8, link-local, ULA, *.local/.lan/.internal) may be admitted over * plaintext by the #2585 gates — agent-registry intake and the summary-endpoint @@ -2263,7 +2307,7 @@ export function parseRouteEntry(v: unknown): ProviderRoute | undefined { // is the KEY in the providers map (identical to the /bili/ string), // so it is NOT repeated inside the value. if (v && typeof v === "object" && !Array.isArray(v)) { - const obj = v as { models?: Record; proxy?: string; compressProtocol?: string; protocol?: unknown; compress?: CompressSettings; compat?: { roles?: unknown; dropFields?: unknown }; passthrough?: boolean; direct?: boolean; imageBilling?: unknown; imageTokenCap?: unknown; calibration?: unknown; baseUrl?: unknown; api?: unknown; apiKeyEnv?: unknown; credentialRef?: unknown }; + const obj = v as { models?: Record; proxy?: string; compressProtocol?: string; protocol?: unknown; compress?: CompressSettings; compat?: { roles?: unknown; dropFields?: unknown }; passthrough?: boolean; direct?: boolean; imageBilling?: unknown; imageTokenCap?: unknown; calibration?: unknown; firstContentTimeoutMs?: unknown; contentGapTimeoutMs?: unknown; baseUrl?: unknown; api?: unknown; apiKeyEnv?: unknown; credentialRef?: unknown }; const route: ProviderRoute = { models: obj.models }; if (typeof obj.proxy === "string") route.proxy = obj.proxy; if (obj.compressProtocol === "marker" || obj.compressProtocol === "tools") route.compressProtocol = obj.compressProtocol; @@ -2284,6 +2328,17 @@ export function parseRouteEntry(v: unknown): ProviderRoute | undefined { if (imageTokenCap !== undefined) route.imageTokenCap = imageTokenCap; const calibration = parseCalibrationPolicy(obj.calibration); if (calibration) route.calibration = calibration; + // #3254: model-content deadlines — strict like calibration (a silently + // dropped safety bound would leave the stall invisible again). Model + // entries used to pass through raw; validate their copies too. + route.firstContentTimeoutMs = parseDeadlineMs("providers entry firstContentTimeoutMs", obj.firstContentTimeoutMs); + route.contentGapTimeoutMs = parseDeadlineMs("providers entry contentGapTimeoutMs", obj.contentGapTimeoutMs); + for (const [modelName, entry] of Object.entries(obj.models ?? {})) { + if (!entry || typeof entry !== "object") continue; + const me = entry as Record; + parseDeadlineMs(`providers entry models.${modelName}.firstContentTimeoutMs`, me.firstContentTimeoutMs); + parseDeadlineMs(`providers entry models.${modelName}.contentGapTimeoutMs`, me.contentGapTimeoutMs); + } // #2336 lane credential override. Only parsed for NON-recipe-shaped // entries: a dialing recipe (baseUrl/api present) validates its own // credential fields in parseNamedProviderRecipe, and letting this @@ -2363,6 +2418,38 @@ export function parseCalibrationPolicy(value: unknown): { disabled?: boolean; ma return out; } +// #3254: strict validator shared by the load-time route parsing and the +// per-request resolver (the global network.* tier bypasses load-time parsing, +// so a typo there must still reject loudly instead of silently disabling the +// safety bound). 0 is a valid value — it means "disabled" at its level. +export function parseDeadlineMs(fieldPath: string, value: unknown): number | undefined { + if (value === undefined) return undefined; + if (typeof value === "number" && Number.isInteger(value) && value >= 0) return value; + throw new Error(`[acp-config] ${fieldPath} must be a non-negative integer number of milliseconds (got ${JSON.stringify(value)})`); +} + +// #3254: three-level cascade (model > provider route > global network.*) for +// the model-content deadlines. An explicit 0 wins AT ITS LEVEL (disables that +// phase, no fall-through); an absent field inherits the next level down. +// contentGapTimeoutMs inherits the resolved initial deadline when left absent. +// Returns undefined when every phase is off or absent (feature fully off). +export function resolveFirstContentDeadlines( + routes: ProviderRoutes, + upstreamUrl: string | undefined, + model: string | undefined, + globals: { firstContentTimeoutMs?: unknown; contentGapTimeoutMs?: unknown } | undefined, +): { initialMs?: number; gapMs?: number } | undefined { + const route = findRoute(routes, upstreamUrl); + const entry = model ? route?.models?.[model] : undefined; + const initialMs = parseDeadlineMs("firstContentTimeoutMs", entry?.firstContentTimeoutMs ?? route?.firstContentTimeoutMs ?? globals?.firstContentTimeoutMs); + const gapMs = parseDeadlineMs("contentGapTimeoutMs", entry?.contentGapTimeoutMs ?? route?.contentGapTimeoutMs ?? globals?.contentGapTimeoutMs); + const out: { initialMs?: number; gapMs?: number } = {}; + if (initialMs !== undefined && initialMs > 0) out.initialMs = initialMs; + if (gapMs !== undefined && gapMs > 0) out.gapMs = gapMs; + else if (gapMs === undefined && out.initialMs !== undefined) out.gapMs = out.initialMs; + return out.initialMs !== undefined || out.gapMs !== undefined ? out : undefined; +} + function parseStreamErrorShape(value: unknown): "protocol" | "completion" { return value === "completion" ? "completion" : "protocol"; } diff --git a/src/loop/core.ts b/src/loop/core.ts index afaf08dac1..76a8f1cf11 100644 --- a/src/loop/core.ts +++ b/src/loop/core.ts @@ -208,6 +208,11 @@ export interface LoopCtx { * host = upstream hostname, fp = `${billing}:${cap}` fingerprint. When * present, each sent round captures its image facts alongside the body. */ imageLearn?: { host: string; fp: string }; + /** #3254: model-content deadlines resolved per request (three-level + * cascade via resolveFirstContentDeadlines). Present iff at least one + * phase is enabled; undefined keeps the legacy path byte-identical + * (no timers armed, zero overhead). */ + firstContentDeadline?: { initialMs?: number; gapMs?: number }; } export interface RequestOptions { @@ -395,6 +400,50 @@ function recordUsage( } } +// #3254: wrapper that lets the model-content deadline KILL a body stream from +// the outside. Plain ReadableStream.cancel() is unusable here: the adapter's +// SSE iterator holds the only reader, and cancel() on a LOCKED stream rejects +// with "Invalid state: ReadableStream is locked" (verified empirically on +// Node 22/24), silently no-oping the kill. So the wrapper owns its own +// controller — kill() errors it (the pending/next read rejects) and releases +// the inner body through ITS OWN reader.cancel (the legal path). Pull-driven +// passthrough keeps backpressure intact (one outstanding inner read). +function interruptibleBody(src: ReadableStream): { stream: ReadableStream; kill: () => void } { + let innerReader: ReadableStreamDefaultReader | null = null; + let outerController: ReadableStreamDefaultController | null = null; + let killed = false; + const killReason = new Error("bili model-content deadline (#3254)"); + const stream = new ReadableStream({ + start(c) { + outerController = c; + innerReader = src.getReader(); + }, + async pull(c) { + if (killed) { c.error(killReason); return; } + try { + const r = await innerReader!.read(); + if (killed) return; + if (r.done) c.close(); + else c.enqueue(r.value); + } catch (e) { + if (!killed) c.error(e instanceof Error ? e : new Error("upstream body read failed")); + } + }, + cancel(reason) { + void innerReader?.cancel(reason).catch(() => {}); + }, + }); + return { + stream, + kill: () => { + if (killed) return; + killed = true; + outerController?.error(killReason); + void innerReader?.cancel(killReason).catch(() => {}); + }, + }; +} + export async function* runCompressLoop( upstream: ReadableStream, ctx: LoopCtx, @@ -422,6 +471,56 @@ export async function* runCompressLoop( // in client-stored history (incoming-history stripping removes it next turn, // but one copy per request is enough signal for humans). const seenMarkers = new Set(); + // #3254: model-content deadline. Byte-level silence is already bounded by + // the upstream idle timer, but SSE keep-alive comments/pings keep it armed + // forever — without this bound a relay that streams pings for 13 minutes + // per attempt leaves the CLIENT's content watchdog as the de-facto circuit + // breaker (the incident: 775s + 773s + 27s). Firing kills the wrapped body + // stream (interruptibleBody: controller.error rejects the pending read; + // every adapter's SSE iterator swallows that rejection as EOF, so the + // consumption loop below exits cleanly) and the turn ends + // with ONE clean in-band error — deliberately NO internal refetch + // (#3254 decision C-rejected: the client's retry ladder re-arms with + // backoff, which beats hammering the same stalled slot). + let activeKill: (() => void) | null = null; + let contentDeadlineTimer: ReturnType | null = null; + let contentDeadlineArmedAt = 0; + let contentDeadlineInfo: { ms: number; phase: "initial" | "gap"; round: number } | null = null; + let contentDeadlineKilled = false; + let contentDeadlineSettled = true; + let upstreamAttemptNo = 0; + const deadlineHost = (() => { try { return new URL(requestOptions.url).host; } catch { return requestOptions.url; } })(); + const clearContentDeadline = () => { + if (contentDeadlineTimer) { + clearTimeout(contentDeadlineTimer); + contentDeadlineTimer = null; + } + }; + const fireContentDeadline = () => { + contentDeadlineTimer = null; + if (contentDeadlineSettled || !contentDeadlineInfo) return; + contentDeadlineKilled = true; + const info = contentDeadlineInfo; + const waitedMs = Date.now() - contentDeadlineArmedAt; + const line = `content-deadline fired: no model content for ${waitedMs}ms (limit ${info.ms}ms, phase=${info.phase}, source=config, upstream attempt ${upstreamAttemptNo}, round ${info.round}) — aborting stalled upstream stream (#3254)`; + ctx.log(`[acp-proxy: ${line}]`); + loggerLog("warn", `[acp-loop] session ${ctx.session.id} ${line}`); + activeKill?.(); + }; + const armContentDeadline = (ms: number, phase: "initial" | "gap", roundNo: number) => { + clearContentDeadline(); + contentDeadlineInfo = { ms, phase, round: roundNo }; + contentDeadlineArmedAt = Date.now(); + contentDeadlineSettled = false; + contentDeadlineTimer = setTimeout(fireContentDeadline, ms); + }; + // #3254: wrap the initial body so the deadline can kill it from outside + // (see interruptibleBody for why plain stream.cancel() cannot do this). + if (ctx.firstContentDeadline !== undefined) { + const wrapped = interruptibleBody(upstream); + activeKill = wrapped.kill; + currentUpstream = wrapped.stream; + } const fetchUpstream = (body: Record) => { // #1592-family seam forensics: remember the body actually sent so the @@ -496,6 +595,15 @@ export async function* runCompressLoop( // consumer parses — the dump twin above keeps RAW upstream bytes, // mirroring the outer request's pre-unwrap tee order in relay.ts. if (body && requestOptions.responseStreamTransform) body = requestOptions.responseStreamTransform(body); + // #3254: wrap the post-#3280-unwrap stream as the OUTERMOST body so + // the deadline kill errors exactly the reads parseStream is making + // and the cancel propagates down through the unwrap transform to the + // raw body (a tee dump branch dying early is acceptable). + if (ctx.firstContentDeadline !== undefined) { + const wrapped = interruptibleBody(body); + activeKill = wrapped.kill; + return wrapped.stream; + } return body; }; @@ -675,8 +783,25 @@ export async function* runCompressLoop( sawSignedThinkingThisAttempt = false; fwdBytes = 0; + // #3254: arm this attempt's initial-phase model-content deadline + // (gap-only configs arm nothing until first content arrives). + upstreamAttemptNo += 1; + contentDeadlineKilled = false; + if (ctx.firstContentDeadline?.initialMs !== undefined && ctx.firstContentDeadline.initialMs > 0) { + armContentDeadline(ctx.firstContentDeadline.initialMs, "initial", round); + } else { + clearContentDeadline(); + } for await (const ev of adapter.parseStream(currentUpstream, round)) { if (signal?.aborted) break; + // #3254: any MODEL CONTENT event re-arms the inter-content + // deadline (owner-tiered like the client's own watchdog); + // pings/comments/meta/usage/diag never count as content. + if (ctx.firstContentDeadline?.gapMs !== undefined && ctx.firstContentDeadline.gapMs > 0 + && ((ev.kind === "text" || ev.kind === "reasoning") ? ev.delta.length > 0 + : ev.kind === "tool_call" || ev.kind === "redacted_thinking" || ev.kind === "standalone_signature")) { + armContentDeadline(ctx.firstContentDeadline.gapMs, "gap", round); + } if (ev.kind === "text") { assistantText += ev.delta; let tp = roundParts[roundParts.length - 1]; @@ -755,6 +880,28 @@ export async function* runCompressLoop( } } } + // #3254: if WE cancelled the body, every adapter's SSE iterator + // swallowed the read rejection as EOF and the loop exited + // cleanly — end in-band exactly once, BEFORE any retry gate: + // a deadline kill must never trigger an internal refetch + // (#3254 decision C-rejected; the client's ladder re-arms). + contentDeadlineSettled = true; + clearContentDeadline(); + if (contentDeadlineKilled && !signal?.aborted) { + // explicit assertion: TS flow analysis does not see the + // closure reassignment inside armContentDeadline, so without + // it the read below is narrowed to the `null` initializer + const info = contentDeadlineInfo as { ms: number; phase: "initial" | "gap"; round: number } | null; + const sec = Math.max(1, Math.round((info?.ms ?? 0) / 1000)); + const phaseDesc = info?.phase === "gap" + ? `no model content for ${sec}s between content events` + : `no model content within ${sec}s of the response starting`; + const msg = `model content deadline exceeded — upstream ${deadlineHost} produced ${phaseDesc} (limit ${info?.ms ?? "?"}ms); your context is intact, resend your last message to continue`; + ctx.log(`[acp-proxy: ${msg}]`); + loggerLog("warn", `[acp-loop] session ${ctx.session.id} content-deadline reached (upstream attempt ${upstreamAttemptNo}, round ${info?.round ?? "?"}); ending turn in-band instead of stalling the client's watchdog (#3254)`); + yield adapter.emitError(msg); + return; + } // #2701: threads acceptance. This round's response started ⇔ // the tail advanced past the id the sent body continued from ⇔ @@ -1480,6 +1627,7 @@ export async function* runCompressLoop( recordThreadTail(ctx.session, id); } } + clearContentDeadline(); if (activeClearTimer) { activeClearTimer(); activeClearTimer = null; diff --git a/src/server/relay.ts b/src/server/relay.ts index 6cf3259dc6..9c86f7a295 100644 --- a/src/server/relay.ts +++ b/src/server/relay.ts @@ -15,7 +15,7 @@ import { applyCompatRoles, applyCompatRolesJson, detectRoleRejection, detectSyst import { compressLoopResponsesJson } from "../compress-loop-responses.js"; import { resolveCompress } from "../compress-settings.js"; import { buildCompressHybridSystemPrompt, buildCompressSystemPrompt, withMarkerIntegrityNote, withProjectAttribution, withSummaryBudgetNote } from "../compress-tool.js"; -import { findRoute, resolveDeclaredProtocol, type ProxyOptions } from "../config.js"; +import { findRoute, loadConfigFile, resolveDeclaredProtocol, resolveFirstContentDeadlines, type ProxyOptions } from "../config.js"; import { makeContinuationRefetch } from "../degenerate-retry.js"; import { dumpRejectedBody } from "../error-dump.js"; import { externalSummaryEnabled } from "../external-summary-surface.js"; @@ -1495,9 +1495,13 @@ export async function forward( // the loop has no access to the provider route table. Same billing // upstream expression as outboundPayloadBreakdown. const loopBillingUpstream = route?.rewrittenUrl ?? (/^https?:\/\//i.test(req.url ?? "") ? req.url ?? undefined : opts.upstream); + // #3254: model-content deadlines — three-level cascade (model > + // route > global network.*), resolved per request so config edits + // apply without a restart; absent everywhere → undefined → legacy. + const firstContentDeadline = resolveFirstContentDeadlines(opts.routes, loopBillingUpstream, (parsedReq as { model?: string }).model, loadConfigFile().network); const loop = runCompressLoop( responseBody, - { core, config, messages: prepared.processedMessages.length > 0 ? prepared.processedMessages : prepared.originalMessages, compressMessages: prepared.originalMessages, session: prepared.session, log: ctx.log, proxyUrl, skippedProxy, upstreamOrigin: targetOrigin, protocol: prepared.protocol, textProtocol, debug: opts.debug, refreshFolded, visibilityMarkers, dumpSse: loopDumpDir ? (name, stream) => dumpStreamToFile(stream, loopDumpDir, name) : undefined, imageLearn: { host: upstreamHost(loopBillingUpstream), fp: `${imageBillingFor(opts, loopBillingUpstream)}:${imageTokenCapFor(opts, loopBillingUpstream)}` } }, + { core, config, messages: prepared.processedMessages.length > 0 ? prepared.processedMessages : prepared.originalMessages, compressMessages: prepared.originalMessages, session: prepared.session, log: ctx.log, proxyUrl, skippedProxy, upstreamOrigin: targetOrigin, protocol: prepared.protocol, textProtocol, debug: opts.debug, refreshFolded, visibilityMarkers, dumpSse: loopDumpDir ? (name, stream) => dumpStreamToFile(stream, loopDumpDir, name) : undefined, imageLearn: { host: upstreamHost(loopBillingUpstream), fp: `${imageBillingFor(opts, loopBillingUpstream)}:${imageTokenCapFor(opts, loopBillingUpstream)}` }, firstContentDeadline }, parsedReq, // #3280: egress goes out enveloped via wireTransform, so every // round-2+ response comes back enveloped too — unwrap at the loop's diff --git a/tests/issue3254-first-content-deadline.test.ts b/tests/issue3254-first-content-deadline.test.ts new file mode 100644 index 0000000000..198f6ce4b4 --- /dev/null +++ b/tests/issue3254-first-content-deadline.test.ts @@ -0,0 +1,276 @@ +import { test } from "node:test"; +import assert from "node:assert/strict"; +import { mkdtempSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import type { Config, CoreMessage } from "acp-kernel"; +import { createCore, createInitialState } from "acp-kernel"; +import type { Session } from "../src/session.ts"; +import { runCompressLoop, createAnthropicAdapter } from "../src/loop/index.ts"; +import { buildCompressSystemPrompt } from "../src/compress-tool.ts"; +import { loadRoutes, parseDeadlineMs, resolveFirstContentDeadlines, type ProviderRoutes } from "../src/config.ts"; + +// #3254: in-request summarization had no bili-side wait bound of its own. The +// upstream idle timer is byte-level — SSE keep-alive comments/pings keep it +// armed forever — so a relay that streams pings for 13 minutes per attempt +// left the CLIENT's content watchdog (780s, codex-webui) as the de-facto +// circuit breaker. Incident: one fold = 775s + 773s + 27s across three client +// attempts, bili.log fully silent. Pinned here: (A) an initial-phase deadline +// fires, cancels the stalled body, ends the turn with ONE clean in-band error +// and NO internal refetch (owner decision C-rejected: the client's retry +// ladder re-arms with backoff); (B) the inter-content (gap) phase fires after +// first content; (C) live content re-arms the deadline and never false-fires; +// (D) absent config stays byte-identical legacy (no timers, no early exit); +// (E-H) the three-level cascade (model > route > global network.*) incl. +// explicit-0-wins-at-level, gap-inherits-initial, and strict validation. + +const ANTHROPIC_BODY = { model: "claude", messages: [], stream: true, max_tokens: 10 }; +const enc = new TextEncoder(); + +function makeCtx(id: string, logs: string[]): { + core: ReturnType; + config: Config; + messages: CoreMessage[]; + session: Session; + log: (m: string) => void; +} { + return { + core: createCore(), + config: { modelContextLimit: 200000 } as Config, + messages: [], + session: { + id, + meta: {}, + stats: { requests: 0, tokensSaved: 0, inputTokens: 0, cachedTokens: 0, outputTokens: 0, cacheSamples: 0, lastInputTokens: 0, contextTokens: 0, compressCreditTokens: 0, retrieveCalls: 0, retrieveHits: 0, retrieveMisses: 0, storedBytes: 0, storeBytesSaved: 0, rangeRestores: 0 }, + metadata: {}, + state: createInitialState(), + createdAt: Date.now(), + lastSeen: Date.now(), + blockContents: new Map(), + inFlight: 0, + persisted: false, + pendingRetrievals: [], + }, + log: (m: string) => { logs.push(m); }, + }; +} + +const ev = (type: string, data: unknown): string => `event: ${type}\ndata: ${JSON.stringify(data)}\n\n`; + +const MESSAGE_START = ev("message_start", { + type: "message_start", + message: { + id: "msg_1", type: "message", role: "assistant", model: "claude", + content: [], stop_reason: null, stop_sequence: null, + usage: { input_tokens: 5, output_tokens: 0 }, + }, +}); +const PING = ev("ping", { type: "ping" }); +const textStart = (index: number) => ev("content_block_start", { type: "content_block_start", index, content_block: { type: "text", text: "" } }); +const textDelta = (index: number, text: string) => ev("content_block_delta", { type: "content_block_delta", index, delta: { type: "text_delta", text } }); +const blockStop = (index: number) => ev("content_block_stop", { type: "content_block_stop", index }); +const MESSAGE_DELTA = (stopReason = "end_turn") => ev("message_delta", { type: "message_delta", delta: { stop_reason: stopReason, stop_sequence: null }, usage: { input_tokens: 5, output_tokens: 3 } }); +const MESSAGE_STOP = ev("message_stop", { type: "message_stop" }); + +// Emits `text` synchronously then closes — everything is consumed before any +// macrotask timer can fire (deterministic, no timing margin needed). +function completeStream(text: string): ReadableStream { + return new ReadableStream({ + start(controller) { + controller.enqueue(enc.encode(text)); + controller.close(); + }, + }); +} + +// Emits `text` then HOLDS OPEN (the incident shape: pings/comments keep the +// byte idle-timer armed; no model content ever comes). Records cancellation. +function holdStream(text: string, sink: { cancelled: boolean }): ReadableStream { + return new ReadableStream({ + start(controller) { + controller.enqueue(enc.encode(text)); + }, + cancel() { sink.cancelled = true; }, + }); +} + +async function drainGen(gen: AsyncGenerator, chunks: Buffer[], timeoutMs: number): Promise { + const drain = (async () => { for await (const c of gen) chunks.push(c); })(); + let timer: ReturnType | undefined; + const guard = new Promise((_, reject) => { + timer = setTimeout(() => reject(new Error(`drain timed out after ${timeoutMs}ms (generator hung)`)), timeoutMs); + }); + try { + await Promise.race([drain, guard]); + } catch (e) { + // fire-and-forget on purpose: if the generator is parked on a read that + // can never settle (a deliberately held-open stream), .return() queues + // BEHIND the stuck .next() and never settles itself — awaiting it + // deadlocks the test (verified empirically). The abandoned generator is + // GC-eligible; it owns no handles. + void gen.return(undefined).catch(() => {}); + throw e; + } finally { + clearTimeout(timer); + } +} + +async function drainAnthropic( + stream: ReadableStream, + deadline: { initialMs?: number; gapMs?: number } | undefined, + id: string, + timeoutMs: number, +): Promise<{ out: string; fetchCalls: number; logs: string[] }> { + let n = 0; + const orig = globalThis.fetch; + // Any fetch here means the loop REFETCHED — the #3254 decision pins that a + // deadline kill never triggers an internal refetch (C-rejected). + globalThis.fetch = (async () => { n += 1; throw new Error("fetch called — internal refetch forbidden (#3254 C-rejected)"); }) as typeof fetch; + const logs: string[] = []; + const ctx = makeCtx(id, logs); + (ctx as Record).protocol = "anthropic"; + if (deadline !== undefined) (ctx as Record).firstContentDeadline = deadline; + const chunks: Buffer[] = []; + try { + const gen = runCompressLoop( + stream, + ctx, + ANTHROPIC_BODY, + { url: "http://mock", headers: {} }, + createAnthropicAdapter(ANTHROPIC_BODY, undefined, undefined, "protocol"), + buildCompressSystemPrompt(), + ); + await drainGen(gen, chunks, timeoutMs); + } finally { + globalThis.fetch = orig; + } + return { out: Buffer.concat(chunks).toString("utf8"), fetchCalls: n, logs }; +} + +const sseBlocks = (s: string): string[] => s.split("\n\n").map((b) => b.trim()).filter(Boolean); + +// --- A: initial-phase deadline fires on a stalled stream ------------------- + +test("#3254 A: initial-phase deadline cancels the stalled body and ends in-band once, no refetch", async () => { + const sink = { cancelled: false }; + const { out, fetchCalls, logs } = await drainAnthropic(holdStream(MESSAGE_START + PING, sink), { initialMs: 150, gapMs: 150 }, "s3254a", 3000); + assert.match(out, /model content deadline exceeded/); + assert.match(out, /no model content within 1s of the response starting \(limit 150ms\)/); + assert.match(out, /your context is intact, resend your last message to continue/); + assert.ok(sseBlocks(out).at(-1)?.startsWith("event: error"), "last SSE block must be the protocol-native error channel"); + assert.ok(logs.some((l) => l.includes("content-deadline fired") && l.includes("phase=initial") && l.includes("source=config") && l.includes("upstream attempt 1") && l.includes("round 1")), `fire log missing/malformed: ${JSON.stringify(logs)}`); + assert.equal(fetchCalls, 0, "deadline kill must NOT trigger an internal refetch"); + assert.equal(sink.cancelled, true, "the stalled upstream body must be cancelled"); +}); + +// --- B: gap phase fires after first content; pings do not re-arm ----------- + +test("#3254 B: inter-content (gap) deadline fires after first content; ping does not re-arm it", async () => { + const sink = { cancelled: false }; + const prefix = MESSAGE_START + textStart(0) + textDelta(0, "hi") + blockStop(0) + PING; + const { out, fetchCalls, logs } = await drainAnthropic(holdStream(prefix, sink), { gapMs: 150 }, "s3254b", 3000); + assert.match(out, /model content deadline exceeded/); + assert.match(out, /no model content for 1s between content events \(limit 150ms\)/); + assert.ok(logs.some((l) => l.includes("content-deadline fired") && l.includes("phase=gap")), `gap fire log missing: ${JSON.stringify(logs)}`); + assert.equal(fetchCalls, 0); + assert.equal(sink.cancelled, true); +}); + +// --- C: live content re-arms the deadline — no false positive -------------- + +test("#3254 C: continuous content re-arms the deadline; a healthy slow stream completes cleanly", async () => { + const text = MESSAGE_START + textStart(0) + textDelta(0, "a") + textDelta(0, "b") + blockStop(0) + MESSAGE_DELTA() + MESSAGE_STOP; + const { out, fetchCalls, logs } = await drainAnthropic(completeStream(text), { initialMs: 100, gapMs: 100 }, "s3254c", 3000); + assert.doesNotMatch(out, /model content deadline exceeded/); + assert.doesNotMatch(out, /event: error/); + assert.match(out, /"text_delta"/); + assert.ok(!logs.some((l) => l.includes("content-deadline")), `healthy stream must not trip the deadline: ${JSON.stringify(logs)}`); + assert.equal(fetchCalls, 0); +}); + +// --- D: absent config = legacy behavior (no timer, stream stays pending) --- + +test("#3254 D: without firstContentDeadline the legacy path is untouched (stall stays pending)", async () => { + // node:test treats "event loop drained while tests still pending" as fatal; + // this test deliberately leaves a stream read dangling, so pin a keep-alive + // handle for its duration or the later tests in this file get cancelled. + const keepAlive = setInterval(() => {}, 50); + const sink = { cancelled: false }; + let hung = false; + try { + try { + await drainAnthropic(holdStream(MESSAGE_START + PING, sink), undefined, "s3254d", 400); + } catch (e) { + hung = e instanceof Error && /drain timed out/.test(e.message); + } + assert.ok(hung, "legacy path must keep waiting on the stalled stream (no early exit)"); + assert.equal(sink.cancelled, false, "nothing may cancel the body when the feature is off"); + } finally { + clearInterval(keepAlive); + } +}); + +// --- E: parseDeadlineMs strictness ----------------------------------------- + +test("#3254 E: parseDeadlineMs accepts undefined/0/integers and rejects everything else", () => { + assert.equal(parseDeadlineMs("x", undefined), undefined); + assert.equal(parseDeadlineMs("x", 0), 0); + assert.equal(parseDeadlineMs("x", 180000), 180000); + for (const bad of [-1, 1.5, Number.NaN, "180000", true, null]) { + assert.throws(() => parseDeadlineMs("x", bad), /\[acp-config\] x must be a non-negative integer number of milliseconds/); + } +}); + +// --- F: three-level cascade semantics -------------------------------------- + +const ROUTES: ProviderRoutes = { + "https://relay-a.example": { + firstContentTimeoutMs: 600000, + models: { "gpt-x": { firstContentTimeoutMs: 90000 } }, + }, + "https://relay-b.example": { + firstContentTimeoutMs: 0, + }, + "https://relay-c.example": { + firstContentTimeoutMs: 180000, + contentGapTimeoutMs: 0, + }, +}; + +test("#3254 F: cascade — model > route > global, 0 wins at its level, gap inherits initial", () => { + // all absent → feature off + assert.equal(resolveFirstContentDeadlines({}, undefined, undefined, undefined), undefined); + // global tier alone; gap inherits the resolved initial + assert.deepEqual(resolveFirstContentDeadlines({}, "https://any.example", undefined, { firstContentTimeoutMs: 180000 }), { initialMs: 180000, gapMs: 180000 }); + // global gap-only (valid: initial stays off) + assert.deepEqual(resolveFirstContentDeadlines({}, "https://any.example", undefined, { contentGapTimeoutMs: 300000 }), { gapMs: 300000 }); + // route beats global + assert.deepEqual(resolveFirstContentDeadlines(ROUTES, "https://relay-a.example/v1/messages", undefined, { firstContentTimeoutMs: 180000 }), { initialMs: 600000, gapMs: 600000 }); + // model beats route + assert.deepEqual(resolveFirstContentDeadlines(ROUTES, "https://relay-a.example/v1/messages", "gpt-x", { firstContentTimeoutMs: 180000 }), { initialMs: 90000, gapMs: 90000 }); + // model ignored when the URL matches no route + assert.deepEqual(resolveFirstContentDeadlines(ROUTES, "https://other.example", "gpt-x", { firstContentTimeoutMs: 180000 }), { initialMs: 180000, gapMs: 180000 }); + // explicit 0 at route level disables the phase (no fall-through to global); + // gap then inherits nothing → feature fully off + assert.equal(resolveFirstContentDeadlines(ROUTES, "https://relay-b.example", undefined, { firstContentTimeoutMs: 180000 }), undefined); + // explicit 0 for gap only → initial survives, gap phase off + assert.deepEqual(resolveFirstContentDeadlines(ROUTES, "https://relay-c.example", undefined, undefined), { initialMs: 180000 }); + // garbage at ANY tier rejects loudly (the global tier bypasses load-time parsing) + assert.throws(() => resolveFirstContentDeadlines({}, "https://any.example", undefined, { firstContentTimeoutMs: "fast" }), /firstContentTimeoutMs must be a non-negative integer/); +}); + +// --- G: load-time validation of providers entries (route + model level) ---- + +test("#3254 G: loadRoutes rejects malformed deadline values at provider and model level, accepts valid ones", () => { + const dir = mkdtempSync(join(tmpdir(), "bili3254-")); + const path = join(dir, "providers.json"); + const load = (obj: unknown) => { + writeFileSync(path, JSON.stringify(obj)); + return loadRoutes({ ACP_PROVIDERS: path }); + }; + assert.throws(() => load({ "https://relay-a.example": { firstContentTimeoutMs: -5 } }), /providers entry firstContentTimeoutMs must be a non-negative integer/); + assert.throws(() => load({ "https://relay-a.example": { models: { m: { contentGapTimeoutMs: "soon" } } } }), /providers entry models\.m\.contentGapTimeoutMs must be a non-negative integer/); + const ok = load({ "https://relay-a.example": { firstContentTimeoutMs: 0, models: { m: { contentGapTimeoutMs: 120000 } } } }); + assert.equal(ok["https://relay-a.example"]?.firstContentTimeoutMs, 0); + assert.equal(ok["https://relay-a.example"]?.models?.["m"]?.contentGapTimeoutMs, 120000); +}); diff --git a/website/config-reference/process.yaml b/website/config-reference/process.yaml index 8159ef2030..7d689aff6a 100644 --- a/website/config-reference/process.yaml +++ b/website/config-reference/process.yaml @@ -10,8 +10,8 @@ entries: site_group: proc desc_en: "Transport timeouts, watchdogs and replay policy for upstream calls." desc_zh: "上游调用的传输超时、看门狗与重放策略。" - site_en: "network — transport timeouts & replays: upstreamTimeoutMs 720000 · requestWatchdogMs 2× · replayRetryMax 3 · replayRetryBaseMs 1500 · keepAliveTimeoutMs 5000 · clientErrorBackstopMs 30000 · exposureLogIntervalMs 3600000 · streamKeepAliveMs 15000 · preflightHoldMs 30000 · preflightDeadEndCooldownMs 300000 · proxyKeepAliveMaxMs 55000." - site_zh: "network —— 传输超时与重放:upstreamTimeoutMs 720000 · requestWatchdogMs 2× · replayRetryMax 3 · replayRetryBaseMs 1500 · keepAliveTimeoutMs 5000 · clientErrorBackstopMs 30000 · exposureLogIntervalMs 3600000 · streamKeepAliveMs 15000 · preflightHoldMs 30000 · preflightDeadEndCooldownMs 300000 · proxyKeepAliveMaxMs 55000。" + site_en: "network — transport timeouts & replays: upstreamTimeoutMs 720000 · requestWatchdogMs 2× · replayRetryMax 3 · replayRetryBaseMs 1500 · keepAliveTimeoutMs 5000 · clientErrorBackstopMs 30000 · exposureLogIntervalMs 3600000 · streamKeepAliveMs 15000 · preflightHoldMs 30000 · preflightDeadEndCooldownMs 300000 · proxyKeepAliveMaxMs 55000 · firstContentTimeoutMs unset (#3254, file-only) · contentGapTimeoutMs unset (#3254, file-only)." + site_zh: "network —— 传输超时与重放:upstreamTimeoutMs 720000 · requestWatchdogMs 2× · replayRetryMax 3 · replayRetryBaseMs 1500 · keepAliveTimeoutMs 5000 · clientErrorBackstopMs 30000 · exposureLogIntervalMs 3600000 · streamKeepAliveMs 15000 · preflightHoldMs 30000 · preflightDeadEndCooldownMs 300000 · proxyKeepAliveMaxMs 55000 · firstContentTimeoutMs 未设置(#3254,仅文件)· contentGapTimeoutMs 未设置(#3254,仅文件)。" - key: network.upstreamTimeoutMs type: number default: "720000" @@ -68,6 +68,18 @@ entries: group: process desc_en: "Cooldown after a dead-end preflight before retrying reconciliation for the same session." desc_zh: "死胡同 preflight 后,对同一会话重试校准的冷却时长。" + - key: network.firstContentTimeoutMs + type: number + default: "unset (off)" + group: process + desc_en: "Model-content deadline, phase 1 (#3254): max ms an upstream attempt may go with zero model content from response start before bili aborts the stalled stream and ends the turn with one clean in-band error (no internal refetch — the client's retry ladder re-arms). Also settable per route / per model under providers; cascade model > route > global, explicit 0 wins at its level. File-only by design." + desc_zh: "模型内容期限,阶段一(#3254):上游一次尝试从响应开始到首个模型内容之间的最长毫秒数;超时则 bili 中止卡死流并以一条干净的带内错误结束回合(不做内部盲重取——客户端重试梯照常接管)。providers 内也可按线路/按模型设置;级联 model > route > global,显式 0 在所在层生效。按设计仅配置文件可设。" + - key: network.contentGapTimeoutMs + type: number + default: "unset (inherits firstContentTimeoutMs)" + group: process + desc_en: "Model-content deadline, phase 2 (#3254): max ms between consecutive model-content events after the first arrived; re-armed on every content event (pings/comments never count). Absent inherits the resolved firstContentTimeoutMs; explicit 0 disables the gap phase only. Same three-level cascade and file-only rationale as firstContentTimeoutMs." + desc_zh: "模型内容期限,阶段二(#3254):首个内容到达之后,相邻两个模型内容事件之间的最长毫秒数;每个内容事件重新计时(ping/注释不算内容)。缺省继承解析后的 firstContentTimeoutMs;显式 0 仅禁用阶段二。三级级联与仅文件可设的理由同 firstContentTimeoutMs。" - key: network.replayRetryMax type: number default: "3 (1 disables replays)" diff --git a/website/config-reference/providers.yaml b/website/config-reference/providers.yaml index 7a10abbbac..9a5dc971ec 100644 --- a/website/config-reference/providers.yaml +++ b/website/config-reference/providers.yaml @@ -176,6 +176,26 @@ entries: desc_zh: "按路由的 k̂ 校准策略(#3211):对虚报报数线路做退出或钳制;见[线路级 k̂ 校准策略](#route-level-k-calibration-policy)。" site_en: "Per-route k̂ calibration policy (#3211): `disabled` suppresses the learned factor on this route; `max` clamps it (0 < max ≤ 4). Applied at publication time — the learned ring stays raw." site_zh: "按路由的 k̂ 校准策略(#3211):`disabled` 在该线路禁用学习系数;`max` 钳制学习系数(0 < max ≤ 4)。作用于发布环节——学习环保持原始值。" + - key: firstContentTimeoutMs + type: number + default: "unset (off)" + group: providers + id: cr.fcdeadline + render: route-row + desc_en: "Per-route model-content deadline, phase 1 (#3254): max ms without model content from response start before bili aborts the stalled upstream and ends the turn in-band. Also settable per model under models[]; cascade model > route > global network.*, explicit 0 wins at its level. File-only (no env)." + desc_zh: "按线路的模型内容期限,阶段一(#3254):响应开始后无模型内容的上限毫秒数;超时则中止卡死上游并以带内错误结束回合。也可在 models[] 按模型设置;级联 model > route > global network.*,显式 0 在所在层生效。仅配置文件可设(无 env)。" + site_en: "Model-content deadline, phase 1 (#3254), per route and per model (`models[]`); on expiry bili aborts the stalled upstream stream and ends the turn with one clean in-band error — no internal refetch. Cascade model > route > global `network.*`; explicit 0 wins at its level; file-only." + site_zh: "模型内容期限,阶段一(#3254),按线路与按模型(`models[]`);到期则中止卡死的上游流并以一条干净的带内错误结束回合——不做内部盲重取。级联 model > route > global `network.*`;显式 0 在所在层生效;仅文件可设。" + - key: contentGapTimeoutMs + type: number + default: "unset (inherits firstContentTimeoutMs)" + group: providers + id: cr.gapdeadline + render: route-row + desc_en: "Per-route model-content deadline, phase 2 (#3254): max ms between consecutive model-content events after the first arrived; re-armed on every content event. Absent inherits the resolved firstContentTimeoutMs; explicit 0 disables the gap phase only. Same cascade and file-only rationale as firstContentTimeoutMs." + desc_zh: "按线路的模型内容期限,阶段二(#3254):首个内容之后相邻模型内容事件间的上限毫秒数,逐事件重新计时。缺省继承解析后的 firstContentTimeoutMs;显式 0 仅禁用阶段二。级联与仅文件可设的理由同 firstContentTimeoutMs。" + site_en: "Model-content deadline, phase 2 (#3254), per route and per model (`models[]`); bounds silence between content events after the first arrived (pings/comments never count); absent inherits the resolved `firstContentTimeoutMs`. File-only." + site_zh: "模型内容期限,阶段二(#3254),按线路与按模型(`models[]`);约束首个内容之后相邻内容事件间的静默(ping/注释不算内容);缺省继承解析后的 `firstContentTimeoutMs`。仅文件可设。" - key: compactionOptIn type: boolean default: "false" From e8f19974b565c88c45ba7f778ce3758b6747d2f8 Mon Sep 17 00:00:00 2001 From: awork Date: Thu, 15 Oct 2026 16:33:49 +0800 Subject: [PATCH 2/7] fix: regenerate config docs from seeds (#3264) docs-sync CI lane failed on the PR head: the #3254 seed entries in website/config-reference/{process,providers}.yaml were added, but the derived artifacts (CONFIGURATION.md/.zh-CN.md index tables, website/main.js i18n strings, website/docs/index.html) were hand-edited instead of regenerated, so they drifted from generator output: - network.firstContentTimeoutMs/contentGapTimeoutMs index rows landed after postResponseLingerMs by hand; the generator emits them in seed order (after preflightDeadEndCooldownMs); - the two provider-level index rows were missing entirely; - cr.fcdeadline/cr.gapdeadline i18n entries and the tt.network summary line were absent from website/main.js + index.html. Re-ran tools/gen-config-docs.mjs generate and committed its output; the seed text (already authored in this PR) is the source of truth. --- CONFIGURATION.md | 6 ++++-- CONFIGURATION.zh-CN.md | 6 ++++-- website/docs/index.html | 3 +++ website/main.js | 6 ++++++ 4 files changed, 17 insertions(+), 4 deletions(-) diff --git a/CONFIGURATION.md b/CONFIGURATION.md index 598ba07987..8e7fd1242a 100644 --- a/CONFIGURATION.md +++ b/CONFIGURATION.md @@ -147,13 +147,13 @@ This index is generated from `website/config-reference/*.yaml` — edit the seed | `network.streamKeepAliveMs` | number | 15000 (0 disables) | BILI_STREAM_KEEPALIVE_MS | Downstream stream keep-alive ping interval between model chunks; 0 disables pinging. | | `network.preflightHoldMs` | number | 30000 | BILI_PREFLIGHT_HOLD_MS | How long a failed preflight holds before the request is allowed through unreconciled. | | `network.preflightDeadEndCooldownMs` | number | 300000 | BILI_PREFLIGHT_DEAD_END_COOLDOWN_MS | Cooldown after a dead-end preflight before retrying reconciliation for the same session. | +| `network.firstContentTimeoutMs` | number | unset (off) | — | Model-content deadline, phase 1 (#3254): max ms an upstream attempt may go with zero model content from response start before bili aborts the stalled stream and ends the turn with one clean in-band error (no internal refetch — the client's retry ladder re-arms). Also settable per route / per model under providers; cascade model > route > global, explicit 0 wins at its level. File-only by design. | +| `network.contentGapTimeoutMs` | number | unset (inherits firstContentTimeoutMs) | — | Model-content deadline, phase 2 (#3254): max ms between consecutive model-content events after the first arrived; re-armed on every content event (pings/comments never count). Absent inherits the resolved firstContentTimeoutMs; explicit 0 disables the gap phase only. Same three-level cascade and file-only rationale as firstContentTimeoutMs. | | `network.replayRetryMax` | number | 3 (1 disables replays) | BILI_REPLAY_RETRY_MAX | Maximum transport-failure replays of an identical request before surfacing the error. | | `network.replayRetryBaseMs` | number | 1500 (0 disables the delay) | BILI_REPLAY_RETRY_BASE_MS | Base backoff delay between replays (exponential). | | `network.maxShrinkPerCompress` | number | unset | BILI_MAX_SHRINK_PER_COMPRESS | Cap on how much one compression may shrink history (anti-collapse guard). | | `network.proxyKeepAliveMaxMs` | number | 55000 (0 = one-shot connections) | BILI_PROXY_KEEPALIVE_MAX_MS | Max lifetime of a connection through an outbound egress proxy before forcing reconnection (avoids stale NAT mappings). | | `network.postResponseLingerMs` | number | 5000 | BILI_POST_RESPONSE_LINGER_MS | Graceful-close budget for proxy-initiated closes after the response completed: the socket is held waiting for the peer's FIN/TLS close_notify and destroyed at the deadline (reason=linger-backstop). | -| `network.firstContentTimeoutMs` | number | unset (off) | — | Model-content deadline, phase 1 (#3254): max ms an upstream attempt may go with zero model content (text/reasoning deltas, tool calls, signatures) from response start before bili aborts it and ends the turn with an in-band error. Also per route (`providers[]`) and per model (`providers[].models[]`); cascade model > route > global, explicit 0 wins at its level. File-only by design (per-route values can't be one process-wide env var). | -| `network.contentGapTimeoutMs` | number | unset (inherits firstContentTimeoutMs) | — | Model-content deadline, phase 2 (#3254): max ms between two consecutive model-content events after the first arrived; re-armed on every content event (pings/comments never count). Absent → inherits the resolved `firstContentTimeoutMs`; 0 disables the gap phase only. Same three-level cascade and file-only rationale as `firstContentTimeoutMs`. | | `network.allowPrivateHttpProviders` | boolean | false | BILI_ALLOW_PRIVATE_HTTP_PROVIDERS | Opt-in (#2984): admit plain-HTTP providers on private networks (RFC1918, 127/8, link-local, ULA, *.local/.lan/.internal) into the agent-registry intake and summary-endpoint guards. Off by default — plaintext stays loopback-only; public IPs/domains and CGNAT (100.64/10) stay refused regardless. 1/true enables. | | `network.receiveBodyBytes` | number | 5368709120 | BILI_RECEIVE_BODY_BYTES | Max inbound wire bytes read into memory per request before a 413 at the receive stage. Raised from a hardcoded 100 MiB to 5 GiB (#3128 owner decision) so multi-gigabyte single requests are accepted instead of the socket being reset. Also the threshold above which a decoded request consumes one of two shared large-decode admission slots. WebSocket lanes keep the fixed 100 MiB surface. | | `network.decodedBodyBytes` | number | 10737418240 | BILI_DECODED_BODY_BYTES | Max DECODED bytes after content-encoding expansion (gzip/deflate/br/zstd), checked incrementally during decode (decode-stage 413). Default is 2x the receive budget, mirroring the legacy decode=2x-wire relationship; guards against decompression bombs. | @@ -310,6 +310,8 @@ This index is generated from `website/config-reference/*.yaml` — edit the seed | `apiKeyEnv` | string (env var name) | unset | — | Lane credential override (#2336): replace the client's credential with this env variable's value for this lane. Exactly one of apiKeyEnv/credentialRef; see [Lane credentials](#lane-credentials-apikeyenv--credentialref). | | `credentialRef` | string (store name) | unset | — | Lane credential override via the private summary-credential store (secret:NAME semantics). | | `calibration` | { disabled?: boolean, max?: number } | unset (learned k̂ applies) | — | Per-route k̂ calibration policy (#3211): opt-out or clamp for routes whose usage reports inflate; see [Route-level k̂ calibration policy](#route-level-k-calibration-policy). | +| `firstContentTimeoutMs` | number | unset (off) | — | Per-route model-content deadline, phase 1 (#3254): max ms without model content from response start before bili aborts the stalled upstream and ends the turn in-band. Also settable per model under models[]; cascade model > route > global network.*, explicit 0 wins at its level. File-only (no env). | +| `contentGapTimeoutMs` | number | unset (inherits firstContentTimeoutMs) | — | Per-route model-content deadline, phase 2 (#3254): max ms between consecutive model-content events after the first arrived; re-armed on every content event. Absent inherits the resolved firstContentTimeoutMs; explicit 0 disables the gap phase only. Same cascade and file-only rationale as firstContentTimeoutMs. | | `compactionOptIn` | boolean | false | BILI_NON_HTTP_PROVIDERS | Named entries only: opt a non-http(s)-baseUrl provider into compaction ownership (pi/omp lanes); unions with env BILI_NON_HTTP_PROVIDERS. | **Environment-only variables** diff --git a/CONFIGURATION.zh-CN.md b/CONFIGURATION.zh-CN.md index 4d529ba838..a5ff74442a 100644 --- a/CONFIGURATION.zh-CN.md +++ b/CONFIGURATION.zh-CN.md @@ -147,13 +147,13 @@ | `network.streamKeepAliveMs` | number | 15000 (0 disables) | BILI_STREAM_KEEPALIVE_MS | 模型块之间的下游流保活心跳间隔;0 禁用心跳。 | | `network.preflightHoldMs` | number | 30000 | BILI_PREFLIGHT_HOLD_MS | preflight 失败后,允许请求未校准放行的持有时长。 | | `network.preflightDeadEndCooldownMs` | number | 300000 | BILI_PREFLIGHT_DEAD_END_COOLDOWN_MS | 死胡同 preflight 后,对同一会话重试校准的冷却时长。 | +| `network.firstContentTimeoutMs` | number | unset (off) | — | 模型内容期限,阶段一(#3254):上游一次尝试从响应开始到首个模型内容之间的最长毫秒数;超时则 bili 中止卡死流并以一条干净的带内错误结束回合(不做内部盲重取——客户端重试梯照常接管)。providers 内也可按线路/按模型设置;级联 model > route > global,显式 0 在所在层生效。按设计仅配置文件可设。 | +| `network.contentGapTimeoutMs` | number | unset (inherits firstContentTimeoutMs) | — | 模型内容期限,阶段二(#3254):首个内容到达之后,相邻两个模型内容事件之间的最长毫秒数;每个内容事件重新计时(ping/注释不算内容)。缺省继承解析后的 firstContentTimeoutMs;显式 0 仅禁用阶段二。三级级联与仅文件可设的理由同 firstContentTimeoutMs。 | | `network.replayRetryMax` | number | 3 (1 disables replays) | BILI_REPLAY_RETRY_MAX | 同一请求在向上游暴露错误前的最大传输失败重放次数。 | | `network.replayRetryBaseMs` | number | 1500 (0 disables the delay) | BILI_REPLAY_RETRY_BASE_MS | 重放之间的基础退避延迟(指数增长)。 | | `network.maxShrinkPerCompress` | number | unset | BILI_MAX_SHRINK_PER_COMPRESS | 单次压缩允许收缩历史的上限(防塌缩护栏)。 | | `network.proxyKeepAliveMaxMs` | number | 55000 (0 = one-shot connections) | BILI_PROXY_KEEPALIVE_MAX_MS | 经出站代理的连接最大存活时长,到期强制重建(规避陈旧 NAT 映射)。 | | `network.postResponseLingerMs` | number | 5000 | BILI_POST_RESPONSE_LINGER_MS | 代理主动发起的响应后关闭的优雅关闭预算:socket 挂起等待对端 FIN/TLS close_notify,到点销毁(reason=linger-backstop)。 | -| `network.firstContentTimeoutMs` | number | 未设置(关闭) | — | 模型内容期限,阶段一(#3254):上游一次尝试从响应开始到首个模型内容(文本/推理增量、工具调用、签名)之间的最长毫秒数;超时则 bili 中止卡死流并以带内错误结束回合。也可按线路(`providers[]`)与按模型(`providers[].models[]`)设置;级联 model > route > global,显式 0 在所在层生效。按设计仅配置文件可设(每线路取值无法用单一进程级环境变量表达)。 | -| `network.contentGapTimeoutMs` | number | 未设置(继承 firstContentTimeoutMs) | — | 模型内容期限,阶段二(#3254):首个内容到达之后,相邻两个模型内容事件之间的最长毫秒数;每个内容事件重新计时(ping/注释不算内容)。缺省继承解析后的 `firstContentTimeoutMs`;0 仅禁用阶段二。三级级联与仅文件可设的理由同 `firstContentTimeoutMs`。 | | `network.allowPrivateHttpProviders` | boolean | false | BILI_ALLOW_PRIVATE_HTTP_PROVIDERS | 可选开关(#2984):允许私网(RFC1918、127/8、link-local、ULA、*.local/.lan/.internal)明文 HTTP provider 进入 agent-registry 接收与摘要端点守卫。默认关——明文仅回环受信;公网 IP/域名与 CGNAT(100.64/10)无论如何都拒。1/true 启用。 | | `network.receiveBodyBytes` | number | 5368709120 | BILI_RECEIVE_BODY_BYTES | 单请求读入内存的入站 wire 字节上限,超限在 receive 阶段返回 413。由硬编码 100 MiB 提升到 5 GiB(#3128 owner 决定),使多 GB 单请求被接受而非断连。同时也是解码请求占用两个共享大解码准入槽位的阈值。WebSocket lane 保持固定 100 MiB 表面。 | | `network.decodedBodyBytes` | number | 10737418240 | BILI_DECODED_BODY_BYTES | content-encoding 解压(gzip/deflate/br/zstd)后的解码字节上限,解码过程中增量检查(decode 阶段 413)。默认为接收预算的 2 倍,沿用旧的 decode=2×wire 关系;用于防解压炸弹。 | @@ -310,6 +310,8 @@ | `apiKeyEnv` | string (env var name) | unset | — | 通道凭据覆盖(#2336):用该环境变量的值替换客户端凭据。与 credentialRef 二选一;见[通道凭据](#lane-credentials-apikeyenv--credentialref)。 | | `credentialRef` | string (store name) | unset | — | 经私有摘要凭据存储的通道凭据覆盖(secret:NAME 语义)。 | | `calibration` | { disabled?: boolean, max?: number } | unset (learned k̂ applies) | — | 按路由的 k̂ 校准策略(#3211):对虚报报数线路做退出或钳制;见[线路级 k̂ 校准策略](#route-level-k-calibration-policy)。 | +| `firstContentTimeoutMs` | number | unset (off) | — | 按线路的模型内容期限,阶段一(#3254):响应开始后无模型内容的上限毫秒数;超时则中止卡死上游并以带内错误结束回合。也可在 models[] 按模型设置;级联 model > route > global network.*,显式 0 在所在层生效。仅配置文件可设(无 env)。 | +| `contentGapTimeoutMs` | number | unset (inherits firstContentTimeoutMs) | — | 按线路的模型内容期限,阶段二(#3254):首个内容之后相邻模型内容事件间的上限毫秒数,逐事件重新计时。缺省继承解析后的 firstContentTimeoutMs;显式 0 仅禁用阶段二。级联与仅文件可设的理由同 firstContentTimeoutMs。 | | `compactionOptIn` | boolean | false | BILI_NON_HTTP_PROVIDERS | 仅命名条目:把非 http(s) baseUrl 供应商纳入压缩所有权(pi/omp 车道);与 env BILI_NON_HTTP_PROVIDERS 取并集。 | **仅环境变量(无配置文件键)** diff --git a/website/docs/index.html b/website/docs/index.html index ce0f9b00ba..bc27acf79d 100644 --- a/website/docs/index.html +++ b/website/docs/index.html @@ -239,6 +239,8 @@

providers — route table fields

apiKeyEnvstring (env var name)Lane credential override (#2336): requests forwarded through this lane carry this env variable's value instead of the client's own credential — x-api-key / x-goog-api-key replaced outright, Bearer authorization replaced, injected when absent. Exactly one of apiKeyEnv / credentialRef per entry. credentialRefstring (store name)Lane credential override via the private summary-credential store (`secret:NAME` semantics); exactly one of apiKeyEnv / credentialRef per entry. calibration{ disabled?: boolean, max?: number }Per-route k̂ calibration policy (#3211): `disabled` suppresses the learned factor on this route; `max` clamps it (0 < max ≤ 4). Applied at publication time — the learned ring stays raw. + firstContentTimeoutMsnumberModel-content deadline, phase 1 (#3254), per route and per model (`models[]`); on expiry bili aborts the stalled upstream stream and ends the turn with one clean in-band error — no internal refetch. Cascade model > route > global `network.*`; explicit 0 wins at its level; file-only. + contentGapTimeoutMsnumberModel-content deadline, phase 2 (#3254), per route and per model (`models[]`); bounds silence between content events after the first arrived (pings/comments never count); absent inherits the resolved `firstContentTimeoutMs`. File-only. compactionOptInbooleanNamed entries only: opt a non-http(s)-baseUrl provider into compaction ownership (pi/omp lanes); unions with env BILI_NON_HTTP_PROVIDERS. @@ -278,6 +280,7 @@

Other top-level fields

  • mitm.enabled / mitm.domains — HTTPS interception switch and whitelist domains for login clients.
  • mitm.handshakeTimeoutMs — TLS handshake timeout (default 10000).
  • Process blocks
  • +
  • network — transport timeouts & replays: upstreamTimeoutMs 720000 · requestWatchdogMs 2× · replayRetryMax 3 · replayRetryBaseMs 1500 · keepAliveTimeoutMs 5000 · clientErrorBackstopMs 30000 · exposureLogIntervalMs 3600000 · streamKeepAliveMs 15000 · preflightHoldMs 30000 · preflightDeadEndCooldownMs 300000 · proxyKeepAliveMaxMs 55000 · firstContentTimeoutMs unset (#3254, file-only) · contentGapTimeoutMs unset (#3254, file-only).
  • network — transport timeouts & replays: upstreamTimeoutMs 720000 · requestWatchdogMs 2× · replayRetryMax 3 · replayRetryBaseMs 1500 · keepAliveTimeoutMs 5000 · clientErrorBackstopMs 30000 · exposureLogIntervalMs 3600000 · streamKeepAliveMs 15000 · preflightHoldMs 30000 · preflightDeadEndCooldownMs 300000 · proxyKeepAliveMaxMs 55000.
  • guardrails — pi/omp plugin tool-output guardrails: bashTimeout 60 · maxOutputBytes 50000 · repetition true.
  • persist — session persistence to disk: enabled true · zstd false · debounceMs 500 · tailTokens 16384 · epermAlertThreshold 5 · epermAlertRepeatMs 0.
  • diff --git a/website/main.js b/website/main.js index c69caa5fa6..9d0b868e24 100644 --- a/website/main.js +++ b/website/main.js @@ -109,6 +109,8 @@ "cr.lanekey": "Lane credential override (#2336): requests forwarded through this lane carry this env variable's value instead of the client's own credential — x-api-key / x-goog-api-key replaced outright, Bearer authorization replaced, injected when absent. Exactly one of apiKeyEnv / credentialRef per entry.", "cr.lanecred": "Lane credential override via the private summary-credential store (`secret:NAME` semantics); exactly one of apiKeyEnv / credentialRef per entry.", "cr.calib": "Per-route k̂ calibration policy (#3211): `disabled` suppresses the learned factor on this route; `max` clamps it (0 < max ≤ 4). Applied at publication time — the learned ring stays raw.", + "cr.fcdeadline": "Model-content deadline, phase 1 (#3254), per route and per model (`models[]`); on expiry bili aborts the stalled upstream stream and ends the turn with one clean in-band error — no internal refetch. Cascade model > route > global `network.*`; explicit 0 wins at its level; file-only.", + "cr.gapdeadline": "Model-content deadline, phase 2 (#3254), per route and per model (`models[]`); bounds silence between content events after the first arrived (pings/comments never count); absent inherits the resolved `firstContentTimeoutMs`. File-only.", "cr.optin": "Named entries only: opt a non-http(s)-baseUrl provider into compaction ownership (pi/omp lanes); unions with env BILI_NON_HTTP_PROVIDERS.", "tg.core": "Server & core", "tg.up": "Upstream & billing", @@ -122,6 +124,7 @@ "tt.ccx": "codexCompact — intercept | pass (default intercept): handle Codex's compaction requests in bili or forward upstream — explicit forms plus the inline auto-compact transiting as an ordinary turn (prompt-form detection, #3008; overridden compact_prompt transits verbatim).", "ttl.ttl": "ccrRetrievalTtlMs — lifetime of CCR retrieval pointers (default 600000; 0 disables retrieval).", "ttl.tmp": "decompressTmpCap — size cap for temporary decompressed blocks (default 50).", + "tt.network": "network — transport timeouts & replays: upstreamTimeoutMs 720000 · requestWatchdogMs 2× · replayRetryMax 3 · replayRetryBaseMs 1500 · keepAliveTimeoutMs 5000 · clientErrorBackstopMs 30000 · exposureLogIntervalMs 3600000 · streamKeepAliveMs 15000 · preflightHoldMs 30000 · preflightDeadEndCooldownMs 300000 · proxyKeepAliveMaxMs 55000 · firstContentTimeoutMs unset (#3254, file-only) · contentGapTimeoutMs unset (#3254, file-only).", "tt.network": "network — transport timeouts & replays: upstreamTimeoutMs 720000 · requestWatchdogMs 2× · replayRetryMax 3 · replayRetryBaseMs 1500 · keepAliveTimeoutMs 5000 · clientErrorBackstopMs 30000 · exposureLogIntervalMs 3600000 · streamKeepAliveMs 15000 · preflightHoldMs 30000 · preflightDeadEndCooldownMs 300000 · proxyKeepAliveMaxMs 55000.", "tt.guardrails": "guardrails — pi/omp plugin tool-output guardrails: bashTimeout 60 · maxOutputBytes 50000 · repetition true.", "tt.persist": "persist — session persistence to disk: enabled true · zstd false · debounceMs 500 · tailTokens 16384 · epermAlertThreshold 5 · epermAlertRepeatMs 0.", @@ -348,6 +351,8 @@ "cr.lanekey": "通道凭据覆盖(#2336):经该通道转发的请求使用此环境变量的值取代客户端自带凭据——x-api-key / x-goog-api-key 直接替换、Bearer authorization 替换、缺失时注入。每个条目与 credentialRef 二选一。", "cr.lanecred": "经私有摘要凭据存储的通道凭据覆盖(`secret:NAME` 语义);每个条目与 apiKeyEnv 二选一。", "cr.calib": "按路由的 k̂ 校准策略(#3211):`disabled` 在该线路禁用学习系数;`max` 钳制学习系数(0 < max ≤ 4)。作用于发布环节——学习环保持原始值。", + "cr.fcdeadline": "模型内容期限,阶段一(#3254),按线路与按模型(`models[]`);到期则中止卡死的上游流并以一条干净的带内错误结束回合——不做内部盲重取。级联 model > route > global `network.*`;显式 0 在所在层生效;仅文件可设。", + "cr.gapdeadline": "模型内容期限,阶段二(#3254),按线路与按模型(`models[]`);约束首个内容之后相邻内容事件间的静默(ping/注释不算内容);缺省继承解析后的 `firstContentTimeoutMs`。仅文件可设。", "cr.optin": "仅命名条目:把非 http(s) baseUrl 供应商纳入压缩所有权(pi/omp 车道);与 env BILI_NON_HTTP_PROVIDERS 取并集。", "tg.core": "服务与核心", "tg.up": "上游与计费", @@ -361,6 +366,7 @@ "tt.ccx": "codexCompact —— intercept | pass(默认 intercept):在 bili 内处理 Codex 压缩请求或转给上游 —— 显式形态 + 以普通请求形态过线的内联自动压缩(prompt 文本检测,#3008;自定义 compact_prompt 按原样转发)。", "ttl.ttl": "ccrRetrievalTtlMs —— CCR 回取指针有效期(默认 600000;0 禁用回取)。", "ttl.tmp": "decompressTmpCap —— 临时展开块的大小上限(默认 50)。", + "tt.network": "network —— 传输超时与重放:upstreamTimeoutMs 720000 · requestWatchdogMs 2× · replayRetryMax 3 · replayRetryBaseMs 1500 · keepAliveTimeoutMs 5000 · clientErrorBackstopMs 30000 · exposureLogIntervalMs 3600000 · streamKeepAliveMs 15000 · preflightHoldMs 30000 · preflightDeadEndCooldownMs 300000 · proxyKeepAliveMaxMs 55000 · firstContentTimeoutMs 未设置(#3254,仅文件)· contentGapTimeoutMs 未设置(#3254,仅文件)。", "tt.network": "network —— 传输超时与重放:upstreamTimeoutMs 720000 · requestWatchdogMs 2× · replayRetryMax 3 · replayRetryBaseMs 1500 · keepAliveTimeoutMs 5000 · clientErrorBackstopMs 30000 · exposureLogIntervalMs 3600000 · streamKeepAliveMs 15000 · preflightHoldMs 30000 · preflightDeadEndCooldownMs 300000 · proxyKeepAliveMaxMs 55000。", "tt.guardrails": "guardrails —— pi/omp 插件工具输出守卫:bashTimeout 60 · maxOutputBytes 50000 · repetition true。", "tt.persist": "persist —— 会话落盘持久化:enabled true · zstd false · debounceMs 500 · tailTokens 16384 · epermAlertThreshold 5 · epermAlertRepeatMs 0。", From 1d84728ed49c934981729ce3f21f79ef3dc3b9c4 Mon Sep 17 00:00:00 2001 From: awork Date: Thu, 15 Oct 2026 16:36:04 +0800 Subject: [PATCH 3/7] fix: resolve the model tier from the path model on the Google wire (#3264) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The per-request cascade read the model from the request body only. Gemini-native requests carry their model in the URL path, not the body, so providers[].models[] deadlines could never match a google-wire request — the model tier silently never applied there, the same silent-failure class this feature rejects for invalid values. Fall back to prepared.google.model (path-derived) when the body has no model field; non-google requests are unaffected (prepared.google is undefined). --- src/server/relay.ts | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/src/server/relay.ts b/src/server/relay.ts index 9c86f7a295..f5c08b0fcc 100644 --- a/src/server/relay.ts +++ b/src/server/relay.ts @@ -1498,7 +1498,10 @@ export async function forward( // #3254: model-content deadlines — three-level cascade (model > // route > global network.*), resolved per request so config edits // apply without a restart; absent everywhere → undefined → legacy. - const firstContentDeadline = resolveFirstContentDeadlines(opts.routes, loopBillingUpstream, (parsedReq as { model?: string }).model, loadConfigFile().network); + // The Google wire carries its model in the PATH, not the body + // (prepared.google.model) — without the fallback the model tier + // could never match a Gemini-native request. + const firstContentDeadline = resolveFirstContentDeadlines(opts.routes, loopBillingUpstream, (parsedReq as { model?: string }).model ?? prepared.google?.model, loadConfigFile().network); const loop = runCompressLoop( responseBody, { core, config, messages: prepared.processedMessages.length > 0 ? prepared.processedMessages : prepared.originalMessages, compressMessages: prepared.originalMessages, session: prepared.session, log: ctx.log, proxyUrl, skippedProxy, upstreamOrigin: targetOrigin, protocol: prepared.protocol, textProtocol, debug: opts.debug, refreshFolded, visibilityMarkers, dumpSse: loopDumpDir ? (name, stream) => dumpStreamToFile(stream, loopDumpDir, name) : undefined, imageLearn: { host: upstreamHost(loopBillingUpstream), fp: `${imageBillingFor(opts, loopBillingUpstream)}:${imageTokenCapFor(opts, loopBillingUpstream)}` }, firstContentDeadline }, From f84af585b0daf1178dc7332ec213114f73a413ac Mon Sep 17 00:00:00 2001 From: awork Date: Thu, 15 Oct 2026 16:36:29 +0800 Subject: [PATCH 4/7] docs: drop stray double space in network Type line (#3264) Cosmetic: the #3254 edit left five spaces between proxyKeepAliveMaxMs and postResponseLingerMs in the hand-maintained `### network` Type line of CONFIGURATION.md and CONFIGURATION.zh-CN.md. --- CONFIGURATION.md | 2 +- CONFIGURATION.zh-CN.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/CONFIGURATION.md b/CONFIGURATION.md index 8e7fd1242a..50bc14957c 100644 --- a/CONFIGURATION.md +++ b/CONFIGURATION.md @@ -851,7 +851,7 @@ Since #2030 every pure-behavior knob has a config-file key alongside its env var ### `network` -- **Type:** `{ upstreamTimeoutMs?: number; requestWatchdogMs?: number; replayRetryMax?: number; replayRetryBaseMs?: number; maxShrinkPerCompress?: number; keepAliveTimeoutMs?: number; clientErrorBackstopMs?: number; exposureLogIntervalMs?: number; streamKeepAliveMs?: number; preflightHoldMs?: number; preflightDeadEndCooldownMs?: number; proxyKeepAliveMaxMs?: number; postResponseLingerMs?: number; firstContentTimeoutMs?: number; contentGapTimeoutMs?: number; allowPrivateHttpProviders?: boolean }` +- **Type:** `{ upstreamTimeoutMs?: number; requestWatchdogMs?: number; replayRetryMax?: number; replayRetryBaseMs?: number; maxShrinkPerCompress?: number; keepAliveTimeoutMs?: number; clientErrorBackstopMs?: number; exposureLogIntervalMs?: number; streamKeepAliveMs?: number; preflightHoldMs?: number; preflightDeadEndCooldownMs?: number; proxyKeepAliveMaxMs?: number; postResponseLingerMs?: number; firstContentTimeoutMs?: number; contentGapTimeoutMs?: number; allowPrivateHttpProviders?: boolean }` - **Default:** `{}` (built-ins: `720000`, `2× upstream timeout`, `3`, `1500`, unset, `5000`, `30000`, `3600000`, `15000`, `30000`, `300000`, `55000`, `5000`, `false`) - **Status:** ACTIVE - **Description:** Process-wide transport and retry timing (ms). Every key except `firstContentTimeoutMs`/`contentGapTimeoutMs` (#3254, file-only — see below) resolves env > file > default via its twin (`BILI_UPSTREAM_TIMEOUT_MS`, `BILI_REQUEST_WATCHDOG_MS`, `BILI_REPLAY_RETRY_MAX`, `BILI_REPLAY_RETRY_BASE_MS`, `BILI_MAX_SHRINK_PER_COMPRESS`, `BILI_KEEP_ALIVE_TIMEOUT_MS`, `BILI_CLIENT_ERROR_BACKSTOP_MS`, `BILI_EXPOSURE_LOG_INTERVAL_MS`, `BILI_STREAM_KEEPALIVE_MS`, `BILI_PREFLIGHT_HOLD_MS`, `BILI_PREFLIGHT_DEAD_END_COOLDOWN_MS`, `BILI_PROXY_KEEPALIVE_MAX_MS`, `BILI_POST_RESPONSE_LINGER_MS`, `BILI_ALLOW_PRIVATE_HTTP_PROVIDERS`). Semantics: `upstreamTimeoutMs` bounds time-to-first-byte and inter-chunk silence end-to-end (#551); `requestWatchdogMs` is the total-request budget (a negative value opts out); `replayRetryMax` counts attempts after transient upstream rejections (#189/#1688); `maxShrinkPerCompress` caps how much one compress may shrink the request before bili steers toward smaller ranges (#189); `streamKeepAliveMs` emits an SSE comment when the response has written zero bytes for that long (#1647); `proxyKeepAliveMaxMs` caps connection reuse through an upstream proxy below its recycle cadence (#1263), `0` uncaps; `postResponseLingerMs` is the graceful-close budget after a proxy-initiated post-response close — the socket is held waiting for the peer's FIN/TLS close_notify and destroyed at the deadline (`reason=linger-backstop`) (#1982); `allowPrivateHttpProviders` is an opt-in (#2984): when true, plain-HTTP provider baseUrls on private networks (RFC1918, 127/8, link-local, IPv6 ULA/link-local, `*.local`/`.lan`/`.internal`) are admitted by the agent-registry intake and summary-endpoint guards that otherwise trust loopback plaintext only — public IPs/domains and CGNAT (100.64/10) stay refused regardless, and the default `false` keeps pre-#2984 behavior byte-for-byte; `firstContentTimeoutMs`/`contentGapTimeoutMs` are FILE-ONLY model-content deadlines (#3254): phase 1 bounds how long an upstream attempt may go with zero model content (text/reasoning deltas, tool calls, signatures) from response start, phase 2 (`contentGapTimeoutMs`, re-armed on every content event — pings/comments never count) bounds silence between content events after the first arrived; on expiry bili aborts the stalled upstream stream and ends the turn with one clean in-band error (context intact, resend your last message) — no internal refetch, so the client's own retry ladder re-arms; cascade model > provider route > global `network.*`, explicit 0 wins at its level, gap inherits the resolved initial when absent; deliberately no env layer because per-route/per-model values cannot be expressed as one process-wide variable. Full details in the [env table](#config-file-keys-for-environment-knobs-2030). diff --git a/CONFIGURATION.zh-CN.md b/CONFIGURATION.zh-CN.md index a5ff74442a..e679033d4a 100644 --- a/CONFIGURATION.zh-CN.md +++ b/CONFIGURATION.zh-CN.md @@ -851,7 +851,7 @@ ### `network` -- **类型:** `{ upstreamTimeoutMs?: number; requestWatchdogMs?: number; replayRetryMax?: number; replayRetryBaseMs?: number; maxShrinkPerCompress?: number; keepAliveTimeoutMs?: number; clientErrorBackstopMs?: number; exposureLogIntervalMs?: number; streamKeepAliveMs?: number; preflightHoldMs?: number; preflightDeadEndCooldownMs?: number; proxyKeepAliveMaxMs?: number; postResponseLingerMs?: number; firstContentTimeoutMs?: number; contentGapTimeoutMs?: number; allowPrivateHttpProviders?: boolean }` +- **类型:** `{ upstreamTimeoutMs?: number; requestWatchdogMs?: number; replayRetryMax?: number; replayRetryBaseMs?: number; maxShrinkPerCompress?: number; keepAliveTimeoutMs?: number; clientErrorBackstopMs?: number; exposureLogIntervalMs?: number; streamKeepAliveMs?: number; preflightHoldMs?: number; preflightDeadEndCooldownMs?: number; proxyKeepAliveMaxMs?: number; postResponseLingerMs?: number; firstContentTimeoutMs?: number; contentGapTimeoutMs?: number; allowPrivateHttpProviders?: boolean }` - **默认:** `{}`(内置值依次为:`720000`、`2× 上游超时`、`3`、`1500`、未设置、`5000`、`30000`、`3600000`、`15000`、`30000`、`300000`、`55000`、`5000`、`false`) - **状态:** ACTIVE - **说明:** 进程级传输与重试时序(毫秒)。除 `firstContentTimeoutMs`/`contentGapTimeoutMs`(#3254,仅文件可设——见下)外,每个键按 env > file > default 解析,对应环境变量分别为 `BILI_UPSTREAM_TIMEOUT_MS`、`BILI_REQUEST_WATCHDOG_MS`、`BILI_REPLAY_RETRY_MAX`、`BILI_REPLAY_RETRY_BASE_MS`、`BILI_MAX_SHRINK_PER_COMPRESS`、`BILI_KEEP_ALIVE_TIMEOUT_MS`、`BILI_CLIENT_ERROR_BACKSTOP_MS`、`BILI_EXPOSURE_LOG_INTERVAL_MS`、`BILI_STREAM_KEEPALIVE_MS`、`BILI_PREFLIGHT_HOLD_MS`、`BILI_PREFLIGHT_DEAD_END_COOLDOWN_MS`、`BILI_PROXY_KEEPALIVE_MAX_MS`、`BILI_POST_RESPONSE_LINGER_MS`、`BILI_ALLOW_PRIVATE_HTTP_PROVIDERS`。要点:`upstreamTimeoutMs` 端到端约束首字节时间与 chunk 间静默(#551);`requestWatchdogMs` 是单请求总预算(负值 = 退出该约束);`replayRetryMax` 统计瞬态上游拒绝后的尝试次数(#189/#1688);`maxShrinkPerCompress` 限制单次压缩对请求的最大缩减比例,超限则引导模型选择更小的范围(#189);`streamKeepAliveMs` 在响应零字节写出超过该时长时发一条 SSE 注释行(#1647);`proxyKeepAliveMaxMs` 把经上游代理的连接复用窗口压在上游回收周期之下(#1263),`0` 取消上限;`postResponseLingerMs` 是代理主动发起的响应后关闭的优雅关闭预算 —— socket 被挂起等待对端 FIN/TLS close_notify,到点销毁(`reason=linger-backstop`)(#1982);`allowPrivateHttpProviders` 是可选开关(#2984):为 true 时,私网(RFC1918、127/8、link-local、IPv6 ULA/link-local、`*.local`/`.lan`/`.internal`)上的明文 HTTP provider baseUrl 被 agent-registry 接收与摘要端点守卫放行——这些守卫默认只信任回环明文;公网 IP/域名与 CGNAT(100.64/10)无论如何都拒;默认 false 保持 #2984 之前的行为逐字不变;`firstContentTimeoutMs`/`contentGapTimeoutMs` 是仅文件可设的模型内容期限(#3254):阶段一约束上游一次尝试从响应开始到首个模型内容(文本/推理增量、工具调用、签名)的最长等待,阶段二(`contentGapTimeoutMs`,每个内容事件重新计时——ping/注释不算内容)约束首个内容之后相邻内容事件间的静默;到期则 bili 中止卡死的上游流并以一条干净的带内错误结束回合(上下文完好,重发最后一条消息即可继续)——不做内部盲重取,客户端自身的重试梯照常接管;级联 model > provider route > global `network.*`,显式 0 在所在层生效,gap 缺省继承解析后的 initial;刻意不设 env 层,因为按线路/按模型的取值无法用单一进程级变量表达。完整细节见[环境变量表](#环境变量的配置键对照-2030)。 From a5cda3c62cae5879b80d9b2d59d8991ee84e6d1b Mon Sep 17 00:00:00 2001 From: awork Date: Thu, 15 Oct 2026 21:44:56 +0800 Subject: [PATCH 5/7] feat: route-adaptive refinement for model-content deadlines (#3254 follow-up) Owner folded the previously filed follow-up (#3265) into this issue: once a route accumulates >=8 healthy attempts, each configured deadline value becomes a ceiling and the effective deadline tightens to clamp(3 x p95 of the route's recent healthy first-content durations, 120s, 600s) per phase. Sampled only from healthy attempts (completed without truncation/error); samples persist in session metadata (ring of 32); fire log records source=config|adaptive. --- CONFIGURATION.md | 7 +- CONFIGURATION.zh-CN.md | 7 +- src/config.ts | 64 +++++++++-- src/loop/core.ts | 96 ++++++++++++++--- src/loop/index.ts | 2 +- src/server/relay.ts | 9 +- .../issue3254-first-content-deadline.test.ts | 101 ++++++++++++++++-- website/config-reference/process.yaml | 8 +- website/config-reference/providers.yaml | 16 +-- 9 files changed, 257 insertions(+), 53 deletions(-) diff --git a/CONFIGURATION.md b/CONFIGURATION.md index 50bc14957c..7b220070be 100644 --- a/CONFIGURATION.md +++ b/CONFIGURATION.md @@ -147,13 +147,14 @@ This index is generated from `website/config-reference/*.yaml` — edit the seed | `network.streamKeepAliveMs` | number | 15000 (0 disables) | BILI_STREAM_KEEPALIVE_MS | Downstream stream keep-alive ping interval between model chunks; 0 disables pinging. | | `network.preflightHoldMs` | number | 30000 | BILI_PREFLIGHT_HOLD_MS | How long a failed preflight holds before the request is allowed through unreconciled. | | `network.preflightDeadEndCooldownMs` | number | 300000 | BILI_PREFLIGHT_DEAD_END_COOLDOWN_MS | Cooldown after a dead-end preflight before retrying reconciliation for the same session. | -| `network.firstContentTimeoutMs` | number | unset (off) | — | Model-content deadline, phase 1 (#3254): max ms an upstream attempt may go with zero model content from response start before bili aborts the stalled stream and ends the turn with one clean in-band error (no internal refetch — the client's retry ladder re-arms). Also settable per route / per model under providers; cascade model > route > global, explicit 0 wins at its level. File-only by design. | -| `network.contentGapTimeoutMs` | number | unset (inherits firstContentTimeoutMs) | — | Model-content deadline, phase 2 (#3254): max ms between consecutive model-content events after the first arrived; re-armed on every content event (pings/comments never count). Absent inherits the resolved firstContentTimeoutMs; explicit 0 disables the gap phase only. Same three-level cascade and file-only rationale as firstContentTimeoutMs. | +| `network.firstContentTimeoutMs` | number | unset (off) | — | Model-content deadline, phase 1 (#3254): max ms an upstream attempt may go with zero model content from response start before bili aborts the stalled stream and ends the turn with one clean in-band error (no internal refetch — the client's retry ladder re-arms). Also settable per route / per model under providers; cascade model > route > global, explicit 0 wins at its level. Once a route has ≥8 healthy attempts, the configured value becomes a ceiling and the effective deadline tightens to `clamp(3 × p95 of the route's recent healthy first-content durations, 120s, 600s)` (never above the configured value). File-only by design. | +| `network.contentGapTimeoutMs` | number | unset (inherits firstContentTimeoutMs) | — | Model-content deadline, phase 2 (#3254): max ms between consecutive model-content events after the first arrived; re-armed on every content event (pings/comments never count). Absent inherits the resolved firstContentTimeoutMs; explicit 0 disables the gap phase only. Same three-level cascade, same route-adaptive ceiling and file-only rationale as firstContentTimeoutMs. | | `network.replayRetryMax` | number | 3 (1 disables replays) | BILI_REPLAY_RETRY_MAX | Maximum transport-failure replays of an identical request before surfacing the error. | | `network.replayRetryBaseMs` | number | 1500 (0 disables the delay) | BILI_REPLAY_RETRY_BASE_MS | Base backoff delay between replays (exponential). | | `network.maxShrinkPerCompress` | number | unset | BILI_MAX_SHRINK_PER_COMPRESS | Cap on how much one compression may shrink history (anti-collapse guard). | | `network.proxyKeepAliveMaxMs` | number | 55000 (0 = one-shot connections) | BILI_PROXY_KEEPALIVE_MAX_MS | Max lifetime of a connection through an outbound egress proxy before forcing reconnection (avoids stale NAT mappings). | | `network.postResponseLingerMs` | number | 5000 | BILI_POST_RESPONSE_LINGER_MS | Graceful-close budget for proxy-initiated closes after the response completed: the socket is held waiting for the peer's FIN/TLS close_notify and destroyed at the deadline (reason=linger-backstop). | + | `network.allowPrivateHttpProviders` | boolean | false | BILI_ALLOW_PRIVATE_HTTP_PROVIDERS | Opt-in (#2984): admit plain-HTTP providers on private networks (RFC1918, 127/8, link-local, ULA, *.local/.lan/.internal) into the agent-registry intake and summary-endpoint guards. Off by default — plaintext stays loopback-only; public IPs/domains and CGNAT (100.64/10) stay refused regardless. 1/true enables. | | `network.receiveBodyBytes` | number | 5368709120 | BILI_RECEIVE_BODY_BYTES | Max inbound wire bytes read into memory per request before a 413 at the receive stage. Raised from a hardcoded 100 MiB to 5 GiB (#3128 owner decision) so multi-gigabyte single requests are accepted instead of the socket being reset. Also the threshold above which a decoded request consumes one of two shared large-decode admission slots. WebSocket lanes keep the fixed 100 MiB surface. | | `network.decodedBodyBytes` | number | 10737418240 | BILI_DECODED_BODY_BYTES | Max DECODED bytes after content-encoding expansion (gzip/deflate/br/zstd), checked incrementally during decode (decode-stage 413). Default is 2x the receive budget, mirroring the legacy decode=2x-wire relationship; guards against decompression bombs. | @@ -854,7 +855,7 @@ Since #2030 every pure-behavior knob has a config-file key alongside its env var - **Type:** `{ upstreamTimeoutMs?: number; requestWatchdogMs?: number; replayRetryMax?: number; replayRetryBaseMs?: number; maxShrinkPerCompress?: number; keepAliveTimeoutMs?: number; clientErrorBackstopMs?: number; exposureLogIntervalMs?: number; streamKeepAliveMs?: number; preflightHoldMs?: number; preflightDeadEndCooldownMs?: number; proxyKeepAliveMaxMs?: number; postResponseLingerMs?: number; firstContentTimeoutMs?: number; contentGapTimeoutMs?: number; allowPrivateHttpProviders?: boolean }` - **Default:** `{}` (built-ins: `720000`, `2× upstream timeout`, `3`, `1500`, unset, `5000`, `30000`, `3600000`, `15000`, `30000`, `300000`, `55000`, `5000`, `false`) - **Status:** ACTIVE -- **Description:** Process-wide transport and retry timing (ms). Every key except `firstContentTimeoutMs`/`contentGapTimeoutMs` (#3254, file-only — see below) resolves env > file > default via its twin (`BILI_UPSTREAM_TIMEOUT_MS`, `BILI_REQUEST_WATCHDOG_MS`, `BILI_REPLAY_RETRY_MAX`, `BILI_REPLAY_RETRY_BASE_MS`, `BILI_MAX_SHRINK_PER_COMPRESS`, `BILI_KEEP_ALIVE_TIMEOUT_MS`, `BILI_CLIENT_ERROR_BACKSTOP_MS`, `BILI_EXPOSURE_LOG_INTERVAL_MS`, `BILI_STREAM_KEEPALIVE_MS`, `BILI_PREFLIGHT_HOLD_MS`, `BILI_PREFLIGHT_DEAD_END_COOLDOWN_MS`, `BILI_PROXY_KEEPALIVE_MAX_MS`, `BILI_POST_RESPONSE_LINGER_MS`, `BILI_ALLOW_PRIVATE_HTTP_PROVIDERS`). Semantics: `upstreamTimeoutMs` bounds time-to-first-byte and inter-chunk silence end-to-end (#551); `requestWatchdogMs` is the total-request budget (a negative value opts out); `replayRetryMax` counts attempts after transient upstream rejections (#189/#1688); `maxShrinkPerCompress` caps how much one compress may shrink the request before bili steers toward smaller ranges (#189); `streamKeepAliveMs` emits an SSE comment when the response has written zero bytes for that long (#1647); `proxyKeepAliveMaxMs` caps connection reuse through an upstream proxy below its recycle cadence (#1263), `0` uncaps; `postResponseLingerMs` is the graceful-close budget after a proxy-initiated post-response close — the socket is held waiting for the peer's FIN/TLS close_notify and destroyed at the deadline (`reason=linger-backstop`) (#1982); `allowPrivateHttpProviders` is an opt-in (#2984): when true, plain-HTTP provider baseUrls on private networks (RFC1918, 127/8, link-local, IPv6 ULA/link-local, `*.local`/`.lan`/`.internal`) are admitted by the agent-registry intake and summary-endpoint guards that otherwise trust loopback plaintext only — public IPs/domains and CGNAT (100.64/10) stay refused regardless, and the default `false` keeps pre-#2984 behavior byte-for-byte; `firstContentTimeoutMs`/`contentGapTimeoutMs` are FILE-ONLY model-content deadlines (#3254): phase 1 bounds how long an upstream attempt may go with zero model content (text/reasoning deltas, tool calls, signatures) from response start, phase 2 (`contentGapTimeoutMs`, re-armed on every content event — pings/comments never count) bounds silence between content events after the first arrived; on expiry bili aborts the stalled upstream stream and ends the turn with one clean in-band error (context intact, resend your last message) — no internal refetch, so the client's own retry ladder re-arms; cascade model > provider route > global `network.*`, explicit 0 wins at its level, gap inherits the resolved initial when absent; deliberately no env layer because per-route/per-model values cannot be expressed as one process-wide variable. Full details in the [env table](#config-file-keys-for-environment-knobs-2030). +- **Description:** Process-wide transport and retry timing (ms). Every key except `firstContentTimeoutMs`/`contentGapTimeoutMs` (#3254, file-only — see below) resolves env > file > default via its twin (`BILI_UPSTREAM_TIMEOUT_MS`, `BILI_REQUEST_WATCHDOG_MS`, `BILI_REPLAY_RETRY_MAX`, `BILI_REPLAY_RETRY_BASE_MS`, `BILI_MAX_SHRINK_PER_COMPRESS`, `BILI_KEEP_ALIVE_TIMEOUT_MS`, `BILI_CLIENT_ERROR_BACKSTOP_MS`, `BILI_EXPOSURE_LOG_INTERVAL_MS`, `BILI_STREAM_KEEPALIVE_MS`, `BILI_PREFLIGHT_HOLD_MS`, `BILI_PREFLIGHT_DEAD_END_COOLDOWN_MS`, `BILI_PROXY_KEEPALIVE_MAX_MS`, `BILI_POST_RESPONSE_LINGER_MS`, `BILI_ALLOW_PRIVATE_HTTP_PROVIDERS`). Semantics: `upstreamTimeoutMs` bounds time-to-first-byte and inter-chunk silence end-to-end (#551); `requestWatchdogMs` is the total-request budget (a negative value opts out); `replayRetryMax` counts attempts after transient upstream rejections (#189/#1688); `maxShrinkPerCompress` caps how much one compress may shrink the request before bili steers toward smaller ranges (#189); `streamKeepAliveMs` emits an SSE comment when the response has written zero bytes for that long (#1647); `proxyKeepAliveMaxMs` caps connection reuse through an upstream proxy below its recycle cadence (#1263), `0` uncaps; `postResponseLingerMs` is the graceful-close budget after a proxy-initiated post-response close — the socket is held waiting for the peer's FIN/TLS close_notify and destroyed at the deadline (`reason=linger-backstop`) (#1982); `allowPrivateHttpProviders` is an opt-in (#2984): when true, plain-HTTP provider baseUrls on private networks (RFC1918, 127/8, link-local, IPv6 ULA/link-local, `*.local`/`.lan`/`.internal`) are admitted by the agent-registry intake and summary-endpoint guards that otherwise trust loopback plaintext only — public IPs/domains and CGNAT (100.64/10) stay refused regardless, and the default `false` keeps pre-#2984 behavior byte-for-byte; `firstContentTimeoutMs`/`contentGapTimeoutMs` are FILE-ONLY model-content deadlines (#3254): phase 1 bounds how long an upstream attempt may go with zero model content (text/reasoning deltas, tool calls, signatures) from response start, phase 2 (`contentGapTimeoutMs`, re-armed on every content event — pings/comments never count) bounds silence between content events after the first arrived; on expiry bili aborts the stalled upstream stream and ends the turn with one clean in-band error (context intact, resend your last message) — no internal refetch, so the client's own retry ladder re-arms; cascade model > provider route > global `network.*`, explicit 0 wins at its level, gap inherits the resolved initial when absent; deliberately no env layer because per-route/per-model values cannot be expressed as one process-wide variable; **route-adaptive refinement** (#3254 follow-up): once a route has accumulated ≥8 healthy attempts (completed with no truncation/error — stalled or killed attempts are never sampled), each configured phase value becomes a ceiling and the effective deadline tightens to `clamp(3 × p95 of that route's recent healthy first-content durations, 120s, 600s)` per phase (gap uses its own inter-content-gap samples); samples live in session metadata (persisted across restarts, ring of 32) and the fire log records `source=config|adaptive` so a fired deadline can be traced to which value class produced it. Full details in the [env table](#config-file-keys-for-environment-knobs-2030). ### `persist` diff --git a/CONFIGURATION.zh-CN.md b/CONFIGURATION.zh-CN.md index e679033d4a..744ac62644 100644 --- a/CONFIGURATION.zh-CN.md +++ b/CONFIGURATION.zh-CN.md @@ -147,13 +147,14 @@ | `network.streamKeepAliveMs` | number | 15000 (0 disables) | BILI_STREAM_KEEPALIVE_MS | 模型块之间的下游流保活心跳间隔;0 禁用心跳。 | | `network.preflightHoldMs` | number | 30000 | BILI_PREFLIGHT_HOLD_MS | preflight 失败后,允许请求未校准放行的持有时长。 | | `network.preflightDeadEndCooldownMs` | number | 300000 | BILI_PREFLIGHT_DEAD_END_COOLDOWN_MS | 死胡同 preflight 后,对同一会话重试校准的冷却时长。 | -| `network.firstContentTimeoutMs` | number | unset (off) | — | 模型内容期限,阶段一(#3254):上游一次尝试从响应开始到首个模型内容之间的最长毫秒数;超时则 bili 中止卡死流并以一条干净的带内错误结束回合(不做内部盲重取——客户端重试梯照常接管)。providers 内也可按线路/按模型设置;级联 model > route > global,显式 0 在所在层生效。按设计仅配置文件可设。 | -| `network.contentGapTimeoutMs` | number | unset (inherits firstContentTimeoutMs) | — | 模型内容期限,阶段二(#3254):首个内容到达之后,相邻两个模型内容事件之间的最长毫秒数;每个内容事件重新计时(ping/注释不算内容)。缺省继承解析后的 firstContentTimeoutMs;显式 0 仅禁用阶段二。三级级联与仅文件可设的理由同 firstContentTimeoutMs。 | +| `network.firstContentTimeoutMs` | number | unset (off) | — | 模型内容期限,阶段一(#3254):上游一次尝试从响应开始到首个模型内容之间的最长毫秒数;超时则 bili 中止卡死流并以一条干净的带内错误结束回合(不做内部盲重取——客户端重试梯照常接管)。providers 内也可按线路/按模型设置;级联 model > route > global,显式 0 在所在层生效。线路积累 ≥8 次健康尝试后,配置值转为上限:有效期限收紧为 `clamp(该线路近期健康首内容耗时的 3 × p95, 120s, 600s)`(不会高于配置值)。按设计仅配置文件可设。 | +| `network.contentGapTimeoutMs` | number | unset (inherits firstContentTimeoutMs) | — | 模型内容期限,阶段二(#3254):首个内容到达之后,相邻两个模型内容事件之间的最长毫秒数;每个内容事件重新计时(ping/注释不算内容)。缺省继承解析后的 firstContentTimeoutMs;显式 0 仅禁用阶段二。三级级联、线路自适应上限与仅文件可设的理由同 firstContentTimeoutMs。 | | `network.replayRetryMax` | number | 3 (1 disables replays) | BILI_REPLAY_RETRY_MAX | 同一请求在向上游暴露错误前的最大传输失败重放次数。 | | `network.replayRetryBaseMs` | number | 1500 (0 disables the delay) | BILI_REPLAY_RETRY_BASE_MS | 重放之间的基础退避延迟(指数增长)。 | | `network.maxShrinkPerCompress` | number | unset | BILI_MAX_SHRINK_PER_COMPRESS | 单次压缩允许收缩历史的上限(防塌缩护栏)。 | | `network.proxyKeepAliveMaxMs` | number | 55000 (0 = one-shot connections) | BILI_PROXY_KEEPALIVE_MAX_MS | 经出站代理的连接最大存活时长,到期强制重建(规避陈旧 NAT 映射)。 | | `network.postResponseLingerMs` | number | 5000 | BILI_POST_RESPONSE_LINGER_MS | 代理主动发起的响应后关闭的优雅关闭预算:socket 挂起等待对端 FIN/TLS close_notify,到点销毁(reason=linger-backstop)。 | + | `network.allowPrivateHttpProviders` | boolean | false | BILI_ALLOW_PRIVATE_HTTP_PROVIDERS | 可选开关(#2984):允许私网(RFC1918、127/8、link-local、ULA、*.local/.lan/.internal)明文 HTTP provider 进入 agent-registry 接收与摘要端点守卫。默认关——明文仅回环受信;公网 IP/域名与 CGNAT(100.64/10)无论如何都拒。1/true 启用。 | | `network.receiveBodyBytes` | number | 5368709120 | BILI_RECEIVE_BODY_BYTES | 单请求读入内存的入站 wire 字节上限,超限在 receive 阶段返回 413。由硬编码 100 MiB 提升到 5 GiB(#3128 owner 决定),使多 GB 单请求被接受而非断连。同时也是解码请求占用两个共享大解码准入槽位的阈值。WebSocket lane 保持固定 100 MiB 表面。 | | `network.decodedBodyBytes` | number | 10737418240 | BILI_DECODED_BODY_BYTES | content-encoding 解压(gzip/deflate/br/zstd)后的解码字节上限,解码过程中增量检查(decode 阶段 413)。默认为接收预算的 2 倍,沿用旧的 decode=2×wire 关系;用于防解压炸弹。 | @@ -854,7 +855,7 @@ - **类型:** `{ upstreamTimeoutMs?: number; requestWatchdogMs?: number; replayRetryMax?: number; replayRetryBaseMs?: number; maxShrinkPerCompress?: number; keepAliveTimeoutMs?: number; clientErrorBackstopMs?: number; exposureLogIntervalMs?: number; streamKeepAliveMs?: number; preflightHoldMs?: number; preflightDeadEndCooldownMs?: number; proxyKeepAliveMaxMs?: number; postResponseLingerMs?: number; firstContentTimeoutMs?: number; contentGapTimeoutMs?: number; allowPrivateHttpProviders?: boolean }` - **默认:** `{}`(内置值依次为:`720000`、`2× 上游超时`、`3`、`1500`、未设置、`5000`、`30000`、`3600000`、`15000`、`30000`、`300000`、`55000`、`5000`、`false`) - **状态:** ACTIVE -- **说明:** 进程级传输与重试时序(毫秒)。除 `firstContentTimeoutMs`/`contentGapTimeoutMs`(#3254,仅文件可设——见下)外,每个键按 env > file > default 解析,对应环境变量分别为 `BILI_UPSTREAM_TIMEOUT_MS`、`BILI_REQUEST_WATCHDOG_MS`、`BILI_REPLAY_RETRY_MAX`、`BILI_REPLAY_RETRY_BASE_MS`、`BILI_MAX_SHRINK_PER_COMPRESS`、`BILI_KEEP_ALIVE_TIMEOUT_MS`、`BILI_CLIENT_ERROR_BACKSTOP_MS`、`BILI_EXPOSURE_LOG_INTERVAL_MS`、`BILI_STREAM_KEEPALIVE_MS`、`BILI_PREFLIGHT_HOLD_MS`、`BILI_PREFLIGHT_DEAD_END_COOLDOWN_MS`、`BILI_PROXY_KEEPALIVE_MAX_MS`、`BILI_POST_RESPONSE_LINGER_MS`、`BILI_ALLOW_PRIVATE_HTTP_PROVIDERS`。要点:`upstreamTimeoutMs` 端到端约束首字节时间与 chunk 间静默(#551);`requestWatchdogMs` 是单请求总预算(负值 = 退出该约束);`replayRetryMax` 统计瞬态上游拒绝后的尝试次数(#189/#1688);`maxShrinkPerCompress` 限制单次压缩对请求的最大缩减比例,超限则引导模型选择更小的范围(#189);`streamKeepAliveMs` 在响应零字节写出超过该时长时发一条 SSE 注释行(#1647);`proxyKeepAliveMaxMs` 把经上游代理的连接复用窗口压在上游回收周期之下(#1263),`0` 取消上限;`postResponseLingerMs` 是代理主动发起的响应后关闭的优雅关闭预算 —— socket 被挂起等待对端 FIN/TLS close_notify,到点销毁(`reason=linger-backstop`)(#1982);`allowPrivateHttpProviders` 是可选开关(#2984):为 true 时,私网(RFC1918、127/8、link-local、IPv6 ULA/link-local、`*.local`/`.lan`/`.internal`)上的明文 HTTP provider baseUrl 被 agent-registry 接收与摘要端点守卫放行——这些守卫默认只信任回环明文;公网 IP/域名与 CGNAT(100.64/10)无论如何都拒;默认 false 保持 #2984 之前的行为逐字不变;`firstContentTimeoutMs`/`contentGapTimeoutMs` 是仅文件可设的模型内容期限(#3254):阶段一约束上游一次尝试从响应开始到首个模型内容(文本/推理增量、工具调用、签名)的最长等待,阶段二(`contentGapTimeoutMs`,每个内容事件重新计时——ping/注释不算内容)约束首个内容之后相邻内容事件间的静默;到期则 bili 中止卡死的上游流并以一条干净的带内错误结束回合(上下文完好,重发最后一条消息即可继续)——不做内部盲重取,客户端自身的重试梯照常接管;级联 model > provider route > global `network.*`,显式 0 在所在层生效,gap 缺省继承解析后的 initial;刻意不设 env 层,因为按线路/按模型的取值无法用单一进程级变量表达。完整细节见[环境变量表](#环境变量的配置键对照-2030)。 +- **说明:** 进程级传输与重试时序(毫秒)。除 `firstContentTimeoutMs`/`contentGapTimeoutMs`(#3254,仅文件可设——见下)外,每个键按 env > file > default 解析,对应环境变量分别为 `BILI_UPSTREAM_TIMEOUT_MS`、`BILI_REQUEST_WATCHDOG_MS`、`BILI_REPLAY_RETRY_MAX`、`BILI_REPLAY_RETRY_BASE_MS`、`BILI_MAX_SHRINK_PER_COMPRESS`、`BILI_KEEP_ALIVE_TIMEOUT_MS`、`BILI_CLIENT_ERROR_BACKSTOP_MS`、`BILI_EXPOSURE_LOG_INTERVAL_MS`、`BILI_STREAM_KEEPALIVE_MS`、`BILI_PREFLIGHT_HOLD_MS`、`BILI_PREFLIGHT_DEAD_END_COOLDOWN_MS`、`BILI_PROXY_KEEPALIVE_MAX_MS`、`BILI_POST_RESPONSE_LINGER_MS`、`BILI_ALLOW_PRIVATE_HTTP_PROVIDERS`。要点:`upstreamTimeoutMs` 端到端约束首字节时间与 chunk 间静默(#551);`requestWatchdogMs` 是单请求总预算(负值 = 退出该约束);`replayRetryMax` 统计瞬态上游拒绝后的尝试次数(#189/#1688);`maxShrinkPerCompress` 限制单次压缩对请求的最大缩减比例,超限则引导模型选择更小的范围(#189);`streamKeepAliveMs` 在响应零字节写出超过该时长时发一条 SSE 注释行(#1647);`proxyKeepAliveMaxMs` 把经上游代理的连接复用窗口压在上游回收周期之下(#1263),`0` 取消上限;`postResponseLingerMs` 是代理主动发起的响应后关闭的优雅关闭预算 —— socket 被挂起等待对端 FIN/TLS close_notify,到点销毁(`reason=linger-backstop`)(#1982);`allowPrivateHttpProviders` 是可选开关(#2984):为 true 时,私网(RFC1918、127/8、link-local、IPv6 ULA/link-local、`*.local`/`.lan`/`.internal`)上的明文 HTTP provider baseUrl 被 agent-registry 接收与摘要端点守卫放行——这些守卫默认只信任回环明文;公网 IP/域名与 CGNAT(100.64/10)无论如何都拒;默认 false 保持 #2984 之前的行为逐字不变;`firstContentTimeoutMs`/`contentGapTimeoutMs` 是仅文件可设的模型内容期限(#3254):阶段一约束上游一次尝试从响应开始到首个模型内容(文本/推理增量、工具调用、签名)的最长等待,阶段二(`contentGapTimeoutMs`,每个内容事件重新计时——ping/注释不算内容)约束首个内容之后相邻内容事件间的静默;到期则 bili 中止卡死的上游流并以一条干净的带内错误结束回合(上下文完好,重发最后一条消息即可继续)——不做内部盲重取,客户端自身的重试梯照常接管;级联 model > provider route > global `network.*`,显式 0 在所在层生效,gap 缺省继承解析后的 initial;刻意不设 env 层,因为按线路/按模型的取值无法用单一进程级变量表达;**线路自适应收紧**(#3254 follow-up):某线路积累 ≥8 次健康尝试(正常完成、无截断/错误——卡死或被期限中止的尝试从不采样)后,每个已配置阶段值转为上限,有效期限按阶段收紧为 `clamp(该线路近期健康首内容耗时的 3 × p95, 120s, 600s)`(gap 阶段用自己的内容间隔样本);样本存于会话 metadata(跨重启持久化,环形窗口 32),触发日志记录 `source=config|adaptive`,可追溯触发值来自哪一类。完整细节见[环境变量表](#环境变量的配置键对照-2030)。 ### `persist` diff --git a/src/config.ts b/src/config.ts index 5cbd43a5e2..dd8011dc18 100644 --- a/src/config.ts +++ b/src/config.ts @@ -2433,20 +2433,70 @@ export function parseDeadlineMs(fieldPath: string, value: unknown): number | und // phase, no fall-through); an absent field inherits the next level down. // contentGapTimeoutMs inherits the resolved initial deadline when left absent. // Returns undefined when every phase is off or absent (feature fully off). +/** #3254 follow-up (owner point 5): per-route adaptive refinement of the + * configured deadlines. Each route's HEALTHY first-content durations are + * sampled into session metadata as requests complete cleanly; once a route + * accumulates enough samples its deadline stops being a guess and hugs its + * own normal water level. */ +export const FC_ADAPTIVE_MIN_SAMPLES = 8; +/** Per-route sample window (oldest evicted past this). */ +export const FC_ADAPTIVE_WINDOW = 32; +/** Owner-specified clamp band for the adaptive value. */ +export const FC_ADAPTIVE_FLOOR_MS = 120_000; +export const FC_ADAPTIVE_CAP_MS = 600_000; + +export function fcP95(values: number[]): number { + const sorted = [...values].sort((a, b) => a - b); + return sorted[Math.max(0, Math.min(sorted.length - 1, Math.ceil(0.95 * sorted.length) - 1))]; +} + +/** clamp(3 × p95(samples), FLOOR, CAP), then capped by the EXPLICIT config + * value: explicit config is always the ceiling — adaptation only tightens + * toward measured reality, never relaxes a value the user pinned. Returns + * undefined while the route is still cold (fewer than MIN_SAMPLES). */ +export function adaptiveDeadlineMs(samples: number[] | undefined, explicitMs: number): number | undefined { + if (!samples || samples.length < FC_ADAPTIVE_MIN_SAMPLES) return undefined; + const clamped = Math.max(FC_ADAPTIVE_FLOOR_MS, Math.min(FC_ADAPTIVE_CAP_MS, 3 * fcP95(samples))); + return Math.min(explicitMs, clamped); +} + +export interface FirstContentDeadlines { + initialMs?: number; + gapMs?: number; + /** Which source produced each phase's value — lands in the fire log so + * config→adaptive cutover never changes the log format (#3254 point 5). */ + initialSource?: "config" | "adaptive"; + gapSource?: "config" | "adaptive"; +} + export function resolveFirstContentDeadlines( routes: ProviderRoutes, upstreamUrl: string | undefined, model: string | undefined, globals: { firstContentTimeoutMs?: unknown; contentGapTimeoutMs?: unknown } | undefined, -): { initialMs?: number; gapMs?: number } | undefined { + samples?: { init: number[]; gap: number[] }, +): FirstContentDeadlines | undefined { const route = findRoute(routes, upstreamUrl); const entry = model ? route?.models?.[model] : undefined; - const initialMs = parseDeadlineMs("firstContentTimeoutMs", entry?.firstContentTimeoutMs ?? route?.firstContentTimeoutMs ?? globals?.firstContentTimeoutMs); - const gapMs = parseDeadlineMs("contentGapTimeoutMs", entry?.contentGapTimeoutMs ?? route?.contentGapTimeoutMs ?? globals?.contentGapTimeoutMs); - const out: { initialMs?: number; gapMs?: number } = {}; - if (initialMs !== undefined && initialMs > 0) out.initialMs = initialMs; - if (gapMs !== undefined && gapMs > 0) out.gapMs = gapMs; - else if (gapMs === undefined && out.initialMs !== undefined) out.gapMs = out.initialMs; + const initialExplicit = parseDeadlineMs("firstContentTimeoutMs", entry?.firstContentTimeoutMs ?? route?.firstContentTimeoutMs ?? globals?.firstContentTimeoutMs); + const gapExplicit = parseDeadlineMs("contentGapTimeoutMs", entry?.contentGapTimeoutMs ?? route?.contentGapTimeoutMs ?? globals?.contentGapTimeoutMs); + const out: FirstContentDeadlines = {}; + if (initialExplicit !== undefined && initialExplicit > 0) { + const adaptive = adaptiveDeadlineMs(samples?.init, initialExplicit); + out.initialMs = adaptive !== undefined ? adaptive : initialExplicit; + out.initialSource = adaptive !== undefined ? "adaptive" : "config"; + } + // Explicit 0 disables the phase at its level (no fall-through); an ABSENT + // gap inherits the resolved initial value and adapts on gap samples. + if (gapExplicit !== undefined && gapExplicit > 0) { + const adaptive = adaptiveDeadlineMs(samples?.gap, gapExplicit); + out.gapMs = adaptive !== undefined ? adaptive : gapExplicit; + out.gapSource = adaptive !== undefined ? "adaptive" : "config"; + } else if (gapExplicit === undefined && out.initialMs !== undefined) { + const adaptive = adaptiveDeadlineMs(samples?.gap, out.initialMs); + out.gapMs = adaptive !== undefined ? adaptive : out.initialMs; + out.gapSource = adaptive !== undefined ? "adaptive" : "config"; + } return out.initialMs !== undefined || out.gapMs !== undefined ? out : undefined; } diff --git a/src/loop/core.ts b/src/loop/core.ts index 76a8f1cf11..e11ecca9f2 100644 --- a/src/loop/core.ts +++ b/src/loop/core.ts @@ -38,6 +38,7 @@ import { log as loggerLog } from "../logger.js"; import { promptInputTotal, type WireProtocol } from "../util.js"; import { DEGENERATE_RETRY_NUDGE } from "../degenerate-retry.js"; import { endsWithDraftClose, lastUserSummaryInstruction, requestExpectsProseSummary } from "../degenerate-turn.js"; +import { FC_ADAPTIVE_WINDOW } from "../config.js"; import { safePrefix, safeSuffix, scrubLoneSurrogatesOnWire } from "../text-safe.js"; export const MAX_LOOP_ROUNDS = 10; @@ -209,10 +210,20 @@ export interface LoopCtx { * present, each sent round captures its image facts alongside the body. */ imageLearn?: { host: string; fp: string }; /** #3254: model-content deadlines resolved per request (three-level - * cascade via resolveFirstContentDeadlines). Present iff at least one - * phase is enabled; undefined keeps the legacy path byte-identical - * (no timers armed, zero overhead). */ - firstContentDeadline?: { initialMs?: number; gapMs?: number }; + * cascade via resolveFirstContentDeadlines, optionally refined by the + * route's adaptive samples). Present iff at least one phase is enabled; + * undefined keeps the legacy path byte-identical (no timers armed, zero + * overhead). */ + firstContentDeadline?: { + initialMs?: number; + gapMs?: number; + initialSource?: "config" | "adaptive"; + gapSource?: "config" | "adaptive"; + }; + /** #3254 follow-up: route key (providers URL prefix, else host) under + * which this session's healthy first-content durations are sampled in + * metadata — feeds the route's adaptive deadline on later requests. */ + fcRouteKey?: string; } export interface RequestOptions { @@ -444,6 +455,34 @@ function interruptibleBody(src: ReadableStream): { stream: ReadableS }; } +// #3254 follow-up: per-route healthy-latency samples live in session +// metadata (persisted with the session record) so adaptive deadlines +// survive restarts. One row per route key; each phase keeps its own window. +const FC_SAMPLES_META_KEY = "firstContentSamples"; + +export function readFcSamples(session: Session, routeKey: string): { init: number[]; gap: number[] } | undefined { + const table = session.metadata[FC_SAMPLES_META_KEY] as Record | undefined; + const row = table?.[routeKey]; + if (!row) return undefined; + const clean = (v: unknown): number[] => Array.isArray(v) ? v.filter((x): x is number => typeof x === "number" && Number.isFinite(x) && x >= 0) : []; + return { init: clean(row.init), gap: clean(row.gap) }; +} + +export function recordFcSample(session: Session, routeKey: string, initMs: number, gapMs?: number): void { + const table = (session.metadata[FC_SAMPLES_META_KEY] ?? {}) as Record; + const row = table[routeKey] ?? { init: [], gap: [] }; + if (initMs >= 0) { + row.init.push(initMs); + if (row.init.length > FC_ADAPTIVE_WINDOW) row.init.shift(); + } + if (gapMs !== undefined && gapMs >= 0) { + row.gap.push(gapMs); + if (row.gap.length > FC_ADAPTIVE_WINDOW) row.gap.shift(); + } + table[routeKey] = row; + session.metadata[FC_SAMPLES_META_KEY] = table; +} + export async function* runCompressLoop( upstream: ReadableStream, ctx: LoopCtx, @@ -485,10 +524,17 @@ export async function* runCompressLoop( let activeKill: (() => void) | null = null; let contentDeadlineTimer: ReturnType | null = null; let contentDeadlineArmedAt = 0; - let contentDeadlineInfo: { ms: number; phase: "initial" | "gap"; round: number } | null = null; + let contentDeadlineInfo: { ms: number; phase: "initial" | "gap"; round: number; source: "config" | "adaptive" } | null = null; let contentDeadlineKilled = false; let contentDeadlineSettled = true; let upstreamAttemptNo = 0; + // #3254 follow-up: per-attempt healthy-latency sampling for the route's + // adaptive deadline (recorded only when a request completes cleanly). + let fcAttemptStartAt = 0; + let fcFirstContentAt = 0; + let fcLastContentAt = 0; + let fcWorstGapMs = 0; + let fcGapMeasured = false; const deadlineHost = (() => { try { return new URL(requestOptions.url).host; } catch { return requestOptions.url; } })(); const clearContentDeadline = () => { if (contentDeadlineTimer) { @@ -502,14 +548,14 @@ export async function* runCompressLoop( contentDeadlineKilled = true; const info = contentDeadlineInfo; const waitedMs = Date.now() - contentDeadlineArmedAt; - const line = `content-deadline fired: no model content for ${waitedMs}ms (limit ${info.ms}ms, phase=${info.phase}, source=config, upstream attempt ${upstreamAttemptNo}, round ${info.round}) — aborting stalled upstream stream (#3254)`; + const line = `content-deadline fired: no model content for ${waitedMs}ms (limit ${info.ms}ms, phase=${info.phase}, source=${info.source}, upstream attempt ${upstreamAttemptNo}, round ${info.round}) — aborting stalled upstream stream (#3254)`; ctx.log(`[acp-proxy: ${line}]`); loggerLog("warn", `[acp-loop] session ${ctx.session.id} ${line}`); activeKill?.(); }; - const armContentDeadline = (ms: number, phase: "initial" | "gap", roundNo: number) => { + const armContentDeadline = (ms: number, phase: "initial" | "gap", roundNo: number, source: "config" | "adaptive") => { clearContentDeadline(); - contentDeadlineInfo = { ms, phase, round: roundNo }; + contentDeadlineInfo = { ms, phase, round: roundNo, source }; contentDeadlineArmedAt = Date.now(); contentDeadlineSettled = false; contentDeadlineTimer = setTimeout(fireContentDeadline, ms); @@ -787,8 +833,13 @@ export async function* runCompressLoop( // (gap-only configs arm nothing until first content arrives). upstreamAttemptNo += 1; contentDeadlineKilled = false; + fcAttemptStartAt = Date.now(); + fcFirstContentAt = 0; + fcLastContentAt = 0; + fcWorstGapMs = 0; + fcGapMeasured = false; if (ctx.firstContentDeadline?.initialMs !== undefined && ctx.firstContentDeadline.initialMs > 0) { - armContentDeadline(ctx.firstContentDeadline.initialMs, "initial", round); + armContentDeadline(ctx.firstContentDeadline.initialMs, "initial", round, ctx.firstContentDeadline.initialSource ?? "config"); } else { clearContentDeadline(); } @@ -797,10 +848,19 @@ export async function* runCompressLoop( // #3254: any MODEL CONTENT event re-arms the inter-content // deadline (owner-tiered like the client's own watchdog); // pings/comments/meta/usage/diag never count as content. - if (ctx.firstContentDeadline?.gapMs !== undefined && ctx.firstContentDeadline.gapMs > 0 - && ((ev.kind === "text" || ev.kind === "reasoning") ? ev.delta.length > 0 - : ev.kind === "tool_call" || ev.kind === "redacted_thinking" || ev.kind === "standalone_signature")) { - armContentDeadline(ctx.firstContentDeadline.gapMs, "gap", round); + const fcHasContent = (ev.kind === "text" || ev.kind === "reasoning") ? ev.delta.length > 0 + : ev.kind === "tool_call" || ev.kind === "redacted_thinking" || ev.kind === "standalone_signature"; + if (fcHasContent && ctx.firstContentDeadline !== undefined) { + const now = Date.now(); + if (fcFirstContentAt === 0) fcFirstContentAt = now; + else { + fcWorstGapMs = Math.max(fcWorstGapMs, now - fcLastContentAt); + fcGapMeasured = true; + } + fcLastContentAt = now; + } + if (fcHasContent && ctx.firstContentDeadline?.gapMs !== undefined && ctx.firstContentDeadline.gapMs > 0) { + armContentDeadline(ctx.firstContentDeadline.gapMs, "gap", round, ctx.firstContentDeadline.gapSource ?? "config"); } if (ev.kind === "text") { assistantText += ev.delta; @@ -902,6 +962,16 @@ export async function* runCompressLoop( yield adapter.emitError(msg); return; } + // #3254 follow-up: a cleanly completed attempt (done, not + // truncated, not deadline-killed, produced content) is a + // HEALTHY sample of this route's normal latency — feed the + // per-route ring that drives adaptive deadlines later. + if (ctx.fcRouteKey !== undefined && ctx.firstContentDeadline !== undefined + && sawDone && !truncatedDone && fcFirstContentAt > 0) { + // A measured sub-millisecond duration is a legitimate fast + // sample — record it; only unmeasured phases are omitted. + recordFcSample(ctx.session, ctx.fcRouteKey, fcFirstContentAt - fcAttemptStartAt, fcGapMeasured ? fcWorstGapMs : undefined); + } // #2701: threads acceptance. This round's response started ⇔ // the tail advanced past the id the sent body continued from ⇔ diff --git a/src/loop/index.ts b/src/loop/index.ts index 5f5ca938c3..e16e1d3725 100644 --- a/src/loop/index.ts +++ b/src/loop/index.ts @@ -1,4 +1,4 @@ -export { runCompressLoop, executeProxyTool, MAX_LOOP_ROUNDS } from "./core.js"; +export { runCompressLoop, executeProxyTool, readFcSamples, recordFcSample, MAX_LOOP_ROUNDS } from "./core.js"; export type { LoopCtx, RequestOptions, diff --git a/src/server/relay.ts b/src/server/relay.ts index f5c08b0fcc..83e0aaba77 100644 --- a/src/server/relay.ts +++ b/src/server/relay.ts @@ -15,7 +15,7 @@ import { applyCompatRoles, applyCompatRolesJson, detectRoleRejection, detectSyst import { compressLoopResponsesJson } from "../compress-loop-responses.js"; import { resolveCompress } from "../compress-settings.js"; import { buildCompressHybridSystemPrompt, buildCompressSystemPrompt, withMarkerIntegrityNote, withProjectAttribution, withSummaryBudgetNote } from "../compress-tool.js"; -import { findRoute, loadConfigFile, resolveDeclaredProtocol, resolveFirstContentDeadlines, type ProxyOptions } from "../config.js"; +import { findRoute, findRouteKey, loadConfigFile, resolveDeclaredProtocol, resolveFirstContentDeadlines, type ProxyOptions } from "../config.js"; import { makeContinuationRefetch } from "../degenerate-retry.js"; import { dumpRejectedBody } from "../error-dump.js"; import { externalSummaryEnabled } from "../external-summary-surface.js"; @@ -28,7 +28,7 @@ import { dumpReqAllowed as knobDumpReqAllowed, rawDumpDir as knobRawDumpDir } fr import { gcDumpDirIfConfigured } from "../state-gc.js"; import { applyLaneCredential, laneCredential } from "../lane-credentials.js"; import { CREDENTIAL_HEADER_RE, maskHeaderForLog, maskHeadersForLog, maskUrlForLog } from "../log-mask.js"; -import { pickAdapter, runCompressLoop } from "../loop/index.js"; +import { pickAdapter, readFcSamples, runCompressLoop } from "../loop/index.js"; import { ABSORB_INSTRUCTION_MARKER, containsToolCallXmlFragment } from "../loop/tag-echo-filter.js"; import { dumpsDir } from "../paths.js"; import { pipePluginChatWithStrip, pipePluginJson, pipePluginResponsesWithStrip, type UpstreamMeta } from "../plugin.js"; @@ -1501,10 +1501,11 @@ export async function forward( // The Google wire carries its model in the PATH, not the body // (prepared.google.model) — without the fallback the model tier // could never match a Gemini-native request. - const firstContentDeadline = resolveFirstContentDeadlines(opts.routes, loopBillingUpstream, (parsedReq as { model?: string }).model ?? prepared.google?.model, loadConfigFile().network); + const fcRouteKey = findRouteKey(opts.routes, loopBillingUpstream) ?? (() => { if (!loopBillingUpstream) return "unknown"; try { return new URL(loopBillingUpstream).host; } catch { return loopBillingUpstream; } })(); + const firstContentDeadline = resolveFirstContentDeadlines(opts.routes, loopBillingUpstream, (parsedReq as { model?: string }).model ?? prepared.google?.model, loadConfigFile().network, readFcSamples(prepared.session, fcRouteKey)); const loop = runCompressLoop( responseBody, - { core, config, messages: prepared.processedMessages.length > 0 ? prepared.processedMessages : prepared.originalMessages, compressMessages: prepared.originalMessages, session: prepared.session, log: ctx.log, proxyUrl, skippedProxy, upstreamOrigin: targetOrigin, protocol: prepared.protocol, textProtocol, debug: opts.debug, refreshFolded, visibilityMarkers, dumpSse: loopDumpDir ? (name, stream) => dumpStreamToFile(stream, loopDumpDir, name) : undefined, imageLearn: { host: upstreamHost(loopBillingUpstream), fp: `${imageBillingFor(opts, loopBillingUpstream)}:${imageTokenCapFor(opts, loopBillingUpstream)}` }, firstContentDeadline }, + { core, config, messages: prepared.processedMessages.length > 0 ? prepared.processedMessages : prepared.originalMessages, compressMessages: prepared.originalMessages, session: prepared.session, log: ctx.log, proxyUrl, skippedProxy, upstreamOrigin: targetOrigin, protocol: prepared.protocol, textProtocol, debug: opts.debug, refreshFolded, visibilityMarkers, dumpSse: loopDumpDir ? (name, stream) => dumpStreamToFile(stream, loopDumpDir, name) : undefined, imageLearn: { host: upstreamHost(loopBillingUpstream), fp: `${imageBillingFor(opts, loopBillingUpstream)}:${imageTokenCapFor(opts, loopBillingUpstream)}` }, firstContentDeadline, fcRouteKey }, parsedReq, // #3280: egress goes out enveloped via wireTransform, so every // round-2+ response comes back enveloped too — unwrap at the loop's diff --git a/tests/issue3254-first-content-deadline.test.ts b/tests/issue3254-first-content-deadline.test.ts index 198f6ce4b4..441eba84d8 100644 --- a/tests/issue3254-first-content-deadline.test.ts +++ b/tests/issue3254-first-content-deadline.test.ts @@ -8,7 +8,8 @@ import { createCore, createInitialState } from "acp-kernel"; import type { Session } from "../src/session.ts"; import { runCompressLoop, createAnthropicAdapter } from "../src/loop/index.ts"; import { buildCompressSystemPrompt } from "../src/compress-tool.ts"; -import { loadRoutes, parseDeadlineMs, resolveFirstContentDeadlines, type ProviderRoutes } from "../src/config.ts"; +import { adaptiveDeadlineMs, FC_ADAPTIVE_MIN_SAMPLES, FC_ADAPTIVE_WINDOW, fcP95, loadRoutes, parseDeadlineMs, resolveFirstContentDeadlines, type ProviderRoutes } from "../src/config.ts"; +import { readFcSamples, recordFcSample } from "../src/loop/index.ts"; // #3254: in-request summarization had no bili-side wait bound of its own. The // upstream idle timer is byte-level — SSE keep-alive comments/pings keep it @@ -22,7 +23,10 @@ import { loadRoutes, parseDeadlineMs, resolveFirstContentDeadlines, type Provide // first content; (C) live content re-arms the deadline and never false-fires; // (D) absent config stays byte-identical legacy (no timers, no early exit); // (E-H) the three-level cascade (model > route > global network.*) incl. -// explicit-0-wins-at-level, gap-inherits-initial, and strict validation. +// explicit-0-wins-at-level, gap-inherits-initial, and strict validation; +// (I-L) route-adaptive refinement (owner point 5): clamp(3×p95(healthy +// first-content durations), 120s, 600s) capped by the explicit config value, +// samples stored per route in session metadata, healthy attempts only. const ANTHROPIC_BODY = { model: "claude", messages: [], stream: true, max_tokens: 10 }; const enc = new TextEncoder(); @@ -120,7 +124,8 @@ async function drainAnthropic( deadline: { initialMs?: number; gapMs?: number } | undefined, id: string, timeoutMs: number, -): Promise<{ out: string; fetchCalls: number; logs: string[] }> { + fcRouteKey?: string, +): Promise<{ out: string; fetchCalls: number; logs: string[]; session: Session }> { let n = 0; const orig = globalThis.fetch; // Any fetch here means the loop REFETCHED — the #3254 decision pins that a @@ -130,6 +135,7 @@ async function drainAnthropic( const ctx = makeCtx(id, logs); (ctx as Record).protocol = "anthropic"; if (deadline !== undefined) (ctx as Record).firstContentDeadline = deadline; + if (fcRouteKey !== undefined) (ctx as Record).fcRouteKey = fcRouteKey; const chunks: Buffer[] = []; try { const gen = runCompressLoop( @@ -144,7 +150,7 @@ async function drainAnthropic( } finally { globalThis.fetch = orig; } - return { out: Buffer.concat(chunks).toString("utf8"), fetchCalls: n, logs }; + return { out: Buffer.concat(chunks).toString("utf8"), fetchCalls: n, logs, session: ctx.session }; } const sseBlocks = (s: string): string[] => s.split("\n\n").map((b) => b.trim()).filter(Boolean); @@ -241,20 +247,20 @@ test("#3254 F: cascade — model > route > global, 0 wins at its level, gap inhe // all absent → feature off assert.equal(resolveFirstContentDeadlines({}, undefined, undefined, undefined), undefined); // global tier alone; gap inherits the resolved initial - assert.deepEqual(resolveFirstContentDeadlines({}, "https://any.example", undefined, { firstContentTimeoutMs: 180000 }), { initialMs: 180000, gapMs: 180000 }); + assert.deepEqual(resolveFirstContentDeadlines({}, "https://any.example", undefined, { firstContentTimeoutMs: 180000 }), { initialMs: 180000, initialSource: "config", gapMs: 180000, gapSource: "config" }); // global gap-only (valid: initial stays off) - assert.deepEqual(resolveFirstContentDeadlines({}, "https://any.example", undefined, { contentGapTimeoutMs: 300000 }), { gapMs: 300000 }); + assert.deepEqual(resolveFirstContentDeadlines({}, "https://any.example", undefined, { contentGapTimeoutMs: 300000 }), { gapMs: 300000, gapSource: "config" }); // route beats global - assert.deepEqual(resolveFirstContentDeadlines(ROUTES, "https://relay-a.example/v1/messages", undefined, { firstContentTimeoutMs: 180000 }), { initialMs: 600000, gapMs: 600000 }); + assert.deepEqual(resolveFirstContentDeadlines(ROUTES, "https://relay-a.example/v1/messages", undefined, { firstContentTimeoutMs: 180000 }), { initialMs: 600000, initialSource: "config", gapMs: 600000, gapSource: "config" }); // model beats route - assert.deepEqual(resolveFirstContentDeadlines(ROUTES, "https://relay-a.example/v1/messages", "gpt-x", { firstContentTimeoutMs: 180000 }), { initialMs: 90000, gapMs: 90000 }); + assert.deepEqual(resolveFirstContentDeadlines(ROUTES, "https://relay-a.example/v1/messages", "gpt-x", { firstContentTimeoutMs: 180000 }), { initialMs: 90000, initialSource: "config", gapMs: 90000, gapSource: "config" }); // model ignored when the URL matches no route - assert.deepEqual(resolveFirstContentDeadlines(ROUTES, "https://other.example", "gpt-x", { firstContentTimeoutMs: 180000 }), { initialMs: 180000, gapMs: 180000 }); + assert.deepEqual(resolveFirstContentDeadlines(ROUTES, "https://other.example", "gpt-x", { firstContentTimeoutMs: 180000 }), { initialMs: 180000, initialSource: "config", gapMs: 180000, gapSource: "config" }); // explicit 0 at route level disables the phase (no fall-through to global); // gap then inherits nothing → feature fully off assert.equal(resolveFirstContentDeadlines(ROUTES, "https://relay-b.example", undefined, { firstContentTimeoutMs: 180000 }), undefined); // explicit 0 for gap only → initial survives, gap phase off - assert.deepEqual(resolveFirstContentDeadlines(ROUTES, "https://relay-c.example", undefined, undefined), { initialMs: 180000 }); + assert.deepEqual(resolveFirstContentDeadlines(ROUTES, "https://relay-c.example", undefined, undefined), { initialMs: 180000, initialSource: "config" }); // garbage at ANY tier rejects loudly (the global tier bypasses load-time parsing) assert.throws(() => resolveFirstContentDeadlines({}, "https://any.example", undefined, { firstContentTimeoutMs: "fast" }), /firstContentTimeoutMs must be a non-negative integer/); }); @@ -274,3 +280,78 @@ test("#3254 G: loadRoutes rejects malformed deadline values at provider and mode assert.equal(ok["https://relay-a.example"]?.firstContentTimeoutMs, 0); assert.equal(ok["https://relay-a.example"]?.models?.["m"]?.contentGapTimeoutMs, 120000); }); + +// --- I: adaptive deadline math ---------------------------------------------- + +test("#3254 I: fcP95 + adaptiveDeadlineMs — cold undefined, clamp band, explicit ceiling", () => { + assert.equal(fcP95([42]), 42); + // nearest-rank: at n=8 the rank is ceil(0.95×8)=8 → the max; at n=20 it is 19 → second-highest + assert.equal(fcP95([5, 10, 15, 20, 25, 30, 35, 40]), 40); + assert.equal(fcP95(Array.from({ length: 20 }, (_, i) => (i + 1) * 5)), 95); + assert.equal(adaptiveDeadlineMs(undefined, 180000), undefined); + assert.equal(adaptiveDeadlineMs(Array.from({ length: FC_ADAPTIVE_MIN_SAMPLES - 1 }, () => 20000), 180000), undefined, "one below MIN_SAMPLES is still cold"); + // p95=20000 → 3×=60000 → clamped UP to the 120s floor + assert.equal(adaptiveDeadlineMs(Array.from({ length: FC_ADAPTIVE_MIN_SAMPLES }, () => 20000), 180000), 120000); + // p95=200000 → 3×=600000 → the cap + assert.equal(adaptiveDeadlineMs(Array.from({ length: FC_ADAPTIVE_MIN_SAMPLES }, () => 200000), 600000), 600000); + // mid-band: p95=50000 → 150000, below the explicit value + assert.equal(adaptiveDeadlineMs(Array.from({ length: FC_ADAPTIVE_MIN_SAMPLES }, () => 50000), 180000), 150000); + // explicit ceiling: a user-pinned tighter value never gets relaxed + assert.equal(adaptiveDeadlineMs(Array.from({ length: FC_ADAPTIVE_MIN_SAMPLES }, () => 20000), 60000), 60000); +}); + +// --- J: resolver with adaptive samples -------------------------------------- + +test("#3254 J: warm routes adapt (source=adaptive) under their configured ceiling; cold stay on config", () => { + const globals = { firstContentTimeoutMs: 180000 }; + const warm = Array.from({ length: FC_ADAPTIVE_MIN_SAMPLES }, () => 20000); + // warm initial, no gap samples yet: initial adapts to the floor; the + // INHERITED gap has no gap samples of its own and stays on config + assert.deepEqual(resolveFirstContentDeadlines({}, "https://any.example", undefined, globals, { init: warm, gap: [] }), { + initialMs: 120000, initialSource: "adaptive", gapMs: 120000, gapSource: "config", + }); + // both phases warm in-band: p95=50000 → 150000 each + const mid = Array.from({ length: FC_ADAPTIVE_MIN_SAMPLES }, () => 50000); + assert.deepEqual(resolveFirstContentDeadlines({}, "https://any.example", undefined, globals, { init: mid, gap: mid }), { + initialMs: 150000, initialSource: "adaptive", gapMs: 150000, gapSource: "adaptive", + }); + // explicit ceiling tighter than the adaptive floor: adaptation engages but + // the user's pinned value wins as the ceiling + assert.deepEqual(resolveFirstContentDeadlines({}, "https://any.example", undefined, { firstContentTimeoutMs: 60000 }, { init: warm, gap: [] }), { + initialMs: 60000, initialSource: "adaptive", gapMs: 60000, gapSource: "config", + }); +}); + +// --- K: sample ring storage in session metadata ------------------------------ + +test("#3254 K: recordFcSample/readFcSamples — window cap, per-phase rows, sanitizing read", () => { + const s = makeCtx("s3254k", []).session; + assert.equal(readFcSamples(s, "r"), undefined); + recordFcSample(s, "r", 1234, 567); + recordFcSample(s, "r", 2222); + let got = readFcSamples(s, "r"); + assert.deepEqual(got, { init: [1234, 2222], gap: [567] }); + const s2 = makeCtx("s3254k2", []).session; + for (let i = 0; i < FC_ADAPTIVE_WINDOW + 5; i++) recordFcSample(s2, "r", i); + got = readFcSamples(s2, "r"); + assert.equal(got?.init.length, FC_ADAPTIVE_WINDOW, "oldest samples must be evicted past the window"); + assert.equal(got?.init[0], 5, "the first five records must have been evicted"); + assert.equal(got?.gap.length, 0); + (s.metadata as Record)["firstContentSamples"] = { r: { init: [1, Number.NaN, -5, "x"], gap: null } }; + assert.deepEqual(readFcSamples(s, "r"), { init: [1], gap: [] }, "reads must sanitize hand-corrupted metadata"); +}); + +// --- L: end-to-end sampling — healthy attempts only -------------------------- + +test("#3254 L: a cleanly completed attempt records one healthy sample per phase; a killed attempt records none", async () => { + const text = MESSAGE_START + textStart(0) + textDelta(0, "a") + textDelta(0, "b") + blockStop(0) + MESSAGE_DELTA() + MESSAGE_STOP; + const healthy = await drainAnthropic(completeStream(text), { initialMs: 100000 }, "s3254l", 3000, "route-l"); + const sample = readFcSamples(healthy.session, "route-l"); + assert.ok(sample !== undefined && sample.init.length === 1 && sample.gap.length === 1, `expected one init + one gap sample: ${JSON.stringify(sample)}`); + // the stream is synchronous, so both measured durations are legitimately 0ms + assert.ok(sample!.init[0] >= 0 && sample!.init[0] < 10000, `first-content duration implausible: ${sample?.init[0]}ms`); + const sink = { cancelled: false }; + const killed = await drainAnthropic(holdStream(MESSAGE_START + PING, sink), { initialMs: 150 }, "s3254l2", 3000, "route-l2"); + assert.match(killed.out, /model content deadline exceeded/); + assert.equal(readFcSamples(killed.session, "route-l2"), undefined, "a deadline-killed attempt must not poison the route's samples"); +}); diff --git a/website/config-reference/process.yaml b/website/config-reference/process.yaml index 7d689aff6a..c20b5bbd3e 100644 --- a/website/config-reference/process.yaml +++ b/website/config-reference/process.yaml @@ -72,14 +72,14 @@ entries: type: number default: "unset (off)" group: process - desc_en: "Model-content deadline, phase 1 (#3254): max ms an upstream attempt may go with zero model content from response start before bili aborts the stalled stream and ends the turn with one clean in-band error (no internal refetch — the client's retry ladder re-arms). Also settable per route / per model under providers; cascade model > route > global, explicit 0 wins at its level. File-only by design." - desc_zh: "模型内容期限,阶段一(#3254):上游一次尝试从响应开始到首个模型内容之间的最长毫秒数;超时则 bili 中止卡死流并以一条干净的带内错误结束回合(不做内部盲重取——客户端重试梯照常接管)。providers 内也可按线路/按模型设置;级联 model > route > global,显式 0 在所在层生效。按设计仅配置文件可设。" + desc_en: "Model-content deadline, phase 1 (#3254): max ms an upstream attempt may go with zero model content from response start before bili aborts the stalled stream and ends the turn with one clean in-band error (no internal refetch — the client's retry ladder re-arms). Also settable per route / per model under providers; cascade model > route > global, explicit 0 wins at its level. Once a route has ≥8 healthy attempts the configured value becomes a ceiling and the effective deadline tightens to clamp(3 × p95 of the route's recent healthy first-content durations, 120s, 600s); stalled/killed attempts are never sampled. File-only by design." + desc_zh: "模型内容期限,阶段一(#3254):上游一次尝试从响应开始到首个模型内容之间的最长毫秒数;超时则 bili 中止卡死流并以一条干净的带内错误结束回合(不做内部盲重取——客户端重试梯照常接管)。providers 内也可按线路/按模型设置;级联 model > route > global,显式 0 在所在层生效。线路积累 ≥8 次健康尝试后,配置值转为上限,有效期限收紧为 clamp(该线路近期健康首内容耗时的 3 × p95, 120s, 600s);卡死或被中止的尝试从不采样。按设计仅配置文件可设。" - key: network.contentGapTimeoutMs type: number default: "unset (inherits firstContentTimeoutMs)" group: process - desc_en: "Model-content deadline, phase 2 (#3254): max ms between consecutive model-content events after the first arrived; re-armed on every content event (pings/comments never count). Absent inherits the resolved firstContentTimeoutMs; explicit 0 disables the gap phase only. Same three-level cascade and file-only rationale as firstContentTimeoutMs." - desc_zh: "模型内容期限,阶段二(#3254):首个内容到达之后,相邻两个模型内容事件之间的最长毫秒数;每个内容事件重新计时(ping/注释不算内容)。缺省继承解析后的 firstContentTimeoutMs;显式 0 仅禁用阶段二。三级级联与仅文件可设的理由同 firstContentTimeoutMs。" + desc_en: "Model-content deadline, phase 2 (#3254): max ms between consecutive model-content events after the first arrived; re-armed on every content event (pings/comments never count). Absent inherits the resolved firstContentTimeoutMs; explicit 0 disables the gap phase only. Same three-level cascade, same route-adaptive ceiling (per-phase samples) and file-only rationale as firstContentTimeoutMs." + desc_zh: "模型内容期限,阶段二(#3254):首个内容到达之后,相邻两个模型内容事件之间的最长毫秒数;每个内容事件重新计时(ping/注释不算内容)。缺省继承解析后的 firstContentTimeoutMs;显式 0 仅禁用阶段二。三级级联、线路自适应上限(按阶段各自采样)与仅文件可设的理由同 firstContentTimeoutMs。" - key: network.replayRetryMax type: number default: "3 (1 disables replays)" diff --git a/website/config-reference/providers.yaml b/website/config-reference/providers.yaml index 9a5dc971ec..66070a899d 100644 --- a/website/config-reference/providers.yaml +++ b/website/config-reference/providers.yaml @@ -182,20 +182,20 @@ entries: group: providers id: cr.fcdeadline render: route-row - desc_en: "Per-route model-content deadline, phase 1 (#3254): max ms without model content from response start before bili aborts the stalled upstream and ends the turn in-band. Also settable per model under models[]; cascade model > route > global network.*, explicit 0 wins at its level. File-only (no env)." - desc_zh: "按线路的模型内容期限,阶段一(#3254):响应开始后无模型内容的上限毫秒数;超时则中止卡死上游并以带内错误结束回合。也可在 models[] 按模型设置;级联 model > route > global network.*,显式 0 在所在层生效。仅配置文件可设(无 env)。" - site_en: "Model-content deadline, phase 1 (#3254), per route and per model (`models[]`); on expiry bili aborts the stalled upstream stream and ends the turn with one clean in-band error — no internal refetch. Cascade model > route > global `network.*`; explicit 0 wins at its level; file-only." - site_zh: "模型内容期限,阶段一(#3254),按线路与按模型(`models[]`);到期则中止卡死的上游流并以一条干净的带内错误结束回合——不做内部盲重取。级联 model > route > global `network.*`;显式 0 在所在层生效;仅文件可设。" + desc_en: "Per-route model-content deadline, phase 1 (#3254): max ms without model content from response start before bili aborts the stalled upstream and ends the turn in-band. Also settable per model under models[]; cascade model > route > global network.*, explicit 0 wins at its level. Once the route has ≥8 healthy attempts the configured value becomes a ceiling and the effective deadline tightens to clamp(3 × p95 of this route's recent healthy first-content durations, 120s, 600s); stalled/killed attempts are never sampled. File-only (no env)." + desc_zh: "按线路的模型内容期限,阶段一(#3254):响应开始后无模型内容的上限毫秒数;超时则中止卡死上游并以带内错误结束回合。也可在 models[] 按模型设置;级联 model > route > global network.*,显式 0 在所在层生效。线路积累 ≥8 次健康尝试后,配置值转为上限,有效期限收紧为 clamp(该线路近期健康首内容耗时的 3 × p95, 120s, 600s);卡死或被中止的尝试从不采样。仅配置文件可设(无 env)。" + site_en: "Model-content deadline, phase 1 (#3254), per route and per model (`models[]`); on expiry bili aborts the stalled upstream stream and ends the turn with one clean in-band error — no internal refetch. Cascade model > route > global `network.*`; explicit 0 wins at its level; once warmed (≥8 healthy attempts) the configured value acts as a ceiling tightened toward the route's measured first-content pace (clamp 3×p95, 120s–600s); file-only." + site_zh: "模型内容期限,阶段一(#3254),按线路与按模型(`models[]`);到期则中止卡死的上游流并以一条干净的带内错误结束回合——不做内部盲重取。级联 model > route > global `network.*`;显式 0 在所在层生效;线路预热后(≥8 次健康尝试)配置值转为上限,向该线路实测首内容水位收紧(clamp 3×p95,120s–600s);仅文件可设。" - key: contentGapTimeoutMs type: number default: "unset (inherits firstContentTimeoutMs)" group: providers id: cr.gapdeadline render: route-row - desc_en: "Per-route model-content deadline, phase 2 (#3254): max ms between consecutive model-content events after the first arrived; re-armed on every content event. Absent inherits the resolved firstContentTimeoutMs; explicit 0 disables the gap phase only. Same cascade and file-only rationale as firstContentTimeoutMs." - desc_zh: "按线路的模型内容期限,阶段二(#3254):首个内容之后相邻模型内容事件间的上限毫秒数,逐事件重新计时。缺省继承解析后的 firstContentTimeoutMs;显式 0 仅禁用阶段二。级联与仅文件可设的理由同 firstContentTimeoutMs。" - site_en: "Model-content deadline, phase 2 (#3254), per route and per model (`models[]`); bounds silence between content events after the first arrived (pings/comments never count); absent inherits the resolved `firstContentTimeoutMs`. File-only." - site_zh: "模型内容期限,阶段二(#3254),按线路与按模型(`models[]`);约束首个内容之后相邻内容事件间的静默(ping/注释不算内容);缺省继承解析后的 `firstContentTimeoutMs`。仅文件可设。" + desc_en: "Per-route model-content deadline, phase 2 (#3254): max ms between consecutive model-content events after the first arrived; re-armed on every content event. Absent inherits the resolved firstContentTimeoutMs; explicit 0 disables the gap phase only. Same cascade, same route-adaptive ceiling (its own inter-content-gap samples) and file-only rationale as firstContentTimeoutMs." + desc_zh: "按线路的模型内容期限,阶段二(#3254):首个内容之后相邻模型内容事件间的上限毫秒数,逐事件重新计时。缺省继承解析后的 firstContentTimeoutMs;显式 0 仅禁用阶段二。级联、线路自适应上限(用自己的内容间隔样本)与仅文件可设的理由同 firstContentTimeoutMs。" + site_en: "Model-content deadline, phase 2 (#3254), per route and per model (`models[]`); bounds silence between content events after the first arrived (pings/comments never count); absent inherits the resolved `firstContentTimeoutMs`; same route-adaptive ceiling as `firstContentTimeoutMs`. File-only." + site_zh: "模型内容期限,阶段二(#3254),按线路与按模型(`models[]`);约束首个内容之后相邻内容事件间的静默(ping/注释不算内容);缺省继承解析后的 `firstContentTimeoutMs`;线路自适应上限同 `firstContentTimeoutMs`。仅文件可设。" - key: compactionOptIn type: boolean default: "false" From d0158d5c99b4a3d238510c636d8fab47bae86f92 Mon Sep 17 00:00:00 2001 From: awork Date: Fri, 16 Oct 2026 10:27:06 +0800 Subject: [PATCH 6/7] feat: out-of-the-box model-content deadline (built-in 720s, ceiling-following adaptive) (#3254) Round 3 per owner mandate (out-of-the-box / self-adapting / zero-config): - resolver now falls back to FC_BUILTIN_DEFAULT_MS=720000 when no tier sets a value, so every deployment is bounded with NO config written; explicit 0 at any level still opts out (escape hatch). source gains 'builtin'. - adaptive clamp upper bound now FOLLOWS the resolved ceiling instead of a hardcoded 600s (removes FC_ADAPTIVE_CAP_MS): warm routes tighten toward clamp(3*p95, 120s, resolvedCeiling); slow routes keep their full ceiling. - gap phase inherits the resolved initial CEILING (raw configured/builtin), not the adaptive-tightened value. - corrected the in-band-error design note: an in-band error fails the turn cleanly and lets clients that HAVE a retry ladder back off; it does not assume every client auto-retries (verification was against a fork, not native codex). - docs en/zh + website seeds updated to on-by-default semantics. --- CONFIGURATION.md | 16 ++-- CONFIGURATION.zh-CN.md | 16 ++-- src/config.ts | 81 +++++++++++-------- src/loop/core.ts | 13 +-- .../issue3254-first-content-deadline.test.ts | 31 ++++--- website/config-reference/process.yaml | 12 +-- website/config-reference/providers.yaml | 10 +-- 7 files changed, 99 insertions(+), 80 deletions(-) diff --git a/CONFIGURATION.md b/CONFIGURATION.md index 7b220070be..28d86ac176 100644 --- a/CONFIGURATION.md +++ b/CONFIGURATION.md @@ -147,8 +147,8 @@ This index is generated from `website/config-reference/*.yaml` — edit the seed | `network.streamKeepAliveMs` | number | 15000 (0 disables) | BILI_STREAM_KEEPALIVE_MS | Downstream stream keep-alive ping interval between model chunks; 0 disables pinging. | | `network.preflightHoldMs` | number | 30000 | BILI_PREFLIGHT_HOLD_MS | How long a failed preflight holds before the request is allowed through unreconciled. | | `network.preflightDeadEndCooldownMs` | number | 300000 | BILI_PREFLIGHT_DEAD_END_COOLDOWN_MS | Cooldown after a dead-end preflight before retrying reconciliation for the same session. | -| `network.firstContentTimeoutMs` | number | unset (off) | — | Model-content deadline, phase 1 (#3254): max ms an upstream attempt may go with zero model content from response start before bili aborts the stalled stream and ends the turn with one clean in-band error (no internal refetch — the client's retry ladder re-arms). Also settable per route / per model under providers; cascade model > route > global, explicit 0 wins at its level. Once a route has ≥8 healthy attempts, the configured value becomes a ceiling and the effective deadline tightens to `clamp(3 × p95 of the route's recent healthy first-content durations, 120s, 600s)` (never above the configured value). File-only by design. | -| `network.contentGapTimeoutMs` | number | unset (inherits firstContentTimeoutMs) | — | Model-content deadline, phase 2 (#3254): max ms between consecutive model-content events after the first arrived; re-armed on every content event (pings/comments never count). Absent inherits the resolved firstContentTimeoutMs; explicit 0 disables the gap phase only. Same three-level cascade, same route-adaptive ceiling and file-only rationale as firstContentTimeoutMs. | +| `network.firstContentTimeoutMs` | number | `720000` (on by default) | — | Model-content deadline, phase 1 (#3254): max ms an upstream attempt may go with zero model content from response start before bili aborts the stalled stream and ends the turn with one clean in-band error (no internal refetch — the turn fails cleanly; clients with a retry ladder back off and retry, clients without one simply stop). **On by default at 720000ms** so every deployment is bounded out of the box; set a value to tighten/override, or `0` to disable (opt-out override of the built-in). Also settable per route / per model under providers; cascade model > route > global, explicit 0 wins at its level. Once a route has ≥8 healthy attempts, the resolved value becomes a ceiling and the effective deadline tightens to `clamp(3 × p95 of the route's recent healthy first-content durations, 120s, that resolved ceiling)` (never above it). File-only by design. | +| `network.contentGapTimeoutMs` | number | inherits firstContentTimeoutMs (`720000` default) | — | Model-content deadline, phase 2 (#3254): max ms between consecutive model-content events after the first arrived; re-armed on every content event (pings/comments never count). On by default through inheritance (absent inherits the resolved firstContentTimeoutMs ceiling); explicit 0 disables the gap phase only (opt-out). Same three-level cascade, same route-adaptive ceiling (which follows the resolved value) and file-only rationale as firstContentTimeoutMs. | | `network.replayRetryMax` | number | 3 (1 disables replays) | BILI_REPLAY_RETRY_MAX | Maximum transport-failure replays of an identical request before surfacing the error. | | `network.replayRetryBaseMs` | number | 1500 (0 disables the delay) | BILI_REPLAY_RETRY_BASE_MS | Base backoff delay between replays (exponential). | | `network.maxShrinkPerCompress` | number | unset | BILI_MAX_SHRINK_PER_COMPRESS | Cap on how much one compression may shrink history (anti-collapse guard). | @@ -311,8 +311,8 @@ This index is generated from `website/config-reference/*.yaml` — edit the seed | `apiKeyEnv` | string (env var name) | unset | — | Lane credential override (#2336): replace the client's credential with this env variable's value for this lane. Exactly one of apiKeyEnv/credentialRef; see [Lane credentials](#lane-credentials-apikeyenv--credentialref). | | `credentialRef` | string (store name) | unset | — | Lane credential override via the private summary-credential store (secret:NAME semantics). | | `calibration` | { disabled?: boolean, max?: number } | unset (learned k̂ applies) | — | Per-route k̂ calibration policy (#3211): opt-out or clamp for routes whose usage reports inflate; see [Route-level k̂ calibration policy](#route-level-k-calibration-policy). | -| `firstContentTimeoutMs` | number | unset (off) | — | Per-route model-content deadline, phase 1 (#3254): max ms without model content from response start before bili aborts the stalled upstream and ends the turn in-band. Also settable per model under models[]; cascade model > route > global network.*, explicit 0 wins at its level. File-only (no env). | -| `contentGapTimeoutMs` | number | unset (inherits firstContentTimeoutMs) | — | Per-route model-content deadline, phase 2 (#3254): max ms between consecutive model-content events after the first arrived; re-armed on every content event. Absent inherits the resolved firstContentTimeoutMs; explicit 0 disables the gap phase only. Same cascade and file-only rationale as firstContentTimeoutMs. | +| `firstContentTimeoutMs` | number | unset (→ global, else built-in `720000`) | — | Per-route model-content deadline, phase 1 (#3254): max ms without model content from response start before bili aborts the stalled upstream and ends the turn in-band. On by default via the built-in (unset falls through to global then the 720000 builtin); set here to override per route, or `0` to disable for this route. Also settable per model under models[]; cascade model > route > global network.*, explicit 0 wins at its level. File-only (no env). | +| `contentGapTimeoutMs` | number | unset (inherits firstContentTimeoutMs) | — | Per-route model-content deadline, phase 2 (#3254): max ms between consecutive model-content events after the first arrived; re-armed on every content event. Absent inherits the resolved firstContentTimeoutMs ceiling (built-in 720000 when nothing else is set); explicit 0 disables the gap phase only. Same cascade and file-only rationale as firstContentTimeoutMs. | | `compactionOptIn` | boolean | false | BILI_NON_HTTP_PROVIDERS | Named entries only: opt a non-http(s)-baseUrl provider into compaction ownership (pi/omp lanes); unions with env BILI_NON_HTTP_PROVIDERS. | **Environment-only variables** @@ -792,8 +792,8 @@ Since #2030 every pure-behavior knob has a config-file key alongside its env var "preflightDeadEndCooldownMs": 300000, // cooldown after a dead-end preflight verdict "proxyKeepAliveMaxMs": 55000, // upstream-proxy connection reuse cap; 0 uncapped "postResponseLingerMs": 5000, // post-response close linger budget (#1982) - "firstContentTimeoutMs": 180000, // #3254: max ms without model content from response start; off by default; per-route/model overrides in providers - "contentGapTimeoutMs": 600000, // #3254: max ms between content events after the first; absent = inherits firstContentTimeoutMs + "firstContentTimeoutMs": 180000, // #3254: max ms without model content from response start; ON by default at 720000 (shown tightened here); explicit 0 disables; per-route/model overrides in providers + "contentGapTimeoutMs": 600000, // #3254: max ms between content events after the first; absent = inherits firstContentTimeoutMs (its 720000 default when unset); explicit 0 disables this phase only "allowPrivateHttpProviders": false // opt-in: private-network plain-HTTP providers (#2984) }, @@ -853,9 +853,9 @@ Since #2030 every pure-behavior knob has a config-file key alongside its env var ### `network` - **Type:** `{ upstreamTimeoutMs?: number; requestWatchdogMs?: number; replayRetryMax?: number; replayRetryBaseMs?: number; maxShrinkPerCompress?: number; keepAliveTimeoutMs?: number; clientErrorBackstopMs?: number; exposureLogIntervalMs?: number; streamKeepAliveMs?: number; preflightHoldMs?: number; preflightDeadEndCooldownMs?: number; proxyKeepAliveMaxMs?: number; postResponseLingerMs?: number; firstContentTimeoutMs?: number; contentGapTimeoutMs?: number; allowPrivateHttpProviders?: boolean }` -- **Default:** `{}` (built-ins: `720000`, `2× upstream timeout`, `3`, `1500`, unset, `5000`, `30000`, `3600000`, `15000`, `30000`, `300000`, `55000`, `5000`, `false`) +- **Default:** `{}` (built-ins: `720000`, `2× upstream timeout`, `3`, `1500`, unset, `5000`, `30000`, `3600000`, `15000`, `30000`, `300000`, `55000`, `5000`, `720000`, `720000`, `false`) - **Status:** ACTIVE -- **Description:** Process-wide transport and retry timing (ms). Every key except `firstContentTimeoutMs`/`contentGapTimeoutMs` (#3254, file-only — see below) resolves env > file > default via its twin (`BILI_UPSTREAM_TIMEOUT_MS`, `BILI_REQUEST_WATCHDOG_MS`, `BILI_REPLAY_RETRY_MAX`, `BILI_REPLAY_RETRY_BASE_MS`, `BILI_MAX_SHRINK_PER_COMPRESS`, `BILI_KEEP_ALIVE_TIMEOUT_MS`, `BILI_CLIENT_ERROR_BACKSTOP_MS`, `BILI_EXPOSURE_LOG_INTERVAL_MS`, `BILI_STREAM_KEEPALIVE_MS`, `BILI_PREFLIGHT_HOLD_MS`, `BILI_PREFLIGHT_DEAD_END_COOLDOWN_MS`, `BILI_PROXY_KEEPALIVE_MAX_MS`, `BILI_POST_RESPONSE_LINGER_MS`, `BILI_ALLOW_PRIVATE_HTTP_PROVIDERS`). Semantics: `upstreamTimeoutMs` bounds time-to-first-byte and inter-chunk silence end-to-end (#551); `requestWatchdogMs` is the total-request budget (a negative value opts out); `replayRetryMax` counts attempts after transient upstream rejections (#189/#1688); `maxShrinkPerCompress` caps how much one compress may shrink the request before bili steers toward smaller ranges (#189); `streamKeepAliveMs` emits an SSE comment when the response has written zero bytes for that long (#1647); `proxyKeepAliveMaxMs` caps connection reuse through an upstream proxy below its recycle cadence (#1263), `0` uncaps; `postResponseLingerMs` is the graceful-close budget after a proxy-initiated post-response close — the socket is held waiting for the peer's FIN/TLS close_notify and destroyed at the deadline (`reason=linger-backstop`) (#1982); `allowPrivateHttpProviders` is an opt-in (#2984): when true, plain-HTTP provider baseUrls on private networks (RFC1918, 127/8, link-local, IPv6 ULA/link-local, `*.local`/`.lan`/`.internal`) are admitted by the agent-registry intake and summary-endpoint guards that otherwise trust loopback plaintext only — public IPs/domains and CGNAT (100.64/10) stay refused regardless, and the default `false` keeps pre-#2984 behavior byte-for-byte; `firstContentTimeoutMs`/`contentGapTimeoutMs` are FILE-ONLY model-content deadlines (#3254): phase 1 bounds how long an upstream attempt may go with zero model content (text/reasoning deltas, tool calls, signatures) from response start, phase 2 (`contentGapTimeoutMs`, re-armed on every content event — pings/comments never count) bounds silence between content events after the first arrived; on expiry bili aborts the stalled upstream stream and ends the turn with one clean in-band error (context intact, resend your last message) — no internal refetch, so the client's own retry ladder re-arms; cascade model > provider route > global `network.*`, explicit 0 wins at its level, gap inherits the resolved initial when absent; deliberately no env layer because per-route/per-model values cannot be expressed as one process-wide variable; **route-adaptive refinement** (#3254 follow-up): once a route has accumulated ≥8 healthy attempts (completed with no truncation/error — stalled or killed attempts are never sampled), each configured phase value becomes a ceiling and the effective deadline tightens to `clamp(3 × p95 of that route's recent healthy first-content durations, 120s, 600s)` per phase (gap uses its own inter-content-gap samples); samples live in session metadata (persisted across restarts, ring of 32) and the fire log records `source=config|adaptive` so a fired deadline can be traced to which value class produced it. Full details in the [env table](#config-file-keys-for-environment-knobs-2030). +- **Description:** Process-wide transport and retry timing (ms). Every key except `firstContentTimeoutMs`/`contentGapTimeoutMs` (#3254, file-only — see below) resolves env > file > default via its twin (`BILI_UPSTREAM_TIMEOUT_MS`, `BILI_REQUEST_WATCHDOG_MS`, `BILI_REPLAY_RETRY_MAX`, `BILI_REPLAY_RETRY_BASE_MS`, `BILI_MAX_SHRINK_PER_COMPRESS`, `BILI_KEEP_ALIVE_TIMEOUT_MS`, `BILI_CLIENT_ERROR_BACKSTOP_MS`, `BILI_EXPOSURE_LOG_INTERVAL_MS`, `BILI_STREAM_KEEPALIVE_MS`, `BILI_PREFLIGHT_HOLD_MS`, `BILI_PREFLIGHT_DEAD_END_COOLDOWN_MS`, `BILI_PROXY_KEEPALIVE_MAX_MS`, `BILI_POST_RESPONSE_LINGER_MS`, `BILI_ALLOW_PRIVATE_HTTP_PROVIDERS`). Semantics: `upstreamTimeoutMs` bounds time-to-first-byte and inter-chunk silence end-to-end (#551); `requestWatchdogMs` is the total-request budget (a negative value opts out); `replayRetryMax` counts attempts after transient upstream rejections (#189/#1688); `maxShrinkPerCompress` caps how much one compress may shrink the request before bili steers toward smaller ranges (#189); `streamKeepAliveMs` emits an SSE comment when the response has written zero bytes for that long (#1647); `proxyKeepAliveMaxMs` caps connection reuse through an upstream proxy below its recycle cadence (#1263), `0` uncaps; `postResponseLingerMs` is the graceful-close budget after a proxy-initiated post-response close — the socket is held waiting for the peer's FIN/TLS close_notify and destroyed at the deadline (`reason=linger-backstop`) (#1982); `allowPrivateHttpProviders` is an opt-in (#2984): when true, plain-HTTP provider baseUrls on private networks (RFC1918, 127/8, link-local, IPv6 ULA/link-local, `*.local`/`.lan`/`.internal`) are admitted by the agent-registry intake and summary-endpoint guards that otherwise trust loopback plaintext only — public IPs/domains and CGNAT (100.64/10) stay refused regardless, and the default `false` keeps pre-#2984 behavior byte-for-byte; `firstContentTimeoutMs`/`contentGapTimeoutMs` are FILE-ONLY model-content deadlines (#3254): phase 1 bounds how long an upstream attempt may go with zero model content (text/reasoning deltas, tool calls, signatures) from response start, phase 2 (`contentGapTimeoutMs`, re-armed on every content event — pings/comments never count) bounds silence between content events after the first arrived; on expiry bili aborts the stalled upstream stream and ends the turn with one clean in-band error (context intact, resend your last message) — no internal refetch, so the turn fails cleanly: clients with a retry ladder back off and retry, clients without one simply stop; on by default at the `720000` built-in (explicit `0` at any level opts out); cascade model > provider route > global `network.*`, explicit 0 wins at its level, gap inherits the resolved initial ceiling when absent; deliberately no env layer because per-route/per-model values cannot be expressed as one process-wide variable; **route-adaptive refinement** (#3254 follow-up): once a route has accumulated ≥8 healthy attempts (completed with no truncation/error — stalled or killed attempts are never sampled), each resolved phase value becomes a ceiling and the effective deadline tightens to `clamp(3 × p95 of that route's recent healthy first-content durations, 120s, that resolved ceiling)` per phase (gap uses its own inter-content-gap samples); samples live in session metadata (persisted across restarts, ring of 32) and the fire log records `source=config|adaptive` so a fired deadline can be traced to which value class produced it. Full details in the [env table](#config-file-keys-for-environment-knobs-2030). ### `persist` diff --git a/CONFIGURATION.zh-CN.md b/CONFIGURATION.zh-CN.md index 744ac62644..a08dbbaa63 100644 --- a/CONFIGURATION.zh-CN.md +++ b/CONFIGURATION.zh-CN.md @@ -147,8 +147,8 @@ | `network.streamKeepAliveMs` | number | 15000 (0 disables) | BILI_STREAM_KEEPALIVE_MS | 模型块之间的下游流保活心跳间隔;0 禁用心跳。 | | `network.preflightHoldMs` | number | 30000 | BILI_PREFLIGHT_HOLD_MS | preflight 失败后,允许请求未校准放行的持有时长。 | | `network.preflightDeadEndCooldownMs` | number | 300000 | BILI_PREFLIGHT_DEAD_END_COOLDOWN_MS | 死胡同 preflight 后,对同一会话重试校准的冷却时长。 | -| `network.firstContentTimeoutMs` | number | unset (off) | — | 模型内容期限,阶段一(#3254):上游一次尝试从响应开始到首个模型内容之间的最长毫秒数;超时则 bili 中止卡死流并以一条干净的带内错误结束回合(不做内部盲重取——客户端重试梯照常接管)。providers 内也可按线路/按模型设置;级联 model > route > global,显式 0 在所在层生效。线路积累 ≥8 次健康尝试后,配置值转为上限:有效期限收紧为 `clamp(该线路近期健康首内容耗时的 3 × p95, 120s, 600s)`(不会高于配置值)。按设计仅配置文件可设。 | -| `network.contentGapTimeoutMs` | number | unset (inherits firstContentTimeoutMs) | — | 模型内容期限,阶段二(#3254):首个内容到达之后,相邻两个模型内容事件之间的最长毫秒数;每个内容事件重新计时(ping/注释不算内容)。缺省继承解析后的 firstContentTimeoutMs;显式 0 仅禁用阶段二。三级级联、线路自适应上限与仅文件可设的理由同 firstContentTimeoutMs。 | +| `network.firstContentTimeoutMs` | number | `720000`(默认开启) | — | 模型内容期限,阶段一(#3254):上游一次尝试从响应开始到首个模型内容之间的最长毫秒数;超时则 bili 中止卡死流并以一条干净的带内错误结束回合(不做内部盲重取——回合干净失败:有重试梯的客户端退避后重试,没有的则直接停止)。**默认以 720000ms 开启**,让每个部署开箱即用即有界;设值可收紧/覆盖,或设 `0` 关闭(对内置值的显式豁免)。providers 内也可按线路/按模型设置;级联 model > route > global,显式 0 在所在层生效。线路积累 ≥8 次健康尝试后,解析值转为上限:有效期限收紧为 `clamp(该线路近期健康首内容耗时的 3 × p95, 120s, 该解析上限)`(不会高于它)。按设计仅配置文件可设。 | +| `network.contentGapTimeoutMs` | number | 继承 firstContentTimeoutMs(默认 `720000`) | — | 模型内容期限,阶段二(#3254):首个内容到达之后,相邻两个模型内容事件之间的最长毫秒数;每个内容事件重新计时(ping/注释不算内容)。通过继承默认开启(缺省继承解析后的 firstContentTimeoutMs 上限);显式 0 仅禁用阶段二(豁免)。三级级联、线路自适应上限(跟随解析值)与仅文件可设的理由同 firstContentTimeoutMs。 | | `network.replayRetryMax` | number | 3 (1 disables replays) | BILI_REPLAY_RETRY_MAX | 同一请求在向上游暴露错误前的最大传输失败重放次数。 | | `network.replayRetryBaseMs` | number | 1500 (0 disables the delay) | BILI_REPLAY_RETRY_BASE_MS | 重放之间的基础退避延迟(指数增长)。 | | `network.maxShrinkPerCompress` | number | unset | BILI_MAX_SHRINK_PER_COMPRESS | 单次压缩允许收缩历史的上限(防塌缩护栏)。 | @@ -311,8 +311,8 @@ | `apiKeyEnv` | string (env var name) | unset | — | 通道凭据覆盖(#2336):用该环境变量的值替换客户端凭据。与 credentialRef 二选一;见[通道凭据](#lane-credentials-apikeyenv--credentialref)。 | | `credentialRef` | string (store name) | unset | — | 经私有摘要凭据存储的通道凭据覆盖(secret:NAME 语义)。 | | `calibration` | { disabled?: boolean, max?: number } | unset (learned k̂ applies) | — | 按路由的 k̂ 校准策略(#3211):对虚报报数线路做退出或钳制;见[线路级 k̂ 校准策略](#route-level-k-calibration-policy)。 | -| `firstContentTimeoutMs` | number | unset (off) | — | 按线路的模型内容期限,阶段一(#3254):响应开始后无模型内容的上限毫秒数;超时则中止卡死上游并以带内错误结束回合。也可在 models[] 按模型设置;级联 model > route > global network.*,显式 0 在所在层生效。仅配置文件可设(无 env)。 | -| `contentGapTimeoutMs` | number | unset (inherits firstContentTimeoutMs) | — | 按线路的模型内容期限,阶段二(#3254):首个内容之后相邻模型内容事件间的上限毫秒数,逐事件重新计时。缺省继承解析后的 firstContentTimeoutMs;显式 0 仅禁用阶段二。级联与仅文件可设的理由同 firstContentTimeoutMs。 | +| `firstContentTimeoutMs` | number | unset(→ global,否则内置 `720000`) | — | 按线路的模型内容期限,阶段一(#3254):响应开始后无模型内容的上限毫秒数;超时则中止卡死上游并以带内错误结束回合。通过内置值默认开启(缺省落到 global 再到内置 720000);在此按线路覆盖,或设 `0` 对该线路关闭。也可在 models[] 按模型设置;级联 model > route > global network.*,显式 0 在所在层生效。仅配置文件可设(无 env)。 | +| `contentGapTimeoutMs` | number | unset (inherits firstContentTimeoutMs) | — | 按线路的模型内容期限,阶段二(#3254):首个内容之后相邻模型内容事件间的上限毫秒数,逐事件重新计时。缺省继承解析后的 firstContentTimeoutMs 上限(其余均未设时为内置 720000);显式 0 仅禁用阶段二。级联与仅文件可设的理由同 firstContentTimeoutMs。 | | `compactionOptIn` | boolean | false | BILI_NON_HTTP_PROVIDERS | 仅命名条目:把非 http(s) baseUrl 供应商纳入压缩所有权(pi/omp 车道);与 env BILI_NON_HTTP_PROVIDERS 取并集。 | **仅环境变量(无配置文件键)** @@ -792,8 +792,8 @@ "preflightDeadEndCooldownMs": 300000, // preflight 死路判定后的冷却 "proxyKeepAliveMaxMs": 55000, // 经上游代理的连接复用上限;0 不限 "postResponseLingerMs": 5000, // 响应后关闭的优雅关闭预算(#1982) - "firstContentTimeoutMs": 180000, // #3254:响应开始后无模型内容的上限毫秒数;默认关闭;providers 内可按线路/模型覆盖 - "contentGapTimeoutMs": 600000, // #3254:首个内容之后相邻内容事件间的上限毫秒数;缺省继承 firstContentTimeoutMs + "firstContentTimeoutMs": 180000, // #3254:响应开始后无模型内容的上限毫秒数;默认以 720000 开启(此处示例收紧);显式 0 关闭;providers 内可按线路/模型覆盖 + "contentGapTimeoutMs": 600000, // #3254:首个内容之后相邻内容事件间的上限毫秒数;缺省继承 firstContentTimeoutMs(未设时其默认 720000);显式 0 仅禁用本阶段 "allowPrivateHttpProviders": false // 私网明文 HTTP provider 可选放行(#2984) }, @@ -853,9 +853,9 @@ ### `network` - **类型:** `{ upstreamTimeoutMs?: number; requestWatchdogMs?: number; replayRetryMax?: number; replayRetryBaseMs?: number; maxShrinkPerCompress?: number; keepAliveTimeoutMs?: number; clientErrorBackstopMs?: number; exposureLogIntervalMs?: number; streamKeepAliveMs?: number; preflightHoldMs?: number; preflightDeadEndCooldownMs?: number; proxyKeepAliveMaxMs?: number; postResponseLingerMs?: number; firstContentTimeoutMs?: number; contentGapTimeoutMs?: number; allowPrivateHttpProviders?: boolean }` -- **默认:** `{}`(内置值依次为:`720000`、`2× 上游超时`、`3`、`1500`、未设置、`5000`、`30000`、`3600000`、`15000`、`30000`、`300000`、`55000`、`5000`、`false`) +- **默认:** `{}`(内置值依次为:`720000`、`2× 上游超时`、`3`、`1500`、未设置、`5000`、`30000`、`3600000`、`15000`、`30000`、`300000`、`55000`、`5000`、`720000`、`720000`、`false`) - **状态:** ACTIVE -- **说明:** 进程级传输与重试时序(毫秒)。除 `firstContentTimeoutMs`/`contentGapTimeoutMs`(#3254,仅文件可设——见下)外,每个键按 env > file > default 解析,对应环境变量分别为 `BILI_UPSTREAM_TIMEOUT_MS`、`BILI_REQUEST_WATCHDOG_MS`、`BILI_REPLAY_RETRY_MAX`、`BILI_REPLAY_RETRY_BASE_MS`、`BILI_MAX_SHRINK_PER_COMPRESS`、`BILI_KEEP_ALIVE_TIMEOUT_MS`、`BILI_CLIENT_ERROR_BACKSTOP_MS`、`BILI_EXPOSURE_LOG_INTERVAL_MS`、`BILI_STREAM_KEEPALIVE_MS`、`BILI_PREFLIGHT_HOLD_MS`、`BILI_PREFLIGHT_DEAD_END_COOLDOWN_MS`、`BILI_PROXY_KEEPALIVE_MAX_MS`、`BILI_POST_RESPONSE_LINGER_MS`、`BILI_ALLOW_PRIVATE_HTTP_PROVIDERS`。要点:`upstreamTimeoutMs` 端到端约束首字节时间与 chunk 间静默(#551);`requestWatchdogMs` 是单请求总预算(负值 = 退出该约束);`replayRetryMax` 统计瞬态上游拒绝后的尝试次数(#189/#1688);`maxShrinkPerCompress` 限制单次压缩对请求的最大缩减比例,超限则引导模型选择更小的范围(#189);`streamKeepAliveMs` 在响应零字节写出超过该时长时发一条 SSE 注释行(#1647);`proxyKeepAliveMaxMs` 把经上游代理的连接复用窗口压在上游回收周期之下(#1263),`0` 取消上限;`postResponseLingerMs` 是代理主动发起的响应后关闭的优雅关闭预算 —— socket 被挂起等待对端 FIN/TLS close_notify,到点销毁(`reason=linger-backstop`)(#1982);`allowPrivateHttpProviders` 是可选开关(#2984):为 true 时,私网(RFC1918、127/8、link-local、IPv6 ULA/link-local、`*.local`/`.lan`/`.internal`)上的明文 HTTP provider baseUrl 被 agent-registry 接收与摘要端点守卫放行——这些守卫默认只信任回环明文;公网 IP/域名与 CGNAT(100.64/10)无论如何都拒;默认 false 保持 #2984 之前的行为逐字不变;`firstContentTimeoutMs`/`contentGapTimeoutMs` 是仅文件可设的模型内容期限(#3254):阶段一约束上游一次尝试从响应开始到首个模型内容(文本/推理增量、工具调用、签名)的最长等待,阶段二(`contentGapTimeoutMs`,每个内容事件重新计时——ping/注释不算内容)约束首个内容之后相邻内容事件间的静默;到期则 bili 中止卡死的上游流并以一条干净的带内错误结束回合(上下文完好,重发最后一条消息即可继续)——不做内部盲重取,客户端自身的重试梯照常接管;级联 model > provider route > global `network.*`,显式 0 在所在层生效,gap 缺省继承解析后的 initial;刻意不设 env 层,因为按线路/按模型的取值无法用单一进程级变量表达;**线路自适应收紧**(#3254 follow-up):某线路积累 ≥8 次健康尝试(正常完成、无截断/错误——卡死或被期限中止的尝试从不采样)后,每个已配置阶段值转为上限,有效期限按阶段收紧为 `clamp(该线路近期健康首内容耗时的 3 × p95, 120s, 600s)`(gap 阶段用自己的内容间隔样本);样本存于会话 metadata(跨重启持久化,环形窗口 32),触发日志记录 `source=config|adaptive`,可追溯触发值来自哪一类。完整细节见[环境变量表](#环境变量的配置键对照-2030)。 +- **说明:** 进程级传输与重试时序(毫秒)。除 `firstContentTimeoutMs`/`contentGapTimeoutMs`(#3254,仅文件可设——见下)外,每个键按 env > file > default 解析,对应环境变量分别为 `BILI_UPSTREAM_TIMEOUT_MS`、`BILI_REQUEST_WATCHDOG_MS`、`BILI_REPLAY_RETRY_MAX`、`BILI_REPLAY_RETRY_BASE_MS`、`BILI_MAX_SHRINK_PER_COMPRESS`、`BILI_KEEP_ALIVE_TIMEOUT_MS`、`BILI_CLIENT_ERROR_BACKSTOP_MS`、`BILI_EXPOSURE_LOG_INTERVAL_MS`、`BILI_STREAM_KEEPALIVE_MS`、`BILI_PREFLIGHT_HOLD_MS`、`BILI_PREFLIGHT_DEAD_END_COOLDOWN_MS`、`BILI_PROXY_KEEPALIVE_MAX_MS`、`BILI_POST_RESPONSE_LINGER_MS`、`BILI_ALLOW_PRIVATE_HTTP_PROVIDERS`。要点:`upstreamTimeoutMs` 端到端约束首字节时间与 chunk 间静默(#551);`requestWatchdogMs` 是单请求总预算(负值 = 退出该约束);`replayRetryMax` 统计瞬态上游拒绝后的尝试次数(#189/#1688);`maxShrinkPerCompress` 限制单次压缩对请求的最大缩减比例,超限则引导模型选择更小的范围(#189);`streamKeepAliveMs` 在响应零字节写出超过该时长时发一条 SSE 注释行(#1647);`proxyKeepAliveMaxMs` 把经上游代理的连接复用窗口压在上游回收周期之下(#1263),`0` 取消上限;`postResponseLingerMs` 是代理主动发起的响应后关闭的优雅关闭预算 —— socket 被挂起等待对端 FIN/TLS close_notify,到点销毁(`reason=linger-backstop`)(#1982);`allowPrivateHttpProviders` 是可选开关(#2984):为 true 时,私网(RFC1918、127/8、link-local、IPv6 ULA/link-local、`*.local`/`.lan`/`.internal`)上的明文 HTTP provider baseUrl 被 agent-registry 接收与摘要端点守卫放行——这些守卫默认只信任回环明文;公网 IP/域名与 CGNAT(100.64/10)无论如何都拒;默认 false 保持 #2984 之前的行为逐字不变;`firstContentTimeoutMs`/`contentGapTimeoutMs` 是仅文件可设的模型内容期限(#3254):阶段一约束上游一次尝试从响应开始到首个模型内容(文本/推理增量、工具调用、签名)的最长等待,阶段二(`contentGapTimeoutMs`,每个内容事件重新计时——ping/注释不算内容)约束首个内容之后相邻内容事件间的静默;到期则 bili 中止卡死的上游流并以一条干净的带内错误结束回合(上下文完好,重发最后一条消息即可继续)——不做内部盲重取,回合干净失败:有重试梯的客户端退避后重试,没有的则直接停止;默认以 `720000` 内置值开启(任意层显式 `0` 即豁免);级联 model > provider route > global `network.*`,显式 0 在所在层生效,gap 缺省继承解析后的 initial 上限;刻意不设 env 层,因为按线路/按模型的取值无法用单一进程级变量表达;**线路自适应收紧**(#3254 follow-up):某线路积累 ≥8 次健康尝试(正常完成、无截断/错误——卡死或被期限中止的尝试从不采样)后,每个解析阶段值转为上限,有效期限按阶段收紧为 `clamp(该线路近期健康首内容耗时的 3 × p95, 120s, 该解析上限)`(gap 阶段用自己的内容间隔样本);样本存于会话 metadata(跨重启持久化,环形窗口 32),触发日志记录 `source=config|adaptive`,可追溯触发值来自哪一类。完整细节见[环境变量表](#环境变量的配置键对照-2030)。 ### `persist` diff --git a/src/config.ts b/src/config.ts index dd8011dc18..3f5a6de42c 100644 --- a/src/config.ts +++ b/src/config.ts @@ -2430,43 +2430,54 @@ export function parseDeadlineMs(fieldPath: string, value: unknown): number | und // #3254: three-level cascade (model > provider route > global network.*) for // the model-content deadlines. An explicit 0 wins AT ITS LEVEL (disables that -// phase, no fall-through); an absent field inherits the next level down. -// contentGapTimeoutMs inherits the resolved initial deadline when left absent. -// Returns undefined when every phase is off or absent (feature fully off). +// phase, no fall-through) AND overrides the built-in default — it is the opt-out +// escape hatch for legitimately-slow routes. An ABSENT field falls back to the +// built-in out-of-the-box default below (the feature is ON by default, #3254 +// round-3 "开箱即用"). contentGapTimeoutMs inherits the resolved initial ceiling +// when left absent. +/** #3254 round-3 out-of-the-box default (owner directive): when no tier sets a + * value, each phase defaults to this bound instead of staying off. Chosen at + * the conservative end of the candidate band — high enough not to false-kill + * legitimate long prefills, still under the codex-webui client's ~780s content + * watchdog so bili acts first with a clean logged error rather than the client + * killing opaquely. SINGLE named constant: retuning the default is one line. */ +export const FC_BUILTIN_DEFAULT_MS = 720_000; /** #3254 follow-up (owner point 5): per-route adaptive refinement of the - * configured deadlines. Each route's HEALTHY first-content durations are - * sampled into session metadata as requests complete cleanly; once a route - * accumulates enough samples its deadline stops being a guess and hugs its - * own normal water level. */ + * deadlines. Each route's HEALTHY first-content durations are sampled into + * session metadata as requests complete cleanly; once a route accumulates + * enough samples its deadline stops being a guess and hugs its own normal + * water level. */ export const FC_ADAPTIVE_MIN_SAMPLES = 8; /** Per-route sample window (oldest evicted past this). */ export const FC_ADAPTIVE_WINDOW = 32; -/** Owner-specified clamp band for the adaptive value. */ +/** Adaptive floor: a warm route's deadline is never tightened below this, so a + * fast route doesn't get an absurdly tight bound. */ export const FC_ADAPTIVE_FLOOR_MS = 120_000; -export const FC_ADAPTIVE_CAP_MS = 600_000; export function fcP95(values: number[]): number { const sorted = [...values].sort((a, b) => a - b); return sorted[Math.max(0, Math.min(sorted.length - 1, Math.ceil(0.95 * sorted.length) - 1))]; } -/** clamp(3 × p95(samples), FLOOR, CAP), then capped by the EXPLICIT config - * value: explicit config is always the ceiling — adaptation only tightens - * toward measured reality, never relaxes a value the user pinned. Returns - * undefined while the route is still cold (fewer than MIN_SAMPLES). */ -export function adaptiveDeadlineMs(samples: number[] | undefined, explicitMs: number): number | undefined { +/** Clamp 3 × p95(samples) into [FLOOR, ceiling]: never tighter than FLOOR on a + * warm route, never looser than the effective ceiling (built-in or explicit). + * The ceiling FOLLOWS the resolved value instead of a hardcoded cap — otherwise + * a cold route running at e.g. 720s would discontinuously drop to ≤600s once it + * warms up (#3254 round-3 point ③). Returns undefined while the route is still + * cold (fewer than MIN_SAMPLES). */ +export function adaptiveDeadlineMs(samples: number[] | undefined, ceilingMs: number): number | undefined { if (!samples || samples.length < FC_ADAPTIVE_MIN_SAMPLES) return undefined; - const clamped = Math.max(FC_ADAPTIVE_FLOOR_MS, Math.min(FC_ADAPTIVE_CAP_MS, 3 * fcP95(samples))); - return Math.min(explicitMs, clamped); + return Math.min(Math.max(3 * fcP95(samples), FC_ADAPTIVE_FLOOR_MS), ceilingMs); } export interface FirstContentDeadlines { initialMs?: number; gapMs?: number; /** Which source produced each phase's value — lands in the fire log so - * config→adaptive cutover never changes the log format (#3254 point 5). */ - initialSource?: "config" | "adaptive"; - gapSource?: "config" | "adaptive"; + * config→adaptive cutover never changes the log format (#3254 point 5). + * "builtin" = the out-of-the-box default (no tier set a value). */ + initialSource?: "config" | "adaptive" | "builtin"; + gapSource?: "config" | "adaptive" | "builtin"; } export function resolveFirstContentDeadlines( @@ -2478,24 +2489,24 @@ export function resolveFirstContentDeadlines( ): FirstContentDeadlines | undefined { const route = findRoute(routes, upstreamUrl); const entry = model ? route?.models?.[model] : undefined; - const initialExplicit = parseDeadlineMs("firstContentTimeoutMs", entry?.firstContentTimeoutMs ?? route?.firstContentTimeoutMs ?? globals?.firstContentTimeoutMs); - const gapExplicit = parseDeadlineMs("contentGapTimeoutMs", entry?.contentGapTimeoutMs ?? route?.contentGapTimeoutMs ?? globals?.contentGapTimeoutMs); + const initialRaw = parseDeadlineMs("firstContentTimeoutMs", entry?.firstContentTimeoutMs ?? route?.firstContentTimeoutMs ?? globals?.firstContentTimeoutMs); + const gapRaw = parseDeadlineMs("contentGapTimeoutMs", entry?.contentGapTimeoutMs ?? route?.contentGapTimeoutMs ?? globals?.contentGapTimeoutMs); + // ABSENT → built-in default (feature ON by default, #3254 round-3); EXPLICIT + // 0 → phase off, overriding the built-in (opt-out escape hatch). A gap left + // absent inherits the initial ceiling and adapts on its own gap samples. + const initialCeiling = initialRaw === undefined ? FC_BUILTIN_DEFAULT_MS : initialRaw; + const gapCeiling = gapRaw === undefined ? initialCeiling : gapRaw; const out: FirstContentDeadlines = {}; - if (initialExplicit !== undefined && initialExplicit > 0) { - const adaptive = adaptiveDeadlineMs(samples?.init, initialExplicit); - out.initialMs = adaptive !== undefined ? adaptive : initialExplicit; - out.initialSource = adaptive !== undefined ? "adaptive" : "config"; + if (initialCeiling > 0) { + const adaptive = adaptiveDeadlineMs(samples?.init, initialCeiling); + out.initialMs = adaptive !== undefined ? adaptive : initialCeiling; + out.initialSource = adaptive !== undefined ? "adaptive" : (initialRaw === undefined ? "builtin" : "config"); } - // Explicit 0 disables the phase at its level (no fall-through); an ABSENT - // gap inherits the resolved initial value and adapts on gap samples. - if (gapExplicit !== undefined && gapExplicit > 0) { - const adaptive = adaptiveDeadlineMs(samples?.gap, gapExplicit); - out.gapMs = adaptive !== undefined ? adaptive : gapExplicit; - out.gapSource = adaptive !== undefined ? "adaptive" : "config"; - } else if (gapExplicit === undefined && out.initialMs !== undefined) { - const adaptive = adaptiveDeadlineMs(samples?.gap, out.initialMs); - out.gapMs = adaptive !== undefined ? adaptive : out.initialMs; - out.gapSource = adaptive !== undefined ? "adaptive" : "config"; + if (gapCeiling > 0) { + const adaptive = adaptiveDeadlineMs(samples?.gap, gapCeiling); + out.gapMs = adaptive !== undefined ? adaptive : gapCeiling; + const baseSource = gapRaw === undefined ? (initialRaw === undefined ? "builtin" : "config") : "config"; + out.gapSource = adaptive !== undefined ? "adaptive" : baseSource; } return out.initialMs !== undefined || out.gapMs !== undefined ? out : undefined; } diff --git a/src/loop/core.ts b/src/loop/core.ts index e11ecca9f2..a5c51ad1f0 100644 --- a/src/loop/core.ts +++ b/src/loop/core.ts @@ -217,8 +217,8 @@ export interface LoopCtx { firstContentDeadline?: { initialMs?: number; gapMs?: number; - initialSource?: "config" | "adaptive"; - gapSource?: "config" | "adaptive"; + initialSource?: "config" | "adaptive" | "builtin"; + gapSource?: "config" | "adaptive" | "builtin"; }; /** #3254 follow-up: route key (providers URL prefix, else host) under * which this session's healthy first-content durations are sampled in @@ -519,12 +519,13 @@ export async function* runCompressLoop( // every adapter's SSE iterator swallows that rejection as EOF, so the // consumption loop below exits cleanly) and the turn ends // with ONE clean in-band error — deliberately NO internal refetch - // (#3254 decision C-rejected: the client's retry ladder re-arms with - // backoff, which beats hammering the same stalled slot). + // (#3254 decision C-rejected: an in-band error fails the turn cleanly and + // lets clients that HAVE a retry ladder back off and retry, instead of + // hammering the same stalled slot; clients without one simply stop). let activeKill: (() => void) | null = null; let contentDeadlineTimer: ReturnType | null = null; let contentDeadlineArmedAt = 0; - let contentDeadlineInfo: { ms: number; phase: "initial" | "gap"; round: number; source: "config" | "adaptive" } | null = null; + let contentDeadlineInfo: { ms: number; phase: "initial" | "gap"; round: number; source: "config" | "adaptive" | "builtin" } | null = null; let contentDeadlineKilled = false; let contentDeadlineSettled = true; let upstreamAttemptNo = 0; @@ -553,7 +554,7 @@ export async function* runCompressLoop( loggerLog("warn", `[acp-loop] session ${ctx.session.id} ${line}`); activeKill?.(); }; - const armContentDeadline = (ms: number, phase: "initial" | "gap", roundNo: number, source: "config" | "adaptive") => { + const armContentDeadline = (ms: number, phase: "initial" | "gap", roundNo: number, source: "config" | "adaptive" | "builtin") => { clearContentDeadline(); contentDeadlineInfo = { ms, phase, round: roundNo, source }; contentDeadlineArmedAt = Date.now(); diff --git a/tests/issue3254-first-content-deadline.test.ts b/tests/issue3254-first-content-deadline.test.ts index 441eba84d8..eb2d2cc632 100644 --- a/tests/issue3254-first-content-deadline.test.ts +++ b/tests/issue3254-first-content-deadline.test.ts @@ -243,13 +243,15 @@ const ROUTES: ProviderRoutes = { }, }; -test("#3254 F: cascade — model > route > global, 0 wins at its level, gap inherits initial", () => { - // all absent → feature off - assert.equal(resolveFirstContentDeadlines({}, undefined, undefined, undefined), undefined); - // global tier alone; gap inherits the resolved initial +test("#3254 F: cascade — model > route > global, 0 wins at its level, absent falls to built-in, gap inherits initial", () => { + // all absent → OUT-OF-THE-BOX built-in default (feature ON by default, #3254 round-3) + assert.deepEqual(resolveFirstContentDeadlines({}, undefined, undefined, undefined), { initialMs: 720000, initialSource: "builtin", gapMs: 720000, gapSource: "builtin" }); + // global tier alone; gap inherits the resolved initial ceiling assert.deepEqual(resolveFirstContentDeadlines({}, "https://any.example", undefined, { firstContentTimeoutMs: 180000 }), { initialMs: 180000, initialSource: "config", gapMs: 180000, gapSource: "config" }); - // global gap-only (valid: initial stays off) - assert.deepEqual(resolveFirstContentDeadlines({}, "https://any.example", undefined, { contentGapTimeoutMs: 300000 }), { gapMs: 300000, gapSource: "config" }); + // global gap-only: initial is ABSENT so it falls to the built-in default; gap takes the explicit value + assert.deepEqual(resolveFirstContentDeadlines({}, "https://any.example", undefined, { contentGapTimeoutMs: 300000 }), { initialMs: 720000, initialSource: "builtin", gapMs: 300000, gapSource: "config" }); + // explicit 0 at the GLOBAL tier disables every phase (opt-out escape hatch overrides the built-in) + assert.equal(resolveFirstContentDeadlines({}, "https://any.example", undefined, { firstContentTimeoutMs: 0 }), undefined); // route beats global assert.deepEqual(resolveFirstContentDeadlines(ROUTES, "https://relay-a.example/v1/messages", undefined, { firstContentTimeoutMs: 180000 }), { initialMs: 600000, initialSource: "config", gapMs: 600000, gapSource: "config" }); // model beats route @@ -283,7 +285,7 @@ test("#3254 G: loadRoutes rejects malformed deadline values at provider and mode // --- I: adaptive deadline math ---------------------------------------------- -test("#3254 I: fcP95 + adaptiveDeadlineMs — cold undefined, clamp band, explicit ceiling", () => { +test("#3254 I: fcP95 + adaptiveDeadlineMs — cold undefined, [floor, ceiling] clamp, ceiling follows resolved value", () => { assert.equal(fcP95([42]), 42); // nearest-rank: at n=8 the rank is ceil(0.95×8)=8 → the max; at n=20 it is 19 → second-highest assert.equal(fcP95([5, 10, 15, 20, 25, 30, 35, 40]), 40); @@ -292,12 +294,16 @@ test("#3254 I: fcP95 + adaptiveDeadlineMs — cold undefined, clamp band, explic assert.equal(adaptiveDeadlineMs(Array.from({ length: FC_ADAPTIVE_MIN_SAMPLES - 1 }, () => 20000), 180000), undefined, "one below MIN_SAMPLES is still cold"); // p95=20000 → 3×=60000 → clamped UP to the 120s floor assert.equal(adaptiveDeadlineMs(Array.from({ length: FC_ADAPTIVE_MIN_SAMPLES }, () => 20000), 180000), 120000); - // p95=200000 → 3×=600000 → the cap + // p95=200000 → 3×=600000 → exactly the 600000 ceiling passed here assert.equal(adaptiveDeadlineMs(Array.from({ length: FC_ADAPTIVE_MIN_SAMPLES }, () => 200000), 600000), 600000); - // mid-band: p95=50000 → 150000, below the explicit value + // mid-band: p95=50000 → 150000, below the ceiling assert.equal(adaptiveDeadlineMs(Array.from({ length: FC_ADAPTIVE_MIN_SAMPLES }, () => 50000), 180000), 150000); - // explicit ceiling: a user-pinned tighter value never gets relaxed + // a user-pinned tighter ceiling is never relaxed upward (floor yields to it) assert.equal(adaptiveDeadlineMs(Array.from({ length: FC_ADAPTIVE_MIN_SAMPLES }, () => 20000), 60000), 60000); + // ceiling FOLLOWS the resolved value (no hardcoded 600s cap): a slow route + // (p95=300s → 3×=900s) under a 720s ceiling keeps its full 720s instead of + // discontinuously dropping to ≤600s once warm (#3254 round-3 point ③) + assert.equal(adaptiveDeadlineMs(Array.from({ length: FC_ADAPTIVE_MIN_SAMPLES }, () => 300000), 720000), 720000); }); // --- J: resolver with adaptive samples -------------------------------------- @@ -306,9 +312,10 @@ test("#3254 J: warm routes adapt (source=adaptive) under their configured ceilin const globals = { firstContentTimeoutMs: 180000 }; const warm = Array.from({ length: FC_ADAPTIVE_MIN_SAMPLES }, () => 20000); // warm initial, no gap samples yet: initial adapts to the floor; the - // INHERITED gap has no gap samples of its own and stays on config + // INHERITED gap has no gap samples of its own and sits at the inherited + // initial CEILING (the configured bound, not the tightened value) assert.deepEqual(resolveFirstContentDeadlines({}, "https://any.example", undefined, globals, { init: warm, gap: [] }), { - initialMs: 120000, initialSource: "adaptive", gapMs: 120000, gapSource: "config", + initialMs: 120000, initialSource: "adaptive", gapMs: 180000, gapSource: "config", }); // both phases warm in-band: p95=50000 → 150000 each const mid = Array.from({ length: FC_ADAPTIVE_MIN_SAMPLES }, () => 50000); diff --git a/website/config-reference/process.yaml b/website/config-reference/process.yaml index c20b5bbd3e..a1e457ecf2 100644 --- a/website/config-reference/process.yaml +++ b/website/config-reference/process.yaml @@ -10,8 +10,8 @@ entries: site_group: proc desc_en: "Transport timeouts, watchdogs and replay policy for upstream calls." desc_zh: "上游调用的传输超时、看门狗与重放策略。" - site_en: "network — transport timeouts & replays: upstreamTimeoutMs 720000 · requestWatchdogMs 2× · replayRetryMax 3 · replayRetryBaseMs 1500 · keepAliveTimeoutMs 5000 · clientErrorBackstopMs 30000 · exposureLogIntervalMs 3600000 · streamKeepAliveMs 15000 · preflightHoldMs 30000 · preflightDeadEndCooldownMs 300000 · proxyKeepAliveMaxMs 55000 · firstContentTimeoutMs unset (#3254, file-only) · contentGapTimeoutMs unset (#3254, file-only)." - site_zh: "network —— 传输超时与重放:upstreamTimeoutMs 720000 · requestWatchdogMs 2× · replayRetryMax 3 · replayRetryBaseMs 1500 · keepAliveTimeoutMs 5000 · clientErrorBackstopMs 30000 · exposureLogIntervalMs 3600000 · streamKeepAliveMs 15000 · preflightHoldMs 30000 · preflightDeadEndCooldownMs 300000 · proxyKeepAliveMaxMs 55000 · firstContentTimeoutMs 未设置(#3254,仅文件)· contentGapTimeoutMs 未设置(#3254,仅文件)。" + site_en: "network — transport timeouts & replays: upstreamTimeoutMs 720000 · requestWatchdogMs 2× · replayRetryMax 3 · replayRetryBaseMs 1500 · keepAliveTimeoutMs 5000 · clientErrorBackstopMs 30000 · exposureLogIntervalMs 3600000 · streamKeepAliveMs 15000 · preflightHoldMs 30000 · preflightDeadEndCooldownMs 300000 · proxyKeepAliveMaxMs 55000 · firstContentTimeoutMs 720000 on-by-default (#3254, file-only) · contentGapTimeoutMs inherits/720000 (#3254, file-only)." + site_zh: "network —— 传输超时与重放:upstreamTimeoutMs 720000 · requestWatchdogMs 2× · replayRetryMax 3 · replayRetryBaseMs 1500 · keepAliveTimeoutMs 5000 · clientErrorBackstopMs 30000 · exposureLogIntervalMs 3600000 · streamKeepAliveMs 15000 · preflightHoldMs 30000 · preflightDeadEndCooldownMs 300000 · proxyKeepAliveMaxMs 55000 · firstContentTimeoutMs 720000 默认开启(#3254,仅文件)· contentGapTimeoutMs 继承/默认 720000(#3254,仅文件)。" - key: network.upstreamTimeoutMs type: number default: "720000" @@ -70,13 +70,13 @@ entries: desc_zh: "死胡同 preflight 后,对同一会话重试校准的冷却时长。" - key: network.firstContentTimeoutMs type: number - default: "unset (off)" + default: "720000 (on by default)" group: process - desc_en: "Model-content deadline, phase 1 (#3254): max ms an upstream attempt may go with zero model content from response start before bili aborts the stalled stream and ends the turn with one clean in-band error (no internal refetch — the client's retry ladder re-arms). Also settable per route / per model under providers; cascade model > route > global, explicit 0 wins at its level. Once a route has ≥8 healthy attempts the configured value becomes a ceiling and the effective deadline tightens to clamp(3 × p95 of the route's recent healthy first-content durations, 120s, 600s); stalled/killed attempts are never sampled. File-only by design." - desc_zh: "模型内容期限,阶段一(#3254):上游一次尝试从响应开始到首个模型内容之间的最长毫秒数;超时则 bili 中止卡死流并以一条干净的带内错误结束回合(不做内部盲重取——客户端重试梯照常接管)。providers 内也可按线路/按模型设置;级联 model > route > global,显式 0 在所在层生效。线路积累 ≥8 次健康尝试后,配置值转为上限,有效期限收紧为 clamp(该线路近期健康首内容耗时的 3 × p95, 120s, 600s);卡死或被中止的尝试从不采样。按设计仅配置文件可设。" + desc_en: "Model-content deadline, phase 1 (#3254): max ms an upstream attempt may go with zero model content from response start before bili aborts the stalled stream and ends the turn with one clean in-band error (no internal refetch — the turn fails cleanly; clients with a retry ladder back off and retry, clients without one simply stop). On by default at 720000ms so every deployment is bounded out of the box; set a value to tighten/override, or 0 to disable (opt-out override of the built-in). Also settable per route / per model under providers; cascade model > route > global, explicit 0 wins at its level. Once a route has ≥8 healthy attempts the resolved value becomes a ceiling and the effective deadline tightens to clamp(3 × p95 of the route's recent healthy first-content durations, 120s, that resolved ceiling); stalled/killed attempts are never sampled. File-only by design." + desc_zh: "模型内容期限,阶段一(#3254):上游一次尝试从响应开始到首个模型内容之间的最长毫秒数;超时则 bili 中止卡死流并以一条干净的带内错误结束回合(不做内部盲重取——回合干净失败:有重试梯的客户端退避后重试,没有的则直接停止)。默认以 720000ms 开启,让每个部署开箱即用即有界;设值可收紧/覆盖,或设 0 关闭(对内置值的显式豁免)。providers 内也可按线路/按模型设置;级联 model > route > global,显式 0 在所在层生效。线路积累 ≥8 次健康尝试后,解析值转为上限,有效期限收紧为 clamp(该线路近期健康首内容耗时的 3 × p95, 120s, 该解析上限);卡死或被中止的尝试从不采样。按设计仅配置文件可设。" - key: network.contentGapTimeoutMs type: number - default: "unset (inherits firstContentTimeoutMs)" + default: "inherits firstContentTimeoutMs (720000 default)" group: process desc_en: "Model-content deadline, phase 2 (#3254): max ms between consecutive model-content events after the first arrived; re-armed on every content event (pings/comments never count). Absent inherits the resolved firstContentTimeoutMs; explicit 0 disables the gap phase only. Same three-level cascade, same route-adaptive ceiling (per-phase samples) and file-only rationale as firstContentTimeoutMs." desc_zh: "模型内容期限,阶段二(#3254):首个内容到达之后,相邻两个模型内容事件之间的最长毫秒数;每个内容事件重新计时(ping/注释不算内容)。缺省继承解析后的 firstContentTimeoutMs;显式 0 仅禁用阶段二。三级级联、线路自适应上限(按阶段各自采样)与仅文件可设的理由同 firstContentTimeoutMs。" diff --git a/website/config-reference/providers.yaml b/website/config-reference/providers.yaml index 66070a899d..279503762a 100644 --- a/website/config-reference/providers.yaml +++ b/website/config-reference/providers.yaml @@ -178,14 +178,14 @@ entries: site_zh: "按路由的 k̂ 校准策略(#3211):`disabled` 在该线路禁用学习系数;`max` 钳制学习系数(0 < max ≤ 4)。作用于发布环节——学习环保持原始值。" - key: firstContentTimeoutMs type: number - default: "unset (off)" + default: "unset (→ global, else 720000 builtin)" group: providers id: cr.fcdeadline render: route-row - desc_en: "Per-route model-content deadline, phase 1 (#3254): max ms without model content from response start before bili aborts the stalled upstream and ends the turn in-band. Also settable per model under models[]; cascade model > route > global network.*, explicit 0 wins at its level. Once the route has ≥8 healthy attempts the configured value becomes a ceiling and the effective deadline tightens to clamp(3 × p95 of this route's recent healthy first-content durations, 120s, 600s); stalled/killed attempts are never sampled. File-only (no env)." - desc_zh: "按线路的模型内容期限,阶段一(#3254):响应开始后无模型内容的上限毫秒数;超时则中止卡死上游并以带内错误结束回合。也可在 models[] 按模型设置;级联 model > route > global network.*,显式 0 在所在层生效。线路积累 ≥8 次健康尝试后,配置值转为上限,有效期限收紧为 clamp(该线路近期健康首内容耗时的 3 × p95, 120s, 600s);卡死或被中止的尝试从不采样。仅配置文件可设(无 env)。" - site_en: "Model-content deadline, phase 1 (#3254), per route and per model (`models[]`); on expiry bili aborts the stalled upstream stream and ends the turn with one clean in-band error — no internal refetch. Cascade model > route > global `network.*`; explicit 0 wins at its level; once warmed (≥8 healthy attempts) the configured value acts as a ceiling tightened toward the route's measured first-content pace (clamp 3×p95, 120s–600s); file-only." - site_zh: "模型内容期限,阶段一(#3254),按线路与按模型(`models[]`);到期则中止卡死的上游流并以一条干净的带内错误结束回合——不做内部盲重取。级联 model > route > global `network.*`;显式 0 在所在层生效;线路预热后(≥8 次健康尝试)配置值转为上限,向该线路实测首内容水位收紧(clamp 3×p95,120s–600s);仅文件可设。" + desc_en: "Per-route model-content deadline, phase 1 (#3254): max ms without model content from response start before bili aborts the stalled upstream and ends the turn in-band. On by default via the built-in (unset falls through to global then the 720000 builtin); set here to override per route, or 0 to disable for this route. Also settable per model under models[]; cascade model > route > global network.*, explicit 0 wins at its level. Once the route has ≥8 healthy attempts the resolved value becomes a ceiling and the effective deadline tightens to clamp(3 × p95 of this route's recent healthy first-content durations, 120s, that resolved ceiling); stalled/killed attempts are never sampled. File-only (no env)." + desc_zh: "按线路的模型内容期限,阶段一(#3254):响应开始后无模型内容的上限毫秒数;超时则中止卡死上游并以带内错误结束回合。通过内置值默认开启(缺省落到 global 再到内置 720000);在此按线路覆盖,或设 0 对该线路关闭。也可在 models[] 按模型设置;级联 model > route > global network.*,显式 0 在所在层生效。线路积累 ≥8 次健康尝试后,解析值转为上限,有效期限收紧为 clamp(该线路近期健康首内容耗时的 3 × p95, 120s, 该解析上限);卡死或被中止的尝试从不采样。仅配置文件可设(无 env)。" + site_en: "Model-content deadline, phase 1 (#3254), per route and per model (`models[]`); on expiry bili aborts the stalled upstream stream and ends the turn with one clean in-band error — no internal refetch. On by default via the built-in (unset → global → 720000 builtin); explicit 0 disables for this route. Cascade model > route > global `network.*`; once warmed (≥8 healthy attempts) the resolved value acts as a ceiling tightened toward the route's measured first-content pace (clamp 3×p95, floor 120s, capped at the resolved value); file-only." + site_zh: "模型内容期限,阶段一(#3254),按线路与按模型(`models[]`);到期则中止卡死的上游流并以一条干净的带内错误结束回合——不做内部盲重取。通过内置值默认开启(缺省 → global → 内置 720000);显式 0 对该线路关闭。级联 model > route > global `network.*`;线路预热后(≥8 次健康尝试)解析值转为上限,向该线路实测首内容水位收紧(clamp 3×p95,下限 120s,上限为解析值);仅文件可设。" - key: contentGapTimeoutMs type: number default: "unset (inherits firstContentTimeoutMs)" From bdcbc056d0836920fb199647c841c63d0c37245d Mon Sep 17 00:00:00 2001 From: awork Date: Fri, 16 Oct 2026 11:39:29 +0800 Subject: [PATCH 7/7] fix: regenerate config docs after merging master (docs-sync gate) Derived docs (CONFIGURATION en/zh, website main.js/index.html) had drifted from the yaml seeds the generator reads: the two follow-up feat commits (route-adaptive refinement, out-of-the-box 720s builtin) updated the yaml wording but the committed derived docs kept stale default-off-era phrasing (e.g. tt.network still said 'firstContentTimeoutMs unset' while the same table row said 'on by default'). Re-ran tools/gen-config-docs.mjs generate; output is byte-stable on a second run. --- CONFIGURATION.md | 9 ++++----- CONFIGURATION.zh-CN.md | 9 ++++----- website/docs/index.html | 7 +++---- website/main.js | 14 ++++++-------- 4 files changed, 17 insertions(+), 22 deletions(-) diff --git a/CONFIGURATION.md b/CONFIGURATION.md index 28d86ac176..112ecc595e 100644 --- a/CONFIGURATION.md +++ b/CONFIGURATION.md @@ -147,14 +147,13 @@ This index is generated from `website/config-reference/*.yaml` — edit the seed | `network.streamKeepAliveMs` | number | 15000 (0 disables) | BILI_STREAM_KEEPALIVE_MS | Downstream stream keep-alive ping interval between model chunks; 0 disables pinging. | | `network.preflightHoldMs` | number | 30000 | BILI_PREFLIGHT_HOLD_MS | How long a failed preflight holds before the request is allowed through unreconciled. | | `network.preflightDeadEndCooldownMs` | number | 300000 | BILI_PREFLIGHT_DEAD_END_COOLDOWN_MS | Cooldown after a dead-end preflight before retrying reconciliation for the same session. | -| `network.firstContentTimeoutMs` | number | `720000` (on by default) | — | Model-content deadline, phase 1 (#3254): max ms an upstream attempt may go with zero model content from response start before bili aborts the stalled stream and ends the turn with one clean in-band error (no internal refetch — the turn fails cleanly; clients with a retry ladder back off and retry, clients without one simply stop). **On by default at 720000ms** so every deployment is bounded out of the box; set a value to tighten/override, or `0` to disable (opt-out override of the built-in). Also settable per route / per model under providers; cascade model > route > global, explicit 0 wins at its level. Once a route has ≥8 healthy attempts, the resolved value becomes a ceiling and the effective deadline tightens to `clamp(3 × p95 of the route's recent healthy first-content durations, 120s, that resolved ceiling)` (never above it). File-only by design. | -| `network.contentGapTimeoutMs` | number | inherits firstContentTimeoutMs (`720000` default) | — | Model-content deadline, phase 2 (#3254): max ms between consecutive model-content events after the first arrived; re-armed on every content event (pings/comments never count). On by default through inheritance (absent inherits the resolved firstContentTimeoutMs ceiling); explicit 0 disables the gap phase only (opt-out). Same three-level cascade, same route-adaptive ceiling (which follows the resolved value) and file-only rationale as firstContentTimeoutMs. | +| `network.firstContentTimeoutMs` | number | 720000 (on by default) | — | Model-content deadline, phase 1 (#3254): max ms an upstream attempt may go with zero model content from response start before bili aborts the stalled stream and ends the turn with one clean in-band error (no internal refetch — the turn fails cleanly; clients with a retry ladder back off and retry, clients without one simply stop). On by default at 720000ms so every deployment is bounded out of the box; set a value to tighten/override, or 0 to disable (opt-out override of the built-in). Also settable per route / per model under providers; cascade model > route > global, explicit 0 wins at its level. Once a route has ≥8 healthy attempts the resolved value becomes a ceiling and the effective deadline tightens to clamp(3 × p95 of the route's recent healthy first-content durations, 120s, that resolved ceiling); stalled/killed attempts are never sampled. File-only by design. | +| `network.contentGapTimeoutMs` | number | inherits firstContentTimeoutMs (720000 default) | — | Model-content deadline, phase 2 (#3254): max ms between consecutive model-content events after the first arrived; re-armed on every content event (pings/comments never count). Absent inherits the resolved firstContentTimeoutMs; explicit 0 disables the gap phase only. Same three-level cascade, same route-adaptive ceiling (per-phase samples) and file-only rationale as firstContentTimeoutMs. | | `network.replayRetryMax` | number | 3 (1 disables replays) | BILI_REPLAY_RETRY_MAX | Maximum transport-failure replays of an identical request before surfacing the error. | | `network.replayRetryBaseMs` | number | 1500 (0 disables the delay) | BILI_REPLAY_RETRY_BASE_MS | Base backoff delay between replays (exponential). | | `network.maxShrinkPerCompress` | number | unset | BILI_MAX_SHRINK_PER_COMPRESS | Cap on how much one compression may shrink history (anti-collapse guard). | | `network.proxyKeepAliveMaxMs` | number | 55000 (0 = one-shot connections) | BILI_PROXY_KEEPALIVE_MAX_MS | Max lifetime of a connection through an outbound egress proxy before forcing reconnection (avoids stale NAT mappings). | | `network.postResponseLingerMs` | number | 5000 | BILI_POST_RESPONSE_LINGER_MS | Graceful-close budget for proxy-initiated closes after the response completed: the socket is held waiting for the peer's FIN/TLS close_notify and destroyed at the deadline (reason=linger-backstop). | - | `network.allowPrivateHttpProviders` | boolean | false | BILI_ALLOW_PRIVATE_HTTP_PROVIDERS | Opt-in (#2984): admit plain-HTTP providers on private networks (RFC1918, 127/8, link-local, ULA, *.local/.lan/.internal) into the agent-registry intake and summary-endpoint guards. Off by default — plaintext stays loopback-only; public IPs/domains and CGNAT (100.64/10) stay refused regardless. 1/true enables. | | `network.receiveBodyBytes` | number | 5368709120 | BILI_RECEIVE_BODY_BYTES | Max inbound wire bytes read into memory per request before a 413 at the receive stage. Raised from a hardcoded 100 MiB to 5 GiB (#3128 owner decision) so multi-gigabyte single requests are accepted instead of the socket being reset. Also the threshold above which a decoded request consumes one of two shared large-decode admission slots. WebSocket lanes keep the fixed 100 MiB surface. | | `network.decodedBodyBytes` | number | 10737418240 | BILI_DECODED_BODY_BYTES | Max DECODED bytes after content-encoding expansion (gzip/deflate/br/zstd), checked incrementally during decode (decode-stage 413). Default is 2x the receive budget, mirroring the legacy decode=2x-wire relationship; guards against decompression bombs. | @@ -311,8 +310,8 @@ This index is generated from `website/config-reference/*.yaml` — edit the seed | `apiKeyEnv` | string (env var name) | unset | — | Lane credential override (#2336): replace the client's credential with this env variable's value for this lane. Exactly one of apiKeyEnv/credentialRef; see [Lane credentials](#lane-credentials-apikeyenv--credentialref). | | `credentialRef` | string (store name) | unset | — | Lane credential override via the private summary-credential store (secret:NAME semantics). | | `calibration` | { disabled?: boolean, max?: number } | unset (learned k̂ applies) | — | Per-route k̂ calibration policy (#3211): opt-out or clamp for routes whose usage reports inflate; see [Route-level k̂ calibration policy](#route-level-k-calibration-policy). | -| `firstContentTimeoutMs` | number | unset (→ global, else built-in `720000`) | — | Per-route model-content deadline, phase 1 (#3254): max ms without model content from response start before bili aborts the stalled upstream and ends the turn in-band. On by default via the built-in (unset falls through to global then the 720000 builtin); set here to override per route, or `0` to disable for this route. Also settable per model under models[]; cascade model > route > global network.*, explicit 0 wins at its level. File-only (no env). | -| `contentGapTimeoutMs` | number | unset (inherits firstContentTimeoutMs) | — | Per-route model-content deadline, phase 2 (#3254): max ms between consecutive model-content events after the first arrived; re-armed on every content event. Absent inherits the resolved firstContentTimeoutMs ceiling (built-in 720000 when nothing else is set); explicit 0 disables the gap phase only. Same cascade and file-only rationale as firstContentTimeoutMs. | +| `firstContentTimeoutMs` | number | unset (→ global, else 720000 builtin) | — | Per-route model-content deadline, phase 1 (#3254): max ms without model content from response start before bili aborts the stalled upstream and ends the turn in-band. On by default via the built-in (unset falls through to global then the 720000 builtin); set here to override per route, or 0 to disable for this route. Also settable per model under models[]; cascade model > route > global network.*, explicit 0 wins at its level. Once the route has ≥8 healthy attempts the resolved value becomes a ceiling and the effective deadline tightens to clamp(3 × p95 of this route's recent healthy first-content durations, 120s, that resolved ceiling); stalled/killed attempts are never sampled. File-only (no env). | +| `contentGapTimeoutMs` | number | unset (inherits firstContentTimeoutMs) | — | Per-route model-content deadline, phase 2 (#3254): max ms between consecutive model-content events after the first arrived; re-armed on every content event. Absent inherits the resolved firstContentTimeoutMs; explicit 0 disables the gap phase only. Same cascade, same route-adaptive ceiling (its own inter-content-gap samples) and file-only rationale as firstContentTimeoutMs. | | `compactionOptIn` | boolean | false | BILI_NON_HTTP_PROVIDERS | Named entries only: opt a non-http(s)-baseUrl provider into compaction ownership (pi/omp lanes); unions with env BILI_NON_HTTP_PROVIDERS. | **Environment-only variables** diff --git a/CONFIGURATION.zh-CN.md b/CONFIGURATION.zh-CN.md index a08dbbaa63..4a8a1a93f0 100644 --- a/CONFIGURATION.zh-CN.md +++ b/CONFIGURATION.zh-CN.md @@ -147,14 +147,13 @@ | `network.streamKeepAliveMs` | number | 15000 (0 disables) | BILI_STREAM_KEEPALIVE_MS | 模型块之间的下游流保活心跳间隔;0 禁用心跳。 | | `network.preflightHoldMs` | number | 30000 | BILI_PREFLIGHT_HOLD_MS | preflight 失败后,允许请求未校准放行的持有时长。 | | `network.preflightDeadEndCooldownMs` | number | 300000 | BILI_PREFLIGHT_DEAD_END_COOLDOWN_MS | 死胡同 preflight 后,对同一会话重试校准的冷却时长。 | -| `network.firstContentTimeoutMs` | number | `720000`(默认开启) | — | 模型内容期限,阶段一(#3254):上游一次尝试从响应开始到首个模型内容之间的最长毫秒数;超时则 bili 中止卡死流并以一条干净的带内错误结束回合(不做内部盲重取——回合干净失败:有重试梯的客户端退避后重试,没有的则直接停止)。**默认以 720000ms 开启**,让每个部署开箱即用即有界;设值可收紧/覆盖,或设 `0` 关闭(对内置值的显式豁免)。providers 内也可按线路/按模型设置;级联 model > route > global,显式 0 在所在层生效。线路积累 ≥8 次健康尝试后,解析值转为上限:有效期限收紧为 `clamp(该线路近期健康首内容耗时的 3 × p95, 120s, 该解析上限)`(不会高于它)。按设计仅配置文件可设。 | -| `network.contentGapTimeoutMs` | number | 继承 firstContentTimeoutMs(默认 `720000`) | — | 模型内容期限,阶段二(#3254):首个内容到达之后,相邻两个模型内容事件之间的最长毫秒数;每个内容事件重新计时(ping/注释不算内容)。通过继承默认开启(缺省继承解析后的 firstContentTimeoutMs 上限);显式 0 仅禁用阶段二(豁免)。三级级联、线路自适应上限(跟随解析值)与仅文件可设的理由同 firstContentTimeoutMs。 | +| `network.firstContentTimeoutMs` | number | 720000 (on by default) | — | 模型内容期限,阶段一(#3254):上游一次尝试从响应开始到首个模型内容之间的最长毫秒数;超时则 bili 中止卡死流并以一条干净的带内错误结束回合(不做内部盲重取——回合干净失败:有重试梯的客户端退避后重试,没有的则直接停止)。默认以 720000ms 开启,让每个部署开箱即用即有界;设值可收紧/覆盖,或设 0 关闭(对内置值的显式豁免)。providers 内也可按线路/按模型设置;级联 model > route > global,显式 0 在所在层生效。线路积累 ≥8 次健康尝试后,解析值转为上限,有效期限收紧为 clamp(该线路近期健康首内容耗时的 3 × p95, 120s, 该解析上限);卡死或被中止的尝试从不采样。按设计仅配置文件可设。 | +| `network.contentGapTimeoutMs` | number | inherits firstContentTimeoutMs (720000 default) | — | 模型内容期限,阶段二(#3254):首个内容到达之后,相邻两个模型内容事件之间的最长毫秒数;每个内容事件重新计时(ping/注释不算内容)。缺省继承解析后的 firstContentTimeoutMs;显式 0 仅禁用阶段二。三级级联、线路自适应上限(按阶段各自采样)与仅文件可设的理由同 firstContentTimeoutMs。 | | `network.replayRetryMax` | number | 3 (1 disables replays) | BILI_REPLAY_RETRY_MAX | 同一请求在向上游暴露错误前的最大传输失败重放次数。 | | `network.replayRetryBaseMs` | number | 1500 (0 disables the delay) | BILI_REPLAY_RETRY_BASE_MS | 重放之间的基础退避延迟(指数增长)。 | | `network.maxShrinkPerCompress` | number | unset | BILI_MAX_SHRINK_PER_COMPRESS | 单次压缩允许收缩历史的上限(防塌缩护栏)。 | | `network.proxyKeepAliveMaxMs` | number | 55000 (0 = one-shot connections) | BILI_PROXY_KEEPALIVE_MAX_MS | 经出站代理的连接最大存活时长,到期强制重建(规避陈旧 NAT 映射)。 | | `network.postResponseLingerMs` | number | 5000 | BILI_POST_RESPONSE_LINGER_MS | 代理主动发起的响应后关闭的优雅关闭预算:socket 挂起等待对端 FIN/TLS close_notify,到点销毁(reason=linger-backstop)。 | - | `network.allowPrivateHttpProviders` | boolean | false | BILI_ALLOW_PRIVATE_HTTP_PROVIDERS | 可选开关(#2984):允许私网(RFC1918、127/8、link-local、ULA、*.local/.lan/.internal)明文 HTTP provider 进入 agent-registry 接收与摘要端点守卫。默认关——明文仅回环受信;公网 IP/域名与 CGNAT(100.64/10)无论如何都拒。1/true 启用。 | | `network.receiveBodyBytes` | number | 5368709120 | BILI_RECEIVE_BODY_BYTES | 单请求读入内存的入站 wire 字节上限,超限在 receive 阶段返回 413。由硬编码 100 MiB 提升到 5 GiB(#3128 owner 决定),使多 GB 单请求被接受而非断连。同时也是解码请求占用两个共享大解码准入槽位的阈值。WebSocket lane 保持固定 100 MiB 表面。 | | `network.decodedBodyBytes` | number | 10737418240 | BILI_DECODED_BODY_BYTES | content-encoding 解压(gzip/deflate/br/zstd)后的解码字节上限,解码过程中增量检查(decode 阶段 413)。默认为接收预算的 2 倍,沿用旧的 decode=2×wire 关系;用于防解压炸弹。 | @@ -311,8 +310,8 @@ | `apiKeyEnv` | string (env var name) | unset | — | 通道凭据覆盖(#2336):用该环境变量的值替换客户端凭据。与 credentialRef 二选一;见[通道凭据](#lane-credentials-apikeyenv--credentialref)。 | | `credentialRef` | string (store name) | unset | — | 经私有摘要凭据存储的通道凭据覆盖(secret:NAME 语义)。 | | `calibration` | { disabled?: boolean, max?: number } | unset (learned k̂ applies) | — | 按路由的 k̂ 校准策略(#3211):对虚报报数线路做退出或钳制;见[线路级 k̂ 校准策略](#route-level-k-calibration-policy)。 | -| `firstContentTimeoutMs` | number | unset(→ global,否则内置 `720000`) | — | 按线路的模型内容期限,阶段一(#3254):响应开始后无模型内容的上限毫秒数;超时则中止卡死上游并以带内错误结束回合。通过内置值默认开启(缺省落到 global 再到内置 720000);在此按线路覆盖,或设 `0` 对该线路关闭。也可在 models[] 按模型设置;级联 model > route > global network.*,显式 0 在所在层生效。仅配置文件可设(无 env)。 | -| `contentGapTimeoutMs` | number | unset (inherits firstContentTimeoutMs) | — | 按线路的模型内容期限,阶段二(#3254):首个内容之后相邻模型内容事件间的上限毫秒数,逐事件重新计时。缺省继承解析后的 firstContentTimeoutMs 上限(其余均未设时为内置 720000);显式 0 仅禁用阶段二。级联与仅文件可设的理由同 firstContentTimeoutMs。 | +| `firstContentTimeoutMs` | number | unset (→ global, else 720000 builtin) | — | 按线路的模型内容期限,阶段一(#3254):响应开始后无模型内容的上限毫秒数;超时则中止卡死上游并以带内错误结束回合。通过内置值默认开启(缺省落到 global 再到内置 720000);在此按线路覆盖,或设 0 对该线路关闭。也可在 models[] 按模型设置;级联 model > route > global network.*,显式 0 在所在层生效。线路积累 ≥8 次健康尝试后,解析值转为上限,有效期限收紧为 clamp(该线路近期健康首内容耗时的 3 × p95, 120s, 该解析上限);卡死或被中止的尝试从不采样。仅配置文件可设(无 env)。 | +| `contentGapTimeoutMs` | number | unset (inherits firstContentTimeoutMs) | — | 按线路的模型内容期限,阶段二(#3254):首个内容之后相邻模型内容事件间的上限毫秒数,逐事件重新计时。缺省继承解析后的 firstContentTimeoutMs;显式 0 仅禁用阶段二。级联、线路自适应上限(用自己的内容间隔样本)与仅文件可设的理由同 firstContentTimeoutMs。 | | `compactionOptIn` | boolean | false | BILI_NON_HTTP_PROVIDERS | 仅命名条目:把非 http(s) baseUrl 供应商纳入压缩所有权(pi/omp 车道);与 env BILI_NON_HTTP_PROVIDERS 取并集。 | **仅环境变量(无配置文件键)** diff --git a/website/docs/index.html b/website/docs/index.html index bc27acf79d..d8f40ae6b9 100644 --- a/website/docs/index.html +++ b/website/docs/index.html @@ -239,8 +239,8 @@

    providers — route table fields

    apiKeyEnvstring (env var name)Lane credential override (#2336): requests forwarded through this lane carry this env variable's value instead of the client's own credential — x-api-key / x-goog-api-key replaced outright, Bearer authorization replaced, injected when absent. Exactly one of apiKeyEnv / credentialRef per entry. credentialRefstring (store name)Lane credential override via the private summary-credential store (`secret:NAME` semantics); exactly one of apiKeyEnv / credentialRef per entry. calibration{ disabled?: boolean, max?: number }Per-route k̂ calibration policy (#3211): `disabled` suppresses the learned factor on this route; `max` clamps it (0 < max ≤ 4). Applied at publication time — the learned ring stays raw. - firstContentTimeoutMsnumberModel-content deadline, phase 1 (#3254), per route and per model (`models[]`); on expiry bili aborts the stalled upstream stream and ends the turn with one clean in-band error — no internal refetch. Cascade model > route > global `network.*`; explicit 0 wins at its level; file-only. - contentGapTimeoutMsnumberModel-content deadline, phase 2 (#3254), per route and per model (`models[]`); bounds silence between content events after the first arrived (pings/comments never count); absent inherits the resolved `firstContentTimeoutMs`. File-only. + firstContentTimeoutMsnumberModel-content deadline, phase 1 (#3254), per route and per model (`models[]`); on expiry bili aborts the stalled upstream stream and ends the turn with one clean in-band error — no internal refetch. On by default via the built-in (unset → global → 720000 builtin); explicit 0 disables for this route. Cascade model > route > global `network.*`; once warmed (≥8 healthy attempts) the resolved value acts as a ceiling tightened toward the route's measured first-content pace (clamp 3×p95, floor 120s, capped at the resolved value); file-only. + contentGapTimeoutMsnumberModel-content deadline, phase 2 (#3254), per route and per model (`models[]`); bounds silence between content events after the first arrived (pings/comments never count); absent inherits the resolved `firstContentTimeoutMs`; same route-adaptive ceiling as `firstContentTimeoutMs`. File-only. compactionOptInbooleanNamed entries only: opt a non-http(s)-baseUrl provider into compaction ownership (pi/omp lanes); unions with env BILI_NON_HTTP_PROVIDERS. @@ -280,8 +280,7 @@

    Other top-level fields

  • mitm.enabled / mitm.domains — HTTPS interception switch and whitelist domains for login clients.
  • mitm.handshakeTimeoutMs — TLS handshake timeout (default 10000).
  • Process blocks
  • -
  • network — transport timeouts & replays: upstreamTimeoutMs 720000 · requestWatchdogMs 2× · replayRetryMax 3 · replayRetryBaseMs 1500 · keepAliveTimeoutMs 5000 · clientErrorBackstopMs 30000 · exposureLogIntervalMs 3600000 · streamKeepAliveMs 15000 · preflightHoldMs 30000 · preflightDeadEndCooldownMs 300000 · proxyKeepAliveMaxMs 55000 · firstContentTimeoutMs unset (#3254, file-only) · contentGapTimeoutMs unset (#3254, file-only).
  • -
  • network — transport timeouts & replays: upstreamTimeoutMs 720000 · requestWatchdogMs 2× · replayRetryMax 3 · replayRetryBaseMs 1500 · keepAliveTimeoutMs 5000 · clientErrorBackstopMs 30000 · exposureLogIntervalMs 3600000 · streamKeepAliveMs 15000 · preflightHoldMs 30000 · preflightDeadEndCooldownMs 300000 · proxyKeepAliveMaxMs 55000.
  • +
  • network — transport timeouts & replays: upstreamTimeoutMs 720000 · requestWatchdogMs 2× · replayRetryMax 3 · replayRetryBaseMs 1500 · keepAliveTimeoutMs 5000 · clientErrorBackstopMs 30000 · exposureLogIntervalMs 3600000 · streamKeepAliveMs 15000 · preflightHoldMs 30000 · preflightDeadEndCooldownMs 300000 · proxyKeepAliveMaxMs 55000 · firstContentTimeoutMs 720000 on-by-default (#3254, file-only) · contentGapTimeoutMs inherits/720000 (#3254, file-only).
  • guardrails — pi/omp plugin tool-output guardrails: bashTimeout 60 · maxOutputBytes 50000 · repetition true.
  • persist — session persistence to disk: enabled true · zstd false · debounceMs 500 · tailTokens 16384 · epermAlertThreshold 5 · epermAlertRepeatMs 0.
  • storage — unified storage sidecar: unified legacy — file tier selects the lane-wide storage mode; env BILI_STORAGE_UNIFIED wins when set.
  • diff --git a/website/main.js b/website/main.js index 9d0b868e24..c464759823 100644 --- a/website/main.js +++ b/website/main.js @@ -109,8 +109,8 @@ "cr.lanekey": "Lane credential override (#2336): requests forwarded through this lane carry this env variable's value instead of the client's own credential — x-api-key / x-goog-api-key replaced outright, Bearer authorization replaced, injected when absent. Exactly one of apiKeyEnv / credentialRef per entry.", "cr.lanecred": "Lane credential override via the private summary-credential store (`secret:NAME` semantics); exactly one of apiKeyEnv / credentialRef per entry.", "cr.calib": "Per-route k̂ calibration policy (#3211): `disabled` suppresses the learned factor on this route; `max` clamps it (0 < max ≤ 4). Applied at publication time — the learned ring stays raw.", - "cr.fcdeadline": "Model-content deadline, phase 1 (#3254), per route and per model (`models[]`); on expiry bili aborts the stalled upstream stream and ends the turn with one clean in-band error — no internal refetch. Cascade model > route > global `network.*`; explicit 0 wins at its level; file-only.", - "cr.gapdeadline": "Model-content deadline, phase 2 (#3254), per route and per model (`models[]`); bounds silence between content events after the first arrived (pings/comments never count); absent inherits the resolved `firstContentTimeoutMs`. File-only.", + "cr.fcdeadline": "Model-content deadline, phase 1 (#3254), per route and per model (`models[]`); on expiry bili aborts the stalled upstream stream and ends the turn with one clean in-band error — no internal refetch. On by default via the built-in (unset → global → 720000 builtin); explicit 0 disables for this route. Cascade model > route > global `network.*`; once warmed (≥8 healthy attempts) the resolved value acts as a ceiling tightened toward the route's measured first-content pace (clamp 3×p95, floor 120s, capped at the resolved value); file-only.", + "cr.gapdeadline": "Model-content deadline, phase 2 (#3254), per route and per model (`models[]`); bounds silence between content events after the first arrived (pings/comments never count); absent inherits the resolved `firstContentTimeoutMs`; same route-adaptive ceiling as `firstContentTimeoutMs`. File-only.", "cr.optin": "Named entries only: opt a non-http(s)-baseUrl provider into compaction ownership (pi/omp lanes); unions with env BILI_NON_HTTP_PROVIDERS.", "tg.core": "Server & core", "tg.up": "Upstream & billing", @@ -124,8 +124,7 @@ "tt.ccx": "codexCompact — intercept | pass (default intercept): handle Codex's compaction requests in bili or forward upstream — explicit forms plus the inline auto-compact transiting as an ordinary turn (prompt-form detection, #3008; overridden compact_prompt transits verbatim).", "ttl.ttl": "ccrRetrievalTtlMs — lifetime of CCR retrieval pointers (default 600000; 0 disables retrieval).", "ttl.tmp": "decompressTmpCap — size cap for temporary decompressed blocks (default 50).", - "tt.network": "network — transport timeouts & replays: upstreamTimeoutMs 720000 · requestWatchdogMs 2× · replayRetryMax 3 · replayRetryBaseMs 1500 · keepAliveTimeoutMs 5000 · clientErrorBackstopMs 30000 · exposureLogIntervalMs 3600000 · streamKeepAliveMs 15000 · preflightHoldMs 30000 · preflightDeadEndCooldownMs 300000 · proxyKeepAliveMaxMs 55000 · firstContentTimeoutMs unset (#3254, file-only) · contentGapTimeoutMs unset (#3254, file-only).", - "tt.network": "network — transport timeouts & replays: upstreamTimeoutMs 720000 · requestWatchdogMs 2× · replayRetryMax 3 · replayRetryBaseMs 1500 · keepAliveTimeoutMs 5000 · clientErrorBackstopMs 30000 · exposureLogIntervalMs 3600000 · streamKeepAliveMs 15000 · preflightHoldMs 30000 · preflightDeadEndCooldownMs 300000 · proxyKeepAliveMaxMs 55000.", + "tt.network": "network — transport timeouts & replays: upstreamTimeoutMs 720000 · requestWatchdogMs 2× · replayRetryMax 3 · replayRetryBaseMs 1500 · keepAliveTimeoutMs 5000 · clientErrorBackstopMs 30000 · exposureLogIntervalMs 3600000 · streamKeepAliveMs 15000 · preflightHoldMs 30000 · preflightDeadEndCooldownMs 300000 · proxyKeepAliveMaxMs 55000 · firstContentTimeoutMs 720000 on-by-default (#3254, file-only) · contentGapTimeoutMs inherits/720000 (#3254, file-only).", "tt.guardrails": "guardrails — pi/omp plugin tool-output guardrails: bashTimeout 60 · maxOutputBytes 50000 · repetition true.", "tt.persist": "persist — session persistence to disk: enabled true · zstd false · debounceMs 500 · tailTokens 16384 · epermAlertThreshold 5 · epermAlertRepeatMs 0.", "tt.storage": "storage — unified storage sidecar: unified legacy — file tier selects the lane-wide storage mode; env BILI_STORAGE_UNIFIED wins when set.", @@ -351,8 +350,8 @@ "cr.lanekey": "通道凭据覆盖(#2336):经该通道转发的请求使用此环境变量的值取代客户端自带凭据——x-api-key / x-goog-api-key 直接替换、Bearer authorization 替换、缺失时注入。每个条目与 credentialRef 二选一。", "cr.lanecred": "经私有摘要凭据存储的通道凭据覆盖(`secret:NAME` 语义);每个条目与 apiKeyEnv 二选一。", "cr.calib": "按路由的 k̂ 校准策略(#3211):`disabled` 在该线路禁用学习系数;`max` 钳制学习系数(0 < max ≤ 4)。作用于发布环节——学习环保持原始值。", - "cr.fcdeadline": "模型内容期限,阶段一(#3254),按线路与按模型(`models[]`);到期则中止卡死的上游流并以一条干净的带内错误结束回合——不做内部盲重取。级联 model > route > global `network.*`;显式 0 在所在层生效;仅文件可设。", - "cr.gapdeadline": "模型内容期限,阶段二(#3254),按线路与按模型(`models[]`);约束首个内容之后相邻内容事件间的静默(ping/注释不算内容);缺省继承解析后的 `firstContentTimeoutMs`。仅文件可设。", + "cr.fcdeadline": "模型内容期限,阶段一(#3254),按线路与按模型(`models[]`);到期则中止卡死的上游流并以一条干净的带内错误结束回合——不做内部盲重取。通过内置值默认开启(缺省 → global → 内置 720000);显式 0 对该线路关闭。级联 model > route > global `network.*`;线路预热后(≥8 次健康尝试)解析值转为上限,向该线路实测首内容水位收紧(clamp 3×p95,下限 120s,上限为解析值);仅文件可设。", + "cr.gapdeadline": "模型内容期限,阶段二(#3254),按线路与按模型(`models[]`);约束首个内容之后相邻内容事件间的静默(ping/注释不算内容);缺省继承解析后的 `firstContentTimeoutMs`;线路自适应上限同 `firstContentTimeoutMs`。仅文件可设。", "cr.optin": "仅命名条目:把非 http(s) baseUrl 供应商纳入压缩所有权(pi/omp 车道);与 env BILI_NON_HTTP_PROVIDERS 取并集。", "tg.core": "服务与核心", "tg.up": "上游与计费", @@ -366,8 +365,7 @@ "tt.ccx": "codexCompact —— intercept | pass(默认 intercept):在 bili 内处理 Codex 压缩请求或转给上游 —— 显式形态 + 以普通请求形态过线的内联自动压缩(prompt 文本检测,#3008;自定义 compact_prompt 按原样转发)。", "ttl.ttl": "ccrRetrievalTtlMs —— CCR 回取指针有效期(默认 600000;0 禁用回取)。", "ttl.tmp": "decompressTmpCap —— 临时展开块的大小上限(默认 50)。", - "tt.network": "network —— 传输超时与重放:upstreamTimeoutMs 720000 · requestWatchdogMs 2× · replayRetryMax 3 · replayRetryBaseMs 1500 · keepAliveTimeoutMs 5000 · clientErrorBackstopMs 30000 · exposureLogIntervalMs 3600000 · streamKeepAliveMs 15000 · preflightHoldMs 30000 · preflightDeadEndCooldownMs 300000 · proxyKeepAliveMaxMs 55000 · firstContentTimeoutMs 未设置(#3254,仅文件)· contentGapTimeoutMs 未设置(#3254,仅文件)。", - "tt.network": "network —— 传输超时与重放:upstreamTimeoutMs 720000 · requestWatchdogMs 2× · replayRetryMax 3 · replayRetryBaseMs 1500 · keepAliveTimeoutMs 5000 · clientErrorBackstopMs 30000 · exposureLogIntervalMs 3600000 · streamKeepAliveMs 15000 · preflightHoldMs 30000 · preflightDeadEndCooldownMs 300000 · proxyKeepAliveMaxMs 55000。", + "tt.network": "network —— 传输超时与重放:upstreamTimeoutMs 720000 · requestWatchdogMs 2× · replayRetryMax 3 · replayRetryBaseMs 1500 · keepAliveTimeoutMs 5000 · clientErrorBackstopMs 30000 · exposureLogIntervalMs 3600000 · streamKeepAliveMs 15000 · preflightHoldMs 30000 · preflightDeadEndCooldownMs 300000 · proxyKeepAliveMaxMs 55000 · firstContentTimeoutMs 720000 默认开启(#3254,仅文件)· contentGapTimeoutMs 继承/默认 720000(#3254,仅文件)。", "tt.guardrails": "guardrails —— pi/omp 插件工具输出守卫:bashTimeout 60 · maxOutputBytes 50000 · repetition true。", "tt.persist": "persist —— 会话落盘持久化:enabled true · zstd false · debounceMs 500 · tailTokens 16384 · epermAlertThreshold 5 · epermAlertRepeatMs 0。", "tt.storage": "storage —— 统一存储边车:unified legacy —— 文件层选定全 lane 存储模式;设置了 BILI_STORAGE_UNIFIED 时环境变量优先。",