diff --git a/CONFIGURATION.md b/CONFIGURATION.md index f398f1d02f..112ecc595e 100644 --- a/CONFIGURATION.md +++ b/CONFIGURATION.md @@ -147,6 +147,8 @@ This index is generated from `website/config-reference/*.yaml` — edit the seed | `network.streamKeepAliveMs` | number | 15000 (0 disables) | BILI_STREAM_KEEPALIVE_MS | Downstream stream keep-alive ping interval between model chunks; 0 disables pinging. | | `network.preflightHoldMs` | number | 30000 | BILI_PREFLIGHT_HOLD_MS | How long a failed preflight holds before the request is allowed through unreconciled. | | `network.preflightDeadEndCooldownMs` | number | 300000 | BILI_PREFLIGHT_DEAD_END_COOLDOWN_MS | Cooldown after a dead-end preflight before retrying reconciliation for the same session. | +| `network.firstContentTimeoutMs` | number | 720000 (on by default) | — | Model-content deadline, phase 1 (#3254): max ms an upstream attempt may go with zero model content from response start before bili aborts the stalled stream and ends the turn with one clean in-band error (no internal refetch — the turn fails cleanly; clients with a retry ladder back off and retry, clients without one simply stop). On by default at 720000ms so every deployment is bounded out of the box; set a value to tighten/override, or 0 to disable (opt-out override of the built-in). Also settable per route / per model under providers; cascade model > route > global, explicit 0 wins at its level. Once a route has ≥8 healthy attempts the resolved value becomes a ceiling and the effective deadline tightens to clamp(3 × p95 of the route's recent healthy first-content durations, 120s, that resolved ceiling); stalled/killed attempts are never sampled. File-only by design. | +| `network.contentGapTimeoutMs` | number | inherits firstContentTimeoutMs (720000 default) | — | Model-content deadline, phase 2 (#3254): max ms between consecutive model-content events after the first arrived; re-armed on every content event (pings/comments never count). Absent inherits the resolved firstContentTimeoutMs; explicit 0 disables the gap phase only. Same three-level cascade, same route-adaptive ceiling (per-phase samples) and file-only rationale as firstContentTimeoutMs. | | `network.replayRetryMax` | number | 3 (1 disables replays) | BILI_REPLAY_RETRY_MAX | Maximum transport-failure replays of an identical request before surfacing the error. | | `network.replayRetryBaseMs` | number | 1500 (0 disables the delay) | BILI_REPLAY_RETRY_BASE_MS | Base backoff delay between replays (exponential). | | `network.maxShrinkPerCompress` | number | unset | BILI_MAX_SHRINK_PER_COMPRESS | Cap on how much one compression may shrink history (anti-collapse guard). | @@ -308,6 +310,8 @@ This index is generated from `website/config-reference/*.yaml` — edit the seed | `apiKeyEnv` | string (env var name) | unset | — | Lane credential override (#2336): replace the client's credential with this env variable's value for this lane. Exactly one of apiKeyEnv/credentialRef; see [Lane credentials](#lane-credentials-apikeyenv--credentialref). | | `credentialRef` | string (store name) | unset | — | Lane credential override via the private summary-credential store (secret:NAME semantics). | | `calibration` | { disabled?: boolean, max?: number } | unset (learned k̂ applies) | — | Per-route k̂ calibration policy (#3211): opt-out or clamp for routes whose usage reports inflate; see [Route-level k̂ calibration policy](#route-level-k-calibration-policy). | +| `firstContentTimeoutMs` | number | unset (→ global, else 720000 builtin) | — | Per-route model-content deadline, phase 1 (#3254): max ms without model content from response start before bili aborts the stalled upstream and ends the turn in-band. On by default via the built-in (unset falls through to global then the 720000 builtin); set here to override per route, or 0 to disable for this route. Also settable per model under models[]; cascade model > route > global network.*, explicit 0 wins at its level. Once the route has ≥8 healthy attempts the resolved value becomes a ceiling and the effective deadline tightens to clamp(3 × p95 of this route's recent healthy first-content durations, 120s, that resolved ceiling); stalled/killed attempts are never sampled. File-only (no env). | +| `contentGapTimeoutMs` | number | unset (inherits firstContentTimeoutMs) | — | Per-route model-content deadline, phase 2 (#3254): max ms between consecutive model-content events after the first arrived; re-armed on every content event. Absent inherits the resolved firstContentTimeoutMs; explicit 0 disables the gap phase only. Same cascade, same route-adaptive ceiling (its own inter-content-gap samples) and file-only rationale as firstContentTimeoutMs. | | `compactionOptIn` | boolean | false | BILI_NON_HTTP_PROVIDERS | Named entries only: opt a non-http(s)-baseUrl provider into compaction ownership (pi/omp lanes); unions with env BILI_NON_HTTP_PROVIDERS. | **Environment-only variables** @@ -787,6 +791,8 @@ Since #2030 every pure-behavior knob has a config-file key alongside its env var "preflightDeadEndCooldownMs": 300000, // cooldown after a dead-end preflight verdict "proxyKeepAliveMaxMs": 55000, // upstream-proxy connection reuse cap; 0 uncapped "postResponseLingerMs": 5000, // post-response close linger budget (#1982) + "firstContentTimeoutMs": 180000, // #3254: max ms without model content from response start; ON by default at 720000 (shown tightened here); explicit 0 disables; per-route/model overrides in providers + "contentGapTimeoutMs": 600000, // #3254: max ms between content events after the first; absent = inherits firstContentTimeoutMs (its 720000 default when unset); explicit 0 disables this phase only "allowPrivateHttpProviders": false // opt-in: private-network plain-HTTP providers (#2984) }, @@ -845,10 +851,10 @@ Since #2030 every pure-behavior knob has a config-file key alongside its env var ### `network` -- **Type:** `{ upstreamTimeoutMs?: number; requestWatchdogMs?: number; replayRetryMax?: number; replayRetryBaseMs?: number; maxShrinkPerCompress?: number; keepAliveTimeoutMs?: number; clientErrorBackstopMs?: number; exposureLogIntervalMs?: number; streamKeepAliveMs?: number; preflightHoldMs?: number; preflightDeadEndCooldownMs?: number; proxyKeepAliveMaxMs?: number; postResponseLingerMs?: number; allowPrivateHttpProviders?: boolean }` -- **Default:** `{}` (built-ins: `720000`, `2× upstream timeout`, `3`, `1500`, unset, `5000`, `30000`, `3600000`, `15000`, `30000`, `300000`, `55000`, `5000`, `false`) +- **Type:** `{ upstreamTimeoutMs?: number; requestWatchdogMs?: number; replayRetryMax?: number; replayRetryBaseMs?: number; maxShrinkPerCompress?: number; keepAliveTimeoutMs?: number; clientErrorBackstopMs?: number; exposureLogIntervalMs?: number; streamKeepAliveMs?: number; preflightHoldMs?: number; preflightDeadEndCooldownMs?: number; proxyKeepAliveMaxMs?: number; postResponseLingerMs?: number; firstContentTimeoutMs?: number; contentGapTimeoutMs?: number; allowPrivateHttpProviders?: boolean }` +- **Default:** `{}` (built-ins: `720000`, `2× upstream timeout`, `3`, `1500`, unset, `5000`, `30000`, `3600000`, `15000`, `30000`, `300000`, `55000`, `5000`, `720000`, `720000`, `false`) - **Status:** ACTIVE -- **Description:** Process-wide transport and retry timing (ms). Each key resolves env > file > default via its twin (`BILI_UPSTREAM_TIMEOUT_MS`, `BILI_REQUEST_WATCHDOG_MS`, `BILI_REPLAY_RETRY_MAX`, `BILI_REPLAY_RETRY_BASE_MS`, `BILI_MAX_SHRINK_PER_COMPRESS`, `BILI_KEEP_ALIVE_TIMEOUT_MS`, `BILI_CLIENT_ERROR_BACKSTOP_MS`, `BILI_EXPOSURE_LOG_INTERVAL_MS`, `BILI_STREAM_KEEPALIVE_MS`, `BILI_PREFLIGHT_HOLD_MS`, `BILI_PREFLIGHT_DEAD_END_COOLDOWN_MS`, `BILI_PROXY_KEEPALIVE_MAX_MS`, `BILI_POST_RESPONSE_LINGER_MS`, `BILI_ALLOW_PRIVATE_HTTP_PROVIDERS`). Semantics: `upstreamTimeoutMs` bounds time-to-first-byte and inter-chunk silence end-to-end (#551); `requestWatchdogMs` is the total-request budget (a negative value opts out); `replayRetryMax` counts attempts after transient upstream rejections (#189/#1688); `maxShrinkPerCompress` caps how much one compress may shrink the request before bili steers toward smaller ranges (#189); `streamKeepAliveMs` emits an SSE comment when the response has written zero bytes for that long (#1647); `proxyKeepAliveMaxMs` caps connection reuse through an upstream proxy below its recycle cadence (#1263), `0` uncaps; `postResponseLingerMs` is the graceful-close budget after a proxy-initiated post-response close — the socket is held waiting for the peer's FIN/TLS close_notify and destroyed at the deadline (`reason=linger-backstop`) (#1982); `allowPrivateHttpProviders` is an opt-in (#2984): when true, plain-HTTP provider baseUrls on private networks (RFC1918, 127/8, link-local, IPv6 ULA/link-local, `*.local`/`.lan`/`.internal`) are admitted by the agent-registry intake and summary-endpoint guards that otherwise trust loopback plaintext only — public IPs/domains and CGNAT (100.64/10) stay refused regardless, and the default `false` keeps pre-#2984 behavior byte-for-byte. Full details in the [env table](#config-file-keys-for-environment-knobs-2030). +- **Description:** Process-wide transport and retry timing (ms). Every key except `firstContentTimeoutMs`/`contentGapTimeoutMs` (#3254, file-only — see below) resolves env > file > default via its twin (`BILI_UPSTREAM_TIMEOUT_MS`, `BILI_REQUEST_WATCHDOG_MS`, `BILI_REPLAY_RETRY_MAX`, `BILI_REPLAY_RETRY_BASE_MS`, `BILI_MAX_SHRINK_PER_COMPRESS`, `BILI_KEEP_ALIVE_TIMEOUT_MS`, `BILI_CLIENT_ERROR_BACKSTOP_MS`, `BILI_EXPOSURE_LOG_INTERVAL_MS`, `BILI_STREAM_KEEPALIVE_MS`, `BILI_PREFLIGHT_HOLD_MS`, `BILI_PREFLIGHT_DEAD_END_COOLDOWN_MS`, `BILI_PROXY_KEEPALIVE_MAX_MS`, `BILI_POST_RESPONSE_LINGER_MS`, `BILI_ALLOW_PRIVATE_HTTP_PROVIDERS`). Semantics: `upstreamTimeoutMs` bounds time-to-first-byte and inter-chunk silence end-to-end (#551); `requestWatchdogMs` is the total-request budget (a negative value opts out); `replayRetryMax` counts attempts after transient upstream rejections (#189/#1688); `maxShrinkPerCompress` caps how much one compress may shrink the request before bili steers toward smaller ranges (#189); `streamKeepAliveMs` emits an SSE comment when the response has written zero bytes for that long (#1647); `proxyKeepAliveMaxMs` caps connection reuse through an upstream proxy below its recycle cadence (#1263), `0` uncaps; `postResponseLingerMs` is the graceful-close budget after a proxy-initiated post-response close — the socket is held waiting for the peer's FIN/TLS close_notify and destroyed at the deadline (`reason=linger-backstop`) (#1982); `allowPrivateHttpProviders` is an opt-in (#2984): when true, plain-HTTP provider baseUrls on private networks (RFC1918, 127/8, link-local, IPv6 ULA/link-local, `*.local`/`.lan`/`.internal`) are admitted by the agent-registry intake and summary-endpoint guards that otherwise trust loopback plaintext only — public IPs/domains and CGNAT (100.64/10) stay refused regardless, and the default `false` keeps pre-#2984 behavior byte-for-byte; `firstContentTimeoutMs`/`contentGapTimeoutMs` are FILE-ONLY model-content deadlines (#3254): phase 1 bounds how long an upstream attempt may go with zero model content (text/reasoning deltas, tool calls, signatures) from response start, phase 2 (`contentGapTimeoutMs`, re-armed on every content event — pings/comments never count) bounds silence between content events after the first arrived; on expiry bili aborts the stalled upstream stream and ends the turn with one clean in-band error (context intact, resend your last message) — no internal refetch, so the turn fails cleanly: clients with a retry ladder back off and retry, clients without one simply stop; on by default at the `720000` built-in (explicit `0` at any level opts out); cascade model > provider route > global `network.*`, explicit 0 wins at its level, gap inherits the resolved initial ceiling when absent; deliberately no env layer because per-route/per-model values cannot be expressed as one process-wide variable; **route-adaptive refinement** (#3254 follow-up): once a route has accumulated ≥8 healthy attempts (completed with no truncation/error — stalled or killed attempts are never sampled), each resolved phase value becomes a ceiling and the effective deadline tightens to `clamp(3 × p95 of that route's recent healthy first-content durations, 120s, that resolved ceiling)` per phase (gap uses its own inter-content-gap samples); samples live in session metadata (persisted across restarts, ring of 32) and the fire log records `source=config|adaptive` so a fired deadline can be traced to which value class produced it. Full details in the [env table](#config-file-keys-for-environment-knobs-2030). ### `persist` diff --git a/CONFIGURATION.zh-CN.md b/CONFIGURATION.zh-CN.md index 316a208b36..4a8a1a93f0 100644 --- a/CONFIGURATION.zh-CN.md +++ b/CONFIGURATION.zh-CN.md @@ -147,6 +147,8 @@ | `network.streamKeepAliveMs` | number | 15000 (0 disables) | BILI_STREAM_KEEPALIVE_MS | 模型块之间的下游流保活心跳间隔;0 禁用心跳。 | | `network.preflightHoldMs` | number | 30000 | BILI_PREFLIGHT_HOLD_MS | preflight 失败后,允许请求未校准放行的持有时长。 | | `network.preflightDeadEndCooldownMs` | number | 300000 | BILI_PREFLIGHT_DEAD_END_COOLDOWN_MS | 死胡同 preflight 后,对同一会话重试校准的冷却时长。 | +| `network.firstContentTimeoutMs` | number | 720000 (on by default) | — | 模型内容期限,阶段一(#3254):上游一次尝试从响应开始到首个模型内容之间的最长毫秒数;超时则 bili 中止卡死流并以一条干净的带内错误结束回合(不做内部盲重取——回合干净失败:有重试梯的客户端退避后重试,没有的则直接停止)。默认以 720000ms 开启,让每个部署开箱即用即有界;设值可收紧/覆盖,或设 0 关闭(对内置值的显式豁免)。providers 内也可按线路/按模型设置;级联 model > route > global,显式 0 在所在层生效。线路积累 ≥8 次健康尝试后,解析值转为上限,有效期限收紧为 clamp(该线路近期健康首内容耗时的 3 × p95, 120s, 该解析上限);卡死或被中止的尝试从不采样。按设计仅配置文件可设。 | +| `network.contentGapTimeoutMs` | number | inherits firstContentTimeoutMs (720000 default) | — | 模型内容期限,阶段二(#3254):首个内容到达之后,相邻两个模型内容事件之间的最长毫秒数;每个内容事件重新计时(ping/注释不算内容)。缺省继承解析后的 firstContentTimeoutMs;显式 0 仅禁用阶段二。三级级联、线路自适应上限(按阶段各自采样)与仅文件可设的理由同 firstContentTimeoutMs。 | | `network.replayRetryMax` | number | 3 (1 disables replays) | BILI_REPLAY_RETRY_MAX | 同一请求在向上游暴露错误前的最大传输失败重放次数。 | | `network.replayRetryBaseMs` | number | 1500 (0 disables the delay) | BILI_REPLAY_RETRY_BASE_MS | 重放之间的基础退避延迟(指数增长)。 | | `network.maxShrinkPerCompress` | number | unset | BILI_MAX_SHRINK_PER_COMPRESS | 单次压缩允许收缩历史的上限(防塌缩护栏)。 | @@ -308,6 +310,8 @@ | `apiKeyEnv` | string (env var name) | unset | — | 通道凭据覆盖(#2336):用该环境变量的值替换客户端凭据。与 credentialRef 二选一;见[通道凭据](#lane-credentials-apikeyenv--credentialref)。 | | `credentialRef` | string (store name) | unset | — | 经私有摘要凭据存储的通道凭据覆盖(secret:NAME 语义)。 | | `calibration` | { disabled?: boolean, max?: number } | unset (learned k̂ applies) | — | 按路由的 k̂ 校准策略(#3211):对虚报报数线路做退出或钳制;见[线路级 k̂ 校准策略](#route-level-k-calibration-policy)。 | +| `firstContentTimeoutMs` | number | unset (→ global, else 720000 builtin) | — | 按线路的模型内容期限,阶段一(#3254):响应开始后无模型内容的上限毫秒数;超时则中止卡死上游并以带内错误结束回合。通过内置值默认开启(缺省落到 global 再到内置 720000);在此按线路覆盖,或设 0 对该线路关闭。也可在 models[] 按模型设置;级联 model > route > global network.*,显式 0 在所在层生效。线路积累 ≥8 次健康尝试后,解析值转为上限,有效期限收紧为 clamp(该线路近期健康首内容耗时的 3 × p95, 120s, 该解析上限);卡死或被中止的尝试从不采样。仅配置文件可设(无 env)。 | +| `contentGapTimeoutMs` | number | unset (inherits firstContentTimeoutMs) | — | 按线路的模型内容期限,阶段二(#3254):首个内容之后相邻模型内容事件间的上限毫秒数,逐事件重新计时。缺省继承解析后的 firstContentTimeoutMs;显式 0 仅禁用阶段二。级联、线路自适应上限(用自己的内容间隔样本)与仅文件可设的理由同 firstContentTimeoutMs。 | | `compactionOptIn` | boolean | false | BILI_NON_HTTP_PROVIDERS | 仅命名条目:把非 http(s) baseUrl 供应商纳入压缩所有权(pi/omp 车道);与 env BILI_NON_HTTP_PROVIDERS 取并集。 | **仅环境变量(无配置文件键)** @@ -787,6 +791,8 @@ "preflightDeadEndCooldownMs": 300000, // preflight 死路判定后的冷却 "proxyKeepAliveMaxMs": 55000, // 经上游代理的连接复用上限;0 不限 "postResponseLingerMs": 5000, // 响应后关闭的优雅关闭预算(#1982) + "firstContentTimeoutMs": 180000, // #3254:响应开始后无模型内容的上限毫秒数;默认以 720000 开启(此处示例收紧);显式 0 关闭;providers 内可按线路/模型覆盖 + "contentGapTimeoutMs": 600000, // #3254:首个内容之后相邻内容事件间的上限毫秒数;缺省继承 firstContentTimeoutMs(未设时其默认 720000);显式 0 仅禁用本阶段 "allowPrivateHttpProviders": false // 私网明文 HTTP provider 可选放行(#2984) }, @@ -845,10 +851,10 @@ ### `network` -- **类型:** `{ upstreamTimeoutMs?: number; requestWatchdogMs?: number; replayRetryMax?: number; replayRetryBaseMs?: number; maxShrinkPerCompress?: number; keepAliveTimeoutMs?: number; clientErrorBackstopMs?: number; exposureLogIntervalMs?: number; streamKeepAliveMs?: number; preflightHoldMs?: number; preflightDeadEndCooldownMs?: number; proxyKeepAliveMaxMs?: number; postResponseLingerMs?: number; allowPrivateHttpProviders?: boolean }` -- **默认:** `{}`(内置值依次为:`720000`、`2× 上游超时`、`3`、`1500`、未设置、`5000`、`30000`、`3600000`、`15000`、`30000`、`300000`、`55000`、`5000`、`false`) +- **类型:** `{ upstreamTimeoutMs?: number; requestWatchdogMs?: number; replayRetryMax?: number; replayRetryBaseMs?: number; maxShrinkPerCompress?: number; keepAliveTimeoutMs?: number; clientErrorBackstopMs?: number; exposureLogIntervalMs?: number; streamKeepAliveMs?: number; preflightHoldMs?: number; preflightDeadEndCooldownMs?: number; proxyKeepAliveMaxMs?: number; postResponseLingerMs?: number; firstContentTimeoutMs?: number; contentGapTimeoutMs?: number; allowPrivateHttpProviders?: boolean }` +- **默认:** `{}`(内置值依次为:`720000`、`2× 上游超时`、`3`、`1500`、未设置、`5000`、`30000`、`3600000`、`15000`、`30000`、`300000`、`55000`、`5000`、`720000`、`720000`、`false`) - **状态:** ACTIVE -- **说明:** 进程级传输与重试时序(毫秒)。每个键按 env > file > default 解析,对应环境变量分别为 `BILI_UPSTREAM_TIMEOUT_MS`、`BILI_REQUEST_WATCHDOG_MS`、`BILI_REPLAY_RETRY_MAX`、`BILI_REPLAY_RETRY_BASE_MS`、`BILI_MAX_SHRINK_PER_COMPRESS`、`BILI_KEEP_ALIVE_TIMEOUT_MS`、`BILI_CLIENT_ERROR_BACKSTOP_MS`、`BILI_EXPOSURE_LOG_INTERVAL_MS`、`BILI_STREAM_KEEPALIVE_MS`、`BILI_PREFLIGHT_HOLD_MS`、`BILI_PREFLIGHT_DEAD_END_COOLDOWN_MS`、`BILI_PROXY_KEEPALIVE_MAX_MS`、`BILI_POST_RESPONSE_LINGER_MS`、`BILI_ALLOW_PRIVATE_HTTP_PROVIDERS`。要点:`upstreamTimeoutMs` 端到端约束首字节时间与 chunk 间静默(#551);`requestWatchdogMs` 是单请求总预算(负值 = 退出该约束);`replayRetryMax` 统计瞬态上游拒绝后的尝试次数(#189/#1688);`maxShrinkPerCompress` 限制单次压缩对请求的最大缩减比例,超限则引导模型选择更小的范围(#189);`streamKeepAliveMs` 在响应零字节写出超过该时长时发一条 SSE 注释行(#1647);`proxyKeepAliveMaxMs` 把经上游代理的连接复用窗口压在上游回收周期之下(#1263),`0` 取消上限;`postResponseLingerMs` 是代理主动发起的响应后关闭的优雅关闭预算 —— socket 被挂起等待对端 FIN/TLS close_notify,到点销毁(`reason=linger-backstop`)(#1982);`allowPrivateHttpProviders` 是可选开关(#2984):为 true 时,私网(RFC1918、127/8、link-local、IPv6 ULA/link-local、`*.local`/`.lan`/`.internal`)上的明文 HTTP provider baseUrl 被 agent-registry 接收与摘要端点守卫放行——这些守卫默认只信任回环明文;公网 IP/域名与 CGNAT(100.64/10)无论如何都拒;默认 false 保持 #2984 之前的行为逐字不变。完整细节见[环境变量表](#环境变量的配置键对照-2030)。 +- **说明:** 进程级传输与重试时序(毫秒)。除 `firstContentTimeoutMs`/`contentGapTimeoutMs`(#3254,仅文件可设——见下)外,每个键按 env > file > default 解析,对应环境变量分别为 `BILI_UPSTREAM_TIMEOUT_MS`、`BILI_REQUEST_WATCHDOG_MS`、`BILI_REPLAY_RETRY_MAX`、`BILI_REPLAY_RETRY_BASE_MS`、`BILI_MAX_SHRINK_PER_COMPRESS`、`BILI_KEEP_ALIVE_TIMEOUT_MS`、`BILI_CLIENT_ERROR_BACKSTOP_MS`、`BILI_EXPOSURE_LOG_INTERVAL_MS`、`BILI_STREAM_KEEPALIVE_MS`、`BILI_PREFLIGHT_HOLD_MS`、`BILI_PREFLIGHT_DEAD_END_COOLDOWN_MS`、`BILI_PROXY_KEEPALIVE_MAX_MS`、`BILI_POST_RESPONSE_LINGER_MS`、`BILI_ALLOW_PRIVATE_HTTP_PROVIDERS`。要点:`upstreamTimeoutMs` 端到端约束首字节时间与 chunk 间静默(#551);`requestWatchdogMs` 是单请求总预算(负值 = 退出该约束);`replayRetryMax` 统计瞬态上游拒绝后的尝试次数(#189/#1688);`maxShrinkPerCompress` 限制单次压缩对请求的最大缩减比例,超限则引导模型选择更小的范围(#189);`streamKeepAliveMs` 在响应零字节写出超过该时长时发一条 SSE 注释行(#1647);`proxyKeepAliveMaxMs` 把经上游代理的连接复用窗口压在上游回收周期之下(#1263),`0` 取消上限;`postResponseLingerMs` 是代理主动发起的响应后关闭的优雅关闭预算 —— socket 被挂起等待对端 FIN/TLS close_notify,到点销毁(`reason=linger-backstop`)(#1982);`allowPrivateHttpProviders` 是可选开关(#2984):为 true 时,私网(RFC1918、127/8、link-local、IPv6 ULA/link-local、`*.local`/`.lan`/`.internal`)上的明文 HTTP provider baseUrl 被 agent-registry 接收与摘要端点守卫放行——这些守卫默认只信任回环明文;公网 IP/域名与 CGNAT(100.64/10)无论如何都拒;默认 false 保持 #2984 之前的行为逐字不变;`firstContentTimeoutMs`/`contentGapTimeoutMs` 是仅文件可设的模型内容期限(#3254):阶段一约束上游一次尝试从响应开始到首个模型内容(文本/推理增量、工具调用、签名)的最长等待,阶段二(`contentGapTimeoutMs`,每个内容事件重新计时——ping/注释不算内容)约束首个内容之后相邻内容事件间的静默;到期则 bili 中止卡死的上游流并以一条干净的带内错误结束回合(上下文完好,重发最后一条消息即可继续)——不做内部盲重取,回合干净失败:有重试梯的客户端退避后重试,没有的则直接停止;默认以 `720000` 内置值开启(任意层显式 `0` 即豁免);级联 model > provider route > global `network.*`,显式 0 在所在层生效,gap 缺省继承解析后的 initial 上限;刻意不设 env 层,因为按线路/按模型的取值无法用单一进程级变量表达;**线路自适应收紧**(#3254 follow-up):某线路积累 ≥8 次健康尝试(正常完成、无截断/错误——卡死或被期限中止的尝试从不采样)后,每个解析阶段值转为上限,有效期限按阶段收紧为 `clamp(该线路近期健康首内容耗时的 3 × p95, 120s, 该解析上限)`(gap 阶段用自己的内容间隔样本);样本存于会话 metadata(跨重启持久化,环形窗口 32),触发日志记录 `source=config|adaptive`,可追溯触发值来自哪一类。完整细节见[环境变量表](#环境变量的配置键对照-2030)。 ### `persist` diff --git a/src/config.ts b/src/config.ts index 4b3980e165..3f5a6de42c 100644 --- a/src/config.ts +++ b/src/config.ts @@ -133,8 +133,27 @@ export type ProviderRoute = { * - `max: ` — clamp the applied factor to ≤ max (e.g. `1` = * allow deflation learning only, never inflation). Must be > 0 and ≤ 4 * (higher is inert anyway — the publication clamp already caps at 4). - * Invalid values reject the config load loudly (strict #1909 discipline). */ + * Invalid values reject the config load loudly (strict #1909 discipline). */ calibration?: { disabled?: boolean; max?: number }; + /** #3254 level-2 (global → provider → model cascade): per-route in-request + * model-content deadline. When armed, EVERY upstream attempt of the + * compression loop must see MODEL CONTENT (text/reasoning delta, tool + * call, thinking-family block) within `firstContentTimeoutMs` of the + * response headers; relay pings / SSE keep-alive comments never count — + * they are exactly what hid the 775s stall. On fire the loop aborts the + * live upstream stream and exits with an in-band protocol-native error + * (context intact; the host/client retry ladder takes over) instead of + * letting a stalled-but-alive stream ride the client's own content + * watchdog. Absent everywhere = feature off (byte-identical legacy); + * explicit 0 disables the phase AT THIS LEVEL (falls back to nothing, + * not to the lower level). Invalid values reject the config load loudly + * (strict #1909 discipline). See contentGapTimeoutMs for the mid-stream + * phase. */ + firstContentTimeoutMs?: number; + /** #3254 level-2: inter-content deadline — re-armed on every model content + * event after the first (mid-stream stalls included). Unset inherits the + * resolved firstContentTimeoutMs value. Same levels/semantics. */ + contentGapTimeoutMs?: number; }; export type ProviderRoutes = Record; // key = upstream URL prefix (the /bili/ string) @@ -153,6 +172,10 @@ export type ModelEntry = { * (CODEARTS_BENEFIT_FALLBACK mirror). Env BILI_RESIGN_BENEFIT wins over * this like every other knob. */ benefit?: boolean; + /** #3254 level-3 (deepest): per-model overrides of the provider route's + * model-content deadlines. Same semantics as the route-level fields. */ + firstContentTimeoutMs?: number; + contentGapTimeoutMs?: number; }; /** User-facing compression tuning. Configurable at three levels — global @@ -1248,6 +1271,11 @@ export function loadOptions(env: NodeJS.ProcessEnv = process.env): ProxyOptions const host = rawHost === "localhost" ? "127.0.0.1" : rawHost; const upstream = (env.ACP_UPSTREAM ?? fileConfig.upstream ?? "https://api.anthropic.com").replace(/\/$/, ""); const routes = loadRoutes(env); + // #3254: the global network.* deadline tier is not covered by route + // parsing — validate it at startup too (a broken safety bound must not + // boot silently disabled). + parseDeadlineMs("network.firstContentTimeoutMs", fileConfig.network?.firstContentTimeoutMs); + parseDeadlineMs("network.contentGapTimeoutMs", fileConfig.network?.contentGapTimeoutMs); const namedProviders = loadNamedProviders(env); warnCcrPluginDivergences(routes, fileConfig.compress); const passthrough = passthroughState(env); @@ -1757,6 +1785,22 @@ type FileConfig = { proxyKeepAliveMaxMs?: number; /** Post-response close linger budget (#1982; was env-only BILI_POST_RESPONSE_LINGER_MS; default 5000). */ postResponseLingerMs?: number; + /** #3254 global in-request model-content deadline (level 1 of the + * global → provider → model cascade): no MODEL CONTENT (text/reasoning + * delta, tool call, thinking-family block) within this many ms of the + * response headers → the compression loop aborts the live upstream + * stream and exits with an in-band protocol-native error (context + * intact; the client's retry ladder takes over) instead of letting a + * stalled-but-alive stream ride the client's content watchdog. Relay + * pings / SSE keep-alive comments never count. Deliberately FILE-ONLY + * (no env layer): the per-route cascade needs a file shape env cannot + * express. 0 = explicitly disabled; default unset = feature off + * (byte-identical legacy). Invalid values reject loudly (strict + * #1909), unlike the lenient numeric siblings above. */ + firstContentTimeoutMs?: number; + /** #3254 inter-content phase: re-armed on every model content event; + * unset inherits firstContentTimeoutMs (see that field for levels). */ + contentGapTimeoutMs?: number; /** Opt-in plain-HTTP trust for private networks (#2984): when true, private hosts * (RFC1918, 127/8, link-local, ULA, *.local/.lan/.internal) may be admitted over * plaintext by the #2585 gates — agent-registry intake and the summary-endpoint @@ -2263,7 +2307,7 @@ export function parseRouteEntry(v: unknown): ProviderRoute | undefined { // is the KEY in the providers map (identical to the /bili/ string), // so it is NOT repeated inside the value. if (v && typeof v === "object" && !Array.isArray(v)) { - const obj = v as { models?: Record; proxy?: string; compressProtocol?: string; protocol?: unknown; compress?: CompressSettings; compat?: { roles?: unknown; dropFields?: unknown }; passthrough?: boolean; direct?: boolean; imageBilling?: unknown; imageTokenCap?: unknown; calibration?: unknown; baseUrl?: unknown; api?: unknown; apiKeyEnv?: unknown; credentialRef?: unknown }; + const obj = v as { models?: Record; proxy?: string; compressProtocol?: string; protocol?: unknown; compress?: CompressSettings; compat?: { roles?: unknown; dropFields?: unknown }; passthrough?: boolean; direct?: boolean; imageBilling?: unknown; imageTokenCap?: unknown; calibration?: unknown; firstContentTimeoutMs?: unknown; contentGapTimeoutMs?: unknown; baseUrl?: unknown; api?: unknown; apiKeyEnv?: unknown; credentialRef?: unknown }; const route: ProviderRoute = { models: obj.models }; if (typeof obj.proxy === "string") route.proxy = obj.proxy; if (obj.compressProtocol === "marker" || obj.compressProtocol === "tools") route.compressProtocol = obj.compressProtocol; @@ -2284,6 +2328,17 @@ export function parseRouteEntry(v: unknown): ProviderRoute | undefined { if (imageTokenCap !== undefined) route.imageTokenCap = imageTokenCap; const calibration = parseCalibrationPolicy(obj.calibration); if (calibration) route.calibration = calibration; + // #3254: model-content deadlines — strict like calibration (a silently + // dropped safety bound would leave the stall invisible again). Model + // entries used to pass through raw; validate their copies too. + route.firstContentTimeoutMs = parseDeadlineMs("providers entry firstContentTimeoutMs", obj.firstContentTimeoutMs); + route.contentGapTimeoutMs = parseDeadlineMs("providers entry contentGapTimeoutMs", obj.contentGapTimeoutMs); + for (const [modelName, entry] of Object.entries(obj.models ?? {})) { + if (!entry || typeof entry !== "object") continue; + const me = entry as Record; + parseDeadlineMs(`providers entry models.${modelName}.firstContentTimeoutMs`, me.firstContentTimeoutMs); + parseDeadlineMs(`providers entry models.${modelName}.contentGapTimeoutMs`, me.contentGapTimeoutMs); + } // #2336 lane credential override. Only parsed for NON-recipe-shaped // entries: a dialing recipe (baseUrl/api present) validates its own // credential fields in parseNamedProviderRecipe, and letting this @@ -2363,6 +2418,99 @@ export function parseCalibrationPolicy(value: unknown): { disabled?: boolean; ma return out; } +// #3254: strict validator shared by the load-time route parsing and the +// per-request resolver (the global network.* tier bypasses load-time parsing, +// so a typo there must still reject loudly instead of silently disabling the +// safety bound). 0 is a valid value — it means "disabled" at its level. +export function parseDeadlineMs(fieldPath: string, value: unknown): number | undefined { + if (value === undefined) return undefined; + if (typeof value === "number" && Number.isInteger(value) && value >= 0) return value; + throw new Error(`[acp-config] ${fieldPath} must be a non-negative integer number of milliseconds (got ${JSON.stringify(value)})`); +} + +// #3254: three-level cascade (model > provider route > global network.*) for +// the model-content deadlines. An explicit 0 wins AT ITS LEVEL (disables that +// phase, no fall-through) AND overrides the built-in default — it is the opt-out +// escape hatch for legitimately-slow routes. An ABSENT field falls back to the +// built-in out-of-the-box default below (the feature is ON by default, #3254 +// round-3 "开箱即用"). contentGapTimeoutMs inherits the resolved initial ceiling +// when left absent. +/** #3254 round-3 out-of-the-box default (owner directive): when no tier sets a + * value, each phase defaults to this bound instead of staying off. Chosen at + * the conservative end of the candidate band — high enough not to false-kill + * legitimate long prefills, still under the codex-webui client's ~780s content + * watchdog so bili acts first with a clean logged error rather than the client + * killing opaquely. SINGLE named constant: retuning the default is one line. */ +export const FC_BUILTIN_DEFAULT_MS = 720_000; +/** #3254 follow-up (owner point 5): per-route adaptive refinement of the + * deadlines. Each route's HEALTHY first-content durations are sampled into + * session metadata as requests complete cleanly; once a route accumulates + * enough samples its deadline stops being a guess and hugs its own normal + * water level. */ +export const FC_ADAPTIVE_MIN_SAMPLES = 8; +/** Per-route sample window (oldest evicted past this). */ +export const FC_ADAPTIVE_WINDOW = 32; +/** Adaptive floor: a warm route's deadline is never tightened below this, so a + * fast route doesn't get an absurdly tight bound. */ +export const FC_ADAPTIVE_FLOOR_MS = 120_000; + +export function fcP95(values: number[]): number { + const sorted = [...values].sort((a, b) => a - b); + return sorted[Math.max(0, Math.min(sorted.length - 1, Math.ceil(0.95 * sorted.length) - 1))]; +} + +/** Clamp 3 × p95(samples) into [FLOOR, ceiling]: never tighter than FLOOR on a + * warm route, never looser than the effective ceiling (built-in or explicit). + * The ceiling FOLLOWS the resolved value instead of a hardcoded cap — otherwise + * a cold route running at e.g. 720s would discontinuously drop to ≤600s once it + * warms up (#3254 round-3 point ③). Returns undefined while the route is still + * cold (fewer than MIN_SAMPLES). */ +export function adaptiveDeadlineMs(samples: number[] | undefined, ceilingMs: number): number | undefined { + if (!samples || samples.length < FC_ADAPTIVE_MIN_SAMPLES) return undefined; + return Math.min(Math.max(3 * fcP95(samples), FC_ADAPTIVE_FLOOR_MS), ceilingMs); +} + +export interface FirstContentDeadlines { + initialMs?: number; + gapMs?: number; + /** Which source produced each phase's value — lands in the fire log so + * config→adaptive cutover never changes the log format (#3254 point 5). + * "builtin" = the out-of-the-box default (no tier set a value). */ + initialSource?: "config" | "adaptive" | "builtin"; + gapSource?: "config" | "adaptive" | "builtin"; +} + +export function resolveFirstContentDeadlines( + routes: ProviderRoutes, + upstreamUrl: string | undefined, + model: string | undefined, + globals: { firstContentTimeoutMs?: unknown; contentGapTimeoutMs?: unknown } | undefined, + samples?: { init: number[]; gap: number[] }, +): FirstContentDeadlines | undefined { + const route = findRoute(routes, upstreamUrl); + const entry = model ? route?.models?.[model] : undefined; + const initialRaw = parseDeadlineMs("firstContentTimeoutMs", entry?.firstContentTimeoutMs ?? route?.firstContentTimeoutMs ?? globals?.firstContentTimeoutMs); + const gapRaw = parseDeadlineMs("contentGapTimeoutMs", entry?.contentGapTimeoutMs ?? route?.contentGapTimeoutMs ?? globals?.contentGapTimeoutMs); + // ABSENT → built-in default (feature ON by default, #3254 round-3); EXPLICIT + // 0 → phase off, overriding the built-in (opt-out escape hatch). A gap left + // absent inherits the initial ceiling and adapts on its own gap samples. + const initialCeiling = initialRaw === undefined ? FC_BUILTIN_DEFAULT_MS : initialRaw; + const gapCeiling = gapRaw === undefined ? initialCeiling : gapRaw; + const out: FirstContentDeadlines = {}; + if (initialCeiling > 0) { + const adaptive = adaptiveDeadlineMs(samples?.init, initialCeiling); + out.initialMs = adaptive !== undefined ? adaptive : initialCeiling; + out.initialSource = adaptive !== undefined ? "adaptive" : (initialRaw === undefined ? "builtin" : "config"); + } + if (gapCeiling > 0) { + const adaptive = adaptiveDeadlineMs(samples?.gap, gapCeiling); + out.gapMs = adaptive !== undefined ? adaptive : gapCeiling; + const baseSource = gapRaw === undefined ? (initialRaw === undefined ? "builtin" : "config") : "config"; + out.gapSource = adaptive !== undefined ? "adaptive" : baseSource; + } + return out.initialMs !== undefined || out.gapMs !== undefined ? out : undefined; +} + function parseStreamErrorShape(value: unknown): "protocol" | "completion" { return value === "completion" ? "completion" : "protocol"; } diff --git a/src/loop/core.ts b/src/loop/core.ts index afaf08dac1..a5c51ad1f0 100644 --- a/src/loop/core.ts +++ b/src/loop/core.ts @@ -38,6 +38,7 @@ import { log as loggerLog } from "../logger.js"; import { promptInputTotal, type WireProtocol } from "../util.js"; import { DEGENERATE_RETRY_NUDGE } from "../degenerate-retry.js"; import { endsWithDraftClose, lastUserSummaryInstruction, requestExpectsProseSummary } from "../degenerate-turn.js"; +import { FC_ADAPTIVE_WINDOW } from "../config.js"; import { safePrefix, safeSuffix, scrubLoneSurrogatesOnWire } from "../text-safe.js"; export const MAX_LOOP_ROUNDS = 10; @@ -208,6 +209,21 @@ export interface LoopCtx { * host = upstream hostname, fp = `${billing}:${cap}` fingerprint. When * present, each sent round captures its image facts alongside the body. */ imageLearn?: { host: string; fp: string }; + /** #3254: model-content deadlines resolved per request (three-level + * cascade via resolveFirstContentDeadlines, optionally refined by the + * route's adaptive samples). Present iff at least one phase is enabled; + * undefined keeps the legacy path byte-identical (no timers armed, zero + * overhead). */ + firstContentDeadline?: { + initialMs?: number; + gapMs?: number; + initialSource?: "config" | "adaptive" | "builtin"; + gapSource?: "config" | "adaptive" | "builtin"; + }; + /** #3254 follow-up: route key (providers URL prefix, else host) under + * which this session's healthy first-content durations are sampled in + * metadata — feeds the route's adaptive deadline on later requests. */ + fcRouteKey?: string; } export interface RequestOptions { @@ -395,6 +411,78 @@ function recordUsage( } } +// #3254: wrapper that lets the model-content deadline KILL a body stream from +// the outside. Plain ReadableStream.cancel() is unusable here: the adapter's +// SSE iterator holds the only reader, and cancel() on a LOCKED stream rejects +// with "Invalid state: ReadableStream is locked" (verified empirically on +// Node 22/24), silently no-oping the kill. So the wrapper owns its own +// controller — kill() errors it (the pending/next read rejects) and releases +// the inner body through ITS OWN reader.cancel (the legal path). Pull-driven +// passthrough keeps backpressure intact (one outstanding inner read). +function interruptibleBody(src: ReadableStream): { stream: ReadableStream; kill: () => void } { + let innerReader: ReadableStreamDefaultReader | null = null; + let outerController: ReadableStreamDefaultController | null = null; + let killed = false; + const killReason = new Error("bili model-content deadline (#3254)"); + const stream = new ReadableStream({ + start(c) { + outerController = c; + innerReader = src.getReader(); + }, + async pull(c) { + if (killed) { c.error(killReason); return; } + try { + const r = await innerReader!.read(); + if (killed) return; + if (r.done) c.close(); + else c.enqueue(r.value); + } catch (e) { + if (!killed) c.error(e instanceof Error ? e : new Error("upstream body read failed")); + } + }, + cancel(reason) { + void innerReader?.cancel(reason).catch(() => {}); + }, + }); + return { + stream, + kill: () => { + if (killed) return; + killed = true; + outerController?.error(killReason); + void innerReader?.cancel(killReason).catch(() => {}); + }, + }; +} + +// #3254 follow-up: per-route healthy-latency samples live in session +// metadata (persisted with the session record) so adaptive deadlines +// survive restarts. One row per route key; each phase keeps its own window. +const FC_SAMPLES_META_KEY = "firstContentSamples"; + +export function readFcSamples(session: Session, routeKey: string): { init: number[]; gap: number[] } | undefined { + const table = session.metadata[FC_SAMPLES_META_KEY] as Record | undefined; + const row = table?.[routeKey]; + if (!row) return undefined; + const clean = (v: unknown): number[] => Array.isArray(v) ? v.filter((x): x is number => typeof x === "number" && Number.isFinite(x) && x >= 0) : []; + return { init: clean(row.init), gap: clean(row.gap) }; +} + +export function recordFcSample(session: Session, routeKey: string, initMs: number, gapMs?: number): void { + const table = (session.metadata[FC_SAMPLES_META_KEY] ?? {}) as Record; + const row = table[routeKey] ?? { init: [], gap: [] }; + if (initMs >= 0) { + row.init.push(initMs); + if (row.init.length > FC_ADAPTIVE_WINDOW) row.init.shift(); + } + if (gapMs !== undefined && gapMs >= 0) { + row.gap.push(gapMs); + if (row.gap.length > FC_ADAPTIVE_WINDOW) row.gap.shift(); + } + table[routeKey] = row; + session.metadata[FC_SAMPLES_META_KEY] = table; +} + export async function* runCompressLoop( upstream: ReadableStream, ctx: LoopCtx, @@ -422,6 +510,64 @@ export async function* runCompressLoop( // in client-stored history (incoming-history stripping removes it next turn, // but one copy per request is enough signal for humans). const seenMarkers = new Set(); + // #3254: model-content deadline. Byte-level silence is already bounded by + // the upstream idle timer, but SSE keep-alive comments/pings keep it armed + // forever — without this bound a relay that streams pings for 13 minutes + // per attempt leaves the CLIENT's content watchdog as the de-facto circuit + // breaker (the incident: 775s + 773s + 27s). Firing kills the wrapped body + // stream (interruptibleBody: controller.error rejects the pending read; + // every adapter's SSE iterator swallows that rejection as EOF, so the + // consumption loop below exits cleanly) and the turn ends + // with ONE clean in-band error — deliberately NO internal refetch + // (#3254 decision C-rejected: an in-band error fails the turn cleanly and + // lets clients that HAVE a retry ladder back off and retry, instead of + // hammering the same stalled slot; clients without one simply stop). + let activeKill: (() => void) | null = null; + let contentDeadlineTimer: ReturnType | null = null; + let contentDeadlineArmedAt = 0; + let contentDeadlineInfo: { ms: number; phase: "initial" | "gap"; round: number; source: "config" | "adaptive" | "builtin" } | null = null; + let contentDeadlineKilled = false; + let contentDeadlineSettled = true; + let upstreamAttemptNo = 0; + // #3254 follow-up: per-attempt healthy-latency sampling for the route's + // adaptive deadline (recorded only when a request completes cleanly). + let fcAttemptStartAt = 0; + let fcFirstContentAt = 0; + let fcLastContentAt = 0; + let fcWorstGapMs = 0; + let fcGapMeasured = false; + const deadlineHost = (() => { try { return new URL(requestOptions.url).host; } catch { return requestOptions.url; } })(); + const clearContentDeadline = () => { + if (contentDeadlineTimer) { + clearTimeout(contentDeadlineTimer); + contentDeadlineTimer = null; + } + }; + const fireContentDeadline = () => { + contentDeadlineTimer = null; + if (contentDeadlineSettled || !contentDeadlineInfo) return; + contentDeadlineKilled = true; + const info = contentDeadlineInfo; + const waitedMs = Date.now() - contentDeadlineArmedAt; + const line = `content-deadline fired: no model content for ${waitedMs}ms (limit ${info.ms}ms, phase=${info.phase}, source=${info.source}, upstream attempt ${upstreamAttemptNo}, round ${info.round}) — aborting stalled upstream stream (#3254)`; + ctx.log(`[acp-proxy: ${line}]`); + loggerLog("warn", `[acp-loop] session ${ctx.session.id} ${line}`); + activeKill?.(); + }; + const armContentDeadline = (ms: number, phase: "initial" | "gap", roundNo: number, source: "config" | "adaptive" | "builtin") => { + clearContentDeadline(); + contentDeadlineInfo = { ms, phase, round: roundNo, source }; + contentDeadlineArmedAt = Date.now(); + contentDeadlineSettled = false; + contentDeadlineTimer = setTimeout(fireContentDeadline, ms); + }; + // #3254: wrap the initial body so the deadline can kill it from outside + // (see interruptibleBody for why plain stream.cancel() cannot do this). + if (ctx.firstContentDeadline !== undefined) { + const wrapped = interruptibleBody(upstream); + activeKill = wrapped.kill; + currentUpstream = wrapped.stream; + } const fetchUpstream = (body: Record) => { // #1592-family seam forensics: remember the body actually sent so the @@ -496,6 +642,15 @@ export async function* runCompressLoop( // consumer parses — the dump twin above keeps RAW upstream bytes, // mirroring the outer request's pre-unwrap tee order in relay.ts. if (body && requestOptions.responseStreamTransform) body = requestOptions.responseStreamTransform(body); + // #3254: wrap the post-#3280-unwrap stream as the OUTERMOST body so + // the deadline kill errors exactly the reads parseStream is making + // and the cancel propagates down through the unwrap transform to the + // raw body (a tee dump branch dying early is acceptable). + if (ctx.firstContentDeadline !== undefined) { + const wrapped = interruptibleBody(body); + activeKill = wrapped.kill; + return wrapped.stream; + } return body; }; @@ -675,8 +830,39 @@ export async function* runCompressLoop( sawSignedThinkingThisAttempt = false; fwdBytes = 0; + // #3254: arm this attempt's initial-phase model-content deadline + // (gap-only configs arm nothing until first content arrives). + upstreamAttemptNo += 1; + contentDeadlineKilled = false; + fcAttemptStartAt = Date.now(); + fcFirstContentAt = 0; + fcLastContentAt = 0; + fcWorstGapMs = 0; + fcGapMeasured = false; + if (ctx.firstContentDeadline?.initialMs !== undefined && ctx.firstContentDeadline.initialMs > 0) { + armContentDeadline(ctx.firstContentDeadline.initialMs, "initial", round, ctx.firstContentDeadline.initialSource ?? "config"); + } else { + clearContentDeadline(); + } for await (const ev of adapter.parseStream(currentUpstream, round)) { if (signal?.aborted) break; + // #3254: any MODEL CONTENT event re-arms the inter-content + // deadline (owner-tiered like the client's own watchdog); + // pings/comments/meta/usage/diag never count as content. + const fcHasContent = (ev.kind === "text" || ev.kind === "reasoning") ? ev.delta.length > 0 + : ev.kind === "tool_call" || ev.kind === "redacted_thinking" || ev.kind === "standalone_signature"; + if (fcHasContent && ctx.firstContentDeadline !== undefined) { + const now = Date.now(); + if (fcFirstContentAt === 0) fcFirstContentAt = now; + else { + fcWorstGapMs = Math.max(fcWorstGapMs, now - fcLastContentAt); + fcGapMeasured = true; + } + fcLastContentAt = now; + } + if (fcHasContent && ctx.firstContentDeadline?.gapMs !== undefined && ctx.firstContentDeadline.gapMs > 0) { + armContentDeadline(ctx.firstContentDeadline.gapMs, "gap", round, ctx.firstContentDeadline.gapSource ?? "config"); + } if (ev.kind === "text") { assistantText += ev.delta; let tp = roundParts[roundParts.length - 1]; @@ -755,6 +941,38 @@ export async function* runCompressLoop( } } } + // #3254: if WE cancelled the body, every adapter's SSE iterator + // swallowed the read rejection as EOF and the loop exited + // cleanly — end in-band exactly once, BEFORE any retry gate: + // a deadline kill must never trigger an internal refetch + // (#3254 decision C-rejected; the client's ladder re-arms). + contentDeadlineSettled = true; + clearContentDeadline(); + if (contentDeadlineKilled && !signal?.aborted) { + // explicit assertion: TS flow analysis does not see the + // closure reassignment inside armContentDeadline, so without + // it the read below is narrowed to the `null` initializer + const info = contentDeadlineInfo as { ms: number; phase: "initial" | "gap"; round: number } | null; + const sec = Math.max(1, Math.round((info?.ms ?? 0) / 1000)); + const phaseDesc = info?.phase === "gap" + ? `no model content for ${sec}s between content events` + : `no model content within ${sec}s of the response starting`; + const msg = `model content deadline exceeded — upstream ${deadlineHost} produced ${phaseDesc} (limit ${info?.ms ?? "?"}ms); your context is intact, resend your last message to continue`; + ctx.log(`[acp-proxy: ${msg}]`); + loggerLog("warn", `[acp-loop] session ${ctx.session.id} content-deadline reached (upstream attempt ${upstreamAttemptNo}, round ${info?.round ?? "?"}); ending turn in-band instead of stalling the client's watchdog (#3254)`); + yield adapter.emitError(msg); + return; + } + // #3254 follow-up: a cleanly completed attempt (done, not + // truncated, not deadline-killed, produced content) is a + // HEALTHY sample of this route's normal latency — feed the + // per-route ring that drives adaptive deadlines later. + if (ctx.fcRouteKey !== undefined && ctx.firstContentDeadline !== undefined + && sawDone && !truncatedDone && fcFirstContentAt > 0) { + // A measured sub-millisecond duration is a legitimate fast + // sample — record it; only unmeasured phases are omitted. + recordFcSample(ctx.session, ctx.fcRouteKey, fcFirstContentAt - fcAttemptStartAt, fcGapMeasured ? fcWorstGapMs : undefined); + } // #2701: threads acceptance. This round's response started ⇔ // the tail advanced past the id the sent body continued from ⇔ @@ -1480,6 +1698,7 @@ export async function* runCompressLoop( recordThreadTail(ctx.session, id); } } + clearContentDeadline(); if (activeClearTimer) { activeClearTimer(); activeClearTimer = null; diff --git a/src/loop/index.ts b/src/loop/index.ts index 5f5ca938c3..e16e1d3725 100644 --- a/src/loop/index.ts +++ b/src/loop/index.ts @@ -1,4 +1,4 @@ -export { runCompressLoop, executeProxyTool, MAX_LOOP_ROUNDS } from "./core.js"; +export { runCompressLoop, executeProxyTool, readFcSamples, recordFcSample, MAX_LOOP_ROUNDS } from "./core.js"; export type { LoopCtx, RequestOptions, diff --git a/src/server/relay.ts b/src/server/relay.ts index 6cf3259dc6..83e0aaba77 100644 --- a/src/server/relay.ts +++ b/src/server/relay.ts @@ -15,7 +15,7 @@ import { applyCompatRoles, applyCompatRolesJson, detectRoleRejection, detectSyst import { compressLoopResponsesJson } from "../compress-loop-responses.js"; import { resolveCompress } from "../compress-settings.js"; import { buildCompressHybridSystemPrompt, buildCompressSystemPrompt, withMarkerIntegrityNote, withProjectAttribution, withSummaryBudgetNote } from "../compress-tool.js"; -import { findRoute, resolveDeclaredProtocol, type ProxyOptions } from "../config.js"; +import { findRoute, findRouteKey, loadConfigFile, resolveDeclaredProtocol, resolveFirstContentDeadlines, type ProxyOptions } from "../config.js"; import { makeContinuationRefetch } from "../degenerate-retry.js"; import { dumpRejectedBody } from "../error-dump.js"; import { externalSummaryEnabled } from "../external-summary-surface.js"; @@ -28,7 +28,7 @@ import { dumpReqAllowed as knobDumpReqAllowed, rawDumpDir as knobRawDumpDir } fr import { gcDumpDirIfConfigured } from "../state-gc.js"; import { applyLaneCredential, laneCredential } from "../lane-credentials.js"; import { CREDENTIAL_HEADER_RE, maskHeaderForLog, maskHeadersForLog, maskUrlForLog } from "../log-mask.js"; -import { pickAdapter, runCompressLoop } from "../loop/index.js"; +import { pickAdapter, readFcSamples, runCompressLoop } from "../loop/index.js"; import { ABSORB_INSTRUCTION_MARKER, containsToolCallXmlFragment } from "../loop/tag-echo-filter.js"; import { dumpsDir } from "../paths.js"; import { pipePluginChatWithStrip, pipePluginJson, pipePluginResponsesWithStrip, type UpstreamMeta } from "../plugin.js"; @@ -1495,9 +1495,17 @@ export async function forward( // the loop has no access to the provider route table. Same billing // upstream expression as outboundPayloadBreakdown. const loopBillingUpstream = route?.rewrittenUrl ?? (/^https?:\/\//i.test(req.url ?? "") ? req.url ?? undefined : opts.upstream); + // #3254: model-content deadlines — three-level cascade (model > + // route > global network.*), resolved per request so config edits + // apply without a restart; absent everywhere → undefined → legacy. + // The Google wire carries its model in the PATH, not the body + // (prepared.google.model) — without the fallback the model tier + // could never match a Gemini-native request. + const fcRouteKey = findRouteKey(opts.routes, loopBillingUpstream) ?? (() => { if (!loopBillingUpstream) return "unknown"; try { return new URL(loopBillingUpstream).host; } catch { return loopBillingUpstream; } })(); + const firstContentDeadline = resolveFirstContentDeadlines(opts.routes, loopBillingUpstream, (parsedReq as { model?: string }).model ?? prepared.google?.model, loadConfigFile().network, readFcSamples(prepared.session, fcRouteKey)); const loop = runCompressLoop( responseBody, - { core, config, messages: prepared.processedMessages.length > 0 ? prepared.processedMessages : prepared.originalMessages, compressMessages: prepared.originalMessages, session: prepared.session, log: ctx.log, proxyUrl, skippedProxy, upstreamOrigin: targetOrigin, protocol: prepared.protocol, textProtocol, debug: opts.debug, refreshFolded, visibilityMarkers, dumpSse: loopDumpDir ? (name, stream) => dumpStreamToFile(stream, loopDumpDir, name) : undefined, imageLearn: { host: upstreamHost(loopBillingUpstream), fp: `${imageBillingFor(opts, loopBillingUpstream)}:${imageTokenCapFor(opts, loopBillingUpstream)}` } }, + { core, config, messages: prepared.processedMessages.length > 0 ? prepared.processedMessages : prepared.originalMessages, compressMessages: prepared.originalMessages, session: prepared.session, log: ctx.log, proxyUrl, skippedProxy, upstreamOrigin: targetOrigin, protocol: prepared.protocol, textProtocol, debug: opts.debug, refreshFolded, visibilityMarkers, dumpSse: loopDumpDir ? (name, stream) => dumpStreamToFile(stream, loopDumpDir, name) : undefined, imageLearn: { host: upstreamHost(loopBillingUpstream), fp: `${imageBillingFor(opts, loopBillingUpstream)}:${imageTokenCapFor(opts, loopBillingUpstream)}` }, firstContentDeadline, fcRouteKey }, parsedReq, // #3280: egress goes out enveloped via wireTransform, so every // round-2+ response comes back enveloped too — unwrap at the loop's diff --git a/tests/issue3254-first-content-deadline.test.ts b/tests/issue3254-first-content-deadline.test.ts new file mode 100644 index 0000000000..eb2d2cc632 --- /dev/null +++ b/tests/issue3254-first-content-deadline.test.ts @@ -0,0 +1,364 @@ +import { test } from "node:test"; +import assert from "node:assert/strict"; +import { mkdtempSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import type { Config, CoreMessage } from "acp-kernel"; +import { createCore, createInitialState } from "acp-kernel"; +import type { Session } from "../src/session.ts"; +import { runCompressLoop, createAnthropicAdapter } from "../src/loop/index.ts"; +import { buildCompressSystemPrompt } from "../src/compress-tool.ts"; +import { adaptiveDeadlineMs, FC_ADAPTIVE_MIN_SAMPLES, FC_ADAPTIVE_WINDOW, fcP95, loadRoutes, parseDeadlineMs, resolveFirstContentDeadlines, type ProviderRoutes } from "../src/config.ts"; +import { readFcSamples, recordFcSample } from "../src/loop/index.ts"; + +// #3254: in-request summarization had no bili-side wait bound of its own. The +// upstream idle timer is byte-level — SSE keep-alive comments/pings keep it +// armed forever — so a relay that streams pings for 13 minutes per attempt +// left the CLIENT's content watchdog (780s, codex-webui) as the de-facto +// circuit breaker. Incident: one fold = 775s + 773s + 27s across three client +// attempts, bili.log fully silent. Pinned here: (A) an initial-phase deadline +// fires, cancels the stalled body, ends the turn with ONE clean in-band error +// and NO internal refetch (owner decision C-rejected: the client's retry +// ladder re-arms with backoff); (B) the inter-content (gap) phase fires after +// first content; (C) live content re-arms the deadline and never false-fires; +// (D) absent config stays byte-identical legacy (no timers, no early exit); +// (E-H) the three-level cascade (model > route > global network.*) incl. +// explicit-0-wins-at-level, gap-inherits-initial, and strict validation; +// (I-L) route-adaptive refinement (owner point 5): clamp(3×p95(healthy +// first-content durations), 120s, 600s) capped by the explicit config value, +// samples stored per route in session metadata, healthy attempts only. + +const ANTHROPIC_BODY = { model: "claude", messages: [], stream: true, max_tokens: 10 }; +const enc = new TextEncoder(); + +function makeCtx(id: string, logs: string[]): { + core: ReturnType; + config: Config; + messages: CoreMessage[]; + session: Session; + log: (m: string) => void; +} { + return { + core: createCore(), + config: { modelContextLimit: 200000 } as Config, + messages: [], + session: { + id, + meta: {}, + stats: { requests: 0, tokensSaved: 0, inputTokens: 0, cachedTokens: 0, outputTokens: 0, cacheSamples: 0, lastInputTokens: 0, contextTokens: 0, compressCreditTokens: 0, retrieveCalls: 0, retrieveHits: 0, retrieveMisses: 0, storedBytes: 0, storeBytesSaved: 0, rangeRestores: 0 }, + metadata: {}, + state: createInitialState(), + createdAt: Date.now(), + lastSeen: Date.now(), + blockContents: new Map(), + inFlight: 0, + persisted: false, + pendingRetrievals: [], + }, + log: (m: string) => { logs.push(m); }, + }; +} + +const ev = (type: string, data: unknown): string => `event: ${type}\ndata: ${JSON.stringify(data)}\n\n`; + +const MESSAGE_START = ev("message_start", { + type: "message_start", + message: { + id: "msg_1", type: "message", role: "assistant", model: "claude", + content: [], stop_reason: null, stop_sequence: null, + usage: { input_tokens: 5, output_tokens: 0 }, + }, +}); +const PING = ev("ping", { type: "ping" }); +const textStart = (index: number) => ev("content_block_start", { type: "content_block_start", index, content_block: { type: "text", text: "" } }); +const textDelta = (index: number, text: string) => ev("content_block_delta", { type: "content_block_delta", index, delta: { type: "text_delta", text } }); +const blockStop = (index: number) => ev("content_block_stop", { type: "content_block_stop", index }); +const MESSAGE_DELTA = (stopReason = "end_turn") => ev("message_delta", { type: "message_delta", delta: { stop_reason: stopReason, stop_sequence: null }, usage: { input_tokens: 5, output_tokens: 3 } }); +const MESSAGE_STOP = ev("message_stop", { type: "message_stop" }); + +// Emits `text` synchronously then closes — everything is consumed before any +// macrotask timer can fire (deterministic, no timing margin needed). +function completeStream(text: string): ReadableStream { + return new ReadableStream({ + start(controller) { + controller.enqueue(enc.encode(text)); + controller.close(); + }, + }); +} + +// Emits `text` then HOLDS OPEN (the incident shape: pings/comments keep the +// byte idle-timer armed; no model content ever comes). Records cancellation. +function holdStream(text: string, sink: { cancelled: boolean }): ReadableStream { + return new ReadableStream({ + start(controller) { + controller.enqueue(enc.encode(text)); + }, + cancel() { sink.cancelled = true; }, + }); +} + +async function drainGen(gen: AsyncGenerator, chunks: Buffer[], timeoutMs: number): Promise { + const drain = (async () => { for await (const c of gen) chunks.push(c); })(); + let timer: ReturnType | undefined; + const guard = new Promise((_, reject) => { + timer = setTimeout(() => reject(new Error(`drain timed out after ${timeoutMs}ms (generator hung)`)), timeoutMs); + }); + try { + await Promise.race([drain, guard]); + } catch (e) { + // fire-and-forget on purpose: if the generator is parked on a read that + // can never settle (a deliberately held-open stream), .return() queues + // BEHIND the stuck .next() and never settles itself — awaiting it + // deadlocks the test (verified empirically). The abandoned generator is + // GC-eligible; it owns no handles. + void gen.return(undefined).catch(() => {}); + throw e; + } finally { + clearTimeout(timer); + } +} + +async function drainAnthropic( + stream: ReadableStream, + deadline: { initialMs?: number; gapMs?: number } | undefined, + id: string, + timeoutMs: number, + fcRouteKey?: string, +): Promise<{ out: string; fetchCalls: number; logs: string[]; session: Session }> { + let n = 0; + const orig = globalThis.fetch; + // Any fetch here means the loop REFETCHED — the #3254 decision pins that a + // deadline kill never triggers an internal refetch (C-rejected). + globalThis.fetch = (async () => { n += 1; throw new Error("fetch called — internal refetch forbidden (#3254 C-rejected)"); }) as typeof fetch; + const logs: string[] = []; + const ctx = makeCtx(id, logs); + (ctx as Record).protocol = "anthropic"; + if (deadline !== undefined) (ctx as Record).firstContentDeadline = deadline; + if (fcRouteKey !== undefined) (ctx as Record).fcRouteKey = fcRouteKey; + const chunks: Buffer[] = []; + try { + const gen = runCompressLoop( + stream, + ctx, + ANTHROPIC_BODY, + { url: "http://mock", headers: {} }, + createAnthropicAdapter(ANTHROPIC_BODY, undefined, undefined, "protocol"), + buildCompressSystemPrompt(), + ); + await drainGen(gen, chunks, timeoutMs); + } finally { + globalThis.fetch = orig; + } + return { out: Buffer.concat(chunks).toString("utf8"), fetchCalls: n, logs, session: ctx.session }; +} + +const sseBlocks = (s: string): string[] => s.split("\n\n").map((b) => b.trim()).filter(Boolean); + +// --- A: initial-phase deadline fires on a stalled stream ------------------- + +test("#3254 A: initial-phase deadline cancels the stalled body and ends in-band once, no refetch", async () => { + const sink = { cancelled: false }; + const { out, fetchCalls, logs } = await drainAnthropic(holdStream(MESSAGE_START + PING, sink), { initialMs: 150, gapMs: 150 }, "s3254a", 3000); + assert.match(out, /model content deadline exceeded/); + assert.match(out, /no model content within 1s of the response starting \(limit 150ms\)/); + assert.match(out, /your context is intact, resend your last message to continue/); + assert.ok(sseBlocks(out).at(-1)?.startsWith("event: error"), "last SSE block must be the protocol-native error channel"); + assert.ok(logs.some((l) => l.includes("content-deadline fired") && l.includes("phase=initial") && l.includes("source=config") && l.includes("upstream attempt 1") && l.includes("round 1")), `fire log missing/malformed: ${JSON.stringify(logs)}`); + assert.equal(fetchCalls, 0, "deadline kill must NOT trigger an internal refetch"); + assert.equal(sink.cancelled, true, "the stalled upstream body must be cancelled"); +}); + +// --- B: gap phase fires after first content; pings do not re-arm ----------- + +test("#3254 B: inter-content (gap) deadline fires after first content; ping does not re-arm it", async () => { + const sink = { cancelled: false }; + const prefix = MESSAGE_START + textStart(0) + textDelta(0, "hi") + blockStop(0) + PING; + const { out, fetchCalls, logs } = await drainAnthropic(holdStream(prefix, sink), { gapMs: 150 }, "s3254b", 3000); + assert.match(out, /model content deadline exceeded/); + assert.match(out, /no model content for 1s between content events \(limit 150ms\)/); + assert.ok(logs.some((l) => l.includes("content-deadline fired") && l.includes("phase=gap")), `gap fire log missing: ${JSON.stringify(logs)}`); + assert.equal(fetchCalls, 0); + assert.equal(sink.cancelled, true); +}); + +// --- C: live content re-arms the deadline — no false positive -------------- + +test("#3254 C: continuous content re-arms the deadline; a healthy slow stream completes cleanly", async () => { + const text = MESSAGE_START + textStart(0) + textDelta(0, "a") + textDelta(0, "b") + blockStop(0) + MESSAGE_DELTA() + MESSAGE_STOP; + const { out, fetchCalls, logs } = await drainAnthropic(completeStream(text), { initialMs: 100, gapMs: 100 }, "s3254c", 3000); + assert.doesNotMatch(out, /model content deadline exceeded/); + assert.doesNotMatch(out, /event: error/); + assert.match(out, /"text_delta"/); + assert.ok(!logs.some((l) => l.includes("content-deadline")), `healthy stream must not trip the deadline: ${JSON.stringify(logs)}`); + assert.equal(fetchCalls, 0); +}); + +// --- D: absent config = legacy behavior (no timer, stream stays pending) --- + +test("#3254 D: without firstContentDeadline the legacy path is untouched (stall stays pending)", async () => { + // node:test treats "event loop drained while tests still pending" as fatal; + // this test deliberately leaves a stream read dangling, so pin a keep-alive + // handle for its duration or the later tests in this file get cancelled. + const keepAlive = setInterval(() => {}, 50); + const sink = { cancelled: false }; + let hung = false; + try { + try { + await drainAnthropic(holdStream(MESSAGE_START + PING, sink), undefined, "s3254d", 400); + } catch (e) { + hung = e instanceof Error && /drain timed out/.test(e.message); + } + assert.ok(hung, "legacy path must keep waiting on the stalled stream (no early exit)"); + assert.equal(sink.cancelled, false, "nothing may cancel the body when the feature is off"); + } finally { + clearInterval(keepAlive); + } +}); + +// --- E: parseDeadlineMs strictness ----------------------------------------- + +test("#3254 E: parseDeadlineMs accepts undefined/0/integers and rejects everything else", () => { + assert.equal(parseDeadlineMs("x", undefined), undefined); + assert.equal(parseDeadlineMs("x", 0), 0); + assert.equal(parseDeadlineMs("x", 180000), 180000); + for (const bad of [-1, 1.5, Number.NaN, "180000", true, null]) { + assert.throws(() => parseDeadlineMs("x", bad), /\[acp-config\] x must be a non-negative integer number of milliseconds/); + } +}); + +// --- F: three-level cascade semantics -------------------------------------- + +const ROUTES: ProviderRoutes = { + "https://relay-a.example": { + firstContentTimeoutMs: 600000, + models: { "gpt-x": { firstContentTimeoutMs: 90000 } }, + }, + "https://relay-b.example": { + firstContentTimeoutMs: 0, + }, + "https://relay-c.example": { + firstContentTimeoutMs: 180000, + contentGapTimeoutMs: 0, + }, +}; + +test("#3254 F: cascade — model > route > global, 0 wins at its level, absent falls to built-in, gap inherits initial", () => { + // all absent → OUT-OF-THE-BOX built-in default (feature ON by default, #3254 round-3) + assert.deepEqual(resolveFirstContentDeadlines({}, undefined, undefined, undefined), { initialMs: 720000, initialSource: "builtin", gapMs: 720000, gapSource: "builtin" }); + // global tier alone; gap inherits the resolved initial ceiling + assert.deepEqual(resolveFirstContentDeadlines({}, "https://any.example", undefined, { firstContentTimeoutMs: 180000 }), { initialMs: 180000, initialSource: "config", gapMs: 180000, gapSource: "config" }); + // global gap-only: initial is ABSENT so it falls to the built-in default; gap takes the explicit value + assert.deepEqual(resolveFirstContentDeadlines({}, "https://any.example", undefined, { contentGapTimeoutMs: 300000 }), { initialMs: 720000, initialSource: "builtin", gapMs: 300000, gapSource: "config" }); + // explicit 0 at the GLOBAL tier disables every phase (opt-out escape hatch overrides the built-in) + assert.equal(resolveFirstContentDeadlines({}, "https://any.example", undefined, { firstContentTimeoutMs: 0 }), undefined); + // route beats global + assert.deepEqual(resolveFirstContentDeadlines(ROUTES, "https://relay-a.example/v1/messages", undefined, { firstContentTimeoutMs: 180000 }), { initialMs: 600000, initialSource: "config", gapMs: 600000, gapSource: "config" }); + // model beats route + assert.deepEqual(resolveFirstContentDeadlines(ROUTES, "https://relay-a.example/v1/messages", "gpt-x", { firstContentTimeoutMs: 180000 }), { initialMs: 90000, initialSource: "config", gapMs: 90000, gapSource: "config" }); + // model ignored when the URL matches no route + assert.deepEqual(resolveFirstContentDeadlines(ROUTES, "https://other.example", "gpt-x", { firstContentTimeoutMs: 180000 }), { initialMs: 180000, initialSource: "config", gapMs: 180000, gapSource: "config" }); + // explicit 0 at route level disables the phase (no fall-through to global); + // gap then inherits nothing → feature fully off + assert.equal(resolveFirstContentDeadlines(ROUTES, "https://relay-b.example", undefined, { firstContentTimeoutMs: 180000 }), undefined); + // explicit 0 for gap only → initial survives, gap phase off + assert.deepEqual(resolveFirstContentDeadlines(ROUTES, "https://relay-c.example", undefined, undefined), { initialMs: 180000, initialSource: "config" }); + // garbage at ANY tier rejects loudly (the global tier bypasses load-time parsing) + assert.throws(() => resolveFirstContentDeadlines({}, "https://any.example", undefined, { firstContentTimeoutMs: "fast" }), /firstContentTimeoutMs must be a non-negative integer/); +}); + +// --- G: load-time validation of providers entries (route + model level) ---- + +test("#3254 G: loadRoutes rejects malformed deadline values at provider and model level, accepts valid ones", () => { + const dir = mkdtempSync(join(tmpdir(), "bili3254-")); + const path = join(dir, "providers.json"); + const load = (obj: unknown) => { + writeFileSync(path, JSON.stringify(obj)); + return loadRoutes({ ACP_PROVIDERS: path }); + }; + assert.throws(() => load({ "https://relay-a.example": { firstContentTimeoutMs: -5 } }), /providers entry firstContentTimeoutMs must be a non-negative integer/); + assert.throws(() => load({ "https://relay-a.example": { models: { m: { contentGapTimeoutMs: "soon" } } } }), /providers entry models\.m\.contentGapTimeoutMs must be a non-negative integer/); + const ok = load({ "https://relay-a.example": { firstContentTimeoutMs: 0, models: { m: { contentGapTimeoutMs: 120000 } } } }); + assert.equal(ok["https://relay-a.example"]?.firstContentTimeoutMs, 0); + assert.equal(ok["https://relay-a.example"]?.models?.["m"]?.contentGapTimeoutMs, 120000); +}); + +// --- I: adaptive deadline math ---------------------------------------------- + +test("#3254 I: fcP95 + adaptiveDeadlineMs — cold undefined, [floor, ceiling] clamp, ceiling follows resolved value", () => { + assert.equal(fcP95([42]), 42); + // nearest-rank: at n=8 the rank is ceil(0.95×8)=8 → the max; at n=20 it is 19 → second-highest + assert.equal(fcP95([5, 10, 15, 20, 25, 30, 35, 40]), 40); + assert.equal(fcP95(Array.from({ length: 20 }, (_, i) => (i + 1) * 5)), 95); + assert.equal(adaptiveDeadlineMs(undefined, 180000), undefined); + assert.equal(adaptiveDeadlineMs(Array.from({ length: FC_ADAPTIVE_MIN_SAMPLES - 1 }, () => 20000), 180000), undefined, "one below MIN_SAMPLES is still cold"); + // p95=20000 → 3×=60000 → clamped UP to the 120s floor + assert.equal(adaptiveDeadlineMs(Array.from({ length: FC_ADAPTIVE_MIN_SAMPLES }, () => 20000), 180000), 120000); + // p95=200000 → 3×=600000 → exactly the 600000 ceiling passed here + assert.equal(adaptiveDeadlineMs(Array.from({ length: FC_ADAPTIVE_MIN_SAMPLES }, () => 200000), 600000), 600000); + // mid-band: p95=50000 → 150000, below the ceiling + assert.equal(adaptiveDeadlineMs(Array.from({ length: FC_ADAPTIVE_MIN_SAMPLES }, () => 50000), 180000), 150000); + // a user-pinned tighter ceiling is never relaxed upward (floor yields to it) + assert.equal(adaptiveDeadlineMs(Array.from({ length: FC_ADAPTIVE_MIN_SAMPLES }, () => 20000), 60000), 60000); + // ceiling FOLLOWS the resolved value (no hardcoded 600s cap): a slow route + // (p95=300s → 3×=900s) under a 720s ceiling keeps its full 720s instead of + // discontinuously dropping to ≤600s once warm (#3254 round-3 point ③) + assert.equal(adaptiveDeadlineMs(Array.from({ length: FC_ADAPTIVE_MIN_SAMPLES }, () => 300000), 720000), 720000); +}); + +// --- J: resolver with adaptive samples -------------------------------------- + +test("#3254 J: warm routes adapt (source=adaptive) under their configured ceiling; cold stay on config", () => { + const globals = { firstContentTimeoutMs: 180000 }; + const warm = Array.from({ length: FC_ADAPTIVE_MIN_SAMPLES }, () => 20000); + // warm initial, no gap samples yet: initial adapts to the floor; the + // INHERITED gap has no gap samples of its own and sits at the inherited + // initial CEILING (the configured bound, not the tightened value) + assert.deepEqual(resolveFirstContentDeadlines({}, "https://any.example", undefined, globals, { init: warm, gap: [] }), { + initialMs: 120000, initialSource: "adaptive", gapMs: 180000, gapSource: "config", + }); + // both phases warm in-band: p95=50000 → 150000 each + const mid = Array.from({ length: FC_ADAPTIVE_MIN_SAMPLES }, () => 50000); + assert.deepEqual(resolveFirstContentDeadlines({}, "https://any.example", undefined, globals, { init: mid, gap: mid }), { + initialMs: 150000, initialSource: "adaptive", gapMs: 150000, gapSource: "adaptive", + }); + // explicit ceiling tighter than the adaptive floor: adaptation engages but + // the user's pinned value wins as the ceiling + assert.deepEqual(resolveFirstContentDeadlines({}, "https://any.example", undefined, { firstContentTimeoutMs: 60000 }, { init: warm, gap: [] }), { + initialMs: 60000, initialSource: "adaptive", gapMs: 60000, gapSource: "config", + }); +}); + +// --- K: sample ring storage in session metadata ------------------------------ + +test("#3254 K: recordFcSample/readFcSamples — window cap, per-phase rows, sanitizing read", () => { + const s = makeCtx("s3254k", []).session; + assert.equal(readFcSamples(s, "r"), undefined); + recordFcSample(s, "r", 1234, 567); + recordFcSample(s, "r", 2222); + let got = readFcSamples(s, "r"); + assert.deepEqual(got, { init: [1234, 2222], gap: [567] }); + const s2 = makeCtx("s3254k2", []).session; + for (let i = 0; i < FC_ADAPTIVE_WINDOW + 5; i++) recordFcSample(s2, "r", i); + got = readFcSamples(s2, "r"); + assert.equal(got?.init.length, FC_ADAPTIVE_WINDOW, "oldest samples must be evicted past the window"); + assert.equal(got?.init[0], 5, "the first five records must have been evicted"); + assert.equal(got?.gap.length, 0); + (s.metadata as Record)["firstContentSamples"] = { r: { init: [1, Number.NaN, -5, "x"], gap: null } }; + assert.deepEqual(readFcSamples(s, "r"), { init: [1], gap: [] }, "reads must sanitize hand-corrupted metadata"); +}); + +// --- L: end-to-end sampling — healthy attempts only -------------------------- + +test("#3254 L: a cleanly completed attempt records one healthy sample per phase; a killed attempt records none", async () => { + const text = MESSAGE_START + textStart(0) + textDelta(0, "a") + textDelta(0, "b") + blockStop(0) + MESSAGE_DELTA() + MESSAGE_STOP; + const healthy = await drainAnthropic(completeStream(text), { initialMs: 100000 }, "s3254l", 3000, "route-l"); + const sample = readFcSamples(healthy.session, "route-l"); + assert.ok(sample !== undefined && sample.init.length === 1 && sample.gap.length === 1, `expected one init + one gap sample: ${JSON.stringify(sample)}`); + // the stream is synchronous, so both measured durations are legitimately 0ms + assert.ok(sample!.init[0] >= 0 && sample!.init[0] < 10000, `first-content duration implausible: ${sample?.init[0]}ms`); + const sink = { cancelled: false }; + const killed = await drainAnthropic(holdStream(MESSAGE_START + PING, sink), { initialMs: 150 }, "s3254l2", 3000, "route-l2"); + assert.match(killed.out, /model content deadline exceeded/); + assert.equal(readFcSamples(killed.session, "route-l2"), undefined, "a deadline-killed attempt must not poison the route's samples"); +}); diff --git a/website/config-reference/process.yaml b/website/config-reference/process.yaml index 8159ef2030..a1e457ecf2 100644 --- a/website/config-reference/process.yaml +++ b/website/config-reference/process.yaml @@ -10,8 +10,8 @@ entries: site_group: proc desc_en: "Transport timeouts, watchdogs and replay policy for upstream calls." desc_zh: "上游调用的传输超时、看门狗与重放策略。" - site_en: "network — transport timeouts & replays: upstreamTimeoutMs 720000 · requestWatchdogMs 2× · replayRetryMax 3 · replayRetryBaseMs 1500 · keepAliveTimeoutMs 5000 · clientErrorBackstopMs 30000 · exposureLogIntervalMs 3600000 · streamKeepAliveMs 15000 · preflightHoldMs 30000 · preflightDeadEndCooldownMs 300000 · proxyKeepAliveMaxMs 55000." - site_zh: "network —— 传输超时与重放:upstreamTimeoutMs 720000 · requestWatchdogMs 2× · replayRetryMax 3 · replayRetryBaseMs 1500 · keepAliveTimeoutMs 5000 · clientErrorBackstopMs 30000 · exposureLogIntervalMs 3600000 · streamKeepAliveMs 15000 · preflightHoldMs 30000 · preflightDeadEndCooldownMs 300000 · proxyKeepAliveMaxMs 55000。" + site_en: "network — transport timeouts & replays: upstreamTimeoutMs 720000 · requestWatchdogMs 2× · replayRetryMax 3 · replayRetryBaseMs 1500 · keepAliveTimeoutMs 5000 · clientErrorBackstopMs 30000 · exposureLogIntervalMs 3600000 · streamKeepAliveMs 15000 · preflightHoldMs 30000 · preflightDeadEndCooldownMs 300000 · proxyKeepAliveMaxMs 55000 · firstContentTimeoutMs 720000 on-by-default (#3254, file-only) · contentGapTimeoutMs inherits/720000 (#3254, file-only)." + site_zh: "network —— 传输超时与重放:upstreamTimeoutMs 720000 · requestWatchdogMs 2× · replayRetryMax 3 · replayRetryBaseMs 1500 · keepAliveTimeoutMs 5000 · clientErrorBackstopMs 30000 · exposureLogIntervalMs 3600000 · streamKeepAliveMs 15000 · preflightHoldMs 30000 · preflightDeadEndCooldownMs 300000 · proxyKeepAliveMaxMs 55000 · firstContentTimeoutMs 720000 默认开启(#3254,仅文件)· contentGapTimeoutMs 继承/默认 720000(#3254,仅文件)。" - key: network.upstreamTimeoutMs type: number default: "720000" @@ -68,6 +68,18 @@ entries: group: process desc_en: "Cooldown after a dead-end preflight before retrying reconciliation for the same session." desc_zh: "死胡同 preflight 后,对同一会话重试校准的冷却时长。" + - key: network.firstContentTimeoutMs + type: number + default: "720000 (on by default)" + group: process + desc_en: "Model-content deadline, phase 1 (#3254): max ms an upstream attempt may go with zero model content from response start before bili aborts the stalled stream and ends the turn with one clean in-band error (no internal refetch — the turn fails cleanly; clients with a retry ladder back off and retry, clients without one simply stop). On by default at 720000ms so every deployment is bounded out of the box; set a value to tighten/override, or 0 to disable (opt-out override of the built-in). Also settable per route / per model under providers; cascade model > route > global, explicit 0 wins at its level. Once a route has ≥8 healthy attempts the resolved value becomes a ceiling and the effective deadline tightens to clamp(3 × p95 of the route's recent healthy first-content durations, 120s, that resolved ceiling); stalled/killed attempts are never sampled. File-only by design." + desc_zh: "模型内容期限,阶段一(#3254):上游一次尝试从响应开始到首个模型内容之间的最长毫秒数;超时则 bili 中止卡死流并以一条干净的带内错误结束回合(不做内部盲重取——回合干净失败:有重试梯的客户端退避后重试,没有的则直接停止)。默认以 720000ms 开启,让每个部署开箱即用即有界;设值可收紧/覆盖,或设 0 关闭(对内置值的显式豁免)。providers 内也可按线路/按模型设置;级联 model > route > global,显式 0 在所在层生效。线路积累 ≥8 次健康尝试后,解析值转为上限,有效期限收紧为 clamp(该线路近期健康首内容耗时的 3 × p95, 120s, 该解析上限);卡死或被中止的尝试从不采样。按设计仅配置文件可设。" + - key: network.contentGapTimeoutMs + type: number + default: "inherits firstContentTimeoutMs (720000 default)" + group: process + desc_en: "Model-content deadline, phase 2 (#3254): max ms between consecutive model-content events after the first arrived; re-armed on every content event (pings/comments never count). Absent inherits the resolved firstContentTimeoutMs; explicit 0 disables the gap phase only. Same three-level cascade, same route-adaptive ceiling (per-phase samples) and file-only rationale as firstContentTimeoutMs." + desc_zh: "模型内容期限,阶段二(#3254):首个内容到达之后,相邻两个模型内容事件之间的最长毫秒数;每个内容事件重新计时(ping/注释不算内容)。缺省继承解析后的 firstContentTimeoutMs;显式 0 仅禁用阶段二。三级级联、线路自适应上限(按阶段各自采样)与仅文件可设的理由同 firstContentTimeoutMs。" - key: network.replayRetryMax type: number default: "3 (1 disables replays)" diff --git a/website/config-reference/providers.yaml b/website/config-reference/providers.yaml index 7a10abbbac..279503762a 100644 --- a/website/config-reference/providers.yaml +++ b/website/config-reference/providers.yaml @@ -176,6 +176,26 @@ entries: desc_zh: "按路由的 k̂ 校准策略(#3211):对虚报报数线路做退出或钳制;见[线路级 k̂ 校准策略](#route-level-k-calibration-policy)。" site_en: "Per-route k̂ calibration policy (#3211): `disabled` suppresses the learned factor on this route; `max` clamps it (0 < max ≤ 4). Applied at publication time — the learned ring stays raw." site_zh: "按路由的 k̂ 校准策略(#3211):`disabled` 在该线路禁用学习系数;`max` 钳制学习系数(0 < max ≤ 4)。作用于发布环节——学习环保持原始值。" + - key: firstContentTimeoutMs + type: number + default: "unset (→ global, else 720000 builtin)" + group: providers + id: cr.fcdeadline + render: route-row + desc_en: "Per-route model-content deadline, phase 1 (#3254): max ms without model content from response start before bili aborts the stalled upstream and ends the turn in-band. On by default via the built-in (unset falls through to global then the 720000 builtin); set here to override per route, or 0 to disable for this route. Also settable per model under models[]; cascade model > route > global network.*, explicit 0 wins at its level. Once the route has ≥8 healthy attempts the resolved value becomes a ceiling and the effective deadline tightens to clamp(3 × p95 of this route's recent healthy first-content durations, 120s, that resolved ceiling); stalled/killed attempts are never sampled. File-only (no env)." + desc_zh: "按线路的模型内容期限,阶段一(#3254):响应开始后无模型内容的上限毫秒数;超时则中止卡死上游并以带内错误结束回合。通过内置值默认开启(缺省落到 global 再到内置 720000);在此按线路覆盖,或设 0 对该线路关闭。也可在 models[] 按模型设置;级联 model > route > global network.*,显式 0 在所在层生效。线路积累 ≥8 次健康尝试后,解析值转为上限,有效期限收紧为 clamp(该线路近期健康首内容耗时的 3 × p95, 120s, 该解析上限);卡死或被中止的尝试从不采样。仅配置文件可设(无 env)。" + site_en: "Model-content deadline, phase 1 (#3254), per route and per model (`models[]`); on expiry bili aborts the stalled upstream stream and ends the turn with one clean in-band error — no internal refetch. On by default via the built-in (unset → global → 720000 builtin); explicit 0 disables for this route. Cascade model > route > global `network.*`; once warmed (≥8 healthy attempts) the resolved value acts as a ceiling tightened toward the route's measured first-content pace (clamp 3×p95, floor 120s, capped at the resolved value); file-only." + site_zh: "模型内容期限,阶段一(#3254),按线路与按模型(`models[]`);到期则中止卡死的上游流并以一条干净的带内错误结束回合——不做内部盲重取。通过内置值默认开启(缺省 → global → 内置 720000);显式 0 对该线路关闭。级联 model > route > global `network.*`;线路预热后(≥8 次健康尝试)解析值转为上限,向该线路实测首内容水位收紧(clamp 3×p95,下限 120s,上限为解析值);仅文件可设。" + - key: contentGapTimeoutMs + type: number + default: "unset (inherits firstContentTimeoutMs)" + group: providers + id: cr.gapdeadline + render: route-row + desc_en: "Per-route model-content deadline, phase 2 (#3254): max ms between consecutive model-content events after the first arrived; re-armed on every content event. Absent inherits the resolved firstContentTimeoutMs; explicit 0 disables the gap phase only. Same cascade, same route-adaptive ceiling (its own inter-content-gap samples) and file-only rationale as firstContentTimeoutMs." + desc_zh: "按线路的模型内容期限,阶段二(#3254):首个内容之后相邻模型内容事件间的上限毫秒数,逐事件重新计时。缺省继承解析后的 firstContentTimeoutMs;显式 0 仅禁用阶段二。级联、线路自适应上限(用自己的内容间隔样本)与仅文件可设的理由同 firstContentTimeoutMs。" + site_en: "Model-content deadline, phase 2 (#3254), per route and per model (`models[]`); bounds silence between content events after the first arrived (pings/comments never count); absent inherits the resolved `firstContentTimeoutMs`; same route-adaptive ceiling as `firstContentTimeoutMs`. File-only." + site_zh: "模型内容期限,阶段二(#3254),按线路与按模型(`models[]`);约束首个内容之后相邻内容事件间的静默(ping/注释不算内容);缺省继承解析后的 `firstContentTimeoutMs`;线路自适应上限同 `firstContentTimeoutMs`。仅文件可设。" - key: compactionOptIn type: boolean default: "false" diff --git a/website/docs/index.html b/website/docs/index.html index ce0f9b00ba..d8f40ae6b9 100644 --- a/website/docs/index.html +++ b/website/docs/index.html @@ -239,6 +239,8 @@

providers — route table fields

apiKeyEnvstring (env var name)Lane credential override (#2336): requests forwarded through this lane carry this env variable's value instead of the client's own credential — x-api-key / x-goog-api-key replaced outright, Bearer authorization replaced, injected when absent. Exactly one of apiKeyEnv / credentialRef per entry. credentialRefstring (store name)Lane credential override via the private summary-credential store (`secret:NAME` semantics); exactly one of apiKeyEnv / credentialRef per entry. calibration{ disabled?: boolean, max?: number }Per-route k̂ calibration policy (#3211): `disabled` suppresses the learned factor on this route; `max` clamps it (0 < max ≤ 4). Applied at publication time — the learned ring stays raw. + firstContentTimeoutMsnumberModel-content deadline, phase 1 (#3254), per route and per model (`models[]`); on expiry bili aborts the stalled upstream stream and ends the turn with one clean in-band error — no internal refetch. On by default via the built-in (unset → global → 720000 builtin); explicit 0 disables for this route. Cascade model > route > global `network.*`; once warmed (≥8 healthy attempts) the resolved value acts as a ceiling tightened toward the route's measured first-content pace (clamp 3×p95, floor 120s, capped at the resolved value); file-only. + contentGapTimeoutMsnumberModel-content deadline, phase 2 (#3254), per route and per model (`models[]`); bounds silence between content events after the first arrived (pings/comments never count); absent inherits the resolved `firstContentTimeoutMs`; same route-adaptive ceiling as `firstContentTimeoutMs`. File-only. compactionOptInbooleanNamed entries only: opt a non-http(s)-baseUrl provider into compaction ownership (pi/omp lanes); unions with env BILI_NON_HTTP_PROVIDERS. @@ -278,7 +280,7 @@

Other top-level fields

  • mitm.enabled / mitm.domains — HTTPS interception switch and whitelist domains for login clients.
  • mitm.handshakeTimeoutMs — TLS handshake timeout (default 10000).
  • Process blocks
  • -
  • network — transport timeouts & replays: upstreamTimeoutMs 720000 · requestWatchdogMs 2× · replayRetryMax 3 · replayRetryBaseMs 1500 · keepAliveTimeoutMs 5000 · clientErrorBackstopMs 30000 · exposureLogIntervalMs 3600000 · streamKeepAliveMs 15000 · preflightHoldMs 30000 · preflightDeadEndCooldownMs 300000 · proxyKeepAliveMaxMs 55000.
  • +
  • network — transport timeouts & replays: upstreamTimeoutMs 720000 · requestWatchdogMs 2× · replayRetryMax 3 · replayRetryBaseMs 1500 · keepAliveTimeoutMs 5000 · clientErrorBackstopMs 30000 · exposureLogIntervalMs 3600000 · streamKeepAliveMs 15000 · preflightHoldMs 30000 · preflightDeadEndCooldownMs 300000 · proxyKeepAliveMaxMs 55000 · firstContentTimeoutMs 720000 on-by-default (#3254, file-only) · contentGapTimeoutMs inherits/720000 (#3254, file-only).
  • guardrails — pi/omp plugin tool-output guardrails: bashTimeout 60 · maxOutputBytes 50000 · repetition true.
  • persist — session persistence to disk: enabled true · zstd false · debounceMs 500 · tailTokens 16384 · epermAlertThreshold 5 · epermAlertRepeatMs 0.
  • storage — unified storage sidecar: unified legacy — file tier selects the lane-wide storage mode; env BILI_STORAGE_UNIFIED wins when set.
  • diff --git a/website/main.js b/website/main.js index c69caa5fa6..c464759823 100644 --- a/website/main.js +++ b/website/main.js @@ -109,6 +109,8 @@ "cr.lanekey": "Lane credential override (#2336): requests forwarded through this lane carry this env variable's value instead of the client's own credential — x-api-key / x-goog-api-key replaced outright, Bearer authorization replaced, injected when absent. Exactly one of apiKeyEnv / credentialRef per entry.", "cr.lanecred": "Lane credential override via the private summary-credential store (`secret:NAME` semantics); exactly one of apiKeyEnv / credentialRef per entry.", "cr.calib": "Per-route k̂ calibration policy (#3211): `disabled` suppresses the learned factor on this route; `max` clamps it (0 < max ≤ 4). Applied at publication time — the learned ring stays raw.", + "cr.fcdeadline": "Model-content deadline, phase 1 (#3254), per route and per model (`models[]`); on expiry bili aborts the stalled upstream stream and ends the turn with one clean in-band error — no internal refetch. On by default via the built-in (unset → global → 720000 builtin); explicit 0 disables for this route. Cascade model > route > global `network.*`; once warmed (≥8 healthy attempts) the resolved value acts as a ceiling tightened toward the route's measured first-content pace (clamp 3×p95, floor 120s, capped at the resolved value); file-only.", + "cr.gapdeadline": "Model-content deadline, phase 2 (#3254), per route and per model (`models[]`); bounds silence between content events after the first arrived (pings/comments never count); absent inherits the resolved `firstContentTimeoutMs`; same route-adaptive ceiling as `firstContentTimeoutMs`. File-only.", "cr.optin": "Named entries only: opt a non-http(s)-baseUrl provider into compaction ownership (pi/omp lanes); unions with env BILI_NON_HTTP_PROVIDERS.", "tg.core": "Server & core", "tg.up": "Upstream & billing", @@ -122,7 +124,7 @@ "tt.ccx": "codexCompact — intercept | pass (default intercept): handle Codex's compaction requests in bili or forward upstream — explicit forms plus the inline auto-compact transiting as an ordinary turn (prompt-form detection, #3008; overridden compact_prompt transits verbatim).", "ttl.ttl": "ccrRetrievalTtlMs — lifetime of CCR retrieval pointers (default 600000; 0 disables retrieval).", "ttl.tmp": "decompressTmpCap — size cap for temporary decompressed blocks (default 50).", - "tt.network": "network — transport timeouts & replays: upstreamTimeoutMs 720000 · requestWatchdogMs 2× · replayRetryMax 3 · replayRetryBaseMs 1500 · keepAliveTimeoutMs 5000 · clientErrorBackstopMs 30000 · exposureLogIntervalMs 3600000 · streamKeepAliveMs 15000 · preflightHoldMs 30000 · preflightDeadEndCooldownMs 300000 · proxyKeepAliveMaxMs 55000.", + "tt.network": "network — transport timeouts & replays: upstreamTimeoutMs 720000 · requestWatchdogMs 2× · replayRetryMax 3 · replayRetryBaseMs 1500 · keepAliveTimeoutMs 5000 · clientErrorBackstopMs 30000 · exposureLogIntervalMs 3600000 · streamKeepAliveMs 15000 · preflightHoldMs 30000 · preflightDeadEndCooldownMs 300000 · proxyKeepAliveMaxMs 55000 · firstContentTimeoutMs 720000 on-by-default (#3254, file-only) · contentGapTimeoutMs inherits/720000 (#3254, file-only).", "tt.guardrails": "guardrails — pi/omp plugin tool-output guardrails: bashTimeout 60 · maxOutputBytes 50000 · repetition true.", "tt.persist": "persist — session persistence to disk: enabled true · zstd false · debounceMs 500 · tailTokens 16384 · epermAlertThreshold 5 · epermAlertRepeatMs 0.", "tt.storage": "storage — unified storage sidecar: unified legacy — file tier selects the lane-wide storage mode; env BILI_STORAGE_UNIFIED wins when set.", @@ -348,6 +350,8 @@ "cr.lanekey": "通道凭据覆盖(#2336):经该通道转发的请求使用此环境变量的值取代客户端自带凭据——x-api-key / x-goog-api-key 直接替换、Bearer authorization 替换、缺失时注入。每个条目与 credentialRef 二选一。", "cr.lanecred": "经私有摘要凭据存储的通道凭据覆盖(`secret:NAME` 语义);每个条目与 apiKeyEnv 二选一。", "cr.calib": "按路由的 k̂ 校准策略(#3211):`disabled` 在该线路禁用学习系数;`max` 钳制学习系数(0 < max ≤ 4)。作用于发布环节——学习环保持原始值。", + "cr.fcdeadline": "模型内容期限,阶段一(#3254),按线路与按模型(`models[]`);到期则中止卡死的上游流并以一条干净的带内错误结束回合——不做内部盲重取。通过内置值默认开启(缺省 → global → 内置 720000);显式 0 对该线路关闭。级联 model > route > global `network.*`;线路预热后(≥8 次健康尝试)解析值转为上限,向该线路实测首内容水位收紧(clamp 3×p95,下限 120s,上限为解析值);仅文件可设。", + "cr.gapdeadline": "模型内容期限,阶段二(#3254),按线路与按模型(`models[]`);约束首个内容之后相邻内容事件间的静默(ping/注释不算内容);缺省继承解析后的 `firstContentTimeoutMs`;线路自适应上限同 `firstContentTimeoutMs`。仅文件可设。", "cr.optin": "仅命名条目:把非 http(s) baseUrl 供应商纳入压缩所有权(pi/omp 车道);与 env BILI_NON_HTTP_PROVIDERS 取并集。", "tg.core": "服务与核心", "tg.up": "上游与计费", @@ -361,7 +365,7 @@ "tt.ccx": "codexCompact —— intercept | pass(默认 intercept):在 bili 内处理 Codex 压缩请求或转给上游 —— 显式形态 + 以普通请求形态过线的内联自动压缩(prompt 文本检测,#3008;自定义 compact_prompt 按原样转发)。", "ttl.ttl": "ccrRetrievalTtlMs —— CCR 回取指针有效期(默认 600000;0 禁用回取)。", "ttl.tmp": "decompressTmpCap —— 临时展开块的大小上限(默认 50)。", - "tt.network": "network —— 传输超时与重放:upstreamTimeoutMs 720000 · requestWatchdogMs 2× · replayRetryMax 3 · replayRetryBaseMs 1500 · keepAliveTimeoutMs 5000 · clientErrorBackstopMs 30000 · exposureLogIntervalMs 3600000 · streamKeepAliveMs 15000 · preflightHoldMs 30000 · preflightDeadEndCooldownMs 300000 · proxyKeepAliveMaxMs 55000。", + "tt.network": "network —— 传输超时与重放:upstreamTimeoutMs 720000 · requestWatchdogMs 2× · replayRetryMax 3 · replayRetryBaseMs 1500 · keepAliveTimeoutMs 5000 · clientErrorBackstopMs 30000 · exposureLogIntervalMs 3600000 · streamKeepAliveMs 15000 · preflightHoldMs 30000 · preflightDeadEndCooldownMs 300000 · proxyKeepAliveMaxMs 55000 · firstContentTimeoutMs 720000 默认开启(#3254,仅文件)· contentGapTimeoutMs 继承/默认 720000(#3254,仅文件)。", "tt.guardrails": "guardrails —— pi/omp 插件工具输出守卫:bashTimeout 60 · maxOutputBytes 50000 · repetition true。", "tt.persist": "persist —— 会话落盘持久化:enabled true · zstd false · debounceMs 500 · tailTokens 16384 · epermAlertThreshold 5 · epermAlertRepeatMs 0。", "tt.storage": "storage —— 统一存储边车:unified legacy —— 文件层选定全 lane 存储模式;设置了 BILI_STORAGE_UNIFIED 时环境变量优先。",