Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 9 additions & 3 deletions CONFIGURATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,8 @@ This index is generated from `website/config-reference/*.yaml` — edit the seed
| `network.streamKeepAliveMs` | number | 15000 (0 disables) | BILI_STREAM_KEEPALIVE_MS | Downstream stream keep-alive ping interval between model chunks; 0 disables pinging. |
| `network.preflightHoldMs` | number | 30000 | BILI_PREFLIGHT_HOLD_MS | How long a failed preflight holds before the request is allowed through unreconciled. |
| `network.preflightDeadEndCooldownMs` | number | 300000 | BILI_PREFLIGHT_DEAD_END_COOLDOWN_MS | Cooldown after a dead-end preflight before retrying reconciliation for the same session. |
| `network.firstContentTimeoutMs` | number | 720000 (on by default) | — | Model-content deadline, phase 1 (#3254): max ms an upstream attempt may go with zero model content from response start before bili aborts the stalled stream and ends the turn with one clean in-band error (no internal refetch — the turn fails cleanly; clients with a retry ladder back off and retry, clients without one simply stop). On by default at 720000ms so every deployment is bounded out of the box; set a value to tighten/override, or 0 to disable (opt-out override of the built-in). Also settable per route / per model under providers; cascade model > route > global, explicit 0 wins at its level. Once a route has ≥8 healthy attempts the resolved value becomes a ceiling and the effective deadline tightens to clamp(3 × p95 of the route's recent healthy first-content durations, 120s, that resolved ceiling); stalled/killed attempts are never sampled. File-only by design. |
| `network.contentGapTimeoutMs` | number | inherits firstContentTimeoutMs (720000 default) | — | Model-content deadline, phase 2 (#3254): max ms between consecutive model-content events after the first arrived; re-armed on every content event (pings/comments never count). Absent inherits the resolved firstContentTimeoutMs; explicit 0 disables the gap phase only. Same three-level cascade, same route-adaptive ceiling (per-phase samples) and file-only rationale as firstContentTimeoutMs. |
| `network.replayRetryMax` | number | 3 (1 disables replays) | BILI_REPLAY_RETRY_MAX | Maximum transport-failure replays of an identical request before surfacing the error. |
| `network.replayRetryBaseMs` | number | 1500 (0 disables the delay) | BILI_REPLAY_RETRY_BASE_MS | Base backoff delay between replays (exponential). |
| `network.maxShrinkPerCompress` | number | unset | BILI_MAX_SHRINK_PER_COMPRESS | Cap on how much one compression may shrink history (anti-collapse guard). |
Expand Down Expand Up @@ -308,6 +310,8 @@ This index is generated from `website/config-reference/*.yaml` — edit the seed
| `apiKeyEnv` | string (env var name) | unset | — | Lane credential override (#2336): replace the client's credential with this env variable's value for this lane. Exactly one of apiKeyEnv/credentialRef; see [Lane credentials](#lane-credentials-apikeyenv--credentialref). |
| `credentialRef` | string (store name) | unset | — | Lane credential override via the private summary-credential store (secret:NAME semantics). |
| `calibration` | { disabled?: boolean, max?: number } | unset (learned k̂ applies) | — | Per-route k̂ calibration policy (#3211): opt-out or clamp for routes whose usage reports inflate; see [Route-level k̂ calibration policy](#route-level-k-calibration-policy). |
| `firstContentTimeoutMs` | number | unset (→ global, else 720000 builtin) | — | Per-route model-content deadline, phase 1 (#3254): max ms without model content from response start before bili aborts the stalled upstream and ends the turn in-band. On by default via the built-in (unset falls through to global then the 720000 builtin); set here to override per route, or 0 to disable for this route. Also settable per model under models[<m>]; cascade model > route > global network.*, explicit 0 wins at its level. Once the route has ≥8 healthy attempts the resolved value becomes a ceiling and the effective deadline tightens to clamp(3 × p95 of this route's recent healthy first-content durations, 120s, that resolved ceiling); stalled/killed attempts are never sampled. File-only (no env). |
| `contentGapTimeoutMs` | number | unset (inherits firstContentTimeoutMs) | — | Per-route model-content deadline, phase 2 (#3254): max ms between consecutive model-content events after the first arrived; re-armed on every content event. Absent inherits the resolved firstContentTimeoutMs; explicit 0 disables the gap phase only. Same cascade, same route-adaptive ceiling (its own inter-content-gap samples) and file-only rationale as firstContentTimeoutMs. |
| `compactionOptIn` | boolean | false | BILI_NON_HTTP_PROVIDERS | Named entries only: opt a non-http(s)-baseUrl provider into compaction ownership (pi/omp lanes); unions with env BILI_NON_HTTP_PROVIDERS. |

**Environment-only variables**
Expand Down Expand Up @@ -787,6 +791,8 @@ Since #2030 every pure-behavior knob has a config-file key alongside its env var
"preflightDeadEndCooldownMs": 300000, // cooldown after a dead-end preflight verdict
"proxyKeepAliveMaxMs": 55000, // upstream-proxy connection reuse cap; 0 uncapped
"postResponseLingerMs": 5000, // post-response close linger budget (#1982)
"firstContentTimeoutMs": 180000, // #3254: max ms without model content from response start; ON by default at 720000 (shown tightened here); explicit 0 disables; per-route/model overrides in providers
"contentGapTimeoutMs": 600000, // #3254: max ms between content events after the first; absent = inherits firstContentTimeoutMs (its 720000 default when unset); explicit 0 disables this phase only
"allowPrivateHttpProviders": false // opt-in: private-network plain-HTTP providers (#2984)
},

Expand Down Expand Up @@ -845,10 +851,10 @@ Since #2030 every pure-behavior knob has a config-file key alongside its env var

### `network`

- **Type:** `{ upstreamTimeoutMs?: number; requestWatchdogMs?: number; replayRetryMax?: number; replayRetryBaseMs?: number; maxShrinkPerCompress?: number; keepAliveTimeoutMs?: number; clientErrorBackstopMs?: number; exposureLogIntervalMs?: number; streamKeepAliveMs?: number; preflightHoldMs?: number; preflightDeadEndCooldownMs?: number; proxyKeepAliveMaxMs?: number; postResponseLingerMs?: number; allowPrivateHttpProviders?: boolean }`
- **Default:** `{}` (built-ins: `720000`, `2× upstream timeout`, `3`, `1500`, unset, `5000`, `30000`, `3600000`, `15000`, `30000`, `300000`, `55000`, `5000`, `false`)
- **Type:** `{ upstreamTimeoutMs?: number; requestWatchdogMs?: number; replayRetryMax?: number; replayRetryBaseMs?: number; maxShrinkPerCompress?: number; keepAliveTimeoutMs?: number; clientErrorBackstopMs?: number; exposureLogIntervalMs?: number; streamKeepAliveMs?: number; preflightHoldMs?: number; preflightDeadEndCooldownMs?: number; proxyKeepAliveMaxMs?: number; postResponseLingerMs?: number; firstContentTimeoutMs?: number; contentGapTimeoutMs?: number; allowPrivateHttpProviders?: boolean }`
- **Default:** `{}` (built-ins: `720000`, `2× upstream timeout`, `3`, `1500`, unset, `5000`, `30000`, `3600000`, `15000`, `30000`, `300000`, `55000`, `5000`, `720000`, `720000`, `false`)
- **Status:** ACTIVE
- **Description:** Process-wide transport and retry timing (ms). Each key resolves env > file > default via its twin (`BILI_UPSTREAM_TIMEOUT_MS`, `BILI_REQUEST_WATCHDOG_MS`, `BILI_REPLAY_RETRY_MAX`, `BILI_REPLAY_RETRY_BASE_MS`, `BILI_MAX_SHRINK_PER_COMPRESS`, `BILI_KEEP_ALIVE_TIMEOUT_MS`, `BILI_CLIENT_ERROR_BACKSTOP_MS`, `BILI_EXPOSURE_LOG_INTERVAL_MS`, `BILI_STREAM_KEEPALIVE_MS`, `BILI_PREFLIGHT_HOLD_MS`, `BILI_PREFLIGHT_DEAD_END_COOLDOWN_MS`, `BILI_PROXY_KEEPALIVE_MAX_MS`, `BILI_POST_RESPONSE_LINGER_MS`, `BILI_ALLOW_PRIVATE_HTTP_PROVIDERS`). Semantics: `upstreamTimeoutMs` bounds time-to-first-byte and inter-chunk silence end-to-end (#551); `requestWatchdogMs` is the total-request budget (a negative value opts out); `replayRetryMax` counts attempts after transient upstream rejections (#189/#1688); `maxShrinkPerCompress` caps how much one compress may shrink the request before bili steers toward smaller ranges (#189); `streamKeepAliveMs` emits an SSE comment when the response has written zero bytes for that long (#1647); `proxyKeepAliveMaxMs` caps connection reuse through an upstream proxy below its recycle cadence (#1263), `0` uncaps; `postResponseLingerMs` is the graceful-close budget after a proxy-initiated post-response close — the socket is held waiting for the peer's FIN/TLS close_notify and destroyed at the deadline (`reason=linger-backstop`) (#1982); `allowPrivateHttpProviders` is an opt-in (#2984): when true, plain-HTTP provider baseUrls on private networks (RFC1918, 127/8, link-local, IPv6 ULA/link-local, `*.local`/`.lan`/`.internal`) are admitted by the agent-registry intake and summary-endpoint guards that otherwise trust loopback plaintext only — public IPs/domains and CGNAT (100.64/10) stay refused regardless, and the default `false` keeps pre-#2984 behavior byte-for-byte. Full details in the [env table](#config-file-keys-for-environment-knobs-2030).
- **Description:** Process-wide transport and retry timing (ms). Every key except `firstContentTimeoutMs`/`contentGapTimeoutMs` (#3254, file-only — see below) resolves env > file > default via its twin (`BILI_UPSTREAM_TIMEOUT_MS`, `BILI_REQUEST_WATCHDOG_MS`, `BILI_REPLAY_RETRY_MAX`, `BILI_REPLAY_RETRY_BASE_MS`, `BILI_MAX_SHRINK_PER_COMPRESS`, `BILI_KEEP_ALIVE_TIMEOUT_MS`, `BILI_CLIENT_ERROR_BACKSTOP_MS`, `BILI_EXPOSURE_LOG_INTERVAL_MS`, `BILI_STREAM_KEEPALIVE_MS`, `BILI_PREFLIGHT_HOLD_MS`, `BILI_PREFLIGHT_DEAD_END_COOLDOWN_MS`, `BILI_PROXY_KEEPALIVE_MAX_MS`, `BILI_POST_RESPONSE_LINGER_MS`, `BILI_ALLOW_PRIVATE_HTTP_PROVIDERS`). Semantics: `upstreamTimeoutMs` bounds time-to-first-byte and inter-chunk silence end-to-end (#551); `requestWatchdogMs` is the total-request budget (a negative value opts out); `replayRetryMax` counts attempts after transient upstream rejections (#189/#1688); `maxShrinkPerCompress` caps how much one compress may shrink the request before bili steers toward smaller ranges (#189); `streamKeepAliveMs` emits an SSE comment when the response has written zero bytes for that long (#1647); `proxyKeepAliveMaxMs` caps connection reuse through an upstream proxy below its recycle cadence (#1263), `0` uncaps; `postResponseLingerMs` is the graceful-close budget after a proxy-initiated post-response close — the socket is held waiting for the peer's FIN/TLS close_notify and destroyed at the deadline (`reason=linger-backstop`) (#1982); `allowPrivateHttpProviders` is an opt-in (#2984): when true, plain-HTTP provider baseUrls on private networks (RFC1918, 127/8, link-local, IPv6 ULA/link-local, `*.local`/`.lan`/`.internal`) are admitted by the agent-registry intake and summary-endpoint guards that otherwise trust loopback plaintext only — public IPs/domains and CGNAT (100.64/10) stay refused regardless, and the default `false` keeps pre-#2984 behavior byte-for-byte; `firstContentTimeoutMs`/`contentGapTimeoutMs` are FILE-ONLY model-content deadlines (#3254): phase 1 bounds how long an upstream attempt may go with zero model content (text/reasoning deltas, tool calls, signatures) from response start, phase 2 (`contentGapTimeoutMs`, re-armed on every content event — pings/comments never count) bounds silence between content events after the first arrived; on expiry bili aborts the stalled upstream stream and ends the turn with one clean in-band error (context intact, resend your last message) — no internal refetch, so the turn fails cleanly: clients with a retry ladder back off and retry, clients without one simply stop; on by default at the `720000` built-in (explicit `0` at any level opts out); cascade model > provider route > global `network.*`, explicit 0 wins at its level, gap inherits the resolved initial ceiling when absent; deliberately no env layer because per-route/per-model values cannot be expressed as one process-wide variable; **route-adaptive refinement** (#3254 follow-up): once a route has accumulated ≥8 healthy attempts (completed with no truncation/error — stalled or killed attempts are never sampled), each resolved phase value becomes a ceiling and the effective deadline tightens to `clamp(3 × p95 of that route's recent healthy first-content durations, 120s, that resolved ceiling)` per phase (gap uses its own inter-content-gap samples); samples live in session metadata (persisted across restarts, ring of 32) and the fire log records `source=config|adaptive` so a fired deadline can be traced to which value class produced it. Full details in the [env table](#config-file-keys-for-environment-knobs-2030).

### `persist`

Expand Down
Loading
Loading