Repository navigation
[PR] Fix #5807: Classify TTFB timeout separately on gateway hang and replay within the proxy | 修 #5807:网关装死时,首字节超时单独分类并在代理内重放 - #5947
Conversation
A first-byte (pre-first-byte) timeout was indistinguishable from a mid-stream idle timeout - both surfaced as a bare AbortError -> upstream-timeout, which is deliberately NOT retried. So a wedged upstream that accepts a request but never answers cost the host a 35s+/12min single-shot stall with no in-proxy replay (#5807/#5831). When the operator tightens the first-byte budget below the idle budget (network.upstreamFirstByteMs / BILI_UPSTREAM_FIRST_BYTE_MS), fetchWithTimeout now stamps the rejection with UPSTREAM_FIRST_BYTE_TIMEOUT_CODE; the classifier returns the new fail-fast kind upstream-headers-timeout, so the main transport path (fetchWithTransportRetry / fetchWithRetry) replays it within the existing BILI_REPLAY_RETRY_MAX budget. Safe because zero response bytes were received - a replay cannot double-deliver. Unset knob => the marker is never stamped => upstream-timeout, non-replayed, byte-identical to the pre-change default (no 3x12min stacking on dead endpoints). Mid-stream (body) idle timeouts stay upstream-timeout. Docs regenerated from the YAML seed.
Unit: the marker classifies to upstream-headers-timeout (fail-fast); a plain abort and undici headers/body timeouts stay upstream-timeout (non-replayed). E2E against a local silent HTTP server: a tightened first-byte budget is replayed in-proxy within the bounded budget; the default (non-split) path stays unmarked and single-attempt; a body-phase timeout under a split stays unmarked and non-replayed (#1706/#1714).
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.8634-pr.5947.1fc40a8This exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr5947.tgz
npm install -g packageInstall instructions are refreshed on each push. |
…un-export internal gate types - website/config-reference/compress.yaml: desc_en/desc_zh/site_en/site_zh for compress.requireNudge now describe the 20-model-round nudge credit window (#5943) instead of the retired current-turn arm — the PR hand-edited the generated docs/configuration.{md,zh-CN.md} without updating the single source of truth, which is what the #2107 docs-sync drift guard caught. Regenerated website/main.js + website/docs/index.html from the updated seed. - src/unprompted-gate.ts: UnpromptedGate / UnpromptedGateStamp are referenced only inside unprompted-gate.ts — drop the leading export per the unused-exports gate (#1440 P6); UnpromptedGateState stays exported (imported by session.ts).
The parenthetical described the trigger backwards: phase 1 splits whenever the first-byte budget source (the knob when set, else the resolved idle) is strictly tighter than the effective full budget — including a caller timeout LONGER than that source, not a short one. Comment-only; no behavior change. (Review supplement for PR #5947.)
(ai: vllm-qwen/qwen3.8-27b) 查重:不重复。 #5947 是 #5807 的唯一实现 PR;相关条目为 #5917(两阶段预算拆分,已合入 master @ 3ac3e95)与 #5831(形状决策,排除了死端点上 3×12 分钟叠加)。本线程无其他在途 [PR],一线程一 PR 满足。 审核结论:可合入。 三步 Triage 均通过——复现路径经代码读证、CI 全绿、方案打到根因而非症状:
回归清单(diff 触碰路径的既有行为):默认路径零变化(用例分析 + 测试 4 钉住「显式短超时」变体);body 阶段行为不变(测试 5 钉住,elapsed≥1500ms 证明是 body 预算触发);其他失败类的重试语义不变;wire/持久化/配置 schema/kernel 均未触碰;两个穷举 Record(TRANSPORT_FAILURE_LABELS / UPSTREAM_FAIL_HINTS)按 typecheck 强制补齐新类条目。未发现回归。 顺手小修(已直接提交):src/fetch-util.ts:208 注释里分裂条件的措辞方向写反了("an explicit short caller timeout"——实际触发条件是调用方超时长于首字节预算来源,短超时反而使 effective=T 永不分裂)。已在 commit 标签:已按 PR 自身 diff 打上 bug + p2 + scope/wiring(opt-in 缓解 p2 级卡顿;默认行为零变化;改动位于代理↔上游传输边界)。 八问:
判定:可合入,无阻塞项。合并留给 owner(agent 不合 PR)。 |
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.1092-master.47db161 # this exact build
npm install -g billion-context@master # always the newest merged build
|
…un-export internal gate types - website/config-reference/compress.yaml: desc_en/desc_zh/site_en/site_zh for compress.requireNudge now describe the 20-model-round nudge credit window (#5943) instead of the retired current-turn arm — the PR hand-edited the generated docs/configuration.{md,zh-CN.md} without updating the single source of truth, which is what the #2107 docs-sync drift guard caught. Regenerated website/main.js + website/docs/index.html from the updated seed. - src/unprompted-gate.ts: UnpromptedGate / UnpromptedGateStamp are referenced only inside unprompted-gate.ts — drop the leading export per the unused-exports gate (#1440 P6); UnpromptedGateState stays exported (imported by session.ts).
…un-export internal gate types - website/config-reference/compress.yaml: desc_en/desc_zh/site_en/site_zh for compress.requireNudge now describe the 20-model-round nudge credit window (#5943) instead of the retired current-turn arm — the PR hand-edited the generated docs/configuration.{md,zh-CN.md} without updating the single source of truth, which is what the #2107 docs-sync drift guard caught. Regenerated website/main.js + website/docs/index.html from the updated seed. - src/unprompted-gate.ts: UnpromptedGate / UnpromptedGateStamp are referenced only inside unprompted-gate.ts — drop the leading export per the unused-exports gate (#1440 P6); UnpromptedGateState stays exported (imported by session.ts).
这是什么问题
上游网关"连着但装死"——TCP 活着,一个字节都不回。之前 bili 只能等到底层把 socket 判死(实测 35.3s,最长 12 分钟),然后整步只失败一次、不重试,宿主被迫重传整个 51 MiB prompt。
根因:首字节前的超时和流中途的超时任意一个,都归到同一个失败类
upstream-timeout,而这个类刻意不重试(怕流中途重放导致重复输出)。但两个阶段语义完全相反——首字节前 0 字节,重放无副作用、可以重试;流中途已有输出,不能重放。分不清阶段,就既不敢给首字节配短预算、也不敢让超时可重试。#5917 已把预算拆成两阶段(新增
BILI_UPSTREAM_FIRST_BYTE_MS)。本 PR 补上另一半:分类 + 代理内重放。改了什么
fetchWithTimeout给首字节超时打一个专属标记(UPSTREAM_FIRST_BYTE_TIMEOUT_CODE)。upstream-headers-timeout,加入 fail-fast 集合 → 主传输路径(fetchWithTransportRetry/fetchWithRetry)在既有BILI_REPLAY_RETRY_MAX(默认 3 次)预算内代理内重放——不再让宿主重传整包。BILI_UPSTREAM_FIRST_BYTE_MS时标记永不产生 → 仍归upstream-timeout、不重试,逐字节等于改动前(也避免了在死端点上把 12 分钟叠成 3×12 分钟)。upstream-timeout、不重试(Opencode failed to send message with error: upstream stream ended before a completion event; this turn may be incomplete #1706/配置面收敛:移除全部 BILI_*/ACP_* 行为类环境变量,统一迁入 config.json #1714 长静默保证不动)。明确不包含
ALERT_KINDS)——网关装死熔断也治不好,与upstream-timeout口径一致。验证
tests/issue5807-first-byte-replay.test.ts5/5:分类单测(标记→新类/裸 AbortError→upstream-timeout/UND_ERR_BODY|HEADERS_TIMEOUT→upstream-timeout)+ 本地静默 HTTP server e2e(收紧预算→代理内重放 3 次;默认路径无标记单次;body 阶段无标记不重放)。npm run typecheck通过;config-surface-guard通过;gen-config-docsi18n EN/ZH 对齐。Closes #5807