Skip to content

[PR] Fix #5807: Classify TTFB timeout separately on gateway hang and replay within the proxy | 修 #5807:网关装死时,首字节超时单独分类并在代理内重放 - #5947

Merged
ranxianglei merged 3 commits into
masterfrom
2026-11-08_headers-timeout-replay
Nov 8, 2026
Merged

ranxianglei merged 3 commits into
masterfrom
2026-11-08_headers-timeout-replay

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

这是什么问题

上游网关"连着但装死"——TCP 活着,一个字节都不回。之前 bili 只能等到底层把 socket 判死(实测 35.3s,最长 12 分钟),然后整步只失败一次、不重试,宿主被迫重传整个 51 MiB prompt。

根因:首字节前的超时和流中途的超时任意一个,都归到同一个失败类 upstream-timeout,而这个类刻意不重试(怕流中途重放导致重复输出)。但两个阶段语义完全相反——首字节前 0 字节,重放无副作用、可以重试;流中途已有输出,不能重放。分不清阶段,就既不敢给首字节配短预算、也不敢让超时可重试。

#5917 已把预算拆成两阶段(新增 BILI_UPSTREAM_FIRST_BYTE_MS)。本 PR 补上另一半:分类 + 代理内重放。

改了什么

明确不包含

验证

  • 新套件 tests/issue5807-first-byte-replay.test.ts 5/5:分类单测(标记→新类/裸 AbortError→upstream-timeout/UND_ERR_BODY|HEADERS_TIMEOUT→upstream-timeout)+ 本地静默 HTTP server e2e(收紧预算→代理内重放 3 次;默认路径无标记单次;body 阶段无标记不重放)。
  • 回归批 83/83(upstream-fail、fix-551/437/78、issue5904、issue3434、main-path-transport-retry、preflight-transport-recovery、issue4254、upstream-alerts、issue1453、issue4050、upstream-truncation)。
  • 负向对照:去掉打标行 → 2 个标记测试失败("rejection must carry the first-byte marker" / "final error keeps the marker kind"),恢复后全绿。
  • npm run typecheck 通过;config-surface-guard 通过;gen-config-docs i18n EN/ZH 对齐。

Closes #5807

A first-byte (pre-first-byte) timeout was indistinguishable from a mid-stream
idle timeout - both surfaced as a bare AbortError -> upstream-timeout, which
is deliberately NOT retried. So a wedged upstream that accepts a request but
never answers cost the host a 35s+/12min single-shot stall with no in-proxy
replay (#5807/#5831).

When the operator tightens the first-byte budget below the idle budget
(network.upstreamFirstByteMs / BILI_UPSTREAM_FIRST_BYTE_MS), fetchWithTimeout
now stamps the rejection with UPSTREAM_FIRST_BYTE_TIMEOUT_CODE; the
classifier returns the new fail-fast kind upstream-headers-timeout, so the
main transport path (fetchWithTransportRetry / fetchWithRetry) replays it
within the existing BILI_REPLAY_RETRY_MAX budget. Safe because zero response
bytes were received - a replay cannot double-deliver.

Unset knob => the marker is never stamped => upstream-timeout, non-replayed,
byte-identical to the pre-change default (no 3x12min stacking on dead
endpoints). Mid-stream (body) idle timeouts stay upstream-timeout. Docs
regenerated from the YAML seed.
Unit: the marker classifies to upstream-headers-timeout (fail-fast); a plain
abort and undici headers/body timeouts stay upstream-timeout (non-replayed).
E2E against a local silent HTTP server: a tightened first-byte budget is
replayed in-proxy within the bounded budget; the default (non-split) path
stays unmarked and single-attempt; a body-phase timeout under a split stays
unmarked and non-replayed (#1706/#1714).
@ranxianglei ranxianglei changed the title 修 #5807:网关装死时,首字节超时单独分类并在代理内重放 [PR] Fix #5807: Classify TTFB timeout separately on gateway hang and replay within the proxy | 修 #5807:网关装死时,首字节超时单独分类并在代理内重放 Nov 8, 2026
@github-actions

github-actions Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-11-08_headers-timeout-replay (3a52fdb)

Latest build — install this exact version

npm install -g billion-context@0.0.8634-pr.5947.1fc40a8

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
3a52fdb billion-context@0.0.8634-pr.5947.1fc40a8 build
16b149e billion-context@0.0.8628-pr.5947.e8a19ef build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr5947.tgz
npm install -g package

Install instructions are refreshed on each push.

ranxianglei pushed a commit that referenced this pull request Nov 8, 2026
…un-export internal gate types

- website/config-reference/compress.yaml: desc_en/desc_zh/site_en/site_zh for
  compress.requireNudge now describe the 20-model-round nudge credit window
  (#5943) instead of the retired current-turn arm — the PR hand-edited the
  generated docs/configuration.{md,zh-CN.md} without updating the single
  source of truth, which is what the #2107 docs-sync drift guard caught.
  Regenerated website/main.js + website/docs/index.html from the updated seed.
- src/unprompted-gate.ts: UnpromptedGate / UnpromptedGateStamp are referenced
  only inside unprompted-gate.ts — drop the leading export per the unused-exports
  gate (#1440 P6); UnpromptedGateState stays exported (imported by session.ts).
The parenthetical described the trigger backwards: phase 1 splits whenever
the first-byte budget source (the knob when set, else the resolved idle) is
strictly tighter than the effective full budget — including a caller timeout
LONGER than that source, not a short one. Comment-only; no behavior change.
(Review supplement for PR #5947.)
@ranxianglei ranxianglei added bug Something isn't working p2 scope/wiring labels Nov 8, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重:不重复。 #5947 是 #5807 的唯一实现 PR;相关条目为 #5917(两阶段预算拆分,已合入 master @ 3ac3e95)与 #5831(形状决策,排除了死端点上 3×12 分钟叠加)。本线程无其他在途 [PR],一线程一 PR 满足。

审核结论:可合入。 三步 Triage 均通过——复现路径经代码读证、CI 全绿、方案打到根因而非症状:

回归清单(diff 触碰路径的既有行为):默认路径零变化(用例分析 + 测试 4 钉住「显式短超时」变体);body 阶段行为不变(测试 5 钉住,elapsed≥1500ms 证明是 body 预算触发);其他失败类的重试语义不变;wire/持久化/配置 schema/kernel 均未触碰;两个穷举 Record(TRANSPORT_FAILURE_LABELS / UPSTREAM_FAIL_HINTS)按 typecheck 强制补齐新类条目。未发现回归。

顺手小修(已直接提交):src/fetch-util.ts:208 注释里分裂条件的措辞方向写反了("an explicit short caller timeout"——实际触发条件是调用方超时长于首字节预算来源,短超时反而使 effective=T 永不分裂)。已在 commit 3a52fdbf55 修正,纯注释、零行为变化,该 commit 触发的新一轮 CI 进行中。另记一处文档级 nuance(不阻塞):若运维把 BILI_UPSTREAM_TIMEOUT_MS 降到低于某调用点的显式常量(如 threads-lane 的 120s),即使不设首字节 knob 标记也会产生——行为本身合理(phase 1 确实更紧),但 PR 正文「不设 knob 则标记永不产生」的表述略绝对,知会即可。

标签:已按 PR 自身 diff 打上 bug + p2 + scope/wiring(opt-in 缓解 p2 级卡顿;默认行为零变化;改动位于代理↔上游传输边界)。

八问:

  1. 是什么问题:上游网关「连着但装死」(TCP 活着不吐字节)时,bili 要等到底层 12 分钟空闲预算才判死,且整步只失败一次不重试,宿主被迫重传整个大包(实测 35.3s–12min、51 MiB prompt)。
  2. 什么原因导致的:首字节前超时与流中途超时被归入同一个刻意不重试的失败类 upstream-timeout;两阶段语义相反(0 字节可安全重放 vs 已有输出不可重放),不分阶段就既不敢给首字节配短预算、也不敢让超时可重试。
  3. 影响范围与严重程度:仅命中「接受连接但不回任何字节」的上游的请求;p2——能用但体验差(长时间挂起 + 整包重传),且既有手工绕行(fail-fast+15s)会杀死健康流([bug][fetch] Upstream TTFB idle inherits the 12-min default → hangs with a pointless 35 s wait; suggest splitting header | [bug][fetch] 上游首字节空闲沿用 12 分钟默认值 → 卡死白等 35 s;建议拆分 headers/body 预算并让 header 超时可重放 #5807 实证),没有安全的绕行方式。
  4. 怎么修复的:当运维把首字节预算收紧到低于空闲预算时,给首字节超时打专属标记 → 分类器返回新类 upstream-headers-timeout → 进入 fail-fast 集合,主传输路径在既有重放预算内代理内重放;安全前提是首字节前 0 字节到达,重放无副作用。
  5. 是否有回归问题:无——默认路径逐字节不变、body 阶段不变、告警集合不变、wire/持久化/kernel 未触碰(详见上方回归清单)。
  6. 是否破坏缓存:不涉及——不触及缓存/增量游标/去重键;重放只发生在「0 字节到达」的失败上。
  7. 是否改了设计方向:否——[PR] Upstream hang on compressed summary no longer silent: added first-byte budget + watchdog/preflight logging (#5904) | 压缩摘要上游卡死不再静默:新增首字节预算+看门狗/预检日志(#5904) #5917 两阶段预算的延续,补上「分类+重放」缺失的一半,无架构转向。
  8. 是否引入了新的配置:否——复用既有 network.upstreamFirstByteMs / BILI_UPSTREAM_FIRST_BYTE_MS([PR] Upstream hang on compressed summary no longer silent: added first-byte budget + watchdog/preflight logging (#5904) | 压缩摘要上游卡死不再静默:新增首字节预算+看门狗/预检日志(#5904) #5917 已引入),本次仅更新文档描述文本。

判定:可合入,无阻塞项。合并留给 owner(agent 不合 PR)。

@ranxianglei
ranxianglei merged commit 47db161 into master Nov 8, 2026
92 of 134 checks passed
@github-actions

github-actions Bot commented Nov 8, 2026

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.1092-master.47db161 (rolling master tag).

npm install -g billion-context@0.0.1092-master.47db161  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

ranxianglei pushed a commit that referenced this pull request Nov 9, 2026
…un-export internal gate types

- website/config-reference/compress.yaml: desc_en/desc_zh/site_en/site_zh for
  compress.requireNudge now describe the 20-model-round nudge credit window
  (#5943) instead of the retired current-turn arm — the PR hand-edited the
  generated docs/configuration.{md,zh-CN.md} without updating the single
  source of truth, which is what the #2107 docs-sync drift guard caught.
  Regenerated website/main.js + website/docs/index.html from the updated seed.
- src/unprompted-gate.ts: UnpromptedGate / UnpromptedGateStamp are referenced
  only inside unprompted-gate.ts — drop the leading export per the unused-exports
  gate (#1440 P6); UnpromptedGateState stays exported (imported by session.ts).
ranxianglei pushed a commit that referenced this pull request Nov 9, 2026
…un-export internal gate types

- website/config-reference/compress.yaml: desc_en/desc_zh/site_en/site_zh for
  compress.requireNudge now describe the 20-model-round nudge credit window
  (#5943) instead of the retired current-turn arm — the PR hand-edited the
  generated docs/configuration.{md,zh-CN.md} without updating the single
  source of truth, which is what the #2107 docs-sync drift guard caught.
  Regenerated website/main.js + website/docs/index.html from the updated seed.
- src/unprompted-gate.ts: UnpromptedGate / UnpromptedGateStamp are referenced
  only inside unprompted-gate.ts — drop the leading export per the unused-exports
  gate (#1440 P6); UnpromptedGateState stays exported (imported by session.ts).
@github-actions github-actions Bot mentioned this pull request Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working p2 scope/wiring

Projects

None yet

1 participant