Repository navigation
[PR] Upstream hang on compressed summary no longer silent: added first-byte budget + watchdog/preflight logging (#5904) | 压缩摘要上游卡死不再静默:新增首字节预算+看门狗/预检日志(#5904) - #5917
Conversation
…tchdog/preflight logging The release-gate Lane A triple-red root cause: an upstream (GPU-restart-cold sglang via relay) accepted the proxy's compress-summary request and never answered. Every existing budget was mid-stream: the 12-min idle watchdog only re-arms AFTER the first body byte, so a zero-byte wedge sat silent until the client's 300s abort — no retry, no error, no log line. Three changes, defaults byte-identical to current behavior: 1. src/fetch-util.ts two-phase watchdog: phase 1 (request sent -> fetch resolves, empirically around the first body data) gets its own budget min(timeoutMs, upstreamFirstByteMs()); phase 2 re-arms to the full idle budget, preserving the #1706/#1714 guarantee that long silent prefills/ thinking phases are never truncated. 2. src/knobs.ts + src/config.ts: BILI_UPSTREAM_FIRST_BYTE_MS / network.upstreamFirstByteMs, default = the full 720000ms idle budget (opt-in tightening only). Documented in CONFIGURATION(.zh-CN).md + website/config-reference/process.yaml (config-surface-guard green). 3. Watchdog + preflight logging: the idle watchdog historically fired SILENTLY — now every trip logs phase + budget + method + URL; preflight logs each summary attempt sent/headers/permanent-fail with timings, so a wedged attempt is visible in bili.log immediately. scripts/release-gate-local.mjs sets BILI_UPSTREAM_FIRST_BYTE_MS=120000 for gate lanes: 1.6x the slowest legitimate observed first byte (73s, cold GPU), two attempts fit the 300s turn so the wedge surfaces as a real error. Tests: tests/issue5904-upstream-first-byte.test.ts (4 cases: knob resolution, phase-1 wedge abort, phase-2 slow-gap non-truncation, explicit timeout cap); regression suites fix-551/fix-437/fix-15/fix-78/issue2652/issue3148 (27), external-summary-* (104), knobs/config (44) all green; tsc clean.
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.8606-pr.5917.da76a7aThis exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr5917.tgz
npm install -g packageInstall instructions are refreshed on each push. |
…dger entry docs-sync and lifecycle-ledger-guard reds on #5917: the new network.upstreamFirstByteMs / BILI_UPSTREAM_FIRST_BYTE_MS knob needs its generated artifacts (CONFIGURATION tables, website i18n tt.network summary) regenerated via tools/gen-config-docs.mjs, and the env var registered in LIFECYCLE.md §2.3 (permanent network-tuning family, default = full idle budget, release gate sets 120000).
Review findings on PR #5917 (comments/docs contradicted the code): - src/config.ts: comment claimed default 120000; actual default is UPSTREAM_FIRST_BYTE_MS = UPSTREAM_TIMEOUT_MS = 720000 - src/knobs.ts + src/fetch-util.ts: 'release gate sets 45s' was stale — scripts/release-gate-local.mjs sets BILI_UPSTREAM_FIRST_BYTE_MS=120000 - src/knobs.ts: garbled '(120000s-free default ...)' phrasing cleaned up - CONFIGURATION.md + website/config-reference/process.yaml: duplicated phrase 'to the client's abort' (en); regenerated from the seed Comment/doc-only change; no behavior change.
(ai: vllm-qwen/qwen3.8-27b) 查重:不重复 —— 按「首字节预算 / 看门狗静默卡死 / preflight 摘要请求」主题检索了 open+closed issue 与在途 [PR] 清单,无其他同机制条目;本线程即 GitHub #5904 的唯一修复 PR(#5917),无并列第二 PR。 标签:bug + scope/wiring + p2(一句话理由:机制在代理请求管道的 fetch 看门狗,属协议接线面;缺陷仅在「上游接收后永不应答」时触发、有绕行(重启/收紧配置),不到 p1)。 实况(刚验证):PR open,mergeable 无冲突——分支下垫的 4 个非 #5904 提交已随 #5905 进 master,对最新 master 的真实 diff 是干净的 12 文件,不含任何 kernel/ 改动(PR 正文的 审查中发现的小问题已直接修掉(独立标注 commit
一个需要你拍板的点(最终八问里展开):旋钮默认值钉死在常量 720000,而不是跟随用户配置的 |
(ai: vllm-qwen/qwen3.8-27b) 审查结论 — 结构化八问(PR #5917,head 现为
测试有效性:新增 4 例是真测试——断言有牙(阶段一卡死 <5s 快速失败 vs 12 分钟旧预算、旋钮解析含非法值回落、负向不误杀、显式 timeoutMs 封顶),gate 双向纪律齐;作者报告的回归面(fix-551/fix-437/fix-15/fix-78/issue2652/issue3148、external-summary、knobs/config)合理覆盖触碰路径,全量裁决以远程 CI 为准。 小问题已直接修掉(commit CI 实况(本回复时刻):静态门禁全绿;test 矩阵仍在跑(12 shard in progress,目前零失败)。合并请等其转绿。 🔶 需要决策
收工判定:机制审查通过、测试有牙、小问题已修;等你拍板默认语义(推荐 A);选 B 或接受现状则 CI 绿即可合并。 |
…mTimeoutMs Review caught a real regression in the first cut: the unset fallback was pinned to the UPSTREAM_FIRST_BYTE_MS constant (720000), so deployments that raise BILI_UPSTREAM_TIMEOUT_MS above 12 min (documented: >12-min local-model prefills) would have had their first-byte budget silently re-capped at 12 min — breaking this PR's own 'unset = byte-identical' claim for exactly the #1706/#1714 population it promises to protect. - src/knobs.ts upstreamFirstByteMs(): fallback is now upstreamTimeoutMs() (the resolved idle budget) — unset holds for EVERY configuration. - src/fetch-util.ts: drop the UPSTREAM_FIRST_BYTE_MS constant (would be an unused export); note documents why no default constant exists here. - tests: the knob-resolution case now also pins BILI_UPSTREAM_TIMEOUT_MS= 3600000 → first-byte follows 3600000, explicit knob still wins, garbage falls back to the RESOLVED budget. - docs regenerated: default '= upstreamTimeoutMs (resolved; 720000 when unset)', desc explains the follow semantics. Adopts option A from the review decision point.
|
评审提到的回归确认属实,已修:采纳选项 A(commit 987de7c)。 确认的回归:首版把未设旋钮的回落钉死在常量 720000——把 修法:
验证:本测试 4/4、fix-551/fix-437/fix-15/config-surface-guard/knobs-precedence 30/30、tsc 干净、lifecycle-ledger OK;等 CI 全绿。 另外回应一下评审第 4 点里沿用的旧说法:看门狗中止按既有语义( |
(ai: vllm-qwen/qwen3.8-27b) 选项 A 修复已逐项核验通过(commit
本地定向验证:在 重试语义更正:核实成立,接受更正。 增量八问(仅覆盖本提交 diff):
状态更新: |
…-byte-budget # Conflicts: # CONFIGURATION.md # CONFIGURATION.zh-CN.md # LIFECYCLE.md
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.1087-master.3ac3e95 # this exact build
npm install -g billion-context@master # always the newest merged build
|
…TS.md and the ranxianglei#5904 first-byte budget Upstream delta 48ee6a7..3ac3e95 (83 files, 20 commits). The substantive ones: - e3434a3 fix(ranxianglei#5904): phase-1 first-byte budget for upstream summary calls + watchdog/preflight logging - b4a4026 docs(ranxianglei#5904): regenerate config artifacts for upstreamFirstByteMs + ledger entry - 9492212 docs(ranxianglei#5904): align first-byte-knob comments and docs with shipped values - c7db020 fix(opencode): route host console chatter to bili.log and teach the config key (PR ranxianglei#5710, closes ranxianglei#5706) - 984d3dc fix(gate): opencode lane one-time warmup round for fresh storage dirs (PR ranxianglei#3497) - 8f1fca8 docs: live architecture hub + module pages + file-map CI guard - 2a15b5a docs: all 8 architecture module pages live - 44faff0 docs: slim AGENTS.md (KDD text to reference/design-decisions.md, triage table to reference/triage.md) - 6f4f248 docs: add docs/README navigation landing page, en+zh - 3ac3e95 Merge pull request ranxianglei#5917 Conflict surface: only the four generated artifacts, all auto-merged. The lane's chat row is still in the regenerated docs/configuration.md (line 294) after `node tools/gen-config-docs.mjs generate`, and the docs fixed point holds. kernel stayed at 0.0.187 and was rebuilt before measuring. Lane anchors intact: src/server/relay.ts:2070 still calls `pipePluginJson(toClient, res, undefined, prepared.protocol, ...)`, src/storage/mysql.ts:77 still exports MYSQL_SCHEMA_VERSION, symbols unchanged (stripDriftResidue 6, isDraftDump 6, containsRenderTagText 21, mayStartDegenerateRenderTag 6, DRIFT_BARE_MISMATCH 3, DRAFT_CLOSE_TAIL 3). Gates: tsc 0, four guards 0, gen-config-docs fixed point. Tests: issue5175 58/0, tag-echo 79/0, issue2190-degen-close 32/0, golden-storage-render 5/0, storage-engine-proofs 12/0, native-compact-terminal 1/0, issue4666 2/0, and the three new upstream suites: issue5904-upstream-first-byte 4/0, issue5706-opencode-host-console 2/0, issue1815-unknown-top-level-keys 5/0.
…rub, x.ai WS recovery Upstream delta fe963d7..f1a4af4 (15 files, 20 commits): - 96fc75d fix: give a tightened first-byte timeout its own replayable failure kind `upstream-headers-timeout`, replayed in-proxy under BILI_REPLAY_RETRY_MAX (ranxianglei#5807/ranxianglei#5831, ranxianglei#5917 review); 16b149e pins the classification; 3a52fdbfd corrects the split-condition wording - 907fcea fix(log-mask): scrub JSON-quoted credentials and Groq/Hugging Face keys - 0c213c0 fix: recover x.ai Responses WS continuations rejected without a type - 204f79f fix: hide the host composer seat while the Context tab is open (ranxianglei#4406) - e7b0cee / 6436b23: the two golden-realign commits are ALREADY ancestors of this branch (they landed on the lane earlier via ci-fixes), so the only golden movement here is none; both verified with merge-base --is-ancestor. Three files overlap and all three auto-merged: src/loop/core.ts (upstream adds one TRANSPORT_FAILURE_LABELS row for upstream-headers-timeout; the lane's isDraftDump gate is elsewhere in the file), docs/configuration.md and docs/configuration.zh-CN.md (regenerated from the seed, fixed point holds), plus website/main.js and website/docs/index.html regenerated too. kernel stayed at 0.0.187 and was rebuilt before measuring. Lane anchors intact: src/server/relay.ts:2097 still calls `pipePluginJson(toClient, res, undefined, prepared.protocol, ...)`, src/storage/mysql.ts:77 still exports MYSQL_SCHEMA_VERSION, symbols unchanged (stripDriftResidue 6, isDraftDump 6, containsRenderTagText 21, mayStartDegenerateRenderTag 6, DRIFT_BARE_MISMATCH 3, DRAFT_CLOSE_TAIL 3). Gates: tsc 0, unused-exports 0, config-surface 0, lifecycle-ledger 0, test-import-graph 0, gen-config-docs fixed point. Tests: issue5175 58/0, tag-echo 79/0, issue2190-degen-close 32/0, golden-storage-render 5/0, storage-engine-proofs 12/0, native-compact-terminal 1/0, issue4666 2/0; plus the upstream suites this delta touches: issue5807-first-byte-replay 5/0, log-mask 26/0, xai-responses-ws-error-envelope 2/0, dsh-settings-ui 27/0 (the 7 failures seen without stripping the live host's BILLION_CONTEXT_ATTACH/DSH_* env are a sandbox artifact, not a regression: pristine master shows the same 20/7, and the same tree is 26/0 clean).
…, PR ranxianglei#5177) Resolves the dirty state against current master (e7b3d28): - src/upstream-fail.ts: keep master's new upstream-headers-timeout kind (ranxianglei#5807/ranxianglei#5917) AND apply the PR's neutral wording to the upstream-timeout docstring + hint (this category alone does not prove idle expiry). - CONFIGURATION.md / CONFIGURATION.zh-CN.md: adopt master's redirect stub (content moved to docs/configuration*.md during docs centralization). - Port the PR's three doc edits (upstream-timeout table row, masked causes= diagnostics paragraph, four-step checklist step 1) into docs/configuration.md / docs/configuration.zh-CN.md where they now live. Verified: typecheck clean; 57/57 targeted timeout/failure diagnostic tests pass.
这是啥问题?
代理向模型发的压缩摘要请求,上游(冷启动的 sglang 中继)接收了请求却永不应答。代理所有现有时长预算都是「流中间」的:12 分钟空闲看门狗只在收到首个字节之后才挂上,所以零字节的卡死会一直静默挂着,直到客户端(gate,300 秒轮次预算)先超时中止——不重试、不报错、日志里一行痕迹都没有。这是发布 gate Lane A 三连红的根因(取证:t2 到达 00:32:36.702,preflight fold 00:32:36.704,之后 300s 零响应、零日志、连接零断开)。
根本原因
src/fetch-util.ts单阶段看门狗:计时器只覆盖「发出→首字节之后」的流中间静默;AbortController触发时无任何日志。src/preflight.tsrequestSummaryBody 每次尝试发出/收到响应均无进度日志,永久失败也不记 error。如何解决
src/fetch-util.ts):阶段一(请求发出→fetch resolve,实测 undici 在首个 body 数据时才 resolve)使用独立预算min(timeoutMs, upstreamFirstByteMs());阶段二重挂到完整空闲预算——保住 Opencode failed to send message with error: upstream stream ended before a completion event; this turn may be incomplete #1706/配置面收敛:移除全部 BILI_*/ACP_* 行为类环境变量,统一迁入 config.json #1714 的保证:长静默预填充/思考永不误杀。src/knobs.ts/src/config.ts):BILI_UPSTREAM_FIRST_BYTE_MS/network.upstreamFirstByteMs,默认 = 720000ms(完整空闲预算)——默认行为与现状逐字节一致,只有显式收紧的部署才变(比如 300s prefill 的本地模型部署:不设置就完全不受影响)。文档已进 CONFIGURATION.md / CONFIGURATION.zh-CN.md / website/config-reference/process.yaml(config-surface-guard 绿)。scripts/release-gate-local.mjs):设BILI_UPSTREAM_FIRST_BYTE_MS=120000——是本环境观测到的最慢合法首字节(冷 GPU 上 73s)的 1.6 倍。按既有语义,看门狗中止不重试、直接永久失败(src/preflight.tsSummaryTransportErrorretryable = !aborted):120s 无首字节即以带原因的真实错误结束该轮,gate 快速红且可诊断,而不是静默拖到 300s 客户端中止。测试
tests/issue5904-upstream-first-byte.test.ts4 用例全绿:旋钮解析、阶段一卡死快速失败、阶段二慢 chunk 间隙不误杀、显式 timeoutMs 封顶。tsc --noEmit干净。Fixes #5904
kernel-change: src/fetch-util.ts src/knobs.ts src/config.ts src/preflight.ts