Repository navigation
fix: warn when upstream accepts a request then stalls before first byte (#3434) - #3441
Conversation
…te (#3434) Between `forward …` and the outcome (client disconnect "0 bytes written" or the idle-watchdog abort) bili logged nothing when the upstream accepted the request and then sent no bytes, so an upstream stall read as proxy-side silence. Arm a one-shot first-byte watchdog around the initial fetch in relay.forward: if response headers have not arrived after 30s, log ONE warn naming method + masked target + lane. Deduped once per origin per process (windowSourceLogged pattern) so a long-prefill local model logs at most one line, not one per slow request. Cleared on fetch settle or client abort; fires only while pending. Additive logging only — deliberately NO fast-fail (would truncate legitimate long prefills, cf. #1706/#1714) and NO new config surface (code constant + _setForTest seam).
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.4053-pr.3441.2289fadThis exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr3441.tgz
npm install -g packageInstall instructions are refreshed on each push. |
(ai: vllm-qwen/qwen3.8-27b) 查重: 不重复。独立检索 open+closed issues/PRs(first-byte / stall / upstream-silence / awaiting / timeout-visibility 等失败签名与标题关键词),无同症状同机制的既有条目;#3434 是本 PR 的需求来源(可见性缺口),#3396 是该缺口的受害者而非报告——两者均非重复工作项。 PR #3441 审核中(独立审 diff + 既有行为清点 + 测试有效性,不照抄 issue 侧结论)。当前 CI 状态:静态门禁全绿(kernel-guard / config-surface-guard / version-guard / docs-coverage / lane-guard 等),6 个 test shard in_progress。完整八问结论稍后在本线程给出。 |
(ai: vllm-qwen/qwen3.8-27b) PR #3441 独立审核完成:逐行读了 diff、清点触碰路径的既有行为、验证测试有效性,本地跑了定向门禁(typecheck + 回归套件),未照抄 issue 侧结论。八问如下:
独立验证证据(非复述 PR 描述):
判定:可合并——方向正确、零回归、代码质量达标、测试钉住行为;待剩余 3 个 test shard 转绿即可由人工合入(我无权合并)。 |
(ai: vllm-qwen/qwen3.8-27b) CI 补充结论(补上一轮审核的 Q5 回归判定与合并判定): 全部 6 个测试分片已跑完:5 绿、1 红( 红分片性质(不是断言失败):
排除链(为何不是本 PR 引入):
已做动作:
判定不变:本 PR 自身零回归(上轮八问结论维持)。红分片属环境 flake,可合并——重跑转绿即可直接人工合入;若重跑再红,则是 #3356 家族第三次观测,届时建议优先修 win22 harness 本身,而不是继续让无关 PR 陪跑。 备注:排查过程中我的沙箱本地磁盘写满(ENOSPC),shell/文件工具暂不可用,只能走网络通道;重跑结果确认待环境恢复后补交。 |
(ai: vllm-qwen/qwen3.8-27b) 收尾更新(补上条备注里的待办):已触发的重跑转绿—— 最终判定维持:可合并。红分片确认为 #3356 家族的 win22 负载 flake(第二次观测,证据链在 #3356),与本 PR 零关联;本 PR 自身零回归、测试有牙、单点覆盖全 lane。请人工合入。 另:沙箱磁盘问题已自行清理恢复(删掉了自己验证时装的 node_modules 与临时日志,释放约 350MB);整盘写满的主因是共享工作区目录总量过大(fleet 级),非本会话造成,留作运维参考。 |
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.521-master.5fc5fea # this exact build
npm install -g billion-context@master # always the newest merged build
|
What & why
#3434: when the upstream accepts a forwarded request and then sends no bytes, bili logged nothing between
forward …and the eventual outcome (client disconnected mid-stream after Nms (0 bytes written to client), or the 12-minute idle-watchdog abort). From the log alone you could not tell whether the stall was upstream-side or inside the proxy. This is the exact visibility gap that sent #3396 down a day-long misattribution (the side lane looked guilty while the stall was intermittent upstream/egress silence).Change
Additive logging only — no behavior change. In
relay.forward, arm a one-shot first-byte watchdog around the initial fetch (fetchWithTransportRetry): if response headers have not arrived after 30 s, log ONE warn naming method + masked target + lane:web_searchon DSH never completes: theside request (server-tool utility call)lane forwards upstream but writes 0 bytes back to the client #3396 is included.windowSourceLoggedpattern) so a long-prefill local model logs at most one line, not one per slow request. The origin is claimed at fire time (not arm time) so a fast request never suppresses a later genuine stall on the same origin.unref()'d so a pending watchdog cannot hold the event loop open at shutdown.Deliberately NOT doing
No fast-fail (e.g. returning 504 after N seconds of upstream silence): that is a new default/behavior and would truncate legitimate local-model long prefills (#1706/#1714 retired exactly such guards on purpose). If ever wanted, it is owner-decision territory (possibly scoped to the side/utility lane only).
Config surface
None. The 30 s threshold is a code constant (
UPSTREAM_FIRST_BYTE_WARN_MS) with a_setFirstByteWarnMsForTestseam — no new env var and no new config key.Behavior-change disclosure
None. The request/response flow is untouched; the watchdog only appends a warn line. On rejection the error still flows into the existing outer
catch; thetry/finallyaround the awaited fetch adds no semantic change.Verification
npm run typecheckclean;npm run buildsuccess;check:unused-exportsclean (on the rebased head).tests/issue3434-first-byte-watchdog.test.ts— 8 cases: fires-once-on-stall, no-warn-on-settle, per-origin dedupe, cross-origin independence, abort-before-threshold, already-aborted disarm, disabled-threshold disarm, reset seam. All pass locally. Full-suite verdict = remote CI on this head.Closes #3434