Skip to content

[PR] fix: terminate committed preflight responses with a protocol error frame (#3978) - #5057

Open
ranxianglei wants to merge 6 commits into
masterfrom
2026-10-27_preflight-terminal-frame
Open

ranxianglei wants to merge 6 commits into
masterfrom
2026-10-27_preflight-terminal-frame

Conversation

@ranxianglei

@ranxianglei ranxianglei commented Oct 28, 2026 •

Copy link
Copy Markdown
Owner

Follow-up to #3978 (floor 17/18 thread); complements open PR #3590. Kept SEPARATE from #3590 per the thread discussion, so it can land whether or not #3590 does.

Problem

Once the #568 keep-alive hold commits early headers (200 + SSE comment lines / JSON whitespace padding), TWO preflight no-forward exits left the client holding a 200 and an open stream that never completes:

  • the abort branch returns respond: false → the consumer writes nothing, no end();
  • the overflow-refold re-run passes respondFailFast=false → the consumer's in-band branch is suppressed entirely, and after the hold has committed, forward()'s upstream-400 relay hits ERR_HTTP_HEADERS_SENT on the committed socket.

(The other no-forward exits — probe doomed-413, generic fail-fast, post-preflight doomed-413 — are ALREADY terminated in-band on current master by the consumer at src/server/handle.ts, a #568 follow-up: protocol-native error frames after commit, plain status codes before it.)

The client cannot distinguish "bili gave up" from a dead network and has to wait out its own watchdog. Verified line-by-line at #3978 floor 17.

Change

New commitPreflightTerminalFrame(res, prepared, message, retryable, log) (src/server.ts, next to beginPreflightHold) called at all FOUR preflight no-forward exits: probe doomed-413, the generic fail-fast closure, post-preflight doomed-413, and the abort branch. When headers are already committed and the socket still lives, it terminates the response with the ESTABLISHED in-band contract — byte-identical to what the consumer already emits for committed fail-fasts (#568/#1455 protocol-native shapes, so client SDKs parse them natively):

  • SSE: delegates to emitPreflightError — per-protocol error event (anthropic event: error carrying {type:"error",error:{type:"server_error",code:"preflight_compress_failed",message}}; openai/responses/google their spec shapes);
  • non-streaming JSON: preflightFailFastBody(...) appended after the whitespace padding (still one valid JSON document).

preflightFailFastBody is extracted into src/stream-error.ts as the single source of truth, now also used by the consumer's status-code bodies in handle.ts. Uncommitted exits keep their normal status-code path byte-for-byte: the helper no-ops when !headersSent || destroyed || writableEnded, and after a successful frame the existing respond: !res.writableEnded self-corrects to false. The abort branch additionally logs its real outcome (terminal frame sent, stream ended / headers committed, terminal frame not sent (socket gone) / nothing when not committed), per request 3 at floor 17.

Behavior change (wire)

OLD (current master): the three normal-path committed fail-fast exits already terminate in-band via the consumer; the ABORT exit and the REFOLD path wrote nothing after commit (dangling stream / ERR_HTTP_HEADERS_SENT). NEW: every committed no-forward exit terminates at the exit itself with the SAME established frames — visible bytes on the normal paths are UNCHANGED (the write just moves one step earlier), and the two dangling exits now deliver the same protocol-native terminal frame + stream end instead of hanging. The abort branch gains outcome logging. No wire-shape change anywhere; no change to uncommitted paths, status codes, message wording, or forwarded traffic.

Interaction with #3590

If #3590 (wall budget) merges first, its post-budget 502 hits the same hole and can call this helper; if this lands first, #3590's stop path benefits automatically once it routes through these exits. Expected conflict surface: none — this PR touches only the four exit sites plus one helper in src/server.ts (and the shared body extractor in src/stream-error.ts), while #3590 works in the src/preflight.ts round loop.

Tests

tests/issue3978-preflight-terminal-frame.test.ts (full-proxy e2e, hermetic, ~29s):

  1. SSE positive: exhausted preflight past a shrunk grace → client receives 200 + keep-alive comment, then the protocol-native event: error frame (anthropic schema asserted), and the body COMPLETES (read raced against a 20s deadline — pre-fix it hangs);
  2. JSON positive: same scenario with stream:false → final body parses as {error:{type:"server_error",code:"preflight_compress_failed",message,retryable}};
  3. negative control: huge grace ⇒ headers never commit ⇒ plain status path unchanged, no frame;
  4. client abort after commit: abort branch logs its outcome and does not crash.

Red run (fix reverted): T1/T2 fail (stream never terminates), T3/T4 pass — the pair pins the bug while guarding the pre-existing behavior. Adjacent suites (preflight-fail-fast, preflight-client-abort, preflight-hold, anthropic-beta-window) 24/24; typecheck clean. Full-suite verdict: CI.

…ame (#3978)

Once #568 has committed early headers (200 + SSE/JSON keep-alive), every
no-forward exit left the client holding a 200 and an open stream that never
completes: respond:false writes nothing, and a later status code cannot be
written after the 200 — so clients could not distinguish 'bili gave up' from
a dead network and waited out their own watchdog (#3978 floor 17/18).

All four preflight no-forward exits now emit one terminal frame before
returning when headers are already committed: SSE gets an 'event: error'
frame, non-streaming JSON gets the error object appended after the
whitespace padding (still one valid JSON document), then end(). Body reuses
the native fail-fast shape {ok:false,error:{message,retryable}}. Uncommitted
exits keep their normal status-code path byte-for-byte, and the abort branch
logs its outcome (terminal frame sent / socket gone / not committed).
@ranxianglei ranxianglei changed the title fix: terminate committed preflight responses with a protocol error frame (#3978) [PR] fix: terminate committed preflight responses with a protocol error frame (#3978) Oct 28, 2026
@github-actions

github-actions Bot commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-27_preflight-terminal-frame (9531a8c)

Latest build — install this exact version

npm install -g billion-context@0.0.7291-pr.5057.b244881

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
9531a8c billion-context@0.0.7291-pr.5057.b244881 build
bd54c53 billion-context@0.0.7150-pr.5057.befa939 build
89e8a78 billion-context@0.0.7130-pr.5057.095168c build
02dc7a9 billion-context@0.0.6895-pr.5057.ae64cd3 build
34020b6 billion-context@0.0.6681-pr.5057.1ccdb67 build
4bcbcf9 billion-context@0.0.6663-pr.5057.d187169 build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr5057.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重: 不重复。这是 #3978(floor 17/18 线程)的独立落地 PR,与在途的 #3590 相关但机制不同——#3590 改的是 src/preflight.ts 轮询循环的 wall budget,本 PR 只动 src/server.ts 的四个 preflight no-forward 出口 + 一个 helper,两者无共享改动面(正文已声明零冲突面)。按线程讨论保持分离,可独立合入。

标签: bug + scope/wiring + p2 —— 缺陷类、请求管道/wire 接线面、能用但请求会挂到客户端自己的 watchdog(非数据丢失/缓存泄露,不升 p1)。

开始独立审核:拉分支核对代码 + 读原 issue 方向 + CI 状态,结论随后按八问给出。

@ranxianglei ranxianglei added bug Something isn't working p2 scope/wiring labels Oct 28, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

进度汇报(会话曾卡住被重启,现继续):审核进行到一半,已有关键发现——

  1. 方向核实:PR 基于当前 master(base == master)。但正文 Problem 段对现状的描述有一处过时:master 上三个常规 fail-fast 出口在头已提交时已经被消费侧在带内终结(src/server/handle.ts:2683-2701,Preflight compression holds the request with zero response bytes; client undici headersTimeout (default 300s) kills it -> repeated [preflight] summarization aborted: client disconnected 5-minute death loop, payload never forwarded #568 后续,emitPreflightError/res.end(JSON))。真正未补的洞是两个:①abort 出口(respond:false 短路,消费侧完全不写);②refold 路径(respondFailFast=false,hold 提交后 forward() 透传 400 撞 ERR_HTTP_HEADERS_SENT)。PR 对这两个洞的修复成立。
  2. 发现的契约漂移(待处理):新 helper 用的 {ok:false,error:{message,retryable}} 帧不是 bili 既有的带内错误契约——本仓纪律(Preflight compression holds the request with zero response bytes; client undici headersTimeout (default 300s) kills it -> repeated [preflight] summarization aborted: client disconnected 5-minute death loop, payload never forwarded #568/Plugin mode: upstream stream truncated before terminal event → client gets bare cut-off SSE (no completion/error event) #721/上游截断在 compress 内部重发路径无法恢复:重试闸门将 ping 计为已转发输出,且 emitError 合成 end_turn 伪完成 #1455,src/stream-error.ts 头部明文)是协议原生失败帧(preflight_compress_failed code + 分协议形状,让客户端 SDK 原生解析)。{ok:false} 只存在于 /__bili/* admin 面。照现在合入,常规已提交 fail-fast 路径的带内错误帧会被换成通用形状 = 未披露的 wire 契约变更(Anthropic 车道上 Claude Code 解析原生 event: error schema 的能力会丢)。
  3. 处置计划:按「对齐既有契约」重构 helper(SSE 走 emitPreflightError,JSON 逐字节复刻 handle.ts 现有 body),更新测试断言,定向跑测试后推一个独立标注 commit,并同步修正 PR 正文的 OLD→NEW 描述;然后按八问给最终结论。

继续干活,结论稍后。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

进度:会话连续多次被卡住重启(非阻塞性错误,疑似运行时长挂起),现切换为小步快跑继续。当前状态:

  • 对齐重构的代码改动已全部落盘(worktree .tmp/pr5057):src/stream-error.ts 新增 preflightFailFastBody()(fail-fast JSON body 单一来源);src/server/handle.ts 两处内联 body 改为引用它;src/server.ts 的 commitPreflightTerminalFrame 改为复用既有带内契约(SSE→emitPreflightError,JSON→preflightFailFastBody),四个出口调用点已传 log;测试文件头部注释与 SSE/JSON 两例断言已改为协议原生形状。
  • 依赖安装+构建已完成(EXIT=0)。
  • 剩余步骤:①补完 JSON 用例最后一处断言;②定向跑新测试+相邻套件+typecheck;③推独立标注 commit 到 PR 分支;④修正 PR 正文 OLD→NEW 描述;⑤按八问给最终结论。

继续执行,有结果即报。

Review alignment for the terminal-frame fix: the first commit introduced a
new {ok:false,error:{message,retryable}} frame for committed no-forward
exits, which would have silently replaced the in-band contract master
already emits on this path (protocol-native preflight_compress_failed
frames per #568/#1455 - client SDKs parse those natively; the {ok:false}
shape exists only on /__bili/* admin endpoints).

- commitPreflightTerminalFrame now delegates to emitPreflightError (SSE)
  and appends preflightFailFastBody (non-streaming): byte-identical to
  what the consumer's committed branch already emits, so no wire-shape
  change anywhere.
- preflightFailFastBody extracted into src/stream-error.ts as the single
  source of truth, shared with handle.ts's status-code bodies.
- The genuinely new terminations are unchanged: the abort exit (respond:
  false used to skip all writes) and the overflow-refold re-run
  (respondFailFast=false suppressed the consumer) now terminate with the
  same established frames instead of dangling / hitting
  ERR_HTTP_HEADERS_SENT.
- Tests assert the native shapes; uncommitted negative control unchanged.

Targeted local: issue3978 suite 4/4, adjacent preflight suites 24/24,
tsc clean. Full-suite verdict: CI.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

PR #5057 审核完成——方向成立,原版的契约漂移已对齐重构并推送(独立标注 commit 34020b656),等 CI 绿即可合。

一句话:这个 PR 修的是 preflight「不转发」出口在头已提交后不终结响应、客户端挂到自己超时的问题;修对了两个真洞,但原版引入的新错误帧形状会静默替换 master 既有带内契约,我已将其对齐回既有契约。

八问:

  1. 是什么问题:Preflight compression holds the request with zero response bytes; client undici headersTimeout (default 300s) kills it -> repeated [preflight] summarization aborted: client disconnected 5-minute death loop, payload never forwarded #568 hold 提交早期头(200+keep-alive)之后,preflight 判定不转发时,有两个出口再写不出任何字节——客户端攥着一个 200 干等,分不清「bili 放弃了」还是网络死了,只能等自家 watchdog。
  2. 什么原因导致的:abort 出口硬编 respond:false,消费侧的带内终结分支被整体跳过;refold 重跑传 respondFailFast=false 同样抑制消费侧,随后 forward() 透传上游 400 在已提交 socket 上撞 ERR_HTTP_HEADERS_SENT。层次上:master 早已修好三个常规 fail-fast 出口的带内终结(handle.ts 消费侧,Preflight compression holds the request with zero response bytes; client undici headersTimeout (default 300s) kills it -> repeated [preflight] summarization aborted: client disconnected 5-minute death loop, payload never forwarded #568 后续),漏的就是这两个出口。另核实:PR 正文 Problem 段「所有出口都悬空」的描述对当前 master 是过时的(写于更早的 master 状态)。
  3. 影响范围与严重程度:仅命中「preflight 不转发 + Preflight compression holds the request with zero response bytes; client undici headersTimeout (default 300s) kills it -> repeated [preflight] summarization aborted: client disconnected 5-minute death loop, payload never forwarded #568 hold 已提交头」的会话——载荷超窗且压缩轮次耗尽/doomed 413/压缩中客户端断开/超窗重折叠这几条路;正常转发流量零影响。严重度:特定条件下挂起(有客户端自带超时兜底)= p2。
  4. 怎么修复的:在全部四个不转发出口就地终结响应(commitPreflightTerminalFrame,头已提交且 socket 存活时生效)。我的对齐重构要点:帧内容不发明新形状——SSE 委托既有 emitPreflightError(协议原生错误帧,各客户端 SDK 原生可解析,Preflight compression holds the request with zero response bytes; client undici headersTimeout (default 300s) kills it -> repeated [preflight] summarization aborted: client disconnected 5-minute death loop, payload never forwarded #568/上游截断在 compress 内部重发路径无法恢复:重试闸门将 ping 计为已转发输出,且 emitError 合成 end_turn 伪完成 #1455 纪律),JSON 追加 preflightFailFastBody(抽到 src/stream-error.ts 作单一来源,与 handle.ts 两处 status-code 响应共用);未提交路径保持正常 status-code 路径逐字节不变。abort 出口补了结果日志(floor 17 request 3)。
  5. 是否有回归问题:逐项审过既有行为——①三个常规 committed fail-fast 出口的写出字节:不变(同一帧,写入点从消费侧提前到出口本身);②未提交出口:helper 恒 no-op,status 路径不变(测试 T3 负控制钉住);③转发流量/status 码/消息措辞:不变;④abort 出口新增一行结果日志(纯观测)。无语义漂移、无 fail-fast 变吞错、无阈值/时序/输出格式变化。
  6. 是否破坏缓存:不涉及——前缀缓存接缝、重试策略、去重键、增量游标均未触碰。
  7. 是否改了设计方向:无架构转向;保留「在决策点终结」的设计,只是把终结点统一收进四个出口。被移除的是原版真正的偏航:用新 {ok:false,error:{...}} 通用帧替换模型线上的既有带内契约(该形状只存在于 /__bili/* admin 面;照原版合入,Anthropic 车道上 Claude Code 解析原生 event: error schema 的能力会丢)。PR 正文 OLD→NEW 描述已同步修正为对当前 master 的准确陈述。
  8. 是否引入了新的配置:无——没有新 env/config/默认值。

验证证据:定向本地——issue3978 套件 4/4(T1/T2 钉挂起 bug:读体竞 20s 超时,pre-fix 必挂;T3/T4 守既有行为)、相邻套件 preflight-fail-fast/preflight-client-abort/preflight-hold/anthropic-beta-window 24/24、tsc 干净。全量裁决以推送头 34020b656 的 CI 为准,尚未出结果。

一个观察:镜像缓存里旧头(4bcbcf9)的 30 项 check 全部记为 skipped——若新头的 run 也整 lane skip,说明触发配置有问题,按规矩我去 #4599 评论一句。

判定:代码就绪,待 CI 绿确认后可合(合并留给你)。标签已在首条回复报过(bug / scope/wiring / p2)。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

CI 状态更新(接续八问结论):推送头 34020b656 的首轮 CI 唯一红腿 test (windows-latest, 22, 1) 已定位并定性——

判定不变:代码就绪,等第二整轮 run 全绿即可合;若其再红我继续跟。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

CI 终局:推送头 34020b656 的整轮 run(40800290013)已全绿——26 success / 1 skipped / 0 failure(含此前红的 w22s1 分片,确认首轮红为 multi-instance-live flake)。全部 e2e 真实客户端车道绿。

收工判定:PR #5057 代码与 CI 均就绪,可合(合并留给你)。本 PR 交付 = 四个 preflight 不转发出口在头已提交后按既有带内契约终结响应(abort + refold 两个真洞补上)+ abort 出口结果日志 + preflightFailFastBody 单一来源抽取;无任何 wire 形状变更。若后续 master 侧再遇该 multi-instance-live 签名,见 #4599 我的记录行。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

结论先行:CI 红不是本 PR 引起的。 唯一红项 test (windows-latest, 22, 1) 挂在 tests/multi-instance-live.test.ts(两个真实代理进程共享一个 sessions dir 的并发持久化 e2e),与本 PR 的 preflight 终止帧改动零交集。证据链齐全后已触发 rerun-failed-jobs(run 40796549458),等重跑结果即可。

查重:不重复——这是既有 PR 的 CI 事件而非新缺陷报告;状态已同步一行到 #4599(新签名首次出现,该文件历史失败记录里没有 multi-instance)。

结构化八问:

  1. 是什么问题:head 34020b6 的 windows-22 分片 1 里,two live instances over one sessions dir: serve, guard, converge (#394/#405/#2401) 子测试的读者循环观察到磁盘上的 stats.requests 从 6 回落到 5(disk requests rolled back: 5 < 6,tests/multi-instance-live.test.ts:241),违反「共享记录计数器单调不减」断言。其余 1994 个测试全过。
  2. 什么原因导致的:判为 Windows runner 高负载下的偶发竞态,不是代码回归。依据:①本 PR diff 只改 preflight no-forward 退出路径的错误契约(src/server.ts、src/stream-error.ts、src/server/handle.ts + 自带测试),不碰 persist/converge/debounce 任何路径;②该测试文件自 merge-base(b08fbf3)以来零改动;③同分支上一 head 4bcbcf9 的 CI(run 40728955786)全绿,含同一 windows-22 s1 分片——同一套 persist 代码昨天刚过;④master@5594c13 今天各 test 分片全绿(CI Control Center: Centralized CI Failure Handling & Status Sync (Comment all CI issues here; do not open new issues) | CI 总控: CI 失败集中处理与状态同步(所有 CI 问题在此评论, 勿开新 issue) #4599 floor 432/433);⑤本地 Linux 复跑 30/30 全绿(含 3 路并行 × 6 次的加压批),CI 上该子测试耗时 14.5s 而本地仅 3.8s,典型负载放大特征。该测试头注释自己也把「counter catch-up re-clobber」列为已知残留面(另由 multi-instance-persist 单元测试钉住),说明这类跨进程时序敏感是已知域。
  3. 影响范围与严重程度:仅影响本次 PR 门禁,无产品行为影响。若它是真缺陷,含义会是多实例 stale-guard 在特定 Windows 调度下放行旧快照覆盖新快照——但目前没有任何证据支持这一点(上一 head 绿 + master 绿 + 本地 30/30),按偶发处理。
  4. 怎么修复的:不改代码——没有可复现的缺陷可修,而放宽/加固那条单调性断言语义属于设计决策,不该在自动修复车道里顺手做。动作 = 对 run 40796549458 触发 rerun-failed-jobs(已受理,queued)。升级预案:若重跑再红,下一步把 origin/master 合入 PR 分支(含 4bf931c persist fork-ledger 原子化修复与 windows-latest CI will always be red: the drain e2e in #4982 uses POSIX-only kill(SIGTERM) to trigger graceful shutdow | windows-latest CI 必红:#4982 的 drain e2e 用 POSIX 专属的 kill(SIGTERM) 触发优雅退出,Node 在 Windows 上是直接硬杀 #5050 的 Windows SIGTERM/RST 测试加固,当前分支 behind master 且无冲突)再推;若仍红,就开 decision request 讨论测试加固方案,不再自行试错。
  5. 是否有回归问题:本轮零代码变更,无新增回归面。PR 自身范围(preflight 终止帧、in-band 错误契约复用 emitPreflightError/preflightFailFastBody)维持上一轮 review 的八问结论不变。
  6. 是否破坏缓存:不涉及。
  7. 是否改了设计方向:不涉及。
  8. 是否引入了新的配置:不涉及。

收工判定:等待 rerun 结果。预期转绿;转绿后本 PR 维持上一轮「可合并」判定(分支目前 behind master 但可干净合并,合并前我会提示先同步 master)。若重跑再红,按上面第 4 问的升级预案走,不会静默等待。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Rerun 结果:绿。 test (windows-latest, 22, 1) 重跑通过(job 123232609164,02:47–02:53Z),同一 head 34020b6、同一测试——偶发判定坐实,非回归。当前该 PR 全部 check 绿。

收工判定更新:本 PR 维持「可合并」。唯一遗留提示:分支目前 behind master(可干净合并,无冲突);合并前建议先同步一次 master(会带入 4bf931c persist fork-ledger 原子化修复与 #5050 的 Windows 测试加固)。需要我现在把 master 合进来并跑一轮 CI 的话说一声即可。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

CI 修复车道收尾汇总(自动修复 attempt 1/2):

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)
update-branch synced origin/master (09a5b5b) into this branch via a clean merge — zero conflicts, new head 02dc7a9; no longer behind (mergeable_state behind → blocked = CI re-running, fresh run 0 failed / 13 pending).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant