Repository navigation
[PR] fix: terminate committed preflight responses with a protocol error frame (#3978) - #5057
ranxianglei wants to merge 6 commits into
Conversation
…ame (#3978) Once #568 has committed early headers (200 + SSE/JSON keep-alive), every no-forward exit left the client holding a 200 and an open stream that never completes: respond:false writes nothing, and a later status code cannot be written after the 200 — so clients could not distinguish 'bili gave up' from a dead network and waited out their own watchdog (#3978 floor 17/18). All four preflight no-forward exits now emit one terminal frame before returning when headers are already committed: SSE gets an 'event: error' frame, non-streaming JSON gets the error object appended after the whitespace padding (still one valid JSON document), then end(). Body reuses the native fail-fast shape {ok:false,error:{message,retryable}}. Uncommitted exits keep their normal status-code path byte-for-byte, and the abort branch logs its outcome (terminal frame sent / socket gone / not committed).
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.7291-pr.5057.b244881This exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr5057.tgz
npm install -g packageInstall instructions are refreshed on each push. |
(ai: vllm-qwen/qwen3.8-27b) 查重: 不重复。这是 #3978(floor 17/18 线程)的独立落地 PR,与在途的 #3590 相关但机制不同——#3590 改的是 标签: 开始独立审核:拉分支核对代码 + 读原 issue 方向 + CI 状态,结论随后按八问给出。 |
(ai: vllm-qwen/qwen3.8-27b) 进度汇报(会话曾卡住被重启,现继续):审核进行到一半,已有关键发现——
继续干活,结论稍后。 |
(ai: vllm-qwen/qwen3.8-27b) 进度:会话连续多次被卡住重启(非阻塞性错误,疑似运行时长挂起),现切换为小步快跑继续。当前状态:
继续执行,有结果即报。 |
Review alignment for the terminal-frame fix: the first commit introduced a
new {ok:false,error:{message,retryable}} frame for committed no-forward
exits, which would have silently replaced the in-band contract master
already emits on this path (protocol-native preflight_compress_failed
frames per #568/#1455 - client SDKs parse those natively; the {ok:false}
shape exists only on /__bili/* admin endpoints).
- commitPreflightTerminalFrame now delegates to emitPreflightError (SSE)
and appends preflightFailFastBody (non-streaming): byte-identical to
what the consumer's committed branch already emits, so no wire-shape
change anywhere.
- preflightFailFastBody extracted into src/stream-error.ts as the single
source of truth, shared with handle.ts's status-code bodies.
- The genuinely new terminations are unchanged: the abort exit (respond:
false used to skip all writes) and the overflow-refold re-run
(respondFailFast=false suppressed the consumer) now terminate with the
same established frames instead of dangling / hitting
ERR_HTTP_HEADERS_SENT.
- Tests assert the native shapes; uncommitted negative control unchanged.
Targeted local: issue3978 suite 4/4, adjacent preflight suites 24/24,
tsc clean. Full-suite verdict: CI.
(ai: vllm-qwen/qwen3.8-27b) PR #5057 审核完成——方向成立,原版的契约漂移已对齐重构并推送(独立标注 commit 一句话:这个 PR 修的是 preflight「不转发」出口在头已提交后不终结响应、客户端挂到自己超时的问题;修对了两个真洞,但原版引入的新错误帧形状会静默替换 master 既有带内契约,我已将其对齐回既有契约。 八问:
验证证据:定向本地——issue3978 套件 4/4(T1/T2 钉挂起 bug:读体竞 20s 超时,pre-fix 必挂;T3/T4 守既有行为)、相邻套件 一个观察:镜像缓存里旧头(4bcbcf9)的 30 项 check 全部记为 skipped——若新头的 run 也整 lane skip,说明触发配置有问题,按规矩我去 #4599 评论一句。 判定:代码就绪,待 CI 绿确认后可合(合并留给你)。标签已在首条回复报过(bug / scope/wiring / p2)。 |
(ai: vllm-qwen/qwen3.8-27b) CI 状态更新(接续八问结论):推送头
判定不变:代码就绪,等第二整轮 run 全绿即可合;若其再红我继续跟。 |
(ai: vllm-qwen/qwen3.8-27b) CI 终局:推送头 收工判定:PR #5057 代码与 CI 均就绪,可合(合并留给你)。本 PR 交付 = 四个 preflight 不转发出口在头已提交后按既有带内契约终结响应(abort + refold 两个真洞补上)+ abort 出口结果日志 + |
(ai: vllm-qwen/qwen3.8-27b) 结论先行:CI 红不是本 PR 引起的。 唯一红项 查重:不重复——这是既有 PR 的 CI 事件而非新缺陷报告;状态已同步一行到 #4599(新签名首次出现,该文件历史失败记录里没有 multi-instance)。 结构化八问:
收工判定:等待 rerun 结果。预期转绿;转绿后本 PR 维持上一轮「可合并」判定(分支目前 behind master 但可干净合并,合并前我会提示先同步 master)。若重跑再红,按上面第 4 问的升级预案走,不会静默等待。 |
(ai: vllm-qwen/qwen3.8-27b) Rerun 结果:绿。 收工判定更新:本 PR 维持「可合并」。唯一遗留提示:分支目前 behind master(可干净合并,无冲突);合并前建议先同步一次 master(会带入 4bf931c persist fork-ledger 原子化修复与 #5050 的 Windows 测试加固)。需要我现在把 master 合进来并跑一轮 CI 的话说一声即可。 |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) CI 修复车道收尾汇总(自动修复 attempt 1/2):
|
…pdate-branch, resync)
… adopt sentinel test fix for inherited master red, #4599)
Follow-up to #3978 (floor 17/18 thread); complements open PR #3590. Kept SEPARATE from #3590 per the thread discussion, so it can land whether or not #3590 does.
Problem
Once the #568 keep-alive hold commits early headers (200 + SSE comment lines / JSON whitespace padding), TWO preflight no-forward exits left the client holding a 200 and an open stream that never completes:
respond: false→ the consumer writes nothing, noend();respondFailFast=false→ the consumer's in-band branch is suppressed entirely, and after the hold has committed,forward()'s upstream-400 relay hitsERR_HTTP_HEADERS_SENTon the committed socket.(The other no-forward exits — probe doomed-413, generic fail-fast, post-preflight doomed-413 — are ALREADY terminated in-band on current master by the consumer at
src/server/handle.ts, a #568 follow-up: protocol-native error frames after commit, plain status codes before it.)The client cannot distinguish "bili gave up" from a dead network and has to wait out its own watchdog. Verified line-by-line at #3978 floor 17.
Change
New
commitPreflightTerminalFrame(res, prepared, message, retryable, log)(src/server.ts, next tobeginPreflightHold) called at all FOUR preflight no-forward exits: probe doomed-413, the generic fail-fast closure, post-preflight doomed-413, and the abort branch. When headers are already committed and the socket still lives, it terminates the response with the ESTABLISHED in-band contract — byte-identical to what the consumer already emits for committed fail-fasts (#568/#1455 protocol-native shapes, so client SDKs parse them natively):emitPreflightError— per-protocol error event (anthropicevent: errorcarrying{type:"error",error:{type:"server_error",code:"preflight_compress_failed",message}}; openai/responses/google their spec shapes);preflightFailFastBody(...)appended after the whitespace padding (still one valid JSON document).preflightFailFastBodyis extracted intosrc/stream-error.tsas the single source of truth, now also used by the consumer's status-code bodies inhandle.ts. Uncommitted exits keep their normal status-code path byte-for-byte: the helper no-ops when!headersSent || destroyed || writableEnded, and after a successful frame the existingrespond: !res.writableEndedself-corrects to false. The abort branch additionally logs its real outcome (terminal frame sent, stream ended/headers committed, terminal frame not sent (socket gone)/ nothing when not committed), per request 3 at floor 17.Behavior change (wire)
OLD (current master): the three normal-path committed fail-fast exits already terminate in-band via the consumer; the ABORT exit and the REFOLD path wrote nothing after commit (dangling stream /
ERR_HTTP_HEADERS_SENT). NEW: every committed no-forward exit terminates at the exit itself with the SAME established frames — visible bytes on the normal paths are UNCHANGED (the write just moves one step earlier), and the two dangling exits now deliver the same protocol-native terminal frame + stream end instead of hanging. The abort branch gains outcome logging. No wire-shape change anywhere; no change to uncommitted paths, status codes, message wording, or forwarded traffic.Interaction with #3590
If #3590 (wall budget) merges first, its post-budget 502 hits the same hole and can call this helper; if this lands first, #3590's stop path benefits automatically once it routes through these exits. Expected conflict surface: none — this PR touches only the four exit sites plus one helper in
src/server.ts(and the shared body extractor insrc/stream-error.ts), while #3590 works in thesrc/preflight.tsround loop.Tests
tests/issue3978-preflight-terminal-frame.test.ts(full-proxy e2e, hermetic, ~29s):event: errorframe (anthropic schema asserted), and the body COMPLETES (read raced against a 20s deadline — pre-fix it hangs);stream:false→ final body parses as{error:{type:"server_error",code:"preflight_compress_failed",message,retryable}};Red run (fix reverted): T1/T2 fail (stream never terminates), T3/T4 pass — the pair pins the bug while guarding the pre-existing behavior. Adjacent suites (
preflight-fail-fast,preflight-client-abort,preflight-hold,anthropic-beta-window) 24/24; typecheck clean. Full-suite verdict: CI.