You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
inline dispatch ignores cancellation carried by a Request object #6503
At master 9674cfa, create a Request pointing through the inline origin to a loopback upstream and pass an AbortController signal in that Request. Call engine.dispatch(request) without a replacement signal. Aborting while the upstream waits to send headers or after the first streamed chunk has no effect. A pre-aborted Request is forwarded too.
Proposed correction
Inherit Request.signal when init.signal is omitted or undefined, honor explicit signal/null overrides, check already-aborted signals around request conversion, and reuse the existing cancellation path.
Evidence
Four regressions fail and two controls pass on unchanged master. The corrected implementation passes 14 targeted inline-engine/streaming tests and full typecheck/build/unused-export/import-graph gates. Tests verify actual upstream connection closure and preservation of a custom cancellation reason.
Real-model E2E/upstream CI remain unverified; the zero-token preflight cannot reach its local endpoint. Existing eager request-body buffering remains unchanged.
The patch is prepared on Afloat16:2026-11-12_inline-request-abort-signal. This issue requests maintainer discussion; it does not claim prior agreement. The linked draft PR will retain the validation limits.
The prepared fix is submitted in #6504 as a draft for maintainer discussion. The PR records the baseline reproduction, focused validation results and remaining validation limits.
Reviewed the fix in draft PR #6504 independently on current master (a007974683) — did not take the issue's claims at face value.
Duplicate screening: not a duplicate. No open/closed issue or PR with the same symptom + mechanism. #3278 is the predecessor feature that introduced the inline engine and init.signal cancel parity; this bug is a gap in that implementation (Request.signal ignored), not a duplicate.
Labels applied: bug / pi / scope/wiring / p2.
Verdict: the fix is correct, hits the root cause, high quality — recommend merging (after a rebase onto current master).
Evidence (all run by me on current master):
Repro (unfixed master): the PR's 6 tests → 4 fail, 2 pass. Failures are all the Request.signal cases; the 2 passes are the init.signal controls. Matches the issue exactly.
Direction: root cause is the single-point const signal = init?.signal in makeDispatcher — a Request's own .signal was never read. The fix aligns signal resolution with fetch's Request constructor semantics (fall back to Request.signal when init.signal is undefined; explicit signal/null override) and adds a throwIfAborted() before and after request conversion (covers an abort racing the conversion), then reuses the existing abortPromise + res.destroy cancel path. No new mechanism, correct layer.
Rebase: confirmed clean — no master commits since the PR's merge-base touch src/inline-engine.ts or the test file.
是否有回归问题: 无。既有 init.signal 行为不变(移植 billion-context-pi 版本的非 server 模式 #3278 取消对齐测试仍过);string/URL 输入无 signal 行为不变;显式 init.signal=null 行为不变。唯一新增是"之前被忽略的 Request.signal 现在生效"。
是否破坏缓存: 不涉及——纯取消路径,不碰缓存/重试/游标/去重键。
是否改了设计方向: 不涉及——复用既有取消机制,无架构/数据流转向。
是否引入了新的配置: 不涉及。
Related finding (does not block this PR; suggest a follow-up):src/agent/pi-inline.ts:46 decides "caller cancel" by err.name === "AbortError". But the inline engine (before and after this fix) rejects with signal.reason — when a caller aborts with a custom reason (e.g. controller.abort(new Error(...))), the reason's name is not "AbortError", so it is misclassified as an engine failure and takes the degrade path (pi-inline.ts:56), re-sending the already-cancelled request direct (uncompressed) — exactly what the comment at pi-inline.ts:43-45 says must be avoided. This is pre-existing (the init.signal path has it too); this PR makes it reachable via Request.signal as well. The default controller.abort() (reason is an AbortError) is unaffected — edge case.
Disposition: tests are real (verify actual upstream connection closure + custom-reason preservation + 3s anti-hang timeout + listen(0) deterministic port); diff is clean (4 lines + 130-line new test, no unrelated changes, no lock-file churn). The PR is a few commits behind master (announcement policy / sqlite default, none touching inline-engine.ts); the rebase is clean — I verified the fix on current master. Recommend rebasing, then merging.
Activity
Afloat16 commentedon Nov 13, 2026
(ai: OpenAI/GPT-6 Astra Pro)
The prepared fix is submitted in #6504 as a draft for maintainer discussion. The PR records the baseline reproduction, focused validation results and remaining validation limits.
ranxianglei commentedon Nov 13, 2026
[bot] (ai: vllm-qwen/qwen3.8-27b)
查重: 不重复。 检索 open+closed issue/PR 无同症状同机制条目。#3278 是引入 inline engine 与
init.signal取消对齐的前置特性(代码注释引用),本 bug 是该实现的一处缺口(Request.signal被忽略),非重复。标签: bug / pi / scope/wiring / p2(inline 车道取消信号丢失,客户端接入层问题,可用但需绕行)。
审核结论: 修复打在根因上、质量高,建议合并(需先 rebase 到当前 master)。 我在当前 master 上独立复现并验证,未照搬 issue 描述:
Request.signal用例,通过的是 2 个init.signal对照,与 issue 所述完全一致。makeDispatcher里const signal = init?.signal单点取值,Request自带的 signal 从未被读。改动让 signal 解析对齐 fetch 的Request构造语义(init.signal未定义时回退Request.signal,显式 signal/null 覆盖),并在请求转换前后各throwIfAborted()一次(覆盖转换期间 abort 的竞态),之后复用既有abortPromise+res.destroy取消路径——不引入新机制,层次正确。结构化八问
dispatch只认init.signal;调用方把AbortSignal放进Request对象而init不带 signal 时,取消信号被丢弃——上游照常跑完,烧 token。src/inline-engine.ts的makeDispatcher用const signal = init?.signal单点取值,input是Request时其.signal从未被读取。new Request(url, {signal})形式(而非fetch(url, {signal}))发起请求时触发;触发后取消失效、上游跑完烧 token。属"特定条件下有问题",p2(绕行: 把 signal 放进 init)。throwIfAborted()防竞态 + 复用既有取消路径。init.signal行为不变(移植 billion-context-pi 版本的非 server 模式 #3278 取消对齐测试仍过);string/URL 输入无 signal 行为不变;显式init.signal=null行为不变。唯一新增是"之前被忽略的Request.signal现在生效"。一个相关发现(不阻塞本 PR,建议单开 follow-up):
src/agent/pi-inline.ts:46用err.name === "AbortError"判定"调用方取消"。但 inline engine(修复前后皆然)以signal.reason作为 reject 原因——调用方用自定义 reason abort(如controller.abort(new Error(...)))时,reason 的name不是"AbortError",会被误判为 engine 故障而走降级路径(pi-inline.ts:56),把已取消的请求原样重发(不压缩)——正是pi-inline.ts:43-45注释明确要避免的。这是既有问题(init.signal路径同样存在),本 PR 让它在Request.signal路径上也可达;默认controller.abort()(reason 为AbortError)不受影响,属边缘场景。处置: 测试真实(验证上游连接实际关闭 + 自定义 reason 保留 + 3s 超时防挂 +
listen(0)确定性端口),diff 干净(4 行 + 130 行新测试,无无关改动/无锁文件 churn)。PR 当前落后 master 几个提交(公告策略 / sqlite 默认等,均不触碰inline-engine.ts),rebase 应干净——我已在当前 master 上验证过修复有效。建议 rebase 后合并。ranxianglei commentedon Nov 13, 2026
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)
Reviewed the fix in draft PR #6504 independently on current master (
a007974683) — did not take the issue's claims at face value.Duplicate screening: not a duplicate. No open/closed issue or PR with the same symptom + mechanism. #3278 is the predecessor feature that introduced the inline engine and
init.signalcancel parity; this bug is a gap in that implementation (Request.signalignored), not a duplicate.Labels applied: bug / pi / scope/wiring / p2.
Verdict: the fix is correct, hits the root cause, high quality — recommend merging (after a rebase onto current master).
Evidence (all run by me on current master):
Request.signalcases; the 2 passes are theinit.signalcontrols. Matches the issue exactly.const signal = init?.signalinmakeDispatcher— aRequest's own.signalwas never read. The fix aligns signal resolution with fetch'sRequestconstructor semantics (fall back toRequest.signalwheninit.signalis undefined; explicit signal/null override) and adds athrowIfAborted()before and after request conversion (covers an abort racing the conversion), then reuses the existingabortPromise+res.destroycancel path. No new mechanism, correct layer.src/inline-engine.tsor the test file.Structured eight questions (八问)
dispatch只认init.signal;调用方把AbortSignal放进Request对象而init不带 signal 时,取消信号被丢弃——上游照常跑完,烧 token。src/inline-engine.ts的makeDispatcher用const signal = init?.signal单点取值,input是Request时其.signal从未被读取。new Request(url, {signal})形式(而非fetch(url, {signal}))发起请求时触发;触发后取消失效、上游跑完烧 token。属"特定条件下有问题",p2(绕行: 把 signal 放进 init)。throwIfAborted()防竞态 + 复用既有取消路径。init.signal行为不变(移植 billion-context-pi 版本的非 server 模式 #3278 取消对齐测试仍过);string/URL 输入无 signal 行为不变;显式init.signal=null行为不变。唯一新增是"之前被忽略的Request.signal现在生效"。Related finding (does not block this PR; suggest a follow-up):
src/agent/pi-inline.ts:46decides "caller cancel" byerr.name === "AbortError". But the inline engine (before and after this fix) rejects withsignal.reason— when a caller aborts with a custom reason (e.g.controller.abort(new Error(...))), the reason'snameis not"AbortError", so it is misclassified as an engine failure and takes the degrade path (pi-inline.ts:56), re-sending the already-cancelled request direct (uncompressed) — exactly what the comment atpi-inline.ts:43-45says must be avoided. This is pre-existing (theinit.signalpath has it too); this PR makes it reachable viaRequest.signalas well. The defaultcontroller.abort()(reason is anAbortError) is unaffected — edge case.Disposition: tests are real (verify actual upstream connection closure + custom-reason preservation + 3s anti-hang timeout +
listen(0)deterministic port); diff is clean (4 lines + 130-line new test, no unrelated changes, no lock-file churn). The PR is a few commits behind master (announcement policy / sqlite default, none touchinginline-engine.ts); the rebase is clean — I verified the fix on current master. Recommend rebasing, then merging.