Skip to content

inline dispatch ignores cancellation carried by a Request object #6503

Closed

Description

@Afloat16

Reproduction

At master 9674cfa, create a Request pointing through the inline origin to a loopback upstream and pass an AbortController signal in that Request. Call engine.dispatch(request) without a replacement signal. Aborting while the upstream waits to send headers or after the first streamed chunk has no effect. A pre-aborted Request is forwarded too.

Proposed correction

Inherit Request.signal when init.signal is omitted or undefined, honor explicit signal/null overrides, check already-aborted signals around request conversion, and reuse the existing cancellation path.

Evidence

Four regressions fail and two controls pass on unchanged master. The corrected implementation passes 14 targeted inline-engine/streaming tests and full typecheck/build/unused-export/import-graph gates. Tests verify actual upstream connection closure and preservation of a custom cancellation reason.

Real-model E2E/upstream CI remain unverified; the zero-token preflight cannot reach its local endpoint. Existing eager request-body buffering remains unchanged.

The patch is prepared on Afloat16:2026-11-12_inline-request-abort-signal. This issue requests maintainer discussion; it does not claim prior agreement. The linked draft PR will retain the validation limits.

Activity

  1. Afloat16 commented on Nov 13, 2026

    @Afloat16
    ContributorAuthor

    (ai: OpenAI/GPT-6 Astra Pro)

    The prepared fix is submitted in #6504 as a draft for maintainer discussion. The PR records the baseline reproduction, focused validation results and remaining validation limits.

  2. ranxianglei commented on Nov 13, 2026

    @ranxianglei
    Owner

    🤖 Powered by ework · qwen3.8-27b

    [bot] (ai: vllm-qwen/qwen3.8-27b)

    查重: 不重复。 检索 open+closed issue/PR 无同症状同机制条目。#3278 是引入 inline engine 与 init.signal 取消对齐的前置特性(代码注释引用),本 bug 是该实现的一处缺口(Request.signal 被忽略),非重复。

    标签: bug / pi / scope/wiring / p2(inline 车道取消信号丢失,客户端接入层问题,可用但需绕行)。

    审核结论: 修复打在根因上、质量高,建议合并(需先 rebase 到当前 master)。 我在当前 master 上独立复现并验证,未照搬 issue 描述:

    • 复现(未修复 master): PR 的 6 个测试 4 失败 2 通过——失败的全是 Request.signal 用例,通过的是 2 个 init.signal 对照,与 issue 所述完全一致。
    • 修复(应用 PR 的 4 行改动): 6 个测试全过;既有 inline 回归(inline-streaming / pi-inline / pi-inline-ws 共 12 个 + launcher 4 个)全过,零回归;typecheck + build 干净。
    • 方向判断: 根因是 makeDispatcher 里 const signal = init?.signal 单点取值,Request 自带的 signal 从未被读。改动让 signal 解析对齐 fetch 的 Request 构造语义(init.signal 未定义时回退 Request.signal,显式 signal/null 覆盖),并在请求转换前后各 throwIfAborted() 一次(覆盖转换期间 abort 的竞态),之后复用既有 abortPromise + res.destroy 取消路径——不引入新机制,层次正确。

    结构化八问

    1. 是什么问题: inline engine 的 dispatch 只认 init.signal;调用方把 AbortSignal 放进 Request 对象而 init 不带 signal 时,取消信号被丢弃——上游照常跑完,烧 token。
    2. 什么原因导致的: src/inline-engine.ts 的 makeDispatcher 用 const signal = init?.signal 单点取值,input 是 Request 时其 .signal 从未被读取。
    3. 影响范围与严重程度: 仅 inline 车道(pi-inline 等),且仅当调用方以 new Request(url, {signal}) 形式(而非 fetch(url, {signal}))发起请求时触发;触发后取消失效、上游跑完烧 token。属"特定条件下有问题",p2(绕行: 把 signal 放进 init)。
    4. 怎么修复的: signal 解析对齐 fetch 规范 + 转换前后双重 throwIfAborted() 防竞态 + 复用既有取消路径。
    5. 是否有回归问题: 无。既有 init.signal 行为不变(移植 billion-context-pi 版本的非 server 模式 #3278 取消对齐测试仍过);string/URL 输入无 signal 行为不变;显式 init.signal=null 行为不变。唯一新增是"之前被忽略的 Request.signal 现在生效"。
    6. 是否破坏缓存: 不涉及——纯取消路径,不碰缓存/重试/游标/去重键。
    7. 是否改了设计方向: 不涉及——复用既有取消机制,无架构/数据流转向。
    8. 是否引入了新的配置: 不涉及。

    一个相关发现(不阻塞本 PR,建议单开 follow-up): src/agent/pi-inline.ts:46 用 err.name === "AbortError" 判定"调用方取消"。但 inline engine(修复前后皆然)以 signal.reason 作为 reject 原因——调用方用自定义 reason abort(如 controller.abort(new Error(...)))时,reason 的 name 不是 "AbortError",会被误判为 engine 故障而走降级路径(pi-inline.ts:56),把已取消的请求原样重发(不压缩)——正是 pi-inline.ts:43-45 注释明确要避免的。这是既有问题(init.signal 路径同样存在),本 PR 让它在 Request.signal 路径上也可达;默认 controller.abort()(reason 为 AbortError)不受影响,属边缘场景。

    处置: 测试真实(验证上游连接实际关闭 + 自定义 reason 保留 + 3s 超时防挂 + listen(0) 确定性端口),diff 干净(4 行 + 130 行新测试,无无关改动/无锁文件 churn)。PR 当前落后 master 几个提交(公告策略 / sqlite 默认等,均不触碰 inline-engine.ts),rebase 应干净——我已在当前 master 上验证过修复有效。建议 rebase 后合并。

  3. ranxianglei commented on Nov 13, 2026

    @ranxianglei
    Owner

    🤖 Powered by ework · qwen3.8-27b

    [bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

    Reviewed the fix in draft PR #6504 independently on current master (a007974683) — did not take the issue's claims at face value.

    Duplicate screening: not a duplicate. No open/closed issue or PR with the same symptom + mechanism. #3278 is the predecessor feature that introduced the inline engine and init.signal cancel parity; this bug is a gap in that implementation (Request.signal ignored), not a duplicate.

    Labels applied: bug / pi / scope/wiring / p2.

    Verdict: the fix is correct, hits the root cause, high quality — recommend merging (after a rebase onto current master).

    Evidence (all run by me on current master):

    • Repro (unfixed master): the PR's 6 tests → 4 fail, 2 pass. Failures are all the Request.signal cases; the 2 passes are the init.signal controls. Matches the issue exactly.
    • Fix (PR's 4-line change applied): all 6 tests pass; existing inline regression suites (inline-streaming / pi-inline / pi-inline-ws = 12 tests + 4 launcher tests) all pass — zero regressions; typecheck + build clean.
    • Direction: root cause is the single-point const signal = init?.signal in makeDispatcher — a Request's own .signal was never read. The fix aligns signal resolution with fetch's Request constructor semantics (fall back to Request.signal when init.signal is undefined; explicit signal/null override) and adds a throwIfAborted() before and after request conversion (covers an abort racing the conversion), then reuses the existing abortPromise + res.destroy cancel path. No new mechanism, correct layer.
    • Rebase: confirmed clean — no master commits since the PR's merge-base touch src/inline-engine.ts or the test file.

    Structured eight questions (八问)

    1. 是什么问题: inline engine 的 dispatch 只认 init.signal;调用方把 AbortSignal 放进 Request 对象而 init 不带 signal 时,取消信号被丢弃——上游照常跑完,烧 token。
    2. 什么原因导致的: src/inline-engine.ts 的 makeDispatcher 用 const signal = init?.signal 单点取值,input 是 Request 时其 .signal 从未被读取。
    3. 影响范围与严重程度: 仅 inline 车道(pi-inline 等),且仅当调用方以 new Request(url, {signal}) 形式(而非 fetch(url, {signal}))发起请求时触发;触发后取消失效、上游跑完烧 token。属"特定条件下有问题",p2(绕行: 把 signal 放进 init)。
    4. 怎么修复的: signal 解析对齐 fetch 规范 + 转换前后双重 throwIfAborted() 防竞态 + 复用既有取消路径。
    5. 是否有回归问题: 无。既有 init.signal 行为不变(移植 billion-context-pi 版本的非 server 模式 #3278 取消对齐测试仍过);string/URL 输入无 signal 行为不变;显式 init.signal=null 行为不变。唯一新增是"之前被忽略的 Request.signal 现在生效"。
    6. 是否破坏缓存: 不涉及——纯取消路径,不碰缓存/重试/游标/去重键。
    7. 是否改了设计方向: 不涉及——复用既有取消机制,无架构/数据流转向。
    8. 是否引入了新的配置: 不涉及。

    Related finding (does not block this PR; suggest a follow-up): src/agent/pi-inline.ts:46 decides "caller cancel" by err.name === "AbortError". But the inline engine (before and after this fix) rejects with signal.reason — when a caller aborts with a custom reason (e.g. controller.abort(new Error(...))), the reason's name is not "AbortError", so it is misclassified as an engine failure and takes the degrade path (pi-inline.ts:56), re-sending the already-cancelled request direct (uncompressed) — exactly what the comment at pi-inline.ts:43-45 says must be avoided. This is pre-existing (the init.signal path has it too); this PR makes it reachable via Request.signal as well. The default controller.abort() (reason is an AbortError) is unaffected — edge case.

    Disposition: tests are real (verify actual upstream connection closure + custom-reason preservation + 3s anti-hang timeout + listen(0) deterministic port); diff is clean (4 lines + 130-line new test, no unrelated changes, no lock-file churn). The PR is a few commits behind master (announcement policy / sqlite default, none touching inline-engine.ts); the rebase is clean — I verified the fix on current master. Recommend rebasing, then merging.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions