Skip to content

[PR] fix: stop inline upstream work when the response body is cancelled - #6571

Merged
ranxianglei merged 14 commits into
ranxianglei:masterfrom
Afloat16:2026-11-13_inline-body-cancel
Nov 14, 2026
Merged

ranxianglei merged 14 commits into
ranxianglei:masterfrom
Afloat16:2026-11-13_inline-body-cancel

Conversation

@Afloat16

Copy link
Copy Markdown
Contributor

Fixes #6570

Problem

InlineRes creates a consumer ReadableStream without a cancel handler. Cancelling response.body or an active reader therefore closes only the consumer side. The upstream can continue generating or remain idle indefinitely because the pipeline never receives its response close/client-disconnect event.

Changes

Wire the stream source cancel callback to InlineRes.destroy(). The existing close and client-abort paths then terminate raw upstream reads and model SSE generation. The caller-owned signal is not changed.

Validation

  • Five new real-loopback HTTP cases cover unread raw bodies, active raw readers, a caller-provided signal, Anthropic model SSE after its first token, and ordinary completion.
  • Unchanged master e1f31ad: all four cancellation cases leave the upstream connection open after the one-second deadline; the completion control passes.
  • Fixed branch: all five pass, with upstream response close observed after cancellation. The existing pi-inline and inline-streaming suites also pass: 13 tests total.
  • Root typecheck, complete build, unused-export gate, test-import-graph gate and git diff --check pass. Hosted CI and real-model E2E remain separate merge gates.

Related scope

Checked prior Afloat16 submissions and fresh all-state body.cancel/InlineRes/cancellation searches. #6504 handles a caller abort signal; this patch handles cancelling response consumption without aborting that signal. It reuses existing lifecycle propagation and adds no configuration, retries, kernel or persistence change.

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重: 不重复。 与 #6504(open,Closes #6503)同触 src/inline-engine.ts 但机制不同:#6504 是「信号来源解析」——把 Request.signal 纳入有效信号(init.signal 缺省时),走的是既有 abort-signal 路径;本 PR 是「响应体消费取消」——response.body.cancel() / reader.cancel() 这条不涉及任何 signal 的触发器。#6051(已合)是真实 HTTP 客户端断连。三者是同一 res.destroy()→"close"→clientAbort 机制的三个独立触发器,本 PR 补上缺失的第三个。#6504 与本 PR 改的是不同区域(信号解析 ~L280 vs 构造函数 L157-160),合入顺序无关、无文本冲突。

独立审核结论(非照抄 issue)

方向关:issue 定位正确——InlineRes 的底层 source 只定义了 start、没有 cancel(src/inline-engine.ts:156-158)。取消体只关消费者流,不传播到 InlineRes 生命周期,故 destroyed/socket.destroyed 不置位、不 emit "close",registerRequestAbort(src/server.ts:336)与 relay 的 res.on("close")(src/server/relay.ts:1106)都不触发,clientAbort 永不 abort,上游挂死。修的是根本原因(缺 cancel 传播),不是治标。

修法:加 cancel: () => { this.destroy(); },复用既有的 destroy→close→clientAbort 传播(与 #6504/#3278 的 abort 路径同一条)。我逐点核过正确性:

  • destroy() 幂等(if (this.destroyed) return;)——abort 与 cancel 同时来不会双触发;
  • destroy() 里的 controller.error() 在 cancel 回调内是 no-op(WHATWG 流规范:cancel 运行时流状态是 closing 非 readable,controller.error() 要求 readable)——不会误报错/崩溃,try/catch 是防御;
  • 正常完成不受影响:end() 置 writableEnded 并 close 流,已关闭流的 cancel() 是 no-op,且 destroy() 的 if (!this.writableEnded) 守卫挡住误 emit;
  • 调用方自有 AbortController 不被 abort——测试显式断言 controller.signal.aborted === false,没有把「停止读体」和「调用方 abort 请求」混为一谈。

回归关(审过的既有行为):

测试有效性(真测试,有牙):5 个真实 loopback HTTP case(真 http.createServer 上游、临时端口,无固定端口碰撞)。我自己复现了 bug:临时还原掉那 3 行 → 4 个取消 case 全红(上游 1s 后仍开着)、完成对照绿;加回修复 → 5/5 绿。上游设计成「非 complete 响应写一帧后保持空闲」,确保测试不会因上游自然关闭而假绿。「reader with caller signal」case 带关键负向断言(体取消不 abort 调用方 signal)。

本地定向验证(全量交给 CI):

  • tests/inline-body-cancel.test.ts:5/5 过(还原修复后 4 红 1 绿,证明有牙)
  • 既有 pi-inline.test.ts + inline-streaming.test.ts:8/8 过
  • npm run typecheck:clean
  • git diff --check:clean;unused-export gate:clean;test-import-graph:ok

代码整洁(外部提交硬标准):3 行改动、命名达意、注释讲「为什么」(无 signal 也算客户端断连)而非复述、无死代码/魔法值/新依赖、风格与周边一致。达标。


结构化八问

  1. 是什么问题:InlineRes 的消费者 ReadableStream 没有 cancel 回调。调用方取消响应体(response.body.cancel() / reader.cancel())时只关消费者一侧,管道收不到 close/断连事件,上游生成一直挂着(空闲或继续烧 token)。
  2. 什么原因导致的:根因是 InlineRes 底层 source 只定义 start、没定义 cancel(src/inline-engine.ts:156-158)。取消体不传播到 InlineRes 生命周期 → 不置 destroyed/socket.destroyed、不 emit "close" → registerRequestAbort(src/server.ts:336)与 relay close 监听(src/server/relay.ts:1106)都不触发 → clientAbort 永不 abort。
  3. 影响范围与严重程度:影响 inline 车道(pi-inline / opencode-native 传输)。触发条件:调用方取消响应体消费(不 abort 自有 signal)。严重度:p2——能用但取消后上游连接/生成不释放,持续占用(空闲连接、token 生成),属资源泄漏/无法及时停止,非完全不可用。
  4. 怎么修复的:在 source 上加 cancel 回调,直接复用既有 res.destroy() 生命周期传播(与 abort 路径同一条 destroy→close→clientAbort 机制)。destroy() 幂等;controller.error() 在 cancel 回调内是 no-op;正常完成路径有 writableEnded 守卫 + 已关闭流 cancel 为 no-op 双重保护。
  5. 是否有回归问题:无。正常完成 / 调用方 abort([PR] fix: honor Request cancellation in the inline engine #6504/移植 billion-context-pi 版本的非 server 模式 #3278)/ 真实断连([PR] fix: cancel idle raw upstream reads on client disconnect #6051)三条路径独立无重叠;src/ 内无内部路径把 inline 响应体当控制流 cancel;两压缩模式均不涉及(传输层生命周期,wire 形状零变化)。
  6. 是否破坏缓存:不涉及。不碰缓存/重试/增量游标/去重键。
  7. 是否改了设计方向:不涉及。无架构/边界/数据流转向,仅把第三个取消触发器(体取消)接进既有机制。
  8. 是否引入了新的配置:不涉及。无新 env/config/默认值。

判定:可合。 满足 §7.4 auto-merge 全部门禁(单模块 fix、回归测试复现 bug 且现过、rebased head typecheck 干净、不碰 config/persistence/wire/kernel、纯 fix:、diff 干净、Fixes #6570、不碰 load-bearing infra)。合并权在 owner,请自行 merge:#6571

The suite baseline preload (preload-storage-baseline.mjs) pins
BILI_STORAGE_UNIFIED="legacy" for the broad corpus. After ranxianglei#6480 flipped the
storage default to pure-sqlite, the notice-registry suite's visibleNotices
assertions expect the pure-sqlite default notice, which is gated on
unifiedSelectionIsSqlite (unified "sqlite" or absent) — the preload's
"legacy" made the notice silent, so the suite was born-red on master. Pin
the selection explicitly (ranxianglei#4011 lesson: a test that depends on the production
default passes locally but fails in CI).
ework-agent and others added 12 commits November 14, 2026 10:20
…test

The dsh plugin lane (ci-e2e-dsh.yml) runs node --import tsx --test
WITHOUT the preload-storage-baseline.mjs preload, so BILI_STORAGE_UNIFIED
is unset and the storage mode defaults to 'full'. Under the pure-sqlite
default flip (5a80e2b, ranxianglei#6480), 'full' mode turns the JSON envelope
writer OFF — so the .content-store.json sidecar is never written, and
the test's readdirSync(namespace) hits ENOENT.

Pin BILI_STORAGE_JSON_WRITER=on in the test so the content-store
sidecar is written regardless of the storage mode. This keeps the
corrupted-store detection path exercised in both full and legacy modes.

Closes: dsh plugin lane red on master (run 43532980164, job 132255459726)
The pure-sqlite default flip (5a80e2b, ranxianglei#6480) turns the JSON envelope
writer OFF, so the .json session file this lane asserts on is never
written and 'proxy must persist the session' / 'no persisted session
carried compressed blocks' fail after the 20s wait. Pin the writer ON in
the spawned client's env (the bootstrapped proxy inherits it) so the
envelope this test was written for is still produced.

Closes: native opencode lane red on master (run 43547593803, job 132302438610)
Same pure-sqlite default flip (5a80e2b, ranxianglei#6480) as the opencode lane:
the JSON envelope writer is OFF, so 'proxy must persist the session
within 20000ms' fails — the on-disk .json session file is never written.
Pin the writer ON in the spawned dsh client's env (the bootstrapped
proxy inherits it).

Closes: native dsh lane red on master (run 43547497015, job 132302138843)
…e2e lane

Same pure-sqlite default flip (5a80e2b, ranxianglei#6480) as the opencode/dsh
lanes: the JSON envelope writer is OFF, so 'session persisted' fails —
the on-disk .json session file is never written. Pin the writer ON in
the spawned pi client's env (the bootstrapped proxy inherits it).

Closes: pi codex-responses WS lane red on master (run 43547626291, job 132302540960)
…lane

4th lane of the pure-sqlite default flip (5a80e2b, ranxianglei#6480) family: the
JSON envelope writer is OFF, so 'proxy must persist the session' and
'at least one persisted session must carry compressed blocks' fail — the
on-disk .json session file is never written. Pin the writer ON in the
spawned pi client's env (the bootstrapped proxy inherits it).

Closes: pi package-native lane red on ci-fixes (run 43558179054, job 132336647955)
Same pure-sqlite default flip (5a80e2b, ranxianglei#6480) family: the JSON writer
is OFF, so the per-session CCR content-store sidecar (.content-store.json)
that 'child persists its own companion and survives a proxy restart'
asserts on is never written (0 !== 1). The earlier b1bc17b pin only
covered tests/plugin-fork.test.ts, not this file. Pin the writer ON at
module load (before any SessionStore is constructed).

Closes: dsh plugin lane red on ci-fixes (run 43558179102, job 132336519539)
…lane

Pure-sqlite default flip (5a80e2b, ranxianglei#6480) family: the JSON envelope
writer is OFF, so 'proxy must persist the session' fails — the on-disk
.json session file is never written. Pin the writer ON in the spawned
opencode client's env (the bootstrapped proxy inherits it).
Pure-sqlite default flip (5a80e2b, ranxianglei#6480) family: the JSON envelope
writer is OFF, so the on-disk .json session envelope this lane asserts on
(exactly ONE persisted session under the raw key) is never written. Pin
the writer ON in the spawned dsh client's env.
Pure-sqlite default flip (5a80e2b, ranxianglei#6480) family: the JSON envelope
writer is OFF, so the on-disk .json session file this lane asserts on
(state survival across the OLD→NEW update) is never written. Pin the
writer ON in the spawned proxy's env (both update phases share envBase).
Resolve CI baseline conflicts using the current upstream test setup.
@ranxianglei
ranxianglei merged commit 535ac76 into ranxianglei:master Nov 14, 2026
50 checks passed
@ranxianglei

Copy link
Copy Markdown
Owner

Superseded by #6677 (merged as 3858b6956, 2026-11-14).

Landed into master via the real merge commit 535ac764a2e (Merge Afloat16/2026-11-13_inline-body-cancel (PR #6571)), so your original commits are preserved verbatim in master history — only hashes of the batch wrapper changed, your authorship and commit messages are intact.

The fix for #6570 is now in master. Closing this PR as superseded — thanks for the careful, well-tested fix.

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Acknowledged — live-verified the closure state:

My review above stands (可合, all §7.4 gates met); the fix for #6570 is now in master. Nothing further needed from me — thread is clean.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cancelling an inline response body leaves upstream generation running

2 participants