Skip to content

[PR] fix: honor Request cancellation in the inline engine - #6504

Closed
Afloat16 wants to merge 17 commits into
ranxianglei:masterfrom
Afloat16:2026-11-12_inline-request-abort-signal
Closed

Afloat16 wants to merge 17 commits into
ranxianglei:masterfrom
Afloat16:2026-11-12_inline-request-abort-signal

Conversation

@Afloat16

@Afloat16 Afloat16 commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Closes #6503

Problem

Inline dispatch accepts a Request object but observes only init.signal. A caller using new Request(url, { signal }) can cancel before response headers or during streaming, yet the inline engine continues forwarding work and the caller's body read remains pending. A pre-aborted Request is also sent upstream.

Changes

Resolve the effective signal from Request when init.signal is omitted or undefined. Preserve explicit replacement signals and null overrides. Check cancellation before and after request conversion, then reuse the existing pre-header rejection and post-header stream/transport cancellation path.

Original feature validation

On unchanged master 9674cfa, the new real loopback HTTP suite reports 4 failing regressions and 2 passing override controls. It covers omitted init, undefined init.signal, a committed streaming body, and a pre-aborted Request. Cancellation tests verify the actual upstream connection closes; pre-header errors retain the caller's exact reason.

After completing the build, all 14 tests across inline-request-signal, inline-streaming and pi-inline pass, including existing POST model dispatch, init.signal cancellation, live token delivery and admin/transport integration.

Passed: npm run typecheck; npm run build; npm run check:unused-exports; npm run check:test-import-graph; git diff --check.

The zero-token Codex preflight finds the CLI/distribution but its local model endpoint is unreachable; its real-model case is skipped. Full real-model E2E and upstream CI remain unverified. Request body conversion keeps the existing eager-buffering behavior; this change does not make a pending request-body read interruptible.

Prior work and submission

Checked 61 author PRs, 166 fork heads, complete inline-engine history, and Request.signal/inline/abort PR and issue searches. The cancellation introduced with #3284 observes init.signal only; no prior change was found for inheritance from a Request.

Current master integration

Merged upstream fcc2e848486cc2152e36b6658c517fbfa51ec428 into this branch without rewriting the existing feature commit. Integration head: 1c6d2c5cee73c3bd951c224be4a342098a291e7f. The merge parents are the original feature head 9bcd9c44488c9225da46e57e86eb81f49f9beddc and the upstream commit above.

The production file and feature test are byte-for-byte unchanged from the original feature head. Relative to current master, this PR still changes only src/inline-engine.ts and tests/inline-request-signal.test.ts.

Revalidated the merged tree on Node 24.22.1:

  • All 14 tests across inline-request-signal, inline-streaming, and pi-inline pass, using the repository's storage and test-environment preloads.
  • npm run typecheck, npm run build, npm run check:unused-exports, and npm run check:test-import-graph pass.
  • git diff fcc2e848486cc2152e36b6658c517fbfa51ec428 --check is clean for this PR's net changes.

The original failure/control reproduction above remains tied to 9674cfafbc78c50245ae0d887483d3d6ce880c15; the validation in this section was rerun after integrating current master. The dispatch and cancellation cases use loopback HTTP fixtures, and full real-model E2E was not rerun locally. Request-body conversion retains its existing eager buffering and does not make an in-progress body read interruptible.

Review status

Linked to #6503 and ready for maintainer review. The branch now includes current master and is mergeable; GitHub no longer reports it as behind. CI run 43377731925 was queued for this exact integration head when this update was prepared. The Checks tab contains the current hosted results.

Please apply the existing ci:full label to run the FULL matrix, including the required Windows / Node 24 checks. The default CORE matrix does not produce those checks.

@ranxianglei ranxianglei changed the title fix: honor Request cancellation in the inline engine [PR] fix: honor Request cancellation in the inline engine Nov 13, 2026
@Afloat16
Afloat16 marked this pull request as ready for review November 13, 2026 04:46
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Reviewed the diff, traced the full inline-lane call chain, verified the signal semantics against the WHATWG fetch spec + empirical Node probes, and ran the targeted suites locally.

What I verified

  • The engine change (src/inline-engine.ts:272) resolves the effective signal as init?.signal !== undefined ? init.signal : input instanceof Request ? input.signal : undefined. I confirmed this matches the platform's fetch semantics exactly: the spec's Request constructor sets signal from the input Request unless init["signal"] exists (property present, even when null), so null = explicit no-signal override, undefined/absent = inherit the Request's signal. Empirical probes on Node 22 confirm: fetch(req, {signal: null}) drops the request signal; fetch(req, {signal: undefined}) / fetch(req, undefined) honor it.
  • new Request(target, requestInstance) creates a linked dependent signal (abort propagates), so the outer makeTarget rebuild in native-intercept.ts:1644 preserves cancellation.
  • Local: 14/14 targeted tests pass (inline-request-signal, inline-streaming, pi-inline); npm run typecheck clean.
  • CI: full matrix is running on the PR head 1c6d2c5cee (ubuntu/windows × node 22/24 × shards); mergeable_state is blocked pending CI. Final green/red is the CI verdict.

Regression surface (existing behaviors the diff touches)

  • init.signal provided (incl. null): unchanged — still the effective signal (override control tests confirm).
  • init.signal already aborted before dispatch: now rejects before toInlineReq (body buffering) instead of after. Strictly an improvement, matches fetch semantics — not a regression.
  • input is a string/URL: unchanged (signal = undefined).
  • input is a Request with a signal, init.signal undefined/absent: now honored (the fix).
  • No high-risk pattern: no semantic drift, no fail-fast→swallow, no threshold/timing/output-format change. The only production consumer of engine.dispatch is the pi-inline transport (pi + opencode inline lanes); the change is at the dispatcher entry and doesn't touch caching/retry/cursor/dedup.

Test quality: real loopback HTTP (not mocked), connection-close assertions, exact-reason identity (error === reason), pre-abort never-reaches-upstream counter, and both-direction override controls (gate discipline #3617). Would catch a regression. Solid.

Two adjacent pre-existing defects this PR surfaces (out of scope, noted for tracking)

  1. native-intercept.ts:433 — withHeaders rebuilds a Request as new Request(input.url, { method, headers, body, duplex }) when stamping headers, which drops input.signal (verified: signal.aborted === false after the host aborts). So on stamped paths the signal never reaches the engine and the host's cancellation is silently lost. In the pi lane (headersFor undefined) this only bites once the pf watermark channel is seeded; in dsh/opencode lanes (headersFor set) it bites on every model request. Affects spawn mode the same way. Suggested fix: add signal: input.signal to that rebuild (the existing try/catch already handles the already-aborted case — the constructor throws and falls back to the original input).
  2. pi-inline.ts:46 — the catch classifies caller-side cancel by err.name === "AbortError" only. A custom abort reason (controller.abort(new Error(...)), name "Error") is misclassified as an engine failure → the abandoned model request is re-sent direct (burns tokens) and the one-shot degrade warning is consumed. Pre-existing (the init.signal path had it); this PR widens the exposure. Suggested fix: also propagate when the effective signal (resolved as the engine does) is aborted.

Both are in separate files/layers from this PR's scope and pre-date it — I did not bundle them in.


八问

  1. 是什么问题:inline 引擎的 dispatch 只认 init.signal,忽略了 Request 对象自带的 signal——用 new Request(url, { signal }) 的调用方在响应头之前或流式过程中都无法取消,引擎继续转发、调用方的 body 读取一直挂起;已 abort 的 Request 也会被发往上游。
  2. 什么原因导致的:dispatcher 把 signal 解析成 init?.signal 一个来源,当 input 是 Request 且 init.signal 缺省时不回退到 input.signal。平台 fetch 语义(WHATWG:Request 构造器在 init["signal"] 不存在时取 input Request 的 signal)没有被镜像,Request 携带的 signal 被静默丢弃。Port billion-context-pi's no-server mode: inline in-process compression lane for pi #3284 引入取消时只观察了 init.signal。
  3. 影响范围与严重程度:影响所有向 inline 车道(pi / opencode)传带 signal 的 Request 的宿主。严重度:该场景下取消完全无效——请求跑完(烧 token)、abort 被忽略;不是「完全不能用」,是受支持的取消路径上的真实功能缺口。属既有缺陷。
  4. 怎么修复的:按平台同款规则解析有效 signal(init?.signal !== undefined ? init.signal : input instanceof Request ? input.signal : undefined),在请求转换前后各做一次 throwIfAborted(),然后复用既有的 pre-header 拒绝与 post-header 流/传输取消路径。与 WHATWG fetch 规范逐字对齐(规范文本 + Node 实测双重验证)。
  5. 是否有回归问题:无。唯一行为变化两处:①已 abort 的 init.signal 现在在 body 缓冲前就拒绝(原先先缓冲再拒绝)——严格改进、对齐 fetch 语义;②Request 携带 signal 且 init.signal 缺省时现在被尊重——即本修复。init.signal 覆盖(含 null)保持不变(override 控制测试钉住)。无高危模式(无语义漂移、无 fail-fast 变吞错、无阈值/时序/输出格式变化)。
  6. 是否破坏缓存:不涉及。改动在 dispatcher 入口的 signal 解析,不触碰缓存/重试/增量游标/去重键。
  7. 是否改了设计方向:不涉及。让引擎对齐平台 fetch 语义,不改架构/边界/数据流。
  8. 是否引入了新的配置:不涉及。无新 env/config/默认值。

判定:引擎边界上的修复正确、测试扎实、diff 干净(2 文件),回归面清白——CI 转绿即可合。上面两处相邻既有缺陷(withHeaders 丢 signal、pi-inline 自定义 abort reason 误判)超出本 PR 范围,建议另行跟踪;本 PR 不捆绑。

ework-agent and others added 13 commits November 14, 2026 04:35
The suite baseline preload (preload-storage-baseline.mjs) pins
BILI_STORAGE_UNIFIED="legacy" for the broad corpus. After ranxianglei#6480 flipped the
storage default to pure-sqlite, the notice-registry suite's visibleNotices
assertions expect the pure-sqlite default notice, which is gated on
unifiedSelectionIsSqlite (unified "sqlite" or absent) — the preload's
"legacy" made the notice silent, so the suite was born-red on master. Pin
the selection explicitly (ranxianglei#4011 lesson: a test that depends on the production
default passes locally but fails in CI).
…test

The dsh plugin lane (ci-e2e-dsh.yml) runs node --import tsx --test
WITHOUT the preload-storage-baseline.mjs preload, so BILI_STORAGE_UNIFIED
is unset and the storage mode defaults to 'full'. Under the pure-sqlite
default flip (5a80e2b, ranxianglei#6480), 'full' mode turns the JSON envelope
writer OFF — so the .content-store.json sidecar is never written, and
the test's readdirSync(namespace) hits ENOENT.

Pin BILI_STORAGE_JSON_WRITER=on in the test so the content-store
sidecar is written regardless of the storage mode. This keeps the
corrupted-store detection path exercised in both full and legacy modes.

Closes: dsh plugin lane red on master (run 43532980164, job 132255459726)
The pure-sqlite default flip (5a80e2b, ranxianglei#6480) turns the JSON envelope
writer OFF, so the .json session file this lane asserts on is never
written and 'proxy must persist the session' / 'no persisted session
carried compressed blocks' fail after the 20s wait. Pin the writer ON in
the spawned client's env (the bootstrapped proxy inherits it) so the
envelope this test was written for is still produced.

Closes: native opencode lane red on master (run 43547593803, job 132302438610)
Same pure-sqlite default flip (5a80e2b, ranxianglei#6480) as the opencode lane:
the JSON envelope writer is OFF, so 'proxy must persist the session
within 20000ms' fails — the on-disk .json session file is never written.
Pin the writer ON in the spawned dsh client's env (the bootstrapped
proxy inherits it).

Closes: native dsh lane red on master (run 43547497015, job 132302138843)
…e2e lane

Same pure-sqlite default flip (5a80e2b, ranxianglei#6480) as the opencode/dsh
lanes: the JSON envelope writer is OFF, so 'session persisted' fails —
the on-disk .json session file is never written. Pin the writer ON in
the spawned pi client's env (the bootstrapped proxy inherits it).

Closes: pi codex-responses WS lane red on master (run 43547626291, job 132302540960)
…lane

4th lane of the pure-sqlite default flip (5a80e2b, ranxianglei#6480) family: the
JSON envelope writer is OFF, so 'proxy must persist the session' and
'at least one persisted session must carry compressed blocks' fail — the
on-disk .json session file is never written. Pin the writer ON in the
spawned pi client's env (the bootstrapped proxy inherits it).

Closes: pi package-native lane red on ci-fixes (run 43558179054, job 132336647955)
Same pure-sqlite default flip (5a80e2b, ranxianglei#6480) family: the JSON writer
is OFF, so the per-session CCR content-store sidecar (.content-store.json)
that 'child persists its own companion and survives a proxy restart'
asserts on is never written (0 !== 1). The earlier b1bc17b pin only
covered tests/plugin-fork.test.ts, not this file. Pin the writer ON at
module load (before any SessionStore is constructed).

Closes: dsh plugin lane red on ci-fixes (run 43558179102, job 132336519539)
…lane

Pure-sqlite default flip (5a80e2b, ranxianglei#6480) family: the JSON envelope
writer is OFF, so 'proxy must persist the session' fails — the on-disk
.json session file is never written. Pin the writer ON in the spawned
opencode client's env (the bootstrapped proxy inherits it).
Pure-sqlite default flip (5a80e2b, ranxianglei#6480) family: the JSON envelope
writer is OFF, so the on-disk .json session envelope this lane asserts on
(exactly ONE persisted session under the raw key) is never written. Pin
the writer ON in the spawned dsh client's env.
Pure-sqlite default flip (5a80e2b, ranxianglei#6480) family: the JSON envelope
writer is OFF, so the on-disk .json session file this lane asserts on
(state survival across the OLD→NEW update) is never written. Pin the
writer ON in the spawned proxy's env (both update phases share envBase).
Resolve CI baseline conflicts using the current upstream test setup.
The pi-inline transport's catch only re-threw errors named "AbortError",
but the engine rejects with the signal's REASON — for a custom abort
(controller.abort(new Error(...))) that reason is not an AbortError, so a
caller cancellation was misclassified as an engine failure and degraded to
a direct (uncompressed) resend that also consumed the one-shot warning.
Classify by the effective signal's aborted state (resolved exactly as the
engine does) so any caller cancellation propagates. Pinned by a regression
test (custom-reason Request.signal cancel must propagate, never degrade).

@ranxianglei ranxianglei left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE — the core fix is correct and well-tested.

The dispatcher now resolves the effective signal exactly as the platform fetch does (init.signal if present, else the Request's signal, else none), throws if already aborted before and after the body conversion, and reuses the existing abort path (res.destroy + reject with the signal's reason). Verified against platform fetch semantics (probe: the Request constructor does NOT throw on an already-aborted signal — it creates a linked dependent signal — so the pre-aborted case is handled by the new throwIfAborted, not by a ctor throw). The test file is solid (real loopback upstream, exact-reason identity, both-direction override controls per #3617).

已顺手修:pi-inline transport 按 signal 状态(非 reason 名)分类取消——custom-reason 取消(engine 以 signal.reason reject,非 AbortError)不再被误判为引擎失败而降级直发(烧 token + 消耗一次性警告),并补回归测试。+ commit dc0efc3


结构化八问

  1. 是什么问题:内联引擎 dispatcher 只读 init.signal,忽略 Request 对象自带的 signal,调用方用 new Request(url, {signal})(无 init)取消的请求未被尊重,仍被转发上游。
  2. 什么原因导致的:signal 解析只有单一来源(init?.signal),Request 的 .signal 从未被读;toInlineReq 的 body 读取 .catch(()=>null) 吞掉 pre-aborted 读取,pre-aborted Request 被转发上游。
  3. 影响范围与严重程度:内联车道(pi + opencode inline)在宿主传带 signal 的 Request(无 init.signal)时触发;调用方取消被静默忽略(烧 token、忽略用户取消)——正确性问题,非数据丢失/可用性。
  4. 怎么修复的:按平台 fetch 语义解析有效 signal(init.signal 存在→用之;否则 Request.signal;否则无),body 转换前后 throwIfAborted(),复用现有 abort 路径(res.destroy + 以 reason reject);另修 pi-inline transport 按 signal 状态(非 reason 名)分类取消,使 custom-reason 取消被传播而非降级直发。
  5. 是否有回归问题:无——只添加之前缺失的 signal 处理,不改变现有 init.signal 路径(表达式与平台 fetch 语义一致,probe 验证);Defect B 只改取消分类,不改引擎失败降级路径。
  6. 是否破坏缓存:无。
  7. 是否改了设计方向:无——复用现有 abort 路径,无新机制。
  8. 是否引入新配置:无。

合并判定:可合并。核心修复正确、测试充分;Defect B 已顺手修(commit dc0efc3);Defect A(withHeaders 丢 signal,跨文件跨车道)已转 🔶 需要决策,见下条评论。

@ranxianglei

Copy link
Copy Markdown
Owner

🔶 需要决策

Defect A: withHeaders drops the Request's signal (cross-file, cross-lane)

src/agent/native-intercept.ts:433 — withHeaders(input, init, extra) rebuilds a Request-object input as:

new Request(input.url, { method: input.method, headers, body: input.body, duplex: "half" })

This omits signal, so the rebuilt Request gets a fresh (non-aborted) signal and the caller's cancellation is silently lost before it reaches the engine. The PR's engine fix is correct, but this adjacent path defeats it for the stamped-header case.

Reachability: only when the input is a Request object AND extra headers are non-empty. In the inline pi lane, headersFor is undefined, so extra is only non-empty after the pf watermark is seeded (2nd model request onward). The dsh lane sets headersFor but is not an inline lane (spawn mode).

Why not fixed on the spot: this is a cross-file (native-intercept.ts, a hot file) and cross-lane (all lanes) change — not "small" per the review-lane rule. The fix would be signal: input.signal added to the rebuild (safe per probe: the Request constructor does not throw on an already-aborted signal; it creates a linked dependent signal).

Decision needed: fold into this PR vs track separately.

@ranxianglei

Copy link
Copy Markdown
Owner

Review complete: APPROVE posted (core fix correct, well-tested). Defect B (pi-inline cancel misclassification) fixed on the spot + commit dc0efc3. Defect A (withHeaders signal drop, cross-file/cross-lane) routed to 🔶 需要决策. CI needs re-run on new head dc0efc3.

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Review of PR #6504 complete.

Verdict: APPROVE (posted as a formal GitHub review with the structured 8-question checklist).

What I did:

  1. Verified the core fix is correct: the dispatcher now resolves the effective signal exactly as platform fetch does (init.signal if present, else the Request's signal, else none), throws if already aborted before/after the body conversion, and reuses the existing abort path (res.destroy + reject with the signal's reason). Verified against platform fetch semantics via probe (the Request constructor does NOT throw on an already-aborted signal — it creates a linked dependent signal, so the pre-aborted case is handled by the new throwIfAborted, not by a ctor throw). The test file is solid (real loopback upstream, exact-reason identity, both-direction override controls per Test Blind Spot Retrospective: Reinforcing Negative Assertion Discipline, Tightening Fake Upstream Strictness, Thin Inst | 测试盲区复盘:补反向断言纪律、假上游严格化、薄安装冒烟、形态矩阵与生命周期覆盖 #3617).
  2. Fixed a small adjacent finding on the spot (Defect B): the pi-inline transport's catch misclassified a custom-reason cancel (the engine rejects with signal.reason, which for controller.abort(new Error(...)) is not an AbortError) as an engine failure → degraded to a direct uncompressed resend that also consumed the one-shot warning. Fixed by classifying the cancel by the signal's state (resolved exactly as the engine does), not the reason's name. Regression test added. Commit dc0efc3d pushed to the PR branch.
  3. Routed a cross-file/cross-lane finding to 🔶 需要决策 (Defect A): withHeaders in native-intercept.ts:433 drops the Request's signal when rebuilding a Request-object input (omits signal from the ctor). Not "small" (hot file, all lanes). Owner decision needed: fold into this PR vs track separately.

Verified:

  • All 14 targeted tests pass (6 inline-request-signal + 8 pi-inline) on the rebased head.
  • Typecheck clean.
  • The author's master-merge (new head c5b41b71ae) did not touch the PR's files; my fix was rebased cleanly on top.

Pending:

  • CI needs re-run on new head dc0efc3d02 (the author merged master, moving the head from abd1dd46fc).
  • needs-decision label: tooling unavailable in this environment (add-label.sh not found in workdir tools/; Gitea label API not implemented in the ework shim).

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Re-triggered review lane — the APPROVE review is already posted on this exact head (dc0efc3d02, review id 5903327407, submitted 2026-11-14T09:02:15Z). The PR is at the reviewed head; no new review needed.

CI status per mirror cache (fetched 09:19:11Z, may be stale): 4 success, 12 skipped, 14 queued. The task reports the PR as green.

The 🔶 需要决策 item (Defect A: withHeaders signal drop, cross-file/cross-lane) remains open for owner decision — fold into this PR vs track separately.

@ranxianglei

Copy link
Copy Markdown
Owner

Superseded by #6677 (merged as 3858b6956, 2026-11-14).

Landed into master via the real merge commit dc27f406c93 (Merge Afloat16/2026-11-12_inline-request-abort-signal (PR #6504)), so your original commits are preserved verbatim in master history — only hashes of the batch wrapper changed, your authorship and commit messages are intact.

The fix for #6503 is now in master. Closing this PR as superseded — thanks for the careful, well-tested fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

inline dispatch ignores cancellation carried by a Request object

2 participants