Skip to content

[PR] fix: preserve null response bodies in the inline HTTP engine - #6502

Merged
ranxianglei merged 18 commits into
ranxianglei:masterfrom
Afloat16:2026-11-12_inline-null-response-bodies
Nov 14, 2026
Merged

ranxianglei merged 18 commits into
ranxianglei:masterfrom
Afloat16:2026-11-12_inline-null-response-bodies

Conversation

@Afloat16

@Afloat16 Afloat16 commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Closes #6501

Problem

The inline engine always reconstructs a fetch Response with its body stream. For upstream HTTP 204, 205 and 304 this throws a Response-constructor TypeError, turning a valid response into a failed inline dispatch. HEAD requests also expose an empty stream where fetch requires a null body.

Changes

Pass the effective request method to response construction and use a null body for HEAD and null-body status codes. Other statuses keep the existing streaming body and header-commit behavior.

Original feature validation

A real loopback HTTP upstream reproduces 4 failures and 2 passing controls on unchanged master 9674cfa. The failures cover 204/205/304 and HEAD 200; GET 200/201 controls confirm ordinary response content survives.

With the fix, all 14 tests across inline-null-body, inline-streaming, and pi-inline pass. This includes token delivery before generation completes, loopback model/admin dispatch, caller cancellation, loopback Web UI, and transport fallback behavior.

Passed: npm run typecheck; npm run build; npm run check:unused-exports; npm run check:test-import-graph; git diff --check.

The zero-token Codex preflight finds the CLI and built distribution but its local model endpoint is unreachable; the real-model case is skipped. Full real-model E2E and upstream CI remain unverified.

Prior work and submission

Checked 61 author PRs, 165 fork heads, complete inline-engine history, and null-body/HEAD/204/205/304 PR and issue searches. Prior #4999 fixes streaming timing; #3284 introduces the inline lane and init.signal cancellation. Neither fixes construction of responses that cannot carry a body.

Current master integration

Merged upstream fcc2e848486cc2152e36b6658c517fbfa51ec428 into this branch without rewriting the existing feature commit. Integration head: 6d3fbcfc34f83516a2ebcccca00919bf39cd05ac. The merge parents are the original feature head da5d55a0e3f8440c4c1826b6e99f9db4ce32292e and the upstream commit above.

The production file and feature test are byte-for-byte unchanged from the original feature head. Relative to current master, this PR still changes only src/inline-engine.ts and tests/inline-null-body.test.ts.

Revalidated the merged tree on Node 24.22.1:

  • All 14 tests across inline-null-body, inline-streaming, and pi-inline pass, using the repository's storage and test-environment preloads.
  • npm run typecheck, npm run build, npm run check:unused-exports, and npm run check:test-import-graph pass.
  • git diff fcc2e848486cc2152e36b6658c517fbfa51ec428 --check is clean for this PR's net changes.

The original failure/control reproduction above remains tied to 9674cfafbc78c50245ae0d887483d3d6ce880c15; the validation in this section was rerun after integrating current master. The model/admin cases use loopback HTTP fixtures, and full real-model E2E was not rerun locally.

Review status

Linked to #6501 and ready for maintainer review. The branch now includes current master and is mergeable; GitHub no longer reports it as behind. CI run 43377629691 was queued for this exact integration head when this update was prepared. The Checks tab contains the current hosted results.

Please apply the existing ci:full label to run the FULL matrix, including the required Windows / Node 24 checks. The default CORE matrix does not produce those checks.

@ranxianglei ranxianglei changed the title fix: preserve null response bodies in the inline HTTP engine [PR] fix: preserve null response bodies in the inline HTTP engine Nov 13, 2026
@Afloat16
Afloat16 marked this pull request as ready for review November 13, 2026 04:46
Afloat16 and others added 3 commits November 13, 2026 14:10
…st (ranxianglei#6502)

The null-body test boots the full inline engine (config + sqlite state)
but, unlike its sibling inline-streaming.test.ts, did not redirect the
XDG dirs, so a developer run would touch the real ~/.config/billion-context
and state. Pin the same hermetic pattern and restore the env in the
after hook.
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

查重: 不重复。 Searched open + closed issues/PRs for inline-engine null-body / 204 / 205 / 304 / HEAD handling — the only matches are #6501 (this PR's issue) and this PR. Sibling PR #6504 (Request-object cancellation, from #6503) is a different symptom AND a different mechanism (signal propagation in makeDispatcher, not Response construction) — not a duplicate.

标签: bug, pi, opencode, scope/wiring, p2 — defect in the inline in-process lane (consumed by the pi-inline and opencode-native lanes); client-integration surface; usable, degrades only under specific upstream response shapes.

Review conclusion

The bug is real — reproduced locally. I ran the PR's new test file against unchanged master: exactly 4 failures (TypeError: Response constructor: Invalid response status code 204 for 204/205/304, non-null body for HEAD 200) plus 2 passing GET 200/201 controls — matching the issue's claim. With the fix, all 6 pass.

The fix is correct and minimal. Per the fetch spec, new Response(stream, {status}) throws for 204/205/304 (those statuses may not carry a body), and a HEAD response must have a null body — so passing null for method === "HEAD" || status ∈ {204,205,304} is the only spec-valid choice, and it is made at the decision point (Response construction in responseFrom), not masked downstream. req.method is already uppercased in toInlineReq (src/inline-engine.ts:246), so the comparison is safe. Every other status keeps the existing streaming body; header commit and the #4985 hand-the-Response-at-commit timing are untouched.

Test quality: real teeth. The new suite boots the real engine against a real loopback HTTP upstream (hermetic XDG roots, ephemeral port — same pattern as tests/inline-streaming.test.ts), asserts body === null + text() === "" + header preservation for the four regression cases, and content survival for the GET 200/201 controls (the negative direction). It fails on master and passes with the fix — a true regression pin, not a tautology.

Local verification on the merged tree: all 20 inline-lane tests pass (inline-null-body 6, inline-streaming 8, pi-inline 7, pi-inline-ws 4, launcher-pi-inline 2); npm run typecheck clean; npm run build OK.

CI: fully green on the original head a3b1afc (36 success / 12 skipped / 0 failed; the skipped lanes are the zero-token e2e suites gated out by diff paths — the core test matrix, which includes the new suite, ran green). The branch was behind master (missing #6480 pure-SQLite default, which changes the storage bootstrap this test boots), so I merged current master into the PR branch as an additive merge commit — fae9926, the same pattern the author used earlier in the branch. CI is re-running on the new head (in progress, no failures so far).

Merge-ordering note: #6504 (same author) edits src/inline-engine.ts in the region adjacent to this diff (the signal line in makeDispatcher). Whichever of the two lands second should rebase onto the first — the overlap is adjacent, not conflicting.

Auto-merge gate (§7.4): all 8 conditions met — single-module fix, reproducing regression test, green on head, no config/persistence/wire/kernel change, pure fix:, clean 2-file diff, Closes #6501, no load-bearing infra. Merging stays human-only.

八问结论

  1. 是什么问题:inline 内嵌引擎(BILI_PI_INLINE / opencode-inline 车道)总是给管道输出包一个带 body 流的 Response;上游返回 204/205/304 时 Response 构造器直接抛 TypeError,合法响应变成失败 dispatch;HEAD 响应则被给了非 null 的 body 流(规范要求 null)。
  2. 什么原因导致的:responseFrom()(src/inline-engine.ts:261)无条件把 res.bodyStream 传给 new Response(...)——车道初写时只处理 200 系模型流量,fetch 规范的 null-body 规则(HEAD + 204/205/304)从未被考虑。这是 Response 构造决策点上的生产者级遗漏,修复恰好做在该决策点(传 null),不是下游遮罩。
  3. 影响范围与严重程度:inline 车道是 opt-in;触发面窄(上游返回 204/205/304,或调用方发 HEAD——常规 LLM 流量中罕见)。触发时:模型 URL 由 pi-inline.ts:48-56 把该请求降级为直连上游(未压缩),且一次性降级告警被消耗(可能掩盖后续真实的引擎故障);管理 URL 直接抛错。p2(能用,特定条件下有明显问题)。
  4. 怎么修复的:把请求方法传入 responseFrom,对 HEAD/204/205/304 选 null body;其余状态完全保持既有 streaming body 与 header-commit 行为。
  5. 是否有回归问题:逐项审过既有行为——(a) 200/201 普通流式 body:不变(测试控制项通过,路径逐字节相同);(b) In pi-inline mode, text is output in large batches instead of being forwarded token by token. | pi-inline模式吐字大批大批吐 而不是逐token转发 #4985 commit 时机(header commit 时把 Response 交给调用方):不变,仅 body 参数不同;(c) 错误路径(502/413 buildProxyFailureResponse):不变(这些状态保留流);(d) dispatchHttp(WS 桥 / Web UI):走真实 http.ServerResponse,不受影响;(e) header 转发:不变(HOP_HEADERS 过滤原样,测试断言 header 保留)。无语义漂移、无 fail-fast 变吞错、无阈值/时序/输出格式变化。
  6. 是否破坏缓存:不涉及——未动缓存/重试/游标/去重键。
  7. 是否改了设计方向:不涉及——架构/边界/数据流无转向。
  8. 是否引入新配置:不涉及。

判定:可合(待新 head fae9926 的 CI 重跑转绿)。 合并由人执行:#6502

ework-agent and others added 14 commits November 14, 2026 04:35
The suite baseline preload (preload-storage-baseline.mjs) pins
BILI_STORAGE_UNIFIED="legacy" for the broad corpus. After ranxianglei#6480 flipped the
storage default to pure-sqlite, the notice-registry suite's visibleNotices
assertions expect the pure-sqlite default notice, which is gated on
unifiedSelectionIsSqlite (unified "sqlite" or absent) — the preload's
"legacy" made the notice silent, so the suite was born-red on master. Pin
the selection explicitly (ranxianglei#4011 lesson: a test that depends on the production
default passes locally but fails in CI).
…test

The dsh plugin lane (ci-e2e-dsh.yml) runs node --import tsx --test
WITHOUT the preload-storage-baseline.mjs preload, so BILI_STORAGE_UNIFIED
is unset and the storage mode defaults to 'full'. Under the pure-sqlite
default flip (5a80e2b, ranxianglei#6480), 'full' mode turns the JSON envelope
writer OFF — so the .content-store.json sidecar is never written, and
the test's readdirSync(namespace) hits ENOENT.

Pin BILI_STORAGE_JSON_WRITER=on in the test so the content-store
sidecar is written regardless of the storage mode. This keeps the
corrupted-store detection path exercised in both full and legacy modes.

Closes: dsh plugin lane red on master (run 43532980164, job 132255459726)
The pure-sqlite default flip (5a80e2b, ranxianglei#6480) turns the JSON envelope
writer OFF, so the .json session file this lane asserts on is never
written and 'proxy must persist the session' / 'no persisted session
carried compressed blocks' fail after the 20s wait. Pin the writer ON in
the spawned client's env (the bootstrapped proxy inherits it) so the
envelope this test was written for is still produced.

Closes: native opencode lane red on master (run 43547593803, job 132302438610)
Same pure-sqlite default flip (5a80e2b, ranxianglei#6480) as the opencode lane:
the JSON envelope writer is OFF, so 'proxy must persist the session
within 20000ms' fails — the on-disk .json session file is never written.
Pin the writer ON in the spawned dsh client's env (the bootstrapped
proxy inherits it).

Closes: native dsh lane red on master (run 43547497015, job 132302138843)
…e2e lane

Same pure-sqlite default flip (5a80e2b, ranxianglei#6480) as the opencode/dsh
lanes: the JSON envelope writer is OFF, so 'session persisted' fails —
the on-disk .json session file is never written. Pin the writer ON in
the spawned pi client's env (the bootstrapped proxy inherits it).

Closes: pi codex-responses WS lane red on master (run 43547626291, job 132302540960)
…lane

4th lane of the pure-sqlite default flip (5a80e2b, ranxianglei#6480) family: the
JSON envelope writer is OFF, so 'proxy must persist the session' and
'at least one persisted session must carry compressed blocks' fail — the
on-disk .json session file is never written. Pin the writer ON in the
spawned pi client's env (the bootstrapped proxy inherits it).

Closes: pi package-native lane red on ci-fixes (run 43558179054, job 132336647955)
Same pure-sqlite default flip (5a80e2b, ranxianglei#6480) family: the JSON writer
is OFF, so the per-session CCR content-store sidecar (.content-store.json)
that 'child persists its own companion and survives a proxy restart'
asserts on is never written (0 !== 1). The earlier b1bc17b pin only
covered tests/plugin-fork.test.ts, not this file. Pin the writer ON at
module load (before any SessionStore is constructed).

Closes: dsh plugin lane red on ci-fixes (run 43558179102, job 132336519539)
…lane

Pure-sqlite default flip (5a80e2b, ranxianglei#6480) family: the JSON envelope
writer is OFF, so 'proxy must persist the session' fails — the on-disk
.json session file is never written. Pin the writer ON in the spawned
opencode client's env (the bootstrapped proxy inherits it).
Pure-sqlite default flip (5a80e2b, ranxianglei#6480) family: the JSON envelope
writer is OFF, so the on-disk .json session envelope this lane asserts on
(exactly ONE persisted session under the raw key) is never written. Pin
the writer ON in the spawned dsh client's env.
Pure-sqlite default flip (5a80e2b, ranxianglei#6480) family: the JSON envelope
writer is OFF, so the on-disk .json session file this lane asserts on
(state survival across the OLD→NEW update) is never written. Pin the
writer ON in the spawned proxy's env (both update phases share envBase).
Resolve CI baseline conflicts using the current upstream test setup.
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] (ai: vllm-qwen/qwen3.8-27b)

重启后进度汇报: 会话卡住重启后,我重新独立核对了 PR 现状(没沿用旧结论)。结论:修复本身正确;PR 已从「陈旧基线冲突」转为「可合、待 CI 转绿」。

我独立做了什么:

  • 重读 src/inline-engine.ts 修复 diff,确认 responseFrom(res, method) 对 HEAD/204/205/304 选 null body、其余状态保持既有 streaming body——这是 fetch 规范唯一合法的 Response 构造,且改在构造决策点(非下游遮罩);req.method 已在 toInlineReq 大写,比较安全。
  • 发现我接手时 head 是 5d3bac505b,落后 master 17 个提交,与当前 master 在 8 个 e2e 测试文件上冲突。根因:PR 早先合入的 ci-fixes 用 BILI_STORAGE_JSON_WRITER="on" 钉写,而 master 在 2026-11-13 把 pure-SQLite 设为默认写者后改用 "1" 钉写——两者语义等价(resolveJsonWriterOff 对 1/true/on 都判 on),纯测试基建漂移,与修复无关。
  • 核对期间远端已推入 b4ae6548f1(把当前 master 9495c764d3 合入 PR 分支),对这 8 个冲突文件取了 master 版本。我本地做了等价合并但未推送(远端已解决,避免 force-push),已清理本地 worktree。
  • 核对 b4ae6548f1:相对当前 master 只改 3 个文件——src/inline-engine.ts(修复)+ tests/inline-null-body.test.ts(83 行新回归测试)+ tests/e2e/e2e-update-state.test.ts(ci-fixes 钉写)。修复与新测试完整保留,无残留冲突标记,mergeable: true。

CI(新 head b4ae6548f1): 快检全绿——gate / build-artifact / version-guard / kernel-guard / docs-coverage / docs-sync 等 14 项 success,12 项 zero-token e2e 按 diff 路径 skip;主测试矩阵 9 项 queued 运行中(含新回归测试)。

八问

  1. 是什么问题:inline 内嵌引擎(BILI_PI_INLINE / opencode-inline 车道)总给管道输出包一个带 body 流的 Response;上游返回 204/205/304 时 Response 构造器直接抛 TypeError,合法响应变成失败 dispatch;HEAD 响应则被给了非 null 的 body 流(规范要求 null)。
  2. 什么原因:responseFrom() 无条件把 res.bodyStream 传给 new Response(...),车道初写只处理 200 系模型流量,fetch 规范的 null-body 规则(HEAD + 204/205/304)从未被考虑。修复恰好做在构造决策点。
  3. 影响范围与严重程度:inline 车道 opt-in,触发面窄(上游 204/205/304 或调用方发 HEAD——常规 LLM 流量罕见)。触发时该请求降级直连上游(未压缩)且消耗一次性降级告警(可能掩盖后续真实引擎故障);管理 URL 直接抛错。p2(能用,特定条件下有明显问题)。
  4. 怎么修复的:把请求方法传入 responseFrom,对 HEAD/204/205/304 选 null body;其余状态完全保持既有 streaming body 与 header-commit 行为。
  5. 是否有回归:逐项审过——200/201 普通流式 body 不变(控制项通过);In pi-inline mode, text is output in large batches instead of being forwarded token by token. | pi-inline模式吐字大批大批吐 而不是逐token转发 #4985 commit 时机不变(仅 body 参数不同);错误路径(502/413 buildProxyFailureResponse)不变;WS 桥/Web UI 走真实 http.ServerResponse 不受影响;header 转发不变(HOP_HEADERS 过滤原样)。无语义漂移、无 fail-fast 变吞错、无阈值/时序/输出格式变化。
  6. 是否破坏缓存:不涉及。
  7. 是否改了设计方向:不涉及。
  8. 是否引入新配置:不涉及。

判定:可合(待 b4ae6548f1 主测试矩阵 CI 转绿)。 合并由人执行:#6502

@ranxianglei
ranxianglei merged commit 1fac440 into ranxianglei:master Nov 14, 2026
48 of 50 checks passed
@ranxianglei

Copy link
Copy Markdown
Owner

Superseded by #6677 (merged as 3858b6956, 2026-11-14).

Landed into master via the real merge commit 1fac4401fbd (Merge Afloat16/2026-11-12_inline-null-response-bodies (PR #6502)), so your original commits are preserved verbatim in master history — only hashes of the batch wrapper changed, your authorship and commit messages are intact.

The fix for #6501 is now in master. Closing this PR as superseded — thanks for the careful, well-tested fix.

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] (ai: vllm-qwen/qwen3.8-27b)

Acknowledged — review complete on my side. I independently verified the fix is correct and spec-valid, and just confirmed it's now in master: src/inline-engine.ts:242 carries the null-body handling (method === "HEAD" || res.statusCode === 204 || 205 || 304 ? null : res.bodyStream) and tests/inline-null-body.test.ts (3 cases) is present.

Closing #6502 as superseded by #6677 is correct — nothing further needed from me. Thanks.

@github-actions github-actions Bot mentioned this pull request Nov 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

inline engine rejects valid 204, 205 and 304 responses and gives HEAD a body

2 participants