Repository navigation
[PR] fix: release request input buffers before long-running responses - #4766
ranxianglei wants to merge 14 commits into
Conversation
Three retention sites kept input-only data alive across the whole
response stream of large requests:
1. readBody never removed its data/end/error listeners, so the closure
holding every network chunk stayed attached to the request for its
whole life; a close before end left the promise dangling forever
(admission leak). All four settlement paths now detach their own
listeners and drop the chunk array; close-before-end rejects with an
explicit abort error instead of hanging.
2. rememberPluginMessages took the raw inbound Buffer AFTER forwarding
and re-parsed + re-walked it. The raw-wire hash and text-comparability
verdict are now computed BEFORE prepare mutates the parsed graph
(anchored on the original parse, gated on the plugin lane) and only
those two scalars cross to the existing post-forward bookkeeping
point; hash-gate semantics are unchanged.
3. handleRequest's frame retained the raw body, decoded text, parsed
graph and the overflow-refold preparation closure across the entire
response stream. forward() now takes an onRetriesSettled callback,
fired exactly once when every upstream attempt has finished (initial
send, role ladder, pck probe, image-count retry, overflow refold),
before the first response byte; handleRequest releases its input
cells there. Accepted outbound bodies (fake-completion/blind retries,
echo provenance) and the session pin stay through the response by
design.
Behavior change: a client disconnect mid-body now surfaces as a
rejected readBody ("request aborted before body completed") instead of
a permanently pending promise; the admission slot is released either
way. Previously the slot leaked until process exit.
Closes #4753
…nput (#4753) Review finding on this PR: affinityMessagesCache (chain-fingerprint / resume-detection memo) aliases parsed.messages or input/contents, so the inbound message array stayed pinned on the suspended handleRequest frame across the whole response stream even after parsed was released. Null it in releaseRequestInput alongside the other input cells. Verified: typecheck clean, tests/issue4753-input-lifetime.test.ts + tests/proxy-readbody.test.ts 30/30 green.
(ai: vllm-qwen/qwen3.8-27b) 查重: 重复 #4756(同症状 + 同机制 + 同根因,先来后到 #4756 优先)。
标签: pr(原有)、duplicate(与 #4756 同机制并行)、bug(内存驻留缺陷)、scope/wiring(请求管道输入生命周期)、p2(功能可用,大请求长响应下慢性内存增长,有 OOM 风险但不破坏功能)。 独立审核(未复用 #4753/#4756 任何一侧的既有结论,diff 两边都重读了)本 PR 实现正确,逐项核实过:
本地(head 1544b5a):typecheck/build clean;issue4753-input-lifetime + proxy-readbody 30/30;refold 族 + plugin-fork + session-transient-memory + context-overflow 共 120/120;e2e-overflow-selfheal 3/3、fix-arm-headroom-clamp 3/3。 审核中发现一个真实缺口(已修,已推): 与 #4756 的对比(同机制家族):
八问
收工判定代码质量通过(缺口已补)。建议合 #4756(先来后到 + 覆盖更完整),前置条件:把它 merge 一次 origin/ci-fixes 清掉 5 片继承红(哨兵批量或作者自行均可);本 PR 已打 duplicate,等 owner 决定关闭。若 owner 偏好本 PR 更干净的 API,缺口已在分支上修复(head 139c9db),可直接用。两个 PR 我都不会执行关闭。 |
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.10511-pr.4766.73ca92cThis exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr4766.tgz
npm install -g packageInstall instructions are refreshed on each push. |
Resolve conflicts from 1340 commits of master drift: - src/server/handle.ts: union import (computeRawWireBookkeeping + pluginCompressionProxyHeader); the forward() call keeps the releaseRequestInput closure + #4753 comment and appends it to master's new onWireBody arg list. - src/server/relay.ts: forwardCaptured() signature — master's throttleModelHint + onWireBody, then onRetriesSettled (last). - tests/issue4753-input-lifetime.test.ts: the settlement-hook probe forward() calls gain two positional undefineds (throttleModelHint, onWireBody) before onRetriesSettled; the frozen comparableHistory oracle is updated to track master's #4266 thinking-family handling (thinking block fixture now expects textComparable=true — the canonical walker treats string-payload thinking as comparable by view).
The #4753 bookkeeping type is only consumed inside src/plugin.ts (return type of computeRawWireBookkeeping, param of rememberPluginMessages); no other module imports the type name. Drop the leading export so the unused-exports gate passes.
[bot] (ai: vllm-qwen/qwen3.8-27b) Resolved the merge conflict and the red CI check on this PR in one pass. Live state verified against the GitHub API on head
Two commits pushed to
Note: the PR is currently 八问
判定:冲突已解、CI 全绿,PR 可合( |
[bot] (ai: vllm-qwen/qwen3.8-27b) Finding: this PR is now redundant — the #4753 fix is already in master. While resolving the new merge conflict (master advanced 570 commits since my last merge), I found that master has already merged the #4753 fix — via a cherry-pick of the earlier duplicate PR #4756 (z0d0dOr's fork) plus two follow-up fixes:
Why our PR is redundant (and inferior):
What I did this run:
Recommendation: close #4766 as redundant (the fix is already in master via #4756 + follow-ups). I can't close PRs (owner-only). #4756 is likewise now redundant (it was the source of the cherry-pick). 八问:
判定: 建议 owner 关闭 #4766(冗余,修复已在 master)。本分支未做任何推送,保持原样。 |
[ework] CI fix verified green ✓ |
The probe anchor PROXY_TOOL_AT=6 was calibrated on 2026-10-31 for #4521 (schedule {4,6,8,9}). The nudge scheduling has shifted again — nudge now lands at {3,5,11,14}. Re-calibrate the anchor to the next nudge-carrying request pair (5 → tool round, 6 → fold round). The fold/proxy-tool-round mechanics under test are unchanged. Closes: master-red on test (windows-latest, 22, 3) and (ubuntu-latest, 22, 1).
The probe's hardcoded request-index anchors (PROXY_TOOL_AT/FOLD_AT) broke every time the nudge gate's schedule shifted. The last recalibration (245ecb1) was measured on a direct local run (timestamps ON by default since #4313), but CI runs the suite preload pin BILI_MESSAGE_TIMESTAMPS="0" (timestamps OFF) — a different wire shape, a different schedule (no nudge at index 5) — so it passed locally and failed on every CI shard. Root-cause fix instead of another anchor whack-a-mole: - Arm the fold on the first request carrying the ephemeral nudge with >=11 tagged refs; fold on the re-request after the acp_status round. The probe now materializes the #2929 scenario wherever the schedule places it. - Pin compress.messageTimestamps:false so the probe's wire shape is hermetic regardless of how the suite is launched (matches the CI preload pin). Supersedes the 245ecb1 recalibration. Verified: passes with and without the preload pin, same deterministic schedule (nudge at inputs[7], fold at inputs[8]).
…already in master) The #4753 input-buffer-release fix landed in master independently via a cherry-pick of the earlier duplicate PR #4756 (6c0079b) plus two follow-up fixes (bdf1ac0 marker-slip repair, 3728f96 watermark-fastpath regression). Master's version is the merged, tested, strictly-better one (releases runPreparedPipeline; handles bodyText conditionally). All 4 source-file conflicts resolved by adopting master's version; the stale test file tests/issue4753-input-lifetime.test.ts (imports the now-removed computeRawWireBookkeeping API) is deleted. The merge result is byte-identical to master, so this PR is now a no-op and should be closed as redundant (see thread).
[bot] (ai: vllm-qwen/qwen3.8-27b) Conflict resolved (no-op merge) — the PR is now redundant and should be closed. This run I completed the conflict resolution I'd aborted last run. The root cause (found last run): master already merged the #4753 fix — via a cherry-pick of the earlier duplicate PR #4756 ( What I did this run:
Live state (verified via single GitHub API GET this run): Result: the PR's diff vs master is now empty — a true no-op. Merging it would change nothing. Recommendation: close #4766 as redundant (the #4753 fix is already in master via #4756 + follow-ups). I can't close PRs (owner-only). #4756 is likewise now redundant (it was the cherry-pick source). 八问:
判定: 冲突已解决(no-op merge,head |
(ai: vllm-qwen/qwen3.8-27b) CI red resolved — inherited master red, fixed by merging the ci-fixes sentinel lane. All three failing test shards (ubuntu-24/3, ubuntu-22/3, windows-22/4) failed on the SAME single test: 七问
|
ranxianglei
left a comment
There was a problem hiding this comment.
Verdict: REQUEST_CHANGES — this PR is redundant and should be closed, not merged.
This is a disposition issue, not a code defect — there is nothing to fix in the code. I independently verified the current head (ef134e385) against origin/master and origin/ci-fixes rather than relying on prior thread claims.
1. The #4753 fix (this PR's stated purpose) is already in master.
6c0079b5f0"fix: release input-only request data before streaming (#4753)" — a cherry-pick of the earlier duplicate PR #4756 (9c8c72c78) — is in master.bdf1ac0d57"fix(handoff): complete #4753 release-wiring dropped by a marker slip" — in master.3728f968fd"fix: keep the raw body text available to the plugin full-resend watermark fastpath" — in master.- master's
src/server/handle.ts:3482carriesreleaseRequestInput;src/plugin.ts:570carriespreparePluginWireMetadata. - The PR's diff vs master contains none of the #4753 source files (
plugin.ts/server.ts/handle.ts/relay.ts). The no-op merge2f76ccf1aaadopted master's version of all four, so this PR's own #4753 implementation is gone from the diff.
2. The only remaining diff vs master is 2 files, both byte-identical to the ci-fixes branch.
tests/issue2929-codex-fold-double-miss.test.ts— the #2929 fold-probe root fix (schedule-independentfold.statemachine + hermeticcompress.messageTimestamps:falsepin). Byte-identical to ci-fixes tip7247410976.reference/architecture.md— one module-map line forsrc/responses-system-sections.ts(#6922). Present in ci-fixes.- Both converge to master via the dedicated ci-fixes→master PR #6974 (open, head=
ci-fixes, base=master, carrying exactly these 2 files) — not through this PR.
3. Correct disposition.
Merging this PR adds nothing: its purpose (#4753) is already in master, and its remaining content is redundant with #6974. The clean action is to close #4766 as redundant (owner action — agents cannot close). The #4753 fix ships via the already-merged master commits; the #2929 test fix + docs line ship via #6974.
Scope checks: no kernel/ changes, no kernel-change declaration, no config surface, no wire/persistence change. CI green on head ef134e385 (0 failed / 0 pending). No small findings to fix on the spot — the 2-file diff is the clean ci-fixes canonical version.
八问
- 是什么问题: 这个 PR 现在冗余了——它声称要修的 #4753(大请求的输入缓冲跨整个响应流驻留 → 慢性内存增长/OOM)已经在 master 里。
- 什么原因导致的: 更早的重复 PR #4756 先被 cherry-pick 进 master(
6c0079b5f0),又带两个后续修复(bdf1ac0d57补 marker slip 丢掉的接线、3728f968fd修 bodyText 无条件释放砸掉 watermark fastpath);本 PR 的 CI 工程师解冲突时把 4 个源文件全部采用 master 版本(no-op merge2f76ccf1aa),自己那份 #4753 实现就从 diff 里消失了,只剩后来为骑掉继承红而 merge 进来的 ci-fixes 内容。 - 影响范围与严重程度: 对用户零影响(#4753 修复已在 master);这个 PR 本身是冗余条目,合入不会改变任何东西,只会在 #6974 之前把同样的 ci-fixes 内容提前带进 master(无害但多余)。
- 怎么修复的: 无需修复——#4753 修复已在 master;剩余 diff(#2929 测试根治 + 一行架构文档)与 ci-fixes 分支(
7247410976)逐字节一致,走 #6974 收敛进 master。 - 是否有回归问题: 无——本 PR 相对 master 没有引入任何新变更(4 个源文件零 diff),合入等于 no-op;真正要防的反而是「强行合入本 PR 旧版本」会回退 master 的
3728f968fd修复,当前 head 已避免这一点。 - 是否破坏缓存: 不涉及——无出站字节/缓存/重试/游标改动。
- 是否改了设计方向: 不涉及——无架构/边界/数据流转向。
- 是否引入新配置: 不涉及——无新 env/config/默认值。
判定: 建议 owner 关闭 #4766(冗余:#4753 修复已在 master,剩余内容走 #6974 收敛)。本审核不动代码(2 文件 diff 是 ci-fixes 规范版,无可修小缺陷),不执行关闭(owner-only)。
[bot] 🏷 Review complete — verdict: PR #4766 is redundant, recommend close (owner action). I independently verified head
Full findings + 八问 in my COMMENT review above ( 八问
判定: 建议 owner 关闭 #4766(冗余)。本审核不动代码、不执行关闭(owner-only)。 |
Closes #4753
What
Three input-lifetime retention sites kept request-side memory alive across the entire response of large requests:
readBody (src/server.ts) never removed its data/end/error listeners, so a closure holding every network chunk stayed attached to the request for its whole life; a close before end left the promise pending forever (admission-slot leak). Now every settlement path (success / error / abort / oversize) detaches its own listeners and drops the chunk array, and close-before-end rejects with an explicit abort error.
rememberPluginMessages (src/plugin.ts) received the raw inbound Buffer after forwarding and re-parsed + re-walked it for the text-comparability verdict. The raw-wire hash and verdict are now computed before prepare mutates the parsed graph (anchored on the original parse, gated on the plugin lane), and only those two scalars cross to the existing post-forward bookkeeping point. Hash-gate semantics unchanged.
handleRequest's frame (src/server/handle.ts) retained the raw body, decoded text, parsed graph and the overflow-refold preparation closure across the entire response stream. forward() now takes an onRetriesSettled callback fired exactly once when every upstream attempt has finished (initial send, role ladder, pck probe, image-count retry, overflow refold), before the first response byte; handleRequest releases its input cells there.
Per the issue scope: accepted outbound bodies (fake-completion/blind retries, echo provenance) and the session pin stay through the response. No admission controls, no storage changes, no kernel changes, no config surface added.
Behavior change (disclosure)
Validation