Skip to content

[PR] fix(server): release bodyText after the prepare path consumed it — restores the watermark fastpath (CI shard #5418 red since #6482) - #6646

Closed
ranxianglei wants to merge 0 commit into
masterfrom
fix-5418-bodytext-release
Closed

ranxianglei wants to merge 0 commit into
masterfrom
fix-5418-bodytext-release

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

What & why

CI shard "#5418 A/B byte-equality" (tests/issue5403-watermark-fastpath.test.ts) has been red on master since the #6482 batch merge, and it goes red on every PR that merges current master — including all of Afloat16's wave-5 branches, which is why 40 unrelated PRs each carry one red shard.

Root cause: #4753's fix (commit 6c0079b, "release input-only request data before streaming") set bodyText = undefined in the sync region of handleRequest before the awaited prepare path ran. The watermark fastpath gate (src/server/prepare-openai.ts:135, if (bodyText !== undefined && ...)) therefore never fired — the feature was silently dead, not failing loudly. Runtime probes (set site → release → consume, in order) confirm the release precedes the only consumer.

Changes

  1. src/server/handle.ts: move the fix: release request input buffers before long-running responses #4753 release to the last consumer — inside runPrepareInner, right after prepareOpenai has read the parsed body (the preparedOut ternary). All other consumers (auth bypass, bodyMightCarryImages, preparedPipeline, forward, streaming path) still see the released text exactly as before; only the fastpath regains it.
  2. tests/issue5403-watermark-fastpath.test.ts: turn 0's "side lane must not stamp" pin pre-dates c15d9ce ("a tool-less MAIN turn must not be mistaken for a dsh sidecar"), which deliberately re-routes a tool-less main turn with a large output budget (64000 here) off the verbatim side lane. That turn is now a full-pipeline turn in both phases and stamps normally — pin it to the measured wire total like the other turns. The side-lane exemption itself stays pinned in tests/issue5656-single-shot-sidecar.test.ts ("side lane must never stamp: no x-bili-pf-turn").

Verification (node v22.23.4, CI preload set)

  • #4753 suites: request-input-lifetime 5/5, proxy-readbody 8/8, plugin-wire-metadata 3/3 — the memory release still happens, just after the last read.
  • tests/issue5403-watermark-fastpath.test.ts: 2/2 green (was B stamp 1: measured wire total — null !== 4).
  • A/B byte equality across all 8 turns holds (phase A = legacy pipeline, phase B = watermark fastpath with per-turn stamps).

@ranxianglei ranxianglei changed the title fix(server): release bodyText after the prepare path consumed it — restores the watermark fastpath (CI shard #5418 red since #6482) [PR] fix(server): release bodyText after the prepare path consumed it — restores the watermark fastpath (CI shard #5418 red since #6482) Nov 14, 2026
@ranxianglei
ranxianglei force-pushed the fix-5418-bodytext-release branch from f162293 to c26e001 Compare November 14, 2026 07:39
@ranxianglei

Copy link
Copy Markdown
Owner Author

Superseded by #6640 (merged as e5e1647): same root cause, same fix — release bodyText after prepareOpenai consumed it. Master is now green on the 5418 shard; closing this duplicate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant