Skip to content

[PR] batch-B: wire-format preservation — 3 fixes absorbed (original PRs superseded) - #7445

Open
ranxianglei wants to merge 4 commits into
masterfrom
batch-B-2026-11-21
Open

ranxianglei wants to merge 4 commits into
masterfrom
batch-B-2026-11-21

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

What this batch is

One PR absorbing three small, already-reviewed wire-format preservation fixes. Each original PR was cherry-picked with -x, so the original authorship and full commit messages are preserved. The three fixes all keep what the model/client already got on the wire instead of dropping or corrupting it in proxy-synthesized frames.

Absorbed original PRs

Original PR Original author What it fixes
#7402 Afloat16 Responses text events carried an item address but no part address (content_index), so the official OpenAI Node SDK threw missing content at index undefined on ordinary answers (e.g. text ending in a buffered m). Now sets content_index: 0 on single-part text/marker lifecycle frames, preserves the source index in proxy and plugin tail flushing (incl. retry reframing), and adds wire-contract rule WC-026 with real OpenAI-SDK consumption regressions.
#7406 Afloat16 The opt-in fake-completion path treated its capped error-body buffer (default 16 MiB) as a complete successful response and silently truncated bigger successes — text, real tool calls, usage and terminal events could vanish, and a JSON success could become unparseable. Now the buffer is inspection-only: on overflow it replays everything retained and keeps reading the live stream through the normal response pipes; the fake-completion streak is only updated when the whole response was inspected; retry cleanup stays alive until the live response (and raw dump) finish.
#7136 myfear The synthesized final completion chunk dropped provider-side usage fields (e.g. cost/credits) that some OpenAI clients price from, and the trailing usage-only frame was only forwarded when a tool call had been seen. Now the last upstream usage object is preserved into the synthesized completion (normalized token fields still win), the usage-only frame is forwarded to clients that set stream_options.include_usage on text-only rounds too, and remembered usage resets per stream round so a later round can't inherit a previous cost.

Fixes

Related references mentioned in the original PR bodies (context only, NOT closed by this PR): #2691, #1464, #5057 (#7402's related history); #378, #486, #1904, #7203 (#7406's related history); #1766 (#7136's follow-up direction).

原 PR 处理

#7402、#7406、#7136 将在本批合并后关单,并注释收编进本批 (this PR). 本批未自行关闭或评论任何原 PR。

Validation (local, pre-push)

  • All three cherry-picks applied cleanly (no conflicts) on current master (98fa957).
  • Targeted suites on the absorbed files: 105/105 pass — responses-content-index, fake-completion-buffer-limit, stream-buffer-budget, openai-usage-frame-forward, proxy-stream-error, fake-completion, fix-stream, wire-contract.
  • npm ci + npm run build (kernel + pi-subagents + tsup) clean.
  • Full-suite verdict is the remote CI test lanes on this head (§7.2).

Afloat16 and others added 3 commits November 21, 2026 15:13
Set content_index=0 on the single-part text and marker lifecycles and
legacy stream-error completions. Retain the source content index when
proxy and plugin filters flush pending text, including plugin retry
reframing. This prevents official SDK consumers from rejecting valid
answers such as text ending in a buffered "m" as a missing content part.

Record WC-026, enforce it in the matching fake and exercise actual OpenAI
SDK consumption. The unchanged production fails ten focused regressions
with ten controls passing; the fixed relevant suites pass all 193 tests.
Typecheck, build, unused-export and test-import-graph gates also pass.

(cherry picked from commit 3eec386)
Stop bounded inspection at the first over-budget chunk and replay its
complete prefix and remaining stream through the existing response pipes.
Do not classify a partial response or alter its fake-completion streak.

Keep retry idle timers and client-abort listeners until the live response
and optional raw dump finish. Error-body truncation remains unchanged.

Validation: 5 baseline behavior failures and 2 passing exact-cap controls;
74 targeted tests pass. Typecheck, build and static gates pass. Codex
preflight finds the CLI/artifact but the live upstream is unavailable;
the real four-phase E2E and full remote CI remain pending.

(cherry picked from commit 2703cbe)
Preserve the last upstream usage object when emitting the synthesized completion chunk. OpenAI clients that price a turn from provider fields (for example cost/credits) otherwise lose them.

Forward the trailing usage-only frame to clients that set stream_options.include_usage, including text-only rounds, and reset the remembered usage per stream round so a later round cannot inherit a previous cost.

Adds five regression tests in tests/openai-usage-frame-forward.test.ts.

(cherry picked from commit 6228c39)
@github-actions

github-actions Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: batch-B-2026-11-21 (0e6b83a)

Latest build — install this exact version

npm install -g billion-context@0.0.11180-pr.7445.bab6e88

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
0e6b83a billion-context@0.0.11180-pr.7445.bab6e88 build
d60f690 billion-context@0.0.11173-pr.7445.111528e build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr7445.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

Picked up the PR review for #7445 (batch-B-2026-11-21, head d60f690). Plan: pull the branch, read the full diff of the three absorbed fixes (#7402 / #7406 / #7136), check CI status from the mirror cache, then post the structured review (八问). Will report back shortly.

…s on Windows (ERR_UNSUPPORTED_ESM_URL_SCHEME)
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重: 不重复 — the only open PR targeting these three wire-format fixes is this batch; the original PRs (#7402, #7406, #7136) are superseded by it, and no sibling batch (A/C/D) touches the same files.
标签: bug + scope/wiring + p2 (three wire-format defect fixes in the request pipeline; local impact with workarounds, not fleet-wide availability).

Review of head d60f690e220 (base = master 98fa9575f, branch up to date, no rebase needed). 15 files, +756/−29, cherry-picked with -x — original authorship and messages preserved.

① Direction (original issues). All three are root-cause wire-fidelity restorations, not symptom masks:

  • [PR] fix(responses): preserve content indices in synthesized streams #7402 (content_index): the producer of the defect was the synthesized lifecycle frames carrying an item address but no part address — the official OpenAI Node SDK keys text deltas by content_index and throws missing content at index undefined on ordinary answers. All four synthesis sites are covered: src/stream-error.ts:93-96, src/loop/adapter-responses.ts:240-243 (buildMessageItemSequence, both emitText and emitMarker route through it) + :421 (flushFilter, lastTextRef.contentIndex), src/plugin.ts:4623 (flushTail + retry reframing { ...lastDeltaMeta, item_id, output_index }); src/codex-compact.ts:316-319 already had it on master.
  • [PR] fix: preserve successful responses beyond the fake-completion buffer cap #7406 (fake-completion buffer cap): the capped error-body buffer (default 16 MiB) was being used as the complete-response carrier. Now inspection-only: on overflow it replays everything retained and keeps reading the live stream through the normal response pipes; the fake-completion streak is only updated when the whole response was inspected; retry cleanup stays alive until the live response (and raw dump) finish.
  • [PR] fix(openai): keep provider usage fields in the synthesized completion #7136 (provider usage fields): the synthesized completion carried only normalized token fields and the trailing usage-only frame was gated on a seen tool call. Now the last upstream usage object is preserved (normalized fields win), the usage-only frame is forwarded to stream_options.include_usage clients on text-only rounds too, and remembered usage resets per stream round.

② Regression review (existing behavior on touched paths).

  • content_index is additive on synthesized frames; flushFilter's lastTextRef.contentIndex is undefined and dropped by JSON.stringify — legacy-safe. The new WC-026 rule in tests/wire-contract-fakes.ts enforces it in the sse() fake, but master's responsesReply (lines 828-831) already emitted content_index: 0, so no golden drift.
  • Behavior change (disclosed in PR + docs/configuration.md en+zh): over-cap responses on the opt-in fake-completion path go from truncated + classified to replayed + live passthrough, no streak change, no classification. Verified the plumbing: resolveFakeCompletion now returns { body, clearTimer? } (relay.ts:3027); overflow → { body: liveStream } (relay.ts:3028-3030); retry over-budget → handedOff=true + clearTimer (relay.ts:3066-3069); both call sites (plugin lane 2308-2320, proxy lane 2477-2490) store clearFakeRetryTimer, cleared in the outer finally AFTER await dumpRaw (plugin 2422-2428, proxy 2976-2989) — on all exit paths incl. client abort. bufferStreamIfFits (src/server/stream-io.ts:31-88) releases the source lock exactly once on cancel/error (ended flag); readStreamToBuffer unchanged (still capped, pinned by test).
  • Behavior change (disclosed): usage-only frame forward condition relaxed from sawRealToolCall to sawRealToolCall || clientWantsUsage (adapter-openai.ts:568) — matches the client's explicit include_usage request. Usage build: ...raw spread first, normalized prompt_tokens/completion_tokens/total_tokens override after, prompt_tokens_details merged { ...rawDetails, cached_tokens } — normalized fields win as claimed. lastUsageRaw captured at 3 sites (adapter-openai.ts:557, 601, 615), reset per round at parseStream entry (:301); all emitCompletion call sites (src/loop/core.ts:1727/1737/1784) run after that round's parseStream; the truncation path exits via emitError (core.ts:1719-1720) — no stale-usage path.
  • No fail-fast→swallow, no threshold/timing drift beyond the two disclosed items.

③ Code review. No blocking findings; the mechanics above were verified line-by-line against the diff, not just the PR description.

④ Test teeth. 105/105 targeted suites pass locally (responses-content-index, fake-completion-buffer-limit, stream-buffer-budget, openai-usage-frame-forward, proxy-stream-error, fake-completion, fix-stream, wire-contract). The WC-026 regression consumes the synthesized stream through the REAL OpenAI Node SDK (resolved via pi-coding-agent's nested node_modules) — it has teeth: it failed on Windows CI precisely because the SDK actually ran. The overflow test pins replay + no-streak semantics; the usage tests pin field precedence and per-round reset.

⑤/⑤b Diff & code cleanliness. Every line is on-topic; cherry-pick -x preserves provenance. Naming clear, no dead code, no magic numbers, style consistent with the touched files.

CI (live, mirror cache): 29/30 green on d60f690e220; the one red is test (windows-latest, 22, 5) (run 44879819182, job 136772543003) — a real test bug, not a flake: all 9 failures in that shard (2587 tests / 2546 pass / 9 fail / 32 skip) are the new tests/responses-content-index.test.ts cases, error ERR_UNSUPPORTED_ESM_URL_SCHEME: Only URLs with a scheme in: file, data, and node are supported… Received protocol 'd:'. Root cause: await import(requireSdk.resolve("openai")) — require().resolve() returns a raw Windows drive path (D:\...), which dynamic import() rejects on Windows (Linux works by accident). Master is green per #4599, so this was the PR's own. You had already pushed the fix (0e6b83aab4c, pathToFileURL(...).href) while I was diagnosing — it is byte-identical to what I independently derived from the job log; I discarded my duplicate local commit and verified 105/105 on the new head 0e6b83aab4c. CI on that head is pending.

八问(中文结构化结论)

  1. 是什么问题:三个 wire 格式缺陷——合成的 Responses 帧缺 content_index(官方 OpenAI SDK 在普通回答上抛错);fake-completion 缓冲上限把超限的成功响应当完整响应截断;合成的 completion 丢 provider 侧 usage 字段(cost/credits)且纯文本轮次不转发 usage-only 帧。
  2. 什么原因导致的:① 合成的生命周期帧只带 item 地址、不带 part 地址;② 用于错误体检查的有上限缓冲被复用作完整响应载体;③ 合成 completion 只带归一化 token 字段,usage-only 帧的转发条件绑死在「见过 tool call」上。
  3. 影响范围与严重程度:① 用官方 OpenAI Node SDK 走 responses 流式的客户端(普通回答即报错);② 开 opt-in fake-completion 且响应超 16MiB 的用户(内容静默丢失);③ 从 provider usage 字段计费的客户端(少计 cost)。整体 p2:局部影响、各有绕行(换客户端/不开 fake-completion/不依赖那些字段)。
  4. 怎么修复的:① 合成帧补 content_index(单 part 为 0,多 part 保留源索引,retry 重帧也保留);② 缓冲降级为 inspection-only,溢出时回放已保留部分 + 继续读活流,仅整响应被检查时才更新 streak;③ 最后一个上游 usage 对象保留进合成 completion(归一化字段优先)+ 纯文本轮次也向 include_usage 客户端转发 usage-only 帧 + 每轮重置。
  5. 是否有回归问题:逐项审过——content_index 为增量字段,legacy 客户端无感(undefined 被 JSON.stringify 丢弃);fake-completion 行为变更已披露(超限响应:截断+分类 → 完整活流透传、streak 不变),docs en+zh 同步;usage 帧转发条件放宽匹配客户端显式 include_usage 请求;归一化 token 字段仍优先,无语义漂移;无 fail-fast→吞错。
  6. 是否破坏缓存:不涉及(缓存/重试/增量游标/去重键均未动,pf 水位车道未触及)。
  7. 是否改了设计方向:没有——三项都是 wire-fidelity 恢复(保住模型/客户端已收到的内容),不是协议变更。
  8. 是否引入新配置:没有。

判定:方向、回归、代码、测试、diff 五关全过;唯一红条(Windows 测试导入 bug)已修(0e6b83aab4c),等新 head CI 转绿即可合并(human merge)。

@ranxianglei ranxianglei added bug Something isn't working p2 scope/wiring labels Nov 21, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working p2 scope/wiring

Projects

None yet

2 participants