Skip to content

[PR] fix: carry the restored output budget across the watermark rebind (#6752) - #6766

Closed
sx4im wants to merge 3 commits into
ranxianglei:masterfrom
sx4im:fix-6752-output-budget-restore
Closed

sx4im wants to merge 3 commits into
ranxianglei:masterfrom
sx4im:fix-6752-output-budget-restore

Conversation

@sx4im

@sx4im sx4im commented Nov 14, 2026

Copy link
Copy Markdown
Contributor

Fixes #6752

What broke
A dsh turn with a client-shrunk output budget (max_tokens: 1) gets rescued by restoreOutputBudget, and the log reports the restore, but on a watermark fastpath hit prepareOpenai re-parses the scalar fields from the raw wire text, so the forwarded body still carries max_tokens: 1. Upstream records completion_tokens = 1 every turn.

What changed

How verified

@ranxianglei ranxianglei changed the title fix: carry the restored output budget across the watermark rebind (#6752) [PR] fix: carry the restored output budget across the watermark rebind (#6752) Nov 14, 2026
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm/qwen3.8-27b)

查重: 不重复. PR #6766 is the only fix for issue #6752. No parallel PRs found.

Review

The fix is correct and complete. The watermark fastpath's rebind logic (prepareOpenai) was discarding the restored output budget because it re-parsed scalar fields from the raw wire text. The fix snapshots the restored budget before the rebind and writes it back after. The budget.ts move is a clean refactor (#1440 P2 cut 2) that doesn't change the design direction.

CI: lane-guard failure (fixed in this review)

The lane-guard CI job failed because the PR modified src/server/side-request.ts (lane-logic file) without updating a side-lane suite. I added a boundary-pinning test (F2) to tests/side-lane-invariants.test.ts that pins the re-export identity (outputBudgetField / readOutputBudget / writeOutputBudget from side-request.ts === budget.ts). This satisfies the lane-guard and also verifies the refactor is correct. Pushed to the PR branch (5bd8909291).

八问 (structured review)

  1. 是什么问题: dsh 轮次中客户端缩小的输出预算 (max_tokens: 1) 在水位快路径命中时丢失——prepareOpenai 从原始 wire 文本重新解析标量字段,丢弃了 restoreOutputBudget 设置的恢复预算。转发的 body 仍然携带 max_tokens: 1,上游每轮记录 completion_tokens = 1。

  2. 什么原因导致的: 水位快路径 (pluginFullResendWatermark, 5% 灰度) 命中时从原始 wire 文本 (bodyText) 重新绑定 parsed,重新解析标量字段包括 max_tokens,丢弃了 handle.ts 通过 restoreOutputBudget 设置的恢复预算。Legacy 路径正常 (parsed 在序列化前被修改),但快路径是唯一丢失预算的路径。

  3. 影响范围与严重程度: 影响 dsh plugin-native 用户在 openai wire 上、处于 5% 水位快路径灰度中、且有客户端缩小输出预算 (max_tokens: 1) 的场景。严重程度: 恢复的预算永远无法到达上游,模型每轮被限制到 1 token——用户看到 "output token limit reached, answer truncated"。这是 p2 问题 (可用但有明显问题,有 compat.dropFields 绕行方案)。

  4. 怎么修复的: 在水位命中时,从 rebind 前的 body 快照恢复的预算,在 rebind 后写回。同时将输出预算字段管道 (outputBudgetField / readOutputBudget / writeOutputBudget) 从 side-request.ts 移到 budget.ts,使 prepare 模块无需触及 side-lane 引擎即可使用 (audit: 全仓冗余审计与重构计划——server.ts 解体、客户端矩阵去重、死代码、AGENTS.md 模块图腐化 #1440 P2 cut 2)。side-request.ts 重新导出,现有导入者不受影响。

  5. 是否有回归问题: 无。Legacy 路径不变 (parsed 在序列化前被修改)。快路径仅限 openai wire (anthropic/responses wire 无 rebind)。定向套件全绿 (watermark/fastpath/scanner, side-lane invariants, side-request isolation, output-ceiling, bili 0.1.181:clampOutputBudget 把小数写进 max_tokens,上游 400 max_tokens: Input should be a valid integer / 422 Endpoint is unavailable #2011 integer-budget, 输出预算裁剪把图像token计了两遍,usage级基线已含图像,ctx.imageTokens又被加一次 #2558 clamp-image — 135 测试)。新的 A/B 回归测试证明修复有效 (两个阶段在 turn 1 都转发恢复的上限 32768)。

  6. 是否破坏缓存: 无。修复仅影响输出预算字段 (max_tokens),不影响消息内容或前缀缓存。水位快路径的驻留检查 (memB.breakdown.lastMessages < memA.breakdown.lastMessages) 证明快路径确实命中,缓存行为不变。

  7. 是否改了设计方向: 无。修复是水位快路径 rebind 逻辑的最小、定向变更。budget.ts 移动是纯重构 (audit: 全仓冗余审计与重构计划——server.ts 解体、客户端矩阵去重、死代码、AGENTS.md 模块图腐化 #1440 P2 cut 2),不改变设计方向。

  8. 是否引入了新的配置: 无。无新的 env 变量、配置键或默认值。现有的灰度旋钮 (pluginFullResendWatermark, 5%) 和 env 变量 (BILI_ROLLOUT_PLUGIN_FULL_RESEND_WATERMARK) 不变。

判定

修复正确、完整、最小化。lane-guard CI 失败已修复 (添加了边界固定测试)。建议合并。

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm/qwen3.8-27b)

CI is now fully green on the updated head (5bd8909291): 38 checks passed, 14 skipped, 0 failed. The lane-guard failure is resolved by the boundary-pinning test (F2) added in this review. Ready to merge.

sx4im and others added 2 commits November 16, 2026 02:25
…nxianglei#6752)

On a watermark fastpath hit, prepareOpenai re-parses scalar fields from
the raw wire text, silently discarding the max_tokens restore that
restoreOutputBudget had just applied upstream of it. The log still
reports the restore while the forwarded body carries the client's
shrunk value (max_tokens: 1).

Snapshot the restored budget from the pre-rebind body and write it
back after the rebind. The output-budget field plumbing moves from
side-request.ts to budget.ts so the prepare modules can use it without
reaching into the side-lane engine (ranxianglei#1440 P2 cut 2); side-request.ts
re-exports it, so existing importers are untouched.

Regression test: hermetic A/B golden over two proxies (rollout off vs
on) asserting the starved turn forwards the restored ceiling on both
paths, with a residency check proving the fastpath actually hit.
…ranxianglei#6752)

The ranxianglei#1440 P2 cut 2 move (output-budget plumbing side-request.ts -> budget.ts)
trips the lane-guard (ranxianglei#2170): side-request.ts changed without a side-lane suite
update. Pin the re-export identity so the re-export and the definition can never
silently diverge.
@ranxianglei
ranxianglei force-pushed the fix-6752-output-budget-restore branch from 5bd8909 to 0299701 Compare November 15, 2026 18:30
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Merge conflict resolved & rebased onto current master. PR #6766 is now mergeable: True (the dirty state is cleared). CI is running on the new head: 30 checks, 0 failed, 12 pending — no red yet.

What I did this run:

  • The PR head (sx4im:fix-6752-output-budget-restore, was 5bd89092) was 91 commits behind master and in dirty (merge-conflict) state.
  • Rebased the PR's 2 commits onto current master (f648217cb8).
  • One conflict, in src/server/budget.ts: master had added import { outputBudgetField, readOutputBudget, writeOutputBudget, type OutputBudgetField } from "./side-request.js" + a re-export (consuming the functions from side-request), while this PR moves those definitions INTO budget.ts and makes side-request.ts re-export from budget.js. Keeping both would be a circular import. Resolved by keeping the PR's definitions in budget.ts and dropping master's import+re-export.
  • Verified coherence: no circular imports; all importers (prepare-openai.ts, relay.ts, server.ts, the tests) resolve correctly; the PR's own re-export-identity test (F2 in tests/side-lane-invariants.test.ts) is consistent with the resolution.
  • Force-pushed with --force-with-lease (remote head confirmed unchanged before push): 5bd89092 → 0299701062.
  • CI on the new head is in progress (0 failed, 12 pending). I'll keep owning red-CI fixing if any check turns red (attempt 1/6).

八问:

  1. 是什么问题: PR [PR] fix: carry the restored output budget across the watermark rebind (#6752) #6766(修 Output-budget restore is discarded by the watermark fastpath rebind, so the restored max_tokens never reaches upstream while the log reports success #6752 的 watermark rebind 丢失已恢复 output budget)与 master 产生合并冲突(dirty)——master 前进了 91 个 commit,且 master 侧在 src/server/budget.ts 里加了「从 side-request.js import + re-export output-budget 函数」,与本 PR「把这些函数定义搬进 budget.ts」撞车。
  2. 什么原因导致的: PR 基于较旧的 master(f21b601893),把 output-budget 字段函数(outputBudgetField/readOutputBudget/writeOutputBudget/OutputBudgetField)从 side-request.ts 搬进 budget.ts(audit: 全仓冗余审计与重构计划——server.ts 解体、客户端矩阵去重、死代码、AGENTS.md 模块图腐化 #1440 P2 cut 2 解耦);master 同期独立地在 budget.ts 里加了反向 import(从 side-request.js 取这些函数)。rebase 到当前 master 时二者在 budget.ts 冲突。
  3. 影响范围与严重程度: 单文件 import 结构冲突,纯合并阻塞(不修则 PR 无法合入);无用户面行为影响。
  4. 怎么修复的: 把 PR 的 2 个 commit rebase 到当前 master;budget.ts 冲突取 PR 侧(函数定义留在 budget.ts),删掉 master 侧的「import from side-request.js + re-export」——因为 side-request.ts 已(自动合并)改成从 budget.js import 并 re-export,再留反向 import 会成循环依赖。
  5. 是否有回归问题: 无。保留 PR 原意(定义在 budget.ts,side-request.ts re-export);prepare-openai.ts / side-request.ts 自动合并结果连贯;PR 自带的 re-export 身份测试(F2)与解法一致。除 PR 自身改动外无行为变化。
  6. 是否破坏缓存: 不涉及。冲突解决是 import 结构(函数定义位置)调整,不碰缓存/重试/游标/去重键。
  7. 是否改了设计方向: 无。PR 的设计(output-budget plumbing 移到 budget.ts 以解耦 audit: 全仓冗余审计与重构计划——server.ts 解体、客户端矩阵去重、死代码、AGENTS.md 模块图腐化 #1440 P2 cut 2)原样保留。
  8. 是否引入了新的配置: 无。无新 env/config/CLI flag。

判定: 冲突已解、已 rebase 到当前 master 并 force-push,PR 现可合入(mergeable: True)。CI 在新 head 上运行中(0 failed / 12 pending),无红;若 CI 转红我会继续修。

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[ework] CI fix verified green ✓

…nxianglei#6752)

When compat.dropFields removes a budget field that restoreOutputBudget
just restored, the drop log now says so explicitly so the log is honest
(the restore fired, but the field is absent from the forwarded body).

One-line note appended to both drop-log sites (graph and string paths).
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[ework] auto-fix stalled: architecture-docs — stopping for human review

1 similar comment
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[ework] auto-fix stalled: architecture-docs — stopping for human review

@ranxianglei

Copy link
Copy Markdown
Owner

Your fix is merged — the watermark-rebind budget carry and the F2 re-export identity test landed via the wave-11 batch merge (#6827 → master a446a3c1c), carrying your commits verbatim.

One follow-up on this PR was pushed after the batch snapshot and is therefore not in master yet: 1fcc31d11 ("note the output-budget override in the compat.dropFields log") — the honest-log note in relay.ts. It is small and self-contained, and it is queued for the next community-fix batch; you will see it land there. (If you'd rather I take it separately sooner, say so.)

GitHub did not auto-flip this PR to Merged for the same reason as the others (the head moved after the snapshot). Closing as merged-via-batch, pending that one carry-over commit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Output-budget restore is discarded by the watermark fastpath rebind, so the restored max_tokens never reaches upstream while the log reports success

2 participants