Repository navigation
[PR] fix: retry context overflow with measured output headroom - #6529
Conversation
(ai: vllm-qwen/qwen3.8-27b) 查重: 不重复。最近的在途工作 — #5880/#5899(estimate-only fit:bili 的本地估计认为能放下但上游拒绝)和 #5382(overflow-stated counts 驱动折叠目标)— 机制不同。本 PR 的机制是独立的:当上游报告的计数证明输入本已放得下时,封顶输出预算并重试,而不是重新折叠。没有并行的 open PR 修同一问题。 标签: bug · dsh · scope/wiring · p2 — relay 溢出恢复路径上的 bug 修复(协议接线层),在 DSH 上报/复现;定 p2 因为它是特定条件失败(overflow 且输入本已放下)且有手工绕行(调小输出预算),不是全局不可用或缓存泄露。 Review 状态 — CI 是红的,且是 PR 自己引入的:
核心修复本身是扎实的。 我读了完整 diff:measured-output cap 的计算和它的守卫(total===input+output、字段匹配、保留 thinking budget、排除 egress-transform、排除 Buffer)都正确且保守;新增测试是真实的(非恒真),覆盖了恢复路径和负例。 我在做什么: 按外部 PR 质量政策,我直接在 PR 分支上修这个边界违规(不开并行 PR):这三个 output-budget helper 是通用的 wire-shape helper, |
…-lane boundary relay.ts (the wire tail) must not import the side-lane engine directly (ranxianglei#1440 P2 cut 4; tests/side-lane-invariants.test.ts test H). The overflow recovery (ranxianglei#6528) added an import of outputBudgetField/readOutputBudget/ writeOutputBudget from ./side-request.js, coupling relay to the side-lane engine and breaking test H on 3 CI shards (master is green on the same shards). The three helpers are generic wire-shape helpers already imported by src/server/budget.ts; re-export them from budget.ts and point relay.ts at ./budget.js. No behavior change, no circular dependency.
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) PR #6529 Review — 结构化八问1. 是什么问题 2. 什么原因导致的 3. 影响范围与严重程度 4. 怎么修复的 5. 是否有回归问题
6. 是否破坏缓存 7. 是否改了设计方向 8. 是否引入了新的配置 判定:核心修复正确且保守;阻断性 CI 失败(side-lane 边界违例)已修并本地验证(typecheck + 69 测试),CI 将在新 head |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) Follow-up — CI re-run on my fix (
Recommendation: re-run the failed shards (or land the notice-test stabilization on master / |
|
Closes pair #6528 was auto-closed by the batch merge. Superseded by #6765 (merged to master as 0a73563). Your commit 9c6e9b3 is in master verbatim — same content, same authorship, only the object ID changed (the batch used a real Thanks for the fix — it is live in master from 0a73563 onward. |
Master advanced 379 commits (incl. nudge-rearm #6531, side-passthrough, preflight/fold, relay output-budget #6529) since the prior sync (9674cfa). 4 of the PR's 5 files overlapped; 3 auto-merged cleanly. The one conflict, src/server/prepare-anthropic.ts, resolved as the union of both intents: keep #4974's if (!anthropicTitleGen) gate wrapping the meter writes (the meter writes are byte-identical on both sides modulo indent) and adopt master's return which adds nudgeStampsBefore (nudge-rearm #6531). typecheck clean; issue4974 + side-lane-invariants + issue2490 + issue2847 = 30/30 and anthropic nudge suites = 7/7 pass.
Fixes #6528.
When upstream reports 108,000 input tokens and a 32,000 output reservation against a 128,000 window, the input already fits. The relay currently goes straight to refolding. If preflight exhausts its compression budget, the client receives the original 400.
Retry the same payload with output capped at 20,000 in that example. The calculation uses the rejected request's measured input and verifies that the reported completion count matches the outbound field. If recovery still needs a refold, apply the same rule to that attempt's counts. Preserve explicit thinking budgets and stop on unrelated errors. Routes with configured egress transforms keep the existing refold path because their retry can carry different input.
Validation
The declared model window, tokenizer, and fork inheritance rules stay unchanged. This PR covers the measured-output recovery path. Human review before merge.