Repository navigation
[PR] Bound output-budget restore & clamp by the model's declared output ceiling (#3833) - #3836
Conversation
…tput ceiling (#3833) A DSH session that ran a model declaring output 256000 (deepseek-flash) and then switched back to glm-5.3 (ceiling 131072) kept replaying the stale session-level high-water of 256000 into every glm request after history grew past the window and the client shrank max_tokens to 1. Neither the #546 rescue nor the #453 clamp honored the CURRENT model's declared ceiling: - restoreOutputBudget only ever RAISED the target to the ceiling (`ceiling > target`), so a high-water above the ceiling passed through verbatim (256000 out the door); - clampOutputBudget capped only by window headroom (1M - ~827K input - margin = 131577), still above glm's 131072. Upstream rejected both with 400 code 1210 ("max_tokens参数非法:限制数值范围 [1,131072]"), failing the session every turn. Fix (both directions, availability over ideology): - restore: when a ceiling is known the target is clamped to min(highWater, ceiling); an over-bound high-water is corrected IN PLACE (outputBudgetHighWater := ceiling) so subsequent turns stop re-poisoning; - clamp: cap = min(window-headroom, knownOutputCeiling), threaded through clampOutgoingOutput and the three prepare lanes via resolveKnownOutputCeiling (the same source chain as the restore floor). Absent a known ceiling every term is byte-identical to before. Behavior change (intended): a healthy high-water ABOVE the declared ceiling is now clamped DOWN to the ceiling instead of restored verbatim. Old behavior trusted "client intent beats declaration"; the ceiling is the hard upstream bound strict providers enforce with a 400, so a water above it is either a stale cross-model artifact or an over-conservative declaration — clamping is safe, verbatim restore is not.
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.4540-pr.3836.52ef589This exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr3836.tgz
npm install -g packageInstall instructions are refreshed on each push. |
(ai: vllm-qwen/qwen3.8-27b) 查重:不重复。 相关项:#3833(本 PR 修复的原始 issue,症状与机制完全对应);#3677(在途 PR,同触 DSH 输出预算但不同面——#3677 管中段救援的入口门与日志措辞,本 PR 管目标值计算,仅在同一函数上最小重叠,PR 正文已注明谁先合谁后就 rebase);#3328 / #3071 / #1842(相关机制、不同失败域)。不存在与任一 open/closed 条目「同症状+同机制+同根因」的重叠。 标签:bug、dsh、scope/wiring、p2(已打)——缺陷类;DSH Desktop 触发;属出站请求参数边界层;仅特定跨模型场景命中且可换会话绕行。 CI:head 审查进行中:已通读全部 diff 并对照原 issue 证据,正在本地跑定向测试。结论随后按八问给出。 |
ceiling-clamp change The function-header doc block still stated 'a healthy water mark above the ceiling keeps winning' — the pre-#3833 behavior this PR deliberately reverses (over-bound waters are now clamped down and corrected in place). Update the sentence to match the code facts; no logic touched.
(ai: vllm-qwen/qwen3.8-27b) 审查结论:可合入。 方向判断正确(修的是根因两处「不认上限」,不是症状层拦截),既有行为逐项清点无意外回归,唯一的行为变更已在 PR 正文披露且有测试钉住。审查中顺手修了一处遗留陈旧注释并已推到 PR 分支(见下)。 八问
测试有效性:新测试钉住事故精确数值(restore diff 整洁度:7 个文件全部与本修复相关,无无关改动、无 lockfile churn、无版本号变更,单一线性提交。 顺手小修(已推 PR 分支): 措辞备注(不必改):PR 正文「restore 目标变为 min(highWater, ceiling)」是简写,实际代码为「有上限时目标恒为上限」(低于上限仍按 #1665 抬升);同段第二句「highWater ≤ ceiling 保持 raise-to-ceiling」才是准确描述,测试钉的也是后者。行为正确,仅首句措辞宽松。 与 #3677 的关系:独立缺陷、同函数最小重叠(#3677 管入口门/日志措辞,本 PR 管目标值计算),谁先合谁后就 rebase,不构成阻塞。 CI 与本地验证:head 判定:可合入。 |
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.616-master.1434ca7 # this exact build
npm install -g billion-context@master # always the newest merged build
|
Summary
A DSH session that ran a model declaring output 256000 (deepseek-flash) then switched back to glm-5.3 (declared ceiling 131072) kept replaying the stale session-level high-water of 256000 into every glm request once history crossed the window and the client shrank
max_tokensto 1. Both bili mechanisms that touch the outgoing output budget ignored the current model's declared ceiling:restoreOutputBudget) only ever raised the target to the ceiling (ceiling > target), so a high-water above the ceiling passed through verbatim → 256000 out the door.clampOutputBudget) capped only by window headroom (1M − ~827K input − margin = 131577), still above glm's 131072.Upstream rejected both with
400 code 1210("max_tokens参数非法:限制数值范围[1,131072]"), failing the session every turn (9× on the reporter's box). Root cause 3:outputBudgetHighWateris a session-level scalar with no model dimension, so a water learned under one model survives a switch to another.Fix (both directions)
src/server/side-request.ts): when a ceiling is known the restore target ismin(highWater, ceiling); an over-bound high-water is corrected in place (outputBudgetHighWater := ceiling) so every subsequent turn stops re-poisoning instead of replaying the stale value.src/server/budget.ts):cap = min(window-headroom, knownOutputCeiling); the ceiling is threaded throughclampOutgoingOutputand all three prepare lanes (openai/responses/google) viaresolveKnownOutputCeiling— the same source chain the restore floor already uses. Absent a known ceiling, every term is byte-identical to before.A healthy high-water above the declared ceiling was previously restored verbatim ("client intent beats declaration"). It is now clamped down to the ceiling. Rationale: the ceiling is the hard upstream bound strict providers enforce with a 400 — a water above it is either a stale cross-model artifact (this incident) or an over-conservative declaration; clamping is always safe, verbatim restore can hard-fail a whole session. Only the
highWater > ceilingsub-case changes;highWater ≤ ceilingkeeps the pre-existing raise-to-ceiling rescue, and the no-ceiling path is untouched.Tests
New
tests/issue3833-cross-model-output-ceiling.test.ts(the issue's exact numbers): restorevalue=1, highWater=256000, ceiling=131072→ writes 131072 and resets the water; clamprequested=256000, input=827069, window=1e6, ceiling=131072→ 131072 (not 131577); plus negative rows proving below-ceiling waters keep the rescue and an absent ceiling stays byte-identical. Updated one assertion intests/side-request-isolation.test.tsthat had pinned the old verbatim-above-ceiling behavior (#1665 "water mark above the floor untouched"). Targeted suites green locally: typecheck + side-request-isolation + issue453-backstop + issue3071-planning-floor-min-output + issue2011-integer-output-budget = 61/61.Notes
kernel/change, no persistence-format change (the in-place correction writes the existingoutputBudgetHighWaterfield to a smaller value).Closes #3833