Skip to content

[PR] Bound output-budget restore & clamp by the model's declared output ceiling (#3833) - #3836

Merged
ranxianglei merged 2 commits into
masterfrom
2026-10-19_cross-model-output-ceiling
Oct 19, 2026
Merged

ranxianglei merged 2 commits into
masterfrom
2026-10-19_cross-model-output-ceiling

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Summary

A DSH session that ran a model declaring output 256000 (deepseek-flash) then switched back to glm-5.3 (declared ceiling 131072) kept replaying the stale session-level high-water of 256000 into every glm request once history crossed the window and the client shrank max_tokens to 1. Both bili mechanisms that touch the outgoing output budget ignored the current model's declared ceiling:

Upstream rejected both with 400 code 1210 ("max_tokens参数非法:限制数值范围[1,131072]"), failing the session every turn (9× on the reporter's box). Root cause 3: outputBudgetHighWater is a session-level scalar with no model dimension, so a water learned under one model survives a switch to another.

Fix (both directions)

  • Restore (src/server/side-request.ts): when a ceiling is known the restore target is min(highWater, ceiling); an over-bound high-water is corrected in place (outputBudgetHighWater := ceiling) so every subsequent turn stops re-poisoning instead of replaying the stale value.
  • Clamp (src/server/budget.ts): cap = min(window-headroom, knownOutputCeiling); the ceiling is threaded through clampOutgoingOutput and all three prepare lanes (openai/responses/google) via resolveKnownOutputCeiling — the same source chain the restore floor already uses. Absent a known ceiling, every term is byte-identical to before.

⚠️ Behavior change (intended)

A healthy high-water above the declared ceiling was previously restored verbatim ("client intent beats declaration"). It is now clamped down to the ceiling. Rationale: the ceiling is the hard upstream bound strict providers enforce with a 400 — a water above it is either a stale cross-model artifact (this incident) or an over-conservative declaration; clamping is always safe, verbatim restore can hard-fail a whole session. Only the highWater > ceiling sub-case changes; highWater ≤ ceiling keeps the pre-existing raise-to-ceiling rescue, and the no-ceiling path is untouched.

Tests

New tests/issue3833-cross-model-output-ceiling.test.ts (the issue's exact numbers): restore value=1, highWater=256000, ceiling=131072 → writes 131072 and resets the water; clamp requested=256000, input=827069, window=1e6, ceiling=131072 → 131072 (not 131577); plus negative rows proving below-ceiling waters keep the rescue and an absent ceiling stays byte-identical. Updated one assertion in tests/side-request-isolation.test.ts that had pinned the old verbatim-above-ceiling behavior (#1665 "water mark above the floor untouched"). Targeted suites green locally: typecheck + side-request-isolation + issue453-backstop + issue3071-planning-floor-min-output + issue2011-integer-output-budget = 61/61.

Notes

Closes #3833

…tput ceiling (#3833)

A DSH session that ran a model declaring output 256000 (deepseek-flash) and
then switched back to glm-5.3 (ceiling 131072) kept replaying the stale
session-level high-water of 256000 into every glm request after history grew
past the window and the client shrank max_tokens to 1. Neither the #546 rescue
nor the #453 clamp honored the CURRENT model's declared ceiling:

  - restoreOutputBudget only ever RAISED the target to the ceiling
    (`ceiling > target`), so a high-water above the ceiling passed through
    verbatim (256000 out the door);
  - clampOutputBudget capped only by window headroom
    (1M - ~827K input - margin = 131577), still above glm's 131072.

Upstream rejected both with 400 code 1210 ("max_tokens参数非法:限制数值范围
[1,131072]"), failing the session every turn.

Fix (both directions, availability over ideology):
  - restore: when a ceiling is known the target is clamped to min(highWater,
    ceiling); an over-bound high-water is corrected IN PLACE
    (outputBudgetHighWater := ceiling) so subsequent turns stop re-poisoning;
  - clamp: cap = min(window-headroom, knownOutputCeiling), threaded through
    clampOutgoingOutput and the three prepare lanes via resolveKnownOutputCeiling
    (the same source chain as the restore floor). Absent a known ceiling every
    term is byte-identical to before.

Behavior change (intended): a healthy high-water ABOVE the declared ceiling is
now clamped DOWN to the ceiling instead of restored verbatim. Old behavior
trusted "client intent beats declaration"; the ceiling is the hard upstream
bound strict providers enforce with a 400, so a water above it is either a
stale cross-model artifact or an over-conservative declaration — clamping is
safe, verbatim restore is not.
@ranxianglei ranxianglei changed the title Bound output-budget restore & clamp by the model's declared output ceiling (#3833) [PR] Bound output-budget restore & clamp by the model's declared output ceiling (#3833) Oct 19, 2026
@github-actions

github-actions Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-19_cross-model-output-ceiling (e679ef0)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.4540-pr.3836.52ef589

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr3836.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重:不重复。 相关项:#3833(本 PR 修复的原始 issue,症状与机制完全对应);#3677(在途 PR,同触 DSH 输出预算但不同面——#3677 管中段救援的入口门与日志措辞,本 PR 管目标值计算,仅在同一函数上最小重叠,PR 正文已注明谁先合谁后就 rebase);#3328 / #3071 / #1842(相关机制、不同失败域)。不存在与任一 open/closed 条目「同症状+同机制+同根因」的重叠。

标签:bug、dsh、scope/wiring、p2(已打)——缺陷类;DSH Desktop 触发;属出站请求参数边界层;仅特定跨模型场景命中且可换会话绕行。

CI:head 5083797 全绿(test lanes ubuntu 22/24×shards + windows 22、static-gates、各 gate job、config-surface-guard、lane-guard、lock-sync);e2e lanes 按路径门控未触发(本 diff 不涉及其触发文件)。

审查进行中:已通读全部 diff 并对照原 issue 证据,正在本地跑定向测试。结论随后按八问给出。

@ranxianglei ranxianglei added bug Something isn't working dsh p2 scope/wiring labels Oct 19, 2026
 ceiling-clamp change

The function-header doc block still stated 'a healthy water mark above the
ceiling keeps winning' — the pre-#3833 behavior this PR deliberately reverses
(over-bound waters are now clamped down and corrected in place). Update the
sentence to match the code facts; no logic touched.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

审查结论:可合入。 方向判断正确(修的是根因两处「不认上限」,不是症状层拦截),既有行为逐项清点无意外回归,唯一的行为变更已在 PR 正文披露且有测试钉住。审查中顺手修了一处遗留陈旧注释并已推到 PR 分支(见下)。

八问

  1. 是什么问题:同一 DSH 会话先跑过声明输出 256000 的 deepseek-flash,切回 glm-5.3(声明上限 131072)后,历史涨过窗口、客户端把 max_tokens 压到 1,bili 把旧高水位 256000 原样还原、clamp 又只按窗口余量钳到 131577,两者都超 131072 → 上游 400(code 1210),该会话此后每轮直接失败。

  2. 什么原因导致的:三层叠加——(a) restoreOutputBudget 只把目标「抬高」到上限(ceiling > target),高于上限的水位原样放行;(b) clampOutputBudget 只按窗口余量算 cap,没跟模型声明上限取 min;(c) outputBudgetHighWater 是会话级标量、无模型维度,切模型后旧水位存活。(a)(b) 是「不认上限」,(c) 是「上限为什么会错位」的根子。

  3. 影响范围与严重程度:仅复用会话 id 跨模型的主机(DSH 为主)且要命中「从高声明输出模型切到低上限模型 + 历史涨过窗口致客户端把 max_tokens 压小」这条链。命中后会话每轮失败无法继续,换会话可绕行。p2。

  4. 怎么修复的:两个决策点都补上硬上限约束。restore:有已知上限时目标取上限值(水位低于上限时保持 #546 输出预算高水位可被客户端衰减值污染,饿死会话永久钉在小预算(restore 缺下限) #1665「抬到上限」的原语义),水位高于上限时就地纠正回写同一字段并记 warn,后续轮次不再重放脏值;clamp:cap 取 min(窗口余量, 已知上限),[bug] 0.1.195 输出预算三路饿死仍复现:无 output 上限模型 restore 无下限(1→252、fork 原样转发 1)、#2490 planning floor 钳到 1479–2623 → 3 会话 9 次 finish=length 空轮不重试(#546/#1665/#1840/#2904/#2756 残留) #3071 floor 也一并取 min 防其把结果顶回上限之上。上限经 resolveKnownOutputCeiling(runtime-info > launcher > route 声明 > registry)穿进 openai/responses/google 三条 prepare 通道——与 restore 同源同链。无已知上限时每一项与旧实现逐字节一致(负向测试行验证)。

  5. 是否有回归问题:无意外回归。逐项对照改动前代码清点:① 水位低于上限→抬到上限(#546 输出预算高水位可被客户端衰减值污染,饿死会话永久钉在小预算(restore 缺下限) #1665/[bug] 长会话频繁「达到输出token上限」:restore 回落到被污染的 high-water,0.1.175 未声明 output 时仍能复现 #1840 救援):保持(新代码有上限即 target=ceiling,旧代码此情形同样抬升);② 无水位+有上限→用上限:不变;③ 无上限→信任已验证水位不抬高(区间审查(#3331)残留 11 项中小问题集中修:守卫旋钮、存储竞态、静默丢参、CI 触发残角等 #3406)/绝对下限([bug] 0.1.195 输出预算三路饿死仍复现:无 output 上限模型 restore 无下限(1→252、fork 原样转发 1)、#2490 planning floor 钳到 1479–2623 → 3 会话 9 次 finish=length 空轮不重试(#546/#1665/#1840/#2904/#2756 残留) #3071):分支逻辑未动;④ 无工具 side request、非饥饿(>200)健康请求:早退路径未动;⑤ clamp 新参数缺省 → Infinity,Math.min(x, Inf)=x 逐字节一致;⑥ plugin mode: no hard backstop when the agent ignores the compress nudge — context grows past the window and upstream 400s #453 的 sub-floor surrender(cap<OUTPUT_CLAMP_FLOOR→不钳)判定顺序保持。唯一变化的子情形是「水位>上限」:原样放行→钳到上限+就地纠正——正是缺陷本身,PR 正文已披露(旧→新+理由)并有测试钉住。另注:CHANGELOG.md 里 [bug] 长会话频繁「达到输出token上限」:restore 回落到被污染的 high-water,0.1.175 未声明 output 时仍能复现 #1840 的历史条目仍写着旧行为「above-ceiling water keeps winning」——那是当时版本的发布记录,不回改,未动。

  6. 是否破坏缓存:不涉及。只改出站 max_tokens 取值决策,并把既有 metadata 字段写成更小值;消息字节/前缀/checkpoint/去重键均未触碰。

  7. 是否改了设计方向:否。仍沿用「会话级高水位 + 单一解析源链」,只在两个决策点加了硬上限约束。「高水位按 (sessionId, model) 分别记账」的可选项未做(要改持久化形状,属 owner 拍板项)。

  8. 是否引入新配置:不涉及。

测试有效性:新测试钉住事故精确数值(restore 1→131072 且水位被重置;clamp 256000→131072 而非 131577),另有两条负向行(低于上限的水位保持救援且不被改写;无上限时与 legacy cap 逐字节一致)。断言有牙(bug 回归即抓)、测的是真实函数行为非 mock;第二次 starve 断言证明就地纠持续效、不再被重新污染。

diff 整洁度:7 个文件全部与本修复相关,无无关改动、无 lockfile churn、无版本号变更,单一线性提交。

顺手小修(已推 PR 分支):restoreOutputBudget 函数头 doc 块仍写着 "a healthy water mark above the ceiling keeps winning"——恰是本 PR 推翻的旧行为,注释与代码事实不符。已改为与新行为一致的表述,独立 commit e679ef041(docs-only,零逻辑改动)。

措辞备注(不必改):PR 正文「restore 目标变为 min(highWater, ceiling)」是简写,实际代码为「有上限时目标恒为上限」(低于上限仍按 #1665 抬升);同段第二句「highWater ≤ ceiling 保持 raise-to-ceiling」才是准确描述,测试钉的也是后者。行为正确,仅首句措辞宽松。

与 #3677 的关系:独立缺陷、同函数最小重叠(#3677 管入口门/日志措辞,本 PR 管目标值计算),谁先合谁后就 rebase,不构成阻塞。

CI 与本地验证:head 5083797 全绿(test lanes ubuntu 22/24×shards + windows 22、static-gates、各 gate job、config-surface-guard、lane-guard、lock-sync);我的注释修复 commit e679ef041 仅 docs,CI 重跑确认即可。本地定向:typecheck 干净 + side-request-isolation / issue3833 / issue453-backstop / issue3071-planning-floor-min-output / issue2011-integer-output-budget = 61/61 绿。

判定:可合入。

@ranxianglei
ranxianglei merged commit 1434ca7 into master Oct 19, 2026
44 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.616-master.1434ca7 (rolling master tag).

npm install -g billion-context@0.0.616-master.1434ca7  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working dsh p2 scope/wiring

Projects

None yet

1 participant