Skip to content

[PR] fix: clamp the output budget against a safety-inflated input estimate (#5278) - #5291

Merged
ranxianglei merged 2 commits into
masterfrom
2026-10-31_clamp-estimate-safety
Oct 31, 2026
Merged

ranxianglei merged 2 commits into
masterfrom
2026-10-31_clamp-estimate-safety

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

What's broken

The #453 output-budget clamp (clampOutputBudget, src/server/budget.ts) must guarantee input + output <= window on request-rebuilding upstreams. Its safety margin is max(2048, ceil(est x 0.05)) — sized FROM the input estimate, while its job is to absorb the ESTIMATOR's error. Self-referential.

The heuristic estimator (defaultCountTokens: CJK 1 token/char, rest chars/4) systematically understates CJK-heavy payloads by ~5% — the same order as the 5% margin — so on those sessions margin < actual error by construction. Three independent field incidents (#5278, bili 0.1.206/0.1.209, DSH desktop, window 262144, client-reported max_tokens 384000): billed/estimated ratios 1.050011 / 1.050016 / 1.050020, each clamped total = 262145 = window + 1 -> upstream ContextWindowExceededError 502.

Fix (option A from the issue)

Behavior change (old -> new)

  • Clamped output budgets tighten by up to ~12% of the input estimate. Owner field validation on the issue's bundle extract: across a 1088-sample sweep, fitting requests stay bit-identical, all oversized requests still clamp, tightening 120–18891 tokens, zero overflows. The three incident requests now clamp to 142946 / 120575 / 175719 (totals under the window with 9K–15K headroom).
  • Previously-unclamped oversized requests may now clamp (conservative direction only).
  • No wire-format change, no persistence change, no kernel change, no new config/env surface.

Verification

Fixes #5278

ci-check added 2 commits October 31, 2026 10:18
…5278)

The clamp's safety margin (est * 5%) was sized from the very estimate whose
error it must absorb. The heuristic estimator understates CJK-heavy payloads
by ~5% — the same order as the margin — so on those sessions margin < actual
error by construction and the clamped total landed at exactly window+1, which
request-rebuilding upstreams reject with ContextWindowExceededError 502
(three field incidents, billed/estimated ratios 1.050011/1.050016/1.050020).

Judge on a safety-inflated estimate (x1.12) before sizing both the subtraction
and the margin: any true input up to 1.12x the estimate provably fits (cap
path and #3071 floor path alike), and the compound reservation covers
underestimates up to ~17.6%. When the inflated estimate drops below the
capability floor (input alone nearly fills the window), fall back to the raw
formula so the #2490/#3071 pinned regimes stay byte-identical instead of
surrendering the clamp entirely.

Behavior change: emitted caps tighten by up to ~12% of the input estimate
(field-validated 120-18891 tokens across a 1088-sample sweep); previously-
unclamped oversized requests may now clamp. No new config surface.
…oldens

New tests/issue5278-clamp-safety-margin.test.ts: the three field incidents
(clamped + real input <= window), the owner-validated patch arithmetic
(142946/120575/175719), a systematic sweep over underestimate rates up to the
safety factor, fitting-request no-churn, and byte-identity pins for the
fallback regimes (#2490/#3071 shapes).

Goldens updated for the intended tightening: issue453-backstop,
issue3071-planning-floor-min-output (136144 -> 121024),
issue3833-cross-model-output-ceiling (incident numbers no longer demonstrate
ceiling binding under the inflation; rewritten with clear-margin numbers),
issue2011-integer-output-budget, issue2096-clamp-headroom-window,
deferred-tool-budget (derived from the documented formula instead of magic
numbers).
@ranxianglei ranxianglei changed the title fix: clamp the output budget against a safety-inflated input estimate (#5278) [PR] fix: clamp the output budget against a safety-inflated input estimate (#5278) Oct 31, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-31_clamp-estimate-safety (786c9d9)

Latest build — install this exact version

npm install -g billion-context@0.0.7320-pr.5291.d5fa3cf

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
786c9d9 billion-context@0.0.7320-pr.5291.d5fa3cf build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr5291.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重: 不重复。 本条是 PR #5291(分支 2026-10-31_clamp-estimate-safety,head 786c9d9)的镜像 issue,修的是已立项的 #5278(#453 output clamp 安全边际自指缺陷,CJK 重负载会话在 request-rebuilding upstream 上被 clamp 到 window+1 → 502 ContextWindowExceededError)。检索 open/closed 无同机制的并行工作。

标签: bug dsh scope/wiring p2(原有 pr)—— 请求管道 budget clamp 缺陷;仅特定条件触发(CJK 重 + rebuild upstream + max_tokens 贴近窗口),属"能用但特定条件下有问题",故 p2 而非 p1。

已核实(进行中):

拿到 CI 全绿结论后我会补最终判定回复。

@ranxianglei ranxianglei added bug Something isn't working dsh p2 scope/wiring labels Oct 31, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

中间状态(本地验证全过,CI 在排队):

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

审查结论 + 状态:代码侧 ready-to-merge(pending CI)。① diff 与 #5278 描述一致:clamp 判定改在 ceil(est×1.12) 膨胀值上(OUTPUT_CLAMP_ESTIMATE_SAFETY 模块常量),膨胀后跌破 OUTPUT_CLAMP_FLOOR 时回退 raw 公式——其余 no-op 路径可证在 raw 上必然同样 no-op,#2490/#3071 钉住的形态字节不变;唯一调用点 src/server/budget.ts:530(两种压缩模式共用),无 wire/持久化/kernel/配置面变更。② 本地定向 7 个受影响测试文件 49/49 绿(head 786c9d9)。③ golden 更新为有意收紧并已在 PR 披露(#3071 136144→121024 等)。④ CI:首轮 4 个 test 分片红(ubuntu24-s2/ubuntu22-s2/win22-s1/win22-s2),rerun 因全舰队 runner 积压停在 queued(#4599 floor 702 同现象);按 PR head 确定性分片算法核对——shard 2 含 claude-native.test.ts(#5136 家族)+ store.test.ts(#4788p2 家族)三平台腿全挂、win22-s1 失败腿对应 issue4982-draining-twin-ladder(#5046/#5236 在档家族),全部落在当日在档继承红签名内,与本 PR diff 零交集。分支落后 master 393 提交且未含 ci-fixes@06ad804ac。处置计划:merge origin/ci-fixes 进 PR 分支推送重跑;当前受阻于本工作区磁盘写满(ENOSPC,所有命令执行失败),环境恢复后立即执行并在本帖与 #4599 补 sha。八问完整版已备好,CI 出结果后一并给出。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

更正上一条的处置计划(经实证核对后不改分支):原计划 merge origin/ci-fixes——实测三个红签名对应的测试文件:tests/claude-native.test.ts 与 tests/store.test.ts 在 ci-fixes@06ad804ac 上与本分支逐字节相同,仅 tests/issue4982-draining-twin-ladder.test.ts 有差异(1+/50-)。即这些家族的修复尚未收敛进 ci-fixes(#5236 等 A/B/C 裁决、#4788p2 在 triage),merge 不会清掉红签名,反而在 runner 积压期取消已在排队的 rerun、白耗一轮全矩阵。故维持分支不动,等 rerun 结果(#5136 家族有「rerun 即绿」先例)。代码侧判定不变:ready-to-merge(pending CI)。rerun 出结果我立即补最终八问结论;若再红且签名仍在档家族内,按政策留哨兵/owner 处置,不自行深挖继承红。

@ranxianglei ranxianglei left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

审查完成(本地分支 @786c9d98,已核对完整补丁 + clamp 家族回归)。结论:代码通过,无阻塞项;CI 全绿即可人工合并。 这是对 issue #5278 修复方向的「方案 A」实现,issue 本身分析准确(算术还原、代码路径、自指涉论证均与 master 代码逐项对上),此 PR 按建议落地并补上了建议里特别提醒的回退路径。

逐项核验:

  1. 补丁形态最小且等价重构干净(src/server/budget.ts):旧函数体逐字抽进内部闭包 clampFor(est)(margin/#2011 floor-at-decision-point/#3833 ceilingBound/#3071 floorOut 全部保留原位注释),新逻辑只有两行语义:clampFor(ceil(est × 1.12)) 有值即返回;否则回退 clampFor(est)。没有顺手改动任何其它行为。✓
  2. 安全路证明成立:cap 路与 #3071 floor 路的输出都以放大后的基准界定——floor 路 floorOut ≤ window − safeEst、cap 路 cap = window − safeEst − margin(safeEst) ⇒ 真实输入 ≤ 1.12×est 时总量恒 ≤ window;叠加 5% margin 后复合预订 1.12×1.05 覆盖至 ~17.6% 低估(实测 5% 的 3.5 倍余量)。我手工推演过两条绑定路径与小输入侧(MIN_MARGIN 2048 兜底段)均闭合。✓
  3. 回退语义正确且无新风险:仅当放大基准把 cap 压到 OUTPUT_CLAMP_FLOOR(1024) 以下、而原始公式仍会钳制的近墙区(#2490/#3071 pinned ledger 形态)才走原始公式——这些形态与 master 逐字节相同,不是新增溢出风险(master 今天在同区域就是这个行为);其余所有安全路 no-op 因单调性必然蕴含原始路同 no-op(更松的估计只抬高各项),注释把这句话说清楚了。✓ 净效果:本 PR 对任何请求的输出要么更安全、要么与现状逐字节一致,不存在变差的分支。
  4. 常数而非配置面:OUTPUT_CLAMP_ESTIMATE_SAFETY = 1.12 是模块常量——加固不变量的量不该做成调参旋钮,判断正确。✓
  5. 测试质量高(tests/issue5278-clamp-safety-margin.test.ts):三次现场事故对钉精确值(142946/120575/175719,来自 issue 里 bundle 抽真实函数的验证算术);系统性性质扫描 u∈{0, .03, .05, .1, .12} × est 1K–220K 全部 ≤ window;装得下的请求零扰动;回退区 #2490/#3071/真投降四个形状逐字节钉死。六个受影响 golden 的改法是从文档公式推导断言值(不再硬编码魔数),且 #453 backstop 的不变式升级为含低估的强形式(capped + ceil(130000×1.05) <= window)——这是借修复机会把回归网织得更密,值得肯定。

非阻塞备注:

  • 分支基于较旧的 master(落后 tip 较多),GitHub 判 MERGEABLE、变更面只限 clamp 域 8 个文件,语义冲突风险低,CI 会按合入态跑终验。
  • 合并后效果可用 #5278 的影响面三条自查:CJK 长会话 + max_tokens 超窗口 + 无新鲜 usage 基线场景下,output budget clamped 日志的钳制值会比旧版收紧最多约 12%×估算值;上游 262145 型 502 应消失。若线上仍见同类 502,说明实际低估率 >17.6%,届时再议 estimator 本身的升级(issue 方案 C)。

本地验证(正式全量留给远程 CI):

  • 定向批 219/219 绿:issue5278 新套件 + issue453-backstop / issue3071 / issue3833 / issue2011 / issue2096 / deferred-tool-budget / issue2490-ledger-floor-doomed-probe / issue2122-clamp-calibration / issue2558-clamp-image / issue3148-upstream-circuit / issue4185 / fix-488-multimodal / lane-decisions-b-class / lane-decisions-matrix / side-request-isolation
  • tsc --noEmit exit 0
  • 无 kernel/配置/env/持久化面变化 ✓

@ranxianglei
ranxianglei merged commit 8acc980 into master Oct 31, 2026
86 of 129 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.964-master.8acc980 (rolling master tag).

npm install -g billion-context@0.0.964-master.8acc980  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working dsh p2 scope/wiring

Projects

None yet

1 participant