Skip to content

fix(#2011): floor output-budget caps so max_tokens is always an integer - #2012

Merged
ranxianglei merged 3 commits into
masterfrom
2026-10-03_fix-2011-integer-max-tokens
Oct 4, 2026
Merged

ranxianglei merged 3 commits into
masterfrom
2026-10-03_fix-2011-integer-max-tokens

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Summary

Fixes #2011 — since v0.1.181 the proxy can write a fractional max_tokens into the outgoing request, which strict OpenAI-compatible upstreams reject and which kills the whole agent turn:

400 {"error":{"type":"server_error","message":"Upstream request failed: max_tokens: Input should be a valid integer"}}
422 {"error":{"type":"server_error","message":"Upstream request failed: Endpoint is unavailable."}}

Root cause

clampOutputBudget() (src/server/budget.ts) computes cap = window - inputEstimate - margin. Since 0.1.181, inputEstimate can carry a fraction: the dual-channel image billing work (#1843) adds ctx.imageTokens (a pixel-prior value) onto the estimate in clampOutgoingOutput, so the cap came out fractional (e.g. 110715.5, 95666.75). That value was written verbatim by writeOutputBudget() into the wire's max_tokens / max_completion_tokens / max_output_tokens — fields that are integer-typed on every protocol.

The issue's control group confirms it: every clamp line on v ≤ 0.1.180 is a whole number; the fractions appear only from 0.1.181 onward. The acp-kernel pin is unchanged at 0.0.100 across both versions, so this is a host-side estimate source, not a kernel change.

Fix

Guarantee an integer at both layers so no caller can ever emit a fractional count:

  1. clampOutputBudget floors the cap — keeps the log line (output budget clamped … -> N) consistent with what is actually written.
  2. writeOutputBudget coerces at the sole writer — also guards the restore path ([bug] bili dsh 长会话中 max_output_tokens 被压到 ≤200,主请求被误判为 side request 并绕过压缩 #546/#546 输出预算高水位可被客户端衰减值污染,饿死会话永久钉在小预算(restore 缺下限) #1665) and any future caller at the type boundary.

Math.floor (not round/ceil) so the emitted budget never exceeds the exact headroom (input + output <= window still holds).

Behavior change (old -> new)

  • Fractional inputEstimate (v0.1.181+, the bug): emitted max_tokens goes from a rejected fraction to its floor — e.g. 110715.5 -> 110715, 95666.75 -> 95666. Strictly more conservative by < 1 token; previously these requests were hard-rejected.
  • Whole-number inputEstimate (all of v <= 0.1.180, and any session whose image share is whole): byte-identical — floor of an integer is itself, and the writer's Number.isInteger short-circuit leaves integers untouched. Pinned by the two "no drift" tests.

No threshold, default, timing, or output-format drift beyond making an already-invalid field conform to its declared integer type.

Config surface

None. No new env/config/CLI flag; no schema or persistence change.

Scope note (preflight)

src/preflight.ts's windowClampedOutput also writes an integer-typed max_output_tokens for the summary side-call. Its inputs are all integer-typed today (defaultCountTokens over text + an integer model window; summary calls carry no images), so it has no fractional source in the current codebase and was not part of the reported incident — left untouched to keep this diff scoped to the actual broken path. Noted here as the one adjacent site, in case a future change makes its inputs fractional.

Tests

New tests/issue2011-integer-output-budget.test.ts anchors to the exact 0.1.181 production values:

  • clampOutputBudget(384000, 85032.5, 200000) -> integer 110715; (..., 99364.25, ...) -> 95666.
  • writeOutputBudget floors a fraction for every field shape (max_tokens, max_completion_tokens, max_output_tokens, generationConfig.maxOutputTokens) and passes integers through unchanged.

Verified teeth: reverting the fix makes the two fractional tests fail; restoring it makes them pass. Targeted run green: issue2011 + issue453-backstop + side-request-isolation = 35 pass / 0 fail. My files add zero type errors (the 12 pre-existing typecheck errors in unrelated test files are identical with/without this change).

Regression review (existing behavior touched)

  • clampOutputBudget: only the return value gains a Math.floor. Integer inputs unchanged; the cap < FLOOR || cap >= requested guard now compares against the floored value (sub-token edge only).
  • writeOutputBudget: adds an integer coercion before writing. All existing callers feed integers (verified across tests/side-request-isolation.test.ts), so their behavior is unchanged.
  • Both compression modes affected identically (proxy + plugin) — the guarantee is mode-agnostic and can only turn a previously-rejected request into an accepted one.

Fixes #2011

Since 0.1.181 the input estimate can carry a fraction (image pixel-prior
billing, #1843, adds ctx.imageTokens to the estimate), so clampOutputBudget
produced a fractional cap that clampOutgoingOutput wrote verbatim into the
wire's max_tokens / max_completion_tokens / max_output_tokens. Those fields
are integer-typed on every protocol; strict upstreams rejected the whole turn
with `max_tokens: Input should be a valid integer` (400) or
`Endpoint is unavailable.` (422).

Guarantee an integer at both layers so no caller can emit a fractional count:
clampOutputBudget floors the cap (keeps log == wire), and writeOutputBudget
coerces at the sole writer (also guards the restore path + future callers).
No-op for whole-number estimates, so v<=0.1.180 behavior is byte-identical.

Regression tests anchor to the exact 0.1.181 production values.

Fixes #2011
@github-actions

github-actions Bot commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-03_fix-2011-integer-max-tokens (3923ddd)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.2044-pr.2012.a74bea9

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr2012.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 收到,正在审核本 PR(拉分支核对代码 + CI 状态 + 回归审查)。结论稍后在此更新。

…et comment

The fraction since v0.1.181 comes from #1843 L1's learned per-route image
cost (learnedImageReserve, cache-ledger.ts — EMA over observed/nImages usage
samples), which takes precedence over the integer pixel/byte priors (#488).
The old wording credited 'image pixel-prior billing' for adding ctx.imageTokens,
but those priors are integers and ctx.imageTokens landed much earlier.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 PR #2012 审核结论:修法正确、覆盖完整、无回归引入;CI 红为 master 继承(#2003 已跟踪),非本 PR 引入。可合并。

结构化七问

  1. 是什么问题:v0.1.181 起,带图片学习成本的会话里 plugin mode: no hard backstop when the agent ignores the compress nudge — context grows past the window and upstream 400s #453 output clamp 算出的 cap 变成小数(如 110715.5),被原样写进 wire 上整数类型的 max_tokens/max_completion_tokens/max_output_tokens,严格 upstream 整轮拒绝(400 max_tokens: Input should be a valid integer / 422)。
  2. 什么原因导致的:分层——表面是 upstream 拒小数;机制是 clampOutputBudget 的 cap = window − inputEstimate − margin,而 inputEstimate += ctx.imageTokens;pixel/byte prior 一直是整数,真正的小数源是 v0.1.181 上线的 Bill images and text separately — the image estimate's ±15× error bar poisons every window decision (dual-channel accounting) #1843 L1 learnedImageReserve(cache-ledger.ts,对 observed/nImages usage 样本的 EMA,alpha=0.5)在有新证据时优先于整数 prior。我用 git log -S 验证了归属链:ctx.imageTokens 入 estimate 是 多模态会话上下文控制失效 + preflight 救援 502(codex responses store:false) #488(整数 prior),learnedImageReserve 由 ff538c9 引入且首个发布版本恰为 v0.1.181——与事故版本精确吻合,生产日志 .5/.25 尾数也符合 EMA 折半特征。
  3. 怎么修复的:决策点 + 唯一写入点双层 floor:① clampOutputBudget 内对 cap 取 Math.floor(floor ≤ 精确余量,input+output ≤ window 不变式保持);② 主出站 body 的唯一 writer writeOutputBudget 再做 Number.isInteger ? value : Math.floor(value) 兜底(覆盖 restore 路径中 client-sourced highWater/outputCeiling)。Preflight 自写 summary 请求的值链全整数(windowClampedOutput = min(32k, window−(token估计+256))),排除出范围是对的。非治标:同一根因换形态再现(未来任何分数 estimate 源)两层都仍成立。
  4. 是否有回归问题:逐项审过——整数输入下 Math.floor 为 no-op,字节级一致(有专门测试钉住);clamp guard(OUTPUT_CLAMP_FLOOR/cap >= requested)逻辑未动;日志行现在打印与写出值一致的 floored cap(此前打印的正是被原样写出导致 400 的小数,现二者一致);restore 路径语义微变:客户端若送小数预算,旧行为原样转发(可能触发 upstream 400),新行为 floor 后转发——向合法方向收敛,按规则披露;plugin/proxy 两种压缩模式走同一 clampOutgoingOutput,无分叉;未触碰 fail-fast、阈值、时序。
  5. 是否破坏缓存:不涉及。不碰 session 状态、prefix-cache 字节流、重试/增量游标、去重键;floor 只作用于出站 body 单个标量字段。
  6. 是否改了设计方向:不涉及。单模块(budget/side-request)修复,无架构变化。
  7. 是否引入了新的配置:不涉及。

测试有效性:4 个新测试有牙——我把两个 src 文件回退到 master 版本复跑,2/4 失败(小数 cap + 小数 wire write 两个用例精确抓到原 bug),恢复后 4/4 通过。定向跑 issue2011 + issue453-backstop + side-request-isolation + fix-1075-side-request:41 pass / 0 fail。

CI 状态:base 为当前 master(33d58e5e,#1989 合入后),无需 rebase。e2e 各泳道全绿(真实 opencode/dsh/pi/codex native、hermetic registry ubuntu+windows、build-artifact、version-guard)。红的 4 个 test job 死在 typecheck 阶段(exit 2,测试根本没跑)——我做了 worktree 对照:master vs PR head 的 typecheck 错误集完全相同(12=12,diff identical),即纯继承自 master,非本 PR 引入。该问题已由 #2003 跟踪(fix PR #2006/#2007 在途),落地后本 PR required checks 自然转绿;也可基于上述证据直接提前合并。

顺手小修(已推 PR 分支,commit aa21543):budget.ts 注释把小数来源误归为 "image pixel-prior billing, #1843 adds ctx.imageTokens"——pixel prior 是整数、ctx.imageTokens 早在 #488 就入 estimate,真正小数源是 #1843 L1 学习 EMA。已改正措辞(独立 docs commit,不影响逻辑)。

diff 整洁度:3 文件 +79/-3(含注释修正后 +81/-5),每行均与本 issue 相关,无跑题内容;version 未动 ✓。

判定:可合并。 合并受继承自 master 的 typecheck 红阻塞(非本 PR 所致)——待 #2006/#2007 落地或凭上述证据提前合并均可。我无法代合并,请自行操作:#2012

@ranxianglei
ranxianglei merged commit 38536f4 into master Oct 4, 2026
15 checks passed
@ranxianglei
ranxianglei deleted the 2026-10-03_fix-2011-integer-max-tokens branch October 15, 2026 05:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bili 0.1.181:clampOutputBudget 把小数写进 max_tokens,上游 400 max_tokens: Input should be a valid integer / 422 Endpoint is unavailable

1 participant