Skip to content

[PR] fix: fail fast with actionable breakdown when fixed overhead alone exceeds budget (#2391) - #2393

Closed
ranxianglei wants to merge 2 commits into
masterfrom
2026-10-08_issue2391-overhead-floor-diagnostic
Closed

ranxianglei wants to merge 2 commits into
masterfrom
2026-10-08_issue2391-overhead-floor-diagnostic

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

What & why

#2391: when the mandatory per-request floor — fixed system/instruction text + tool-schema overhead, plus any verbatim images — already reaches the effective input budget, folding conversation can provably never bring the payload under it (the fit gate is < limit, and this floor lives outside the fold space). The reported incident: context ~313402 tokens exceeds the model window 204000 (model=gpt-6.1-sol), with 0 compressible range(s) still visible, where metadata.systemPromptTokens = 271375 already exceeded the effective budget 204000.

Before this change that state walked the doomed preflight loop, ended on a misleading "no compressible ranges" detail, and armed the #726 dead-end cooldown — telling operators to wait when waiting cannot help, with no breakdown of which component dominates and no recovery path.

This change detects the overhead-dominant floor at the decision point and fails fast with an actionable diagnostic instead.

Changes

  • src/server/budget.ts — extract wireOverheadBreakdown(protocol, body) returning { systemTokens, toolTokens } so a diagnostic can name which component dominates; estimateWireOverhead() now delegates to it (sum unchanged — no trigger/fit drift).
  • src/server.ts (preflightCompressIfNeeded) — early fail-fast when !payloadFitsWindow && overheadEstimate + imageTokens >= limit: names instruction/system vs tool definitions vs verbatim images, the shortfall vs the effective budget, why compression cannot reduce it, and concrete recovery options. No summarization call is spent and no [Bug] Codex 额度中断后恢复长会话,preflight SSE 摘要为空并持续 502 #726 cooldown is armed.
  • tests/issue2391-overhead-floor.test.ts — regression fixture: a Responses/Codex-shaped payload whose fixed overhead alone exceeds the budget (no window bump, no instruction/tool/history deletion), asserting the actionable 502 breakdown, zero upstream calls, no forward, and no cooldown masking on retry; plus an Anthropic guard proving the check defers to the normal fold path when overhead < budget.

Behavior change (old → new)

Uses the raw (un-deflated) overhead reading the existing no-progress fit gate already trusts. No other behavior changes.

Config surface

None added. The diagnostic references the existing compress.outputHeadroomMaxPct knob as one recovery option; no new / renamed / re-semantics fields.

Verification

  • npm run typecheck clean on the rebased head (built against current master, v0.1.188).
  • Targeted suites green: issue2391-overhead-floor 2/2, wire-overhead 10/10, preflight-fail-fast 7/7, issue1841-preflight-futility 3/3, preflight-compress-responses 1/1, issue532-panel-system-tokens 4/4.
  • Full suite + codex e2e run in CI.

Closes #2391

@github-actions

github-actions Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-08_issue2391-overhead-floor-diagnostic (3704fe0)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.2630-pr.2393.1937c7d

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr2393.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

已接手。按 PR Review 规范走:①原 issue #2391 方向核查 → ②回归审查(既有 preflight/fit/cooldown 行为清点)→ ③代码精读 → ④测试有效性 → ⑤diff 整洁度;本地只跑定向套件,全量交 CI。结论会按结构化八问回复。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

Review 结论:初版发现一个阻塞性回归(宽触发踩掉两个既有 pin),已在 PR 分支直接修复(commit 92afec170),等 CI 最终确认。

验证过程(证据)

① 原 issue 方向 ✓ — #2391 的诊断成立:mandatory per-request floor(固定 system/instruction 文本 + tool schema[+ 逐字图片])≥ 有效预算时,fit gate 是 < limit 且 floor 在可折叠空间之外,folding 在数学上永远无法达标;旧路径却要走完 doomed walk、以误导性的 0 compressible range(s) 收尾,并武装 #726 的 5 分钟冷却(等待帮不上忙却告诉 operator 去等)。事故数字(systemPromptTokens=271375 > 有效预算 204000,gpt-6.1-sol,272k 窗口 × 25% output reservation)被本修法的触发条件直接覆盖。用户硬约束(保持 272k 窗口、不扩窗)不受影响——诊断只列选项。

② 代码逐行 —

③ 测试有效性 ✓ — 两个新测试都有牙:主断言钉住 actionable breakdown / 零上游调用 / 不转发 / 重试不被冷却掩盖(旧代码会产生不同文案+冷却,对旧代码必挂);guard 测试证明 overhead < 预算时不误伤正常折叠路径。hermetic、listen(0) 确定性。

④ 本地验证 — typecheck clean;PR 正文点名的 6 个定向套件(issue2391 / wire-overhead / preflight-fail-fast / issue1841 / preflight-compress-responses / issue532)= 27/27 绿。

发现的阻塞问题(回归审查):图片计入触发器,踩掉两条 pin 住的 house 路径

原 head c0ef2cb 上 CI 4/4 test job 全红(ubuntu-22/24、windows-22/24,同一组 2 个用例,确定性失败,本地复现):

  1. tests/issue1800-image-arbitration-compress.test.ts(疑似#767后续:图片计价高估会使preflight自动压缩永久失效,同时请求仍被转发 #1800):image-dominated 估计 + 可折叠文本的既有契约 = 先折文本部分再 forward 一次,让上游仲裁计费(pixel-tile 上游实际计费远低于字节估计——这正是 疑似#767后续:图片计价高估会使preflight自动压缩永久失效,同时请求仍被转发 #1800 存在的原因)。初版条件 overheadEstimate + imageTokens >= limit 直接 502,抢在这个逃生门之前。
  2. tests/issue857-image-baseline.test.ts C([Question] 非白名单上游(如 DeepSeek)图片按 b64/4 估算,实测高估约 100×;且该估算被写回 lastInputTokens 并自愈成 confirmed 窗口,使 #496 逃生门永久失效 #857/模型压缩输出是输入的数倍 #1841):usage-grounded over-window 基线 + 图片单独 ≥ 窗口的既有契约 = futility verdict(futile round);初版的新消息抢先替换了它。

根因分层:verdict 本身("floor 无法被 folding 缩小")数学上没错,但图片 token 是不可靠估计(byte-based,非 pixel-tile 上游上 ±15x),arbitration/futility 路径正是为这类情况设计的——把它算进 fail-fast 触发器等于静默移除一个刻意设计的逃生门,也与 PR 正文 "does not alter any fitting-forward path" 的说法矛盾(该说法只对 pre-walk 分支成立,post-walk 仲裁分支被改了)。

修复(commit 92afec170,已推 PR 分支):触发条件收窄为 overheadEstimate >= limit(仅固定 system+tools 文本,图片不入触发器)。理由:

本地复验:原先失败的 2 个文件 + issue2391 + wire-overhead + preflight-fail-fast + issue1841 = 26/26 绿;typecheck clean。

遗留设计问题(请 owner 拍板,不阻塞本次):若希望图片计入的 floor(overhead + images ≥ limit)也 fail-fast,那意味着对这类 payload 有意移除 #1800 仲裁逃生门——需要刻意重写上述两个 pin 测试并更新 PR 正文。我的建议:按当前窄版本合并;图片计入 fail-fast 另立决策。另注意:PR 正文 Changes/Behavior-change 节描述的仍是原宽触发,合并前需同步更新。

结构化八问

  1. 是什么问题:mandatory 固定开销(system/instructions + tool schema)单独达到有效预算时,folding 可证地永远无法满足 < limit gate,旧路径却走 doomed 循环、以误导性 "0 compressible range(s)" 收尾并武装 5 分钟冷却死路。
  2. 什么原因导致的:决策点缺少 floor 检查——floor 位于可折叠空间之外,由它造成的溢出结构性不可修,但 preflight 只在走完 walk 之后才发现这一点(还顺带武装了冷却)。
  3. 影响范围与严重程度:任何固定 instruction/system+tools 文本达到有效预算 ~100% 的会话(如 [Bug][Codex Responses] fixed instruction/tool overhead exceeds 272k budget; preflight has zero compressible ranges #2391:271375 vs 204000)每个请求完全卡死,伴随误导诊断 + 重复冷却;严重度 = 特定条件下不可用(大固定提示词 / 小窗口 / 高 output reservation),有 ranges 的变体还额外浪费 summary 调用。
  4. 怎么修复的:在决策点(walk 之前)检测 floor 并立即 fail-fast——组件级 breakdown(system vs tools)、shortfall、明确的 "compression CANNOT reduce it"、恢复选项;不花 summarization 调用、不武装冷却。review 修正:图片移出触发器(其估计不可靠,该情形归 house 路径管)。
  5. 是否回归:逐项审过——① trigger/fit 总和:不变(委托保持同项);② calibrated forward:保持(!payloadFitsWindow 守卫);③ pre-walk image-arbitration 立即返回:保持(新检查在其后);④ post-walk 疑似#767后续:图片计价高估会使preflight自动压缩永久失效,同时请求仍被转发 #1800 仲裁 forward:初版破坏(CI 红)→ 收窄后恢复;⑤ 模型压缩输出是输入的数倍 #1841 futility verdict:初版破坏 → 收窄后恢复;⑥ [Bug] Codex 额度中断后恢复长会话,preflight SSE 摘要为空并持续 502 #726 冷却:下游耗尽路径照常武装,新路径不武装;⑦ HTTP 形状(code/retryable/Retry-After):wrapper 未动。未见其他回归。
  6. 是否破坏缓存:否——持久化格式/基线/快照/校准状态/去重键均未变;新路径不写任何 session 状态(不武装 preflightDeadEnd,这正是要点之一)。
  7. 是否改设计方向:无架构转向——在既有 preflight 流程内加一个决策点守卫。明确了一条语义边界:fail-fast verdict 基于 raw 文本 overhead 读数;图片留在 estimate-arbitration 体制内。
  8. 是否引入新配置:无。诊断仅引用既有 compress.outputHeadroomMaxPct 作为恢复选项之一,无新增/改名/改语义字段。

判定:方向与实现质量成立、测试有效;唯一阻塞项(宽触发踩 #1800/#857 pin)已在 PR 分支修复并本地复验。新 head 92afec170 的 CI 正在跑(e2e 各 lane 已绿,test job 进行中)——全绿后我再确认一条;届时即可人工合并。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

CI 确认:新 head 92afec170 全绿(28/28 checks success)。

Review 工作到此收工:可合并(merge 请人工操作)。遗留两件事供 owner 参考(均不阻塞):

  1. PR 正文 Changes/Behavior-change 节仍描述原宽触发(overhead + images ≥ limit),实际代码为 overheadEstimate >= limit(仅固定文本),合并前建议同步更新;
  2. 图片计入 floor 的 fail-fast 是否要另做(需有意移除 疑似#767后续:图片计价高估会使preflight自动压缩永久失效,同时请求仍被转发 #1800 仲裁逃生门并重写两个 pin 测试)——留待单独决策。

@ranxianglei

Copy link
Copy Markdown
Owner Author

冲突

@ranxianglei
ranxianglei force-pushed the 2026-10-08_issue2391-overhead-floor-diagnostic branch from 92afec1 to cb32eff Compare October 8, 2026 14:50
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

上游冲突处理完成 — 分支已 rebase 到当前 master cff9eb029,新 head cb32effec(force-push 仅限本 PR 分支,非 master)。八问:

  1. 是什么问题:PR 分支与基线 dirty——期间 master 合入 8 个 PR,其中 fix: account for deferred Responses tools in context budgets (#2391) #2398(deferred Responses tools)恰好改动了本 PR 拆分所在的同一个函数 estimateWireOverhead,产生文本冲突 + 语义交互。
  2. 什么原因导致的:冲突点在 src/server/budget.ts 的 return 语句——master 侧把工具计数从「全量 tools」改成「modelVisibleTools() 可见投影 + countLoadedToolTokens() 已加载定义」,fix: account for deferred Responses tools in context budgets (#2391) #2398 的预算语义与本 PR 的 breakdown 拆分落在同一行。
  3. 影响范围与严重程度:冲突本身不影响已发布行为;风险在解法——若保留 PR 侧旧计数(全量 tools)会回退 fix: account for deferred Responses tools in context budgets (#2391) #2398 的预算语义;若只留 master 侧则诊断失去分量拆解。按「union of intent」解决后两者兼得。
  4. 怎么修复的:rebase 而非 merge(线性历史)。budget.ts 冲突解为并集:wireOverheadBreakdown 返回 { systemTokens, toolTokens },toolTokens 采用 master 的可见投影+已加载定义语义,总和与 master 的 estimateWireOverhead 结构上恒等(同两项相加)→ 触发器/fit 判定零漂移。server.ts 自动合并后逐 hunk 核查:master 在该函数内的改动仅两处注释(k̂ 自 长会话中token用量被系统性低估(CJK 文本),导致agent误判上下文压力、进入「小收益折叠」循环 #2366 起双向 0.25–4),无代码变化。
  5. 是否有回归问题:逐项过既有行为——① estimateWireOverhead 总和:与 master 恒等,无漂移;② fix: account for deferred Responses tools in context budgets (#2391) #2398 deferred-tools 预算会计:完整保留;③ 长会话中token用量被系统性低估(CJK 文本),导致agent误判上下文压力、进入「小收益折叠」循环 #2366 双向 k̂:未触碰,fail-fast 触发用原始 overheadEstimate(与校准方向无关——固定文本开销 ≥ limit 时任何 k̂ 都救不回 fit),注释仍准确;④ 疑似#767后续:图片计价高估会使preflight自动压缩永久失效,同时请求仍被转发 #1800/模型压缩输出是输入的数倍 #1841/[Question] 非白名单上游(如 DeepSeek)图片按 b64/4 估算,实测高估约 100×;且该估算被写回 lastInputTokens 并自愈成 confirmed 窗口,使 #496 逃生门永久失效 #857 图片路径:触发器保持上一轮收窄后的「仅固定文本」形态,随 rebase 原样带入;⑤ responses 协议下诊断的 "tool definitions" 数值现在反映模型可见投影+已加载定义——有意为之,与真实 per-request schema 成本一致。本地定向验证:8 个相关套件(issue2391/wire-overhead/preflight-fail-fast/issue1841/preflight-compress-responses/issue532/issue1800-image-arbitration/issue857)31/31 绿,typecheck 干净;全量套件 + codex e2e 交 CI。
  6. 是否破坏缓存:不涉及——未触碰缓存/重试/游标/去重逻辑。
  7. 是否改了设计方向:不涉及。
  8. 是否引入了新的配置:不涉及。

判定:rebase 完成、本地定向全绿,等 CI 对新 head 的全量确认;绿了即可合并(merge 请人工操作)。CI 结果我会跟进补报。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

CI 确认:rebase 后 head cb32effec 全量 28/28 全绿(4 个 test job ubuntu-22/24 + windows-22/24,以及 e2e、kernel-guard、pi-subagents-guard、docs-sync 等全部通过)。

冲突处理收工:分支干净、本地定向验证 + CI 全量双绿,可合并(merge 请人工操作)。

awork added 2 commits October 8, 2026 23:09
… exceeds the budget (#2391)

When the mandatory per-request floor (fixed system/instruction text + tool-schema
overhead + any verbatim images) already reaches the effective input budget, folding
conversation can provably never bring the payload under it (the fit gate is '< limit'
and this floor lives outside the fold space). Preflight previously ran the doomed walk,
ended on a misleading 'no compressible ranges' detail, and armed the #726 dead-end
cooldown that told operators to wait when waiting cannot help.

Detect it at the decision point and fail fast with a component breakdown
(instruction/system vs tool definitions vs verbatim images), the shortfall vs the
effective budget, why compression cannot reduce it, and concrete recovery options.
No summarization call is spent and no cooldown is armed.

- budget.ts: extract wireOverheadBreakdown() so the overflow diagnostic can name which
  component dominates; estimateWireOverhead() now delegates to it (sum unchanged).
- server.ts: early fail-fast in preflightCompressIfNeeded guarded on !payloadFitsWindow
  (keeps k-hat-calibrated forwards intact) using the raw overhead reading the
  no-progress fit gate already trusts.

Closes #2391
… (images out of the trigger)

The original gate (overheadEstimate + imageTokens >= limit) preempted two pinned
house paths for image-dominated floors — CI red on every test job:

- tests/issue1800-image-arbitration-compress.test.ts (#1800): an image-dominated
  estimate with compressible text must fold the text portion and forward once so
  the upstream arbitrates billing; the new check turned it into an immediate 502.
- tests/issue857-image-baseline.test.ts C (#857/#1841): a usage-grounded
  over-window baseline with images alone >= window must reach the futility
  verdict; the new check substituted its own message first.

Root cause: the image token estimate is byte-based and unreliable on pixel-tile
billing upstreams (#1800) — the arbitration/futility paths exist precisely for
that case, so an estimate-inclusive floor must not outrank them. The #2391
incident itself (systemPromptTokens=271375 > effective budget 204000, no images)
is fully covered by fixed text overhead alone, so narrow the trigger to
overheadEstimate >= limit. Window fit keeps the raw chars/4 text reading (k^ is a
billing scale per #1933, not a token count), so no calibrated-forward path
changes either; whenever raw text overhead alone >= limit the payload genuinely
cannot fit the window regardless of billing.

Verified locally: issue1800 + issue857 + issue2391 + wire-overhead +
preflight-fail-fast + issue1841 suites green (26/26); typecheck clean.
@ranxianglei
ranxianglei force-pushed the 2026-10-08_issue2391-overhead-floor-diagnostic branch from cb32eff to 3704fe0 Compare October 8, 2026 15:14
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

冲突已处理 — rebase 到当前 master d189be532,新 head 3704fe0ab(force-push 仅限本 PR 分支)。八问:

  1. 是什么问题:master 在上次 rebase 后又前移 12 个 commit(含 refactor(server): extract proxy request pipeline handle() into src/server/handle.ts (#1440 P2 cut 3) #2424「extract proxy request pipeline handle() into src/server/handle.ts」对 src/server.ts 的大规模搬移 −2007/+17),PR 分支落后报冲突。
  2. 什么原因导致的:热文件竞争——refactor(server): extract proxy request pipeline handle() into src/server/handle.ts (#1440 P2 cut 3) #2424 重构与 PR 改动的 preflightCompressIfNeeded 同文件;本次 git 自动合并干净(无文本冲突),因为 refactor(server): extract proxy request pipeline handle() into src/server/handle.ts (#1440 P2 cut 3) #2424 只搬移调用方、函数本体留在 server.ts,budget.ts 在窗口期内未被 master 触碰。
  3. 影响范围与严重程度:仅 PR 分支新鲜度问题,不影响任何已发布行为。
  4. 怎么修复的:rebase(保持线性历史)。落点逐一验证:fail-fast 触发块现位于 src/server.ts:2591-2611(行号因搬移上移,位置关系不变)——仍在 PR #489 后续: 默认图片计费 b64/4 在像素 tile 计费上游(官方 Anthropic/OpenAI)会硬 502 误杀多模态会话 #496/疑似#767后续:图片计价高估会使preflight自动压缩永久失效,同时请求仍被转发 #1800 图片仲裁 immediate-return(2541-2546)之后、compressible-ranges 检查(2612)之前,[Bug] Codex 额度中断后恢复长会话,preflight SSE 摘要为空并持续 502 #726 cooldown 与 walk 及 post-walk 仲裁(2803)全部在其下游;触发条件 !payloadFitsWindow && overheadEstimate >= limit(图片不入触发)原样保留,注释块准确。
  5. 是否有回归问题:窗口期 master 变更对本 PR 触碰路径的既有行为逐项过——① budget.ts:零改动,estimateWireOverhead 总和恒等结论不变;② refactor(server): extract proxy request pipeline handle() into src/server/handle.ts (#1440 P2 cut 3) #2424 handle 抽取:preflightCompressIfNeeded 函数体除本 PR hunk 外零变化,调用点 handle.ts:1860 语义不变;③ 【bug】bili 0.1.188 · ACP 标签形态异常 + 摘要泄漏 + 剥离日志可观测性 #2405 strip-log / fix: treat empty finish_reason as absent, not terminal (#2405 part b) #2420 empty-finish-reason:不触碰 preflight/budget 路径,无交互。本地定向验证:8 个相关套件(issue2391/wire-overhead/preflight-fail-fast/issue1841/preflight-compress-responses/issue532/issue1800-image-arbitration/issue857)31/31 绿,typecheck 干净;全量 + e2e 交 CI。
  6. 是否破坏缓存:不涉及。
  7. 是否改了设计方向:不涉及。
  8. 是否引入了新的配置:不涉及。

判定:分支已对齐当前 master、本地定向全绿,等 CI 对新 head 全量确认;绿了即可合并(merge 请人工操作)。CI 结果我会跟进补报。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

CI 确认:head 3704fe0ab 全绿(28/28)。

过程说明(诚实披露):首跑时 test (windows-latest, 24) 挂了一个用例——tests/issue411-abort-usage.test.ts「#411 plugin stream: client abort…」的 waitFor 3s 超时。判定为环境性时序抖动,与本 PR 无关,证据:① 本 PR diff 只触碰 preflight/budget 路径,不碰 plugin stream abort 代码;② 同一 commit 在 ubuntu-22/24、windows-22 三个 test job 全过;③ 本地 Linux 跑该文件 5/5 绿;④ 该测试文件自带注释(#2305 review)承认此断言在 loaded windows-latest runner 上有已知 flake 史,本次失败点正是该 waitFor。已 rerun-failed-jobs,复跑通过 → 全量 28/28 success。

判定:分支对齐当前 master、CI 全绿,可合并(merge 请人工操作)。

@ranxianglei

Copy link
Copy Markdown
Owner Author

(ai: zhipuai-lb/glm-5.3)

陈旧票清理(第三批动作①):目标 issue #2391 已关。为避免重复劳动关闭本 PR;如判断有误请直接 reopen 并留言。

@ranxianglei ranxianglei changed the title fix: fail fast with actionable breakdown when fixed overhead alone exceeds budget (#2391) [PR] fix: fail fast with actionable breakdown when fixed overhead alone exceeds budget (#2391) Oct 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug][Codex Responses] fixed instruction/tool overhead exceeds 272k budget; preflight has zero compressible ranges

1 participant