Repository navigation
[PR] fix: fail fast with actionable breakdown when fixed overhead alone exceeds budget (#2391) - #2393
ranxianglei wants to merge 2 commits into
Conversation
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.2630-pr.2393.1937c7dThis exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr2393.tgz
npm install -g packageInstall instructions are refreshed on each push. |
Review 结论:初版发现一个阻塞性回归(宽触发踩掉两个既有 pin),已在 PR 分支直接修复(commit 验证过程(证据)① 原 issue 方向 ✓ — #2391 的诊断成立:mandatory per-request floor(固定 system/instruction 文本 + tool schema[+ 逐字图片])≥ 有效预算时,fit gate 是 ② 代码逐行 —
③ 测试有效性 ✓ — 两个新测试都有牙:主断言钉住 actionable breakdown / 零上游调用 / 不转发 / 重试不被冷却掩盖(旧代码会产生不同文案+冷却,对旧代码必挂);guard 测试证明 overhead < 预算时不误伤正常折叠路径。hermetic、 ④ 本地验证 — typecheck clean;PR 正文点名的 6 个定向套件(issue2391 / wire-overhead / preflight-fail-fast / issue1841 / preflight-compress-responses / issue532)= 27/27 绿。 发现的阻塞问题(回归审查):图片计入触发器,踩掉两条 pin 住的 house 路径原 head
根因分层:verdict 本身("floor 无法被 folding 缩小")数学上没错,但图片 token 是不可靠估计(byte-based,非 pixel-tile 上游上 ±15x),arbitration/futility 路径正是为这类情况设计的——把它算进 fail-fast 触发器等于静默移除一个刻意设计的逃生门,也与 PR 正文 "does not alter any fitting-forward path" 的说法矛盾(该说法只对 pre-walk 分支成立,post-walk 仲裁分支被改了)。 修复(commit
本地复验:原先失败的 2 个文件 + issue2391 + wire-overhead + preflight-fail-fast + issue1841 = 26/26 绿;typecheck clean。 遗留设计问题(请 owner 拍板,不阻塞本次):若希望图片计入的 floor( 结构化八问
判定:方向与实现质量成立、测试有效;唯一阻塞项(宽触发踩 #1800/#857 pin)已在 PR 分支修复并本地复验。新 head |
CI 确认:新 head
Review 工作到此收工:可合并(merge 请人工操作)。遗留两件事供 owner 参考(均不阻塞):
|
|
冲突 |
92afec1 to
cb32eff
Compare
上游冲突处理完成 — 分支已 rebase 到当前 master
判定:rebase 完成、本地定向全绿,等 CI 对新 head 的全量确认;绿了即可合并(merge 请人工操作)。CI 结果我会跟进补报。 |
CI 确认:rebase 后 head 冲突处理收工:分支干净、本地定向验证 + CI 全量双绿,可合并(merge 请人工操作)。 |
… exceeds the budget (#2391) When the mandatory per-request floor (fixed system/instruction text + tool-schema overhead + any verbatim images) already reaches the effective input budget, folding conversation can provably never bring the payload under it (the fit gate is '< limit' and this floor lives outside the fold space). Preflight previously ran the doomed walk, ended on a misleading 'no compressible ranges' detail, and armed the #726 dead-end cooldown that told operators to wait when waiting cannot help. Detect it at the decision point and fail fast with a component breakdown (instruction/system vs tool definitions vs verbatim images), the shortfall vs the effective budget, why compression cannot reduce it, and concrete recovery options. No summarization call is spent and no cooldown is armed. - budget.ts: extract wireOverheadBreakdown() so the overflow diagnostic can name which component dominates; estimateWireOverhead() now delegates to it (sum unchanged). - server.ts: early fail-fast in preflightCompressIfNeeded guarded on !payloadFitsWindow (keeps k-hat-calibrated forwards intact) using the raw overhead reading the no-progress fit gate already trusts. Closes #2391
… (images out of the trigger) The original gate (overheadEstimate + imageTokens >= limit) preempted two pinned house paths for image-dominated floors — CI red on every test job: - tests/issue1800-image-arbitration-compress.test.ts (#1800): an image-dominated estimate with compressible text must fold the text portion and forward once so the upstream arbitrates billing; the new check turned it into an immediate 502. - tests/issue857-image-baseline.test.ts C (#857/#1841): a usage-grounded over-window baseline with images alone >= window must reach the futility verdict; the new check substituted its own message first. Root cause: the image token estimate is byte-based and unreliable on pixel-tile billing upstreams (#1800) — the arbitration/futility paths exist precisely for that case, so an estimate-inclusive floor must not outrank them. The #2391 incident itself (systemPromptTokens=271375 > effective budget 204000, no images) is fully covered by fixed text overhead alone, so narrow the trigger to overheadEstimate >= limit. Window fit keeps the raw chars/4 text reading (k^ is a billing scale per #1933, not a token count), so no calibrated-forward path changes either; whenever raw text overhead alone >= limit the payload genuinely cannot fit the window regardless of billing. Verified locally: issue1800 + issue857 + issue2391 + wire-overhead + preflight-fail-fast + issue1841 suites green (26/26); typecheck clean.
cb32eff to
3704fe0
Compare
冲突已处理 — rebase 到当前 master
判定:分支已对齐当前 master、本地定向全绿,等 CI 对新 head 全量确认;绿了即可合并(merge 请人工操作)。CI 结果我会跟进补报。 |
CI 确认:head 过程说明(诚实披露):首跑时 判定:分支对齐当前 master、CI 全绿,可合并(merge 请人工操作)。 |
|
(ai: zhipuai-lb/glm-5.3) 陈旧票清理(第三批动作①):目标 issue #2391 已关。为避免重复劳动关闭本 PR;如判断有误请直接 reopen 并留言。 |
What & why
#2391: when the mandatory per-request floor — fixed system/instruction text + tool-schema overhead, plus any verbatim images — already reaches the effective input budget, folding conversation can provably never bring the payload under it (the fit gate is
< limit, and this floor lives outside the fold space). The reported incident:context ~313402 tokens exceeds the model window 204000 (model=gpt-6.1-sol), with 0 compressible range(s) still visible, wheremetadata.systemPromptTokens = 271375already exceeded the effective budget204000.Before this change that state walked the doomed preflight loop, ended on a misleading "no compressible ranges" detail, and armed the #726 dead-end cooldown — telling operators to wait when waiting cannot help, with no breakdown of which component dominates and no recovery path.
This change detects the overhead-dominant floor at the decision point and fails fast with an actionable diagnostic instead.
Changes
src/server/budget.ts— extractwireOverheadBreakdown(protocol, body)returning{ systemTokens, toolTokens }so a diagnostic can name which component dominates;estimateWireOverhead()now delegates to it (sum unchanged — no trigger/fit drift).src/server.ts(preflightCompressIfNeeded) — early fail-fast when!payloadFitsWindow && overheadEstimate + imageTokens >= limit: names instruction/system vs tool definitions vs verbatim images, the shortfall vs the effective budget, why compression cannot reduce it, and concrete recovery options. No summarization call is spent and no [Bug] Codex 额度中断后恢复长会话,preflight SSE 摘要为空并持续 502 #726 cooldown is armed.tests/issue2391-overhead-floor.test.ts— regression fixture: a Responses/Codex-shaped payload whose fixed overhead alone exceeds the budget (no window bump, no instruction/tool/history deletion), asserting the actionable 502 breakdown, zero upstream calls, no forward, and no cooldown masking on retry; plus an Anthropic guard proving the check defers to the normal fold path when overhead < budget.Behavior change (old → new)
0 compressible range(s) still visible502 + a 5-minute dead-end cooldown (preflightDeadEnd).!payloadFitsWindow, placed after image arbitration so the PR #489 后续: 默认图片计费 b64/4 在像素 tile 计费上游(官方 Anthropic/OpenAI)会硬 502 误杀多模态会话 #496/疑似#767后续:图片计价高估会使preflight自动压缩永久失效,同时请求仍被转发 #1800 house pattern keeps first chance).Uses the raw (un-deflated) overhead reading the existing no-progress fit gate already trusts. No other behavior changes.
Config surface
None added. The diagnostic references the existing
compress.outputHeadroomMaxPctknob as one recovery option; no new / renamed / re-semantics fields.Verification
npm run typecheckclean on the rebased head (built against current master, v0.1.188).issue2391-overhead-floor2/2,wire-overhead10/10,preflight-fail-fast7/7,issue1841-preflight-futility3/3,preflight-compress-responses1/1,issue532-panel-system-tokens4/4.Closes #2391