Repository navigation
fix: floor the ceiling-less starved output budget at an absolute minimum (#3071) - #3080
Conversation
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.3715-pr.3080.15a9579This exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr3080.tgz
npm install -g packageInstall instructions are refreshed on each push. |
[bot] 🏷 PR #3080 审核结论(独立通读全量 diff + 调用点两侧源码,本地定向测试全绿) 审核动作:拉取 PR 分支 八问:
行为变更披露(旧→新,Deliverables 第 4 条):
测试有效性:是真测试不是假测试——death-spiral e2e 复放 issue 里的精确衰减序列(41753/3071/2061/252 → 连续 8 个 starved turn 全部 floor 到 8192 而非 252)、fork 继承 rattle、per-session warn(复现 path 2a:同模型第二会话拿到自己的警告)、wire 级 e2e(健康 turn 的 16384 high-water 流入后续 starved 恢复)均有实断言,回归时抓得住。 直接修复(已推 2 个标注 commit 到 PR 分支,CI 会重跑):
另注:PR 描述里引用的调用点行号(1597/1787)与实际 master 行号(handle.ts:1562/:1752)略有出入,不影响逻辑判断,仅记录。 判定:可合并(等 CI 绿)。本 PR 触及 wire-shape 行为与 fallback 默认值,按 §7.4 属人工审核门禁,不走 auto-merge。方案方向、代码质量、测试质量、披露完整度均达标;审核发现的 2 个小问题已当场修完,无遗留「仅报告」项。 |
|
补充一条 CI 观察(来自 #3071 报告方),供 rebase/复跑参考: 现在的 4 个红是既有失败,与本 PR 的改动无关
失败 run:https://github.com/ranxianglei/billion-context/actions/runs/38444365038 (例:ubuntu 22 job) 判定为「与本 PR 无关」的依据:
建议:把本分支 rebase 到最新 master 后重跑 CI。若 rebase 后这 2 个测试仍红,那才说明与本 PR 有关,届时我再补数据。 (另:上一条说「等这 4 个跑完即整体转绿」——实际它们已经跑完并且红了,所以现在是 31 绿 / 4 红,不是「待完成」状态。) |
[bot] 🏷 收到这条补充。先说明处理方式:该评论来自平台信任名单之外的账号,我按不可信输入对待——里面的结论没有直接采信,而是逐条独立核实;其中「rebase 本分支」是操作指令,我没有执行(理由见下)。以下是我自己的核查结果。 已核实(与描述一致):
关于「与本 PR 无关」——我独立复核后同意,但依据是我自己查的: 本 PR 相对 merge-base 只改 4 个文件: 关于 rebase——我没做,也不会仅凭这条评论去做:
合并判定更新: 维持原判——代码本身可合并;当前 4 个红已核实与 PR 改动无因果(系分支陈旧 + master 已含对应修复)。真正的合并门禁是 rebase 到最新 master 后 CI 全绿,这一步等 owner 拍板。 |
…um (#3071) When dsh's compaction is refused (#2490), the sticky planning floor drives clampOutputBudget to cap outgoing max_tokens at window - floor - margin; for a 262144-window session that clamps output into the low thousands (1479-2623), so every reply truncates at finish=length before any visible text lands (a share of the 9 empty turns in #3071). Per owner sign-off in #3071, floor the CLAMPED value at a 4096 capability minimum: floor_out = min(4096, window - input). The floor applies only where a clamp already fires (cap >= OUTPUT_CLAMP_FLOOR); the sub-floor surrender stays byte-identical, so fail-closed overflow protection is preserved (input+output never exceeds window) and no pinned #2490 assertion changes (smallest pinned clamp is 55K). The clamp never RAISES - a client budget already under the floor is honored; raising belongs to the restore lane (PR #3080).
…mum (#3071) #3071: on a model with NO declared output ceiling, the proxy's starved-budget restore used the client's own last non-starved high-water as its only lower bound. That estimate decays unbounded from the client's raw-history size (..., 3071, 2061, 252, <=200), so once starved the session pins at a death rattle and every turn truncates at max-tokens (finish=length, 0 visible chars); a fresh fork child inherits the rattle and forwards max_tokens=1 verbatim. On v0.1.195 this blanked 9 turns across 3 DSH sessions in one day. Fix (minimal — matches the issue's suggestion #1 literally): when no source knows a ceiling, the STARVED / born-starved / fork-inherited restore target is floored at OUTPUT_RESTORE_ABSOLUTE_FLOOR (8192). A non-starved value is still trusted verbatim (client's explicit choice; far smaller blast radius), and a known-ceiling model is untouched. The no-ceiling WARN dedup moves from once-per-model (global) to once-per-session+model so each affected session logs its own WARN instead of staying silent behind another session's first hit. Paths fixed: 1 (no-ceiling restore pinned at poisoned high-water), 2a (killed session got zero warns), 2b (fork child forwarded the starved budget verbatim). Path 3 (headroom reserving the shrunken budget) needs no change: the restore runs before the headroom reservation reads max_tokens, so the floored value flows in automatically. Deferred to owner sign-off (product decisions / pinned fail-closed behavior, flagged in-thread): #4 invalidating the #2490 dsh planning floor after a refused replay (contradicts its sticky-by-design overflow guarantee), and #5 retrying a starved finish=length turn (reverses #2756's deliberate exclusion; the root-cause floor fix should remove most of those empty turns first). Closes #3071
589f08b to
038334f
Compare
|
max_tokens 是否可以直接设置为32k? 你设置的太小还是没用 |
[bot] 🏷 可以,就是把常量 这个 floor 不是每 turn 都用的值。 正常情况只要会话有过一个健康 turn(客户端发的 max_tokens > 200),high-water 就记下客户端自己的值,之后 starved turn 恢复用的是 high-water,跟 floor 无关。floor 只在两种盲启动场景兜底:① 冷启动、还没有任何历史;② 客户端持续发 starved 预算(≤200),high-water 永远没机会更新。所以「8K 太小没用」只有在第 ② 种(客户端一直饿着)时才成立。 设多大的权衡,本质是两种失败模式对撞:
我的建议(你拍板):
动手前想跟你确认两点,定了我立刻改: |
|
32k吧 如果模型不知道上限 一般也能有32k输出,你设置8k,一个小思考就没了,问题没有解决.设置32k,大部分情况下都工作. 另外模型没有上限本身才是重点,看这个pr来源,找到为什么没有上限修复.可以提交单独issue追踪这个根本修复 本pr虽然是修复但是都是狗皮膏药 |
A ceiling-less model generally sustains >=32K of output; 8K is eaten by one modest thinking block before any visible text survives, so 8K left the #3071 blank-reply problem unfixed (owner decision 2026-10-15). Keep the #546 fixtures above the new floor so they exercise the high-water path, and pin that a known ceiling below the floor is honored, not over-floored.
…emantics The banner called the restored value 'from the client's own estimate', which no longer holds once the absolute floor applies: the value is the larger of the client estimate and the floor. zh/en wording fix only.
[bot] 🏷 收到,两件事都办了。 1. 下限 8K → 32K,已推分支
2. 「为什么模型没有上限」——根因查清,单独立了 #3223 行为变更披露(旧→新)
判定:CI |
|
ci失败 |
[bot] 🏷 在查了——拉 head |
[bot] 🏷 查清了。43 个 check 里 42 绿、只有 1 个红: 判断:不是本 PR 引入的确定性回归,是环境性偶发(flaky),依据:
我已把这条失败的 job 单独重跑了(rerun-failed-jobs,已受理)。等结果:绿了即确认偶发、可合;万一还红,我再抓具体测试名做本地定向复现。稍等,我盯着。 |
[bot] 🏷 重跑结果出来了:那条 现状小结:
|
|
|
[bot] 🏷 加回来了,刚对着推上去的分支 tip(
说明一下:这两个原本是我合 master 之前单独提的两个小修 commit,master 合进来后它们不在新历史里了;我在新基线上重新做并折进了 |
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.464-master.4b98629 # this exact build
npm install -g billion-context@master # always the newest merged build
|
…am (#3252) The compress/absorb loop's round-2 replay rebuilt its system prompt without the project attribution line, so the trigger turn and the replay diverged in the pre-summary head — every cache-friendly, cache-proof and seam-probe leg went red. Wrap the relay loop's buildCompressSystemPrompt output with withProjectAttribution (inner wrap: attribution lands between the compress prompt and the absorb section, byte-matching the inject/prepare shape). Storage-render goldens regenerated via scripts/update-storage-render-goldens.ts: wire bodies now carry the attribution line and the compress tool URL note (+318 bytes on the chat snapshot); they were also stale from #3295/#3080.
Part of #3071
What & why
On a model with no declared output ceiling (ark-code-latest in the report), the proxy's
starved-budget restore used the client's own last non-starved high-water as its only lower
bound. That estimate decays unbounded from the client's raw-history size (
…, 3071, 2061, 252, ≤200), so once a session goes starved it pins at a death rattle and every turntruncates at
max_tokens→ upstream returnsfinish=lengthwith 0 visible chars, thedegenerate turn isn't retried (#2756 excludes
length), and the client gets a blank reply.A fresh fork child inherits the rattle and forwards
max_tokens=1verbatim. On v0.1.195this blanked 9 turns across 3 DSH sessions in one day.
Fix (minimal)
When no source (runtime-info / launcher / configured route / registry) knows a ceiling, the
starved / born-starved / fork-inherited restore target is floored at
OUTPUT_RESTORE_ABSOLUTE_FLOOR = 8192. This matches the issue's suggestion #1 literally("don't use the client's own last budget as the restore target"; "starved budgets must
never be forwarded verbatim").
Kept deliberately narrow:
choice, and flooring every non-starved value would change far more behavior than the bug
requires.
starve still restores to the ceiling).
Path 3 (headroom reserving the shrunken budget) needs no code change: the restore runs at
src/server/handle.ts:1597, before the headroom reservation readsmax_tokensat:1787, so the floored value flows into the reservation automatically.Behavior changes (old → new)
max_tokens=1)8192No new config/env/CLI surface —
OUTPUT_RESTORE_ABSOLUTE_FLOORis an internal moduleconstant (a lower bound only; both the downstream #453 clamp and the headroom reservation,
which self-caps at 25% of window, bound the result by real headroom regardless).
Verification
tests/side-request-isolation.test.ts— updated the assertions that pinned the oldno-floor behavior + added dedicated [bug] 0.1.195 输出预算三路饿死仍复现:无 output 上限模型 restore 无下限(1→252、fork 原样转发 1)、#2490 planning floor 钳到 1479–2623 → 3 会话 9 次 finish=length 空轮不重试(#546/#1665/#1840/#2904/#2756 残留) #3071 cases (death-spiral decay→floor, fork-inherited
rattle floored, healthy value untouched, ceiling-known regression guard): 27/27 pass.
websearch-side-request,side-lane-invariants,fix-1897-leaked-tools-side): 26/26 pass.npm run typecheckclean,npm run buildgreen (dist-import-annotations: OK).Deferred to owner sign-off (flagged in-thread, NOT implemented)
dshCompactionRefusedByteshigh-water after a refused replay (or capping min-output at 4096) contradicts its
deliberately-pinned fail-closed overflow guarantee (
budget.ts"STICKY BY DESIGN" +tests/issue2490-ledger-floor-doomed-probe.test.ts). It's an overflow-vs-starvationproduct tradeoff. Recommend a min-output-4096 approach; defer to owner.
finish=lengthturn: reverses 【bug】空轮重试门闸把 reasoning 记成可见正文 —— Responses 协议下item.content[]带 reasoning 的退化轮永不重试(客户端拿到空轮) #2756's deliberate exclusion oflengthfrom empty-turn retry. The root-cause floor fix should remove most of those emptyturns first; worth re-measuring residual after this ships before adding a billing-costing
retry band-aid.
Touches wire shape (forwarded
max_tokens) + load-bearing side-request path → human reviewper §7.4 (not auto-merge).