Repository navigation
fix: floor output-budget restore by best-known model output ceiling; warn once per model when none exists (#1840) - #1842
Conversation
…warn once per model when none exists (#1840)
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.1826-pr.1842.c2d291aThis exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr1842.tgz
npm install -g packageInstall instructions are refreshed on each push. |
[bot] 🏷 Review 结论(PR #1842)— 可合并(merge 由人执行) 按结构化七问逐项作答: 1. 是什么问题 2. 什么原因导致的 3. 怎么修复的 4. 是否回归问题 — 审过的既有行为及结论:
5. 是否破坏缓存 — 不涉及。不改 session state 格式、message ref、前缀字节;water-mark metadata key 不变;预算改写是 #546 既有行为,本 PR 只改改写目标值。 6. 是否改了设计方向 — 不涉及新方向。这是完成 #1665 既定设计("floor 恢复目标"),把单一来源扩成既有四源链;架构/边界/数据流未转向。 7. 是否引入新配置 — 不涉及。无新 env/CLI/config 字段;CONFIGURATION.md en+zh 同步更新的是既有 验证证据(本地,基于 rebase 到当前 master 97e7420 后的树):
测试质量:有牙。warn 单测钉住 once-per-model/去重/born-starved 措辞/ceiling 抑制警告;resolver 单测钉住四级排序 + stale-entry 门控 + 未宣告 header 惰性;e2e 走真实代理 rig、无 route 声明,断言的是线上 wire 真值( 备注:auto-merge gate 层面此 PR 属"fallback 语义变更",本就留给人工合并;代码与行为审查均通过,建议直接合。 |
What & why
#1840: long sessions on DSH (stepfun/step-5-preview) truncated every turn at
reason=max-tokenswith context far below the window. The client sizes its output budget from UNCOMPRESSED raw history (available = contextWindow − estimate − 4096); as history grows the estimate decays through small positives (restored 113 -> 983,restored 1 -> 583) and the #546 high-water mark ends up holding a death rattle learned from the truncation — every starved tool-carrying turn is restored back to a value that truncates again.#1665 floored the restore target at the operator-declared route
output— but only that source. Routes without a declaredoutput(the DSH case: the client's profilemaxTokensnever reaches bili's config) stayed pinned at a few hundred tokens silently. This PR completes #1665's design:resolveKnownOutputCeiling()resolves the model's best-known output ceiling through the SAME source chain and rank order the output-headroom fallback already uses: [native] runtime-info protocol: plugins report real model config (model / contextWindow / maxOutput) to the proxy #955 client runtime-info (per-requestx-bili-plugin-max-outputheader > agent/conversation-scoped table) > [launcher] BILI_LAUNCHER_MODEL_MAX_OUTPUTS — carry configured max output through the launcher env channel #971 launcher channel > Codex Responses 不发送 max_output_tokens 时 output headroom 不生效,是否可 fallback 到模型 output 配置? #924 operator-declared route value > [Question] preflight 摘要为空就判定"上游没返回内容" —— thinking 默认模型的 reasoning_content 是否有意不参与? #853 models.dev registry (cache-only, bundled snapshot as offline floor). One shared resolution for "this model's output ceiling" keeps the headroom reservation and the restore from ever disagreeing. In the reported scenario the dsh native plugin ALREADY reportsdefaultMaxTokensvia runtime-info + per-request header — v0.1.175's restore path simply never consulted it; now it does. And even without any client report, the bundled snapshot knowsstepfun/step-5-preview(output 1M).⚠ Behavior change (old → new)
output: restore target washighWaterverbatim (e.g. 583); nowmax(highWater, knownCeiling)where knownCeiling comes from the chain above (e.g. registry listing / client-reported maxOutput). Healthy (>200) requests are NEVER rewritten; a healthy water mark above the ceiling still wins (capability floor, not instruction cap — same semantics #546 输出预算高水位可被客户端衰减值污染,饿死会话永久钉在小预算(restore 缺下限) #1665 established for the declared limit); the plugin mode: no hard backstop when the agent ignores the compress nudge — context grows past the window and upstream 400s #453 clamp downstream still bounds the outgoing budget by real window headroom, so no overflow risk.output-budget restore has NO known output ceiling for model=<model> (…) — … Declare the model's max output in the bili config (providers.<url>.models."<model>".output) or have the client report it to floor the restore (#1840).…using configured output limit (#1665)→…using known output ceiling (#1665/#1840)(log consumers matching the old string should update).Config surface
None added. No new env var, CLI flag, or config field — the fix reuses the existing four sources already documented for
outputHeadroomMaxPct. TheBILI_OUTPUT_BUDGET_FLOORconstant-floor idea from #1665 stays out pending an owner decision.Tests
Targeted (
tests/side-request-isolation.test.ts, all 21 pass):Preflight:
npm run typecheck✓,npm run build✓, targeted test file ✓ (full suite left to CI).Fixes #1840