Skip to content

fix: floor output-budget restore by best-known model output ceiling; warn once per model when none exists (#1840) - #1842

Merged
ranxianglei merged 1 commit into
masterfrom
wt-1840-ework-daemon
Oct 1, 2026
Merged

ranxianglei merged 1 commit into
masterfrom
wt-1840-ework-daemon

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

What & why

#1840: long sessions on DSH (stepfun/step-5-preview) truncated every turn at reason=max-tokens with context far below the window. The client sizes its output budget from UNCOMPRESSED raw history (available = contextWindow − estimate − 4096); as history grows the estimate decays through small positives (restored 113 -> 983, restored 1 -> 583) and the #546 high-water mark ends up holding a death rattle learned from the truncation — every starved tool-carrying turn is restored back to a value that truncates again.

#1665 floored the restore target at the operator-declared route output — but only that source. Routes without a declared output (the DSH case: the client's profile maxTokens never reaches bili's config) stayed pinned at a few hundred tokens silently. This PR completes #1665's design:

  1. Widened floor — new resolveKnownOutputCeiling() resolves the model's best-known output ceiling through the SAME source chain and rank order the output-headroom fallback already uses: [native] runtime-info protocol: plugins report real model config (model / contextWindow / maxOutput) to the proxy #955 client runtime-info (per-request x-bili-plugin-max-output header > agent/conversation-scoped table) > [launcher] BILI_LAUNCHER_MODEL_MAX_OUTPUTS — carry configured max output through the launcher env channel #971 launcher channel > Codex Responses 不发送 max_output_tokens 时 output headroom 不生效,是否可 fallback 到模型 output 配置? #924 operator-declared route value > [Question] preflight 摘要为空就判定"上游没返回内容" —— thinking 默认模型的 reasoning_content 是否有意不参与? #853 models.dev registry (cache-only, bundled snapshot as offline floor). One shared resolution for "this model's output ceiling" keeps the headroom reservation and the restore from ever disagreeing. In the reported scenario the dsh native plugin ALREADY reports defaultMaxTokens via runtime-info + per-request header — v0.1.175's restore path simply never consulted it; now it does. And even without any client report, the bundled snapshot knows stepfun/step-5-preview (output 1M).
  2. No more silent degradation — when NO source knows the model's output ceiling, the proxy logs a one-time-per-model warning naming the model and what was actually restored (or forwarded verbatim in the born-starved case). No built-in constant floor is introduced: a hard default would be a product decision on the owner-gated config surface (explicitly deferred in #546 输出预算高水位可被客户端衰减值污染,饿死会话永久钉在小预算(restore 缺下限) #1665) — the warning makes the gap visible instead of guessing.

⚠ Behavior change (old → new)

  • Starved (≤200) tool-carrying main requests on routes WITHOUT a declared output: restore target was highWater verbatim (e.g. 583); now max(highWater, knownCeiling) where knownCeiling comes from the chain above (e.g. registry listing / client-reported maxOutput). Healthy (>200) requests are NEVER rewritten; a healthy water mark above the ceiling still wins (capability floor, not instruction cap — same semantics #546 输出预算高水位可被客户端衰减值污染,饿死会话永久钉在小预算(restore 缺下限) #1665 established for the declared limit); the plugin mode: no hard backstop when the agent ignores the compress nudge — context grows past the window and upstream 400s #453 clamp downstream still bounds the outgoing budget by real window headroom, so no overflow risk.
  • New warn log line (once per model per process): output-budget restore has NO known output ceiling for model=<model> (…) — … Declare the model's max output in the bili config (providers.<url>.models."<model>".output) or have the client report it to floor the restore (#1840).
  • Info log note text changed for the floored case: …using configured output limit (#1665) → …using known output ceiling (#1665/#1840) (log consumers matching the old string should update).

Config surface

None added. No new env var, CLI flag, or config field — the fix reuses the existing four sources already documented for outputHeadroomMaxPct. The BILI_OUTPUT_BUDGET_FLOOR constant-floor idea from #1665 stays out pending an owner decision.

Tests

Targeted (tests/side-request-isolation.test.ts, all 21 pass):

  • unit: ceiling-less restore warns exactly once per model, names model + restored value, dedupes per model, born-starved says "forwarded verbatim", usable ceiling suppresses the warn;
  • unit: resolver rank order (registry last resort incl. cross-provider suffix scan for unknown hosts, declared > registry, runtime-info table > declared, per-request header > table, stale-entry model gate, unannounced-header inertness);
  • e2e through the real proxy rig with NO route declaration: poisoned water mark seeded at 234 → starved request reaches upstream at the registry ceiling 8192; born-starved session also floored; client-reported per-request header (16384) outranks the registry listing.

Preflight: npm run typecheck ✓, npm run build ✓, targeted test file ✓ (full suite left to CI).

Fixes #1840

@github-actions

github-actions Bot commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: wt-1840-ework-daemon (5f944d5)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.1826-pr.1842.c2d291a

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr1842.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 收到,开始 review PR #1842(floor output-budget restore by known ceiling)。步骤:拉分支核对代码 → 回归审查(既有行为清点)→ 定向验证(typecheck + 指定测试文件,全量交给 CI)→ 按结构化七问给结论。稍后回帖。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Review 结论(PR #1842)— 可合并(merge 由人执行)

按结构化七问逐项作答:

1. 是什么问题
DSH(stepfun/step-5-preview)长会话每轮在 reason=max-tokens 截断、而 context 远低于窗口。#1665 已把 #546 恢复目标钉在"运营者声明的 route output"上,但只认这一个来源——没声明 output 的路线(本例)仍被钉死在几百 token,且完全静默。

2. 什么原因导致的
因果链分三层:客户端按未压缩原始历史估输出预算(available = window − estimate − 4096)→ 随历史增长 estimate 衰减、预算穿过小正值(…, 680, 234)后在 ≤200 饥饿 → #546 的 high-water 记住的是衰减末端的"死亡余音"(234/583),每次恢复都恢复到会再次截断的值。#1665 治了"无声明时恢复目标过低"但来源面太窄,声明缺失的路线依旧静默退化。

3. 怎么修复的
机制层:新增 resolveKnownOutputCeiling()(src/server.ts:1091),用与 output-headroom 回退(#924/#955/#971/#853)完全相同的来源链和排序(per-request runtime-info header > agent/conversation 表 > launcher 通道 > 声明值 > models.dev registry 缓存兜底)解析"该模型已知最大输出",作为 #546 恢复的下限;一个共享解析保证预留与恢复永不打架。无任何来源知道上限时,按模型打一次性警告(Set 去重 + 4096 cap,与 plugin.ts warnedNoModelRequests 同款约定),把静默缺口变成可见信号。不引入硬编码默认下限——正确留给了 owner-gated config surface。修法打在"坏恢复目标的产生机制"(来源链)上,不是症状层。

4. 是否回归问题 — 审过的既有行为及结论:

5. 是否破坏缓存 — 不涉及。不改 session state 格式、message ref、前缀字节;water-mark metadata key 不变;预算改写是 #546 既有行为,本 PR 只改改写目标值。

6. 是否改了设计方向 — 不涉及新方向。这是完成 #1665 既定设计("floor 恢复目标"),把单一来源扩成既有四源链;架构/边界/数据流未转向。

7. 是否引入新配置 — 不涉及。无新 env/CLI/config 字段;CONFIGURATION.md en+zh 同步更新的是既有 output 字段的说明,CHANGELOG 条目齐备。BILI_OUTPUT_BUDGET_FLOOR 常量方案明确留在 owner 决策外。


验证证据(本地,基于 rebase 到当前 master 97e7420 后的树):

  • rebase 干净无冲突(分支原落后 15 个提交,GitHub 显示 mergeable=True,未推送任何东西,分支未动);
  • npm run typecheck ✓、npm run build ✓;
  • 定向测试 tests/side-request-isolation.test.ts 21/21 通过;全量套件交给 CI;
  • 原 head sha 的 CI:12/13 绿(e2e、hermetic registry、各真实 lane 全过),剩 windows node-22 test 一个 job 在跑。

测试质量:有牙。warn 单测钉住 once-per-model/去重/born-starved 措辞/ceiling 抑制警告;resolver 单测钉住四级排序 + stale-entry 门控 + 未宣告 header 惰性;e2e 走真实代理 rig、无 route 声明,断言的是线上 wire 真值(rig.lastBody.max_tokens:毒水位 234 → registry 8192、born-starved 同钉、client header 16384 压过 registry)——bug 回归时抓得住。

备注:auto-merge gate 层面此 PR 属"fallback 语义变更",本就留给人工合并;代码与行为审查均通过,建议直接合。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[bug] 长会话频繁「达到输出token上限」:restore 回落到被污染的 high-water,0.1.175 未声明 output 时仍能复现

1 participant