Skip to content

feat: output-headroom fallback to model output when the request carries no budget (#924) - #929

Merged
ranxianglei merged 1 commit into
masterfrom
2026-09-18_output-headroom-model-fallback
Sep 18, 2026
Merged

ranxianglei merged 1 commit into
masterfrom
2026-09-18_output-headroom-model-fallback

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

What

Harnesses that omit every output-budget field (max_tokens / max_completion_tokens / max_output_tokens) previously got no output-headroom reservation at all: maxOutput fell through to 0, reserveOutputHeadroom() short-circuited, and effectiveWindow stayed the full native window. Codex native Responses is exactly this case (it sends no max_output_tokens — openai/codex#36180), so the #896/#898 headroom policy never participated there.

This adds the fallback chain from #924:

  1. request max_tokens / max_completion_tokens / max_output_tokens (unchanged, always wins)
  2. per-route providers[url].models[model].output (resolveConfiguredOutputLimit, new) — operator-declared, outranks auto-fetched data, same order as the window resolution (zcode 用的是gpt-5.6-sol 上下文设置了272000,识别错了上下文空间了 #344)
  3. models.dev registry output ceiling (peekRegistryOutputLimit — cache-only, bundled-snapshot offline floor, never fetches; the same source preflight's summary cap uses, [Question] preflight 摘要为空就判定"上游没返回内容" —— thinking 默认模型的 reasoning_content 是否有意不参与? #853)
  4. neither → 0 (today's behavior preserved)

The fallback value runs through the same capped reservation, so reserved = min(fallbackBudget, outputHeadroomMaxPct × window) — the #898 cap (default 0.25) bounds it and large model outputs can't re-create the #896 over-reservation.

Example (from the issue): window 272000, model output 128000 → reserved = min(128000, 68000) = 68000 → effective 204000 instead of 272000.

Behavior note

Sessions whose harness omits the budget now reserve up to 25% of the window (cost: slightly earlier compression; benefit: long replies can't push input+output past strict-backend limits). One-time-per-model info log marks the fallback: [headroom] model=X: request carries no output budget; reserving against configured|registry max output N (#924).

Changes

  • src/config.ts: resolveConfiguredOutputLimit() (mirror of resolveConfiguredContextLimit)
  • src/server.ts: fallback wiring in the headroom block + imports + one-time log Set
  • tests/routing.test.ts: resolver unit cases (longest-key, undeclared model/route, context-only entry, non-positive output)
  • tests/headroom-output-fallback.test.ts (new): 4 e2e scenarios through a real proxy + kernel session, observed via the OVER-LIMIT nudge on the forwarded payload (same technique as fallback-window-floor.test.ts): configured fallback fires at 80% of the reserved window; registry fallback same; unknown model keeps no-reservation; explicit max_tokens outranks the configured output
  • CONFIGURATION.md / CONFIGURATION.zh-CN.md: outputHeadroomMaxPct budget-source paragraph + models.output field role

Pre-flight (rebased on origin/master @ 0cfde7a)

  • typecheck: clean
  • test: 1747 pass / 0 fail / 2 skipped (e2e-gated)
  • build: ok
  • codex e2e E2E_CHECK=1 preflight: logged (codex-cli 0.147.0 present; local sglang upstream unreachable from this dev sandbox, so the full ACP_TEST_E2E run needs a dispatch/CI with an upstream — the changed path itself is covered by the 4 new e2e scenarios above)

Closes #924

@github-actions

github-actions Bot commented Sep 18, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-09-18_output-headroom-model-fallback (3baaee9)

Option A — Install from npm PR tag (recommended)

npm install -g billion-context@pr-929

Each push to this PR publishes a new version under the pr-929 npm tag.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr929.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 PR #929 review — code verified & mergeable; one flaky CI gate re-running

Code review (diff clean)

Fetched branch, reviewed against current master (0cfde7a = master tip, so not stale, no rebase needed). The diff is tight and every line serves the stated purpose:

One cosmetic edge (not blocking): !(maxOutput > 0) also catches a non-numeric max_tokens (e.g. a string), which would route into the fallback instead of being ignored. Degenerate/malformed input only; left as-is.

Local verification (on PR head 191838f)

  • typecheck: clean
  • test: 1747 pass / 0 fail / 2 skipped (e2e-gated) — matches the pre-flight claim exactly
  • build: ok

CI

8 check-runs on 191838f: all green except test (windows-latest, 24). That failure is not caused by this PR:

  • It fails in tests/ca-combined.test.ts (#152: ensureRootCA writes combined-ca.pem…, #152: combined bundle re-merges…) — MITM CA bundle generation. This PR touches zero lines in src/ca.ts, src/launcher.ts, or that test file.
  • master @ 0cfde7a — the exact base commit of this PR — ran the full matrix green, including test (windows-latest, 24) (run 35294293755).
  • The identical suite is green on ubuntu-22/24 and windows-22 in this very run.
  • Re-ran tests/ca-combined.test.ts locally on the PR head: 5/5 pass.
  • The failing runner's log shows ENOSPC: no space left on device warnings — disk pressure on that specific runner, consistent with a transient/environmental failure.

Action taken: triggered rerun-failed-jobs on run 35304635023 (only the red windows-24 job re-runs). I'll follow up here with the result as soon as it completes — if it goes green, the branch is fully green and ready for you to merge.


中文摘要:代码审查与本地验证全部通过(fallback 链正确、cap 不变、双模式一致、diff 干净、1747 测试全绿);唯一红的 windows-24 是本 PR 完全未触碰的 CA 证书测试偶发失败(同基线 master 该矩阵全绿、本地复跑 5/5 通过、runner 日志有磁盘满告警),已触发该 job 重跑,重跑变绿后即可合并。

…es no budget (#924)

Harnesses that omit every output-budget field (Codex native Responses sends no max_output_tokens) previously got no headroom reservation at all — maxOutput fell through to 0 and effectiveWindow stayed the full native window. Fall back to the model's declared max output through the same capped reservation: per-route ModelEntry.output first (operator-declared, outranks auto-fetched data, same order as the window resolution #344), then the models.dev registry ceiling (cache-only peek, bundled-snapshot floor), else 0 (today's behavior). The #898 cap (outputHeadroomMaxPct, default 0.25) bounds the fallback value, so large model outputs never re-create the #896 over-reservation.

Tests: resolveConfiguredOutputLimit unit cases (routing.test.ts) + 4 e2e scenarios through a real proxy/kernel session (headroom-output-fallback.test.ts): configured fallback nudges at 80% of the reserved window, registry fallback same, unknown model keeps no-reservation, explicit max_tokens outranks the configured output. Preflight on rebased base: typecheck clean, 1747 pass / 0 fail, build ok; codex e2e preflight logged (upstream unreachable from dev sandbox).
@ranxianglei
ranxianglei force-pushed the 2026-09-18_output-headroom-model-fallback branch from 191838f to 3baaee9 Compare September 18, 2026 04:29
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Follow-up: flake confirmed & branch rebased onto live master

1. The red gate was a flake. The rerun-failed-jobs I triggered completed green — test (windows-latest, 24) now passes (all 5 jobs in run 35304635023 success). That confirms the two #152 CA-bundle failures were transient/environmental on that runner (disk-pressure ENOSPC in its log), not a regression from this change.

2. Meanwhile master advanced, so I rebased before calling it mergeable (per review discipline — stale base is the top cause of second-round rework):

3. Re-verified on the rebased head 3baaee9:

  • typecheck: clean
  • test: 1764 pass / 0 fail / 2 skipped (total rose 1749→1766 because the merged master PRs added their own tests — all green)
  • build: ok

Action taken: pushed the rebased head to the PR branch (--force-with-lease, feature branch only). A fresh CI run is now executing on 3baaee9; I'll confirm it's fully green here when it lands. Once that's up, the branch is current with master and ready for you to merge.


中文补充:重跑后 windows-24 变绿,确认那次红是 runner 偶发(磁盘满告警),非本 PR 引入;期间 master 合入了 #923/#804(同改 server.ts 与文档),已先核对无交叠再 rebase 到最新 master(无冲突、diff 不变、version 未动),并在 rebase 后的头提交上重跑 typecheck+全量测试+构建全部通过,已推送更新分支,新 CI 绿后即可合并。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 ✅ Fully green & ready to merge

Fresh CI on the rebased head 3baaee9 (base = live master 3e6f33e) is all green:

Workflow Result
CI (test matrix: ubuntu-22/24, windows-22/24 + version-guard) ✅ all 5 jobs pass
e2e-fake-codex (zero-token preflight) ✅
PR Build Artifact ✅
CI Windows codex overlay (#686) ✅

Note the previously-flaky test (windows-latest, 24) passed again here, consistent with it being a transient runner issue rather than a code problem.

PR status: state: open, mergeable: true, mergeable_state: clean — current with master, no conflicts, all required checks green.

Nothing left on my side. Per repo rules I can't merge PRs myself — please merge when you're ready: #929


中文总结:本 PR 为无 output budget 的请求补上 fallback 预留(请求字段 → 配置 models.output → registry 上限 → 0),沿用既有 cap 逻辑、双压缩模式一致、diff 干净;此前唯一的红是 runner 偶发的 CA 证书测试(重跑即绿、与改动无关),期间 master 前移后已 rebase 到最新并全量复验(typecheck+1764 测试+构建全过),现在所有 CI 全绿、分支与 master 同步、可合并。

@ranxianglei
ranxianglei merged commit 4a1b4e7 into master Sep 18, 2026
8 checks passed
This was referenced Sep 18, 2026
@ranxianglei
ranxianglei deleted the 2026-09-18_output-headroom-model-fallback branch October 15, 2026 05:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Codex Responses 不发送 max_output_tokens 时 output headroom 不生效,是否可 fallback 到模型 output 配置?

1 participant