Repository navigation
feat: output-headroom fallback to model output when the request carries no budget (#924) - #929
Conversation
📦 Built Package ArtifactBranch: Option A — Install from npm PR tag (recommended)npm install -g billion-context@pr-929Each push to this PR publishes a new version under the Option B — Download artifact
tar xzf billion-context-pr929.tgz
npm install -g packageInstall instructions are refreshed on each push. |
[bot] 🏷 PR #929 review — code verified & mergeable; one flaky CI gate re-running Code review (diff clean)Fetched branch, reviewed against current master (
One cosmetic edge (not blocking): Local verification (on PR head
|
…es no budget (#924) Harnesses that omit every output-budget field (Codex native Responses sends no max_output_tokens) previously got no headroom reservation at all — maxOutput fell through to 0 and effectiveWindow stayed the full native window. Fall back to the model's declared max output through the same capped reservation: per-route ModelEntry.output first (operator-declared, outranks auto-fetched data, same order as the window resolution #344), then the models.dev registry ceiling (cache-only peek, bundled-snapshot floor), else 0 (today's behavior). The #898 cap (outputHeadroomMaxPct, default 0.25) bounds the fallback value, so large model outputs never re-create the #896 over-reservation. Tests: resolveConfiguredOutputLimit unit cases (routing.test.ts) + 4 e2e scenarios through a real proxy/kernel session (headroom-output-fallback.test.ts): configured fallback nudges at 80% of the reserved window, registry fallback same, unknown model keeps no-reservation, explicit max_tokens outranks the configured output. Preflight on rebased base: typecheck clean, 1747 pass / 0 fail, build ok; codex e2e preflight logged (upstream unreachable from dev sandbox).
191838f to
3baaee9
Compare
[bot] 🏷 Follow-up: flake confirmed & branch rebased onto live master 1. The red gate was a flake. The 2. Meanwhile master advanced, so I rebased before calling it mergeable (per review discipline — stale base is the top cause of second-round rework):
3. Re-verified on the rebased head
Action taken: pushed the rebased head to the PR branch ( 中文补充:重跑后 windows-24 变绿,确认那次红是 runner 偶发(磁盘满告警),非本 PR 引入;期间 master 合入了 #923/#804(同改 server.ts 与文档),已先核对无交叠再 rebase 到最新 master(无冲突、diff 不变、version 未动),并在 rebase 后的头提交上重跑 typecheck+全量测试+构建全部通过,已推送更新分支,新 CI 绿后即可合并。 |
[bot] 🏷 ✅ Fully green & ready to merge Fresh CI on the rebased head
Note the previously-flaky PR status: Nothing left on my side. Per repo rules I can't merge PRs myself — please merge when you're ready: #929 中文总结:本 PR 为无 output budget 的请求补上 fallback 预留(请求字段 → 配置 |
What
Harnesses that omit every output-budget field (
max_tokens/max_completion_tokens/max_output_tokens) previously got no output-headroom reservation at all:maxOutputfell through to0,reserveOutputHeadroom()short-circuited, andeffectiveWindowstayed the full native window. Codex native Responses is exactly this case (it sends nomax_output_tokens— openai/codex#36180), so the #896/#898 headroom policy never participated there.This adds the fallback chain from #924:
max_tokens/max_completion_tokens/max_output_tokens(unchanged, always wins)providers[url].models[model].output(resolveConfiguredOutputLimit, new) — operator-declared, outranks auto-fetched data, same order as the window resolution (zcode 用的是gpt-5.6-sol 上下文设置了272000,识别错了上下文空间了 #344)peekRegistryOutputLimit— cache-only, bundled-snapshot offline floor, never fetches; the same source preflight's summary cap uses, [Question] preflight 摘要为空就判定"上游没返回内容" —— thinking 默认模型的 reasoning_content 是否有意不参与? #853)0(today's behavior preserved)The fallback value runs through the same capped reservation, so
reserved = min(fallbackBudget, outputHeadroomMaxPct × window)— the #898 cap (default 0.25) bounds it and large model outputs can't re-create the #896 over-reservation.Example (from the issue): window 272000, model output 128000 → reserved = min(128000, 68000) = 68000 → effective 204000 instead of 272000.
Behavior note
Sessions whose harness omits the budget now reserve up to 25% of the window (cost: slightly earlier compression; benefit: long replies can't push input+output past strict-backend limits). One-time-per-model info log marks the fallback:
[headroom] model=X: request carries no output budget; reserving against configured|registry max output N (#924).Changes
src/config.ts:resolveConfiguredOutputLimit()(mirror ofresolveConfiguredContextLimit)src/server.ts: fallback wiring in the headroom block + imports + one-time log Settests/routing.test.ts: resolver unit cases (longest-key, undeclared model/route, context-only entry, non-positive output)tests/headroom-output-fallback.test.ts(new): 4 e2e scenarios through a real proxy + kernel session, observed via the OVER-LIMIT nudge on the forwarded payload (same technique asfallback-window-floor.test.ts): configured fallback fires at 80% of the reserved window; registry fallback same; unknown model keeps no-reservation; explicitmax_tokensoutranks the configured outputCONFIGURATION.md/CONFIGURATION.zh-CN.md:outputHeadroomMaxPctbudget-source paragraph +models.outputfield rolePre-flight (rebased on origin/master @ 0cfde7a)
E2E_CHECK=1preflight: logged (codex-cli 0.147.0 present; local sglang upstream unreachable from this dev sandbox, so the full ACP_TEST_E2E run needs a dispatch/CI with an upstream — the changed path itself is covered by the 4 new e2e scenarios above)Closes #924