Skip to content

[PR] fix: do not reserve output headroom twice on a codex live-table window - #6165

Closed
PatelVro wants to merge 3 commits into
ranxianglei:masterfrom
PatelVro:2026-11-10_codex-window-no-double-headroom
Closed

PatelVro wants to merge 3 commits into
ranxianglei:masterfrom
PatelVro:2026-11-10_codex-window-no-double-headroom

Conversation

@PatelVro

Copy link
Copy Markdown
Contributor

Fixes #6164

Problem

On the codex lane the window is clamped to codex's live table value, context_window x effective_context_window_percent (272000 x 95% = 258400). Codex defines that percent as the input share after reserving model output. bili then reserved its default 25% output headroom again (258400 -> 193800), which puts the preflight wall below codex's own auto-compact point (90% of 272000 = 244800). Sessions between those two numbers fail with an unrecoverable 413 ... futile round that codex retries forever. Details and logs in the linked issue.

Fix

Test

New tests/codex-window-no-double-headroom.test.ts drives a real proxy with a codex user agent and a temp CODEX_HOME/models_cache.json (272000, default 95%):

  • before the fix: fails with no second output reservation on a codex-netted window (pre-fix: 258400 -> 193800);
  • after the fix: passes, 3 of 3 local runs; the window stays at 258400 and the skip is logged once;
  • operator outputHeadroomMaxPct: 0.1 still reserves (258400 -> 232560);
  • helper unit test pins which sources count as netted.

Related existing tests (codex window align, live window, #5243 catalog, #2096 headroom, #5191, fork nudge dead zone and others matching codex/headroom): 923 pass, 15 fail, 1 skipped across 44 files; the same 15 (tests/host-usage-postfold.test.ts and one in tests/launcher.test.ts, mostly 9.6 s timeouts) fail identically on unmodified master on this machine, so they are local-environment failures, not this change. npm run typecheck: clean. Local runs on Windows 11 / Node 24.19.0 only; the full-suite verdict is the CI test lanes.

Scope

Two source files plus one test. No config, wire or persistence change. Non-codex traffic is unaffected.

🤖 Generated with Claude Code

The codex clamp already applies effective_context_window_percent, which
codex defines as the input share after reserving model output. Reserving
bili's default 25% on top walled sessions below codex's own auto-compact
point, so long sessions hit an unrecoverable 413 futile round.

Skip the default reservation when the window came from codex-catalog or
codex-models-cache. An explicit operator outputHeadroomMaxPct, a proven
completion share and every raw-window source still reserve.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ranxianglei ranxianglei changed the title fix: do not reserve output headroom twice on a codex live-table window [PR] fix: do not reserve output headroom twice on a codex live-table window Nov 11, 2026
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

查重: 不重复. Searched open+closed issues and PRs (mirror + GitHub full-text for headroom codex, 55 hits reviewed): no other open fix targets this defect. Related lineage, all distinct mechanisms: #924 (fallback to declared max output when the request carries no budget field — feeds the reservation this PR gates), #3900/#4236 (always-reserve policy by route, not wire), #5230 (25% default on local models in general, explicitly called out as related-but-distinct by the reporter), #798/#324 (codex window alignment history). This is the only PR fixing #6164.

标签: bug + codex + scope/kernel + p1 — a long codex session becomes permanently stuck (unrecoverable 413 loop below codex's own auto-compact point); session-unrecoverable class → p1 per the 2026-10-19 ruling.

Review in progress — preliminary state, full verdict to follow in a second comment:

  • Issue mechanism verified against code (v0.1.213 == master at 58a87962, which is exactly this PR's fork point): readCodexLiveModelWindow applies effective_context_window_percent (default 95) only for the two live-table sources; DEFAULT_OUTPUT_HEADROOM_MAX_PCT = 0.25 then reserves min(declared 128000, 0.25 × 258400) = 64600 → wall 193800 < codex auto-compact 244800. The deadlock window [193800, 244800] reproduces the reported futile round 413s exactly. Root cause (double-counting the output budget) rather than symptom — the fix removes the second reservation at the decision point.
  • Diff (3 files, +161/−3, single commit): source-set helper pinned to exactly the two percent-applying sources; skip gated on clamped-from-live-table AND no explicit operator outputHeadroomMaxPct; proven completion share ([Bug][dsh native] preflight fold target sits ABOVE the engine's own denominator (1048576 vs 1000000) and the upstream-stated window from a 400 never feeds it — 'deficit 0' forwards an over-window payload → context_length_exceeded kills the turn #3450) still forces the reservation; raw-window sources and non-codex traffic byte-identical to before (verified case-by-case against shouldReserveOutputHeadroom semantics).
  • Still pending: targeted local test runs (npm install in progress), CI lanes, and two small nits I'm evaluating for direct-fix commits on the branch (a duplicated resolveCompress call and one log line whose "protocol is exempt" wording is now reachable via the new skip path with a different cause).

@ranxianglei ranxianglei added bug Something isn't working codex p1 scope/kernel labels Nov 11, 2026
@ranxianglei

Copy link
Copy Markdown
Owner

ework

ci-check added 2 commits November 11, 2026 22:38
…ew nit for ranxianglei#6165)

The ranxianglei#6165 change re-ran resolveCompress with identical arguments to read
outputHeadroomMaxPct, duplicating the resolution already done at the
headroomCap site whose comment says it is resolved once so the side-request
guard and the main-path reservation measure against the SAME capped window.
Capture the resolved config once and reuse it; behavior unchanged.

Reviewed by agent (ai: vllm-qwen/qwen3.8-27b).
…kips the reservation (review nit for ranxianglei#6165)

The 'although protocol X is exempt' clause is now also reachable via the new
codex-netted skip path, where the protocol is NOT exempt — the default
reservation was skipped because the codex-aligned window already nets out
codex's own output headroom. Make the clause conditional so incident logs
attribute the right cause; old-path wording byte-identical.

Reviewed by agent (ai: vllm-qwen/qwen3.8-27b).
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

Review complete — full verdict below (your ping arrived mid-work; this is the promised second comment).

Verdict: correct root-cause fix, high quality, mergeable after human gate. Two small review nits were fixed directly as labeled commits on the PR branch (pushed, now part of the PR head 03af668f2e).

What I verified independently

  • Mechanism & math: codex live-table window = 272000 × 95% = 258400; pre-fix bili reserved min(declared 128000, 0.25 × 258400 = 64600) → wall 193800, below codex's own auto-compact point 244800 → deadlock zone [193800, 244800] reproduces the reported futile round 413 loop exactly. Only the two live-table sources (codex-catalog, codex-models-cache) apply the percent — I read the whole source-resolution chain to confirm the helper's source set is exact.
  • Case-by-case regression walk: non-codex traffic byte-identical (flag never set); codex raw-window sources (client metadata / config.toml / bundled snapshot / fallback) unchanged; operator-tuned windows keep the reservation (they never enter the clamp branch); explicit operator outputHeadroomMaxPct at any level still reserves (test pins 258400→232560 at 0.1); proven completion share ([Bug][dsh native] preflight fold target sits ABOVE the engine's own denominator (1048576 vs 1000000) and the upstream-stated window from a 400 never feeds it — 'deficit 0' forwards an over-window payload → context_length_exceeded kills the turn #3450) still forces the reservation and outranks everything; outgoing max_tokens clamp vs the native window untouched; applies symmetrically in plugin mode and proxy mode (it's a window value, not a summary carrier).
  • Tests: npm run typecheck clean on the PR head; the new file passes 3/3 locally (real proxy + temp CODEX_HOME/models_cache.json, asserts the clamp log, zero reservation lines, one-time skip line; plus the operator-override direction); headroom cluster around it 25/25. In a wider 15-file codex/headroom run, exactly one failure (issue2929-codex-fold-double-miss) — it fails identically on unmodified master on this machine (token-probe miss), so it's local-environment, not this change.
  • Rebase: PR is behind current master by 5 commits (the compressAtFoldPoint removal family); git merge-tree confirms a conflict-free rebase, and none of those 5 commits touch src/server/handle.ts or src/codex-models.ts.
  • CI (live-checked): 30 checks — all guard lanes green (kernel-guard, version-guard, config-surface-guard, docs-sync, architecture-docs, lock-sync, …), but every test lane is skipped: this is an unapproved external-fork PR, so the heavy suites won't run until you approve it. That approval is the only thing standing between this PR and a real full-suite verdict.

Nits fixed in-place (additive labeled commits on the contributor's branch):

  • 025270203f — the new code re-ran resolveCompress with identical args already resolved once at the headroomCap site (whose comment says "resolved once"); captured once, reused. Behavior unchanged.
  • 03af668f2e — the proven-share log line said "although protocol X is exempt", which is factually wrong on the new codex-skip path (the protocol isn't exempt there; codex netting turned the default off). Clause is now conditional; old-path wording byte-identical.

Behavior change (disclosed, old → new + why): on the codex lane with a live-table-clamped window and no explicit operator cap, the default 25% output reservation is no longer applied — effective judging window 193800 → 258400 (the fix itself: codex's own 95% already nets out its output headroom, reserving again double-counted it and wall'd sessions below codex's own compact point). Side effect to be aware of: an explicit per-request budget field would also be ignored on such windows (real codex sends none — openai/codex#36180 — so no practical impact today). No config/wire/persistence change; non-codex traffic unaffected.

Merge path: §7.4 auto-merge does NOT apply (default-value change = product decision → human gate). Suggested order: approve the external PR (unskips CI) → author rebases onto master (verified clean) → merge after green test lanes.


八问(中文结论)

  1. 是什么问题:codex 车道的窗口已经按 codex 自己的 95%(effective_context_window_percent)扣过了输出预留,bili 又在上面默认再扣 25%,把判定墙压到 codex 自身 auto-compact 点之下,会话落入死区后 413 无限重试、无法恢复。
  2. 什么原因导致的:输出预算被双重计算——codex 的百分比窗口已含输出预留,bili 的默认 25% 预留是第二次计账。修的是决策点(第二次预留本身),不是症状层拦截。
  3. 影响范围与严重程度:codex 车道 + 走实时模型表窗口的长会话;上下文落在两墙之间的区间时完全卡死(p1)。非 codex 流量、raw-window 来源的 codex 流量均不受影响。
  4. 怎么修复的:只对「确实应用了百分比」的两个来源(codex-catalog/codex-models-cache)跳过默认预留;操作者显式 outputHeadroomMaxPct 与已证实完成份额([Bug][dsh native] preflight fold target sits ABOVE the engine's own denominator (1048576 vs 1000000) and the upstream-stated window from a 400 never feeds it — 'deficit 0' forwards an over-window payload → context_length_exceeded kills the turn #3450)仍然生效并优先。
  5. 是否有回归问题:逐项过——非 codex 字节级不变;raw 来源不变;操作者调过窗口的路径不进 clamp 分支、保留原行为;出站 max_tokens 对原生窗口的钳制未动;状态面口径("Output reservation extended to all wires" (#4184) landed without test coverage: widespread red on master is blocking th | 「输出预留扩到所有 wire」(#4184) 落地没带测试面: master 大面积红卡住 v0.1.204 发版 (已 revert, 重做要求见内) #4205)未动;plugin/proxy 两种压缩模式对称适用。未发现回归。
  6. 是否破坏缓存:不涉及缓存/重试/游标/去重键;仅改变每请求判定的有效窗口值。
  7. 是否改了设计方向:没有架构转向;是在既有「按来源决定窗口」机制内补一个「该来源是否已含输出预留」的属性。
  8. 是否引入了新的配置:无新增配置;沿用既有 compress.outputHeadroomMaxPct(跳过日志里也指明了这个逃生门)。

收工判定:代码与测试质量达标,方向正确,两处小瑕疵已就地修掉并推上分支;等 owner 批准外部 PR 触发 CI 全量测试 + 作者 rebase 后即可合并。

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

Status check (re-wake): nothing new on the thread since the full verdict above — live-verified just now. Work is complete; the ball is with you:

  1. Approve the external PR on GitHub (that unskips all test CI lanes — currently every heavy suite is skipped, guard lanes green);
  2. Author then rebases onto master (I verified the rebase is conflict-free) and merges after the test lanes go green.

PR head 03af668f2e already includes the two in-place nit fixes; no further agent action pending unless something changes.

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

Closing summary (full verdict above): review of PR #6165 is done.

  • Verified: mechanism + math against code (deadlock zone [193800, 244800] reproduces the 413 loop); case-by-case regression walk clean; npm run typecheck clean; new test file 3/3 + headroom cluster 25/25 locally; the one wider-run failure (issue2929-codex-fold-double-miss) fails identically on unmodified master → environmental, not this change; rebase onto current master conflict-free (5 commits behind, none touching the two source files).
  • Nits: two small flaws fixed directly as labeled commits on the contributor's branch — 025270203f (duplicate resolveCompress call removed) and 03af668f2e (proven-share log wording now names the real cause on the codex-skip path). PR head now carries both.
  • CI (live): all guard lanes green; every test lane skipped because this is an unapproved external-fork PR.
  • Behavior change disclosed: netted codex live-table windows skip the default 25% output reservation (effective judging window 193800 → 258400 — the fix itself); operator outputHeadroomMaxPct and proven completion share still reserve; non-codex traffic byte-identical.

Remaining (yours): approve the external PR (unskips CI) → author rebases → merge after green test lanes. §7.4 keeps this human-gated (default-value change). Nothing further pending agent-side.

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

Restart notice acknowledged — nothing was actually stuck: the review finished before this and all deliverables are in the thread (full verdict + closing summary above). Live re-check just now: PR #6165 still open, unmerged, head 03af668f2e (carries both nit fixes), behind master as expected. No blockers, no pending agent work — only your approval of the external PR (unskips CI) → author rebase → merge remain.

@ranxianglei

Copy link
Copy Markdown
Owner

Landed via batch PR #6464 (merged) — closing as superseded.

Louie0806 pushed a commit to Louie0806/billion-context that referenced this pull request Nov 13, 2026
…ew nit for ranxianglei#6165)

The ranxianglei#6165 change re-ran resolveCompress with identical arguments to read
outputHeadroomMaxPct, duplicating the resolution already done at the
headroomCap site whose comment says it is resolved once so the side-request
guard and the main-path reservation measure against the SAME capped window.
Capture the resolved config once and reuse it; behavior unchanged.

Reviewed by agent (ai: vllm-qwen/qwen3.8-27b).

(cherry picked from commit 0252702)
Louie0806 pushed a commit to Louie0806/billion-context that referenced this pull request Nov 13, 2026
…kips the reservation (review nit for ranxianglei#6165)

The 'although protocol X is exempt' clause is now also reachable via the new
codex-netted skip path, where the protocol is NOT exempt — the default
reservation was skipped because the codex-aligned window already nets out
codex's own output headroom. Make the clause conditional so incident logs
attribute the right cause; old-path wording byte-identical.

Reviewed by agent (ai: vllm-qwen/qwen3.8-27b).

(cherry picked from commit 03af668)
@github-actions github-actions Bot mentioned this pull request Nov 13, 2026
@ranxianglei ranxianglei reopened this Nov 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working codex p1 scope/kernel

Projects

None yet

2 participants