Skip to content

[PR] fix: reserve output headroom by route, not wire — caliber split (redo of reverted #4184) - #4236

Merged
ranxianglei merged 4 commits into
masterfrom
2026-10-22_output-reservation-caliber
Oct 23, 2026
Merged

ranxianglei merged 4 commits into
masterfrom
2026-10-22_output-reservation-caliber

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

(ai: vllm-qwen/qwen3.8-27b)

Closes #4205
Refs: #3900 (original bug), #4184 (first attempt, reverted), #4198 (revert), #3973 (companion PR), #3837 (caliber discussion)

What this PR does

Re-lands output-headroom reservation for all wires — reverted by #4198 after #4184 landed red — and fixes both defects #4205 calls out: the test fallout and the global-boolean exemption.

1. Exemption decided per-route, not per-wire

shouldReserveOutputHeadroom(protocol) becomes shouldReserveOutputHeadroom(protocol, origin, modelId). Reservation happens by default everywhere; the anthropic wire is exempt only with positive separate-output-budget evidence:

  • upstream hostname is exactly api.anthropic.com, or
  • the model id starts with claude- (covers Bedrock/Vertex/other fronts).

Wrapper routes that bill completion INTO the window (the #3900 shape: DeepSeek /anthropic serving DeepSeek-V3.2-Exp) have no such evidence and reserve like every other wire. This replaces the global boolean #4184 introduced (which shifted every real-Claude band down by maxOutput for no safety gain — #3900 thread analysis) while still fixing the wrapper class. #3973 composes above this: once a session has a PROVEN completion share from an overflow, its override can exempt a real-Claude route this heuristic cannot see.

2. Caliber split: display N, decide R

metadata.effectiveContextLimit again records the UNRESERVED window N (post self-heal) — what the status panel / acp_status / admin / web surfaces show. A new per-request field metadata.reservedContextLimit records R = the window the kernel actually judges against (post headroom), and every decision consumer reads it (relay judging limit / #3738 arm, no-window overflow arm, side-lane guard call site). reqConfig.modelContextLimit still carries R into the kernel/preflight paths — no kernel change. Fork inheritance copies both fields. The [headroom] log keeps printing both numbers (effective window N -> R (reserved …)), matching the #3837 caliber discussion.

Behavior changes (old → new)

Surface Old (post-revert master) New Why
Wrapper /anthropic routes (non-claude model, or non-anthropic host) no reservation (wire-level exemption) reserve min(max_tokens, cap·window) THE #3900 fix: hosts billing completion against the wall otherwise 400-loop at the wall
Real Claude routes (api.anthropic.com or claude-* model) no reservation no reservation byte-identical to today
effectiveContextLimit on non-anthropic wires (panel/acp_status/web) R (reserved) N (unreserved) owner caliber, #4205 req 1 / #3837: status shows the declared window; decision fields carry R
Egress fuse bar (relay.ts) sized vs R sized vs N the bar guards the physical wall; payloads between R and N now ride forward→400→refold rescue instead of instant 413 (slightly looser than pre-#4205)
sideRequestGuard reserved internally for non-anthropic protocols call site passes a reserveHeadroom flag; guard stays pure mechanics one policy decision point; existing unit rows keep their behavior via default false
#453 outgoing max_tokens clamp structurally absent from the anthropic prepare path unchanged boundary documented in src/server/budget.ts: wrapper-anthropic routes get band-layer reservation but no egress clamp; #4093 self-heal + #3973 learning cover the residual one-first-shot 400

No kernel/ changes. Config surface: nothing added — no new config key, env var, or CLI flag; the reservation cap stays compress.outputHeadroomMaxPct (default 0.25).

Tests

The whole family that went red under #4184 — codex-window-align, bridge, sideRequestGuard, the registry/window suites, the #3900 e2e pair — is green WITHOUT modification; 188 targeted tests green locally including the side-lane e2e set. Full-suite verdict: CI.

CI

Green before merge — requirement 4 of #4205. #4184 landing red was a process incident; this PR does not merge on red checks.

Note for #3973

#3973 (proven completion-wall persistence) composes above this PR: its override sits at the same call sites and can exempt a real-Claude route this heuristic cannot see. Its test currently expects effectiveContextLimit === R; after rebasing onto this PR that assertion moves to reservedContextLimit === R (one-line tweak). Suggested order: merge this first, rebase #3973 second.

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-22_output-reservation-caliber (cc77646)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.5034-pr.4236.05d187a

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr4236.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重: 不重复。 本 PR 是 #4184(已被 #4198 回滚)的重做,针对 #4205 的在途修复仅此一条,无并行工作。

标签: bug scope/wiring p2(保留原 pr)——修的是 wrapper 路由在窗口墙处 400 循环的缺陷(#3900 类);改动面在请求管道/预算层(scope/wiring);属特定条件下有明显问题、有绕行(#4093 自愈),定 p2。

审查进展(进行中):

  1. 代码审查已完成:src diff 逐段读完,shouldReserveOutputHeadroom / sideRequestGuard / effectiveContextLimit 全部调用点与消费方逐一核对——两个 src 调用点都传了 route+model 证据,无遗漏的决策消费方仍读旧口径;display 面(plugin status / admin / web sessions-data)读 N 符合 "Output reservation extended to all wires" (#4184) landed without test coverage: widespread red on master is blocking th | 「输出预留扩到所有 wire」(#4184) 落地没带测试面: master 大面积红卡住 v0.1.204 发版 (已 revert, 重做要求见内) #4205 定的口径;fork 继承同时复制两个字段;normalizeUpstreamOrigin 对 mitm:// 与不可解析输入的处理正确。定向本地测试全绿(context-overflow / side-request-isolation / fix-arm-headroom-clamp / issue4205 新 e2e / fork-nudge-deadzone / golden-storage-render,共 81 项;另补跑了所有断言 effectiveContextLimit 的其余套件 30 项)。
  2. CI 发现一个红并已修:旧 head(adfbf5c)上 test (ubuntu-latest, 22, 3) 失败——tests/runtime-info.test.ts 的三个 [Bug][OMP native][0.1.163] contextWindow 上报后仍使用 registry 窗口,runtime-info 存在跨会话覆盖 #1531 子测仍在断言 plugin status 的 contextLimit 为保留后的值(229376 / 983040 / 90000),而口径拆分后该展示面按设计报未保留值 N(262144 / 1048576 / 100000,保留带移到 reservedContextLimit)。这是 PR 测试更新漏改的一处断言(本地 node --import tsx --test tests/runtime-info.test.ts 复现同一失败)。已按新口径更新三处断言并保留各子测的判据能力(agent-slot vs conv entry 区分不变),独立标注 commit 6ca565bab 已推到 PR 分支。
  3. 待办:等 6ca565b 的全量 CI 绿后给最终八问结论。PR 描述里「CI green before merge」这条要求目前由我的修复补齐。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Review 完成,CI 全绿(head cc77646:34 success / 16 skipped-gated / 0 fail)。可合并。

结构化八问:

  1. 是什么问题:[PR] fix: reserve output headroom by route, not wire — caliber split (redo of reverted #4184) #4236 是已回滚的 [PR] Reserve output headroom on every wire (drop the Anthropic exemption) #4184 的重做,修 [BUG] acp-proxy upstream HTTP 400: Request with 1,198,269 tokens exceeds the model's 1,048,576 token limit, consistently | [BUG] acp-proxy upstream HTTP 400:请求 1,198,269 tokens 超过模型上限 1,048,576,一直复现——是插件压缩未生效,还是 DSH 的问题? #3900——上游 wrapper 路由(如 DeepSeek 的 /anthropic 网关跑 DeepSeek-V3.2-Exp)把 completion tokens 计入窗口计费,会话顶到墙后陷入 400 循环。本 PR 让输出预留改为按路由判定:默认所有 wire 都预留,anthropic wire 仅在存在独立输出预算的正证据(上游 host 恰为 api.anthropic.com,或 modelId 以 claude- 开头)时豁免。同时做口径拆分:effectiveContextLimit=N(未预留窗口,仅展示面)vs 新增 reservedContextLimit=R(预留后,全部决策消费者读它)。
  2. 什么原因导致的:原豁免判定按 wire 协议(protocol === "anthropic"),无法区分「真 Anthropic API」与「第三方网关讲 anthropic 协议」;协议级豁免放过了 wrapper 路由。根修 = 判定从「哪种协议」移到「这条路由有没有独立输出预算」,用 host+model 前缀两个正证据条件。
  3. 影响范围与严重程度:修复前 wrapper /anthropic 路由无预留 → 载荷顶窗 → 上游 400 → 重试循环([BUG] acp-proxy upstream HTTP 400: Request with 1,198,269 tokens exceeds the model's 1,048,576 token limit, consistently | [BUG] acp-proxy upstream HTTP 400:请求 1,198,269 tokens 超过模型上限 1,048,576,一直复现——是插件压缩未生效,还是 DSH 的问题? #3900,体验毁灭级);真实 Claude 路由前后字节一致不受影响;展示面(panel/acp_status/web)由显示 R 变为 N(owner 口径要求,非回归);egress fuse bar 由 R 口径变 N 口径、比 "Output reservation extended to all wires" (#4184) landed without test coverage: widespread red on master is blocking th | 「输出预留扩到所有 wire」(#4184) 落地没带测试面: master 大面积红卡住 v0.1.204 发版 (已 revert, 重做要求见内) #4205 前略松(R..N 的载荷走 forward→400→refold 救援而非立即 413)——有意取舍,见下方披露。
  4. 怎么修复的:证据函数 hasSeparateOutputBudgetEvidence(origin, modelId)(host 精确匹配 + model 前缀匹配;URL 不可解析按无证据处理 → 预留);两个决策点(sideRequestGuard flag @src/server/handle.ts:1998、预留门 @:2085)共用同一证据源;metadata 双字段每请求同写、恒同步;relay/side-request 决策点统一读 reservedContextLimit(对旧持久化会话有 legacy fallback);fork 继承同时拷贝两字段。无 kernel 改动、无配置面新增。
  5. 是否有回归问题:对触碰路径做了既有行为清点——src 内全部 effectiveContextLimit 读取点逐一分类:plugin status endpoint(src/plugin.ts:1611)、admin.ts:946、web/sessions-data.ts:752 均为纯展示(按口径应显 N);relay fuse bar/fallback/warning 行为变更属有意设计;无漏网的决策侧消费者。shouldReserveOutputHeadroom 在 src 恰有两个调用点、均已更新;sideRequestGuard 在 src 恰有一个调用方。storage-render golden diff 确认仅 metadata 段变化(state/blockContents/stats digest 未动)。高危模式逐项过:无语义漂移(wrapper 路由行为变化即修复目标本身)、无 fail-fast 变吞错、除披露项外无阈值/时序漂移。另发现并处置两处 CI 红(PR 描述声称的 "green without modification" 恰好在这两个文件上不成立):① tests/runtime-info.test.ts [Bug][OMP native][0.1.163] contextWindow 上报后仍使用 registry 窗口,runtime-info 存在跨会话覆盖 #1531 三个子测试把旧口径 R 钉死在 status-surface 断言上 → 本 PR 引起,我已在分支上直接修掉(commit 6ca565b:期望值 R→N,本地 29/29 绿);② tests/external-summary-autofold.test.ts 三个用例红 → master 既有红(pristine fe5ba59 干净环境本地复现一致;机制:fixture 裸路径直连代理 → embeddedUrl=undefined → routes 表 miss → 窗口判为 fallback-derived → bili出现连续压缩情况 #282 下限把 <100K 窗口抬到 100K → 对照组 usage 从 84% 稀释到 42%,跌出压力带)→ 分支上已有更优修法(cc77646b2:fixture 改走 /bili/ 前缀命中 routes 表,source=configured 不触发下限),我核对正确性后采纳;我另开发了一个平行修法(bill 参数化,本地该套件 8/8 + 相关 12 套 122/122 全绿)但判其劣于且晚于前者,放弃,保留在本地分支 backup-autofold-billparam @ a238133d6,未推送。
  6. 是否破坏缓存:否。头部 system 消息字节稳定,prefix cache 不受影响;golden diff 证实 wire body 逐字节未变(仅会话 metadata 段变化)。
  7. 是否改了设计方向:否。同一架构、判定粒度细化(wire → route)+ 两种口径显式化。plugin mode: no hard backstop when the agent ignores the compress nudge — context grows past the window and upstream 400s #453 egress clamp 的残留边界(anthropic prepare 路径结构性没有出站 max_tokens clamp)在 src/server/budget.ts 有文档化说明,本 PR 未改动。
  8. 是否引入新配置:否。cap 字段沿用 compress.outputHeadroomMaxPct(默认 0.25)。

行为变更披露(旧→新):

面 旧 新 原因
wrapper /anthropic 路由(非 api.anthropic.com host、非 claude-* model) 不预留 预留 min(max_tokens, cap·window) THE #3900 修复
真实 Claude 路由 不预留 不预留 字节一致
panel/acp_status/web 的 effectiveContextLimit(非 anthropic wire) R N owner 口径(#4205/#3837)
egress fuse bar R 口径 N 口径(略松) 有意:R..N 载荷走 forward→400→refold 救援,避免误 413
sideRequestGuard 内部自行预留 调用点传 reserveHeadroom flag(默认 false) 策略单点化

新发现已立项:#4239 —— #282 回退下限在输出预留之后生效,会把任何 <100K 的回退基线抬到高于声明值(日志可见 reserved -91808 负预留),nudge/growth-fold 节奏静默停摆直到自愈;该缺陷独立于本 PR(master 今天就有)、影响真实用户,属 owner 决策项(§7.4 fallback/默认值变更),已附两个候选修法待拍板。autofold fixture 对齐(cc77646b2)与该产品决策相互独立。

合并判定:代码审查通过(调用点审计完整、双压缩模式均已推演、测试真值表双向覆盖符合 #3617),CI 在 cc77646 全绿,建议合并(合并操作归你)。提醒:#3973 的测试断言在本 PR 合并后需一行 rebase 调整(effectiveContextLimit===R → reservedContextLimit===R),合入顺序建议本 PR 在前。

awork added 4 commits October 23, 2026 13:07
The caliber split moves metadata.effectiveContextLimit to the UNRESERVED window N on every display surface; the reserved band now lives in metadata.reservedContextLimit. The three #1531 subtests in tests/runtime-info.test.ts still pinned the reserved values (229376 / 983040 / 90000) on the plugin status contextLimit field, red-lining CI shard test (ubuntu-latest, 22, 3) with '262144 !== 229376'. Update the expectations to N (262144 / 1048576 / 100000); the entry-wins discriminating power of each subtest is kept (agent-slot 100000 vs conv 262144).
…ix (#4205)

Bare-path /v1/messages requests missed the routes table, fell back to the
option-level window, and read as fallback-derived — where the #282 floor
lifts any reserved sub-100K window to 100K. Once anthropic wrapper routes
reserve output headroom (#4205), that dilution pushed the control-arm usage
from ~84% down to 42%, out of the >=75% pressure band these tests exercise.
Routing through the zero-config prefix makes the declared routes-table
window authoritative (source=configured, no floor).

Complements 6ca565b (runtime-info status-surface alignment).
@ranxianglei
ranxianglei force-pushed the 2026-10-22_output-reservation-caliber branch from cc77646 to 1750866 Compare October 23, 2026 05:21
@ranxianglei

Copy link
Copy Markdown
Owner Author

(ai: vllm-qwen/qwen3.8-27b)

已 rebase 到 master@07888d74 并强推到本分支(cc77646 → 1750866),4 个 commit 全部保留。

冲突解法(逐文件)

src/ 全部无冲突,PR 语义原样保留:

  • src/util.ts(shouldReserveOutputHeadroom(protocol, origin, modelId) 按路由豁免)、src/server/handle.ts、src/server/relay.ts、src/server/side-request.ts、src/server/budget.ts、src/server.ts、src/plugin.ts — 7 个源文件在 commit 1(fix: reserve output headroom by route, not wire)干净落上 master,未动一字。

tests/ 4 个 golden 文件冲突(唯一冲突点,commit 2):

已丢弃/无对应丢弃的 hunk

无丢弃。全部冲突只有 golden digest 一类,按上述重新生成解决;没有发现「PR hunk 已在 master 等价存在」需要丢弃的情况(PR diff 与 master 的最终差异 = 原 PR diff 内容,仅行号漂移 + golden digest 值变化)。

本地验证

  • npm run typecheck 干净(kernel/pi-subagents 预构建通过)。
  • PR 触及的 8 个测试文件全绿,共 118 pass / 0 fail / 0 skip:
    • context-overflow 34✓ · external-summary-autofold 8✓ · fix-arm-headroom-clamp 3✓ · fork-nudge-deadzone 3✓ · issue4205-output-headroom-caliber 3✓(新回归钉)· runtime-info 29✓ · side-request-isolation 33✓ · golden-storage-render 5✓(重生 golden 自洽)
  • 相邻受影响面(原 [PR] Reserve output headroom on every wire (drop the Anthropic exemption) #4184 被打红的那批)抽查全绿:codex-window-align 8✓、claude-bridge-e2e 3✓、issue3900-overflow-teaches-calibration 3✓、e2e-issue3900-overflow-teaches 2✓。

全量判定看 CI test lanes。

@github-actions

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.767-master.924b852 (rolling master tag).

npm install -g billion-context@0.0.767-master.924b852  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

rebase 收尾独立复核完成——冲突已解、PR 已合入 master 并发布,任务闭环。以下结论是本轮对远端逐项重新验证的结果(不是转述上面的 rebase 报告):

独立核对(全部通过):

  1. 分支头无误:远端 PR 分支头 = 17508661,正是强推目标;其上恰好 4 个 commit 落在 rebase 基点 master@07888d74 上(720a6e331 fix 主体 + e2aa53da6/b370b965a/175086612 三个测试 commit),与报告一致。
  2. 无丢失 hunk:merge commit 924b85233 的第二父节点就是 17508661;merge 相对 master 的 diff 恰好 = 原 PR 范围(7 个 src + 10 个测试文件 + 4 个 golden,384+/98-),没有多余改动、没有「PR hunk 已在 master 等价存在」需要丢弃的情况。
  3. golden 重生正确:4 个 storage-render golden 相对 merge 前 master 每个只动 stateDigest + sectionDigests.metadata 两行;blockContents/stats 等 wire-body digest 全部未变——与「compress calls rejected when the summary text rides in content next to top-level startId/endId — plus U+2212 dash silently folds the wrong range #4252 schema 文案 + 本 PR headroom 元数据的并集」说法吻合,出站消息体逐字节未变。
  4. 功能存活:当前 master(50e250685)上 shouldReserveOutputHeadroom(protocol, origin, modelId)(src/util.ts:608)与 reservedContextLimit(src/server/handle.ts:2205)均在;合入后的后续提交(CI 修复、nudge 探针等)无回滚、无触碰该功能。

结构化八问:

  1. 是什么问题:上游 merge_conflict 转发——本 PR 分支与 master 冲突,需 rebase 后恢复可合并状态。
  2. 什么原因导致的:master 侧 compress calls rejected when the summary text rides in content next to top-level startId/endId — plus U+2212 dash silently folds the wrong range #4252(compress schema 文案)与本 PR("Output reservation extended to all wires" (#4184) landed without test coverage: widespread red on master is blocking th | 「输出预留扩到所有 wire」(#4184) 落地没带测试面: master 大面积红卡住 v0.1.204 发版 (已 revert, 重做要求见内) #4205 口径拆分)各自重新生成过同一批 storage-render golden,digest 叠不上 → 4 个 golden 文件冲突;src/ 七个文件零冲突。
  3. 影响范围与严重程度:仅阻塞本 PR 合入,不影响任何已发布行为。
  4. 怎么修复的:golden 不取任何一方,在 rebase 中途用仓库自带的 golden 更新脚本重新生成(两侧变更的自然并集);src 语义零改动落上 master。
  5. 是否有回归问题:无。merge diff 逐文件对照原 PR 范围无意外行;真实 Claude 路由行为不变(字节一致),wrapper 路由开始预留即本 PR 的修复目标本身;除 floor 4 已披露的口径变化外无新增漂移。
  6. 是否破坏缓存:否——golden 证实出站 wire body 逐字节未变,仅会话 metadata 段 digest 变化,prefix cache 不受影响。
  7. 是否改了设计方向:否,rebase 不改语义,只消冲突。
  8. 是否引入新配置:否。

CI 实况(如实记录):全量绿的基线在 rebase 前的 head cc77646b(34 success / 16 gated-skip / 0 fail,floor 4);rebase 后最终头 17508661 上的 test lanes 于 07:40Z 被取消、未跑完(该头的本地定向验证 118/118 绿见 floor 7,且本轮 git 级核对确认合入树与其同 SHA)。合入后 master 构建管线在 12:49–12:51Z 跑绿并发布 0.0.767-master.924b852(floor 9)。合并动作由 owner 完成(12:27Z),不在 agent 权限内,也不在此复述其决策依据。

收工判定:冲突处理完成,PR 已合入 master 并已发布 master 构建,无遗留项,本任务闭环。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

1 participant