Repository navigation
feat: output-side compression (verbosity steering + effort routing) - #1096
ranxianglei wants to merge 5 commits into
Conversation
📦 Built Package ArtifactBranch: Option A — Install from npm PR tag (recommended)npm install -g billion-context@pr-1096Each push to this PR publishes a new version under the Option B — Download artifact
tar xzf billion-context-pr1096.tgz
npm install -g packageInstall instructions are refreshed on each push. |
…ks (#1093) Review round on PR #1096: the Responses turn classifier scanned the flat input list FORWARD for any user signal. Clients re-send full history every request (codex does), so the original ask was present on every turn and the classifier returned new_user_ask unconditionally — effort routing could only fire on delta-only inputs that never occur through bili. The other three wires classify the LAST message correctly, so this made effort routing dead on the flagship codex lane while appearing functional in tests (whose synthetic inputs carry no prior user message). - classifyResponsesTurn now walks BACKWARD from the tail: tool outputs after the last user signal = mechanical continuation; a user signal at the tail or with no outputs after it = new_user_ask. Assistant-side items between the last user signal and the trailing outputs no longer break the continuation (message/function_call/custom_tool_call/local_shell_call/ apply_patch_call/reasoning — previously custom/local/apply-patch calls marked the turn unknown even in the delta case). - Anthropic + Google classifiers: unrecognized block/part composition now returns unknown instead of being silently skipped (all four wires share one conservative stance: unrecognized structure => do not classify). - 6 regression tests incl. the realistic full-history repro; steering suite 23 -> 29, all green. - Docs: outputSteering entries in CONFIGURATION.md + CONFIGURATION.zh-CN.md (global + per-provider), CHANGELOG [Unreleased] entry.
[bot] 🏷 PR review — verified end-to-end, one real bug found & fixed on the branch, otherwise ready for your gate. What I checked (branch
Bug found & fixed (new commit Minor observation (report only, no action taken): one transient single-test failure appeared in 1 of 4 full-suite runs in this environment — not reproduced in the other 3 runs, and CI ubuntu 22/24 were green on the original head. That run's log wasn't retained, so the test name is unknown; if it recurs anywhere it's worth hunting per the determinism rule (§7.2). Docs gap closed: the PR carried no documentation for the new config block — added CI is re-running on the updated head; merging stays yours either way (feat ⇒ off the auto-merge path regardless). 中文摘要:审查了输出侧压缩 PR——发现并修复一个真实缺陷:Responses wire 的轮次分类器正向扫描整个 input,而 codex 每轮重发完整历史,原始提问永远在场,导致机械续轮恒被判为新提问、effort routing 在主力 codex 通道上实际失效(合成测试因无历史用户消息而未暴露);已改为从尾部反向扫描并统一四条 wire 的保守分类口径,补 6 个回归测试与中/英配置文档及 CHANGELOG,其余安全不变量(skip-if-absent、幂等尾追加、只降不注、默认关闭零行为变化)逐项核实通过,可以合并。 |
[bot] 🏷 Follow-up: CI is now fully green on the updated head |
|
Fix follow-up on the earlier review findings (cwst pass): fixed on this branch in b8bf783, gates green (typecheck / 2194 tests / build). 1. 2. Compress rounds are exempt from verbosity steering — 3. Bad config values now warn — Non-blocking notes from the review stay as documented behavior (per-route replace-not-merge; no env override; Google floor 128 assumed). @ework — please re-review the current head b8bf783 in full (fresh pass over the whole diff, not just these hunks). |
[bot] 🏷 Re-reviewed head b8bf783 in full — fresh pass over the whole diff, not just the three hunks. Verdict: fixes 1 and 2 are correct as implemented; fix 3 shipped with a false-positive bug, which I fixed directly (commit Fix 1 — minimal effort guard ✓. Both guards verified ( Fix 2 — compress-round exemption ✓ (one documented coarseness). I traced every request flowing through the loop transform: in Fix 3 — bad-config warning ⚠ had a false positive, fixed. As shipped in b8bf783, Also cleaned: b8bf783 left Fresh pass over the rest of the diff: all round-1 invariants still hold — skip-if-absent on all four wires, sentinel/tail/idempotent placement, clamp-only lowering (never injects, never toggles Gates at b4df2a1: typecheck clean · build success · full suite 2197 pass / 0 fail / 2 skipped (the two env-gated e2e-codex skips, unchanged). CI is running on the new head — early checks green; will confirm here when it finishes. 复审了 b8bf783 的全部改动:fix 1(minimal 不再被抬到 low)与 fix 2(压缩轮免 verbosity 指令)验证无误;fix 3 的告警在字段缺省时误报(合法配置被当成非法并打 warn),已直接在分支上修复(presence-aware 的 resolveVerbosityLevel + 3 个回归测试)并顺手清掉一处死代码,本地三道门全绿,CI 绿后可合并。 |
[bot] 🏷 CI follow-up: all 8 checks green on head Full re-review of b8bf783 is complete and reported above — the only defect found (false-positive config warning on omitted CI 全绿(8/8),PR 无冲突可合并;复审发现的唯一缺陷已在 b4df2a1 修复并带回归测试,等人工合并即可。 |
|
审查结论:实现质量过关,建议按 owner 拍板保留为参考实现 + 回归基准,待 acp-kernel#355 落地后收敛为 wire 落位层。 验证(rebased head)
硬边界逐条核对(全部落实 ✅)
与 #1092(锚定)的交互 — 无冲突锚定在 prepare 内比较的是客户端发来的 fresh 并存为 lastSeen;verbosity 在 forward boundary 之后追加到出站 body,不回流进锚的比较基准,不会产生假 diff。锚头字节 + compress 注入 + steering 尾注三者逐轮字节稳定,前缀缓存兼容。 亮点
残留风险(不阻塞,记录在案)
定位确认按 2026-09-22 拍板:轮次分类 / verbosity 档位 / effort 路由的决策逻辑下沉 acp-kernel#355,本 PR 保持打开作为规格与回归基准。届时按标准跨仓流程 bump 内核版本,再删除 |
Port headroom's two output-token levers to bili's forward boundary (#1093): - Verbosity steering: append a byte-stable conciseness directive (L0-L4, default L2) at the TAIL of the system prompt on all four wires (anthropic system, openai system/developer msg, responses instructions, google systemInstruction). Never prepends, never fabricates a system carrier when absent (skip-if-absent), idempotent so prefix cache stays hot. - Effort routing: structural turn classification of the final message list (no content pattern-matching); lowers an already-present effort field to its floor on mechanical continuations only. Never injects an effort field the client did not send, never toggles thinking.type. Default off; config in billion-context.json (global + per-provider route). Lives at the forward boundary after compat.roles so launcher/native-plugin/ /bili/ lanes inherit it; compress-retry re-sends share the same transform. Closes #1093
…ks (#1093) Review round on PR #1096: the Responses turn classifier scanned the flat input list FORWARD for any user signal. Clients re-send full history every request (codex does), so the original ask was present on every turn and the classifier returned new_user_ask unconditionally — effort routing could only fire on delta-only inputs that never occur through bili. The other three wires classify the LAST message correctly, so this made effort routing dead on the flagship codex lane while appearing functional in tests (whose synthetic inputs carry no prior user message). - classifyResponsesTurn now walks BACKWARD from the tail: tool outputs after the last user signal = mechanical continuation; a user signal at the tail or with no outputs after it = new_user_ask. Assistant-side items between the last user signal and the trailing outputs no longer break the continuation (message/function_call/custom_tool_call/local_shell_call/ apply_patch_call/reasoning — previously custom/local/apply-patch calls marked the turn unknown even in the delta case). - Anthropic + Google classifiers: unrecognized block/part composition now returns unknown instead of being silently skipped (all four wires share one conservative stance: unrecognized structure => do not classify). - 6 regression tests incl. the realistic full-history repro; steering suite 23 -> 29, all green. - Docs: outputSteering entries in CONFIGURATION.md + CONFIGURATION.zh-CN.md (global + per-provider), CHANGELOG [Unreleased] entry.
- treat explicit minimal effort as at-floor (clamp-only invariant: never raise a client-sent minimal to low on openai/responses) - compress rounds re-send without the verbosity directive (summary fidelity: the never-restate clause contradicts the compress prompt's verbatim-preservation contract); effort routing stays - warn (instead of silently clamping) on out-of-range verbosityLevel
…form (#1093) b8bf783 made clampVerbosityLevel warn on any non-integer-0..4 value, which includes undefined: every config that omits verbosityLevel (the documented default case) logged a false 'must be an integer 0-4' warning at load. resolveVerbosityLevel now warns only when the field is PRESENT but invalid (honest output); omitted silently takes L2. Also removes the now-unused wireTransform(true) local left behind when both compress-loop call sites moved to compressWireTransform(false), and corrects its stale comment.
b4df2a1 to
1986ffc
Compare
|
决定调整:不走 bili 侧先行实现,直接走内核路线。 决策逻辑(轮次结构分类 / verbosity L0–L4 / effort clamp)全部在 acp-kernel#355 落地并发版,billion-context 只做 wire 侧落位 + 配置管道。 本 PR 关闭重做:等 acp-kernel#355 就绪、按标准跨仓流程 bump 内核版本后重新开 PR,新 PR 只保留:
本实现的价值不丢:继续作为 acp-kernel#355 的规格参考与回归基准(内核 issue 已引用 PR 链接),审查结论见上一条评论(硬边界逐条核对通过、与 #1092 锚定无交互)。 |
Add two opt-in output levers at the proxy forward boundary (#1093): - Verbosity steering: append a deterministic conciseness directive to the system-prompt tail (skip-if-absent, sentinel-idempotent, never prepends). - Effort routing: clamp an already-sent effort field down on mechanical continuation turns (never injects an absent field, never raises, never toggles thinking.type). Division of labor (per #1096 rework spec): ALL decision logic — turn classification, verbosity-level resolution, lower-effort choice — lives in acp-kernel (decideOutputSteering / resolveOutputSteeringConfig, pinned 0.0.85). bili's src/output-steering.ts only LANDS those decisions on the four wires (openai chat / responses / anthropic / google-native); no local cwst/level classification is duplicated into bili. Config: `compress.outputSteering` rides the standard three-level compress cascade (global -> provider -> model) via mergeCompress; resolved + validated by the kernel's resolveOutputSteeringConfig at the forward boundary. Default off => byte-for-byte passthrough when disabled (no prefix-cache regression). Covers both forward paths: the initial string-body forward and compress-retry re-sends (via wireTransform). Launcher / native-plugin / /bili/ lanes inherit it automatically since it lives at the shared forward boundary. 35 wire-shape regression tests (tests/output-steering.test.ts).
…ks (ranxianglei#1093) Review round on PR ranxianglei#1096: the Responses turn classifier scanned the flat input list FORWARD for any user signal. Clients re-send full history every request (codex does), so the original ask was present on every turn and the classifier returned new_user_ask unconditionally — effort routing could only fire on delta-only inputs that never occur through bili. The other three wires classify the LAST message correctly, so this made effort routing dead on the flagship codex lane while appearing functional in tests (whose synthetic inputs carry no prior user message). - classifyResponsesTurn now walks BACKWARD from the tail: tool outputs after the last user signal = mechanical continuation; a user signal at the tail or with no outputs after it = new_user_ask. Assistant-side items between the last user signal and the trailing outputs no longer break the continuation (message/function_call/custom_tool_call/local_shell_call/ apply_patch_call/reasoning — previously custom/local/apply-patch calls marked the turn unknown even in the delta case). - Anthropic + Google classifiers: unrecognized block/part composition now returns unknown instead of being silently skipped (all four wires share one conservative stance: unrecognized structure => do not classify). - 6 regression tests incl. the realistic full-history repro; steering suite 23 -> 29, all green. - Docs: outputSteering entries in CONFIGURATION.md + CONFIGURATION.zh-CN.md (global + per-provider), CHANGELOG [Unreleased] entry.
What
Ports headroom's two output-token levers to bili's forward boundary. Output tokens cost ~5x input and are billed the moment they stream out, so the only lever is at request time. Both levers are default off — zero behavior change until a user opts in.
1. Verbosity steering — appends a byte-stable conciseness directive (levels L0-L4, default L2) at the tail of the system prompt. Never prepends (that would bust the prefix cache), skip-if-absent (never fabricates a system carrier the client didn't send), and idempotent (re-applying the same level is a no-op; a level change replaces the block in place). Covers all four wires: Anthropic
system, OpenAIsystem/developermessage, Responsesinstructions, GooglesystemInstruction.2. Effort routing — structural turn classification of the final message list (block composition only, no content pattern-matching). On a mechanical continuation turn it lowers an already-present effort field to its floor (
reasoning_effort,reasoning.effort,output_config.effort+thinking.budget_tokens->1024,generationConfig.thinkingConfig.thinkingBudget->128). Never injects an effort field the client did not send, never togglesthinking.type; dynamic budgets (-1) and non-mechanical turns are left untouched.Both run at the forward boundary after
compat.roles, so the classifier sees the final wire list; the launcher / native-plugin //bili/lanes inherit it automatically, and compress-retry re-sends share the same idempotent transform.Config
A per-provider route entry overlays the global block:
"providers": { "<url>": { "outputSteering": { ... } } }.Files
src/output-steering.ts(new): directives, idempotent tail placement, structural classifiers, clamp-only effort lowering, wire-shape detection.src/config.ts:outputSteeringin the global + per-route schema.src/server.ts: forward-boundary hook (initial forward + retry re-sends).tests/output-steering.test.ts(new): 23 regression tests across all four wires.Pre-flight
npm run typecheck— cleannpm run build— successnpm test— 2185 pass / 0 fail / 2 skipped (the skips are the env-gated e2e)e2e-codexrun needs a live Responses-compatible upstream (seetests/e2e/README.md), which isn't available in this environment; its zero-token preflight confirms the codex binary + built dist are ready. Because both levers are inert by default and the entire existing suite passes, the request pipeline is unaffected unless a user enables the feature.While validating I caught and fixed a bug in this very change: the resolved steering config wasn't defaulted when a
ProxyOptionsobject omitted the new field (hand-built options in the test harnesses), which threw aTypeErroron every forward (surfacing as 502s). Fixed by null-coalescing toDEFAULT_OUTPUT_STEERINGinforward().Closes #1093