Skip to content

feat: output-side compression (verbosity steering + effort routing) - #1257

Merged
ranxianglei merged 2 commits into
masterfrom
2026-09-24_output-side-compression
Sep 24, 2026
Merged

ranxianglei merged 2 commits into
masterfrom
2026-09-24_output-side-compression

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Implements output-side compression (#1093): two opt-in levers at the proxy forward boundary, ported from headroom and repositioned per the #1096 rework spec.

What

  • Verbosity steering — append a deterministic conciseness directive to the system-prompt tail (skip-if-absent, sentinel-idempotent, never prepends → no prefix-cache bust).
  • Effort routing — clamp an already-sent effort field down on mechanical continuation turns (never injects an absent field, never raises, never toggles thinking.type).

Design (division of labor)

Per #1096 close-out: all decision logic lives in acp-kernel (decideOutputSteering / resolveOutputSteeringConfig, pinned 0.0.85) — turn classification, verbosity-level resolution, lower-effort choice. bili's new src/output-steering.ts only lands those decisions on the four wires (openai chat / responses / anthropic / google-native). No local turn-classifier or level logic is duplicated into bili.

Config

compress.outputSteering rides the standard three-level compress cascade (global → provider → model) via mergeCompress, resolved + validated at the forward boundary by the kernel's resolveOutputSteeringConfig.

Behavior-change disclosure

Default off. When disabled (the default) the forward path is byte-for-byte passthrough — steerCfg resolves null and wireTransform reduces to the original compatProtocol condition, so there is zero change to any existing request and no prefix-cache impact. The levers only act when explicitly enabled in billion-context.json.

Coverage

Covers both forward paths — the initial string-body forward and compress-retry re-sends (via wireTransform). Launcher / native-plugin / /bili/ lanes inherit automatically since it lives at the shared forward boundary.

Tests & gates

  • 35 wire-shape regression tests (tests/output-steering.test.ts) — all pass.
  • npm run typecheck — green.
  • npm run build (tsup, acp-kernel inlined) — green.
  • Full suite: only pre-existing environment-dependent failures remain (4 in claude-native.test.ts — needs a real claude binary + live ps walk, unavailable in this sandbox); identical with and without this change. CI runs the complete suite.

Cross-repo dependency

Uses acp-kernel 0.0.85 (already pinned in master) — no new cross-repo bump required by this PR.

Fixes #1093

@github-actions

github-actions Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-09-24_output-side-compression (3f1075c)

Option A — Install from npm PR tag (recommended)

npm install -g billion-context@pr-1257

Each push to this PR publishes a new version under the pr-1257 npm tag.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr1257.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Review of PR #1257 — direction ✓, safety boundaries runtime-verified, gates green. I pushed three small labeled fix commits directly onto the branch (new head 42c77aab), so CI is re-running there. Details below.

① 原 issue / 方向关 — PASS. Checked against the #1096 close-out spec: all decision logic lives in acp-kernel 0.0.85 (decideOutputSteering / resolveOutputSteeringConfig / clampEffortToFloor / renderSteeringBlock); bili's new src/output-steering.ts only lands those decisions on the four wires (openai-chat / responses / anthropic / google-native). No local turn-classifier or level logic duplicated into bili. Config pipeline = global + per-provider overlay via mergeCompress, exactly as specced. Fixes #1093 present.

Safety boundaries — verified at runtime, not just by reading: I probed the installed acp-kernel 0.0.85 directly:

  • clampEffortToFloor: undefined/null/minimal/low/garbage → null (never injects an absent field, never raises, garbage untouched); medium/xhigh → low. Only-lower-only holds kernel-side.
  • decideOutputSteering: mechanical continuation → lowerEffort:true; fresh user ask and error continuation → false.
  • renderSteeringBlock(0) → null; default config {enabled:false, verbosityLevel:2, effortRouting:true} → off by default.

Default-off passthrough also proven structurally: when unset at every level mergeCompress leaves outputSteering as undefined (not {}), so steerCfg === null and wireTransform reduces to the original compatProtocol ? fn : undefined — zero change to any existing request, no prefix-cache impact, as the PR claims.

② 回归审查(既有行为清单):

  • forward() string-body path (src/server.ts): steering runs strictly after the compat.roles mutation and only when steerCfg.enabled — preserved.
  • wireTransform re-send path (billion-context: 400 Bad Request when codex sends role: "developer" messages — request a request-body compat transform hook #552 mechanism: compress-retry / truncation / degraded retries): guard changed from if (compatRoles) to if (compatProtocol && compatRoles) — equivalent, since transform existence implied compatProtocol before; the steering addition is independently gated. Preserved.
  • mergeCompress (src/compress-settings.ts): additive key only; other keys untouched. Preserved.
  • parseCompressSettings (src/config.ts): shape-guard only, sub-field validation deferred to the kernel resolver which falls back + warns at resolution time — nothing silently dropped. Preserved.
  • Compress rounds (my main open question — can a verbosity directive land on a compress-prompt round?): no. Proxy-mode compression executes locally/deterministically (executeProxyTool → applyRanges, src/loop/core.ts:162-169; zero HTTP in the session/persist/compress modules), and every model request goes through forward() or the loop's fetchUpstream, which applies wireTransform to every body (src/loop/core.ts:266-273). So the directive never reaches compress rounds and does reach all user-turn re-sends. Correct placement.
  • CI on original head e2b8f31c: 9/9 green. Local full suite on the PR head: 2468 pass / 3 fail / 2 skip — all 3 failures in tests/claude-native.test.ts (live ps-walk + real claude binary unavailable in this sandbox); verified identical failures on the master baseline → pre-existing, unrelated (PR test: gate claude-native live e2e subtests behind ACP_TEST_CLAUDE_NATIVE (#1248) #1253 addresses this class).

③ 代码审查: clean, single-purpose module; sentinel replaceOrAppend handles same-level idempotency (byte-stable no-op) and level change (in-place replace, exactly one block); skip-if-absent verified per-wire including cache_control preservation. Two findings:

  1. (flagged, not fixed) src/server.ts (~L4147): warnings from resolveOutputSteeringConfig are logged per request with no dedup, while siblings dedupe once-per-process (warnedPromptsRisk flag, warnedUnknownPack Set). A misconfigured value would warn forever, once per request. Recommend a once-per-process/per-route dedup set. Left unfixed because src/server.ts is a hot file under concurrent PRs and this is a behavior call for you.
  2. (flagged) The PR body says "three-level cascade (global → provider → model)" but the call site deliberately passes model=undefined (inline comment: provider granularity matches the owner spec "global + per-provider overlay"). Consequence: an outputSteering entry under a model key is silently ignored (sibling keys do pass real models, e.g. reasoningGuard at src/server.ts:4684). Suggest dropping "→ model" from the body; my docs commit makes the supported scope explicit instead.

④ 测试有效性: the 35 tests have teeth — they pin every hard boundary (per-wire tail-append, skip-if-absent byte-identical, idempotent byte-stability, in-place level replacement, mechanical-vs-non-mechanical classification per wire incl. codex custom_tool_call/apply_patch shapes, error continuation untouched, no field injection of unsent fields, minimal never raised to low, -1 dynamic budget untouched, default-off transparency). One gap fixed: the anthropic clamp test asserted budget_tokens===1024 but not that thinking.type stays put — added the assertion (pins hard boundary #3 directly).

⑤ diff 整洁度: pure after my fixes — every line serves the feature or its docs/tests. My commits:

  • cf9427d7 docs: CONFIGURATION.md/.zh-CN.md had no outputSteering entry while sibling keys were documented (doc-sync rule) — sections added to both, pure additions.
  • fccd715d test: pin anthropic thinking.type unchanged under effort clamp.
  • 42c77aab fix: two nits — (a) the PR had re-indented an unchanged line in src/config.ts from 4→5 spaces (diff noise); restored. (b) stale comment in src/output-steering.ts claiming the reserved {verbosity:false} seam was "used by the kernel's compress rounds" — no such production caller exists (compression executes locally, see ②); reworded as a reserved/test seam.

Post-fix gates: typecheck ✓, output-steering 35/35 ✓, build ✓. Pushed e2b8f31c..42c77aab; CI is starting on the new head.

Verdict: mergeable once CI is green on 42c77aab. Direction matches the owner spec exactly, the only-lower/no-injection/thinking.type boundaries are enforced kernel-side and pinned by tests, default-off is byte-transparent, and the diff is clean. Remaining items (warning dedup, PR-body wording) are non-blocking polish.

修复了输出侧压缩 PR 的三个审查小问题(CONFIGURATION 文档缺失、anthropic thinking.type 未断言、config.ts 缩进噪音+过期注释)并直接推到 PR 分支;方向与 owner 规格完全一致、只降不升等安全边界已在 acp-kernel 运行时验证、默认关闭零行为变更,CI 绿后可合并。

ework-agent and others added 2 commits September 24, 2026 17:44
Add two opt-in output levers at the proxy forward boundary (#1093):

- Verbosity steering: append a deterministic conciseness directive to the
  system-prompt tail (skip-if-absent, sentinel-idempotent, never prepends).
- Effort routing: clamp an already-sent effort field down on mechanical
  continuation turns (never injects an absent field, never raises, never
  toggles thinking.type).

Division of labor (per #1096 rework spec): ALL decision logic — turn
classification, verbosity-level resolution, lower-effort choice — lives in
acp-kernel (decideOutputSteering / resolveOutputSteeringConfig, pinned 0.0.85).
bili's src/output-steering.ts only LANDS those decisions on the four wires
(openai chat / responses / anthropic / google-native); no local cwst/level
classification is duplicated into bili.

Config: `compress.outputSteering` rides the standard three-level compress
cascade (global -> provider -> model) via mergeCompress; resolved + validated
by the kernel's resolveOutputSteeringConfig at the forward boundary. Default
off => byte-for-byte passthrough when disabled (no prefix-cache regression).

Covers both forward paths: the initial string-body forward and compress-retry
re-sends (via wireTransform). Launcher / native-plugin / /bili/ lanes inherit
it automatically since it lives at the shared forward boundary.

35 wire-shape regression tests (tests/output-steering.test.ts).
…ty-option doc, CONFIGURATION.md (+zh-CN) for outputSteering
@ranxianglei
ranxianglei force-pushed the 2026-09-24_output-side-compression branch from e2b8f31 to 3f1075c Compare September 24, 2026 09:46
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · 人工接手复核(替代 ework 自审)

结论:复核通过,已直接在 PR 分支修复并推送(3f1075c)。 CI 绿后可合。

复核发现并已修复

  1. 无关行缩进损坏:src/config.ts 把 reasoningGuard?: ReasoningGuardConfig; 的缩进从 4 空格改成 5 空格(§7.2 禁止的 incidental whitespace),新增的 outputSteering 行同样 5 空格——已恢复 4 空格对齐。
  2. 误导性死代码注释:ApplySteeringOptions.verbosity 的注释声称 "Used by the kernel's compress rounds",但全仓无任何调用点传它。核实:preflight summarize 走独立的 summaryPayload()/requestSummaryBody() 路径,完全绕过 steering 注入,所以这不是活 bug(摘要质量不受污染),但注释必须诚实——已改为 "Reserved for summarize-shaped requests should they ever route through here",并注明现状绕过。
  3. CONFIGURATION.md / zh-CN 双语缺 outputSteering 章节:新配置面无文档,违反文档纪律——已按 reasoningGuard 同款风格补齐(类型/默认/两杠杆语义/四 wire effort 字段与下限/三级合并/示例)。
  4. rebase:分支原落后 master 26 提交(test: gate claude-native live e2e subtests behind ACP_TEST_CLAUDE_NATIVE (#1248) #1253/feat: /acp-rule command — human entry point for recorded acp_rule rules (#1251) #1259/docs: record flat-50K nudge cadence as deliberate design (not window-scaled) #1265/fix: strip JSDoc import() annotations from dist bundles — dead opencode plugin lane (#1255) #1256/release v0.1.147 #1261/v0.1.147/fix: pin acp-kernel 0.0.86 — count-triggered distillation default-off (#379) #1260)——已 rebase 到 c96d91f。

复核通过的关键点(未改动)

  • 两种压缩模式(§6):proxy 模式 forward 边界注入 ✓,loop 重发经 wireTransform 同样携带 ✓;plugin 模式 agent 请求照走 forward ✓;compress 摘要不受污染(preflight 独立路径)✓
  • skip-if-absent:四 wire 无 system 载体绝不伪造 ✓;幂等:sentinel 原地替换,重试不累积,等级切换不重复(有测试)✓
  • effort clamp-only:不注入客户端没发的字段、不切 thinking.type、minimal 不上调(有测试)✓
  • 前缀缓存:默认关=字节透传;开=首请求一次重热,之后稳定(等级不变时)✓
  • 测试 37 个覆盖面扎实(四 wire/幂等/minimal guard/codex 形状/字节透明/协议推断),hermetic ✓

门禁(rebase+修复后,本地)

typecheck 0 错 / 2487 过 0 败 / build ✓(annotations OK)。

Fixes #1093。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature: Output-side compression (verbosity steering + effort routing)

1 participant