Repository navigation
feat(compress): opt-in nudge-low-effort — clamp thinking/effort on first compression-nudge turn - #1780
feat(compress): opt-in nudge-low-effort — clamp thinking/effort on first compression-nudge turn#1780ranxianglei wants to merge 1 commit into
Conversation
…rst compression-nudge turn Add compress.nudgeLowEffort (boolean, default off): clamps the outbound request's existing effort/thinking fields to wire floors on the FIRST response of a compression-nudge episode, restoring on the next request (one-shot). Reuses outputSteering lowerEffort clamp (Anthropic thinking.budget_tokens floor 1024 / Gemini thinkingBudget floor 128 / OpenAI+Responses reasoning_effort->low); only lowers, never raises/injects absent fields, so it cannot bust the prompt cache. Cuts thinking tokens spent writing the summary (#853 ~9.5k reasoning tokens). Default off = byte-for-byte unchanged behavior. Part of #1640.
[bot] 🏷 收到,开始按 PR Review 规范审查 #1780(nudge-low-effort)。初步状态:分支 |
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.1747-pr.1780.3217f78This exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr1780.tgz
npm install -g packageInstall instructions are refreshed on each push. |
[bot] 🏷 Review complete — mergeable (merge itself stays human-only). ① Original issue direction (#1640)Matches the owner-approved mechanism exactly: floor 45 chose "压缩注入后的第一个回应直接降低思考参数,然后下一个请求立马恢复" (low implementation cost, no cache risk), floor 49 split into two PRs with this one first. Config placement in bili's own ② Regression review — existing behaviors touched, and verdictsThe diff touches these load-bearing paths; each checked against pre-change code:
No unintended regressions found. ③ Code review
④ Test validity
⑤ Diff cleanliness & hot-file coordinationDiff is tight: 6 files, every line on-purpose, no whitespace churn, docs en/zh fully mirrored (verified the zh section line-by-line). Coordination note: four open PRs also touch Behavior change disclosure (per policy)
Evidence
中文摘要:审查了 nudge-low-effort(压缩 nudge 首轮一次性压低思考参数、默认关闭的可选功能)——方向与 #1640 拍板一致,既有行为(nudge 注入判定、双压缩模式、输出转向、egress 摘要、side-passthrough、默认路径)逐项清点均无回归,测试有牙但缺一个 forward() 集成级用例(不阻塞),CI 13/13 全绿,可以合并;注意与 #1449/#1491 的热文件排序问题。 |
Adds an opt-in
compress.nudgeLowEffort(boolean, default off). When enabled, on the first response of a compression-nudge episode bili clamps the outbound request's existing effort/thinking fields down to the wire floors, then restores on the next request (one-shot). Reuses the existingoutputSteeringlowerEffortclamp infra.Motivation (#853 / #1640): on the turn where the model writes a compression summary, thinking models burn ~9.5k reasoning tokens bounded only by the output budget. Cutting that one turn's effort is the cheapest "save money on compression" lever — no bypass model, no wire-shape change beyond the effort param.
How it works
Prepared.injectedNudge).nudgeLowEffort === trueand this is the first turn of a nudge episode (tracked viasession.metadata.nudgeLowEffortPrev, refreshed every pass), the outbound body's effort fields are clamped viaapplyNudgeEffortClamp: Anthropicthinking.budget_tokens-> floor 1024, GeminithinkingConfig.thinkingBudget-> floor 128, OpenAI/Responsesreasoning_effort/reasoning.effort->low.outputSteering.enabled.Config surface
New field in the
compressblock (level 1 / global), in bili's own schema — NOT the kernel-ownedoutputSteeringschema (avoids a cross-repo acp-kernel change; we reuse its already-existinglowerEffortinternally).compress.nudgeLowEffort?: boolean, defaultfalse; non-boolean rejects the wholecompressblock (existing strict-parse behavior).outputSteering(cross-repo, rejected); a dedicated side-model summarization route (the separate follow-up PR under Feature suggestion: Asynchronous compression support for billion-context | 功能建议-billion-context 支持异步压缩 #1640).Behavior change (default off = zero change)
false: byte-for-byte identical behavior — no regression path.true: one-shot reduction of thinking/effort on the first compression-nudge turn only. Tradeoff: fewer reasoning tokens (cost) vs. possibly reduced deferral judgment — the model has ~10% chance to defer compression, which is valuable; lowering effort may reduce it. Mitigation: a[nudge-low-effort]info log marks every clamp so operators can A/B the compress/defer distribution across runs.Pre-flight (current master, acp-kernel 0.0.99)
npm run typecheck: cleannpm test: 3424 tests, 3405 pass, 0 fail, 19 skippednpm run build: successtests/nudge-low-effort.test.ts(per-wire clamp + floors, idempotency, safety on bad JSON / null protocol / array body, classifier-independence, one-shot episode decision cycle, config parse round-trip + non-boolean rejection).Scope
Part of #1640 (split into two PRs by owner). This PR = low-thinking approach (first). Pure async compression is a separate follow-up PR (not here).