Skip to content

feat: model-decided nudge timing (compress.nudgeModelDecided) - #2243

Merged
ranxianglei merged 3 commits into
masterfrom
2026-10-06_nudge-model-decided
Oct 8, 2026
Merged

ranxianglei merged 3 commits into
masterfrom
2026-10-06_nudge-model-decided

Conversation

@ranxianglei

@ranxianglei ranxianglei commented Oct 6, 2026 •

Copy link
Copy Markdown
Owner

Closes #2228

What

Model-decided compression timing (compress.nudgeModelDecided, default off). When enabled, an armed gentle-growth tier-1 nudge no longer injects the advisory text immediately. It first sends a short decision side call to the model over the session's already-cached prefix — same system/tools/messages prefix, tiny output budget (default 200 tokens), 15 s idle timeout, no extra tools — asking whether compressing NOW is net-beneficial for the current task. The answer must be strict JSON:

{"compress": true|false, "range": "mNNNNN-mNNNNN", "topic": "<short>"}
  • valid yes → injects an explicit directive with a program-finalized span: the model-proposed range is honored only when fully contained in a live compressible range, otherwise the largest live range is used (the model never dictates range boundaries outright);
  • no, malformed answer, timeout or HTTP error → nothing is injected that round; the session simply continues;
  • after 3 consecutive hard failures the next arm falls back to the legacy advisory once and the counter resets (self-healing ladder — a broken/quirky upstream can never silently kill proactive compression forever);
  • OVER-LIMIT / EMERGENCY pressure arms and tier-2/3 distillation bypass the decision entirely and keep the legacy advisory byte-for-byte — at that pressure the model holds no veto (owner ruling, 新功能:增加模型完全决策的压缩时机nudge触发 #2228 thread); host-side escalation (emergencyNudge) does the same.

Host-only: new module src/nudge-decide.ts (pure logic) + injection-site wiring in src/server.ts. No kernel changes. The decision call mirrors the forward path exactly (same route/proxy/header copy/APIG re-signing, non-stream response parsed per protocol with SSE fallback).

config surface

Two new fields under the existing three-level compress cascade (global → provider → model):

field type default meaning
compress.nudgeModelDecided boolean off (unset) enable model-decided tier-1 nudge timing
compress.nudgeDecisionMaxTokens number 200 output budget (tokens) of the decision side call; must be > 0

Why the existing surface cannot express this: everything currently in compress shapes when the proxy arms the legacy advisory (growth step, thresholds, tiers, preserve rules) or switches the whole advisory on/off (injectNudge). None of them can express "ask the model before injecting" — there is no consultation channel at all in the current schema, and bolting it onto injectNudge would conflate a boolean switch with a behavioral mode. Considered and rejected: the providers table (this is compression behavior, not provider routing); new BILI_* env vars (forbidden by the #2030 env-var discipline — a behavior knob belongs in the config file behind the existing cascade). Compat/migration: purely additive; unset = previous behavior byte-for-byte (verified by the off-path integration test below).

Behavior flags (past-behavior audit)

  • Default OFF → zero behavior change for existing users.
  • When ON, only GENTLE-GROWTH tier-1 arms consult the model; OVER-LIMIT / EMERGENCY pressure arms and tier≥2 keep the immediate legacy advisory exactly as before (owner ruling in the 新功能:增加模型完全决策的压缩时机nudge触发 #2228 thread: at that pressure the model holds no veto). Documented in CONFIGURATION.md en/zh.
  • New outbound request class: one extra upstream round-trip per armed tier-1 nudge (bounded by the 15 s idle timeout; counted against the same session/log as other traffic).
  • No persistence format, cache-key, retry, or dedup changes. Nudge/directive messages remain ephemeral trailing user messages (never persisted, never enter the agent's re-sent history) — same anchor safety as the legacy nudge.

Tests

  • tests/nudge-decide.test.ts — 9 unit tests: prompt shape, strict parser accept/reject matrix (garbage never degrades to yes-without-range), span finalization rules (contained proposal honored, straddling/out-of-view/garbage/reversed fall back to largest live range), 3-strike fallback ladder, per-protocol answer extraction.
  • tests/nudge-model-decided.test.ts — 5 end-to-end scenarios through the real pipeline (mock upstream discriminates the side call by its ≤200-token budget): gentle-growth yes → directive with program-finalized span injected; no → nothing; garbage prose → nothing; feature off → forwarded body identical to legacy advisory and zero side calls; over-limit (pressure band) → immediate legacy advisory, model never consulted (owner ruling). The gentle-growth fixture needs three turns because reported usage lags one turn (settleUsageReport runs per response): turn 1 seeds the baseline, turn 2's report carries turn 3's prepare past the growth floor at 62.5% usage — under the 75% over-limit line.
  • Regression control: tests/plugin-nudge-injection.test.ts still green.
  • On the rebased head: npm run typecheck clean, npm run build green; targeted runs green — 9/9 unit (nudge-decide), 5/5 e2e (nudge-model-decided), 2/2 legacy-nudge regression control (plugin-nudge-injection). (One full-suite failure observed on the OLD base in tests/pi-subagent-registry.test.ts was reproduced on a clean tree without my diff — pre-existing, and already fixed upstream by master red: pi-subagent-registry factory test broken by #2209 + #2186 semantic merge (fake pi Map shadows selfRegister handler) #2234; green on the rebased head.)

Docs: CONFIGURATION.md (en) + CONFIGURATION.zh-CN.md quick-ref rows and detail sections added.

@github-actions

github-actions Bot commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-06_nudge-model-decided (c053d26)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.2579-pr.2243.4425341

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr2243.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 收到,正在处理这个 PR(#2243,分支 2026-10-06_nudge-model-decided,1 commit / 8 files / +868 −31)。当前状态:mergeable=true,mergeStateStatus=blocked(等 CI/审批)。我将按 review 规范执行:拉取分支核对代码 → 检查 CI → 本地定向验证(typecheck + nudge 相关测试)→ 按结构化七问给出结论。稍后回复完整审查报告。

ranxianglei pushed a commit that referenced this pull request Oct 6, 2026
…mpress.yaml

Review fix for #2243: the PR hand-edited CONFIGURATION.md/.zh-CN.md but the
config docs are generated from the YAML seeds (#2107 single source of truth),
so the docs-sync CI job failed — regeneration dropped the two rows. Add the
seed entries (descriptions taken from the hand-written rows; backticks in
desc_* are generator-forbidden, so the nudgeDecisionMaxTokens row loses its
inline key backticks) and regenerate all four doc artifacts.
ranxianglei pushed a commit that referenced this pull request Oct 6, 2026
The module header said 'the emergency/over-limit path bypasses the decision
outright', but the implemented gate (and the owner-approved #2228 policy)
routes OVER-LIMIT tier-1 arms through the decision; only the EMERGENCY arm
bypasses it. Comment-only fix.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

PR #2243 审查完成。代码本身审查通过(两种压缩模式都推演过、内核零改动、测试有牙),审查中发现 2 处缺陷已直接在分支上修掉并推送(89eb8ec37..f019ef2b2),CI 全绿。按七问作答:

1. 是什么问题:#2228——现有 nudge 节奏是纯增长驱动(每 50K 增长必注入),经常在不值得压缩的时机打扰模型;本 PR 让 tier-1 nudge 先问模型一句「现在压不压」,模型说压才注入指令。

2. 什么原因导致的:机制层原因是 nudge 的触发决策权完全在宿主(growth 计数器),模型对「此刻上下文是否真的需要压缩」没有发言渠道;PR 用一次带缓存前缀的决策 side-call 补上这个渠道,而不是改触发阈值(改阈值会动 #379/#380 定下的 flat-50K 契约,方向不对)。

3. 怎么修复的:纯宿主侧新增 src/nudge-decide.ts(纯逻辑)+ src/server.ts 接线。armed 的 tier-1 nudge 先发一个与主请求同路由、同稳定前缀、≤200 token 预算、15s 超时的非流式 side-call,要求严格 JSON {compress, range, topic};yes+合法 range 才注入指令且 span 由程序终裁(只接受完整落在某个 live compressible range 内的提议,否则取最大 live range,没有则本轮不注入);no/畸形/超时/HTTP 错误一律静默跳过;连续 3 次硬失败后下一轮回落一次旧版 advisory 再复位(自愈阶梯)。EMERGENCY 臂与 tier-2/3 distillation 完全旁路,走原路径。

4. 是否有回归问题(逐项核过的既有行为):默认关闭时零变更——else 分支逐行复刻改前代码,e2e 对照组断言 side-call 数为 0 且 advisory 字节级一致;EMERGENCY/tier-2/3 旁路保持;plugin/proxy 双模式下指令走同一 wrapper 链(withMarkerIntegrityNote(withSummaryBudgetNote(...))),回归套件绿;strict-echo 修复在 openai/responses 两条线上镜像了主流程的 normalize 函数(anthropic 无此修复,正确地未镜像);nudge 仍是临时性尾部 user message,不落盘、不进 agent 重发历史;message-ref/id 与持久化格式未触碰。一处有意行为变更(披露):nudgeModelDecided: true 开启后,over-limit 的 tier-1 臂从「必注入 advisory」变为「先经模型决策」——这是 #2228 拍板的策略本身,en/zh 文档均已写明,默认值下无任何变化。

5. 是否破坏缓存:side-call 复用与主请求完全相同的稳定前缀(同路由——resolveUpstream/buildForwardTarget 均为纯函数,我验证了重解析结果与主请求转发目标一致;同 headers、同消息体字节),即同一 cache line 上的追加调用,不产生新缓存条目也不使既有条目失效;side 体上剥掉 prompt_cache_key/prompt_cache_retention;会话状态只新增一个阶梯计数键,未触碰任何既有缓存/去重/重试键。

6. 是否改了设计方向:没有。内核边界完整(kernel-guard 绿,kernel/** 零改动),压缩决策权仍归 ACP agent/代理执行,本 PR 只在「何时提示」这一时机上加了一个可关的咨询环节;Install-Lane 契约、两模式摘要载体等关键设计均未动。

7. 是否引入了新的配置:是,两个字段挂在既有三级 compress 级联(global→provider→model)下,无新 env var(#2030 合规):

  • compress.nudgeModelDecided:boolean,默认 off/unset;
  • compress.nudgeDecisionMaxTokens:number,默认 200,必须 >0(向下取整)。
    CONFIGURATION.md 中英文已同步。

审查中发现并已直接修复的 2 处缺陷(additive commits,见上):

  1. docs-sync CI 红(阻断项):PR 手改了生成物 CONFIGURATION.md/.zh-CN.md 但没往 website/config-reference/compress.yaml 加种子([docs][website] 配置文档单一事实源:从声明式 schema 生成 CONFIGURATION.md 与网站 /docs/(一步到位) #2107 单一事实源),重新生成时两行被丢弃导致 diff 校验失败。已补种子并重生成全部 4 个产物(commit bb7e7351d)。附带披露:生成器禁止 desc 字段含反引号,所以生成的 nudgeDecisionMaxTokens 行文案从 used by \nudgeModelDecided`变为used by nudgeModelDecided`(仅生成行的措辞差异,无功能影响)。
  2. 模块头注释失实:src/nudge-decide.ts 头注释写「emergency/over-limit 路径直接绕过决策」,与实现不符(实际只有 EMERGENCY 绕过,over-limit 走决策)。纯注释修正(commit f019ef2b2),无行为变化。

验证:本地定向 65/65 通过(9 单测 + 4 e2e + plugin-nudge 回归对照 + compress-settings/knobs 级联),typecheck 干净,全量交给 CI;CI 在 f019ef2 上 25/25 全绿(含此前红的 docs-sync)。测试有效性:mock upstream 用 ≤200 token 预算精确区分 side-call,解析器有完整的接受/拒绝矩阵,e2e 有 OFF 对照组钉住字节级兼容——是真测试,bug 回归时抓得住。

收工判定:可合并(feat + 配置面新增,按 §7.4 不走自动合并,请人工合并):#2243

@ranxianglei
ranxianglei force-pushed the 2026-10-06_nudge-model-decided branch 2 times, most recently from 9eec3fa to 92c8f5b Compare October 8, 2026 01:03
@ranxianglei

Copy link
Copy Markdown
Owner Author

又冲突

ework-agent added 3 commits October 8, 2026 14:09
When enabled, an armed tier-1 (gentle growth or over-limit) compression
nudge first asks the model - over the session's already-cached prefix, in a
short side call with a tiny output budget and no extra tools - whether
compressing NOW is net-beneficial for the current task. The answer must be
strict JSON ({"compress": true|false, "range"?, "topic"?}); a valid yes
injects an explicit directive with a program-finalized span (the proposed
range is honored only when fully contained in a live range, else the largest
live range), while no/malformed/timeout injects nothing that round. After 3
consecutive hard failures the next arm falls back to the legacy advisory once
and resets (self-healing ladder). EMERGENCY arms and tier>=2 distillation
always keep the legacy advisory byte-for-byte. Default off; host-only (not
passed to the kernel); no new env vars (#2030).
…mpress.yaml

Review fix for #2243: the PR hand-edited CONFIGURATION.md/.zh-CN.md but the
config docs are generated from the YAML seeds (#2107 single source of truth),
so the docs-sync CI job failed — regeneration dropped the two rows. Add the
seed entries (descriptions taken from the hand-written rows; backticks in
desc_* are generator-forbidden, so the nudgeDecisionMaxTokens row loses its
inline key backticks) and regenerate all four doc artifacts.
The module header said 'the emergency/over-limit path bypasses the decision
outright', but the implemented gate (and the owner-approved #2228 policy)
routes OVER-LIMIT tier-1 arms through the decision; only the EMERGENCY arm
bypasses it. Comment-only fix.
@ranxianglei
ranxianglei force-pushed the 2026-10-06_nudge-model-decided branch from 92c8f5b to c053d26 Compare October 8, 2026 06:10
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 冲突已解决,rebase 到最新 master 并推送完成。

结构化八问

  1. 是什么问题:PR 分支与 master 冲突——本 PR 的 base(46fb5a7)之后 master 前进了 468 个提交。
  2. 什么原因导致的:master 的 a233b6efe(audit: 全仓冗余审计与重构计划——server.ts 解体、客户端矩阵去重、死代码、AGENTS.md 模块图腐化 #1440 P2 cut 2)把 prepareAnthropic/Openai/Google/Responses 四个函数从 src/server.ts 拆到了 src/server/prepare-{anthropic,openai,google,responses}.ts;而本 PR 的主体改动正是这四个函数内的 nudge 注入块 + server.ts 里的调用点与 helper,文件结构变了,文本冲突不可避免。
  3. 影响范围与严重程度:仅代码落位变化,零行为影响;default-OFF 路径保持字节级不变(有 e2e 控制场景钉住)。
  4. 怎么修复的:rebase 到最新 master(8c62d64cb),保持原 3-commit 结构(feat + 两个 review 小修)。nudge 三段式逻辑(fallback 梯子 / side-call / legacy advisory)按原设计重铺进四个新 prepare 模块;吸收 master 侧两处签名漂移:withSummaryBudgetNote 现在带第二参 externalSummaryEnabled(config)(feat: 统一外部摘要模型、顺序故障转移与多客户端压缩接线 #2167)、分析缓存泄露问题又一个日志 #2302 的 renderedWithPayload 模式;runNudgeDecision 从模块私有改为 export(原先同文件无需导出,拆文件后跨模块需要)。
  5. 是否有回归问题:无。逐项验证:typecheck clean;定向测试 65/65(tests/nudge-decide.test.ts、tests/nudge-model-decided.test.ts、tests/plugin-nudge-injection.test.ts、tests/compress-settings.test.ts、tests/knobs-config-precedence.test.ts);CI 28/28 全绿(head c053d2636,含 docs-sync、wire-contract、kernel-guard)。default-off 时转发体与 legacy 逐字节一致由 e2e 控制场景断言(零 side call + legacy advisory)。
  6. 是否破坏缓存:否。side call 仍经纯函数 resolveUpstream/buildForwardTarget 取与主请求相同的前向目标,落同一条 cache line;prompt_cache_key/prompt_cache_retention 照旧从 side body 剥离;nudge/directive 仍是 ephemeral trailing user message,不进持久化历史。
  7. 是否改了设计方向:否。仍为 host-only(kernel 未动、无 kernel-change 声明需求)、config surface 不变、两压缩模式的推理路径不变。
  8. 是否引入了新的配置:否。rebase 未新增任何配置;PR 本身的两个字段(compress.nudgeModelDecided 布尔默认 off、compress.nudgeDecisionMaxTokens 数字默认 200)已在原 PR 描述的 config surface 一节报告过。

判定:冲突解除(PR mergeable=true,head c053d2636),CI 全绿,可以合并——合并操作按规则留给人工:#2243

@ranxianglei
ranxianglei merged commit 89da0a8 into master Oct 8, 2026
28 checks passed
@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.137-master.89da0a8 (rolling master tag).

npm install -g billion-context@0.0.137-master.89da0a8  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

新功能:增加模型完全决策的压缩时机nudge触发

1 participant