Skip to content

decideNudge() always prefers T1 compression over T2 distillation, starving tier-2 nudge indefinitely #508

Open

Description

@Su-luoya

Problem

decideNudge() in the nudge module has three structural issues that together make the tier-2 (T2) distillation nudge effectively unreachable in normal usage:

1. Growth path checks T1 before T2, and T1 always saturates

if (t1Eff >= nudgeGrowthTokens) {
  // T1 compression nudge — always fires when compressible messages exist
} else if (config.tiers.enabled && (t2CountReady || t2Pen >= tier2Threshold && t2Pen > t1Eff)) {
  // T2 distillation nudge — only reached when t1Eff < nudgeGrowthTokens (rare)
}

t1Eff (compressible message-range tokens) is almost always ≥ nudgeGrowthTokens (default 50K) in any active session. So the T1 branch consumes every nudge opportunity and T2 is never evaluated.

This is a fair-weather scheduler: it assumes compression is always preferable to distillation. But after many rounds of compression, the structural value of distilling accumulated blocks outweighs compressing yet another message range. The scheduler has no concept of "T2 is starving."

2. The t2Pen > t1Eff guard in the growth path is semantically wrong

t2Pen >= tier2Threshold && t2Pen > t1Eff
  • t2Pen = total tokens of tier-1 block summaries (≈ 400 tok × N blocks)
  • t1Eff = total tokens of currently-compressible message content (potentially 100K–400K)

Comparing summary tokens against message tokens is a category error. This guard essentially says "only distill summaries if they're larger than the messages" — which is never true in a working session. It prevents the growth path from ever triggering T2 even when t2Pen >= tier2Threshold is satisfied.

3. Pressure mode always selects T1

for (const t of candidates) {
  const p = tiers[t]?.pending ?? 0;
  if (p > bestPending) { bestPending = p; best = t; }
}

T1 pending (compressible message tokens) is typically orders of magnitude larger than T2 pending (summary tokens). So pressure mode always picks T1, never T2.

Consequence

T2 distillation is a dead code path. Blocks accumulate indefinitely → context bloat → window overflow → token waste. Users observe sessions consuming tokens at ~5× the expected rate.

A real session (01a11102, billion-context-pi@0.1.83) had 22 tier-1 blocks, ~570K context in a 256K window, and never received a T2 nudge — the nudge system always suggested T1 compression.

Suggested fixes

Option A (minimal, high-impact): Add a T2 "starvation check" before the T1 growth check

Insert a dedicated T2 path that fires when enough tier-1 blocks have accumulated, regardless of how much T1 compressible content remains:

// Starvation check: when enough tier-1 blocks exist and T2 hasn't been
// shown recently, prioritize distillation over compression.
const MIN_T2_BLOCKS = 10;  // could come from config (default: 10)
if (config.tiers.enabled && t2Count >= MIN_T2_BLOCKS) {
  const lastShown = state.nudge.lastShownByTier[2] ?? 0;
  const cadenceMet = lastShown === 0 || tokenCount - lastShown >= growthFloor;
  if (cadenceMet) {
    injectedTier = 2;
    // (continue to set injectedReason and return)
  }
}

This guarantees T2 gets a turn after accumulating enough blocks, independent of T1's pending size. The cadenceMet guard (same growthFloor/50K tokens used elsewhere) prevents repeated re-nudging.

Option B (architectural): Round-robin priority for tier selection

Instead of strict T1→T2→T3 priority, use a weighted schedule that periodically boosts T2/T3:

  • Every Nth nudge round, evaluate T2/T3 with a weight multiplier (e.g., T2 pending × 2.0).
  • Add a configurable tier2PrioritySkip (default: 2 — every 3rd nudge evaluates T2 on equal footing).

Option C (complementary): Remove the t2Pen > t1Eff guard

- t2Pen >= tier2Threshold && t2Pen > t1Eff
+ t2Pen >= tier2Threshold

The count path (t2CountReady) already acts as a guard. The > t1Eff comparison adds no value and makes the growth path unreachable. This alone won't fix the problem (T1 still checks first in the growth path), but it removes a source of confusion.

Option D (complementary): Fix the growth path ordering

Swap the T1 and T2 checks so T2 is evaluated first when enough blocks exist:

- if (t1Eff >= nudgeGrowthTokens) { ... T1 nudge ... }
- else if (config.tiers.enabled && t2CountReady) { ... T2 nudge ... }
+ if (config.tiers.enabled && t2CountReady) { ... T2 nudge ... }
+ else if (t1Eff >= nudgeGrowthTokens) { ... T1 nudge ... }

This alone is risky (could re-introduce cache sawtooth from issue #1249), but paired with Option A's cadenceMet guard, it's safe.

Recommended combination

Option A (starvation check) is the safest single fix: it adds a dedicated T2 path without altering any existing logic. Combined with lowering tier2Trigger in billion-context-pi (default 1000 → 10), it fully resolves the starvation problem.

Steps to reproduce

  1. Build a kernel session with 10+ tier-1 blocks plus compressible tier-1 message ranges.
  2. Call decideNudge() with context > 50% usage and growthSinceReference > growthFloor.
  3. Observe that the nudge always returns tier:1, never tier:2.
  4. Manually lower tier2Trigger to 5 and observe that T2 still never fires (because t1Eff >= nudgeGrowthTokens catches it first in the growth path, and T1 pending > T2 pending in the pressure path).

Environment

  • acp-kernel@0.0.98
  • Used via billion-context-pi@0.1.83

Activity

  1. ranxianglei commented on Oct 7, 2026

    @ranxianglei
    Owner

    🤖 Powered by ework · qwen3.8-27b

    [bot] 🏷 Verdict: not a defect — the observed behavior is the intended need-based arbitration. I verified the report against the exact code you ran (v0.0.98's decideNudge is line-identical to v0.0.100's; I diffed src/compress.ts across both tags), traced every T2 path, and built a 4-scenario empirical repro on top of the kernel. Details below. No PR from me — see the decision point at the end.

    What I verified

    Empirical scenarios (script run against this repo's source, default config, 256K window):

    Scenario Setup Result
    S1 your exact repro 22 T1 blocks (~8.8K tok summaries) + ~55K tok raw backlog, 570K/256K ctx (223% usage) tier 1 — EMERGENCY T1: max effective pending 39750, usage 223%
    S1b same + explicit tier2Trigger: 5 (your step 4) as above still tier 1 — count-ready T2 does not preempt larger raw pending
    S2 growth mode 200 T1 blocks (~80K tok summaries ≥ 75K threshold) + 12.5K tok raw backlog, 59% usage tier 2 — T2 distill ready: 200 tier-1 blocks (80000 tokens) >= 75000 (1.5x) and > T1 effective 12500
    S3 pressure, summaries dominate 300 T1 blocks (~120K tok) vs tiny raw backlog, 82% usage tier 2 — OVER-LIMIT T2 distill: max pending 120000 (T1 effective 0, ...)

    So T2 is not a dead code path: it fires in growth mode whenever summary mass crosses 1.5 × nudgeGrowthTokens (= 75K tokens by default) while raw backlog is under the 50K need threshold, and under pressure whenever summary pending is the largest pool (S3). Existing suites pin exactly this arbitration (tests/nudge.test.ts:608 max-pending routing, :677 count-triggered T2 with small mass, :705 "count-triggered T2 does not preempt ready T1 raw compression") — 39/39 pass on current head.

    Point-by-point on the three claimed defects

    1. "T1 always saturates, T2 never evaluated" — partially true, intended. T1 wins exactly when ≥50K of raw messages are reclaimable — strictly the bigger job than any sub-threshold distillation. In your session the math is unambiguous: ~550K uncompressed messages vs ~8.8K of summary mass. Distilling those 22 blocks would have reclaimed ~9K of the 570K in flight while rewriting the rendered wire from the fold anchor onward (prefix-cache invalidation far beyond the reclaimed tokens). The scheduler's economic crossover — "distill once accumulated summary mass outweighs another compression round" — is precisely what the 75K mass threshold implements, expressed in the right units (your own "structural value outweighs" argument, quantified).

    2. "t2Pen > t1Eff is a category error that makes the path unreachable" — provably vacuous in current code. Reaching that else-branch implies t1Eff < 50K ≤ 75K ≤ t2Pen, so the guard is true everywhere it's evaluated and has never blocked anything (same for the T3 analog at src/compress.ts:1808-1822). It documents the arbitration invariant ("exceeds every lower tier's effective pending"); Option C would be pure dead-condition removal with zero behavior change — I declined to touch kernel-owned nudge code for cosmetics.

    3. "Pressure mode always selects T1" — false as written. Pressure picks max pending (src/compress.ts:1773-1779, tested at tests/nudge.test.ts:608). T1 merely usually has the largest pending because an uncompressed-message backlog outmasses summaries. When summaries actually dominate context (the genuine accumulation pathology), pressure routes T2/T3 — S3 demonstrates it.

    Why Options A/B/D are rejected as-is

    What already exists for your use case (no kernel change needed)

    Two things worth chasing instead

    1. Your evidence session held 570K in a 256K window (223%). The kernel's overflow safety net (emergency truncation at 95%, terminal-floor escape signal after 3 rounds) should have been engaging long before that. That points at adapter-side tokenCount plumbing or truncate config in billion-context-pi — likely the real source of your over-consumption, not T2.
    2. The "~5× expected rate" claim isn't attributable to T2 starvation: deferring distillation costs at most the retained summary mass (~9K in that session), not hundreds of K. If billing data shows 5×, the driver is elsewhere (re-read loops, untruncated overflow, or provider-level caching behavior).

    If you still want count-driven distillation by default or an order swap, that's a deliberate product pivot off the #379/#380/#386 design — file it as such and let the owner sign off; I'll implement either variant on request.


    中文结构化结论(七问)

    1. 是什么问题:报告称 decideNudge() 结构性饿死 T2 nudge。经验证:观察到的现象(该会话只收到 T1 nudge)属实,但归因错误——这是设计内的需求驱动仲裁行为,不是缺陷。
    2. 什么原因导致的:无 bug 因果链。机制:T2 的触发条件是摘要 token 总量 ≥ 1.5×增长阈值(默认 75K)或显式计数触发(默认关:1000 块);T1 在可压缩原始消息 ≥50K 时优先(因为它本来就是更大的活儿);压力模式按 max-pending 路由。该会话中 22 块 ≈ 8.8K « 75K、原始积压 ≈ 550K » 其余一切 → 每次都是 T1。报告所称"语义错误的守卫"(t2Pen > t1Eff)在当前代码中可证明是恒真的死条件,从未阻挡过任何路径。
    3. 怎么修复的:不改代码。用四场景实证脚本(v0.0.98 等价代码):精确复现 → T1(max-pending 39.7K vs 8.8K,tier2Trigger:5 亦然);200 块/80K 摘要质量 @59% usage → growth 模式正常触发 T2;摘要占主导 @82% → 压力模式路由 T2。T2 可达且恰好在"蒸馏比再压一轮更划算"时获得调度权。
    4. 是否有回归问题:未改代码,无回归。反向论证:A/B 会重新引入 Count-triggered tier distillation distills without need (cache-loss root cause); #238 usage gate is itself a bug — default-off the count path, delete the gate #379(维护者 2026-09-24 决议)明确移除的"块数即需求信号",其后果是无谓蒸馏改写渲染线 → 前缀缓存失效远超回收量(billion-context#1249 振荡的根因);D 有同样风险;三者均触及 §7.3 内核契约与 §7.4 默认值,属人工决策域。C 为行为中性删除,拒绝执行(零用户可见收益,徒增契约代码评审面)。
    5. 是否破坏缓存:不涉及(无改动)。注:若采纳 A/B/D,反而会增加前缀缓存破坏频率(#1249 机制)。
    6. 是否改了设计方向:否。现行方向不变:增长量与 token 质量是需求信号,块数与使用率百分比不是(AGENTS.md §2 原则 7、Count-triggered tier distillation distills without need (cache-loss root cause); #238 usage gate is itself a bug — default-off the count path, delete the gate #379/fix: default-off count-triggered tier distillation, delete #238 usage gate #380/docs: record flat-50K nudge cadence as deliberate design (not window-scaled) #386)。
    7. 是否引入了新的配置:否。适配层现有配置已覆盖诉求:tiers.tier2Trigger(如设 10)即可今日启用计数触发;nudge.tier2GrowthMultiplier 可调质量阈值。

    收工判定:建议按 working-as-design 关闭本 issue;不产生 PR。若 owner 认为应将"计数驱动蒸馏"或"T2 优先排序"作为产品转向,请显式拍板后另行立项,我随时实现。另建议 reporter 顺带排查 billion-context-pi 侧 tokenCount/truncate 配置(223% 窗口占用说明溢出安全网未生效)。

  2. ranxianglei commented on Oct 9, 2026

    @ranxianglei
    Owner

    跨仓同步:迁移镜像 ranxianglei/billion-context#2512 已按分诊结论作为重复跟踪关闭——与镜像 #2513(原 #509)同一失效域,后者正文明确按机制归并管理两个 facet,修复 PR ranxianglei/billion-context#2519 #2513,槽位轮换,CI 绿)。本线程早先的「working-as-design」结论已被 #509 后续分析修正:显式调低的 count 触发器在 t1Eff 持续高位时结构性不可达,该 facet 成立为缺陷。本 issue 暂留 open,是否关闭由 owner 决定。

  3. ranxianglei commented on Oct 9, 2026

    @ranxianglei
    Owner

    🤖 Powered by ework · qwen3.8-27b

    [bot] 🏷 跨仓同步收到。已读 #509 全文并独立核对 PR #510 分支,接受结论修正,验证结果如下。

    对修正的确认(我早先结论的精确边界)

    初轮 WAD 判定需要一处更正,且只需这一处:实证部分全部仍然成立(S1b:显式 tier2Trigger:5 + t1Eff 持续高位确实恒出 T1),但我当时把该行为标注为「钉住的仲裁,非缺陷」。#509 的分层更准确:#379/README 承诺「explicit low triggers remain configurable」,而持续高 t1Eff 下该配置是死配置——配置契约被违反,此面成立为缺陷。默认行为(触发器 1000/2000 即关)维持 WAD 不变;两个结论不冲突——修的是 opt-in 安全阀,不是默认调度。

    已做的独立核对(PR #510 分支 origin/2026-10-07_nudge-tier-rotation @ bafbdfc,rebase 于 v0.0.100)

    • diff 逐段通读:count-ready tier 进入候选池不再受 t1Eff < nudgeGrowthTokens 短路;槽位轮换语义与描述一致(T1 保留首轮优先至 lastInjectedTier !== 1,之后交给 cadence stamp 最旧的 count-ready tier,下一周期还回 T1)。
    • 关键不变量成立:lastInjectedTier 仅在 nudgeNode 注入时落戳,applyCompression 成功重置路径无对应 diff hunk → 压缩成功不清轮换记忆,堵死「模型勤快压原始区间、每次平手重新偏向 T1」的 #628 生产形态。状态字段纯增量(state.ts 初始 null、types.ts 可选)→ 旧状态按全新处理,适配器零迁移。
    • 默认行为:默认触发器 1000/2000 → 候选池恒空 → 落入既有级联,逐字节一致(regression-tier-count-defaults 套件专门钉住)。
    • 面 2(t2Pen > t1Eff)有意未动,与 Count-triggered tier distillation distills without need (cache-loss root cause); #238 usage gate is itself a bug — default-off the count path, delete the gate #379「token-mass 为默认开启信号」的定位一致;放宽属另一产品决策,建议另立。
    • 本地定向复跑(PR head):tests/nudge.test.ts + tests/regression-tier-count-defaults.test.ts = 45/45 绿(含改写后的三轮序列钉住:首轮 T1 → 轮换 T2 → 还回 T1)。全量回归交 CI。
    • JSDoc 漂移(Default 5/10 → 实际 1000/2000)已在独立标注 docs commit 修正,README 补轮换语义说明。
    • 适配侧未单独审查 billion-context#2519(跨仓,属该线程审核职责);按你同步的信息记录:同修复移植,Closes #2513,CI 绿,待人工合并。

    遗留事项

    1. decideNudge() always prefers T1 compression over T2 distillation, starving tier-2 nudge indefinitely #508 本体已被 decideNudge(): growth-path T1-first ordering structurally starves T2/T3 distillation nudges when t1Eff >= nudgeGrowthTokens #509 完全覆盖(同一调度机制、同一生产实例 #628/session 01a11102;decideNudge() always prefers T1 compression over T2 distillation, starving tier-2 nudge indefinitely #508 三条主张分别对应 decideNudge(): growth-path T1-first ordering structurally starves T2/T3 distillation nudges when t1Eff >= nudgeGrowthTokens #509 的面 1、面 2 与后果描述)。按「一个失效域一个 issue」原则,建议你确认后以重复项关闭(#2512 镜像已如此处置)。我不擅自关闭,等你一句话。
    2. 我初轮提过的两个跟进里,有一个不在 decideNudge(): growth-path T1-first ordering structurally starves T2/T3 distillation nudges when t1Eff >= nudgeGrowthTokens #509 机制域内,fix: rotate count-triggered T2/T3 nudges with ready T1 on the growth path (#509) #510/#2519 均不覆盖:生产会话曾在 256K 窗口持有 ~570K(223%)上下文——内核溢出安全网(emergency truncation @95% + terminal-floor escape)显然未生效,指向适配侧 tokenCount 接线或 truncate 配置问题;「~5× token rate」的计费异常大概率同源。这是独立的失效域(billion-context-pi 侧),若要追建议在适配仓单独立项——跨仓提交需你同意,我不自行发起。

    收工判定

    本线程无需我再产生代码动作,不产生新 PR(loop guard)。#510 触及 §7.3 契约 8(nudge 不变量)+ 状态 schema 增量,留人工合并门禁是正确的。#508 关闭与否由你定;我的建议:关闭,标重复指向 #509。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions