fix: remove context-relative suppression from filterRecommendedRanges (Issue #251)

devlog/2026-07-30_filter-recommended-ranges-fix/REQ.md

@@ -0,0 +1,35 @@
+# REQ: Fix filterRecommendedRanges suppressing ranges at large context windows (Issue #251)
+
+## Problem
+
+`filterRecommendedRanges` used a context-relative threshold (`modelContextLimit × 5%`) to suppress compression recommendations when the "effective compressible" tokens were below the threshold. At 1M context windows (Claude Sonnet 4.5), this threshold was 50K — meaning sessions with 30-40K of genuinely compressible content never received compression recommendations, even when the nudge system correctly detected growth.
+
+## Impact
+
+- Sessions at 1M context never get compression recommendations until 50K+ tokens accumulate
+- Nudge fires (growth ≥ threshold) but immediately suppressed → `nothingToCompress = true` → no nudge text injected
+- Model never sees compressible ranges → context grows unbounded → eventually hits hard limit
+
+## Root Cause
+
+`filterRecommendedRanges` in `lib/messages/inject/utils.ts`:
+- `growthThreshold = modelContextLimit × 0.05` (50K at 1M, 10K at 200K)
+- `lastSegmentFloor = growthThreshold × 2` (100K at 1M, 20K at 200K)
+- Aggregate `effectiveCompressible` (non-last ranges + excess of last range beyond floor) compared against `growthThreshold`
+- If below threshold → return `[]` → `filterSuppressed = true` → `nothingToCompress = true`
+
+## Fix
+
+**Simplify `filterRecommendedRanges`**: remove all suppression logic. Always return all input ranges, marking the last segment as `dangerous: true`.
+
+**Backstop**: `minCompressRange` in `range.ts:185` (5000 chars ≈ 1250 tokens) already prevents garbage compressions at the compress tool level.
+
+**Clean up dead code**: Remove `filterSuppressed` variable from `nothingToCompress` calculation (always `false` after fix).
+
+## Files Changed
+
+- `lib/messages/inject/utils.ts` — rewrite `filterRecommendedRanges`, simplify `RangeFilterOptions`
+- `lib/messages/inject/inject.ts` — update call site, remove `filterSuppressed`, update debug logging
+- `tests/smart-nudge-gating.test.ts` — rewrite for new behavior (8 tests)
+- `tests/property-invariants.test.ts` — rewrite INV5 (always returns all ranges)
+- `tests/inject.test.ts` — update 2 tests for new behavior

devlog/2026-07-30_filter-recommended-ranges-fix/WORKLOG.md

@@ -0,0 +1,38 @@
+# WORKLOG: Fix filterRecommendedRanges suppressing ranges at large context windows
+
+## Changes
+
+### `lib/messages/inject/utils.ts`
+- Rewrote `filterRecommendedRanges`: removed `growthThreshold`, `lastSegmentFloor`, `suppressed`, `effectiveCompressible` logic
+- New behavior: always return all input ranges, last segment marked `dangerous: true`
+- Simplified `RangeFilterOptions`: removed `modelContextLimit` and `growthRatio`, only `logger` remains
+
+### `lib/messages/inject/inject.ts`
+- Updated `filterRecommendedRanges` call: `{ logger }` instead of `{ modelContextLimit, growthRatio: 0.05, logger }`
+- Removed `filterSuppressed` from `nothingToCompress` calculation (always false after fix)
+- Updated debug logging: removed `growthThreshold`, `lastSegmentFloor`, `suppressed` references
+
+### Tests
+- `tests/smart-nudge-gating.test.ts`: Rewritten — 8 tests covering new behavior (no suppression, last gets dangerous, tiny ranges shown, etc.)
+- `tests/property-invariants.test.ts`: INV5 rewritten — property: always returns all input ranges (300 random runs)
+- `tests/inject.test.ts`: 2 tests updated — old "nudge suppressed" tests now assert nudge fires
+
+## Verification
+
+- `npm run typecheck` — clean
+- `npm test` — 942 tests pass, 0 failures
+
+## Additional Fix: Nudge Loop Bug (lastNudgeShownTokens reset)
+
+### Problem
+When `nothingToCompress` was true (all ranges protected), `lastNudgeShownTokens` was reset to `undefined`. This caused a loop: next turn, `growthReference` fell back to stale `lastPerMessageNudgeTokens` (potentially very old), producing artificially huge growth → nudge fires → nothingToCompress again → reset → repeat every turn.
+
+### Changes
+- `lib/messages/inject/inject.ts`:
+  - Removed `lastNudgeShownTokens = undefined` from `nothingToCompress` path (lines 367-369 deleted). Baseline preserved, half-threshold gate applies naturally.
+  - Fixed growth display: `currentTokens - lastPerMessageNudgeTokens` → `currentTokens - (lastNudgeShownTokens ?? lastPerMessageNudgeTokens)`. Display now matches the actual growthReference used for nudge decisions.
+
+### Tests
+- `tests/inject.test.ts`:
+  - Updated "pending nudge cleared when suppressed" → "pending nudge preserved when all-protected — no loop" (asserts baseline kept, not reset)
+  - Added "multi-turn: all-protected does not loop (lastNudgeShownTokens stable)" — 3-turn regression test verifying baseline stability

lib/messages/inject/inject.ts

@@ -343,42 +343,30 @@ export const injectCompressNudges = (
     const recommendedRanges = filterRecommendedRanges(
         unprotectedCompressible,
         contextRanges.protected,
-        { modelContextLimit, growthRatio: 0.05, logger },
+        { logger },
     )
     const hasRecommendations = recommendedRanges.length > 0
 
-    if (config.debug && contextRanges.compressible.length > 0 && modelContextLimit) {
-        const growthThreshold = modelContextLimit * 0.05
-        const lastSegmentFloor = growthThreshold * 2
+    if (config.debug && contextRanges.compressible.length > 0) {
         const compressible = contextRanges.compressible
-        const lastRange = compressible[compressible.length - 1]
-        const lastIncluded = lastRange.tokens >= lastSegmentFloor
-        const nonLastTokens = compressible
-            .slice(0, -1)
-            .reduce((s, r) => s + r.tokens, 0)
-        const effective = nonLastTokens + Math.max(0, lastRange.tokens - lastSegmentFloor)
-        const suppressed = effective < growthThreshold
         const fmt = (n: number) => (n >= 1000 ? `${(n / 1000).toFixed(1)}K` : String(n))
         const lines = [
-            `[ACP Debug] Recommendation filter decision:`,
+            `[ACP Debug] Recommendation filter:`,
             `  Input: ${compressible.length} range(s), ${fmt(compressible.reduce((s, r) => s + r.tokens, 0))} tokens`,
-            `  Thresholds: growth=${fmt(growthThreshold)}, lastSegmentFloor=${fmt(lastSegmentFloor)}`,
-            `  Last segment: ${lastRange.startRef}–${lastRange.endRef} ${fmt(lastRange.tokens)} tokens → ${lastIncluded ? "included (dangerous)" : "excluded (< floor)"}`,
-            `  Effective compressible: ${fmt(effective)} → ${suppressed ? "SUPPRESSED (< growth threshold)" : `${recommendedRanges.length} range(s) recommended`}`,
+            `  Output: ${recommendedRanges.length} range(s) (last segment marked dangerous)`,
         ]
         logger.debug(lines.join("\n"))
     }
 
-    const filterSuppressed = contextRanges.compressible.length > 0 && !hasRecommendations
     const allProtected = contextRanges.compressible.length === 0 && contextRanges.protected.length > 0
     const allInProtectedZone = protectedRefs.size > 0 && unprotectedCompressible.length === 0
-    const nothingToCompress = filterSuppressed || allProtected || allInProtectedZone
+    const nothingToCompress = allProtected || allInProtectedZone
     const shouldInjectNudge = nudgeAllowed && (!nothingToCompress || emergencyOverride)
     let shouldInject = shouldInjectNudge
 
-    if (nudgeAllowed && nothingToCompress && !emergencyOverride) {
-        state.nudges.lastNudgeShownTokens = undefined
-    }
+    // Keep lastNudgeShownTokens when nothingToCompress — resetting it
+    // reintroduces the nudge loop (baseline wiped → stale growthReference
+    // → nudge fires every turn).
 
     // Issue #216 Defect 1: only apply anchored nudge text when there IS something
     // to compress. Previously applyAnchoredNudges ran before nothingToCompress was
@@ -506,8 +494,9 @@ export const injectCompressNudges = (
             const pct = (n: number) =>
                 n > 0 ? Math.max(1, Math.round((n / composition.total) * 100)) : 0
             const growth =
-                currentTokens !== undefined && state.nudges.lastPerMessageNudgeTokens !== undefined
-                    ? currentTokens - state.nudges.lastPerMessageNudgeTokens
+                currentTokens !== undefined &&
+                (state.nudges.lastNudgeShownTokens ?? state.nudges.lastPerMessageNudgeTokens) !== undefined
+                    ? currentTokens - (state.nudges.lastNudgeShownTokens ?? state.nudges.lastPerMessageNudgeTokens!)
                     : 0
             const growthStr = growth > 0 ? ` (+${fmt(growth)} since last nudge)` : ""
 

lib/messages/inject/utils.ts

@@ -761,92 +761,46 @@ export function buildCompressibleRanges(
 }
 
 export interface RangeFilterOptions {
-    modelContextLimit: number | undefined
-    growthRatio: number
     logger?: { debug: (msg: string, data?: any) => void }
 }
 
 /**
  * Filter compressible ranges for the recommendation list.
  *
- * Last segment: excluded if < 2× growth threshold; included with
- * `dangerous: true` flag if >= 2× growth threshold.
+ * All ranges are shown to the model — the model decides what to compress.
+ * The last segment is always marked `dangerous: true` (it may still be in
+ * active use; the model is warned in the suffix text).
  *
- * Gate: "effective compressible" = non-last-segment tokens + max(0,
- * last-segment tokens − 2× growth threshold). If effective compressible
- * < growth threshold, suppress all recommendations.
+ * Issue #251: Previously this function used `growthThreshold` (5% of context
+ * window = 50K at 1M) as an aggregate gate — if "effective compressible"
+ * was below the threshold, ALL ranges were suppressed and the nudge was
+ * hidden. At large context windows, individual ranges rarely exceeded this
+ * threshold, so compression was permanently blocked. The aggregate gate
+ * has been removed; `minCompressRange` in `range.ts` (5000 chars) already
+ * prevents garbage compressions as a backstop.
  */
 export function filterRecommendedRanges(
     compressible: CompressibleRange[],
     _protectedRanges: ProtectedRange[],
     options: RangeFilterOptions,
 ): CompressibleRange[] {
-    const { modelContextLimit, growthRatio, logger } = options
+    const { logger } = options
     const log = logger?.debug.bind(logger)
 
-    if (!modelContextLimit || modelContextLimit <= 0) {
-        log?.("filterRecommendedRanges: modelContextLimit unknown, passthrough", {
-            inputCount: compressible.length,
-        })
-        if (compressible.length === 0) return []
-        const passthrough = [...compressible]
-        passthrough[passthrough.length - 1] = {
-            ...passthrough[passthrough.length - 1],
-            dangerous: true,
-        }
-        return passthrough
-    }
-
     if (compressible.length === 0) {
         log?.("filterRecommendedRanges: no compressible ranges, returning empty")
         return []
     }
 
-    const growthThreshold = modelContextLimit * growthRatio
-    const lastSegmentFloor = growthThreshold * 2
-
-    const lastIndex = compressible.length - 1
-    const lastRange = compressible[lastIndex]
-    const lastSegmentIncluded = lastRange.tokens >= lastSegmentFloor
-
-    let effectiveCompressible = 0
-    const result: CompressibleRange[] = []
-
-    for (let i = 0; i < compressible.length; i++) {
-        const r = compressible[i]
-        if (i === lastIndex) {
-            const excess = Math.max(0, r.tokens - lastSegmentFloor)
-            effectiveCompressible += excess
-            if (lastSegmentIncluded) {
-                result.push({ ...r, dangerous: true })
-            }
-        } else {
-            effectiveCompressible += r.tokens
-            result.push(r)
-        }
-    }
-
-    const suppressed = effectiveCompressible < growthThreshold
+    const result = compressible.map((r, i) =>
+        i === compressible.length - 1 ? { ...r, dangerous: true } : r,
+    )
 
-    log?.("filterRecommendedRanges: decision", {
+    log?.("filterRecommendedRanges: passthrough (last segment marked dangerous)", {
         inputRanges: compressible.length,
-        inputTokens: compressible.reduce((s, r) => s + r.tokens, 0),
-        growthThreshold,
-        lastSegmentFloor,
-        lastSegment: {
-            ref: `${lastRange.startRef}–${lastRange.endRef}`,
-            tokens: lastRange.tokens,
-            included: lastSegmentIncluded,
-        },
-        effectiveCompressible,
-        suppressed,
-        outputRanges: suppressed ? 0 : result.length,
+        outputRanges: result.length,
     })
 
-    if (suppressed) {
-        return []
-    }
-
     return result
 }
 

tests/inject.test.ts

@@ -751,11 +751,11 @@ test("growth floor: 98% emergency override fires regardless of growth", () => {
     )
 })
 
-test("nudge suppressed when filter has no recommendations (all ranges below last-segment floor)", () => {
-    // 1M model: growthThreshold=50K, lastSegmentFloor=100K
+test("nudge fires when small ranges exist — Issue #251: no floor suppression at large context", () => {
+    // 1M model: growthThreshold=50K
     // Growth of 55K > 50K threshold → nudgeAllowed = true
-    // But tool output is 80K chars (~20K tokens) < 100K floor → filtered out
-    // shouldInjectThisTurn = false (no ranges to recommend, not emergency)
+    // Tool output is 80K chars (~20K tokens) — before #251 this was < 100K floor → suppressed
+    // After #251: filterRecommendedRanges never suppresses → range shown → nudge fires
     const state = createSessionState()
     state.modelContextLimit = 1_000_000
     state.nudges.lastPerMessageNudgeTokens = 200_000
@@ -776,14 +776,13 @@ test("nudge suppressed when filter has no recommendations (all ranges below last
 
     assert.equal(
         state.nudges.shouldInjectThisTurn,
-        false,
-        "55K growth triggers nudgeAllowed but 20K tool output < 100K floor → no recommendations → nudge suppressed",
+        true,
+        "55K growth + 20K tool output → range recommended → nudge fires (Issue #251 fix)",
     )
 
     const injected = suffixText(messages)
-    assert.ok(!injected.includes("Breakdown:"), "no breakdown when no recommendations")
-    assert.ok(!injected.includes("efficiency nudge"), "no efficiency nudge text")
-    assert.ok(!injected.includes("Context limit reached"), "no emergency alert")
+    assert.ok(injected.includes("Breakdown:"), "breakdown shown when range recommended")
+    assert.ok(!injected.includes("Context limit reached"), "no emergency alert — not at max limit")
 })
 
 test("nudge suppressed when all content is protected (nothing to compress)", () => {
@@ -901,7 +900,7 @@ test("baseline preserved when nudge suppressed — growth accumulates (all prote
     )
 })
 
-test("baseline preserved when filter suppressed — compressible too small", () => {
+test("baseline preserved when nudge fires for small compressible — Issue #251", () => {
     const state = createSessionState()
     state.modelContextLimit = 1_000_000
     state.messageIds.byRawId.set("u1", "m00001")
@@ -919,15 +918,15 @@ test("baseline preserved when filter suppressed — compressible too small", ()
         ]),
     ]
     injectCompressNudges(state, config, logger, turn1, {} as any)
-    assert.equal(state.nudges.shouldInjectThisTurn, false, "55K growth but 20K compressible < 100K floor → suppressed")
+    assert.equal(state.nudges.shouldInjectThisTurn, true, "55K growth + 20K compressible → nudge fires (Issue #251)")
     assert.equal(
         state.nudges.lastPerMessageNudgeTokens,
         200_000,
-        "baseline preserved — growth accumulates until compressible content exists",
+        "baseline preserved on nudge fire — only advances after actual compression (inject.ts:537)",
     )
 })
 
-test("pending nudge cleared when suppressed — threshold resets to full", () => {
+test("pending nudge preserved when all-protected — no loop", () => {
     const state = createSessionState()
     state.modelContextLimit = 1_000_000
     state.messageIds.byRawId.set("a1", "m00001")
@@ -952,8 +951,8 @@ test("pending nudge cleared when suppressed — threshold resets to full", () =>
     assert.equal(state.nudges.shouldInjectThisTurn, false, "nudge suppressed — all protected")
     assert.equal(
         state.nudges.lastNudgeShownTokens,
-        undefined,
-        "pending nudge cleared — threshold resets to full (not halved) for next check",
+        200_000,
+        "pending nudge baseline preserved — prevents loop (stale fallback → huge growth → re-fire)",
     )
     assert.equal(
         state.nudges.lastPerMessageNudgeTokens,
@@ -962,6 +961,55 @@ test("pending nudge cleared when suppressed — threshold resets to full", () =>
     )
 })
 
+test("multi-turn: all-protected does not loop (lastNudgeShownTokens stable)", () => {
+    const state = createSessionState()
+    state.modelContextLimit = 1_000_000
+    state.messageIds.byRawId.set("a1", "m00001")
+    state.messageIds.byRawId.set("a2", "m00002")
+    state.messageIds.byRawId.set("a3", "m00003")
+
+    const config = buildConfig()
+    config.compress.protectedTools = ["skill"]
+    config.compress.maxContextLimit = 500_000
+    config.compress.minContextLimit = 200_000
+
+    state.nudges.lastPerMessageNudgeTokens = 200_000
+
+    const protectedTurn = (id: string, inputTokens: number) =>
+        assistantMsgWithTokens(id, "work", { input: inputTokens, output: 30_000 }, [
+            {
+                id: `${id}-part`, messageID: id, sessionID: SID,
+                type: "tool" as const, tool: "skill", callID: `${id}-call`,
+                state: { status: "completed" as const, input: {}, output: "x".repeat(80_000) },
+            },
+        ])
+
+    // Turn 1: nudge suppressed (all protected)
+    injectCompressNudges(state, config, logger, [protectedTurn("a1", 225_000)], {} as any)
+    assert.equal(state.nudges.shouldInjectThisTurn, false, "turn 1: all protected, nudge suppressed")
+    assert.equal(state.nudges.lastNudgeShownTokens, undefined, "turn 1: no nudge shown yet")
+
+    state.nudges.lastNudgeShownTokens = 225_000
+
+    // Turn 2: growth continues, still all-protected
+    injectCompressNudges(state, config, logger, [protectedTurn("a2", 230_000)], {} as any)
+    assert.equal(state.nudges.shouldInjectThisTurn, false, "turn 2: still all protected")
+    assert.equal(
+        state.nudges.lastNudgeShownTokens,
+        225_000,
+        "turn 2: baseline preserved — NOT reset (prevents loop)",
+    )
+
+    // Turn 3: more growth, still all-protected
+    injectCompressNudges(state, config, logger, [protectedTurn("a3", 240_000)], {} as any)
+    assert.equal(state.nudges.shouldInjectThisTurn, false, "turn 3: still all protected")
+    assert.equal(
+        state.nudges.lastNudgeShownTokens,
+        225_000,
+        "turn 3: baseline still preserved — no loop",
+    )
+})
+
 test("voluntary compress after suppression does not trigger proportional baseline adjustment", () => {
     const state = createSessionState()
     state.modelContextLimit = 1_000_000