Skip to content

[PR] kernel: lift the permanent exemption for FAILED compress exchanges (#4289) - #4293

Closed
ranxianglei wants to merge 1 commit into
masterfrom
2026-10-22_4289-failed-compress-foldable
Closed

ranxianglei wants to merge 1 commit into
masterfrom
2026-10-22_4289-failed-compress-foldable

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

kernel-change: outcome-aware compress exemption. The hard exemption lives in-kernel (kernel/src/protected.ts ALWAYS_PROTECTED_TOOLS) and all three foldability judgment sites are kernel-side (assignRefs, unit-projection filterProtectedToolMessages, recommend buildCompressibleRanges) — the host cannot express "exempt by call OUTCOME", so this must be a kernel change.

Closes #4289

What & why

isMessageProtected exempted every compress tool exchange by tool NAME alone. The design comment ("compress calls carry the summaries that decompress/search rely on") only holds for SUCCESSFUL calls: a total-failure receipt means zero ranges were applied, so the call+result carry no summary carrier — decompress/search can never read them. Yet they enjoyed the same permanent exemption: BLOCKED refs, never advertised, never folded, pinning the fold floor while the session starved on its own compression residue (~75% of one measured 227K session was residue; 70 failed calls = 116KB of pinned summary drafts).

The exemption is now outcome-aware:

  • A compress exchange whose paired result carries a total-failure receipt ([Compression FAILED, zero blocks created — distinct from partial-success receipts which start [Compressed …) loses its hard protection at ALL THREE judgment sites (KDD#12: assignRefs / unit-projection / recommend share one set built from each site's own array class) and folds like an ordinary message. The thinking text pinned behind the failure nail folds along once the nail is gone.
  • Success / partial receipts and result-less in-flight calls STAY exempt (conservative: their summaries are load-bearing or still pending).
  • acp_rule untouched; host hooks (config.isMessageProtected) and protectedTools patterns still outrank the carve-out (host intent wins).
  • Recognition goes through config.isCompressTool, so MCP-dialect lanes (mcp__bili__compress, protected via the host isToolProtected hook rather than the bare-name check) get identical treatment — the carve-out skips both the ALWAYS_PROTECTED hit AND the hook hit, only for failed exchanges.

Existing sessions self-heal without any migration: stale BLOCKED refs migrate to fresh numeric refs on the next assignRefs pass via the existing #2607 path (both halves become unprotected simultaneously, so the #3945 sibling guard never blocks; renderDeferredRefs deferral applies as usual). That also makes the exchanges eligible for the #3769 orphan sweep once a real fold moves the breakpoint — sweep TIMING is unchanged, only eligibility is restored, so the "failures stay observable between folds" policy (#3769/billion-context-pi#9) is preserved.

Behavior change (old → new)

old new
FAILED compress call+result (total-failure receipt) permanently BLOCKED — never advertised, never folded, pins fold floor numeric ref, advertised, foldable; eligible for #3769 sweep after a real fold
successful / partial compress exchanges BLOCKED BLOCKED (unchanged)
in-flight (no result yet) BLOCKED BLOCKED (unchanged)
acp_rule BLOCKED BLOCKED (unchanged)
hosts pinning compress via protectedTools/isMessageProtected pinned pinned (carve-out runs BEFORE those checks only for the two hard paths; pattern/host-hook pins still win)
non-protecting hosts (no protection machinery configured) byte-identical byte-identical (set computed lazily under the existing hasProtection gate in assignRefsNode)

No config surface added (owner-gated area untouched). No wire/message-shape change: folding a previously-unfoldable range only removes already-visible messages into an ordinary block. Mode-agnostic: kernel-level, applies identically in plugin and proxy modes.

Complementary to #4252 (parser-reject storm = the SOURCE of mass failed calls): both must land to end the compression-loop starvation; this PR fixes the independent "failed calls are unreclaimable" defect.

Tests

  • NEW kernel/tests/issue4289-failed-compress-foldable.test.ts — 12 cases incl. negative controls: partial receipts NOT treated as failed, in-flight stays BLOCKED, acp_rule stays BLOCKED, host isMessageProtected override still pins a failed exchange, protectedTools:["compress"] still pins, unnamed results unpinned via toolCallId pairing, stale-BLOCKED self-heal migration (+ renderDeferredRefs), MCP dialect via host hooks, ref-interval coverage invariant (advertised ranges address the failed exchange, never the successful one).
  • Local targeted (full-suite verdict = CI): kernel typecheck clean; kernel suites issue4289 (12) + message-protection + issue3236-recommend-apply-consistency + issue3769-orphan-sweep + refs + compress core → all green; verif-subset-guard green (compress.ts Tier-A additions stay inside the sublanguage) + all 10 no-solver verif judges green; bili protection-area suites (issue2578-mcp-hard-protection, protected-tools/-latest/-paths, issue3769-orphan-sweep-ledger) green; root typecheck clean; wire-contract gate green (goldens unchanged).

Kernel version: 0.0.151 → 0.0.152.

…4289)

ALWAYS_PROTECTED_TOOLS exempted every compress call+result by tool name alone, so total-failure retries (9-21KB summary-draft args, zero ranges applied) stayed BLOCKED forever: never advertised, never folded, pinning the fold floor while the session starved on its own compression residue (~75% of one measured 227K session).

Exemption is now outcome-aware: a compress exchange whose paired result carries a total-failure receipt ('[Compression FAILED', zero blocks created) loses its hard protection at all three judgment sites (assignRefs, unit-projection filterProtectedToolMessages, recommend buildCompressibleRanges) and folds like an ordinary message; success/partial receipts ('[Compressed ...') and result-less in-flight calls stay exempt, acp_rule is untouched, and host hooks (isMessageProtected) or protectedTools patterns still outrank the carve-out. Recognition goes through config.isCompressTool so MCP-dialect lanes (mcp__bili__compress) get identical treatment through their isToolProtected hook path.

Existing sessions self-heal without migration: stale BLOCKED refs migrate to fresh numeric refs on the next assignRefs pass via the existing #2607 path (renderDeferredRefs deferral applies), which also makes the exchanges eligible for the #3769 orphan sweep once a real fold moves the breakpoint.

Tests: kernel/tests/issue4289-failed-compress-foldable.test.ts (12 cases incl. negative controls: partial receipts, in-flight, acp_rule, host-override wins, unnamed-result pairing, self-heal migration, MCP dialect).
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重: 不重复。 本条是 PR #4293 的镜像 issue(关闭 #4289);按失败签名「失败 compress 交换永久 BLOCKED、残渣不可回收」检索 open+closed issue 与 PR,无同机制的独立条目。与 #4252(解析器拒绝风暴)是上下游互补关系,非重复——#4252 治"为什么大量失败",本 PR 治"失败了不可回收"。

标签: bug + scope/kernel + p1(压缩残渣只增不减地撑大上下文,属缓存泄露类失控增长,按 10-19 规则定 p1;kernel 级修复,跨车道,不打客户端标)。

当前状态:PR head d4f979b(branch 2026-10-22_4289-failed-compress-foldable)基于旧 master fe5ba5912,与当前 master 处于 dirty 冲突态,floor 2 已转发 conflict-worker 处理 rebase。我正在做独立复核:拉分支核对代码、查 CI 状态、按八问出审查结论,稍后回帖。

@ranxianglei ranxianglei added bug Something isn't working p1 scope/kernel labels Oct 23, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

(ai: vllm-qwen/qwen3.8-27b)

结论:本 PR 已被先合入的 #4291 完整取代(superseded),按重复处置关闭。

核对过程(对 master 当前头 72926f6 逐一验证):

  1. 同一修复已在 master。Failed compression invocations are never folded back for reclamation, and the context keeps ballooning under the weight | 失败的压缩调用永远不会被折叠回收,上下文被压缩残渣越撑越大 #4289 的修复经由 PR [PR] Failed compression calls become collapsible regular messages, no longer permanently occupying the context. | 失败的压缩调用变成可折叠的普通消息,不再永久占着上下文 #4291(分支 fix/unprotect-failed-compress,merge fcf3e82)先行合入:kernel/src/protected.ts 的 FAILED_COMPRESS_RESULT_PREFIXES / isCompressToolName / isFailedCompressResult / collectFailedCompressCallIds,三个判定位全部接线——compress.ts:1019(assignRefsNode)、unit-projection.ts:466(filterProtectedToolMessages)、recommend.ts:300(buildCompressibleRanges)。后续还有两次同机制扩展:[PR] Refused compress exchanges become foldable like failed ones, no longer permanently stuck on the wire | 被拒绝的压缩调用像失败的一样可折叠,不再永久占着上下文 #4310(Compression calls with lost ACKs are permanently stuck in context: client-side message loss / model-generation switch re | 回执丢失的压缩调用永久滞留上下文:客户端丢消息/切模型世代重置产生孤儿,且无任何回收路径 #4307:[Compression refused 前缀的拒绝回执同样可折叠)与 Accident re-anchor debris is unrecoverable: the overlay island lands mid-turn, and the turn-split guard means the orphan | 事故重锚碎屑无法回收:覆盖岛落在 turn 中段,turn-split 守卫让孤另一半永远折不了 #4348(成功但陈旧的 compress 交换也折叠,仅保留最近 3 条)。kernel 版本已到 0.0.160。
  2. 测试覆盖等价。master 的 kernel/tests/issue4289-failed-compress-fold.test.ts(14 个用例)覆盖本 PR 测试的全部关键角度:失败交换拿到数字 ref、整段可折叠、pending 调用仍保护、partial 回执([Compressed)仍保护、旧 BLOCKED ref 自愈迁移、单元形状、recommend 广告、path-pattern 分支、MCP 方言正/负用例,另加 Compression calls with lost ACKs are permanently stuck in context: client-side message loss / model-generation switch re | 回执丢失的压缩调用永久滞留上下文:客户端丢消息/切模型世代重置产生孤儿,且无任何回收路径 #4307 拒绝回执用例。本 PR 独有的「无名结果靠 toolCallId 配对解锁」在 master 是有意的保守取舍(要求 result 自带 toolName,未知方言保持钉住),真实车道都投影 toolName,无实际差距,不移植。
  3. 一处语义差异,以已合入版本为准:本 PR 让 protectedTools 模式钉住优先于豁免(失败交换仍可被模式钉住);[PR] Failed compression calls become collapsible regular messages, no longer permanently occupying the context. | 失败的压缩调用变成可折叠的普通消息,不再永久占着上下文 #4291 的语义是失败交换脱离一切内置保护源(ALWAYS 表、模式、工具钩子),只有宿主消息级 isMessageProtected 钩子(开启 bili 后「规矩与 skill 好像没被使用」? #2419)压过豁免——该语义已经 owner review 合入,为规范行为。若合并本 PR 反而会把模式钉回失败交换,并与 Compression calls with lost ACKs are permanently stuck in context: client-side message loss / model-generation switch re | 回执丢失的压缩调用永久滞留上下文:客户端丢消息/切模型世代重置产生孤儿,且无任何回收路径 #4307/Accident re-anchor debris is unrecoverable: the overlay island lands mid-turn, and the turn-split guard means the orphan | 事故重锚碎屑无法回收:覆盖岛落在 turn 中段,turn-split 守卫让孤另一半永远折不了 #4348 的扩展冲突(kernel 版本 0.0.152 vs 0.0.160、同名函数重定义)。

无 salvage 价值,不再 rebase。duplicate of #4291(同一 issue #4289、同一机制:解除零块 compress 交换的永久豁免),先到的 #4291 保留且已合入,后到的本 PR 关闭。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working duplicate This issue or pull request already exists p1 scope/kernel

Projects

None yet

1 participant